The Annals of Applied Probability

General notions of indexability for queueing control and asset management

Kevin D. Glazebrook, David J. Hodge, and Chris Kirkbride

Full-text: Open access

Enhanced PDF (283 KB)

Abstract

We develop appropriately generalized notions of indexability for problems of dynamic resource allocation where the resource concerned may be assigned more flexibility than is allowed, for example, in classical multi-armed bandits. Most especially we have in mind the allocation of a divisible resource (manpower, money, equipment) to a collection of objects (projects) requiring it in cases where its over-concentration would usually be far from optimal. The resulting project indices are functions of both a resource level and a state. They have a simple interpretation as a fair charge for increasing the resource available to the project from the specified resource level when in the specified state. We illustrate ideas by reference to two model classes which are of independent interest. In the first, a pool of servers is assigned dynamically to a collection of service teams, each of which mans a service station. We demonstrate indexability under a natural assumption that the service rate delivered is increasing and concave in the team size. The second model class is a generalization of the spinning plates model for the optimal deployment of a divisible investment resource to a collection of reward generating assets. Asset indexability is established under appropriately drawn laws of diminishing returns for resource deployment. For both model classes numerical studies provide evidence that the proposed greedy index heuristic performs strongly.

Article information

Source
Ann. Appl. Probab., Volume 21, Number 3 (2011), 876-907.

Dates
First available in Project Euclid: 2 June 2011

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1307020386

Digital Object Identifier
doi:10.1214/10-AAP705

Mathematical Reviews number (MathSciNet)
MR2830607

Zentralblatt MATH identifier
1229.91173

Subjects
Primary: 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx]
Secondary: 90B22: Queues and service [See also 60K25, 68M20] 90B36: Scheduling theory, stochastic [See also 68M20]

Keywords
Asset management dynamic programming dynamic resource allocation full indexability index policy Lagrangian relaxation monotone policy queueing control

Citation

Glazebrook, Kevin D.; Hodge, David J.; Kirkbride, Chris. General notions of indexability for queueing control and asset management. Ann. Appl. Probab. 21 (2011), no. 3, 876--907. doi:10.1214/10-AAP705. https://projecteuclid.org/euclid.aoap/1307020386


Export citation

References

  • [1] Archibald, T. W., Black, D. P. and Glazebrook, K. D. (2009). Indexability and index heuristics for a simple class of inventory routing problems. Oper. Res. 57 314–326.
    Mathematical Reviews (MathSciNet): MR2555573
    Zentralblatt MATH: 1181.90004
    Digital Object Identifier: doi:10.1287/opre.1070.0505
  • [2] Argon, N. T., Ding, L., Glazebrook, K. D. and Ziya, S. (2009). Dynamic routing of customers with general delay costs in a multiserver queuing sysem. Probab. Engrg. Inform. Sci. 23 175–203.
    Mathematical Reviews (MathSciNet): MR2480086
    Digital Object Identifier: doi:10.1017/S0269964809000138
  • [3] Cox, D. R. and Smith, W. L. (1961). Queues. Methuen, London.
    Mathematical Reviews (MathSciNet): MR133178
  • [4] Crabill, T. B. (1972). Optimal control of a service facility with variable exponential service times and constant arrival rate. Management Sci. 18 560–566.
    Mathematical Reviews (MathSciNet): MR317434
    Digital Object Identifier: doi:10.1287/mnsc.18.9.560
    JSTOR: links.jstor.org
  • [5] Doshi, B. T. (1978). Optimal control of the service rate in an M/G/1 queueing system. Adv. in Appl. Probab. 10 682–701.
    Mathematical Reviews (MathSciNet): MR499221
    Zentralblatt MATH: 0381.60086
    Digital Object Identifier: doi:10.2307/1426641
    JSTOR: links.jstor.org
  • [6] Gallisch, E. (1979). On monotone optimal policies in a queueing model of (M/G/1) type with controllable service time distribution. Adv. in Appl. Probab. 11 870–887.
    Mathematical Reviews (MathSciNet): MR544200
    Zentralblatt MATH: 0434.60095
    Digital Object Identifier: doi:10.2307/1426864
    JSTOR: links.jstor.org
  • [7] George, J. M. and Harrison, J. M. (2001). Dynamic control of a queue with adjustable service rate. Oper. Res. 49 720–731.
    Mathematical Reviews (MathSciNet): MR1860424
    Zentralblatt MATH: 1163.90420
    Digital Object Identifier: doi:10.1287/opre.49.5.720.10605
    JSTOR: links.jstor.org
  • [8] Gittins, J. C. (1979). Bandit processes and dynamic allocation indices (with discussion). J. Roy. Statist. Soc. Ser. B 41 148–177.
    Mathematical Reviews (MathSciNet): MR547241
    JSTOR: links.jstor.org
  • [9] Gittins, J. C. (1989). Multi-Armed Bandit Allocation Indices. Wiley, Chichester.
    Mathematical Reviews (MathSciNet): MR996417
  • [10] Glazebrook, K. D. and Kirkbride, C. (2007). Dynamic routing to heterogeneous collections of unreliable servers. Queueing Syst. 55 9–25.
    Mathematical Reviews (MathSciNet): MR2293563
    Digital Object Identifier: doi:10.1007/s11134-006-9002-9
  • [11] Glazebrook, K. D., Kirkbride, C. and Ruiz-Hernandez, D. (2006). Spinning plates and squad systems: Policies for bi-directional restless bandits. Adv. in Appl. Probab. 38 95–115.
    Mathematical Reviews (MathSciNet): MR2213966
    Zentralblatt MATH: 1105.90103
    Digital Object Identifier: doi:10.1239/aap/1143936142
    Project Euclid: euclid.aap/1143936142
  • [12] Glazebrook, K. D. and Minty, R. (2009). A generalized Gittins index for a class of multiarmed bandits with general resource requirements. Math. Oper. Res. 34 26–44.
    Mathematical Reviews (MathSciNet): MR2542987
    Zentralblatt MATH: 05881615
    Digital Object Identifier: doi:10.1287/moor.1080.0342
  • [13] Glazebrook, K. D., Mitchell, H. M. and Ansell, P. S. (2005). Index policies for the maintenance of a collection of machines by a set of repairmen. European J. Oper. Res. 165 267–284.
    Mathematical Reviews (MathSciNet): MR2121966
    Zentralblatt MATH: 1112.90323
    Digital Object Identifier: doi:10.1016/j.ejor.2004.01.036
  • [14] Mahajan, A. and Teneketzis, D. (2007). Multi-armed bandit problems. In Foundations and Applications of Sensor Management (A. Hero, D. Castanon, D. Cochran and K. Kastella, eds.) 121–151. Springer, New York.
  • [15] Mitchell, B. (1973). Optimal service-rate selection in an M/G/1 queue. SIAM J. Appl. Math. 24 19–35.
    Mathematical Reviews (MathSciNet): MR326863
    Zentralblatt MATH: 0233.60081
    Digital Object Identifier: doi:10.1137/0124004
    JSTOR: links.jstor.org
  • [16] Papadimitriou, C. H. and Tsitsiklis, J. N. (1999). The complexity of optimal queuing network control. Math. Oper. Res. 24 293–305.
    Mathematical Reviews (MathSciNet): MR1853877
    Zentralblatt MATH: 0977.90008
    Digital Object Identifier: doi:10.1287/moor.24.2.293
    JSTOR: links.jstor.org
  • [17] Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1270015
  • [18] Sobel, M. (1982). The optimality of full-service policies. Oper. Res. 30 636–649.
  • [19] Stidham Jr., S. and Weber, R. R. (1989). Monotonic and insensitive optimal policies for control of queues with undiscounted costs. Oper. Res. 37 611–625.
    Mathematical Reviews (MathSciNet): MR1006813
    Zentralblatt MATH: 0674.90029
    Digital Object Identifier: doi:10.1287/opre.37.4.611
    JSTOR: links.jstor.org
  • [20] Weber, R. R. and Weiss, G. (1990). On an index policy for restless bandits. J. Appl. Probab. 27 637–648. [Addendum: Adv. Appl. Probab. 23 (1991) 429–430.]
    Mathematical Reviews (MathSciNet): MR1067028
    Zentralblatt MATH: 0735.90072
    Digital Object Identifier: doi:10.2307/3214547
    JSTOR: links.jstor.org
  • [21] Whittle, P. (1988). Restless bandits: Activity allocation in a changing world. J. Appl. Probab. 25A 287–298. A celebration of applied probability.
    Mathematical Reviews (MathSciNet): MR974588
    Digital Object Identifier: doi:10.2307/3214163

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more