The Annals of Applied Probability

General notions of indexability for queueing control and asset management

Kevin D. Glazebrook, David J. Hodge, and Chris Kirkbride

Full-text: Open access


We develop appropriately generalized notions of indexability for problems of dynamic resource allocation where the resource concerned may be assigned more flexibility than is allowed, for example, in classical multi-armed bandits. Most especially we have in mind the allocation of a divisible resource (manpower, money, equipment) to a collection of objects (projects) requiring it in cases where its over-concentration would usually be far from optimal. The resulting project indices are functions of both a resource level and a state. They have a simple interpretation as a fair charge for increasing the resource available to the project from the specified resource level when in the specified state. We illustrate ideas by reference to two model classes which are of independent interest. In the first, a pool of servers is assigned dynamically to a collection of service teams, each of which mans a service station. We demonstrate indexability under a natural assumption that the service rate delivered is increasing and concave in the team size. The second model class is a generalization of the spinning plates model for the optimal deployment of a divisible investment resource to a collection of reward generating assets. Asset indexability is established under appropriately drawn laws of diminishing returns for resource deployment. For both model classes numerical studies provide evidence that the proposed greedy index heuristic performs strongly.

Article information

Ann. Appl. Probab., Volume 21, Number 3 (2011), 876-907.

First available in Project Euclid: 2 June 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx]
Secondary: 90B22: Queues and service [See also 60K25, 68M20] 90B36: Scheduling theory, stochastic [See also 68M20]

Asset management dynamic programming dynamic resource allocation full indexability index policy Lagrangian relaxation monotone policy queueing control


Glazebrook, Kevin D.; Hodge, David J.; Kirkbride, Chris. General notions of indexability for queueing control and asset management. Ann. Appl. Probab. 21 (2011), no. 3, 876--907. doi:10.1214/10-AAP705.

Export citation


  • [1] Archibald, T. W., Black, D. P. and Glazebrook, K. D. (2009). Indexability and index heuristics for a simple class of inventory routing problems. Oper. Res. 57 314–326.
  • [2] Argon, N. T., Ding, L., Glazebrook, K. D. and Ziya, S. (2009). Dynamic routing of customers with general delay costs in a multiserver queuing sysem. Probab. Engrg. Inform. Sci. 23 175–203.
  • [3] Cox, D. R. and Smith, W. L. (1961). Queues. Methuen, London.
  • [4] Crabill, T. B. (1972). Optimal control of a service facility with variable exponential service times and constant arrival rate. Management Sci. 18 560–566.
  • [5] Doshi, B. T. (1978). Optimal control of the service rate in an M/G/1 queueing system. Adv. in Appl. Probab. 10 682–701.
  • [6] Gallisch, E. (1979). On monotone optimal policies in a queueing model of (M/G/1) type with controllable service time distribution. Adv. in Appl. Probab. 11 870–887.
  • [7] George, J. M. and Harrison, J. M. (2001). Dynamic control of a queue with adjustable service rate. Oper. Res. 49 720–731.
  • [8] Gittins, J. C. (1979). Bandit processes and dynamic allocation indices (with discussion). J. Roy. Statist. Soc. Ser. B 41 148–177.
  • [9] Gittins, J. C. (1989). Multi-Armed Bandit Allocation Indices. Wiley, Chichester.
  • [10] Glazebrook, K. D. and Kirkbride, C. (2007). Dynamic routing to heterogeneous collections of unreliable servers. Queueing Syst. 55 9–25.
  • [11] Glazebrook, K. D., Kirkbride, C. and Ruiz-Hernandez, D. (2006). Spinning plates and squad systems: Policies for bi-directional restless bandits. Adv. in Appl. Probab. 38 95–115.
  • [12] Glazebrook, K. D. and Minty, R. (2009). A generalized Gittins index for a class of multiarmed bandits with general resource requirements. Math. Oper. Res. 34 26–44.
  • [13] Glazebrook, K. D., Mitchell, H. M. and Ansell, P. S. (2005). Index policies for the maintenance of a collection of machines by a set of repairmen. European J. Oper. Res. 165 267–284.
  • [14] Mahajan, A. and Teneketzis, D. (2007). Multi-armed bandit problems. In Foundations and Applications of Sensor Management (A. Hero, D. Castanon, D. Cochran and K. Kastella, eds.) 121–151. Springer, New York.
  • [15] Mitchell, B. (1973). Optimal service-rate selection in an M/G/1 queue. SIAM J. Appl. Math. 24 19–35.
  • [16] Papadimitriou, C. H. and Tsitsiklis, J. N. (1999). The complexity of optimal queuing network control. Math. Oper. Res. 24 293–305.
  • [17] Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
  • [18] Sobel, M. (1982). The optimality of full-service policies. Oper. Res. 30 636–649.
  • [19] Stidham Jr., S. and Weber, R. R. (1989). Monotonic and insensitive optimal policies for control of queues with undiscounted costs. Oper. Res. 37 611–625.
  • [20] Weber, R. R. and Weiss, G. (1990). On an index policy for restless bandits. J. Appl. Probab. 27 637–648. [Addendum: Adv. Appl. Probab. 23 (1991) 429–430.]
  • [21] Whittle, P. (1988). Restless bandits: Activity allocation in a changing world. J. Appl. Probab. 25A 287–298. A celebration of applied probability.