Translator Disclaimer
March 2013 Monotone policies and indexability for bidirectional restless bandits
K. D. Glazebrook, D. J. Hodge, C. Kirkbride
Author Affiliations +
Adv. in Appl. Probab. 45(1): 51-85 (March 2013). DOI: 10.1239/aap/1363354103


Motivated by a wide range of applications, we consider a development of Whittle's restless bandit model in which project activation requires a state-dependent amount of a key resource, which is assumed to be available at a constant rate. As many projects may be activated at each decision epoch as resource availability allows. We seek a policy for project activation within resource constraints which minimises an aggregate cost rate for the system. Project indices derived from a Lagrangian relaxation of the original problem exist provided the structural requirement of indexability is met. Verification of this property and derivation of the related indices is greatly simplified when the solution of the Lagrangian relaxation has a state monotone structure for each constituent project. We demonstrate that this is indeed the case for a wide range of bidirectional projects in which the project state tends to move in a different direction when it is activated from that in which it moves when passive. This is natural in many application domains in which activation of a project ameliorates its condition, which otherwise tends to deteriorate or deplete. In some cases the state monotonicity required is related to the structure of state transitions, while in others it is also related to the nature of costs. Two numerical studies demonstrate the value of the ideas for the construction of policies for dynamic resource allocation, most especially in contexts which involve a large number of projects.


Download Citation

K. D. Glazebrook. D. J. Hodge. C. Kirkbride. "Monotone policies and indexability for bidirectional restless bandits." Adv. in Appl. Probab. 45 (1) 51 - 85, March 2013.


Published: March 2013
First available in Project Euclid: 15 March 2013

zbMATH: 1274.90473
MathSciNet: MR3077541
Digital Object Identifier: 10.1239/aap/1363354103

Primary: 90C40
Secondary: 49L20 , 49M20 , 90C39

Keywords: asset management , Gittins index , indexability , inventory management , Lagrangian relaxation , machine maintenance , monotone policy , restless bandit , stochastic dynamic programming , Whittle index

Rights: Copyright © 2013 Applied Probability Trust


This article is only available to subscribers.
It is not available for individual sale.

Vol.45 • No. 1 • March 2013
Back to Top