September 2015 On the asymptotic optimality of greedy index heuristics for multi-action restless bandits
D. J. Hodge, K. D. Glazebrook
Author Affiliations +
Adv. in Appl. Probab. 47(3): 652-667 (September 2015). DOI: 10.1239/aap/1444308876


The class of restless bandits as proposed by Whittle (1988) have long been known to be intractable. This paper presents an optimality result which extends that of Weber and Weiss (1990) for restless bandits to a more general setting in which individual bandits have multiple levels of activation but are subject to an overall resource constraint. The contribution is motivated by the recent works of Glazebrook et al. (2011a), (2011b) who discussed the performance of index heuristics for resource allocation in such systems. Hitherto, index heuristics have been shown, under a condition of full indexability, to be optimal for a natural Lagrangian relaxation of such problems in which a resource is purchased rather than constrained. We find that under key assumptions about the nature of solutions to a deterministic differential equation that the index heuristics above are asymptotically optimal in a sense described by Whittle. We then demonstrate that these assumptions always hold for three-state bandits.


Download Citation

D. J. Hodge. K. D. Glazebrook. "On the asymptotic optimality of greedy index heuristics for multi-action restless bandits." Adv. in Appl. Probab. 47 (3) 652 - 667, September 2015.


Published: September 2015
First available in Project Euclid: 8 October 2015

zbMATH: 1326.90102
MathSciNet: MR3406602
Digital Object Identifier: 10.1239/aap/1444308876

Primary: 90C40
Secondary: 49L20 , 49M20 , 93E20

Keywords: asymptotic optimality , Index heuristic , multi-action restless bandit , stochastic resource allocation

Rights: Copyright © 2015 Applied Probability Trust


This article is only available to subscribers.
It is not available for individual sale.

Vol.47 • No. 3 • September 2015
Back to Top