Open Access
Translator Disclaimer
August 2016 Asymptotically optimal priority policies for indexable and nonindexable restless bandits
I. M. Verloop
Ann. Appl. Probab. 26(4): 1947-1995 (August 2016). DOI: 10.1214/15-AAP1137


We study the asymptotic optimal control of multi-class restless bandits. A restless bandit is a controllable stochastic process whose state evolution depends on whether or not the bandit is made active. Since finding the optimal control is typically intractable, we propose a class of priority policies that are proved to be asymptotically optimal under a global attractor property and a technical condition. We consider both a fixed population of bandits as well as a dynamic population where bandits can depart and arrive. As an example of a dynamic population of bandits, we analyze a multi-class $\mathit{M/M/S+M}$ queue for which we show asymptotic optimality of an index policy.

We combine fluid-scaling techniques with linear programming results to prove that when bandits are indexable, Whittle’s index policy is included in our class of priority policies. We thereby generalize a result of Weber and Weiss [J. Appl. Probab. 27 (1990) 637–648] about asymptotic optimality of Whittle’s index policy to settings with (i) several classes of bandits, (ii) arrivals of new bandits and (iii) multiple actions.

Indexability of the bandits is not required for our results to hold. For nonindexable bandits, we describe how to select priority policies from the class of asymptotically optimal policies and present numerical evidence that, outside the asymptotic regime, the performance of our proposed priority policies is nearly optimal.


Download Citation

I. M. Verloop. "Asymptotically optimal priority policies for indexable and nonindexable restless bandits." Ann. Appl. Probab. 26 (4) 1947 - 1995, August 2016.


Received: 1 June 2014; Revised: 1 August 2015; Published: August 2016
First available in Project Euclid: 1 September 2016

zbMATH: 1349.90834
MathSciNet: MR3543887
Digital Object Identifier: 10.1214/15-AAP1137

Primary: 90C40
Secondary: 68M20 , 90B36

Keywords: arm-aquiring bandits , asymptotic optimality , nonindexable bandits , Restless bandits , Whittle’s index policy

Rights: Copyright © 2016 Institute of Mathematical Statistics


Vol.26 • No. 4 • August 2016
Back to Top