Advances in Applied Probability

The performance of index-based policies for bandit problems with stochastic machine availability

R. T. Dunn and K. D. Glazebrook

Full-text: Access denied (no subscription detected) We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We consider generalisations of two classical stochastic scheduling models, namely the discounted branching bandit and the discounted multi-armed bandit, to the case where the collection of machines available for processing is itself a stochastic process. Under rather mild conditions on the machine availability process we obtain performance guarantees for a range of controls based on Gittins indices. Various forms of asymptotic optimality are established for index-based limit policies as the discount rate approaches 0.

Article information

Adv. in Appl. Probab. Volume 33, Number 2 (2001), 365-390.

First available in Project Euclid: 30 August 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 90B36: Scheduling theory, stochastic [See also 68M20]
Secondary: 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx] 90B22: Queues and service [See also 60K25, 68M20]

average-overtaking optimal average-reward optimal branching bandit discounted rewards Gittins index machine breakdowns multi-armed bandit problem parallel machines suboptimality bound


Dunn, R. T.; Glazebrook, K. D. The performance of index-based policies for bandit problems with stochastic machine availability. Adv. in Appl. Probab. 33 (2001), no. 2, 365--390. doi:10.1239/aap/999188319.

Export citation