Advances in Applied Probability
- Adv. in Appl. Probab.
- Volume 40, Number 2 (2008), 377-400.
Index policies for discounted bandit problems with availability constraints
A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.
Adv. in Appl. Probab. Volume 40, Number 2 (2008), 377-400.
First available in Project Euclid: 1 July 2008
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 93E20: Optimal stochastic control
Secondary: 90B36: Scheduling theory, stochastic [See also 68M20]
Dayanik, Savas; Powell, Warren; Yamazaki, Kazutoshi. Index policies for discounted bandit problems with availability constraints. Adv. in Appl. Probab. 40 (2008), no. 2, 377--400. doi:10.1239/aap/1214950209. https://projecteuclid.org/euclid.aap/1214950209