The Annals of Applied Probability

A Short Proof of the Gittins Index Theorem

John N. Tsitsiklis

Full-text: Open access

Abstract

We provide a short and elementary proof of the Gittins index theorem for the multi-armed bandit problem, for the case where each bandit is modeled as a finite-state semi-Markov process. We also indicate how this proof can be extended to the branching bandits and Klimov problems.

Article information

Source
Ann. Appl. Probab., Volume 4, Number 1 (1994), 194-199.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1177005207

Digital Object Identifier
doi:10.1214/aoap/1177005207

Mathematical Reviews number (MathSciNet)
MR1258180

Zentralblatt MATH identifier
0816.60040

JSTOR
links.jstor.org

Subjects
Primary: 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60]
Secondary: 90B35: Scheduling theory, deterministic [See also 68M20] 62L05: Sequential design 90C40: Markov and semi-Markov decision processes

Keywords
Multi-armed bandit problem branching bandits Klimov's problem priority scheduling

Citation

Tsitsiklis, John N. A Short Proof of the Gittins Index Theorem. Ann. Appl. Probab. 4 (1994), no. 1, 194--199. doi:10.1214/aoap/1177005207. https://projecteuclid.org/euclid.aoap/1177005207


Export citation