The Annals of Applied Probability
- Ann. Appl. Probab.
- Volume 4, Number 2 (1994), 255-286.
Dynamic Allocation Problems in Continuous Time
We present an approach to the general, non-Markovian dynamic allocation (or multiarmed bandit) problem, formulated in continuous time as a problem of stochastic control for multiparameter processes in the manner of Mandelbaum. This approach is based on a direct, martingale study of auxiliary questions in optimal stopping. Using a methodology similar to that of Whittle and relying on simple time-change arguments, we construct Gittins-index-type strategies, verify their optimality, provide explicit expressions for the values of dynamic allocation and associated optimal stopping problems, explore interesting dualities and derive various characterizations of Gittins indices. This paper extends results of our recent work on discrete-parameter dynamic allocation to the continuous time setup; it can be read independently of that work.
Ann. Appl. Probab., Volume 4, Number 2 (1994), 255-286.
First available in Project Euclid: 19 April 2007
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 93E20: Optimal stochastic control
Secondary: 60G60: Random fields 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60] 90B85: Continuous location 62L10: Sequential analysis
Karoui, Nicole El; Karatzas, Ioannis. Dynamic Allocation Problems in Continuous Time. Ann. Appl. Probab. 4 (1994), no. 2, 255--286. doi:10.1214/aoap/1177005062. https://projecteuclid.org/euclid.aoap/1177005062