The Annals of Applied Probability

Dynamic Allocation Problems in Continuous Time

Nicole El Karoui and Ioannis Karatzas

Full-text: Open access

Abstract

We present an approach to the general, non-Markovian dynamic allocation (or multiarmed bandit) problem, formulated in continuous time as a problem of stochastic control for multiparameter processes in the manner of Mandelbaum. This approach is based on a direct, martingale study of auxiliary questions in optimal stopping. Using a methodology similar to that of Whittle and relying on simple time-change arguments, we construct Gittins-index-type strategies, verify their optimality, provide explicit expressions for the values of dynamic allocation and associated optimal stopping problems, explore interesting dualities and derive various characterizations of Gittins indices. This paper extends results of our recent work on discrete-parameter dynamic allocation to the continuous time setup; it can be read independently of that work.

Article information

Source
Ann. Appl. Probab., Volume 4, Number 2 (1994), 255-286.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1177005062

Digital Object Identifier
doi:10.1214/aoap/1177005062

Mathematical Reviews number (MathSciNet)
MR1272729

Zentralblatt MATH identifier
0831.93069

JSTOR
links.jstor.org

Subjects
Primary: 93E20: Optimal stochastic control
Secondary: 60G60: Random fields 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60] 90B85: Continuous location 62L10: Sequential analysis

Keywords
Multiarmed bandit problem optimal stopping stochastic control multiparameter random time-change Gittins index Brownian local time

Citation

Karoui, Nicole El; Karatzas, Ioannis. Dynamic Allocation Problems in Continuous Time. Ann. Appl. Probab. 4 (1994), no. 2, 255--286. doi:10.1214/aoap/1177005062. https://projecteuclid.org/euclid.aoap/1177005062


Export citation