Open Access
Translator Disclaimer
May, 1994 Dynamic Allocation Problems in Continuous Time
Nicole El Karoui, Ioannis Karatzas
Ann. Appl. Probab. 4(2): 255-286 (May, 1994). DOI: 10.1214/aoap/1177005062


We present an approach to the general, non-Markovian dynamic allocation (or multiarmed bandit) problem, formulated in continuous time as a problem of stochastic control for multiparameter processes in the manner of Mandelbaum. This approach is based on a direct, martingale study of auxiliary questions in optimal stopping. Using a methodology similar to that of Whittle and relying on simple time-change arguments, we construct Gittins-index-type strategies, verify their optimality, provide explicit expressions for the values of dynamic allocation and associated optimal stopping problems, explore interesting dualities and derive various characterizations of Gittins indices. This paper extends results of our recent work on discrete-parameter dynamic allocation to the continuous time setup; it can be read independently of that work.


Download Citation

Nicole El Karoui. Ioannis Karatzas. "Dynamic Allocation Problems in Continuous Time." Ann. Appl. Probab. 4 (2) 255 - 286, May, 1994.


Published: May, 1994
First available in Project Euclid: 19 April 2007

zbMATH: 0831.93069
MathSciNet: MR1272729
Digital Object Identifier: 10.1214/aoap/1177005062

Primary: 93E20
Secondary: 60G40 , 60G60 , 62L10 , 90B85

Keywords: Brownian local time , Gittins index , Multiarmed bandit problem , multiparameter random time-change , Optimal stopping , Stochastic control

Rights: Copyright © 1994 Institute of Mathematical Statistics


Vol.4 • No. 2 • May, 1994
Back to Top