Translator Disclaimer
March, 1988 The Two-Armed Bandit with Delayed Responses
Stephen G. Eick
Ann. Statist. 16(1): 254-264 (March, 1988). DOI: 10.1214/aos/1176350703


A general model for a two-armed bandit with delayed responses is introduced and solved with dynamic programming. One arm has geometric lifetime with parameter $\theta$, which has prior distribution $\mu$. The other arm has known lifetime with mean $\kappa$. The response delays completely change the character of the optimal strategies from the no delay case; in particular, the bandit is no longer a stopping problem. The delays also introduce an extra parameter $p$ into the state space. In clinical trial applications, this parameter represents the number of patients previously treated with the unknown arm who are still living. The value function is introduced and investigated as $p, \mu$ and $\kappa$ vary. Under a regularity condition on the discount sequence, there exists a manifold in the state space such that both arms are optimal on the manifold, arm $x$ is optimal on one side and arm $y$ on the other. Properties of the manifold are described.


Download Citation

Stephen G. Eick. "The Two-Armed Bandit with Delayed Responses." Ann. Statist. 16 (1) 254 - 264, March, 1988.


Published: March, 1988
First available in Project Euclid: 12 April 2007

zbMATH: 0637.62074
MathSciNet: MR924869
Digital Object Identifier: 10.1214/aos/1176350703

Primary: 62L05
Secondary: 62L15

Rights: Copyright © 1988 Institute of Mathematical Statistics


Vol.16 • No. 1 • March, 1988
Back to Top