Open Access
Translator Disclaimer
August 2004 When can the two-armed bandit algorithm be trusted?
Damien Lamberton, Gilles Pagès, Pierre Tarrès
Ann. Appl. Probab. 14(3): 1424-1454 (August 2004). DOI: 10.1214/105051604000000350


We investigate the asymptotic behavior of one version of the so-called two-armed bandit algorithm. It is an example of stochastic approximation procedure whose associated ODE has both a repulsive and an attractive equilibrium, at which the procedure is noiseless. We show that if the gain parameter is constant or goes to 0 not too fast, the algorithm does fall in the noiseless repulsive equilibrium with positive probability, whereas it always converges to its natural attractive target when the gain parameter goes to zero at some appropriate rates depending on the parameters of the model. We also elucidate the behavior of the constant step algorithm when the step goes to 0. Finally, we highlight the connection between the algorithm and the Polya urn. An application to asset allocation is briefly described.


Download Citation

Damien Lamberton. Gilles Pagès. Pierre Tarrès. "When can the two-armed bandit algorithm be trusted?." Ann. Appl. Probab. 14 (3) 1424 - 1454, August 2004.


Published: August 2004
First available in Project Euclid: 13 July 2004

zbMATH: 1048.62079
MathSciNet: MR2071429
Digital Object Identifier: 10.1214/105051604000000350

Primary: 62L20
Secondary: 68T05 , 91B32 , 91E40 , 93C40

Keywords: asset allocation , learning automata , Pólya urn , stochastic approximation , Two-armed bandit algorithm

Rights: Copyright © 2004 Institute of Mathematical Statistics


Vol.14 • No. 3 • August 2004
Back to Top