The Annals of Statistics

Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn

Caterina May and Nancy Flournoy

Full-text: Open access


This paper illustrates asymptotic properties for a response-adaptive design generated by a two-color, randomly reinforced urn model. The design considered is optimal in the sense that it assigns patients to the best treatment, with probability converging to one. An approach to show the joint asymptotic normality of the estimators of the mean responses to the treatments is provided in spite of the fact that allocation proportions converge to zero and one. Results on the rate of convergence of the number of patients assigned to each treatment are also obtained. Finally, we study the asymptotic behavior of a suitable test statistic.

Article information

Ann. Statist., Volume 37, Number 2 (2009), 1058-1078.

First available in Project Euclid: 10 March 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62L05: Sequential design
Secondary: 60F15: Strong theorems 60F05: Central limit and other weak theorems

Generalized Pólya urn adaptive designs asymptotic normality rate of convergence optimal allocation estimation and inference clinical trials ethical allocation testing mean differences treatment allocation two-sample t-test mixing convergence


May, Caterina; Flournoy, Nancy. Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn. Ann. Statist. 37 (2009), no. 2, 1058--1078. doi:10.1214/08-AOS596.

Export citation


  • [1] Aletti, G., May, C. and Secchi, P. (2007). On the distribution of the limit proportion for a two-color, randomly reinforced urn with equal reinforcement distributions. Adv. in Appl. Probab. 39 690–707.
  • [2] Atkinson, A. C. and Biswas, A. (2005). Adaptive biased-coin designs for skewing the allocation proportion in clinical trials with normal responses. Stat. Med. 24 2477–2492.
  • [3] Bai, Z.-D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Probab. 15 914–940.
  • [4] Bai, Z. D., Hu, F. and Zhang, L.-X. (2002). Gaussian approximation theorems for urn models and their applications. Ann. Appl. Probab. 12 1149–1173.
  • [5] Beggs, A. W. (2005). On the convergence of reinforcement learning. J. Econom. Theory 122 1–36.
  • [6] Belisle, C. and Melfi, V. (2008). Independence after adaptive allocation. Statist. Probab. Lett. 78 214–224.
  • [7] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • [8] Durham, S. D., Flournoy, N. and Li, W. (1998). A sequential design for maximizing the probability of a favourable response. Canad. J. Statist. 26 479–495.
  • [9] Durham, S. D. and Yu, K. F. (1990). Randomized play-the leader rules for sequential sampling from two populations. Probability in Engineering and Information Science 4 355–367.
  • [10] Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and its Application. Academic Press, New York.
  • [11] Heyde, C. C. (1997). Quasi-Likelihood and its Application. Springer, New York.
  • [12] Hu, F. and Rosenberger, W. (2003). Optimality, variability, power: Evaluating response-adaptive randomization procedures for treatment comparisons. J. Amer. Statist. Assoc. 98 671–678.
  • [13] Hu, F. and Rosenberger, W. F. (2006). The Theory of Response-Adaptive Randomization in Clinical Trials. Wiley, Hoboken, NJ.
  • [14] Hu, F. and Zhang, L.-X. (2004). Asymptotic properties of doubly adaptive biased coin designs for multitreatment clinical trials. Ann. Statist. 32 268–301.
  • [15] Huang, P. (2007). A numerical study of a randomized reinforced urn design for binary responses. Technical report, Univ. Missouri–Columbia. A Masters’ project available from N. Flournoy upon request.
  • [16] Janson, S. (2004). Functional limit theorems for multitype branching processes and generalized Pólya urns. Stochastic Process. Appl. 110 177–245.
  • [17] Li, W., Durham, S. D. and Flournoy, N. (1996). Randomized Pólya urn designs. Proc. Biometric Section of the American Statistical Association 166–170. Amer. Statist. Assoc., Alexandria, VA.
  • [18] May, C. (2007). Asymptotics in a randomly reinforced Pólya urn model for sequential, response-adaptive designs. Ph.D. thesis, Univ. Milan.
  • [19] May, C., Paganoni, A. and Secchi, P. (2007). Response-adaptive designs targeting the best treatment for clinical trials with continuous responses. In S. Co. 2007 Fifth Conference—Complex Models and Computational Intensive Methods for Estimation and Prediction—Book of Short Papers 326–331. Cleup, Padova.
  • [20] May, C., Paganoni, A. M. and Secchi, P. (2005). On a two-color generalized Pólya urn. Metron 63 115–134.
  • [21] Melfi, V. F. and Page, C. (2000). Estimation after adaptive allocation. J. Statist. Plann. Inference 87 353–363.
  • [22] Muliere, P., Paganoni, A. and Secchi, P. (2006a). A randomly reinforced urn. J. Statist. Plann. Inference 136 1853–1874.
  • [23] Muliere, P., Paganoni, A. and Secchi, P. (2006b). Randomly reinforced urns for clinical trials with continuous responses. In SIS—Proceedings of the XLIII Scientific Meeting 403–414. Cleup, Padova.
  • [24] Paganoni, A. and Secchi, P. (2007). A numerical study for comparing two response-adaptive designs for continuous treatment effects. Statist. Methods Appl. 16 321–346.
  • [25] Pemantle, R. and Volkov, S. (1999). Vertex-reinforced random walk on Z has finite range. Ann. Probab. 27 1368–1388.
  • [26] Rényi, A. (1963). On stable sequences of events. Sankhyā Ser. A 25 293–302.
  • [27] Rosenberger, W. F. (2002). Randomized urn models and sequential design. Sequential Anal. 21 1–28.
  • [28] Rosenberger, W. F. and Lachin, J. M. (2002). Randomization in Clinical Trials. Wiley, New York.
  • [29] Zhang, L. and Rosenberger, W. F. (2006). Response-adaptive randomization for clinical trials with continuous responses. Biometrics 62 562–569.
  • [30] Zhang, L.-X., Hu, F. and Cheung, S. H. (2006). Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Probab. 16 340–369.