Advances in Applied Probability

Asymptotic properties of multicolor randomly reinforced Pólya urns

Li-Xin Zhang, Feifang Hu, Siu Hung Cheung, and Wai Sum Chan

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


The generalized Pólya urn has been extensively studied and is widely applied in many disciplines. An important application of urn models is in the development of randomized treatment allocation schemes in clinical studies. The randomly reinforced urn was recently proposed, but, although the model has some intuitively desirable properties, it lacks theoretical justification. In this paper we obtain important asymptotic properties for multicolor reinforced urn models. We derive results for the rate of convergence of the number of patients assigned to each treatment under a set of minimum required conditions and provide the distributions of the limits. Furthermore, we study the asymptotic behavior for the nonhomogeneous case.

Article information

Adv. in Appl. Probab., Volume 46, Number 2 (2014), 585-602.

First available in Project Euclid: 29 May 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60F15: Strong theorems 62G10: Hypothesis testing
Secondary: 60F05: Central limit and other weak theorems 60F10: Large deviations

Response-adaptive design asymptotic normality clinical trial urn model branching process with immigration rate of convergence


Zhang, Li-Xin; Hu, Feifang; Cheung, Siu Hung; Chan, Wai Sum. Asymptotic properties of multicolor randomly reinforced Pólya urns. Adv. in Appl. Probab. 46 (2014), no. 2, 585--602. doi:10.1239/aap/1401369708.

Export citation


  • Aletti, G., Ghiglietti, A. and Paganoni, A. M. (2013). Randomly reinforced urn designs with prespecified allocations. J. Appl. Prob. 50, 486–498.
  • Aletti, G., May, C. and Secchi, P. (2007). On the distribution of the limit proportion for a two-color, randomly reinforced urn with equal reinforcement distributions. Adv. Appl. Prob. 39, 690–707.
  • Aletti, G., May, C. and Secchi, P. (2009). A central limit theorem, and related results, for a two-color randomly reinforced urn. Adv. Appl. Prob. 41, 829–844.
  • Aletti, G., May, C. and Secchi, P. (2012). A functional equation whose unknown is $\mathcal{P}([0,1])$ valued. J. Theoret. Prob. 25, 1207–1232.
  • Athreya, K. B. and Ney, P. E. (1972). Branching Processes. Springer, New York.
  • Bai, Z. D. and Hu, F. (1999). Asymptotic theorems for urn models with nonhomogeneous generating matrices. Stoch. Process. Appl. 80, 87–101.
  • Bai, Z.-D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Prob. 15, 914–940.
  • Bai, Z. D., Hu, F. and Rosenberger, W. F. (2002). Asymptotic properties of adaptive designs for clinical trials with delayed response. Ann. Statist. 30, 122–139.
  • Beggs, A. W. (2005). On the convergence of reinforcement learning. J. Econom. Theory 122, 1–36.
  • Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2010). Central limit theorems for multicolor urns with dominated colors. Stoch. Process. Appl. 120, 1473–1491.
  • Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2011). A central limit theorem and its applications to multicolor randomly reinforced urns. J. Appl. Prob. 48, 527–546.
  • Chauvin, B., Pouyanne, N. and Sahnoun, R. (2011). Limit distributions for large Pólya urns. Ann. Appl. Prob. 21, 1–32.
  • Durham, S. D. and Yu, K. F. (1990). Randomized play-the leader rules for sequential sampling from two populations. Prob. Eng. Inf. Sci. 4, 355–367.
  • Durham, S. D., Flournoy, N. and Li, W. (1998). A sequential design for maximizing the probability of a favourable response. Canad. J. Statist. 26, 479–495.
  • Erev, I. and Roth, A. E. (1998). Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econom. Rev. 88, 848–881.
  • Ghiglietti, A. and Paganoni, A. M. (2013). Randomly reinforced urn designs whose allocation proportions converge to arbitrary prespecifed values. In mODa 10–Advances in Model-Oriented Design and Analysis, Springer, Cham, pp. 99–106.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Hu, F. and Rosenberger, W. F. (2006). The Theory of Response-Adaptive Randomization in Clinical Trials. John Wiley, Hoboken, NJ.
  • Hu, F. and Zhang, L.-X. (2004). Asymptotic properties of doubly adaptive biased coin designs for multi-treatment clinical trials. Ann. Statist. 32, 268–301.
  • Janson, S. (2004). Functional limit theorems for multitype branching processes and generalized Pólya urns. Stoch. Process. Appl. 110, 177–245.
  • Janson, S. (2006). Limit theorems for triangular urn schemes. Prob. Theory Relat. Fields 134, 417–452.
  • Martin, C. F. and Ho, Y. C. (2002). Value of information in the Polya urn process. Inf. Sci. 147, 65–90.
  • May, C. and Flournoy, N. (2009). Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn. Ann. Statist. 37, 1058–1078.
  • Melfi, V. F. and Page, C. (2000). Estimation after adaptive allocation. J. Statist. Planning Inference 87, 353–363.
  • Muliere, P., Paganoni, A. M. and Secchi, P. (2006). A randomly reinforced urn. J. Statist. Planning Inference 136, 1853–1874.
  • Muliere, P., Paganoni, A. M. and Secchi, P. (2006). Randomly reinforced urns for clinical trials with continuous responses. In SIS Proc. XLIII Scientific Meeting, Cleup, Padova, pp. 403–414.
  • Paganoni, A. and Secchi, P. (2007). A numerical study for comparing two response-adaptive designs for continuous treatment effects. Statist. Meth. Appl. 16, 321–346.
  • Pólya, G. (1930). Sur quelques points de la théorie des probabilités. Ann. Inst. H. Poincaré 1, 117–161.
  • Rosenberger, W. F. (2002). Randomized urn models and sequential design. Sequential Anal. 21, 1–41.
  • Rosenberger, W. F. and Lachin, J. M. (2002). Randomization in Clinical Trials. Theory and Practice. John Wiley, New York.
  • Wei, L. J. and Durham, S. (1978). The randomized play-the-winner rule in medical trials. J. Amer. Statist. Assoc. 73, 840–843.
  • Zhang, L. X., Hu, F. and Cheung, S. H. (2006). Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Prob. 16, 340–369.
  • Zhang, L. X., Hu, F., Cheung. S. H. and Chan, W. S. (2011). Immigrated urn models–-theoretical properties and applications. Ann. Statist. 39, 643–671.