### Asymptotic properties of multicolor randomly reinforced Pólya urns

#### Abstract

The generalized Pólya urn has been extensively studied and is widely applied in many disciplines. An important application of urn models is in the development of randomized treatment allocation schemes in clinical studies. The randomly reinforced urn was recently proposed, but, although the model has some intuitively desirable properties, it lacks theoretical justification. In this paper we obtain important asymptotic properties for multicolor reinforced urn models. We derive results for the rate of convergence of the number of patients assigned to each treatment under a set of minimum required conditions and provide the distributions of the limits. Furthermore, we study the asymptotic behavior for the nonhomogeneous case.

#### Article information

Source
Adv. in Appl. Probab., Volume 46, Number 2 (2014), 585-602.

Dates
First available in Project Euclid: 29 May 2014

https://projecteuclid.org/euclid.aap/1401369708

Digital Object Identifier
doi:10.1239/aap/1401369708

Mathematical Reviews number (MathSciNet)
MR3215547

Zentralblatt MATH identifier
1296.60079

#### Citation

Zhang, Li-Xin; Hu, Feifang; Cheung, Siu Hung; Chan, Wai Sum. Asymptotic properties of multicolor randomly reinforced Pólya urns. Adv. in Appl. Probab. 46 (2014), no. 2, 585--602. doi:10.1239/aap/1401369708. https://projecteuclid.org/euclid.aap/1401369708

#### References

• Aletti, G., Ghiglietti, A. and Paganoni, A. M. (2013). Randomly reinforced urn designs with prespecified allocations. J. Appl. Prob. 50, 486–498.
• Aletti, G., May, C. and Secchi, P. (2007). On the distribution of the limit proportion for a two-color, randomly reinforced urn with equal reinforcement distributions. Adv. Appl. Prob. 39, 690–707.
• Aletti, G., May, C. and Secchi, P. (2009). A central limit theorem, and related results, for a two-color randomly reinforced urn. Adv. Appl. Prob. 41, 829–844.
• Aletti, G., May, C. and Secchi, P. (2012). A functional equation whose unknown is $\mathcal{P}([0,1])$ valued. J. Theoret. Prob. 25, 1207–1232.
• Athreya, K. B. and Ney, P. E. (1972). Branching Processes. Springer, New York.
• Bai, Z. D. and Hu, F. (1999). Asymptotic theorems for urn models with nonhomogeneous generating matrices. Stoch. Process. Appl. 80, 87–101.
• Bai, Z.-D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Prob. 15, 914–940.
• Bai, Z. D., Hu, F. and Rosenberger, W. F. (2002). Asymptotic properties of adaptive designs for clinical trials with delayed response. Ann. Statist. 30, 122–139.
• Beggs, A. W. (2005). On the convergence of reinforcement learning. J. Econom. Theory 122, 1–36.
• Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2010). Central limit theorems for multicolor urns with dominated colors. Stoch. Process. Appl. 120, 1473–1491.
• Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2011). A central limit theorem and its applications to multicolor randomly reinforced urns. J. Appl. Prob. 48, 527–546.
• Chauvin, B., Pouyanne, N. and Sahnoun, R. (2011). Limit distributions for large Pólya urns. Ann. Appl. Prob. 21, 1–32.
• Durham, S. D. and Yu, K. F. (1990). Randomized play-the leader rules for sequential sampling from two populations. Prob. Eng. Inf. Sci. 4, 355–367.
• Durham, S. D., Flournoy, N. and Li, W. (1998). A sequential design for maximizing the probability of a favourable response. Canad. J. Statist. 26, 479–495.
• Erev, I. and Roth, A. E. (1998). Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econom. Rev. 88, 848–881.
• Ghiglietti, A. and Paganoni, A. M. (2013). Randomly reinforced urn designs whose allocation proportions converge to arbitrary prespecifed values. In mODa 10–Advances in Model-Oriented Design and Analysis, Springer, Cham, pp. 99–106.
• Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
• Hu, F. and Rosenberger, W. F. (2006). The Theory of Response-Adaptive Randomization in Clinical Trials. John Wiley, Hoboken, NJ.
• Hu, F. and Zhang, L.-X. (2004). Asymptotic properties of doubly adaptive biased coin designs for multi-treatment clinical trials. Ann. Statist. 32, 268–301.
• Janson, S. (2004). Functional limit theorems for multitype branching processes and generalized Pólya urns. Stoch. Process. Appl. 110, 177–245.
• Janson, S. (2006). Limit theorems for triangular urn schemes. Prob. Theory Relat. Fields 134, 417–452.
• Martin, C. F. and Ho, Y. C. (2002). Value of information in the Polya urn process. Inf. Sci. 147, 65–90.
• May, C. and Flournoy, N. (2009). Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn. Ann. Statist. 37, 1058–1078.
• Melfi, V. F. and Page, C. (2000). Estimation after adaptive allocation. J. Statist. Planning Inference 87, 353–363.
• Muliere, P., Paganoni, A. M. and Secchi, P. (2006). A randomly reinforced urn. J. Statist. Planning Inference 136, 1853–1874.
• Muliere, P., Paganoni, A. M. and Secchi, P. (2006). Randomly reinforced urns for clinical trials with continuous responses. In SIS Proc. XLIII Scientific Meeting, Cleup, Padova, pp. 403–414.
• Paganoni, A. and Secchi, P. (2007). A numerical study for comparing two response-adaptive designs for continuous treatment effects. Statist. Meth. Appl. 16, 321–346.
• Pólya, G. (1930). Sur quelques points de la théorie des probabilités. Ann. Inst. H. Poincaré 1, 117–161.
• Rosenberger, W. F. (2002). Randomized urn models and sequential design. Sequential Anal. 21, 1–41.
• Rosenberger, W. F. and Lachin, J. M. (2002). Randomization in Clinical Trials. Theory and Practice. John Wiley, New York.
• Wei, L. J. and Durham, S. (1978). The randomized play-the-winner rule in medical trials. J. Amer. Statist. Assoc. 73, 840–843.
• Zhang, L. X., Hu, F. and Cheung, S. H. (2006). Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Prob. 16, 340–369.
• Zhang, L. X., Hu, F., Cheung. S. H. and Chan, W. S. (2011). Immigrated urn models–-theoretical properties and applications. Ann. Statist. 39, 643–671.