The Annals of Applied Probability

Asymptotic theorems of sequential estimation-adjusted urn models

Li-X. Zhang, Feifang Hu, and Siu Hung Cheung

Full-text: Open access


The Generalized Pólya Urn (GPU) is a popular urn model which is widely used in many disciplines. In particular, it is extensively used in treatment allocation schemes in clinical trials. In this paper, we propose a sequential estimation-adjusted urn model (a nonhomogeneous GPU) which has a wide spectrum of applications. Because the proposed urn model depends on sequential estimations of unknown parameters, the derivation of asymptotic properties is mathematically intricate and the corresponding results are unavailable in the literature. We overcome these hurdles and establish the strong consistency and asymptotic normality for both the patient allocation and the estimators of unknown parameters, under some widely satisfied conditions. These properties are important for statistical inferences and they are also useful for the understanding of the urn limiting process. A superior feature of our proposed model is its capability to yield limiting treatment proportions according to any desired allocation target. The applicability of our model is illustrated with a number of examples.

Article information

Ann. Appl. Probab., Volume 16, Number 1 (2006), 340-369.

First available in Project Euclid: 6 March 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62L05: Sequential design 62F12: Asymptotic properties of estimators
Secondary: 60F05: Central limit and other weak theorems 60F15: Strong theorems

Responsive adaptive design clinical trial asymptotic normality consistency generalized Pólya urn treatment allocation


Zhang, Li-X.; Hu, Feifang; Cheung, Siu Hung. Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Probab. 16 (2006), no. 1, 340--369. doi:10.1214/105051605000000746.

Export citation


  • Andersen, J., Faries, D. and Tamura, R. N. (1994). Randomized play-the-winner design for multi-arm clinical trials. Comm. Statist. Theory Methods 23 309–323.
  • Athreya, K. B. and Karlin, S. (1967). Limit theorems for the split times of branching processes. J. Math. Mech. 17 257–277.
  • Athreya, K. B. and Karlin, S. (1968). Embedding of urn schemes into continuous time Markov branching processes and related limit theorems. Ann. Math. Statist. 39 1801–1817.
  • Bai, Z. D. and Hu, F. (1999). Asymptotic theorem for urn models with nonhomogeneous generating matrices. Stoch. Process. Appl. 80 87–101.
  • Bai, Z. D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Probab. 15 914–940.
  • Bai, Z. D., Hu, F. and Shen, L. (2002). An adaptive design for multi-arm clinical trials. J. Multivariate Anal. 81 1–18.
  • Bai, Z. D., Hu, F. and Zhang, L. X. (2002). The Gaussian approximation theorems for urn models and their applications. Ann. Appl. Probab. 12 1149–1173.
  • Beggs, A. W. (2005). On the convergence of reinforcement learning. J. Econom. Theory 122 1–36.
  • Benaïm, M., Schreiber, S. J. and Tarrés, P. (2004). Generalized urn models of evolutionary processes. Ann. Appl. Probab. 14 1455–1478.
  • Eggenberger, F. and Pólya, G. (1923). Über die statistik verketetter vorgänge. Z. Ang. Math. Mech. 1 279–289.
  • Eisele, J. (1994). The doubly adaptive biased coin design for sequential clinical trials. J. Statist. Plann. Inference 38 249–262.
  • Eisele, J. and Woodroofe, M. (1995). Central limit theorems for doubly adaptive biased coin designs. Ann. Statist. 23 234–254.
  • Erev, I. and Roth, A. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econ. Rev. 88 848–881.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Applications. Academic Press, London.
  • Hu, F. and Rosenberger, W. F. (2003). Optimality, variability, power: Evaluating response-adaptive randomization procedures for treatment comparisons. J. Amer. Statist. Assoc. 98 671–678.
  • Hu, F. and Zhang, L.-X. (2004). Asymptotic properties of doubly adaptive biased coin designs for multi-treatment clinical trials. Ann. Statist. 32 268–301.
  • Janson, S. (2004). Functional limit theorems for multitype branching processes and generalized Pólya urns. Stoch. Process. Appl. 110 177–245.
  • Jennison, C. and Turnbull, B. W. (2000). Group Sequential Methods with Applications to Clinical Trials. Chapman and Hall/CRC Press, Florida.
  • Johnson, N. L. and Kotz, S. (1977). Urn Models and Their Applications. Wiley, New York.
  • Kotz, S. and Balakrishnan, N. (1997). Advances in urn models during the past two decades. In Advances in Combinatorial Methods and Applications to Probability and Statistics (N. Balakrishnan, ed.). Birkhäuser, Boston.
  • Martin, C. F. and Ho, Y. C. (2002). Value of information in the Pólya urn process. Inform. Sci. 147 65–90.
  • Melfi, V. F. and Page, C. (1998). Variability in adaptive designs for estimation of success probabilities. In New Developments and Applications in Experimental Design (N. Flournoy, W. F. Rosenberger and W. K. Wong, eds.) 106–114. IMS, Hayward, CA.
  • Melfi, V. F. and Page, C. (2000). Estimation after adaptive allocation. J. Statist. Plann. Inference 87 353–363.
  • Robbins, H. (1952). Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. 58 527–535.
  • Rosenberger, W. F. and Lachin, J. M. (2002). Randomization in Clinical Trials: Theory and Practice. Wiley, New York.
  • Rosenberger, W. F., Stallard, N., Ivanova, A., Harper, C. N. and Ricks, M. L. (2001). Optimal adaptive designs for binary response trials. Biometrics 57 909–913.
  • Smythe, R. T. (1996). Central limit theorems for urn models. Stoch. Process. Appl. 65 115–137.
  • Smythe, R. T. and Rosenberger, W. F. (1995). Play-the-winner designs, generalized Pólya urns, and Markov branching processes. In Adaptive Designs (N. Flournoy and W. F. Rosenberger, eds.) 13–22. IMS, Hayward, CA.
  • Thompson, W. R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of the two samples. Biometrika 25 285–294.
  • Wei, L. J. (1979). The generalized Pólya's urn design for sequential medical trials. Ann. Statist. 7 291–296.
  • Wei, L. J. and Durham, S. (1978). The randomized pay-the-winner rule in medical trials. J. Amer. Statist. Assoc. 73 840–843.
  • Windrum, P. (2004). Leveraging technological externalities in complex technologies: Microsoft's exploitation of standards, in the browser wars. Research Policy 33 385–394.