## Bernoulli

• Bernoulli
• Volume 24, Number 3 (2018), 2204-2255.

### Dynamics of an adaptive randomly reinforced urn

#### Abstract

Adaptive randomly reinforced urn (ARRU) is a two-color urn model where the updating process is defined by a sequence of non-negative random vectors $\{(D_{1,n},D_{2,n});n\geq1\}$ and randomly evolving thresholds which utilize accruing statistical information for the updates. Let $m_{1}=E[D_{1,n}]$ and $m_{2}=E[D_{2,n}]$. In this paper, we undertake a detailed study of the dynamics of the ARRU model. First, for the case $m_{1}\neq m_{2}$, we establish $L_{1}$ bounds on the increments of the urn proportion, that is, the proportion of ball colors in the urn, at fixed and increasing times under very weak assumptions on the random threshold sequences. As a consequence, we deduce weak consistency of the evolving urn proportions. Second, under slightly stronger conditions, we establish the strong consistency of the urn proportions for all finite values of $m_{1}$ and $m_{2}$. Specifically, we show that when $m_{1}=m_{2}$, the proportion converges to a non-degenerate random variable. Third, we establish the asymptotic distribution, after appropriate centering and scaling, for the proportion of sampled ball colors and urn proportions for the case $m_{1}=m_{2}$. In the process, we resolve the issue concerning the asymptotic distribution of the proportion of sampled ball colors for a randomly reinforced urn (RRU). To address the technical issues, we establish results on the harmonic moments of the total number of balls in the urn at different times under very weak conditions, which is of independent interest.

#### Article information

Source
Bernoulli, Volume 24, Number 3 (2018), 2204-2255.

Dates
Revised: February 2017
First available in Project Euclid: 2 February 2018

Permanent link to this document
https://projecteuclid.org/euclid.bj/1517540473

Digital Object Identifier
doi:10.3150/17-BEJ926

Mathematical Reviews number (MathSciNet)
MR3757528

Zentralblatt MATH identifier
06839265

#### Citation

Aletti, Giacomo; Ghiglietti, Andrea; Vidyashankar, Anand N. Dynamics of an adaptive randomly reinforced urn. Bernoulli 24 (2018), no. 3, 2204--2255. doi:10.3150/17-BEJ926. https://projecteuclid.org/euclid.bj/1517540473

#### References

• [1] Aletti, G. and Ghiglietti, A. (2017). Interacting generalized Friedman’s urn systems. Stochastic Process. Appl. DOI:10.1016/j.spa.2016.12.003.
• [2] Aletti, G., Ghiglietti, A. and Paganoni, A.M. (2013). Randomly reinforced urn designs with prespecified allocations. J. Appl. Probab. 50 486–498.
• [3] Aletti, G., May, C. and Secchi, P. (2009). A central limit theorem, and related results, for a two-color randomly reinforced urn. Adv. in Appl. Probab. 41 829–844.
• [4] Aletti, G., May, C. and Secchi, P. (2012). A functional equation whose unknown is $\mathcal{P}([0,1])$ valued. J. Theoret. Probab. 25 1207–1232.
• [5] Athreya, K.B. and Karlin, S. (1968). Embedding of urn schemes into continuous time Markov branching processes and related limit theorems. Ann. Math. Stat. 39 1801–1817.
• [6] Bai, Z.D. and Hu, F. (1999). Asymptotic theorems for urn models with nonhomogeneous generating matrices. Stochastic Process. Appl. 80 87–101.
• [7] Bai, Z.D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Probab. 15 914–940.
• [8] Chen, L.H.Y. (1978). A short note on the conditional Borel–Cantelli lemma. Ann. Probab. 6 699–700.
• [9] Crimaldi, I. (2009). An almost sure conditional convergence result and an application to a generalized Pólya urn. Int. Math. Forum 4 1139–1156.
• [10] Crimaldi, I., Letta, G. and Pratelli, L. (2007). A strong form of stable convergence. In Séminaire de Probabilités XL. Lecture Notes in Math. 1899 203–225. Berlin: Springer.
• [11] Dembo, A. and Zeitouni, O. (1998). Large Deviations Techniques and Applications, 2nd ed. Applications of Mathematics (New York) 38. New York: Springer.
• [12] Durham, S.D., Flournoy, N. and Li, W. (1998). A sequential design for maximizing the probability of a favourable response. Canad. J. Statist. 26 479–495.
• [13] Durrett, R. (2010). Probability: Theory and Examples, 4th ed. Cambridge Series in Statistical and Probabilistic Mathematics 31. Cambridge: Cambridge Univ. Press.
• [14] Ghiglietti, A. and Paganoni, A.M. (2014). Statistical properties of two-color randomly reinforced urn design targeting fixed allocations. Electron. J. Stat. 8 708–737.
• [15] Ghiglietti, A. and Paganoni, A.M. (2016). An urn model to construct an efficient test procedure for response adaptive designs. Stat. Methods Appl. 25 211–226.
• [16] Ghiglietti, A., Vidyashankar, A.N. and Rosenberger, W.F. (2017). Central limit theorem for an adaptive randomly reinforced urn model. Ann. Appl. Probab. To appear.
• [17] Hall, P. and Heyde, C.C. (1980). Martingale Limit Theory and Its Application. Probability and Mathematical Statistics. New York–London: Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers].
• [18] Hu, F. and Rosenberger, W.F. (2006). The Theory of Response-Adaptive Randomization in Clinical Trials. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley-Interscience [John Wiley & Sons].
• [19] Laruelle, S. and Pagès, G. (2013). Randomized urn models revisited using stochastic approximation. Ann. Appl. Probab. 23 1409–1436.
• [20] Mahmoud, H. (2008). Pólya Urn Models. Boca Raton, FL: CRC press.
• [21] May, C. and Flournoy, N. (2009). Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn. Ann. Statist. 37 1058–1078.
• [22] Muliere, P., Paganoni, A.M. and Secchi, P. (2006). A randomly reinforced urn. J. Statist. Plann. Inference 136 1853–1874.
• [23] Pemantle, R. and Volkov, S. (1999). Vertex-reinforced random walk on $\mathbf{Z}$ has finite range. Ann. Probab. 27 1368–1388.
• [24] Smythe, R.T. (1996). Central limit theorems for urn models. Stochastic Process. Appl. 65 115–137.
• [25] Zhang, L.-X., Hu, F. and Cheung, S.H. (2006). Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Probab. 16 340–369.