The Annals of Applied Probability

Strongly reinforced Pólya urns with graph-based competition

Remco van der Hofstad, Mark Holmes, Alexey Kuznetsov, and Wioletta Ruszel

Full-text: Open access


We introduce a class of reinforcement models where, at each time step $t$, one first chooses a random subset $A_{t}$ of colours (independently of the past) from $n$ colours of balls, and then chooses a colour $i$ from this subset with probability proportional to the number of balls of colour $i$ in the urn raised to the power $\alpha>1$. We consider stability of equilibria for such models and establish the existence of phase transitions in a number of examples, including when the colours are the edges of a graph; a context which is a toy model for the formation and reinforcement of neural connections. We conjecture that for any graph $G$ and all $\alpha$ sufficiently large, the set of stable equilibria is supported on so-called whisker-forests, which are forests whose components have diameter between 1 and 3.

Article information

Ann. Appl. Probab., Volume 26, Number 4 (2016), 2494-2539.

Received: May 2014
Revised: July 2015
First available in Project Euclid: 1 September 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]
Secondary: 37C10: Vector fields, flows, ordinary differential equations

Reinforcement model Pólya urn stochastic approximation algorithm stable equilibria


van der Hofstad, Remco; Holmes, Mark; Kuznetsov, Alexey; Ruszel, Wioletta. Strongly reinforced Pólya urns with graph-based competition. Ann. Appl. Probab. 26 (2016), no. 4, 2494--2539. doi:10.1214/16-AAP1153.

Export citation


  • [1] Benaïm, M. (1996). A dynamical system approach to stochastic approximations. SIAM J. Control Optim. 34 437–472.
  • [2] Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math. 1709 1–68. Springer, Berlin.
  • [3] Benaïm, M., Benjamini, I., Chen, J. and Lima, Y. (2015). A generalized Pólya’s urn with graph based interactions. Random Structures Algorithms 46 614–634.
  • [4] Benaïm, M., Raimond, O. and Schapira, B. (2013). Strongly vertex-reinforced-random-walk on a complete graph. ALEA Lat. Am. J. Probab. Math. Stat. 10 767–782.
  • [5] Brouwer, A. E. and Haemers, W. H. (2012). Spectra of Graphs. Springer, New York.
  • [6] Chen, H.-F. (2002). Stochastic Approximation and Its Applications. Nonconvex Optimization and Its Applications 64. Kluwer Academic, Dordrecht.
  • [7] Davis, B. (1990). Reinforced random walk. Probab. Theory Related Fields 84 203–229.
  • [8] Durrett, R., Kesten, H. and Limic, V. (2002). Once edge-reinforced random walk on a tree. Probab. Theory Related Fields 122 567–592.
  • [9] Gopnic, A., Meltzoff, A. and Kuhl, P. (1999). The Scientist in the Crib: What Early Learning Tells Us About the Mind. HarperCollins Publishers, New York, NY.
  • [10] Hirsch, M. W. (1976). Differential Topology. Springer, New York.
  • [11] Holmes, M. and Kelptsyn, V. Proof of the WARM whisker conjecture for neuronal connections.
  • [12] Kushner, H. J. and Yin, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed. Applications of Mathematics (New York) 35. Springer, New York.
  • [13] Launay, M. (2012). Urnes interagissantes. Ph.D. thesis, Univ. Aix-Marseille.
  • [14] Launay, M. (2012). Interacting urn models. Available at arXiv:1101.1410.
  • [15] Launay, M. (2012). Urns with simultaneous drawing. Available at arXiv:1201.3495.
  • [16] Launay, M. and Limic, V. (2012). Generalized interacting urn models. Available at arXiv:1207.5635.
  • [17] Limic, V. and Tarrès, P. (2007). Attracting edge and strongly edge reinforced walks. Ann. Probab. 35 1783–1806.
  • [18] Mahmoud, H. M. (2009). Pólya Urn Models. Texts in Statistical Science Series. CRC Press, Boca Raton, FL.
  • [19] Merkl, F. and Rolles, S. W. W. (2009). Recurrence of edge-reinforced random walk on a two-dimensional graph. Ann. Probab. 37 1679–1714.
  • [20] Nishino, T. (2001). Function Theory in Several Complex Variables. Translations of Mathematical Monographs 193. Amer. Math. Soc., Providence, RI.
  • [21] Pemantle, R. (1988). Phase transition in reinforced random walk and RWRE on trees. Ann. Probab. 16 1229–1241.
  • [22] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surv. 4 1–79.
  • [23] Shabat, B. V. (1992). Introduction to Complex Analysis. Part II. Functions of Several Variables. Translations of Mathematical Monographs 110. Amer. Math. Soc., Providence, RI. Translated from the third (1985) Russian edition by J. S. Joel.
  • [24] Stiles, J. and Jernigan, T. L. (2010). The basics of brain development. Neuropsychol. Rev. 20 327–348.
  • [25] van den Heuvel, M. P., Stam, C. J., Kahn, R. S. and Hulshoff Pol, H. E. (2009). Efficiency of functional brain networks and intellectual performance. J. Neurosci. 29 7619–7624.