Weakly reinforced P\'olya urns on countable networks

We study the long-time asymptotics of a network of weakly reinforced P\'olya urns. In this system, which extends the WARM introduced by R. van der Hofstad et. al. (2016) to countable networks, the nodes fire at times given by a Poisson point process. When a node fires, one of the incident edges is selected with a probability proportional to its weight raised to a power $\alpha<1$, and then this weight is increased by $1$. We show that for $\alpha<1/2$ on a network of bounded degrees, every edge is reinforced a positive proportion of time, and that the limiting proportion can be interpreted as an equilibrium in a countable network. Moreover, in the special case of regular graphs, this homogenization remains valid beyond the threshold $\alpha = 1/2$.


Introduction
Pólya's urn process is the paradigm model for a random process incorporating reinforcement effects. However, when thinking in the direction of applications, a single urn often does not represent complex interactions accurately. For instance, in the field of social sciences, the formation of friendship networks could be related to reinforcement effects in social interactions [6]. In economy, in a network of companies competing on a variety of products, the global reputation could influence the market shares of the products differently [1,2]. Finally, in the domain of neuroscience, it is plausible that synapses that were successful in the past should be selected again in the future and reinforced with a higher probability [9].
In the present paper, we focus on the network formation model proposed in [9] and study its long-time behavior in the regime of weak reinforcement. In that model, starting from a base network, nodes are picked sequentially at random. Once a node is selected, we choose one of the incident edges with a probability proportional to its weight to some power α > 0 and increase that weight by 1. The analysis of [9] concerns the asymptotic proportion of times that edges are reinforced in the regime of strong reinforcement, where α > 1. In this regime, the limiting proportion is random and coincides with some stable equilibrium in an associated dynamical system, which is typically concentrated on a small subset of the edges if α is large.
In contrast, we consider the regime of weak reinforcement, where α < 1 and find that in many settings the reinforcement proportions converge to a uniquely determined limit equilibrium, which is supported on all edges of the base network. Figure 1 illustrates the network at an early and at a late time point.
Hence, together with the analysis in [9,5], our main result is a first step towards a networkbased analog of Rubin's dichotomy for classical Pólya urns: while for strong reinforcement some edges are only reinforced finitely often, in the weakly reinforced regime all edges are reinforced a positive proportion of time. Although this description outlines the broader picture, more research is needed to carve out the precise conditions for this dichotomy. Indeed, [3] describes an example of a meticulously designed network together with firing rates where even for α > 1 there is percolation of edges that are reinforced a positive proportion of time.
Similarly to the setting of [4], one major difficulty in the analysis stems from the choice of the base graph. In contrast to [1,9], we do not restrict our attention to finite base graphs, but work on a countable network with uniformly bounded degrees. In particular, the highly developed machinery of stochastic approximation algorithms invoked in [1,9] is not available as the state space of the associated dynamical system would be infinite-dimensional. In [4], the very strong reinforcement leads to an effective decomposition of the countable network into finite islands separated by edges that are never reinforced. This trick recovers the finite-dimensional setting. However, the strategy cannot work in the regime of weak reinforcement, where we expect all edges to be reinforced a positive proportion of time.
Hence, in the present paper, we follow an entirely different plan to prove our main result. Namely, convergence of the normalized edge weights to an equilibrium for (1) all graphs of uniformly bounded degree if α < 1/2, (2) regular graphs if α < 1/2 + ε for some ε > 0, and (3) Z if α < 1. This plan consists of three critical steps. First, we invoke a compactness argument to obtain an equilibrium on the entire countable network. Second, we establish that all edges are reinforced a positive proportion of time. This step rests on a percolation argument, where we decompose the network again into finite islands separated by edges that are now reinforced a positive proportion of time. Finally, to obtain convergence to the equilibrium, we rely on a homogenizing bootstrap argument. It formalizes the intuition that for weak reinforcement, deviations from the equilibrium decrease over time.
The rest of the manuscript is organized as follows. Section 2 introduces the model and states the main contribution of the paper, a homogenization result in the sub-linear regime for three different combinations of graphs and exponents. In Section 3, we consider graphs of bounded degree and α < 1/2. Finally, Section 4 deals with regular graphs of degree ∆ and covers α < 1/2 + ε for ∆ > 2 and α < 1 for the graph Z.

Model definition and main result
Let G = (V, E) be a countable graph with uniformly bounded degrees. That is, the vertex set V is countable and If ∆ = deg(v) for every v ∈ V then the graph is regular with degree ∆, which we write as ∆-regular. We investigate a system of random variables of interacting Pólya-type urns on the edge set E at continuous time t ≥ 0 on a probability space (Ω, P). The dynamics of N t are a continuous-time analog of the process considered in [9]. Loosely speaking, every vertex has a Poisson clock and whenever that clock rings at a node v ∈ V , the dynamics choose and increment the weights on one of the incident edges E v = {e ∈ E : v ∈ e} by 1. The choice of the edge happens using the power-weighted Pólya-scheme Henceforth, we tacitly assume that α < 1. More precisely, the weights N t (e) are initialized at 1 and the dynamics of N t are governed by an iid family of Poisson processes For the existence of the process {N t } t≥0 on bounded-degree graphs, see [4].
holds for all e ∈ E with µ(e) > 0. This is a straightforward extension of the notion of finite equilibria from [9] to countable networks. Now, let X t := N t t denote the time-normalized system of weights at time t > 0. We say that X t exhibits homogenization if the equilibrium measure µ exists, is unique and X t → µ almost surely.

Conjecture 2.1 (Homogenization). Let G be a countable graph with uniformly bounded degrees.
Then, X t exhibits homogenization.
In the present work, we verify this conjecture for three combinations of G and α. Theorem 2.2 (Homogenization). X t exhibits homogenization in the following cases.

Proof of Theorem 2.2 (1)
In this section, we establish part (1) of Theorem 2.2. That is, we show homogenization for α < 1/2 and graphs of bounded degree. First, in Section 3.1, we prove existence of a nonvanishing equilibrium. Then, in Section 3.2, we show that X t converges to such an equilibrium, which is in fact uniquely determined.
3.1. Existence of equilibrium. Henceforth, we call an equilibrium µ ∈ R E + non-vanishing if µ(e) > 0 for every e ∈ E. Before discussing existence of non-vanishing equilibria on general graphs, we present the ∆-regular case as a particularly easy example.
The general case of bounded-degree graphs does not admit such an easy analysis as the equilibrium could in principle assume an infinite number of different values. Nevertheless, we deduce existence from a compactness argument.

Proposition 3.2 (Existence of non-vanishing equilibria). Every graph with degrees uniformly bounded by ∆ exhibits at least one non-vanishing equilibrium
Proof. By Equation (2), the equilibria correspond to fixed points of the continuous operator As a product of locally convex and Hausdorff spaces, R E + is again locally convex and Hausdorff. Now, define the closed set C = [2∆ −1/(1−α) , 2] E , which as a product of convex and compact sets, is again convex and compact. We claim that T (C) ⊆ C. Once this claim is established, Schauder's fixed point theorem yields a non-vanishing equilibrium in C.
To prove T (C) ⊆ C, note that for every µ ∈ C and e = {v 1 , v 2 } ∈ E we have as claimed.

Proof of Theorem 2.2 (1). The proof of part (1) of Theorem 2.2 is based on three pivotal ingredients.
First, as each edge is incident to two vertices firing at rate 1, its weight grows at rate at most 2. In other words, for every e ∈ E, almost surely,  This task is slightly more subtle than the upper bound, since arbitrarily small rates can occur with positive probability. To approach this challenge, we fix throughout this section a nonvanishing equilibrium µ as in Proposition 3.2 and write for the family of edges adjacent to a given edge e ∈ E. Then, an edge e ∈ E is δ-stable if

Lemma 3.4 (Rate lower bound).
There exists δ 0 > 0 such that with probability 1, all connected components of δ 0 -unstable edges are finite.
Finally, we rely on a bootstrapping procedure to push the weights iteratively closer to 1. To that end, define  Before establishing Lemmas 3.3-3.5, we elucidate how they can be combined to yield the proof of part (1) of Theorem 2.2.

̺µ(e)] for every e ∈ E}
is at most δ −1 0 . Now, ̺ * = 1 almost surely since otherwise the bootstrap property yields that for every e ∈ E, thereby contradicting the choice of ̺ * . It remains to establish the auxiliary results. We start by proving Lemma 3.3.

Proof of Lemma 3.3.
One extremal case is that each clock-ring event for P v increments the value for an edge e = {v, w} ∈ E. The process t counts the normalized occurrences for this upper bound and has an expected value of Then, the strong law of large numbers for homogeneous Poisson point processes, gives that almost surely lim t→∞ Y t = 2. Now, X t (e) ≤ Y t + 1 t for all t ≥ 0, implies that almost surely, lim sup t→∞ X t (e) ≤ lim t→∞ Y t = 2.
Next, we verify the bootstrap property.
Proof of Lemma 3.5. Let ̺ > 1 and e = {v 1 , v 2 } ∈ E be such that C(e ′ ) ⊆ [̺ −1 µ(e ′ ), ̺µ(e ′ )] for all e ′ ∈ E e . Moreover, for ε > 0 set ̺ ε = (1+ ε)̺. Then, there exists a random time T < ∞ such that X t (e ′ ) ∈ [̺ −1 ε µ(e ′ ), ̺ ε µ(e ′ )] for all t ≥ T and e ′ ∈ E e . In particular, for every i ∈ {1, 2}, Therefore, using that µ is an equilibrium, {N t (e)} t≥T is dominated from below by a Poisson process with intensity Hence, almost surely, X − (e) ≥ ̺ −2α ε µ(e). Similar arguments yield that X + (e) ≤ ̺ 2α ε µ(e). The proof of Lemma 3.4 is the most challenging part of the auxiliary results. The main ingredient is Lemma 3.6 below. It states that X − (e) is bounded away from 0 with a high probability, even when conditioning on F e c , where for e ∈ E we let denote the σ-algebra generated by all Poisson processes at nodes that are at distance at least 2 away from the edge e. Lemma 3.6 (Compact containment of C(e)). For every ε > 0 there exists δ > 0 such that almost surely inf In particular, P(X − (e) > 0 | F e c ) = 1 for every e ∈ E.
Hence, we conclude from dependent percolation theory in the form of [7, Theorem 0.0] that the edges violating the lower bound are restricted to finite, well-separated islands. To make the presentation self-contained, we give an elementary direct proof.

Proof of Lemma 3.4.
Let v ∈ V be arbitrary. We resort to a first-moment argument and show that for sufficiently small δ the expected number of length-n self-avoiding paths of δ-unstable edges tends to 0 as n → ∞. Since the number of length-n self-avoiding paths in G starting from v is at most ∆ n , it suffices to show that the probability for any fixed self-avoiding path to consist of δ-unstable edges only is of the order at most (2∆) −n .
Note that the number of edges that are at graph distance at most 3 of a given edge is bounded above by 2∆ 3 . Hence, any self-avoiding path of length n contains at least n/(2∆ 3 ) edges that are of pairwise distance at least 4, so that by Lemma 3.6, the probability that they are all δ-unstable is at most ε(δ) n/(2∆ 3 ) . In particular, choosing δ > 0 such that ε(δ) 1/(2∆ 3 ) ≤ 1/(2∆) concludes the proof.
It remains to prove Lemma 3.6. To that end, we invoke a conditioning argument in the spirit of [1, Lemma 5.2] to provide a bootstrap result propagating large edge weights at a current time point to a considerable duration into the future. For k, ℓ ≥ 1 put Proof. Let ε > 0 be arbitrary. Then, the Poisson concentration inequality [8,Lemma 1.2] implies that the event has a probability tending to 1 with an error decaying exponentially in 2 kℓ . Further, under the event {N 2 kℓ (e) ≥ a k,ℓ } ∩ A e , Equation (1) has a lower bound for times t ∈ T ℓ := [2 kℓ , 2 k(ℓ+1) ] given by Hence, In particular, for t ∈ T ℓ , the Poisson processes P v and P w having points in [2 kℓ , t] × U e where U e ⊆ [0, 1] and |U e | ≥ G k,ℓ implies weight increases of the edge e (by one or more) in the time interval [2 kℓ , t]. Therefore, using Equation (3), we find a constant c such that P (N 2 k(ℓ+1) (e) ≤ a k,ℓ+1 and N 2 kℓ (e) ≥ a k,ℓ ) Now, we conclude the proof by noting that a k,ℓ+1 ≥ ∆ − 1 1−α 2 kℓ (1−α) and that the coefficient of a k,ℓ+1 is negative. Lemma 3.7 propagates the lower bounds on the edge weights through time. As t → ∞, these lower bounds allow to exclude vanishing edge weights.

Regular graphs
Throughout this section, we assume G to be ∆-regular. The key to the proof of parts (2) and (3) of Theorem 2.2 is the following bootstrap property. Since the first three items imply that a i and b i converge to 1, we arrive at Proof. Straightforward calculation as f (a, 2) = 2a α a α + (∆ − 1)2 α > 2a α 2 α ∆ = a 2 1−α a 1−α ∆ > a.
Proof of part (2) of Theorem 2.2. Take the ε from Lemma 4.2 and consider α = 1/2 + ε. X − (e) is bounded above by Lemma 3.3 and strictly positive by Lemma 3.4 and Lemma 4.3 using analogous arguments as in the proof of part (1) of this theorem. So we find a ≤ 2/∆ and b ≥ 2/∆ such that C(e) ⊆ [a, b] for all e ∈ E. We choose a maximal and b minimal with that property. To derive a contradiction, assume that a < b. Then, by Lemma 4.1 and Lemma 4.2 we can tighten the bounds as This contradicts the maximality/minimality of a and b, and since f (a, b) ≤ 2/∆ ≤ f (b, a) therefore gives a = b = 2/∆.