Open Access
May 2020 Interacting reinforced stochastic processes: Statistical inference based on the weighted empirical means
Giacomo Aletti, Irene Crimaldi, Andrea Ghiglietti
Bernoulli 26(2): 1098-1138 (May 2020). DOI: 10.3150/19-BEJ1143


This work deals with a system of interacting reinforced stochastic processes, where each process $X^{j}=(X_{n,j})_{n}$ is located at a vertex $j$ of a finite weighted directed graph, and it can be interpreted as the sequence of “actions” adopted by an agent $j$ of the network. The interaction among the dynamics of these processes depends on the weighted adjacency matrix $W$ associated to the underlying graph: indeed, the probability that an agent $j$ chooses a certain action depends on its personal “inclination” $Z_{n,j}$ and on the inclinations $Z_{n,h}$, with $h\neq j$, of the other agents according to the entries of $W$. The best known example of reinforced stochastic process is the Pólya urn.

The present paper focuses on the weighted empirical means $N_{n,j}=\sum_{k=1}^{n}q_{n,k}X_{k,j}$, since, for example, the current experience is more important than the past one in reinforced learning. Their almost sure synchronization and some central limit theorems in the sense of stable convergence are proven. The new approach with weighted means highlights the key points in proving some recent results for the personal inclinations $Z^{j}=(Z_{n,j})_{n}$ and for the empirical means $\overline{X}^{j}=(\sum_{k=1}^{n}X_{k,j}/n)_{n}$ given in recent papers (e.g. Aletti, Crimaldi and Ghiglietti (2019), Ann. Appl. Probab. 27 (2017) 3787–3844, Crimaldi et al. Stochastic Process. Appl. 129 (2019) 70–101). In fact, with a more sophisticated decomposition of the considered processes, we can understand how the different convergence rates of the involved stochastic processes combine. From an application point of view, we provide confidence intervals for the common limit inclination of the agents and a test statistics to make inference on the matrix $W$, based on the weighted empirical means. In particular, we answer a research question posed in Aletti, Crimaldi and Ghiglietti (2019).


Download Citation

Giacomo Aletti. Irene Crimaldi. Andrea Ghiglietti. "Interacting reinforced stochastic processes: Statistical inference based on the weighted empirical means." Bernoulli 26 (2) 1098 - 1138, May 2020.


Received: 1 February 2019; Revised: 1 June 2019; Published: May 2020
First available in Project Euclid: 31 January 2020

zbMATH: 07166558
MathSciNet: MR4058362
Digital Object Identifier: 10.3150/19-BEJ1143

Keywords: asymptotic normality , complex networks , interacting random systems , reinforced learning , reinforced stochastic processes , synchronization , urn models , weighted empirical means

Rights: Copyright © 2020 Bernoulli Society for Mathematical Statistics and Probability

Vol.26 • No. 2 • May 2020
Back to Top