Electronic Communications in Probability

A stochastic approximation approach to quasi-stationary distributions on finite spaces

Michel Benaïm and Bertrand Cloez

Full-text: Open access

Abstract

This work is concerned with the analysis of a stochastic approximation algorithm for the simulation of quasi-stationary distributions on finite state spaces. This is a generalization of a methodintroduced by Aldous, Flannery and Palacios. It is shown that the asymptotic behavior of the empirical occupation measure of this process is precisely related to the asymptotic behavior of somedeterministic dynamical system induced by a vector field on the unit simplex. This approach provides new proof of convergence as well as precise asymptotic rates for this type of algorithm. Inthe last part, our convergence results are compared with those of a particle system algorithm (adiscrete-time version of the Fleming-Viot algorithm).

Article information

Source
Electron. Commun. Probab., Volume 20 (2015), paper no. 37, 13 pp.

Dates
Accepted: 12 May 2015
First available in Project Euclid: 7 June 2016

Permanent link to this document
https://projecteuclid.org/euclid.ecp/1465320964

Digital Object Identifier
doi:10.1214/ECP.v20-3956

Mathematical Reviews number (MathSciNet)
MR3352332

Zentralblatt MATH identifier
1321.65009

Subjects
Primary: 65C20: Models, numerical methods [See also 68U20] 60B12: Limit theorems for vector-valued random variables (infinite- dimensional case) 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 34F05: Equations and systems with randomness [See also 34K50, 60H10, 93E03] 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40]

Keywords
Quasi-stationary distributions approximation method reinforced random walks random perturbations of dynamical

Rights
This work is licensed under a Creative Commons Attribution 3.0 License.

Citation

Benaïm, Michel; Cloez, Bertrand. A stochastic approximation approach to quasi-stationary distributions on finite spaces. Electron. Commun. Probab. 20 (2015), paper no. 37, 13 pp. doi:10.1214/ECP.v20-3956. https://projecteuclid.org/euclid.ecp/1465320964


Export citation

References

  • D. Aldous, B. Flannery, and J.-L. Palacios. Two applications of urn processes: the fringe analysis of search trees and the simulation of quasi-stationary distributions of markov chains. Probab. Eng. Inf. Sci., 2(3):293–307, 1988.
  • Athreya, Krishna B.; Ney, Peter E. Branching processes. Die Grundlehren der mathematischen Wissenschaften, Band 196. Springer-Verlag, New York-Heidelberg, 1972. xi+287 pp.
  • Benaïm, Michel. Vertex-reinforced random walks and a conjecture of Pemantle. Ann. Probab. 25 (1997), no. 1, 361–392.
  • Benaïm, Michel. Dynamics of stochastic approximation algorithms. Séminaire de Probabilités, XXXIII, 1–68, Lecture Notes in Math., 1709, Springer, Berlin, 1999.
  • M. Benaim. On Gradient like Properties of Population games, Learning models and Self Reinforced Processes. ArXiv e-prints, Sept. 2014. To appear in textitDynamics, Games and Science, DGS II, Springer-Verlag (J.P Bourguignon, R. Jelstch, A. Pindo and M. Viana, eds).
  • Benaïm, Michel; Ledoux, Michel; Raimond, Olivier. Self-interacting diffusions. Probab. Theory Related Fields 122 (2002), no. 1, 1–41.
  • Benaïm, Michel; Weibull, Jürgen W. Deterministic approximation of stochastic evolution in games. Econometrica 71 (2003), no. 3, 873–903.
  • Benveniste, Albert; Métivier, Michel; Priouret, Pierre. Adaptive algorithms and stochastic approximations. Translated from the French by Stephen S. Wilson. Applications of Mathematics (New York), 22. Springer-Verlag, Berlin, 1990. xii+365 pp. ISBN: 3-540-52894-6
  • J. Blanchet, P. Glynn, and S. Zheng. Theoretical analysis of a Stochastic Approximation approach for computing Quasi-Stationary distributions. ArXiv e-prints, Jan. 2014.
  • Burdzy, Krzysztof; Hołyst, Robert; March, Peter. A Fleming-Viot particle representation of the Dirichlet Laplacian. Comm. Math. Phys. 214 (2000), no. 3, 679–703.
  • B. Cloez. Limit theorems for some branching measure-valued processes. ArXiv e-prints, June 2011.
  • B. Cloez. Comportement asymptotique de processus avec sauts et applications pour des modèles avec branchement. PhD thesis, Université Paris-Est, 2013.
  • B. Cloez and M.-N. Thai. Quantitative results for the Fleming-Viot particle system in discrete space. ArXiv e-prints, Dec. 2013.
  • Collet, Pierre; Martinez, Servet; San Martin, Jaime. Quasi-stationary distributions. Markov chains, diffusions and dynamical systems. Probability and its Applications (New York). Springer, Heidelberg, 2013. xvi+280 pp. ISBN: 978-3-642-33130-5; 978-3-642-33131-2
  • Darroch, J. N.; Seneta, E. On quasi-stationary distributions in absorbing discrete-time finite Markov chains. J. Appl. Probability 2 1965 88–100.
  • Del Moral, Pierre. Mean field simulation for Monte Carlo integration. Monographs on Statistics and Applied Probability, 126. CRC Press, Boca Raton, FL, 2013. xlvii+578 pp. ISBN: 978-1-4665-0405-9
  • Del Moral, Pierre; Guionnet, Alice. On the stability of measure valued processes with applications to filtering. C. R. Acad. Sci. Paris Sér. I Math. 329 (1999), no. 5, 429–434.
  • Del Moral, P.; Miclo, L. A Moran particle system approximation of Feynman-Kac formulae. Stochastic Process. Appl. 86 (2000), no. 2, 193–216.
  • Del Moral, Pierre; Miclo, Laurent. On convergence of chains with occupational self-interactions. Stochastic analysis with applications to mathematical finance. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 460 (2004), no. 2041, 325–346.
  • Del Moral, P.; Miclo, L. Self-interacting Markov chains. Stoch. Anal. Appl. 24 (2006), no. 3, 615–660.
  • Duflo, Marie. Algorithmes stochastiques. (French) [Stochastic algorithms] Mathématiques & Applications (Berlin) [Mathematics & Applications], 23. Springer-Verlag, Berlin, 1996. xiv+319 pp. ISBN: 3-540-60699-8
  • Ferrari, Pablo A.; Marić, Nevena. Quasi stationary distributions and Fleming-Viot processes in countable spaces. Electron. J. Probab. 12 (2007), no. 24, 684–702.
  • G. Fort. Central Limit Theorems for Stochastic Approximation with controlled Markov chain dynamics. ArXiv e-prints, Sept. 2013. Accepted for publication in Esaim Probab. Stat.
  • Gantmacher, F. R. The theory of matrices. Vols. 1, 2. Translated by K. A. Hirsch Chelsea Publishing Co., New York 1959 Vol. 1, x+374 pp. Vol. 2, ix+276 pp.
  • Groisman, P.; Jonckheere, M. Simulation of quasi-stationary distributions on countable spaces. Markov Process. Related Fields 19 (2013), no. 3, 521–542.
  • Méléard, Sylvie; Villemonais, Denis. Quasi-stationary distributions and population processes. Probab. Surv. 9 (2012), 340–410.
  • Métivier, M.; Priouret, P. Théorémes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant. (French) [Almost sure convergence theorems for a class of decreasing-step stochastic algorithms] Probab. Theory Related Fields 74 (1987), no. 3, 403–428.
  • Pemantle, Robin. A survey of random processes with reinforcement. Probab. Surv. 4 (2007), 1–79.
  • P. Pollett. Quasi-stationary distributions: A bibliography. rlhttp://www.maths.uq.edu.au/~pkp/papers/qsds/qsds.pdf.
  • van Doorn, Erik A.; Pollett, Philip K. Quasi-stationary distributions for discrete-state models. European J. Oper. Res. 230 (2013), no. 1, 1–14.
  • D. Villemonais. Distributions quasi-stationnaires et méthodes particulaires pour l'approximation de processus conditionné. PhD thesis, École Polytechnique, 2011.
  • Villemonais, Denis. General approximation method for the distribution of Markov processes conditioned not to be killed. ESAIM Probab. Stat. 18 (2014), 441–467.