## The Annals of Applied Probability

### Stochastic approximation of quasi-stationary distributions on compact spaces and applications

#### Abstract

As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a nonirreducible setting.

#### Article information

Source
Ann. Appl. Probab., Volume 28, Number 4 (2018), 2370-2416.

Dates
Revised: September 2017
First available in Project Euclid: 9 August 2018

https://projecteuclid.org/euclid.aoap/1533780276

Digital Object Identifier
doi:10.1214/17-AAP1360

Mathematical Reviews number (MathSciNet)
MR3843832

Zentralblatt MATH identifier
06974754

#### Citation

Benaim, Michel; Cloez, Bertrand; Panloup, Fabien. Stochastic approximation of quasi-stationary distributions on compact spaces and applications. Ann. Appl. Probab. 28 (2018), no. 4, 2370--2416. doi:10.1214/17-AAP1360. https://projecteuclid.org/euclid.aoap/1533780276

#### References

• [1] Aldous, D., Flannery, B. and Palacios, J. (1988). Two applications of urn processes: The fringe analysis of search trees and the simulation of quasi-stationary distributions of Markov chains. Probab. Engrg. Inform. Sci. 2 293–307.
• [2] Bansaye, V., Méléard, S. and Richard, M. (2016). Speed of coming down from infinity for birth-and-death processes. Adv. in Appl. Probab. 48 1183–1210.
• [3] Bass, R. F. (1998). Diffusions and Elliptic Operators. Springer, New York.
• [4] Ben-Ari, I. and Pinsky, R. G. (2007). Spectral analysis of a family of second-order elliptic operators with nonlocal boundary condition indexed by a probability measure. J. Funct. Anal. 251 122–140.
• [5] Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math. 1709 1–68. Springer, Berlin.
• [6] Benaïm, M. and Cloez, B. (2015). A stochastic approximation approach to quasi-stationary distributions on finite spaces. Electron. Commun. Probab. 20 no. 37.
• [7] Benaïm, M. and Hirsch, M. W. (1996). Asymptotic pseudotrajectories and chain recurrent flows, with applications. J. Dynam. Differential Equations 8 141–176.
• [8] Benaïm, M., Ledoux, M. and Raimond, O. (2002). Self-interacting diffusions. Probab. Theory Related Fields 122 1–41.
• [9] Berglund, N. and Landon, D. (2012). Mixed-mode oscillations and interspike interval statistics in the stochastic FitzHugh–Nagumo model. Nonlinearity 25 2303–2335.
• [10] Bieniek, M., Burdzy, K. and Finch, S. (2012). Non-extinction of a Fleming–Viot particle model. Probab. Theory Related Fields 153 293–332.
• [11] Bieniek, M., Burdzy, K. and Pal, S. (2012). Extinction of Fleming–Viot-type particle systems with strong drift. Electron. J. Probab. 17 no. 11.
• [12] Blanchet, J., Glynn, P. and Zheng, S. (2016). Analysis of a stochastic approximation algorithm for computing quasi-stationary distributions. Adv. in Appl. Probab. 48 792–811.
• [13] Bouleau, N. and Lépingle, D. (1994). Numerical Methods for Stochastic Processes. Wiley, New York.
• [14] Burdzy, K., Hołyst, R. and March, P. (2000). A Fleming–Viot particle representation of the Dirichlet Laplacian. Comm. Math. Phys. 214 679–703.
• [15] Cattiaux, P., Collet, P., Lambert, A., Martínez, S., Méléard, S. and San Martín, J. (2009). Quasi-stationary distributions and diffusion models in population dynamics. Ann. Probab. 37 1926–1969.
• [16] Champagnat, N. and Villemonais, D. (2016). Exponential convergence to quasi-stationary distribution and $Q$-process. Probab. Theory Related Fields 164 243–283.
• [17] Cloez, B. and Thai, M.-N. (2016). Fleming–Viot processes: Two explicit examples. ALEA Lat. Am. J. Probab. Math. Stat. 13 337–356.
• [18] Cloez, B. and Thai, M.-N. (2016). Quantitative results for the Fleming–Viot particle system and quasi-stationary distributions in discrete space. Stochastic Process. Appl. 126 680–702.
• [19] Collet, P., Martínez, S., Méléard, S. and San Martín, J. (2011). Quasi-stationary distributions for structured birth and death processes with mutations. Probab. Theory Related Fields 151 191–231.
• [20] Del Moral, P. and Guionnet, A. (1999). On the stability of measure valued processes with applications to filtering. C. R. Acad. Sci. Paris, Sér. I Math. 329 429–434.
• [21] Del Moral, P. and Miclo, L. (2000). A Moran particle system approximation of Feynman–Kac formulae. Stochastic Process. Appl. 86 193–216.
• [22] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics 74. Cambridge Univ. Press, Cambridge. Revised reprint of the 1989 original.
• [23] Duflo, M. (2000). Random Iterative Models. Springer, Berlin.
• [24] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
• [25] Ferrari, P. A., Kesten, H., Martinez, S. and Picco, P. (1995). Existence of quasi-stationary distributions. A renewal dynamical approach. Ann. Probab. 23 501–521.
• [26] Ferrari, P. A. and Marić, N. (2007). Quasi stationary distributions and Fleming–Viot processes in countable spaces. Electron. J. Probab. 12 684–702.
• [27] Gobet, E. (2000). Weak approximation of killed diffusion using Euler schemes. Stochastic Process. Appl. 87 167–197.
• [28] Gong, G. L., Qian, M. P. and Zhao, Z. X. (1988). Killed diffusions and their conditioning. Probab. Theory Related Fields 80 151–167.
• [29] Grigorescu, I. and Kang, M. (2012). Immortal particle for a catalytic branching process. Probab. Theory Related Fields 153 333–361.
• [30] Lamberton, D. and Pagès, G. (2008). A penalized bandit algorithm. Electron. J. Probab. 13 341–373.
• [31] Lamberton, D., Pagès, G. and Tarrès, P. (2004). When can the two-armed bandit algorithm be trusted? Ann. Appl. Probab. 14 1424–1454.
• [32] Lemaire, V. (2007). An adaptive scheme for the approximation of dissipative systems. Stochastic Process. Appl. 117 1491–1518.
• [33] Lemaire, V. and Menozzi, S. (2010). On some non asymptotic bounds for the Euler scheme. Electron. J. Probab. 15 1645–1681.
• [34] Méléard, S. and Villemonais, D. (2012). Quasi-stationary distributions and population processes. Probab. Surv. 9 340–410.
• [35] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
• [36] Oçafrain, W. and Villemonais, D. (2016). Non-failable approximation method for conditioned distributions. Preprint. Available at arXiv:1606.08978.
• [37] Panloup, F. (2008). Recursive computation of the invariant measure of a stochastic differential equation driven by a Lévy process. Ann. Appl. Probab. 18 379–426.
• [38] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surv. 4 1–79.
• [39] Pinsky, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Studies in Advanced Mathematics 45. Cambridge Univ. Press, Cambridge.
• [40] Pollock, M., Fearnhead, P., Johansen, A. M. and Roberts, G. O. (2016). The scalable Langevin exact algorithm: Bayesian inference for big data. Preprint. Available at arXiv:1609.03436.
• [41] Rachev, S. T., Klebanov, L. B., Stoyanov, S. V. and Fabozzi, F. J. (2013). The Methods of Distances in the Theory of Probability and Statistics. Springer, New York.
• [42] Villemonais, D. (2011). Interacting particle systems and Yaglom limit approximation of diffusions with unbounded drift. Electron. J. Probab. 16 1663–1692.
• [43] Villemonais, D. (2014). General approximation method for the distribution of Markov processes conditioned not to be killed. ESAIM Probab. Stat. 18 441–467.