The Annals of Applied Probability

Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Michel Benaim, Bertrand Cloez, and Fabien Panloup

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a nonirreducible setting.

Article information

Source
Ann. Appl. Probab., Volume 28, Number 4 (2018), 2370-2416.

Dates
Received: December 2016
Revised: September 2017
First available in Project Euclid: 9 August 2018

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1533780276

Digital Object Identifier
doi:10.1214/17-AAP1360

Mathematical Reviews number (MathSciNet)
MR3843832

Zentralblatt MATH identifier
06974754

Subjects
Primary: 65C20: Models, numerical methods [See also 68U20] 60B12: Limit theorems for vector-valued random variables (infinite- dimensional case) 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 34F05: Equations and systems with randomness [See also 34K50, 60H10, 93E03] 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40] 60J60: Diffusion processes [See also 58J65]

Keywords
Quasi-stationary distributions stochastic approximation reinforced random walks random perturbations of dynamical systems extinction rate Euler scheme

Citation

Benaim, Michel; Cloez, Bertrand; Panloup, Fabien. Stochastic approximation of quasi-stationary distributions on compact spaces and applications. Ann. Appl. Probab. 28 (2018), no. 4, 2370--2416. doi:10.1214/17-AAP1360. https://projecteuclid.org/euclid.aoap/1533780276


Export citation

References

  • [1] Aldous, D., Flannery, B. and Palacios, J. (1988). Two applications of urn processes: The fringe analysis of search trees and the simulation of quasi-stationary distributions of Markov chains. Probab. Engrg. Inform. Sci. 2 293–307.
    Zentralblatt MATH: 1134.68592
    Digital Object Identifier: doi:10.1017/S026996480000084X
  • [2] Bansaye, V., Méléard, S. and Richard, M. (2016). Speed of coming down from infinity for birth-and-death processes. Adv. in Appl. Probab. 48 1183–1210.
    Zentralblatt MATH: 1358.60087
    Digital Object Identifier: doi:10.1017/apr.2016.70
    Project Euclid: euclid.aap/1482548434
  • [3] Bass, R. F. (1998). Diffusions and Elliptic Operators. Springer, New York.
    Zentralblatt MATH: 0914.60009
  • [4] Ben-Ari, I. and Pinsky, R. G. (2007). Spectral analysis of a family of second-order elliptic operators with nonlocal boundary condition indexed by a probability measure. J. Funct. Anal. 251 122–140.
    Zentralblatt MATH: 1137.35045
    Digital Object Identifier: doi:10.1016/j.jfa.2007.05.019
  • [5] Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math. 1709 1–68. Springer, Berlin.
  • [6] Benaïm, M. and Cloez, B. (2015). A stochastic approximation approach to quasi-stationary distributions on finite spaces. Electron. Commun. Probab. 20 no. 37.
    Zentralblatt MATH: 1321.65009
    Digital Object Identifier: doi:10.1214/ECP.v20-3956
  • [7] Benaïm, M. and Hirsch, M. W. (1996). Asymptotic pseudotrajectories and chain recurrent flows, with applications. J. Dynam. Differential Equations 8 141–176.
    Mathematical Reviews (MathSciNet): MR1388167
    Zentralblatt MATH: 0878.58053
    Digital Object Identifier: doi:10.1007/BF02218617
  • [8] Benaïm, M., Ledoux, M. and Raimond, O. (2002). Self-interacting diffusions. Probab. Theory Related Fields 122 1–41.
    Zentralblatt MATH: 1042.60060
    Digital Object Identifier: doi:10.1007/s004400100161
  • [9] Berglund, N. and Landon, D. (2012). Mixed-mode oscillations and interspike interval statistics in the stochastic FitzHugh–Nagumo model. Nonlinearity 25 2303–2335.
    Zentralblatt MATH: 1248.60059
    Digital Object Identifier: doi:10.1088/0951-7715/25/8/2303
  • [10] Bieniek, M., Burdzy, K. and Finch, S. (2012). Non-extinction of a Fleming–Viot particle model. Probab. Theory Related Fields 153 293–332.
    Zentralblatt MATH: 1253.60089
    Digital Object Identifier: doi:10.1007/s00440-011-0372-5
  • [11] Bieniek, M., Burdzy, K. and Pal, S. (2012). Extinction of Fleming–Viot-type particle systems with strong drift. Electron. J. Probab. 17 no. 11.
    Zentralblatt MATH: 1258.60031
    Digital Object Identifier: doi:10.1214/EJP.v17-1770
  • [12] Blanchet, J., Glynn, P. and Zheng, S. (2016). Analysis of a stochastic approximation algorithm for computing quasi-stationary distributions. Adv. in Appl. Probab. 48 792–811.
    Zentralblatt MATH: 1352.60106
    Digital Object Identifier: doi:10.1017/apr.2016.28
    Project Euclid: euclid.aap/1474296315
  • [13] Bouleau, N. and Lépingle, D. (1994). Numerical Methods for Stochastic Processes. Wiley, New York.
  • [14] Burdzy, K., Hołyst, R. and March, P. (2000). A Fleming–Viot particle representation of the Dirichlet Laplacian. Comm. Math. Phys. 214 679–703.
    Zentralblatt MATH: 0982.60078
    Digital Object Identifier: doi:10.1007/s002200000294
  • [15] Cattiaux, P., Collet, P., Lambert, A., Martínez, S., Méléard, S. and San Martín, J. (2009). Quasi-stationary distributions and diffusion models in population dynamics. Ann. Probab. 37 1926–1969.
    Zentralblatt MATH: 1176.92041
    Digital Object Identifier: doi:10.1214/09-AOP451
    Project Euclid: euclid.aop/1253539860
  • [16] Champagnat, N. and Villemonais, D. (2016). Exponential convergence to quasi-stationary distribution and $Q$-process. Probab. Theory Related Fields 164 243–283.
    Zentralblatt MATH: 1334.60015
    Digital Object Identifier: doi:10.1007/s00440-014-0611-7
  • [17] Cloez, B. and Thai, M.-N. (2016). Fleming–Viot processes: Two explicit examples. ALEA Lat. Am. J. Probab. Math. Stat. 13 337–356.
    Zentralblatt MATH: 1337.60241
  • [18] Cloez, B. and Thai, M.-N. (2016). Quantitative results for the Fleming–Viot particle system and quasi-stationary distributions in discrete space. Stochastic Process. Appl. 126 680–702.
    Zentralblatt MATH: 1333.60200
    Digital Object Identifier: doi:10.1016/j.spa.2015.09.016
  • [19] Collet, P., Martínez, S., Méléard, S. and San Martín, J. (2011). Quasi-stationary distributions for structured birth and death processes with mutations. Probab. Theory Related Fields 151 191–231.
  • [20] Del Moral, P. and Guionnet, A. (1999). On the stability of measure valued processes with applications to filtering. C. R. Acad. Sci. Paris, Sér. I Math. 329 429–434.
    Zentralblatt MATH: 0935.92001
    Digital Object Identifier: doi:10.1016/S0764-4442(00)88619-X
  • [21] Del Moral, P. and Miclo, L. (2000). A Moran particle system approximation of Feynman–Kac formulae. Stochastic Process. Appl. 86 193–216.
    Zentralblatt MATH: 1030.65004
    Digital Object Identifier: doi:10.1016/S0304-4149(99)00094-0
  • [22] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics 74. Cambridge Univ. Press, Cambridge. Revised reprint of the 1989 original.
    Zentralblatt MATH: 1023.60001
  • [23] Duflo, M. (2000). Random Iterative Models. Springer, Berlin.
  • [24] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR838085
    Zentralblatt MATH: 0592.60049
  • [25] Ferrari, P. A., Kesten, H., Martinez, S. and Picco, P. (1995). Existence of quasi-stationary distributions. A renewal dynamical approach. Ann. Probab. 23 501–521.
    Zentralblatt MATH: 0827.60061
    Digital Object Identifier: doi:10.1214/aop/1176988277
    Project Euclid: euclid.aop/1176988277
  • [26] Ferrari, P. A. and Marić, N. (2007). Quasi stationary distributions and Fleming–Viot processes in countable spaces. Electron. J. Probab. 12 684–702.
    Zentralblatt MATH: 1127.60088
    Digital Object Identifier: doi:10.1214/EJP.v12-415
  • [27] Gobet, E. (2000). Weak approximation of killed diffusion using Euler schemes. Stochastic Process. Appl. 87 167–197.
    Zentralblatt MATH: 1045.60082
    Digital Object Identifier: doi:10.1016/S0304-4149(99)00109-X
  • [28] Gong, G. L., Qian, M. P. and Zhao, Z. X. (1988). Killed diffusions and their conditioning. Probab. Theory Related Fields 80 151–167.
    Zentralblatt MATH: 0631.60073
    Digital Object Identifier: doi:10.1007/BF00348757
  • [29] Grigorescu, I. and Kang, M. (2012). Immortal particle for a catalytic branching process. Probab. Theory Related Fields 153 333–361.
    Zentralblatt MATH: 1251.60064
    Digital Object Identifier: doi:10.1007/s00440-011-0347-6
  • [30] Lamberton, D. and Pagès, G. (2008). A penalized bandit algorithm. Electron. J. Probab. 13 341–373.
    Zentralblatt MATH: 1206.62139
    Digital Object Identifier: doi:10.1214/EJP.v13-489
  • [31] Lamberton, D., Pagès, G. and Tarrès, P. (2004). When can the two-armed bandit algorithm be trusted? Ann. Appl. Probab. 14 1424–1454.
    Zentralblatt MATH: 1048.62079
    Digital Object Identifier: doi:10.1214/105051604000000350
    Project Euclid: euclid.aoap/1089736291
  • [32] Lemaire, V. (2007). An adaptive scheme for the approximation of dissipative systems. Stochastic Process. Appl. 117 1491–1518.
    Zentralblatt MATH: 1126.65005
    Digital Object Identifier: doi:10.1016/j.spa.2007.02.004
  • [33] Lemaire, V. and Menozzi, S. (2010). On some non asymptotic bounds for the Euler scheme. Electron. J. Probab. 15 1645–1681.
    Zentralblatt MATH: 1225.60117
    Digital Object Identifier: doi:10.1214/EJP.v15-814
  • [34] Méléard, S. and Villemonais, D. (2012). Quasi-stationary distributions and population processes. Probab. Surv. 9 340–410.
    Zentralblatt MATH: 1261.92056
    Digital Object Identifier: doi:10.1214/11-PS191
  • [35] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
    Zentralblatt MATH: 0781.60053
    Digital Object Identifier: doi:10.2307/1427522
  • [36] Oçafrain, W. and Villemonais, D. (2016). Non-failable approximation method for conditioned distributions. Preprint. Available at arXiv:1606.08978.
    arXiv: 1606.08978
  • [37] Panloup, F. (2008). Recursive computation of the invariant measure of a stochastic differential equation driven by a Lévy process. Ann. Appl. Probab. 18 379–426.
    Zentralblatt MATH: 1136.60049
    Digital Object Identifier: doi:10.1214/105051607000000285
    Project Euclid: euclid.aoap/1206018192
  • [38] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surv. 4 1–79.
    Zentralblatt MATH: 1189.60138
    Digital Object Identifier: doi:10.1214/07-PS094
  • [39] Pinsky, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Studies in Advanced Mathematics 45. Cambridge Univ. Press, Cambridge.
    Zentralblatt MATH: 0858.31001
  • [40] Pollock, M., Fearnhead, P., Johansen, A. M. and Roberts, G. O. (2016). The scalable Langevin exact algorithm: Bayesian inference for big data. Preprint. Available at arXiv:1609.03436.
    arXiv: 1609.03436
  • [41] Rachev, S. T., Klebanov, L. B., Stoyanov, S. V. and Fabozzi, F. J. (2013). The Methods of Distances in the Theory of Probability and Statistics. Springer, New York.
    Zentralblatt MATH: 1280.60005
  • [42] Villemonais, D. (2011). Interacting particle systems and Yaglom limit approximation of diffusions with unbounded drift. Electron. J. Probab. 16 1663–1692.
    Zentralblatt MATH: 1244.82052
    Digital Object Identifier: doi:10.1214/EJP.v16-925
  • [43] Villemonais, D. (2014). General approximation method for the distribution of Markov processes conditioned not to be killed. ESAIM Probab. Stat. 18 441–467.
    Zentralblatt MATH: 1310.82032
    Digital Object Identifier: doi:10.1051/ps/2013045

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more