The Annals of Statistics

Convergence of adaptive and interacting Markov chain Monte Carlo algorithms

G. Fort, E. Moulines, and P. Priouret

Full-text: Open access

Abstract

Adaptive and interacting Markov chain Monte Carlo algorithms (MCMC) have been recently introduced in the literature. These novel simulation algorithms are designed to increase the simulation efficiency to sample complex distributions. Motivated by some recently introduced algorithms (such as the adaptive Metropolis algorithm and the interacting tempering algorithm), we develop a general methodological and theoretical framework to establish both the convergence of the marginal distribution and a strong law of large numbers. This framework weakens the conditions introduced in the pioneering paper by Roberts and Rosenthal [J. Appl. Probab. 44 (2007) 458–475]. It also covers the case when the target distribution π is sampled by using Markov transition kernels with a stationary distribution that differs from π.

Article information

Source
Ann. Statist., Volume 39, Number 6 (2011), 3262-3289.

Dates
First available in Project Euclid: 5 March 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1330958679

Digital Object Identifier
doi:10.1214/11-AOS938

Mathematical Reviews number (MathSciNet)
MR3012408

Zentralblatt MATH identifier
1246.65003

Subjects
Primary: 65C05: Monte Carlo methods 60F05: Central limit and other weak theorems 62L10: Sequential analysis 65C05: Monte Carlo methods
Secondary: 65C40: Computational Markov chains 60J05: Discrete-time Markov processes on general state spaces 93E35: Stochastic learning and adaptive control

Keywords
Markov chains Markov chain Monte Carlo adaptive Monte Carlo ergodic theorems law of large numbers adaptive Metropolis equi-energy sampler parallel tempering interacting tempering

Citation

Fort, G.; Moulines, E.; Priouret, P. Convergence of adaptive and interacting Markov chain Monte Carlo algorithms. Ann. Statist. 39 (2011), no. 6, 3262--3289. doi:10.1214/11-AOS938. https://projecteuclid.org/euclid.aos/1330958679


Export citation

References

  • Andrieu, C. and Moulines, É. (2006). On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16 1462–1505.
  • Andrieu, C. and Thoms, J. (2008). A tutorial on adaptive MCMC. Stat. Comput. 18 343–373.
  • Andrieu, C., Jasra, A., Doucet, A. and Del Moral, P. (2007). Non-linear Markov chain Monte Carlo. In Conference Oxford sur les Méthodes de Monte Carlo Séquentielles. ESAIM Proceedings 19 79–84. EDP Sci., Les Ulis.
  • Andrieu, C., Jasra, A., Doucet, A. and Del Moral, P. (2008). A note on convergence of the equi-energy sampler. Stoch. Anal. Appl. 26 298–312.
  • Andrieu, C., Jasra, A., Doucet, A. and Del Moral, P. (2011). On non-linear Markov chain Monte Carlo. Bernoulli 17 987–1014.
  • Atchadé, Y. F. (2010). A cautionary tale on the efficiency of some adaptive Monte Carlo schemes. Ann. Appl. Probab. 20 841–868.
  • Atchadé, Y. and Fort, G. (2010). Limit theorems for some adaptive MCMC algorithms with subgeometric kernels. Bernoulli 16 116–154.
  • Atchadé, Y. F. and Liu, J. S. (2006). Discussion of “Equi-energy sampler,” by Kou, Zhou and Wong. Ann. Statist. 34 1620–1628.
  • Atchade, Y., Roberts, G. O. and Rosenthal, J. S. (2011). Towards optimal scaling of Metropolis-coupled Markov chain Monte Carlo. Stat. Comput. 21 555–568.
  • Atchadé, Y. F. and Rosenthal, J. S. (2005). On adaptive Markov chain Monte Carlo algorithms. Bernoulli 11 815–828.
  • Atchadé, Y., Fort, G., Moulines, E. and Priouret, P. (2011). Adaptive Markov chain Monte Carlo: Theory and methods. In Bayesian Time Series Models (D. Barber, A. T. Cemgil and S. Chiappia, eds.) 33–53. Cambridge Univ. Press, Cambridge.
  • Bai, Y., Roberts, G. O. and Rosenthal, J. S. (2011). On the containement condition for adaptive Markov chain Monte Carlo algorithms. Adv. Appl. Stat. 21 1–54.
  • Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700–738.
  • Bercu, B., Del Moral, P. and Doucet, A. (2009). A functional central limit theorem for a class of interacting Markov chain Monte Carlo methods. Electron. J. Probab. 14 2130–2155.
  • Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York.
  • Brockwell, A., Del Moral, P. and Doucet, A. (2010). Sequentially interacting Markov chain Monte Carlo methods. Ann. Statist. 38 3387–3411.
  • Douc, R., Moulines, E. and Rosenthal, J. S. (2004). Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14 1643–1665.
  • Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics 74. Cambridge Univ. Press, Cambridge.
  • Fort, G. and Moulines, E. (2003). Polynomial ergodicity of Markov transition kernels. Stochastic Process. Appl. 103 57–99.
  • Fort, G., Moulines, E. and Priouret, P. (2011). Supplement to “Convergence of adaptive and interacting Markov chain Monte Carlo algorithms.” DOI:10.1214/11-AOS938SUPP.
  • Gelman, A., Roberts, G. O. and Gilks, W. R. (1996). Efficient Metropolis jumping rules. In Bayesian Statistics, 5 (Alicante, 1994) 599–607. Oxford Univ. Press, New York.
  • Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proc. 23rd Symposium on the Interface, Interface Foundation, Fairfax Station, VA 156–163. Amer. Statist. Assoc., Alexandria, VA.
  • Haario, H., Saksman, E. and Tamminen, J. (1999). Adaptive proposal distribution for random walk Metropolis algorithm. Comput. Statist. 14 375–395.
  • Haario, H., Saksman, E. and Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli 7 223–242.
  • Haario, H., Laine, M., Lehtinen, M., Saksman, E. and Tamminen, J. (2004). Markov chain Monte Carlo methods for high dimensional inversion in remote sensing. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 591–607.
  • Haario, H., Laine, M., Mira, A. and Saksman, E. (2006). DRAM: Efficient adaptive MCMC. Stat. Comput. 16 339–354.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Jarner, S. F. and Hansen, E. (2000). Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl. 85 341–361.
  • Kato, T. (1980). Perturbation Theory for Linear Operators. Springer, Berlin.
  • Kou, S. C., Zhou, Q. and Wong, W. H. (2006). Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Statist. 34 1581–1615.
  • Laine, M. and Tamminen, J. (2008). Aerosol model selection and uncertainty modelling by adaptive MCMC technique. Atmospheric and Chemistry Physics 8 7697–7707.
  • Mengersen, K. L. and Tweedie, R. L. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist. 24 101–121.
  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equations of state calculations by fast computing machines. J. Chem. Phys. 21 1087–1092.
  • Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability. Cambridge Univ. Press, London.
  • Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York.
  • Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.
  • Roberts, G. O. and Rosenthal, J. S. (2007). Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. J. Appl. Probab. 44 458–475.
  • Roberts, G. O. and Tweedie, R. L. (1996). Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika 83 95–110.
  • Rosenthal, J. S. (2009). Optimal proposal distributions and adaptive MCMC. In Handbook of MCMC. Chapman & Hall/CRC Press, Boca Raton, FL.
  • Saksman, E. and Vihola, M. (2010). On the ergodicity of the adaptive Metropolis algorithm on unbounded domains. Ann. Appl. Probab. 20 2178–2203.

Supplemental materials

  • Supplementary material: Supplement to paper “Convergence of adaptive and interacting Markov chain Monte Carlo algorithms”. This supplement provides a detailed proof of Lemma 4.2 and Propositions 3.1, 4.3 and 5.2. It also contains a discussion on the setwise convergence of transition kernels.