The Annals of Applied Probability

On the ergodicity properties of some adaptive MCMC algorithms

Christophe Andrieu and Éric Moulines

Full-text: Open access

Abstract

In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the independent Metropolis–Hastings algorithm and the random walk Metropolis algorithm with symmetric increments. Finally, we propose an application of these results to the case where the proposal distribution of the Metropolis–Hastings update is a mixture of distributions from a curved exponential family.

Article information

Source
Ann. Appl. Probab., Volume 16, Number 3 (2006), 1462-1505.

Dates
First available in Project Euclid: 2 October 2006

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1159804988

Digital Object Identifier
doi:10.1214/105051606000000286

Mathematical Reviews number (MathSciNet)
MR2260070

Zentralblatt MATH identifier
1114.65001

Subjects
Primary: 65C05: Monte Carlo methods 65C40: Computational Markov chains 60J27: Continuous-time Markov processes on discrete state spaces 60J35: Transition functions, generators and resolvents [See also 47D03, 47D07]
Secondary: 93E35: Stochastic learning and adaptive control

Keywords
Adaptive Markov chain Monte Carlo self-tuning algorithm Metropolis–Hastings algorithm stochastic approximation state-dependent noise randomly varying truncation martingale Poisson method

Citation

Andrieu, Christophe; Moulines, Éric. On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16 (2006), no. 3, 1462--1505. doi:10.1214/105051606000000286. https://projecteuclid.org/euclid.aoap/1159804988


Export citation

References

  • Andrieu, C. (2004). Discussion on the meeting on statistical approaches to inverse problems, December 2003. J. R. Statist. Soc. Ser. B Stat. Methodol. 66 633--634.
  • Andrieu, C. and Moulines, E. (2003). On the ergodicity property of some adaptive MCMC algorithms. Technical report, Univ. Bristol.
  • Andrieu, C., Moulines, E. and Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 283--312.
  • Andrieu, C. and Robert, C. (2001). Controlled MCMC for optimal sampling. Cahiers du Cérémade 0125.
  • Arcidiacono, P. and Jones, J. B. (2003). Finite mixture distributions, sequential likelihood and the EM algorithm. Econometrica 71 933--946.
  • Atchadé, Y. and Rosenthal, J. (2003). On adaptive Markov chain Monte Carlo algorithms. Technical report.
  • Bartusek, J. (2000). Stochastic approximation and optimization of Markov chains. Ph.D. thesis.
  • Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700--738.
  • Benveniste, A., Métivier, M. and Priouret, P. (1990). Adaptive Algorithms and Stochastic Approximations. Springer, New York.
  • Birnbaum, Z. W. and Marshall, A. W. (1961). Some multivariate Chebyshev inequalities with extensions to continuous parameter processes. Ann. Math. Statist. 32 687--703.
  • Chen, H., Guo, L. and Gao, A. (1988). Convergence and robustness of the Robbins--Monro algorithm truncated at randomly varying bounds. Stochastic Process. Appl. 27 217--231.
  • Chen, H. and Zhu, Y.-M. (1986). Stochastic approximation procedures with randomly varying truncations. Sci. Sinica Ser. A 29 914--926.
  • Duflo, M. (1997). Random Iterative Systems. Springer, New York.
  • Dvoretzky, A. (1972). Asymptotic normality for sums of dependent random variables. Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability II 513--535. Univ. California Press, Berkeley.
  • Gasemyr, J. (2003). On an adaptive version of the Metropolis--Hastings with independent proposal distribution. Scand. J. Statist. 30 159--173.
  • Gelman, A., Roberts, G. and Gilks, W. (1995). Efficient Metropolis jumping rules. In Bayesian Statistics 5 (J. O. Berger, J. M. Bernardo, A. P. Dawid and A. F. M. Smith, eds.) 599--608. Oxford University Press, New York.
  • Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 473--483.
  • Glynn, P. W. and Meyn, S. P. (1996). A Liapounov bound for solutions of the Poisson equation. Ann. Probab. 24 916--931.
  • Haario, H., Saksman, E. and Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli 7 223--242.
  • Hall, P. and Heyde, C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Holden, L. (1998). Adaptive chains. Technical report, Norwegian Computing Center.
  • Jarner, S. and Hansen, E. (2000). Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl. 85 341--361.
  • Kushner, H. and Yin, G. (1997). Stochastic Approximation Algorithms and Applications. Springer, New York.
  • McLeish, D. (1975). A maximal inequality and dependent strong laws. Ann. Probab. 3 829--839.
  • Meng, X. L. and Van Dyk, D. (1997). The EM algorithm---an old folk song sung to a fast new tune. J. R. Statist. Soc. Ser B Stat. Methodol. 59 511--567.
  • Mengersen, K. and Tweedie, R. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist. 24 101--121.
  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A. and Teller, M. (1953). Equations of state calculations by fast computing machines. J. Chemical Physics 21 1087--1091.
  • Meyn, S. and Tweedie, R. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Meyn, S. and Tweedie, R. (1994). Computable bounds for convergence rates of Markov chains. Ann. Appl. Probab. 4 981--1011.
  • Nummelin, E. (1991). On the Poisson equation in the potential theory of a single kernel. Math. Scand. 68 59--82.
  • Roberts, G. and Tweedie, R. (1996). Geometric convergence and central limit theorem for multidimensional Hastings and Metropolis algorithms. Biometrika 83 95--110.
  • Titterington, D., Smith, A. F. M. and Makov, U. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester.
  • Wu, C. (1983). On the convergence properties of the EM algorithm. Ann. Statist. 11 95--103.