The Annals of Applied Probability

On the ergodicity properties of some adaptive MCMC algorithms

Christophe Andrieu and Éric Moulines

Full-text: Open access

Enhanced PDF (436 KB)

Abstract

In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the independent Metropolis–Hastings algorithm and the random walk Metropolis algorithm with symmetric increments. Finally, we propose an application of these results to the case where the proposal distribution of the Metropolis–Hastings update is a mixture of distributions from a curved exponential family.

Article information

Source
Ann. Appl. Probab., Volume 16, Number 3 (2006), 1462-1505.

Dates
First available in Project Euclid: 2 October 2006

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1159804988

Digital Object Identifier
doi:10.1214/105051606000000286

Mathematical Reviews number (MathSciNet)
MR2260070

Zentralblatt MATH identifier
1114.65001

Subjects
Primary: 65C05: Monte Carlo methods 65C40: Computational Markov chains 60J27: Continuous-time Markov processes on discrete state spaces 60J35: Transition functions, generators and resolvents [See also 47D03, 47D07]
Secondary: 93E35: Stochastic learning and adaptive control

Keywords
Adaptive Markov chain Monte Carlo self-tuning algorithm Metropolis–Hastings algorithm stochastic approximation state-dependent noise randomly varying truncation martingale Poisson method

Citation

Andrieu, Christophe; Moulines, Éric. On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16 (2006), no. 3, 1462--1505. doi:10.1214/105051606000000286. https://projecteuclid.org/euclid.aoap/1159804988


Export citation

References

  • Andrieu, C. (2004). Discussion on the meeting on statistical approaches to inverse problems, December 2003. J. R. Statist. Soc. Ser. B Stat. Methodol. 66 633--634.
  • Andrieu, C. and Moulines, E. (2003). On the ergodicity property of some adaptive MCMC algorithms. Technical report, Univ. Bristol.
  • Andrieu, C., Moulines, E. and Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 283--312.
    Mathematical Reviews (MathSciNet): MR2177157
    Digital Object Identifier: doi:10.1137/S0363012902417267
    Zentralblatt MATH: 1083.62073
  • Andrieu, C. and Robert, C. (2001). Controlled MCMC for optimal sampling. Cahiers du Cérémade 0125.
  • Arcidiacono, P. and Jones, J. B. (2003). Finite mixture distributions, sequential likelihood and the EM algorithm. Econometrica 71 933--946.
    Mathematical Reviews (MathSciNet): MR1983232
    Digital Object Identifier: doi:10.1111/1468-0262.00431
    JSTOR: links.jstor.org
    Zentralblatt MATH: 1154.62360
  • Atchadé, Y. and Rosenthal, J. (2003). On adaptive Markov chain Monte Carlo algorithms. Technical report.
  • Bartusek, J. (2000). Stochastic approximation and optimization of Markov chains. Ph.D. thesis.
  • Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700--738.
    Mathematical Reviews (MathSciNet): MR2114987
    Digital Object Identifier: doi:10.1214/105051604000000710
    Project Euclid: euclid.aoap/1107271665
  • Benveniste, A., Métivier, M. and Priouret, P. (1990). Adaptive Algorithms and Stochastic Approximations. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1082341
    Zentralblatt MATH: 0752.93073
  • Birnbaum, Z. W. and Marshall, A. W. (1961). Some multivariate Chebyshev inequalities with extensions to continuous parameter processes. Ann. Math. Statist. 32 687--703.
    Mathematical Reviews (MathSciNet): MR0148106
    Digital Object Identifier: doi:10.1214/aoms/1177704964
    Project Euclid: euclid.aoms/1177704964
  • Chen, H., Guo, L. and Gao, A. (1988). Convergence and robustness of the Robbins--Monro algorithm truncated at randomly varying bounds. Stochastic Process. Appl. 27 217--231.
    Mathematical Reviews (MathSciNet): MR0931029
    Digital Object Identifier: doi:10.1016/0304-4149(87)90039-1
    Zentralblatt MATH: 0632.62082
  • Chen, H. and Zhu, Y.-M. (1986). Stochastic approximation procedures with randomly varying truncations. Sci. Sinica Ser. A 29 914--926.
    Mathematical Reviews (MathSciNet): MR0869196
  • Duflo, M. (1997). Random Iterative Systems. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1485774
    Zentralblatt MATH: 0868.62069
  • Dvoretzky, A. (1972). Asymptotic normality for sums of dependent random variables. Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability II 513--535. Univ. California Press, Berkeley.
    Mathematical Reviews (MathSciNet): MR0415728
    Zentralblatt MATH: 0256.60009
  • Gasemyr, J. (2003). On an adaptive version of the Metropolis--Hastings with independent proposal distribution. Scand. J. Statist. 30 159--173.
    Mathematical Reviews (MathSciNet): MR1965100
    Digital Object Identifier: doi:10.1111/1467-9469.00324
  • Gelman, A., Roberts, G. and Gilks, W. (1995). Efficient Metropolis jumping rules. In Bayesian Statistics 5 (J. O. Berger, J. M. Bernardo, A. P. Dawid and A. F. M. Smith, eds.) 599--608. Oxford University Press, New York.
    Mathematical Reviews (MathSciNet): MR1425429
  • Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 473--483.
  • Glynn, P. W. and Meyn, S. P. (1996). A Liapounov bound for solutions of the Poisson equation. Ann. Probab. 24 916--931.
    Mathematical Reviews (MathSciNet): MR1404536
    Digital Object Identifier: doi:10.1214/aop/1039639370
    Project Euclid: euclid.aop/1039639370
    Zentralblatt MATH: 0863.60063
  • Haario, H., Saksman, E. and Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli 7 223--242.
    Mathematical Reviews (MathSciNet): MR1828504
    Digital Object Identifier: doi:10.2307/3318737
    Project Euclid: euclid.bj/1080222083
    Zentralblatt MATH: 0989.65004
  • Hall, P. and Heyde, C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
    Mathematical Reviews (MathSciNet): MR0624435
    Zentralblatt MATH: 0462.60045
  • Holden, L. (1998). Adaptive chains. Technical report, Norwegian Computing Center.
  • Jarner, S. and Hansen, E. (2000). Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl. 85 341--361.
    Mathematical Reviews (MathSciNet): MR1731030
    Digital Object Identifier: doi:10.1016/S0304-4149(99)00082-4
    Zentralblatt MATH: 0997.60070
  • Kushner, H. and Yin, G. (1997). Stochastic Approximation Algorithms and Applications. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1453116
    Zentralblatt MATH: 0914.60006
  • McLeish, D. (1975). A maximal inequality and dependent strong laws. Ann. Probab. 3 829--839.
    Mathematical Reviews (MathSciNet): MR0400382
    Digital Object Identifier: doi:10.1214/aop/1176996269
    Zentralblatt MATH: 0353.60035
  • Meng, X. L. and Van Dyk, D. (1997). The EM algorithm---an old folk song sung to a fast new tune. J. R. Statist. Soc. Ser B Stat. Methodol. 59 511--567.
    Mathematical Reviews (MathSciNet): MR1452025
    Digital Object Identifier: doi:10.1111/1467-9868.00082
    JSTOR: links.jstor.org
  • Mengersen, K. and Tweedie, R. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist. 24 101--121.
    Mathematical Reviews (MathSciNet): MR1389882
    Digital Object Identifier: doi:10.1214/aos/1033066201
    Project Euclid: euclid.aos/1033066201
    Zentralblatt MATH: 0854.60065
  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A. and Teller, M. (1953). Equations of state calculations by fast computing machines. J. Chemical Physics 21 1087--1091.
  • Meyn, S. and Tweedie, R. (1993). Markov Chains and Stochastic Stability. Springer, London.
    Mathematical Reviews (MathSciNet): MR1287609
    Zentralblatt MATH: 0925.60001
  • Meyn, S. and Tweedie, R. (1994). Computable bounds for convergence rates of Markov chains. Ann. Appl. Probab. 4 981--1011.
    Mathematical Reviews (MathSciNet): MR1304770
    Digital Object Identifier: doi:10.1214/aoap/1177004900
    Project Euclid: euclid.aoap/1177004900
    Zentralblatt MATH: 0812.60059
  • Nummelin, E. (1991). On the Poisson equation in the potential theory of a single kernel. Math. Scand. 68 59--82.
    Mathematical Reviews (MathSciNet): MR1124820
  • Roberts, G. and Tweedie, R. (1996). Geometric convergence and central limit theorem for multidimensional Hastings and Metropolis algorithms. Biometrika 83 95--110.
    Mathematical Reviews (MathSciNet): MR1399158
    Zentralblatt MATH: 0888.60064
    Digital Object Identifier: doi:10.1093/biomet/83.1.95
    JSTOR: links.jstor.org
  • Titterington, D., Smith, A. F. M. and Makov, U. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester.
    Mathematical Reviews (MathSciNet): MR0838090
    Zentralblatt MATH: 0646.62013
  • Wu, C. (1983). On the convergence properties of the EM algorithm. Ann. Statist. 11 95--103.
    Mathematical Reviews (MathSciNet): MR0684867
    Digital Object Identifier: doi:10.1214/aos/1176346060
    Project Euclid: euclid.aos/1176346060
    Zentralblatt MATH: 0517.62035

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more