• Bernoulli
  • Volume 11, Number 5 (2005), 815-828.

On adaptive Markov chain Monte Carlo algorithms

Yves F. Atchadé and Jeffrey S. Rosenthal

Full-text: Open access


We look at adaptive Markov chain Monte Carlo algorithms that generate stochastic processes based on sequences of transition kernels, where each transition kernel is allowed to depend on the history of the process. We show under certain conditions that the stochastic process generated is ergodic, with appropriate stationary distribution. We use this result to analyse an adaptive version of the random walk Metropolis algorithm where the scale parameter σ is sequentially adapted using a Robbins-Monro type algorithm in order to find the optimal scale parameter σopt. We close with a simulation example.

Article information

Bernoulli Volume 11, Number 5 (2005), 815-828.

First available in Project Euclid: 23 October 2005

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

adaptive Markov chain Monte Carlo Metropolis algorithm mixingales parameter tuning Robbins-Monro algorithm


Atchadé, Yves F.; Rosenthal, Jeffrey S. On adaptive Markov chain Monte Carlo algorithms. Bernoulli 11 (2005), no. 5, 815--828. doi:10.3150/bj/1130077595.

Export citation


  • [1] Andrieu, C. and Moulines, E. (2003) Ergodicity of some adaptive Markov Chain Monte Carlo algorithms. Technical report.
  • [2] Andrieu, C., Moulines, E. and Priouret, P. (2002) Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim., To appear.
  • [3] Atchadé, Y.F. (2005) An adaptive version for the Metropolis adjusted Langevin algorithm with a truncated drift. MCMC Preprint.
  • [4] Chen, H. and Zhu, Y.-M. (1986) Stochastic approximation procedures with randomly varying truncations. Sci. Sinica Ser. A, 1, 914-926.
  • [5] Davidson, J. and de Jong, R. (1997) Strong laws of large numbers for dependent heteregeneous processes: a synthesis of recent and new results. Econometric Rev., 16, 251-279.
  • [6] Duflo, M. (1997) Random Iterative Models. Berlin: Springer-Verlag. Gilks, W.R., Richardson, S. and Spiegelhalter, D.J. (eds) (1996) Markov Chain Monte Carlo in Practice. London: Chapman & Hall.
  • [7] Gilks, W.R., Roberts, G.O. and Sahu, S.K. (1998) Adaptive Markov chain Monte Carlo through regeneration. J. Amer. Statist. Assoc., 93, 1045-1054.
  • [8] Haario, H., Saksman, E. and Tamminen, J. (2001) An adaptive Metropolis algorithm. Bernoulli, 7, 223-242.
  • [9] Hall, P. and Heyde, C.C. (1980) Martingale Limit Theory and Its Application. New York: Academic Press.
  • [10] Jarner, S.F. and Hansen, E. (2000) Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl., 85, 341-361.
  • [11] Kushner, K. and Yin, Y. (2003) Stochastic Approximation and Recursive Algorithms and Applications. New York: Springer-Verlag.
  • [12] Liu, J.S. (2001) Monte Carlo Strategies in Scientific Computing. New York: Springer-Verlag.
  • [13] Meyn, S.P. and Tweedie, R.L. (1994) Computable bounds for convergence rates of Markov chains. Ann. Appl. Probab., 4, 981-1011.
  • [14] Robbins, H. and Monro, S. (1951) A stochastic approximation method. Ann. Math. Statist., 22, 400-407.
  • [15] Roberts, G.O. and Rosenthal, J.S. (2001) Optimal scaling of various Metropolis-Hastings algorithms. Statist. Sci., 16, 351-367.
  • [16] Roberts, G.O. and Gelman, A. and Gilks, W. (1997) Weak convergence and optimal scaling of random walk Metropolis algorithm. Ann. Applied Probab., 7, 110-120.
  • [17] Rosenthal, J.S. (2004) Adaptive MCMC Java applet.