The Annals of Applied Probability

Renewal theory and computable convergence rates for geometrically ergodic Markov chains

Peter H. Baxendale

Full-text: Open access

Abstract

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and Tweedie, and from estimates using coupling, although we start from essentially the same assumptions of a drift condition toward a “small set.” The estimates show a noticeable improvement on existing results if the Markov chain is reversible with respect to its stationary distribution, and especially so if the chain is also positive. The method of proof uses the first-entrance–last-exit decomposition, together with new quantitative versions of a result of Kendall from discrete renewal theory.

Article information

Source
Ann. Appl. Probab., Volume 15, Number 1B (2005), 700-738.

Dates
First available in Project Euclid: 1 February 2005

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1107271665

Digital Object Identifier
doi:10.1214/105051604000000710

Mathematical Reviews number (MathSciNet)
MR2114987

Zentralblatt MATH identifier
1070.60061

Subjects
Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 60K05: Renewal theory 65C05: Monte Carlo methods

Keywords
Geometric ergodicity renewal theory reversible Markov chain Markov chain Monte Carlo Metropolis–Hastings algorithm spectral gap

Citation

Baxendale, Peter H. Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 (2005), no. 1B, 700--738. doi:10.1214/105051604000000710. https://projecteuclid.org/euclid.aoap/1107271665


Export citation

References

  • Baxendale, P. (1994). Uniform estimates for geometric ergodicity of recurrent Markov chains. Technical report, Univ. Southern California.
  • Douc, R., Moulines, E. and Rosenthal, J. S. (2004). Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14 1643–1665.
  • Feller, W. (1968). An Introduction to Probability Theory and Its Applications 1, 3rd ed. Wiley, New York.
  • Fort, G. (2003). Computable bounds for $V$-geometric ergodicity of Markov transition kernels. Preprint.
  • Kendall, D. G. (1959). Unitary dilations of Markov transition operators and the corresponding integral representation of transition probability matrices. In Probability and Statistics (U. Grenander, ed.) 138–161. Almqvist and Wiksell, Stockholm.
  • Liu, J., Wong, W. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. J. R. Stat. Soc. Ser. B Stat. Methodol. 57 157–169.
  • Lund, R. B. and Tweedie, R. L. (1996). Geometric convergence rates for stochastically ordered Markov chains. Math. Oper. Res. 21 182–194.
  • Meyn, S. and Tweedie, R. (1993). Markov Chains and Stochastic Stability. Springer, Berlin.
  • Meyn, S. and Tweedie, R. (1994). Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4 981–1011.
  • Nummelin, E. (1984). General Irreducible Markov Chains and Non-Negative Operators. Cambridge Univ. Press.
  • Nummelin, E. and Tuominen, P. (1982). Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory. Stochastic Process. Appl. 12 187–202.
  • Nummelin, E. and Tweedie, R. L. (1978). Geometric ergodicity and $R$-positivity for general Markov chains. Ann. Probab. 6 404–420.
  • Roberts, G. O. and Rosenthal, J. S. (1997). Geometric ergodicity and hybrid Markov chains. Electron. Comm. Probab. 2 13–25.
  • Roberts, G. O. and Rosenthal, J. S. (1999). Shift-coupling and convergence rates of ergodic averages. Comm. Statist. Stochastic Models 13 147–165.
  • Roberts, G. O. and Tweedie, R. L. (1999). Bounds on regeneration times and convergence rates for Markov chains. Stochastic. Process. Appl. 80 211–229.
  • Roberts, G. O. and Tweedie, R. L. (2000). Rates of convergence of stochastically monotone and continuous time Markov models. J. Appl. Probab. 37 359–373.
  • Roberts, G. O. and Tweedie, R. L. (2001). Geometric $L^2$ and $L^1$ convergence are equivalent for reversible Markov chains. Probability, statistics and seismology. J. Appl. Probab. 38A 37–41.
  • Rosenthal, J. S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558–566.
  • Rosenthal, J. S. (2002). Quantitative convergence rates of Markov chains: A simple account. Electron. Comm. Probab. 7 123–128.
  • Rosenthal, J. S. (2003). Asymptotic variance and convergence rates of nearly periodic MCMC algorithms. J. Amer. Statist. Assoc. 98 169–177.
  • Scott, D. J. and Tweedie, R. L. (1996). Explicit rates of convergence of stochastically monotone Markov chains. In Proceedings of the Athens Conference on Applied Probability and Time Series Analysis: Papers in Honour of J. M. Gani and E. J. Hannan (C. C. Heyde, Yu. V. Prohorov, R. Pyke and S. T. Rachev, eds.) 176–191. Springer, New York.
  • Vere-Jones, D. (1963). On the spectra of some linear operators associated with queuing systems. Z. Wahrsch. Verw. Gebiete 2 12–21.
  • Yosida, K. (1980). Functional Analysis, 6th ed. Springer, Berlin.