Institute of Mathematical Statistics Collections

On the Geometric Ergodicity of Two-Variable Gibbs Samplers

Aixin Tan, Galin L. Jones, and James P. Hobert

Full-text: Open access

Abstract

A Markov chain is geometrically ergodic if it converges to its invariant distribution at a geometric rate in total variation norm. We study geometric ergodicity of deterministic and random scan versions of the two-variable Gibbs sampler. We give a sufficient condition which simultaneously guarantees both versions are geometrically ergodic. We also develop a method for simultaneously establishing that both versions are subgeometrically ergodic. These general results allow us to characterize the convergence rate of two-variable Gibbs samplers in a particular family of discrete bivariate distributions.

Chapter information

Source
Galin Jones and Xiaotong Shen, eds., Advances in Modern Statistical Theory and Applications: A Festschrift in honor of Morris L. Eaton, (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2013) , 25-42

Dates
First available in Project Euclid: 23 September 2013

Permanent link to this document
https://projecteuclid.org/euclid.imsc/1379942046

Digital Object Identifier
doi:10.1214/12-IMSCOLL1002

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 62F15: Bayesian inference

Keywords
geometric ergodicity Gibbs sampler Markov chain Monte Carlo

Rights
Copyright © 2013, Institute of Mathematical Statistics

Citation

Tan, Aixin; Jones, Galin L.; Hobert, James P. On the Geometric Ergodicity of Two-Variable Gibbs Samplers. Advances in Modern Statistical Theory and Applications: A Festschrift in honor of Morris L. Eaton, 25--42, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2013. doi:10.1214/12-IMSCOLL1002. https://projecteuclid.org/euclid.imsc/1379942046.


Export citation

References

  • Chan, K. S. and Geyer, C. J. (1994). Comment on “Markov Chains for exploring posterior distributions”. The Annals of Statistics 22 1747–1758.
  • Diaconis, P., Khare, K. and Saloff-Coste, L. (2008). Gibbs sampling, exponential families and orthogonal polynomials. Statistical Science 23 151–178.
  • Flegal, J. M., Haran, M. and Jones, G. L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure? Statistical Science 23 250–260.
  • Flegal, J. M. and Jones, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo. The Annals of Statistics 38 1034-1070.
  • Flegal, J. M. and Jones, G. L. (2011). Implementing Markov chain Monte Carlo: Estimating with confidence. In Handbook of Markov Chain Monte Carlo ( S. Brooks, A. Gelman, G. L. Jones and X.-L. Meng, eds.) 175–197. CRC Press, Boca Raton, FL.
  • Geyer, C. J. (1992). Practical Markov chain Monte Carlo. Statistical Science 7 473-511.
  • Geyer, C. J. (2011). Introduction to Markov chain Monte Carlo. In Handbook of Markov Chain Monte Carlo ( S. P. Brooks, A. Gelman, G. L. Jones and X.-L. Meng, eds.) 3–48. CRC Press, Boca Raton, FL.
  • Hobert, J. P. (2011). The data augmentation algorithm: Theory and methodology. In Handbook of Markov Chain Monte Carlo ( S. P. Brooks, A. Gelman, G. L. Jones and X.-L. Meng, eds.) 253–293. CRC Press, Boca Raton, FL.
  • Hobert, J. P. and Geyer, C. J. (1998). Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model. Journal of Multivariate Analysis 67 414-430.
  • Hobert, J. P., Jones, G. L., Presnell, B. and Rosenthal, J. S. (2002). On the applicability of regenerative simulation in Markov chain Monte Carlo. Biometrika 89 731–743.
  • Johnson, A. A. (2009). Geometric Ergodicity of Gibbs Samplers PhD thesis, University of Minnesota, School of Statistics.
  • Johnson, A. A. and Jones, G. L. (2010). Gibbs sampling for a Bayesian hierarchical version of the general linear mixed model. Electronic Journal of Statistics 4 313–333.
  • Johnson, A. A., Jones, G. L. and Neath, R. C. (2011). Component-wise Markov chain Monte Carlo Technical Report, University of Minnesota, School of Statistics.
  • Jones, G. L. (2004). On the Markov chain central limit theorem. Probability Surveys 1 299–320.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statistical Science 16 312–334.
  • Jones, G. L. and Hobert, J. P. (2004). Sufficient burn-in for Gibbs samplers for a hierarchical random effects model. The Annals of Statistics 32 784–817.
  • Jones, G. L., Haran, M., Caffo, B. S. and Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. Journal of the American Statistical Association 101 1537–1547.
  • Liu, J. S., Wong, W. H. and Kong, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27–40.
  • Liu, J. S., Wong, W. H. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. Journal of the Royal Statistical Society, Series B 57 157–169.
  • Marchev, D. and Hobert, J. P. (2004). Geometric ergodicity of van Dyk and Meng’s algorithm for the multivariate Student’s $t$ model. Journal of the American Statistical Association 99 228–238.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer-Verlag, London.
  • Robert, C. P. (1995). Convergence control methods for Markov chain Monte Carlo algorithms. Statistical Science 10 231–253.
  • Roberts, G. O. and Polson, N. G. (1994). On the geometric convergence of the Gibbs sampler. Journal of the Royal Statistical Society, Series B 56 377–384.
  • Roberts, G. O. and Rosenthal, J. S. (1997). Geometric ergodicity and hybrid Markov chains. Electronic Communications in Probability 2 13-25.
  • Roberts, G. O. and Rosenthal, J. S. (1999). Convergence of slice ampler Markov chains. Journal of the Royal Statistical Society, Series B 61 643–660.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes. Scandinavian Journal of Statistics 28 489–504.
  • Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probability Surveys 1 20–71.
  • Román, J. C. (2012). Convergence Analysis of Block Gibbs Samplers for Bayesian General Linear Mixed Models PhD thesis, Department of Statistics, University of Florida.
  • Román, J. C. and Hobert, J. P. (2011). Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors. Technical Report, University of Florida, Department of Statistics.
  • Rosenthal, J. S. (1996). Analysis of the Gibbs sampler for a model related to James-Stein estimators. Statistics and Computing 6 269–275.
  • Roy, V. and Hobert, J. P. (2007). Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression. Journal of the Royal Statistical Society, Series B 69 607–623.
  • Tan, A. and Hobert, J. P. (2009). Block Gibbs sampling for Bayesian random effects models with improper priors: Convergence and regeneration. Journal of Computational and Graphical Statistics 18 861–878.
  • Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association 82 528–550.
  • Tierney, L. (1994). Markov chains for exploring posterior distributions. The Annals of Statistics 22 1701-1762.