The Annals of Statistics

Sufficient burn-in for Gibbs samplers for a hierarchical random effects model

Galin L. Jones and James P. Hobert

Full-text: Open access

Abstract

We consider Gibbs and block Gibbs samplers for a Bayesian hierarchical version of the one-way random effects model. Drift and minorization conditions are established for the underlying Markov chains. The drift and minorization are used in conjunction with results from J. S. Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558–566] and G. O. Roberts and R. L. Tweedie [Stochastic Process. Appl. 80 (1999) 211–229] to construct analytical upper bounds on the distance to stationarity. These lead to upper bounds on the amount of burn-in that is required to get the chain within a prespecified (total variation) distance of the stationary distribution. The results are illustrated with a numerical example.

Article information

Source
Ann. Statist., Volume 32, Number 2 (2004), 784-817.

Dates
First available in Project Euclid: 28 April 2004

Permanent link to this document
https://projecteuclid.org/euclid.aos/1083178947

Digital Object Identifier
doi:10.1214/009053604000000184

Mathematical Reviews number (MathSciNet)
MR2060178

Zentralblatt MATH identifier
1048.62069

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 62F15: Bayesian inference

Keywords
Block Gibbs sampler burn-in convergence rate drift condition geometric ergodicity Markov chain minorization condition Monte Carlo total variation distance

Citation

Jones, Galin L.; Hobert, James P. Sufficient burn-in for Gibbs samplers for a hierarchical random effects model. Ann. Statist. 32 (2004), no. 2, 784--817. doi:10.1214/009053604000000184. https://projecteuclid.org/euclid.aos/1083178947


Export citation

References

  • Cowles, M. K. and Carlin, B. P. (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review. J. Amer. Statist. Assoc. 91 883--904.
  • Cowles, M. K., Roberts, G. O. and Rosenthal, J. S. (1999). Possible biases induced by MCMC convergence diagnostics. J. Statist. Comput. Simulation 64 87--104.
  • Cowles, M. K. and Rosenthal, J. S. (1998). A simulation approach to convergence rates for Markov chain Monte Carlo algorithms. Statist. Comput. 8 115--124.
  • Douc, R., Moulines, E. and Rosenthal, J. S. (2002). Quantitative bounds for geometric convergence rates of Markov chains. Technical report, Dept. Statistics, Univ. Toronto.
  • Gelfand, A. E., Hills, S. E., Racine-Poon, A. and Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. J. Amer. Statist. Assoc. 85 972--985.
  • Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398--409.
  • Hobert, J. P. (2001). Discussion of ``The art of data augmentation,'' by D. A. van Dyk and X.-L. Meng. J. Comput. Graph. Statist. 10 59--68.
  • Hobert, J. P. and Geyer, C. J. (1998). Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model. J. Multivariate Anal. 67 414--430.
  • Jones, G. L. (2001). Convergence rates and Monte Carlo standard errors for Markov chain Monte Carlo algorithms. Ph.D. dissertation, Univ. Florida.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312--334.
  • Marchev, D. and Hobert, J. P. (2004). Geometric ergodicity of van Dyk and Meng's algorithm for the multivariate Student's $t$ model. J. Amer. Statist. Assoc. 99 228--238.
  • Meng, X.-L. and van Dyk, D. A. (1999). Seeking efficient data augmentation schemes via conditional and marginal augmentation. Biometrika 86 301--320.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Meyn, S. P. and Tweedie, R. L. (1994). Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4 981--1011.
  • Natarajan, R. and McCulloch, C. E. (1998). Gibbs sampling with diffuse proper priors: A valid approach to data-driven inference? J. Comput. Graph. Statist. 7 267--277.
  • Nummelin, E. (1984). General Irreducible Markov Chains and Non-negative Operators. Cambridge Univ. Press.
  • Roberts, G. O. and Tweedie, R. L. (1999). Bounds on regeneration times and convergence rates for Markov chains. Stochastic Process. Appl. 80 211--229.
  • Roberts, G. O. and Tweedie, R. L. (2001). Corrigendum to ``Bounds on regeneration times and convergence rates for Markov chains.'' Stochastic Process. Appl. 91 337--338.
  • Rosenthal, J. S. (1995a). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558--566.
  • Rosenthal, J. S. (1995b). Rates of convergence for Gibbs sampling for variance component models. Ann. Statist. 23 740--761.
  • Rosenthal, J. S. (1996). Analysis of the Gibbs sampler for a model related to James--Stein estimators. Statist. Comput. 6 269--275.
  • Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Statist. Assoc. 82 528--550.
  • Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist. 22 1701--1762.
  • van Dyk, D. A. and Meng, X.-L. (2001). The art of data augmentation (with discussion). J. Comput. Graph. Statist. 10 1--111.