Electronic Journal of Statistics

Gibbs sampling for a Bayesian hierarchical general linear model

Alicia A. Johnson and Galin L. Jones

Full-text: Open access


We consider a Bayesian hierarchical version of the normal theory general linear model which is practically relevant in the sense that it is general enough to have many applications and it is not straightforward to sample directly from the corresponding posterior distribution. Thus we study a block Gibbs sampler that has the posterior as its invariant distribution. In particular, we establish that the Gibbs sampler converges at a geometric rate. This allows us to establish conditions for a central limit theorem for the ergodic averages used to estimate features of the posterior. Geometric ergodicity is also a key requirement for using batch means methods to consistently estimate the variance of the asymptotic normal distribution. Together, our results give practitioners the tools to be as confident in inferences based on the observations from the Gibbs sampler as they would be with inferences based on random samples from the posterior. Our theoretical results are illustrated with an application to data on the cost of health plans issued by health maintenance organizations.

Article information

Electron. J. Statist., Volume 4 (2010), 313-333.

First available in Project Euclid: 15 March 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Gibbs sampler convergence rate drift condition general linear model geometric ergodicity Markov chain Monte Carlo


Johnson, Alicia A.; Jones, Galin L. Gibbs sampling for a Bayesian hierarchical general linear model. Electron. J. Statist. 4 (2010), 313--333. doi:10.1214/09-EJS515. https://projecteuclid.org/euclid.ejs/1268655652

Export citation


  • Bednorz, W. and Latuszynski, K. (2007). A few remarks on “Fixed-width output analysis for Markov chain Monte Carlo” by Jones et al., Journal of the American Statatistical Association, 102 1485–1486.
  • Chan, K. S. and Geyer, C. J. (1994). Comment on “Markov chains for exploring posterior distributions”., The Annals of Statistics, 22 1747–1758.
  • Flegal, J. M., Haran, M. and Jones, G. L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure?, Statistical Science, 23 250–260.
  • Flegal, J. M. and Jones, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo., The Annals of Statistics, 38 1034–1070.
  • Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004)., Bayesian Data Analysis, Second edition. Chapman & Hall/CRC.
  • Glynn, P. W. and Whitt, W. (1992). The asymptotic validity of sequential stopping rules for stochastic simulations., The Annals of Applied Probability, 2 180–198.
  • Henderson, H. V. and Searle, S. R. (1981). On deriving the inverse of a sum of matrices., SIAM Review, 23 53–60.
  • Hobert, J. P. and Geyer, C. J. (1998). Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model., Journal of Multivariate Analysis, 67 414–430.
  • Hobert, J. P., Jones, G. L., Presnell, B. and Rosenthal, J. S. (2002). On the applicability of regenerative simulation in Markov chain Monte Carlo., Biometrika, 89 731–743.
  • Hobert, J. P., Jones, G. L. and Robert, C. P. (2006). Using a Markov chain to construct a tractable approximation of an intractable probability distribution., Scandinavian Journal of Statistics, 33 37–51.
  • Hodges, J. S. (1998). Some algebra and geometry for hierarchical models, applied to diagnostics., Journal of the Royal Statistical Society, Series B, 60 497–536.
  • Jones, G. L., Haran, M., Caffo, B. S. and Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo., Journal of the American Statistical Association, 101 1537–1547.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo., Statistical Science, 16 312–334.
  • Jones, G. L. and Hobert, J. P. (2004). Sufficient burn-in for Gibbs samplers for a hierarchical random effects model., The Annals of Statistics, 32 784–817.
  • Meyn, S. P. and Tweedie, R. L. (1993)., Markov chains and Stochastic Stability. Springer, London.
  • Mykland, P., Tierney, L. and Yu, B. (1995). Regeneration in Markov chain samplers., Journal of the American Statistical Association, 90 233–241.
  • Papaspiliopoulos, O. and Roberts, G. (2008). Stability of the Gibbs sampler for Bayesian hierarchical models., The Annals of Statistics, 36 95–117.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes., Scandinavian Journal of Statistics, 28 489–504.
  • Rosenthal, J. S. (1995). Rates of convergence for Gibbs sampling for variance component models., The Annals of Statistics, 23 740–761.
  • Spiegelhalter, D., Thomas, A., Best, N. and Lunn, D. (2005). Winbugs version 2.10. Tech. rep., MRC Biostatistics Unit, Cambridge:, UK.
  • Tan, A. and Hobert, J. P. (2009). Block Gibbs sampling for Bayesian random effects models with improper priors: convergence and regeneration., Journal of Computational and Graphical Statistics, 18 861–878.