Annals of Statistics

Stability of the Gibbs sampler for Bayesian hierarchical models

Omiros Papaspiliopoulos and Gareth Roberts

Full-text: Open access


We characterize the convergence of the Gibbs sampler which samples from the joint posterior distribution of parameters and missing data in hierarchical linear models with arbitrary symmetric error distributions. We show that the convergence can be uniform, geometric or subgeometric depending on the relative tail behavior of the error distributions, and on the parametrization chosen. Our theory is applied to characterize the convergence of the Gibbs sampler on latent Gaussian process models. We indicate how the theoretical framework we introduce will be useful in analyzing more complex models.

Article information

Ann. Statist., Volume 36, Number 1 (2008), 95-117.

First available in Project Euclid: 1 February 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 65C05: Monte Carlo methods
Secondary: 60J27: Continuous-time Markov processes on discrete state spaces

Geometric ergodicity capacitance collapsed Gibbs sampler state-space models parametrization Bayesian robustness


Papaspiliopoulos, Omiros; Roberts, Gareth. Stability of the Gibbs sampler for Bayesian hierarchical models. Ann. Statist. 36 (2008), no. 1, 95--117. doi:10.1214/009053607000000749.

Export citation


  • Yali, A. (1991). On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions. J. Multivariate Anal. 38 82–99.
  • Andrews, D. F. and Mallows, C. L. (1974). Scale mixtures of normal distributions. J. Roy. Statist. Soc. Ser. B 36 99–102.
  • Besag, J., York, J. and Mollié, A. (1991). Bayesian image restoration, with two applications in spatial statistics (with discussion). Ann. Inst. Statist. Math. 43 1–59.
  • Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541–553.
  • Choy, S. T. B. and Walker, S. G. (2003). The extended exponential power distribution and Bayesian robustness. Statist. Probab. Lett. 65 227–232.
  • Christensen, O. F., Roberts, G. O. and Sköld, M. (2006). Robust MCMC methods for spatial GLMM’s. J. Comput. Graph. Statist. 15 1–17.
  • Dawid, A. P. (1973). Posterior expectations for large observations. Biometrika 60 664–667.
  • Diggle, P. J., Tawn, J. A. and Moyeed, R. A. (1998). Model-based geostatistics (with discussion). J. Roy. Statist. Soc. Ser. C 47 299–350.
  • Diggle, P., Heagerty, P. J., Liang, K.-Y. and Zeger, S. L. (2002). Analysis of Longitudinal Data. Oxford Univ. Press.
  • Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1995). Efficient parameterisations for normal linear mixed models. Biometrika 82 479–488.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312–334.
  • Kitagawa, G. (1987). Non-Gaussian state-space modeling of nonstationary time series (with comments). J. Amer. Statist. Assoc. 82 1032–1063.
  • Laird, N. M. and Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics 38 963–974.
  • Lawler, G. and Sokal, A. (1988). Bounds on the L2 spectrum for Markov chains and Markov processes. Trans. Amer. Math. Soc. 309 557–580.
  • Liu, J. S. (1994). The collapsed Gibbs sampler in Bayesian computations with applications to a gene regulation problem. J. Amer. Statist. Assoc. 89 958–966.
  • Liu, J. S., Wong, W. H. and Kong, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27–40.
  • Liu, J. S. and Wu, Y. N. (1999). Parameter expansion for data augmentation. J. Amer. Statist. Assoc. 94 1264–1274.
  • Meng, X.-L. and van Dyk, D. (1997). The EM algorithm—an old folk-song sung to a fast new tune (with discussion). J. Roy. Statist. Soc. Ser. B 59 511–567.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • O’Hagan, A. (1979). On outlier rejection phenomena in Bayes inference. J. Roy. Statist. Soc. Ser. B 41 358–367.
  • Papaspiliopoulos, O., Roberts, G. O. and Sköld, M. (2003). Non-centered parameterizations for hierarchical models and data augmentation (with discussion). In Bayesian Statistics 7 (Tenerife 2002) 307–326. Oxford Univ. Press, New York.
  • Pericchi, L. R. and Smith, A. F. M. (1992). Exact and approximate posterior moments for a normal location parameter. J. Roy. Statist. Soc. Ser. B 54 793–804.
  • Roberts, G. O., Papaspiliopoulos, O. and Dellaportas, P. (2004). Bayesian inference for non-Gaussian Ornstein–Uhlenbeck stochastic volatility processes. J. Roy. Statist. Soc. Ser. B 66 369–394.
  • Roberts, G. O. and Sahu, S. K. (1997). Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. J. Roy. Statist. Soc. Ser. B 59 291–317.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes. Scand. J. Statist. 28 489–504.
  • Rosenthal, J. S. (1995). Rates of convergence for Gibbs sampling for variance component models. Ann. Statist. 23 740–761.
  • Shephard, N. (1994). Partial non-Gaussian state space. Biometrika 81 115–131.
  • Smith, A. F. M. and Roberts, G. O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 55 3–23.
  • Wakefield, J. C., Smith, A. F. M., Racine-Poon, A. and Gelfand, A. E. (1994). Bayesian analysis of linear and non-linear population models by using the Gibbs sampler. J. Roy. Statist. Soc. Ser. C 43 201–221.