Statistical Science

A General Framework for the Parametrization of Hierarchical Models

Omiros Papaspiliopoulos, Gareth O. Roberts, and Martin Sköld

Full-text: Open access

Abstract

In this paper, we describe centering and noncentering methodology as complementary techniques for use in parametrization of broad classes of hierarchical models, with a view to the construction of effective MCMC algorithms for exploring posterior distributions from these models. We give a clear qualitative understanding as to when centering and noncentering work well, and introduce theory concerning the convergence time complexity of Gibbs samplers using centered and noncentered parametrizations. We give general recipes for the construction of noncentered parametrizations, including an auxiliary variable technique called the state-space expansion technique. We also describe partially noncentered methods, and demonstrate their use in constructing robust Gibbs sampler algorithms whose convergence properties are not overly sensitive to the data.

Article information

Source
Statist. Sci. Volume 22, Number 1 (2007), 59-73.

Dates
First available in Project Euclid: 1 August 2007

Permanent link to this document
https://projecteuclid.org/euclid.ss/1185975637

Digital Object Identifier
doi:10.1214/088342307000000014

Mathematical Reviews number (MathSciNet)
MR2408661

Zentralblatt MATH identifier
1246.62195

Keywords
Parametrization hierarchical models latent stochastic processes MCMC

Citation

Papaspiliopoulos, Omiros; Roberts, Gareth O.; Sköld, Martin. A General Framework for the Parametrization of Hierarchical Models. Statist. Sci. 22 (2007), no. 1, 59--73. doi:10.1214/088342307000000014. https://projecteuclid.org/euclid.ss/1185975637.


Export citation

References

  • Amit, Y. (1991). On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions. J. Multivariate Anal. 38 82--99.
  • Beskos, A., Papaspiliopoulos, O., Roberts, G. O. and Fearnhead, P. (2006). Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 68 333--382.
  • Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541--553.
  • Catlin, S. N., Abkowitz, J. L. and Guttorp, P. (2001). Statistical inference in a two-compartment model for hematopoiesis. Biometrics 57 546--553.
  • Chib, S., Pitt, M. and Shephard, N. (2004). Likelihood based inference for diffusion driven models. Working paper. Available at www.nuff.ox.ac.uk/economics/papers/2004/w20/chibpittshephard.pdf.
  • Christensen, O. F., Roberts, G. O. and Sköld, M. (2006). Robust Markov chain Monte Carlo methods for spatial generalized linear mixed models J. Comput. Graph. Statist. 15 1--17.
  • Christensen, O. F. and Waagepetersen, R. (2002). Bayesian prediction of spatial count data using generalized linear mixed models. Biometrics 58 280--286.
  • Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363--375.
  • Diggle, P. J., Tawn, J. A. and Moyeed, R. A. (1998). Model-based geostatistics (with discussion). Appl. Statist. 47 299--350.
  • Escobar, M. D. (1994). Estimating normal means with a Dirichlet process prior. J. Amer. Statist. Assoc. 89 268--277.
  • Ferguson, T. S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2 615--629.
  • Gelfand, A. E. (2000). Gibbs sampling. J. Amer. Statist. Assoc. 95 1300--1304.
  • Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1995). Efficient parameterisations for normal linear mixed models. Biometrika 82 479--488.
  • Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1996). Efficient parametrizations for generalized linear mixed models. In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 165--180. Oxford Univ. Press, New York.
  • Ghysels, E., Harvey, A. C. and Renault, E. (1996). Stochastic volatility. In Statistical Methods in Finance (G. S. Maddala and C. R. Rao, eds.) 119--191. North-Holland, Amsterdam.
  • Griffin, J. E. and Steel, M. F. J. (2003). Flexible mixture modelling of stochastic frontiers. Technical Report 418, Dept. Statistics, Univ. Warwick.
  • Griffin, J. E. and Steel, M. F. J. (2004). Semiparametric Bayesian inference for stochastic frontier models. J. Econometrics 123 121--152.
  • Ishwaran, H. and Zarepour, M. (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika 87 371--390.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312--334.
  • Kitagawa, G. (1987). Non-Gaussian state-space modeling of nonstationary time series (with discussion). J. Amer. Statist. Assoc. 82 1032--1063.
  • Knorr-Held, L. and Rue, H. (2002). On block updating in Markov random field models for disease mapping. Scand. J. Statist. 29 597--614.
  • Lavine, M. and West, M. (1992). A Bayesian method for classification and discrimination. Canad. J. Statist. 20 451--461.
  • Liu, J. S. (1994). Fraction of missing information and convergence rate of data augmentation. In Computing Science and Statistics: Proc. 26th Symposiom on the Interface 490--496. Interface Foundation of North America, Fairfax Station, VA.
  • Liu, J. S., Wong, W. H. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. J. Roy. Statist. Soc. Ser. B 57 157--169.
  • Møller, J., Syversveen, A. R. and Waagepetersen, R. P. (1998). Log Gaussian Cox processes. Scand. J. Statist. 25 451--482.
  • Müller, P., Erkanli, A. and West, M. (1996). Bayesian curve fitting using multivariate normal mixtures. Biometrika 83 67--79.
  • Neal, P. and Roberts, G. O. (2005). A case study in noncentering for data augmentention: Stochastic epidemics. Stat. Comput. 15 315--327.
  • Nielsen, B. and Shephard, N. (2003). Likelihood analysis of a first-order autoregressive model with exponential innovations. J. Time Ser. Anal. 24 337--344.
  • O'Neill, P. D. and Roberts, G. O. (1999). Bayesian inference for partially observed stochastic epidemics. J. Roy. Statist. Soc. Ser. A 162 121--129.
  • Papaspiliopoulos, O. (2003). Non-centered parametrizations for hierarchical models and data augmentation. Ph.D. dissertation, Dept. Mathematics and Statistics, Lancaster Univ.
  • Papaspiliopoulos, O. and Roberts, G. (2004). Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Available at www.warwick.ac.uk/staff/O.Papaspiliopoulos/research.html.
  • Papaspiliopoulos, O. and Roberts, G. (2005). Stability of the Gibbs sampler for Bayesian hierarchical models. Submitted.
  • Papaspiliopoulos, O., Roberts, G. O., and Sköld, M. (2003). Non-centered parameterizations for hierarchical models and data augmentation (with discussion). In Bayesian Statistics 7 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.) 307--326. Oxford Univ. Press, New York.
  • Peruggia, M., Van Zandt, T. and Chen, M. (2002). Was it a car or a cat I saw? An analysis of response times for word recognition. Case Studies in Bayesian Statistics VI. Lecture Notes in Statist. 167 319--334. Springer, New York.
  • Renshaw, E. and Gibson, G. J. (1998). Can Markov chain Monte Carlo be usefully applied to stochastic processes with hidden birth times? Inverse Problems 14 1581--1606.
  • Roberts, G. O., Papaspiliopoulos, O. and Dellaportas, P. (2004). Bayesian inference for non-Gaussian Ornstein--Uhlenbeck stochastic volatility processes. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 369--393.
  • Roberts, G. O. and Sahu, S. K. (1997). Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. J. Roy. Statist. Soc. Ser. B 59 291--317.
  • Roberts, G. O. and Stramer, O. (2001). On inference for partially observed nonlinear diffusion models using the Metropolis--Hastings algorithm. Biometrika 88 603--621.
  • Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley, New York.
  • Rue, H. (2001). Fast sampling of Gaussian Markov random fields. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 325--338.
  • Rue, H. and Tjelmeland, H. (2002). Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Statist. 29 31--49.
  • Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4 639--650.
  • Shephard, N. (1994). Partial non-Gaussian state space. Biometrika 81 115--131.
  • Walker, S. G., Damien, P., Laud, P. W. and Smith, A. F. M. (1999). Bayesian nonparametric inference for random distributions and related functions (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 61 485--527.
  • White, J. S. (1958). The limiting distribution of the serial correlation coefficient in the explosive case. Ann. Math. Statist. 29 1188--1197.