Bayesian Analysis

Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper)

Andrew Gelman

Full-text: Open access

Abstract

Various noninformative prior distributions have been suggested for scale parameters in hierarchical models. We construct a new folded-noncentral-$t$ family of conditionally conjugate priors for hierarchical standard deviation parameters, and then consider noninformative and weakly informative priors in this family. We use an example to illustrate serious problems with the inverse-gamma family of "noninformative" prior distributions. We suggest instead to use a uniform prior on the hierarchical standard deviation, using the half-$t$ family when the number of groups is small and in other settings where a weakly informative prior is desired. We also illustrate the use of the half-$t$ family for hierarchical modeling of multiple variance parameters such as arise in the analysis of variance.

Article information

Source
Bayesian Anal. Volume 1, Number 3 (2006), 515-534.

Dates
First available in Project Euclid: 22 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340371048

Digital Object Identifier
doi:10.1214/06-BA117A

Mathematical Reviews number (MathSciNet)
MR2221284

Zentralblatt MATH identifier
1331.62139

Keywords
Bayesian inference conditional conjugacy folded-noncentral-$t$ distribution half-$t$ distribution hierarchical model multilevel model noninformative prior distribution weakly informative prior distribution

Citation

Gelman, Andrew. Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Anal. 1 (2006), no. 3, 515--534. doi:10.1214/06-BA117A. https://projecteuclid.org/euclid.ba/1340371048.


Export citation

References

  • Barnard, J., McCulloch, R. E., and Meng, X. L. (2000). "Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage." Statistica Sinica, 10: 1281–1311.
  • Bayarri, M. J. and Berger, J. (2000). "P-values for composite null models." Journal of the American Statistical Association, 95: 1127–1142. (with discussion).
  • Bernardo, J. M. (1979). "Reference posterior distributions for Bayesian inference." Journal of the Royal Statistical Society B, 41: 113–147. (with discussion).
  • Bickel, P. and Blackwell, D. (1967). "A note on Bayes estimates." Annals of Mathematical Statistics, 38: 1907–1911.
  • Box, G. E. P. (1980). "Sampling and Bayes inference in scientific modelling and robustness." Journal of the Royal Statistical Society A, 143: 383–430.
  • Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Analysis. Reading, Mass.: Addison-Wesley.
  • Browne, W. J. and Draper, D. (2005). "A comparison of Bayesian and likelihood-based methods for fitting multilevel models." Bayesian Analysis, This issue.
  • Carlin, B. P. and Louis, T. A. (2001). Bayes and Empirical Bayes Methods for Data Analysis. Chapman and Hall, second edition edition.
  • Christiansen, C. and Morris, C. (1997). "Hierarchical Poisson regression models." Journal of the American Statistical Association, 92: 618–632.
  • Daniel, C. (1959). "Use of half-normal plots in interpreting factorial two-level experiments." Technometrics, 1: 311–341.
  • Daniels, M. J. (1999). "A prior for the variance in hierarchical models." Canadian Journal of Statistics, 27: 569–580.
  • Daniels, M. J. and Kass, R. E. (1999)). "Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models." Journal of the American Statistical Association, 94: 1254–1263.
  • –- (2001). "Shrinkage estimators for covariance matrices." Biometrics, 57: 1173–1184.
  • Efron, B. and Morris, C. (1975). "Data analysis using Stein's estimator and its generalizations." Journal of the American Statistical Association, 70: 311–319.
  • Gelfand, A. E. and Smith, A. F. M. (1990). "Sampling-based approaches to calculating marginal densities." Journal of the American Statistical Association, 85: 398–409.
  • Gelman, A. (2003). "Bugs.R: functions for calling Bugs from R." http://www.stat.columbia.edu/$\sim$gelman/bugsR/.
  • –- (2004). "Parameterization and Bayesian modeling." Journal of the American Statistical Association, 99: 537 – 545.
  • –- (2005). "Analysis of variance: why it is more important than ever." Annals of Statistics, 33: 1 – 53. With discussion.
  • Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2003). Bayesian Data Analysis. London: Chapman and Hall, second edition edition.
  • Gelman, A., Huang, Z., van Dyk, D., and Boscardin, W. J. (2005). "Transformed and parameter-expanded Gibbs samplers for multilevel linear and generalized linear models." Technical report, Department of Statistics, Columbia University.
  • Gelman, A., Meng, X. L., and Stern, H. S. (1996). "Posterior predictive assessment of model fitness via realized discrepancies." Statistica Sinica, 6: 733–807. (with discussion).
  • Hill, B. M. (1965). "Inference about variance components in the one-way model." Journal of the American Statistical Association, 60: 806–825.
  • James, W. and Stein, C. (1960). "Estimation with quadratic loss." In Neyman, J. (ed.), Proceedings of the Fourth Berkeley Symposium, volume 1, 361–380. Berkeley: University of California Press.
  • Jaynes, E. T. (1983). Papers on Probability, Statistics, and Statistical Physics. Dordrecht, Netherlands: Reidel.
  • Jeffreys, H. (1961). Theory of Probability. Oxford University Press, third edition edition.
  • Johnson, N. L. and Kotz, S. (1972). Distributions in Statistics. New York: Wiley. 4 vols.
  • Kass, R. E. and Raftery, A. E. (1995). "Bayes factors and model uncertainty." Journal of the American Statistical Association, 90: 773–795.
  • Kass, R. E. and Wasserman, L. (1996). "The selection of prior distributions by formal rules." Journal of the American Statistical Association, 91: 1343–1370.
  • Kreft, I. and De Leeuw, J. (1998). Introducing Multilevel Modeling. Sage.
  • Liu, C. (2001). "Bayesian analysis of multivariate probit models. Discussion of “The art of data augmentation” by D. A. van Dyk and X. L. Meng." Journal of Computational and Graphical Statistics, 10: 75–81.
  • Liu, C., Rubin, D. B., and Wu, Y. N. (1998). "Parameter expansion to accelerate EM: the PX-EM algorithm." Biometrika, 85: 755–770.
  • Liu, J. and Wu, Y. N. (1999). "Parameter expansion for data augmentation." Journal of the American Statistical Association, 94: 1264–1274.
  • Meng, X. L. and Zaslavsky, A. M. (2002). "Single observation unbiased priors." Annals of Statistics, 30: 1345–1375.
  • Morris, C. (1983). "Parametric empirical Bayes inference: theory and applications (with discussion)." Journal of the American Statistical Association, 78: 47–65.
  • Natarajan, R. and Kass, R. E. (2000). "Reference Bayesian methods for generalized linear mixed models." Journal of the American Statistical Association, 95: 227–237.
  • O'Hagan, A. (1995). "Fractional Bayes factors for model comparison (with discussion)." Journal of the Royal Statistical Society B, 57: 99–138.
  • Pauler, D. K., Wakefield, J. C., and Kass, R. E. (1999). "Bayes factors for variance component models." Journal of the American Statistical Association, 94: 1242–1253.
  • Portnoy, S. (1971). "Formal Bayes estimation with applications to a random effects model." Annals of Mathematical Statistics, 42: 1379–1402.
  • R Development Core Team (2003). "R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing." http://www.r-project.org.
  • Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical Linear Models. Thousand Oaks, Calif.: Sage., second edition.
  • Rubin, D. B. (1981). "Estimation in parallel randomized experiments." Journal of Educational Statistics, 6: 377–401.
  • Sargent, D. J. and Hodges, J. S. (1997). "Smoothed ANOVA with application to subgroup analysis." Technical report, Department of Biostatistics, University of Minnesota.
  • Savage, L. J. (1954). The Foundations of Statistics. New York: Dover.
  • Snijders, T. A. B. and Bosker, R. J. (1999). Multilevel Analysis. London: Sage.
  • Spiegelhalter, D. J., Abrams, K. R., and Myles, J. P. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation, chapter section 5.7.3. Chichester: Wiley.
  • Spiegelhalter, D. J., Thomas, A., Best, N. G., Gilks, W. R., and Lunn, D. (1994, 2003). "BUGS: Bayesian inference using Gibbs sampling." MRC Biostatistics Unit, Cambridge, England,http://www.mrc-bsu.cam.ac.uk/bugs/.
  • Stein, C. (1955). "Inadmissibility of the usual estimator for the mean of a multivariate normal distribution." In Neyman, J. (ed.), Proceedings of the Third Berkeley Symposium, volume 1, 197–206. Berkeley: University of California Press.
  • Stone, M. and Springer, B. G. F. (1965). "A paradox involving quasi-prior distributions." Biometrika, 52: 623–627.
  • Tiao, G. C. and Tan, W. Y. (1965). "Bayesian analysis of random-effect models in the analysis of variance. I: Posterior distribution of variance components." Biometrika, 52: 37–53.
  • van Dyk, D. A. and Meng, X. L. (2001). "The art of data augmentation (with discussion)." Journal of Computational and Graphical Statistics, 10: 1–111.

See also

  • Related item: William J. Brown, David Draper. A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Anal., Vol. 1, Iss. 3 (2006), 473-514.