Electronic Journal of Statistics

Neutral noninformative and informative conjugate beta and gamma prior distributions

Jouni Kerman

Full-text: Open access

Abstract

The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I call ‘neutral’ priors because they lead to posterior distributions with approximately 50 per cent probability that the true value is either smaller or larger than the maximum likelihood estimate. This holds for all sample sizes and exposures as long as the point estimate is not at the boundary of the parameter space. I also discuss the construction of informative prior distributions. Under the suggested formulation, the posterior median coincides approximately with the weighted average of the prior median and the sample mean, yielding priors that perform more intuitively than those obtained by matching moments and quantiles.

Article information

Source
Electron. J. Statist. Volume 5 (2011), 1450-1470.

Dates
First available in Project Euclid: 4 November 2011

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1320416981

Digital Object Identifier
doi:10.1214/11-EJS648

Mathematical Reviews number (MathSciNet)
MR2851686

Zentralblatt MATH identifier
1271.62045

Keywords
Prior distributions noninformative distributions Bayesian inference conjugate analysis beta distribution gamma distribution

Citation

Kerman, Jouni. Neutral noninformative and informative conjugate beta and gamma prior distributions. Electron. J. Statist. 5 (2011), 1450--1470. doi:10.1214/11-EJS648. https://projecteuclid.org/euclid.ejs/1320416981


Export citation

References

  • [1] Agresti, A. and Coull, B. A. (1998). Approximate Is Better than “Exact” for Interval Estimation of Binomial Proportions., The American Statistician 52 119–126.
  • [2] Agresti, A. and Min, Y. (2001). On Small-Sample Confidence Intervals for Parameters in Discrete Distributions., Biometrics 57 963–971.
  • [3] Bayarri, M. J. and Berger, J. O. (2004). The Interplay of Bayesian and Frequentist Analysis., Statistical Science 19 pp. 58-80.
  • [4] Berg, C. and Pedersen, H. L. (2006). The Chen-Rubin Conjecture in a Continuous Setting., Methods and Applications of Analysis 13 63–88.
  • [5] Berger, J. (2006). The case for objective Bayesian analysis., Bayesian Analysis 1 385–402.
  • [6] Bernardo, J. M. (1979). Reference Posterior Distributions for Bayesian Inference., Journal of the Royal Statistical Society. Series B (Methodological) 41 113–147.
  • [7] Bernardo, J. M. (2005). Reference Analysis. In, Handbook of Statistics 25: Bayesian Thinking, Modeling and Computation (D. K. Dey and C. R. Rao, eds.) 17–90. Elsevier, Amsterdam.
  • [8] Box, G. E. P. and Tiao, G. C. (1973)., Bayesian Inference in Statistical Data Analysis, 1st ed. Wiley-Interscience, New York.
  • [9] Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004)., Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, London.
  • [10] Gelman, A., Jakulin, A., Pittau, M. G. and Yu, S.-S. (2008). A weakly informative default prior distrbution for logistic and other regression models., The Annals of Applied Statistics 2 1360–1383.
  • [11] Haldane, J. B. S. (1948). The Precision of Observed Values of Small Frequencies., Biometrika 35 297–300.
  • [12] Hanley, J. A. and Lippman-Hand, A. (1983). If nothing goes wrong, is everything all right? Interpreting zero numerators., Journal of the American Medical Association 249 1743–1745.
  • [13] Jeffreys, H. (1961)., Theory of probability, 3rd ed. Oxford University Press, New York.
  • [14] Jovanovic, B. D. and Levy, P. S. (1997). A Look at the Rule of Three., The American Statistician 51 137–139.
  • [15] Kerman, J. (2011). A closed-form approximation for the median of the beta distribution. arXiv:1111.0433v1, [math.ST]
  • [16] Neuenschwander, B., Rouyrre, N., Hollaender, N., Zuber, E. and Branson, M. (2011). A proof of concept phase II non-inferiority criterion., Statistics in Medicine 30 1618–1627.
  • [17] O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006)., Uncertain judgements: Eliciting experts’ Probabilities. Wiley, Hoboken, NJ.
  • [18] Rubin, D. B. (1984). Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician., The Annals of Statistics 12 pp. 1151-1172.
  • [19] Spiegelhalter, D. J., Abrams, K. R. and Myles, J. P. (2004)., Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley, Chichester.
  • [20] Tuyl, F., Gerlach, R. and Mengersen, K. (2008). A comparison of Bayes-Laplace, Jeffreys, and other priors: the case of zero events., The American Statistician 62 40–44.
  • [21] Winkler, R. L., Smith, J. E. and Fryback, D. G. (2002). The Role of Informative Priors in Zero-Numerator Problems: Being Conservative versus Being Candid., The American Statistician 56 1–4.