Bayesian Analysis

An enriched conjugate prior for Bayesian nonparametric inference

Sara Wade, Silvia Mongelluzzo, and Sonia Petrone

Full-text: Open access

Abstract

The precision parameter $\alpha$ plays an important role in the Dirichlet Pro- cess. When assigning a Dirichlet Process prior to the set of probability measures on $\mathbb{R}^k, k \gt 1$, this can be restrictive in the sense that the variability is determined by a single parameter. The aim of this paper is to construct an enrichment foof the Dirichlet Process that is more flexible with respect to the precision parameter yet still conjugate, starting from the notion of enriched conjugate priors, which have been proposed to address an analogous lack of flexibility of standard conjugate priors in a parametric setting. The resulting enriched conjugate prior allows more flexibility in modelling uncertainty on the marginal and conditionals. We describe an enriched urn scheme which characterizes this process and show that it can also be obtained from the stick-breaking representation of the marginal and conditionals. For non atomic base measures, this allows global clustering of the marginal variables and local clustering of the conditional variables. Finally, we consider an application to mixture models that allows for uncertainty between homoskedasticity and heteroskedasticity.

Article information

Source
Bayesian Anal. Volume 6, Number 3 (2011), 359-385.

Dates
First available in Project Euclid: 13 June 2012

Permanent link to this document
http://projecteuclid.org/euclid.ba/1339616468

Digital Object Identifier
doi:10.1214/11-BA614

Mathematical Reviews number (MathSciNet)
MR2843536

Zentralblatt MATH identifier
1330.62219

Subjects
Primary: 62F15: Bayesian inference
Secondary: 60G57: Random measures 62G07: Density estimation

Keywords
Bayesian nonparametric inference conjugate priors generalized Dirichlet Dirichlet process mixture models Pólya urns multivariate random distribution functions

Citation

Wade, Sara; Mongelluzzo, Silvia; Petrone, Sonia. An enriched conjugate prior for Bayesian nonparametric inference. Bayesian Anal. 6 (2011), no. 3, 359--385. doi:10.1214/11-BA614. http://projecteuclid.org/euclid.ba/1339616468.


Export citation

References

  • Blackwell, D. and MacQueen, J. B. (1973). "Ferguson Distributions Via Pólya Urn Schemes." Annals of Statistics, 1: 353–355.
  • Cifarelli, D. and Regazzini, E. (1978). "Problemi statistici nonparametrici in condizioni di scambiabilità parziale e impiego di medie associative." Quaderni Istituto di Matematica Finanziaria, Università di Torino, 12: 1–36. English translation available at www.unibocconi.it/wps/allegatiCTP/CR-Scamb-parz[1].20080528.135739.pdf.
  • –- (1996). "De Finetti's contribution to probability and statistics." Statistical Science, 11: 253–282.
  • Connor, R. and Mosimann, J. E. (1969). "Concepts of Independence for Proportions with a Generalization of the Dirichlet Distribution." Journal of the American Statistical Association, 64: 194–206.
  • Consonni, G. and Veronese, P. (2001). "Conditionally reducible natural exponential families and enriched conjugate priors." Scandinavian Journal of Statistics, 28: 377–406.
  • Diaconis, P. and Ylvisaker, D. (1979). "Conjugate priors for exponential families." Annals of Statistics, 7: 269–281.
  • Doksum, K. A. (1974). "Tailfree and Neutral random probabilities and their posterior distributions." Annals of Probability, 2: 183–201.
  • Escobar, M. and West, M. (1995). "Bayesian density estimation and inference using mixtures." Journal of the American Statistical Association, 90: 577–588.
  • Ferguson, T. (1973). "A Bayesian analysis of some nonparametric problems." Annals of Statistics, 1: 209–230.
  • Geiger, D. and Heckerman, D. (1997). "A characterization of the Dirichlet distribution through global and local parameter independence." Annals of Statistics, 25: 1344–1369.
  • Ghosh, J. and Ramamoorthi, R. (2003). Bayesian nonparametrics. New York: Springer-Verlag, Springer Series in Statistics.
  • MacEachern, S. (1999). "Dependent nonparametric processes." In ASA Proceedings of the Section on Bayesian Statistical Science, 50–55. Alexandria, VA: American Statistical Association.
  • Neal, R. (2000). "Markov Chain Sampling Methods for Dirichlet Process Mixture Models." Journal of Computational and Graphical Statistcs, 9: 249–265.
  • Rabe-Hesketh, S. and Skrondal, A. (2005). Multilevel and longitudinal modeling using stata. College Station, Texas: Stata Press.
  • Ramamoorthi, R. and Sangalli, L. (2006). "On a Characterization of Dirichlet Distribution." In Upadhyay, S., Singh, U., and Dey, D. (eds.), Proceedings of the International Conference on Bayesian Statistics and its Applications, Jan. 6-8, 2005, 385–397. Varanasi, India: Banaras Hindu University.
  • Rodriguez, A., Dunson, D., and Gelfand, A. (2006). "The nested Dirichlet Process." Journal of the American Statistical Association, 103: 1131–1154.
  • Sethuraman, J. (1994). "A Constructive Definition of Dirichlet Priors." Statistica Sinica, 4: 639–650.
  • Springer, M. and Thompson, W. (1970). "The distribution of Products of Beta, Gamma and Gaussian Random Variables." Journal on Applied Mathematics, 18: 721–737.
  • Teh, Y., Jordan, M., Beal, M., and Blei, D. (2006). "Hierarchical Dirichlet Process." Journal of the American Statistical Association, 101: 1566–1581.