The Annals of Statistics

On the posterior distribution of the number of components in a finite mixture

Agostino Nobile

Full-text: Open access

Abstract

The posterior distribution of the number of components k in a finite mixture satisfies a set of inequality constraints. The result holds irrespective of the parametric form of the mixture components and under assumptions on the prior distribution weaker than those routinely made in the literature on Bayesian analysis of finite mixtures. The inequality constraints can be used to perform an “internal” consistency check of MCMC estimates of the posterior distribution of k and to provide improved estimates which are required to satisfy the constraints. Bounds on the posterior probability of k components are derived using the constraints. Implications on prior distribution specification and on the adequacy of the posterior distribution of k as a tool for selecting an adequate number of components in the mixture are also explored.

Article information

Source
Ann. Statist., Volume 32, Number 5 (2004), 2044-2073.

Dates
First available in Project Euclid: 27 October 2004

Permanent link to this document
https://projecteuclid.org/euclid.aos/1098883781

Digital Object Identifier
doi:10.1214/009053604000000788

Mathematical Reviews number (MathSciNet)
MR2102502

Zentralblatt MATH identifier
1056.62037

Subjects
Primary: 62F15: Bayesian inference

Keywords
Bayesian analysis constrained estimation finite mixture distribution Markov chain Monte Carlo prior distribution

Citation

Nobile, Agostino. On the posterior distribution of the number of components in a finite mixture. Ann. Statist. 32 (2004), no. 5, 2044--2073. doi:10.1214/009053604000000788. https://projecteuclid.org/euclid.aos/1098883781


Export citation

References

  • Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions. National Bureau of Standards, Washington.
  • Carlin, B. P. and Chib, S. (1995). Bayesian model choice via Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 57 473--484.
  • Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363--375.
  • Geyer, C. J. (1992). Practical Markov chain Monte Carlo (with discussion). Statist. Sci. 7 473--503.
  • Goodall, C. (1995). $S$ functions for quadratic programming. Available from StatLib at http://lib. stat.cmu.edu/S/quadratic.
  • Mengersen, K. L. and Robert, C. P. (1996). Testing for mixtures: A Bayesian entropic approach. In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 255--276. Oxford Univ. Press.
  • Nobile, A. (1994). Bayesian analysis of finite mixture distributions. Ph.D. dissertation, Dept. Statistics, Carnegie Mellon Univ., Pittsburgh.
  • Phillips, D. B. and Smith, A. F. M. (1996). Bayesian model comparison via jump diffusions. In Markov Chain Monte Carlo in Practice (W. R. Gilks, S. Richardson and D. J. Spiegelhalter, eds.) 215--239. Chapman and Hall, London.
  • Raftery, A. E. (1996). Hypothesis testing and model selection. In Markov Chain Monte Carlo in Practice (W. R. Gilks, S. Richardson and D. J. Spiegelhalter, eds.) 163--187. Chapman and Hall, London.
  • Richardson, S. and Green, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. Roy. Statist. Soc. Ser. B 59 731--792.
  • Ripley, B. D. (1987). Stochastic Simulation. Wiley, New York.
  • Robert, C. P. (1996). Mixtures of distributions: Inference and estimation. In Markov Chain Monte Carlo in Practice (W. R. Gilks, S. Richardson and D. J. Spiegelhalter, eds.) 441--464. Chapman and Hall, London.
  • Roeder, K. and Wasserman, L. (1997). Practical Bayesian density estimation using mixtures of normals. J. Amer. Statist. Assoc. 92 894--902.
  • Stephens, M. (2000). Bayesian analysis of mixture models with an unknown number of components: An alternative to reversible jump methods. Ann. Statist. 28 40--74.
  • Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley, New York.