The Annals of Statistics

Posterior convergence rates of Dirichlet mixtures at smooth densities

Subhashis Ghosal and Aad van der Vaart

Full-text: Open access

Abstract

We study the rates of convergence of the posterior distribution for Bayesian density estimation with Dirichlet mixtures of normal distributions as the prior. The true density is assumed to be twice continuously differentiable. The bandwidth is given a sequence of priors which is obtained by scaling a single prior by an appropriate order. In order to handle this problem, we derive a new general rate theorem by considering a countable covering of the parameter space whose prior probabilities satisfy a summability condition together with certain individual bounds on the Hellinger metric entropy. We apply this new general theorem on posterior convergence rates by computing bounds for Hellinger (bracketing) entropy numbers for the involved class of densities, the error in the approximation of a smooth density by normal mixtures and the concentration rate of the prior. The best obtainable rate of convergence of the posterior turns out to be equivalent to the well-known frequentist rate for integrated mean squared error n−2/5 up to a logarithmic factor.

Article information

Source
Ann. Statist. Volume 35, Number 2 (2007), 697-723.

Dates
First available in Project Euclid: 5 July 2007

Permanent link to this document
http://projecteuclid.org/euclid.aos/1183667289

Digital Object Identifier
doi:10.1214/009053606000001271

Mathematical Reviews number (MathSciNet)
MR2336864

Zentralblatt MATH identifier
1117.62046

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Keywords
Bracketing Dirichlet mixture entropy maximum likelihood mixture of normals posterior distribution rate of convergence sieve

Citation

Ghosal, Subhashis; van der Vaart, Aad. Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 (2007), no. 2, 697--723. doi:10.1214/009053606000001271. http://projecteuclid.org/euclid.aos/1183667289.


Export citation

References

  • Birgé, L. and Massart, P. (1998). Minimum contract estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 329--375.
  • Browder, A. (1996). Mathematical Analysis. An Introduction. Springer, New York.
  • Escobar, M. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577--588.
  • Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209--230.
  • Ferguson, T. S. (1983). Bayesian density estimation by mixtures of normal distributions. In Recent Advances in Statistics (M. H. Rizvi, J. S. Rustagi and D. Siegmund, eds.) 287--302. Academic Press, New York.
  • Genovese, C. and Wasserman, L. (2000). Rates of convergence for the Gaussian mixture sieve. Ann. Statist. 28 1105--1127.
  • Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143--158.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500--531.
  • Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233--1263.
  • Kleijn, B. and van der Vaart, A. W. (2006). Misspecification in infinite-dimensional Bayesian statistics. Ann. Statist. 34 837--877.
  • Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates. I. Density estimates. Ann. Statist. 12 351--357.
  • Scricciolo, C. (2001). Convergence rates of posterior distributions for Dirichlet mixtures of normal densities. Working Paper 2001.21, Univ. Padova.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York.
  • Walker, S. G. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028--2043.
  • Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339--362.