Electronic Journal of Statistics

Adaptive Bayesian density estimation with location-scale mixtures

Willem Kruijer, Judith Rousseau, and Aad van der Vaart

Full-text: Open access

Abstract

We study convergence rates of Bayesian density estimators based on finite location-scale mixtures of exponential power distributions. We construct approximations of β-Hölder densities be continuous mixtures of exponential power distributions, leading to approximations of the β-Hölder densities by finite mixtures. These results are then used to derive posterior concentration rates, with priors based on these mixture models. The rates are minimax (up to a logn term) and since the priors are independent of the smoothness the rates are adaptive to the smoothness.

Article information

Source
Electron. J. Statist., Volume 4 (2010), 1225-1257.

Dates
First available in Project Euclid: 8 November 2010

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1289226500

Digital Object Identifier
doi:10.1214/10-EJS584

Mathematical Reviews number (MathSciNet)
MR2735885

Zentralblatt MATH identifier
1329.62188

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Keywords
Rate-adaptive density estimation Bayesian density estimation nonparametric density estimation convergence rates location-scale mixtures

Citation

Kruijer, Willem; Rousseau, Judith; van der Vaart, Aad. Adaptive Bayesian density estimation with location-scale mixtures. Electron. J. Statist. 4 (2010), 1225--1257. doi:10.1214/10-EJS584. https://projecteuclid.org/euclid.ejs/1289226500


Export citation

References

  • [1] Milton Abramowitz and Irene A. Stegun., Handbook of mathematical functions with formulas, graphs, and mathematical tables, volume 55 of National Bureau of Standards Applied Mathematics Series. For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C., http://www.math.sfu.ca/~cbm/aands/, 1964.
  • [2] R. De Jonge and H. Van Zanten. Adaptive nonparametric bayesian inference using location-scale mixture priors., Ann. Statist., 38(6) :3300–3320, 2010.
  • [3] Ronald A. DeVore and George G. Lorentz., Constructive approximation, volume 303 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin, 1993.
  • [4] Luc Devroye. A note on the usefulness of superkernels in density estimation., Ann. Statist., 20(4) :2037–2056, 1992.
  • [5] Jean Diebolt and Christian P. Robert. Estimation of finite mixture distributions through Bayesian sampling., J. Roy. Statist. Soc. Ser. B, 56(2):363–375, 1994.
  • [6] Michael D. Escobar and Mike West. Bayesian density estimation and inference using mixtures., J. Amer. Statist. Assoc., 90(430):577–588, 1995.
  • [7] Stuart Geman and Chii-Ruey Hwang. Nonparametric maximum likelihood estimation by the method of sieves., Ann. Statist., 10(2):401–414, 1982.
  • [8] Christopher R. Genovese and Larry Wasserman. Rates of convergence for the Gaussian mixture sieve., Ann. Statist., 28(4) :1105–1127, 2000.
  • [9] S. Ghosal, J. K. Ghosh, and R. V. Ramamoorthi. Posterior consistency of Dirichlet mixtures in density estimation., Ann. Statist., 27(1):143–158, 1999.
  • [10] Subhashis Ghosal. Convergence rates for density estimation with Bernstein polynomials., Ann. Statist., 29(5) :1264–1280, 2001.
  • [11] Subhashis Ghosal, Jayanta K. Ghosh, and Aad W. van der Vaart. Convergence rates of posterior distributions., Ann. Statist., 28(2):500–531, 2000.
  • [12] Subhashis Ghosal, Jüri Lember, and Aad Van Der Vaart. On Bayesian adaptation. In, Proceedings of the Eighth Vilnius Conference on Probability Theory and Mathematical Statistics, Part II (2002), volume 79, pages 165–175, 2003.
  • [13] Subhashis Ghosal and Aad van der Vaart. Posterior convergence rates of Dirichlet mixtures at smooth densities., Ann. Statist., 35(2):697–723, 2007.
  • [14] Subhashis Ghosal and Aad W. van der Vaart. Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities., Ann. Statist., 29(5) :1233–1263, 2001.
  • [15] Ulf Grenander., Abstract inference. John Wiley & Sons Inc., New York, 1981. Wiley Series in Probability and Mathematical Statistics.
  • [16] Nils Lid Hjort, Chris Holmes, Peter Müller, and Stephen G. Walker (Editors)., Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 2010.
  • [17] Tzee-Ming Huang. Convergence rates for posterior distributions and adaptive estimation., Ann. Statist., 32(4) :1556–1593, 2004.
  • [18] A. P. Korostelëv and A. B. Tsybakov., Minimax theory of image reconstruction, volume 82 of Lecture Notes in Statistics. Springer-Verlag, New York, 1993.
  • [19] Willem Kruijer., Convergence Rates in Nonparametric Bayesian Density Estimation. PhD-thesis. Department of Mathematics, Vrije Universiteit Amsterdam, http://www.math.vu.nl/~kruijer/PhDthesis_Kruijer.pdf, 2008.
  • [20] Willem Kruijer and Aad Van der Vaart. Posterior convergence rates for dirichlet mixtures of beta densities., Journal of Statistical Planning and Inference, 138(7) :1981–1992, 2008.
  • [21] Michael Lavine. Some aspects of Pólya tree distributions for statistical modelling., Ann. Statist., 20(3) :1222–1235, 1992.
  • [22] J.M. Marin, K. Mengersen, and C.P. Robert., Bayesian modelling and inference on mixtures of distributions. Elsevier-Sciences, 2005.
  • [23] C. Maugis and B. Michel. A non asymptotic penalized criterion for gaussian mixture model selection, forthcoming., ESAIM P&S.
  • [24] Whitney K. Newey, Fushing Hsieh, and James M. Robins. Twicing kernels and a small bias property of semiparametric estimators., Econometrica, 72(3):947–962, 2004.
  • [25] Sylvia Richardson and Peter J. Green. On Bayesian analysis of mixtures with an unknown number of components., J. Roy. Statist. Soc. Ser. B, 59(4):731–792, 1997.
  • [26] Judith Rousseau. Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density., Ann. Statist., 38:146–180, 2010.
  • [27] C. Scricciolo. Convergence rates of posterior distributions for dirichlet mixtures of normal densities. working paper 2001-21. Technical report, 2001.
  • [28] C. Scricciolo. Posterior rates of convergence for dirichlet mixtures of exponential power densities, preprint. Technical report, 2010.
  • [29] C. Shalizi. Dynamics of bayesian updating with dependent data and misspecified models, preprint., 2009.
  • [30] M. P. Wand and M. C. Jones., Kernel smoothing, volume 60 of Monographs on Statistics and Applied Probability. Chapman and Hall Ltd., London, 1995.
  • [31] Yuefeng Wu and Subhashis Ghosal. Kullback Leibler property of kernel mixture priors in Bayesian density estimation., Electron. J. Stat., 2:298–331, 2008.
  • [32] Tong Zhang. From, ε-entropy to KL-entropy: analysis of minimum information complexity density estimation. Ann. Statist., 34(5) :2180–2210, 2006.
  • [33] Tong Zhang. Information-theoretic upper and lower bounds for statistical estimation., IEEE Trans. Inform. Theory, 52(4) :1307–1321, 2006.