The Annals of Statistics

Bernstein–von Mises theorem for linear functionals of the density

Vincent Rivoirard and Judith Rousseau

Full-text: Open access

Abstract

In this paper, we study the asymptotic posterior distribution of linear functionals of the density by deriving general conditions to obtain a semi-parametric version of the Bernstein–von Mises theorem. The special case of the cumulative distributive function, evaluated at a specific point, is widely considered. In particular, we show that for infinite-dimensional exponential families, under quite general assumptions, the asymptotic posterior distribution of the functional can be either Gaussian or a mixture of Gaussian distributions with different centering points. This illustrates the positive, but also the negative, phenomena that can occur in the study of Bernstein–von Mises results.

Article information

Source
Ann. Statist. Volume 40, Number 3 (2012), 1489-1523.

Dates
First available in Project Euclid: 5 September 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1346850063

Digital Object Identifier
doi:10.1214/12-AOS1004

Mathematical Reviews number (MathSciNet)
MR3015033

Zentralblatt MATH identifier
1257.62036

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Keywords
Bayesian nonparametric rates of convergence Bernstein–von Mises adaptive estimation

Citation

Rivoirard, Vincent; Rousseau, Judith. Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist. 40 (2012), no. 3, 1489--1523. doi:10.1214/12-AOS1004. https://projecteuclid.org/euclid.aos/1346850063


Export citation

References

  • [1] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • [2] Bickel, P. and Kleijn, B. (2011). The semiparametric Bernstein–von Mises theorem. Preprint. Available at arXiv:1007.0179v2.
  • [3] Boucheron, S. and Gassiat, E. (2009). A Bernstein–von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • [4] Castillo, I. (2012). Semiparametric Bernstein–von Mises theorem and bias, illustrated with Gaussian process priors. Sankhyā. To appear.
  • [5] Castillo, I. (2012). A semi-parametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields 152 53–99.
  • [6] Chambaz, A. and Rousseau, J. (2008). Bounds for Bayesian order identification with application to mixtures. Ann. Statist. 36 938–962.
  • [7] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [8] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [9] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • [10] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [11] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [12] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer, New York.
  • [13] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • [14] Huang, T.-M. (2004). Convergence rates for posterior distributions and adaptive estimation. Ann. Statist. 32 1556–1593.
  • [15] Kim, Y. (2006). The Bernstein–von Mises theorem for the proportional hazard model. Ann. Statist. 34 1678–1700.
  • [16] Kim, Y. and Lee, J. (2004). A Bernstein–von Mises theorem in the nonparametric right-censoring model. Ann. Statist. 32 1492–1512.
  • [17] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems. Preprint. Available at http://arxiv.org/abs/1103.2692v1.
  • [18] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [19] LeCam, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes’ estimates. Univ. California Publ. Statist. 1 277–329.
  • [20] Mallat, S. (1998). A Wavelet Tour of Signal Processing. Academic Press, San Diego, CA.
  • [21] Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics 37. Cambridge Univ. Press, Cambridge.
  • [22] Rivoirard, V. and Rousseau, J. (2012). Posterior concentration rates for infinite dimensional exponential families. Bayesian Anal. 7 311–334.
  • [23] Rivoirard, V. and Rousseau, J. (2012). Supplement to “Bernstein–von Mises theorem for linear functionals of the density.” DOI:10.1214/12-AOS1004SUPP.
  • [24] Robert, C. P. (2007). The Bayesian Choice. From Decision-Theoretic Foundations to Computational Implementation, 2nd ed. Springer, New York.
  • [25] Scricciolo, C. (2006). Convergence rates for Bayesian density estimation of infinite-dimensional exponential families. Ann. Statist. 34 2897–2920.
  • [26] Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • [27] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • [28] van der Vaart, A. (2002). Semiparametric statistics. In Lectures on Probability Theory and Statistics (Saint-Flour, 1999). Lecture Notes in Math. 1781 331–457. Springer, Berlin.
  • [29] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [30] Verdinelli, I. and Wasserman, L. (1998). Bayesian goodness-of-fit testing using infinite-dimensional exponential families. Ann. Statist. 26 1215–1241.

Supplemental materials

  • Supplementary material: Bernstein–von Mises theorem for linear functionals of the density: Supplementary material. The supplementary material gives the proofs of Proposition 2.1 and of Lemma 2.1.