The Annals of Statistics

A Bernstein–von Mises theorem for smooth functionals in semiparametric models

Ismaël Castillo and Judith Rousseau

Full-text: Open access

Abstract

A Bernstein–von Mises theorem is derived for general semiparametric functionals. The result is applied to a variety of semiparametric problems in i.i.d. and non-i.i.d. situations. In particular, new tools are developed to handle semiparametric bias, in particular for nonlinear functionals and in cases where regularity is possibly low. Examples include the squared $L^{2}$-norm in Gaussian white noise, nonlinear functionals in density estimation, as well as functionals in autoregressive models. For density estimation, a systematic study of BvM results for two important classes of priors is provided, namely random histograms and Gaussian process priors.

Article information

Source
Ann. Statist., Volume 43, Number 6 (2015), 2353-2383.

Dates
Received: May 2013
Revised: April 2015
First available in Project Euclid: 7 October 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1444222078

Digital Object Identifier
doi:10.1214/15-AOS1336

Mathematical Reviews number (MathSciNet)
MR3405597

Zentralblatt MATH identifier
1327.62302

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62M15: Spectral analysis

Keywords
Bayesian nonparametrics Bernstein–von Mises theorem posterior concentration semiparametric inference

Citation

Castillo, Ismaël; Rousseau, Judith. A Bernstein–von Mises theorem for smooth functionals in semiparametric models. Ann. Statist. 43 (2015), no. 6, 2353--2383. doi:10.1214/15-AOS1336. https://projecteuclid.org/euclid.aos/1444222078


Export citation

References

  • [1] Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior. Scand. J. Stat. 40 549–570.
  • [2] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • [3] Bickel, P. J. and Kleijn, B. J. K. (2012). The semiparametric Bernstein–von Mises theorem. Ann. Statist. 40 206–237.
  • [4] Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā Ser. A 50 381–393.
  • [5] Bickel, P. J. and Ritov, Y. (2003). Nonparametric estimators which can be “plugged-in”. Ann. Statist. 31 1033–1053.
  • [6] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.
  • [7] Boucheron, S. and Gassiat, E. (2009). A Bernstein–von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • [8] Cai, T. T. and Low, M. G. (2006). Optimal adaptive estimation of a quadratic functional. Ann. Statist. 34 2298–2325.
  • [9] Castillo, I. (2008). Lower bounds for posterior rates with Gaussian process priors. Electron. J. Stat. 2 1281–1299.
  • [10] Castillo, I. (2012). A semiparametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields 152 53–99.
  • [11] Castillo, I. (2012). Semiparametric Bernstein–von Mises theorem and bias, illustrated with Gaussian process priors. Sankhyā 74 194–221.
  • [12] Castillo, I. (2014). On Bayesian supremum norm contraction rates. Ann. Statist. 42 2058–2091.
  • [13] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999–2028.
  • [14] Castillo, I. and Nickl, R. (2014). On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941–1969.
  • [15] Castillo, I. and Rousseau, J. (2015). Supplement to “A Bernstein–von Mises theorem for smooth functionals in semiparametric models.” DOI:10.1214/15-AOS1336SUPP.
  • [16] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [17] De Blasi, P. and Hjort, N. L. (2009). The Bernstein–von Mises theorem in semiparametric competing risks models. J. Statist. Plann. Inference 139 2316–2328.
  • [18] Efromovich, S. and Low, M. (1996). On optimal adaptive estimation of a quadratic functional. Ann. Statist. 24 1106–1125.
  • [19] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [20] Gayraud, G. and Tribouley, K. (1999). Wavelet methods to estimate an integrated quadratic functional: Adaptivity and asymptotic law. Statist. Probab. Lett. 44 109–122.
  • [21] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [22] Ghosal, S. and van der Vaart, A. W. (2007). Convergence rates of posterior distributions for noniid observations. Ann. Statist. 35 192–223.
  • [23] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer, New York.
  • [24] Kim, Y. (2006). The Bernstein–von Mises theorem for the proportional hazard model. Ann. Statist. 34 1678–1700.
  • [25] Knapik, B. T., Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2012). Bayes procedures for adaptive inference in inverse problems for the white noise model. Available at arXiv:1209.3628.
  • [26] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [27] Kruijer, W. and Rousseau, J. (2013). Bayesian semi-parametric estimation of the long-memory parameter under FEXP-priors. Electron. J. Stat. 7 2947–2969.
  • [28] Laurent, B. (1996). Efficient estimation of integral functionals of a density. Ann. Statist. 24 659–681.
  • [29] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [30] Rivoirard, V. and Rousseau, J. (2012). Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489–1523.
  • [31] Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • [32] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [33] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.
  • [34] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.

Supplemental materials

  • Supplement to “A Bernstein–von Mises theorem for smooth functionals in semiparametric models”. In the supplementary material, we state and prove several technical results used in the paper and provide the remaining proofs.