The Annals of Statistics

On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures

Ismaël Castillo and Richard Nickl

Full-text: Open access

Abstract

We continue the investigation of Bernstein–von Mises theorems for nonparametric Bayes procedures from [Ann. Statist. 41 (2013) 1999–2028]. We introduce multiscale spaces on which nonparametric priors and posteriors are naturally defined, and prove Bernstein–von Mises theorems for a variety of priors in the setting of Gaussian nonparametric regression and in the i.i.d. sampling model. From these results we deduce several applications where posterior-based inference coincides with efficient frequentist procedures, including Donsker– and Kolmogorov–Smirnov theorems for the random posterior cumulative distribution functions. We also show that multiscale posterior credible bands for the regression or density function are optimal frequentist confidence bands.

Article information

Source
Ann. Statist., Volume 42, Number 5 (2014), 1941-1969.

Dates
First available in Project Euclid: 11 September 2014

Permanent link to this document
https://projecteuclid.org/euclid.aos/1410440630

Digital Object Identifier
doi:10.1214/14-AOS1246

Mathematical Reviews number (MathSciNet)
MR3262473

Zentralblatt MATH identifier
1305.62190

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G15: Tolerance and confidence regions 62G08: Nonparametric regression

Keywords
Bayesian inference posterior asymptotics multiscale statistics

Citation

Castillo, Ismaël; Nickl, Richard. On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 (2014), no. 5, 1941--1969. doi:10.1214/14-AOS1246. https://projecteuclid.org/euclid.aos/1410440630


Export citation

References

  • [1] Bogachev, V. I. (1998). Gaussian Measures. Amer. Math. Soc., Providence, RI.
  • [2] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.
  • [3] Castillo, I. (2014). On Bayesian supremum norm contraction rates. Ann. Statist. To appear. Available at arXiv:1304.1761v2.
  • [4] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999–2028.
  • [5] Castillo, I. and Rousseau, J. (2013). A general Bernstein–von Mises theorem in semiparametric models. Preprint. Available at arXiv:1305.4482.
  • [6] Ciesielski, Z., Kerkyacharian, G. and Roynette, B. (1993). Quelques espaces fonctionnels associés à des processus Gaussiens. Studia Math. 107 171–204.
  • [7] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [8] Conti, P. L. (1999). Large sample Bayesian analysis for $\mathrmGeo/G/1$ discrete-time queueing models. Ann. Statist. 27 1785–1807.
  • [9] Conti, P. L. (2004). Approximated inference for the quantile function via Dirichlet processes. Metron 62 201–222.
  • [10] Davies, P. L. and Kovac, A. (2001). Local extremes, runs, strings and multiresolution. Ann. Statist. 29 1–65.
  • [11] Davies, P. L., Kovac, A. and Meise, M. (2009). Nonparametric regression, confidence regions and regularization. Ann. Statist. 37 2597–2625.
  • [12] Dümbgen, L. and Spokoiny, V. G. (2001). Multiscale testing of qualitative hypotheses. Ann. Statist. 29 124–152.
  • [13] Dümbgen, L. and Walther, G. (2008). Multiscale inference about a density. Ann. Statist. 36 1758–1785.
  • [14] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [15] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [16] Giné, E. and Nickl, R. (2009). Uniform limit theorems for wavelet density estimators. Ann. Probab. 37 1605–1646.
  • [17] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • [18] Giné, E. and Zinn, J. (1990). Bootstrapping general empirical measures. Ann. Probab. 18 851–869.
  • [19] Hall, P. (1979). On the rate of convergence of normal extremes. J. Appl. Probab. 16 433–439.
  • [20] Hall, P. (1992). Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Statist. 20 675–694.
  • [21] Hjort, N. L. and Petrone, S. (2007). Nonparametric quantile inference using Dirichlet processes. In Advances in Statistical Modeling and Inference 463–492. World Sci. Publ., Hackensack, NJ.
  • [22] Hjort, N. L. and Walker, S. G. (2009). Quantile pyramids for Bayesian nonparametrics. Ann. Statist. 37 105–131.
  • [23] Hoffmann-Jørgensen, J., Shepp, L. A. and Dudley, R. M. (1979). On the lower tail of Gaussian seminorms. Ann. Probab. 7 319–342.
  • [24] Johnstone, I. M. (2010). High dimensional Bernstein–von Mises: Simple examples. In Borrowing Strength: Theory Powering Applications—a Festschrift for Lawrence D. Brown. Inst. Math. Stat. Collect. 6 87–98. IMS, Beachwood, OH.
  • [25] Kato, K. (2013). Quasi-Bayesian analysis of nonparametric instrumental variables models. Ann. Statist. 41 2359–2390.
  • [26] Kim, Y. (2006). The Bernstein–von Mises theorem for the proportional hazard model. Ann. Statist. 34 1678–1700.
  • [27] Kim, Y. and Lee, J. (2004). A Bernstein–von Mises theorem in the nonparametric right-censoring model. Ann. Statist. 32 1492–1512.
  • [28] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [29] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI.
  • [30] Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • [31] Lo, A. Y. (1983). Weak convergence for Dirichlet processes. Sankhyā Ser. A 45 105–111.
  • [32] Rivoirard, V. and Rousseau, J. (2012). Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489–1523.
  • [33] Schmidt-Hieber, J., Munk, A. and Dümbgen, L. (2013). Multiscale methods for shape constraints in deconvolution: Confidence statements for qualitative features. Ann. Statist. 41 1299–1328.
  • [34] Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2013). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Preprint. Available at arXiv:1310.4489.
  • [35] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge.
  • [36] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.
  • [37] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.