The Annals of Statistics

Nonparametric Bernstein–von Mises theorems in Gaussian white noise

Ismaël Castillo and Richard Nickl

Full-text: Open access

Abstract

Bernstein–von Mises theorems for nonparametric Bayes priors in the Gaussian white noise model are proved. It is demonstrated how such results justify Bayes methods as efficient frequentist inference procedures in a variety of concrete nonparametric problems. Particularly Bayesian credible sets are constructed that have asymptotically exact $1-\alpha$ frequentist coverage level and whose $L^{2}$-diameter shrinks at the minimax rate of convergence (within logarithmic factors) over Hölder balls. Other applications include general classes of linear and nonlinear functionals and credible bands for auto-convolutions. The assumptions cover nonconjugate product priors defined on general orthonormal bases of $L^{2}$ satisfying weak conditions.

Article information

Source
Ann. Statist., Volume 41, Number 4 (2013), 1999-2028.

Dates
First available in Project Euclid: 23 October 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1382547511

Digital Object Identifier
doi:10.1214/13-AOS1133

Mathematical Reviews number (MathSciNet)
MR3127856

Zentralblatt MATH identifier
1285.62052

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G15: Tolerance and confidence regions 62G08: Nonparametric regression

Keywords
Bayesian inference plug-in property efficiency

Citation

Castillo, Ismaël; Nickl, Richard. Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist. 41 (2013), no. 4, 1999--2028. doi:10.1214/13-AOS1133. https://projecteuclid.org/euclid.aos/1382547511


Export citation

References

  • [1] Araujo, A. and Giné, E. (1980). The Central Limit Theorem for Real and Banach Valued Random Variables. Wiley, New York.
  • [2] Bickel, P. J. and Kleijn, B. J. K. (2012). The semiparametric Bernstein–von Mises theorem. Ann. Statist. 40 206–237.
  • [3] Bickel, P. J. and Ritov, Y. (2003). Nonparametric estimators which can be “plugged-in.” Ann. Statist. 31 1033–1053.
  • [4] Billingsley, P. and Topsøe, F. (1967). Uniformity in weak convergence. Z. Wahrsch. Verw. Gebiete 7 1–16.
  • [5] Bogachev, V. I. (1998). Gaussian Measures. Mathematical Surveys and Monographs 62. Amer. Math. Soc., Providence, RI.
  • [6] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.
  • [7] Boucheron, S. and Gassiat, E. (2009). A Bernstein–von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • [8] Castillo, I. (2012). A semiparametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields 152 53–99.
  • [9] Clarke, B. and Ghosal, S. (2010). Reference priors for exponential families with increasing dimension. Electron. J. Stat. 4 737–780.
  • [10] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [11] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [12] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Univ. Press, Cambridge.
  • [13] Folland, G. B. (1999). Real Analysis: Modern Techniques and Their Applications, 2nd ed. Wiley, New York.
  • [14] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [15] Frees, E. W. (1994). Estimating densities of functions of observations. J. Amer. Statist. Assoc. 89 517–525.
  • [16] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [17] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • [18] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [19] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [20] Giné, E. and Mason, D. M. (2007). On local $U$-statistic processes and the estimation of densities of functions of several sample variables. Ann. Statist. 35 1105–1145.
  • [21] Giné, E. and Nickl, R. (2008). Uniform central limit theorems for kernel density estimators. Probab. Theory Related Fields 141 333–387.
  • [22] Giné, E. and Nickl, R. (2009). Uniform limit theorems for wavelet density estimators. Ann. Probab. 37 1605–1646.
  • [23] Giné, E. and Nickl, R. (2011). Rates on contraction for posterior distributions in $L^{r}$-metrics, $1\leq r\leq\infty$. Ann. Statist. 39 2883–2911.
  • [24] Kiefer, J. and Wolfowitz, J. (1976). Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrsch. Verw. Gebiete 34 73–85.
  • [25] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [26] Laplace, P. S. (1810). Mémoire sur les formules qui sont fonctions de très grands nombres et sur leurs applications aux probabilités. Oeuvres de Laplace 12 301–345.
  • [27] Laurent, B. (1996). Efficient estimation of integral functionals of a density. Ann. Statist. 24 659–681.
  • [28] Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • [29] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [30] Nickl, R. (2007). Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Related Fields 138 411–449. Correction ibid. 141 331–332.
  • [31] Nickl, R. (2009). On convergence and convolutions of random signed measures. J. Theoret. Probab. 22 38–56.
  • [32] Rivoirard, V. and Rousseau, J. (2012). Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489–1523.
  • [33] Schick, A. and Wefelmeyer, W. (2004). Root $n$ consistent density estimators for sums of independent random variables. J. Nonparametr. Stat. 16 925–935.
  • [34] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • [35] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [36] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.
  • [37] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.
  • [38] van der Vaart, A. W. and van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. Ann. Statist. 37 2655–2675.
  • [39] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [40] von Mises, R. (1931). Wahrscheinlichkeitsrechnung. Deuticke, Vienna.