Electronic Journal of Statistics

Semiparametric Bernstein–von Mises for the error standard deviation

René de Jonge and Harry van Zanten

Full-text: Open access

Abstract

We study Bayes procedures for nonparametric regression problems with Gaussian errors, giving conditions under which a Bernstein–von Mises result holds for the marginal posterior distribution of the error standard deviation. We apply our general results to show that a single Bayes procedure using a hierarchical spline-based prior on the regression function and an independent prior on the error variance, can simultaneously achieve adaptive, rate-optimal estimation of a smooth, multivariate regression function and efficient, $\sqrt{n}$-consistent estimation of the error standard deviation.

Article information

Source
Electron. J. Statist. Volume 7 (2013), 217-243.

Dates
First available in Project Euclid: 24 January 2013

Permanent link to this document
http://projecteuclid.org/euclid.ejs/1359041590

Digital Object Identifier
doi:10.1214/13-EJS768

Mathematical Reviews number (MathSciNet)
MR3020419

Zentralblatt MATH identifier
1337.62087

Subjects
Primary: 62G09: Resampling methods
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62G20: Asymptotic properties

Keywords
Nonparametric regression Bayesian inference estimation of error variance semiparametric Bernstein-von Mises

Citation

de Jonge, René; van Zanten, Harry. Semiparametric Bernstein–von Mises for the error standard deviation. Electron. J. Statist. 7 (2013), 217--243. doi:10.1214/13-EJS768. http://projecteuclid.org/euclid.ejs/1359041590.


Export citation

References

  • Bhattacharya, A., Pati, D. and Dunson, D. B. (2012). Adaptive dimension reduction with a Gaussian process prior., Preprint.
  • Bickel, P. and Kleijn, B. J. K. (2012). The semiparametric Bernstein-von Mises theorem., Ann. Statist. 40 206–237.
  • Brown, L. D. and Levine, M. (2007). Variance estimation in nonparametric regression via the difference sequence method., Ann. Statist. 35 2219–2232.
  • Castillo, I. (2008). Lower bounds for posterior rates with Gaussian process priors., Electron. J. Stat. 2 1281–1299.
  • Castillo, I. (2012a). A semiparametric Bernstein - von Mises theorem for Gaussian process priors., Probab. Theory Related Fields 152 53–99.
  • Castillo, I. (2012b). Semiparametric Bernstein-von Mises theorem and bias, illustrated with Gaussian process priors., Sankhya A to appear.
  • De Blasi, P. and Hjort, N. L. (2009). The Bernstein-von Mises theorem in semiparametric competing risks models., J. Statist. Plann. Inference 139 2316–2328.
  • De Jonge, R. and Van Zanten, J. H. (2010). Adaptive nonparametric Bayesian inference using location-scale mixture priors., Ann. Statist. 38 3300–3320.
  • De Jonge, R. and Van Zanten, J. H. (2012). Adaptive estimation of multivariate functions using conditionally Gaussian tensor-product spline priors., Electron. J. Stat. 6 1984–2001.
  • Edmunds, D. E. and Triebel, H. (1996)., Function spaces, entropy numbers, differential operators. Cambridge Tracts in Mathematics 120. Cambridge University Press, Cambridge.
  • Ghosal, S. and Van der Vaart, A. W. (2007). Convergence rates for posterior distributions for noniid observations., Ann. Statist. 35 697–723.
  • Huang, T.-M. (2004). Convergence rates for posterior distributions and adaptive estimation., Ann. Statist. 32 1556–1593.
  • Li, W. V. and Linde, W. (1999). Approximation, metric entropy and small ball estimates for Gaussian measures., The Annals of Probability 27 1556–1578.
  • Lifshits, M. and Simon, T. (2005). Small deviations for fractional stable processes., Ann. Inst. H. Poincaré Probab. Statist. 41 725–752.
  • Rasmussen, C. E. and Williams, C. K. I. (2006)., Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachusetts.
  • Rivoirard, V. and Rousseau, J. (2012). Bernstein–Von Mises Theorem for linear functionals of the density., Ann. Statist. to appear.
  • Schumaker, L. L. (1981)., Spline functions: basic theory. John Wiley & Sons Inc., New York. Pure and Applied Mathematics, A Wiley-Interscience Publication.
  • Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions., J. Amer. Statist. Assoc. 97 222–235.
  • Tokdar, S. A. (2011). Dimension adaptability of Gaussian process models with variable selection and projection., Preprint.
  • Van der Vaart, A. W. (1998)., Asymptotic statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge University Press, Cambridge.
  • Van der Vaart, A. W. and Van Zanten, J. H. (2008a). Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist. 36 1435–1463.
  • Van der Vaart, A. W. and Van Zanten, J. H. (2008b)., Reproducing Kernel Hilbert Spaces of Gaussian priors. IMS Collections 3 200–222. Institute of Mathematical Statistics.
  • Van der Vaart, A. W. and Van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth., Ann. Statist. 37 2655–2675.
  • Van der Vaart, A. W. and Van Zanten, J. H. (2011). Information rates of nonparametric Gaussian process methods., J. Mach. Learn. Res. 12 2095–2119.
  • Van der Vaart, A. W. and Wellner, J. A. (1996)., Weak convergence and empirical processes. Springer Series in Statistics. Springer-Verlag, New York. With applications to statistics.
  • Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression., J. Roy. Statist. Soc. Ser. B 40 364–372.