Electronic Journal of Statistics

Adaptive Bayesian credible sets in regression with a Gaussian process prior

Suzanne Sniekers and Aad van der Vaart

Full-text: Open access

Abstract

We investigate two empirical Bayes methods and a hierarchical Bayes method for adapting the scale of a Gaussian process prior in a nonparametric regression model. We show that all methods lead to a posterior contraction rate that adapts to the smoothness of the true regression function. Furthermore, we show that the corresponding credible sets cover the true regression function whenever this function satisfies a certain extrapolation condition. This condition depends on the specific method, but is implied by a condition of self-similarity. The latter condition is shown to be satisfied with probability one under the prior distribution.

Article information

Source
Electron. J. Statist., Volume 9, Number 2 (2015), 2475-2527.

Dates
Received: April 2015
First available in Project Euclid: 19 November 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1447943704

Digital Object Identifier
doi:10.1214/15-EJS1078

Mathematical Reviews number (MathSciNet)
MR3425364

Zentralblatt MATH identifier
1327.62300

Subjects
Primary: 62G15: Tolerance and confidence regions 62G05: Estimation
Secondary: 62G20: Asymptotic properties

Keywords
Credible set coverage uncertainty quantification

Citation

Sniekers, Suzanne; van der Vaart, Aad. Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Statist. 9 (2015), no. 2, 2475--2527. doi:10.1214/15-EJS1078. https://projecteuclid.org/euclid.ejs/1447943704


Export citation

References

  • [1] A. Bull. Honest adaptive confidence bands and self-similar functions., Electron. J. Statist., 6 :1490–1516, 2012.
  • [2] T. T. Cai, M. Low, and Z. Ma. Adaptive confidence bands for nonparametric regression functions., J. Amer. Statist. Assoc., 109(507) :1054–1070, 2014.
  • [3] T. T. Cai and M. G. Low. An adaptation theory for nonparametric confidence intervals., Ann. Statist., 32(5) :1805–1840, 2004.
  • [4] T. T. Cai and M. G. Low. Adaptive confidence balls., Ann. Statist., 34(1):202–228, 2006.
  • [5] D. D. Cox. An analysis of Bayesian inference for nonparametric regression., Ann. Statist., 21(2):903–923, 1993.
  • [6] D. Freedman. On the Bernstein-von Mises theorem with infinite-dimensional parameters., Ann. Statist., 27(4) :1119–1140, 1999.
  • [7] C. Genovese and L. Wasserman. Adaptive confidence bands., Ann. Statist., 36(2):875–905, 2008.
  • [8] S. Ghosal, J. K. Ghosh, and A. W. van der Vaart. Convergence rates of posterior distributions., Ann. Statist., 28(2):500–531, 2000.
  • [9] E. Giné and R. Nickl. Confidence bands in density estimation., Ann. Statist., 38(2) :1122–1170, 2010.
  • [10] M. Hoffmann and R. Nickl. On adaptive inference and confidence bands., Ann. Statist., 39(5) :2383–2409, 2011.
  • [11] A. Juditsky and S. Lambert-Lacroix. On nonparametric confidence set estimation., Math. Meth. of Stat, 19(4):410–428, 2003.
  • [12] G. Kimeldorf and G. Wahba. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines., The Annals of Mathematical Statistics, 41(2):495–502, 1970.
  • [13] B. Knapik, A. W. van der Vaart, and J. H. van Zanten. Bayesian inverse problems with Gaussian priors., Ann. Statist., 39(5) :2626–2657, 2011.
  • [14] M. G. Low. On nonparamteric confidence intervals., Ann. Statist., 25(6) :2547–2554, 1997.
  • [15] J. Robins and A. W. van der Vaart. Adaptive nonparametric confidence sets., Ann. Statist., 34(1):229–253, 2006.
  • [16] S. Sniekers and A. van der Vaart. Credible sets in the fixed design model with brownian motion prior., Journal of Statistical Planning and Inference, (0):–, 2014.
  • [17] B. T. Szabo, A. W. van der Vaart, and J. H. van Zanten. Empirical Bayes scaling of Gaussian priors in the white noise model., Electron. J. Statist., 7:991 –1018, 2013.
  • [18] B. T. Szabo, A. W. van der Vaart, and J. H. van Zanten. Frequentist coverage of adaptive nonparametric Bayesian credible sets., To appear in The Annals of Statistics, 2015.
  • [19] A. van der Vaart and H. van Zanten. Bayesian inference with rescaled Gaussian process priors., Electron. J. Stat., 1:433–448 (electronic), 2007.
  • [20] A. W. van der Vaart and J. H. van Zanten. Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist., 36(3) :1435–1463, 2008.
  • [21] A. W. van der Vaart and J. A. Wellner., Weak convergence and empirical processes. Springer Series in Statistics. Springer-Verlag, New York, 1996. With applications to statistics.
  • [22] G. Wahba. Bayesian “confidence intervals” for the cross-validated smoothing spline., J. Roy. Statist. Soc. Ser. B, 45(1):133–150, 1983.