The Annals of Statistics

Supremum norm posterior contraction and credible sets for nonparametric multivariate regression

William Weimin Yoo and Subhashis Ghosal

Full-text: Open access

Abstract

In the setting of nonparametric multivariate regression with unknown error variance $\sigma^{2}$, we study asymptotic properties of a Bayesian method for estimating a regression function $f$ and its mixed partial derivatives. We use a random series of tensor product of B-splines with normal basis coefficients as a prior for $f$, and $\sigma$ is either estimated using the empirical Bayes approach or is endowed with a suitable prior in a hierarchical Bayes approach. We establish pointwise, $L_{2}$ and $L_{\infty}$-posterior contraction rates for $f$ and its mixed partial derivatives, and show that they coincide with the minimax rates. Our results cover even the anisotropic situation, where the true regression function may have different smoothness in different directions. Using the convergence bounds, we show that pointwise, $L_{2}$ and $L_{\infty}$-credible sets for $f$ and its mixed partial derivatives have guaranteed frequentist coverage with optimal size. New results on tensor products of B-splines are also obtained in the course.

Article information

Source
Ann. Statist., Volume 44, Number 3 (2016), 1069-1102.

Dates
Received: November 2014
Revised: September 2015
First available in Project Euclid: 11 April 2016

Permanent link to this document
https://projecteuclid.org/euclid.aos/1460381687

Digital Object Identifier
doi:10.1214/15-AOS1398

Mathematical Reviews number (MathSciNet)
MR3485954

Zentralblatt MATH identifier
1338.62121

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G05: Estimation 62G15: Tolerance and confidence regions 62G20: Asymptotic properties

Keywords
Tensor product B-splines sup-norm posterior contraction nonparametric multivariate regression mixed partial derivatives anisotropic smoothness

Citation

Yoo, William Weimin; Ghosal, Subhashis. Supremum norm posterior contraction and credible sets for nonparametric multivariate regression. Ann. Statist. 44 (2016), no. 3, 1069--1102. doi:10.1214/15-AOS1398. https://projecteuclid.org/euclid.aos/1460381687


Export citation

References

  • [1] Bickel, P. J. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. Ann. Statist. 1 1071–1095.
  • [2] Castillo, I. (2014). On Bayesian supremum norm contraction rates. Ann. Statist. 42 2058–2091.
  • [3] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999–2028.
  • [4] Castillo, I. and Nickl, R. (2014). On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941–1969.
  • [5] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014). Anti-concentration and honest, adaptive confidence bands. Ann. Statist. 42 1787–1818.
  • [6] Claeskens, G. and Van Keilegom, I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Ann. Statist. 31 1852–1884.
  • [7] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [8] de Boor, C. (2001). A Practical Guide to Splines, Revised ed. Springer, New York.
  • [9] de Jonge, R. and van Zanten, J. H. (2012). Adaptive estimation of multivariate functions using conditionally Gaussian tensor-product spline priors. Electron. J. Stat. 6 1984–2001.
  • [10] de Jonge, R. and van Zanten, J. H. (2013). Semiparametric Bernstein–von Mises for the error standard deviation. Electron. J. Stat. 7 217–243.
  • [11] Demko, S., Moss, W. F. and Smith, P. W. (1984). Decay rates for inverses of band matrices. Math. Comp. 43 491–499.
  • [12] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • [13] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • [14] Giné, E. and Nickl, R. (2011). Rates on contraction for posterior distributions in $L^{r}$-metrics, $1\leq r\leq\infty$. Ann. Statist. 39 2883–2911.
  • [15] Harville, D. A. (1997). Matrix Algebra from a Statistician’s Perspective. Springer, New York.
  • [16] Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression. Ann. Statist. 30 325–396.
  • [17] Hoffmann, M., Rousseau, J. and Schmidt-Hieber, J. (2015). On adaptive posterior concentration rates. Ann. Statist. 43 2259–2295.
  • [18] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [19] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2013). Bayesian recovery of the initial condition for the heat equation. Comm. Statist. Theory Methods 42 1294–1313.
  • [20] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
  • [21] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Springer, Berlin.
  • [22] Neumann, M. H. and von Sachs, R. (1997). Wavelet thresholding in anisotropic function classes and application to adaptive estimation of evolutionary spectra. Ann. Statist. 25 38–76.
  • [23] Ray, K. (2015). Adaptive Bernstein–von Mises theorems in Gaussian white noise. Preprint. Available at arXiv:1407.3397v2.
  • [24] Schumaker, L. L. (2007). Spline Functions: Basic Theory, 3rd ed. Cambridge Univ. Press, Cambridge.
  • [25] Scricciolo, C. (2014). Adaptive Bayesian density estimation in $L^{p}$-metrics with Pitman–Yor or normalized inverse-Gaussian process kernel mixtures. Bayesian Anal. 9 475–520.
  • [26] Searle, S. R. (1982). Matrix Algebra Useful for Statistics. Wiley, Chichester.
  • [27] Serra, P. and Krivobokova, T. (2014). Adaptive empirical Bayesian smoothing splines. Preprint. Available at arXiv:1411.6860.
  • [28] Shen, W. and Ghosal, S. (2015). Adaptive Bayesian procedures using random series priors. Scand. J. Stat. 42 1194–1213.
  • [29] Shen, W. and Ghosal, S. (2016). Adaptive Bayesian density regression for high-dimensional data. Bernoulli 22 396–420.
  • [30] Smirnov, N. V. (1950). On the construction of confidence regions for the density of distribution of random variables. Doklady Akad. Nauk SSSR (N.S.) 74 189–191.
  • [31] Sniekers, S. and van der Vaart, A. W. (2015). Credible sets in the fixed design model with Brownian motion prior. J. Statist. Plann. Inference 166 78–86.
  • [32] Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8 1348–1360.
  • [33] Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040–1053.
  • [34] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428.
  • [35] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • [36] Zhou, S., Shen, X. and Wolfe, D. A. (1998). Local asymptotics for regression splines and confidence regions. Ann. Statist. 26 1760–1782.