Bayesian Analysis

Adaptive Empirical Bayesian Smoothing Splines

Paulo Serra and Tatyana Krivobokova

Full-text: Open access

Abstract

In this paper we develop and study adaptive empirical Bayesian smoothing splines. These are smoothing splines with both smoothing parameter and penalty order determined via the empirical Bayes method from the marginal likelihood of the model. The selected order and smoothing parameter are used to construct adaptive credible sets with good frequentist coverage for the underlying regression function. We use these credible sets as a proxy to show the superior performance of adaptive empirical Bayesian smoothing splines compared to frequentist smoothing splines.

Article information

Source
Bayesian Anal., Volume 12, Number 1 (2017), 219-238.

Dates
First available in Project Euclid: 7 March 2016

Permanent link to this document
https://projecteuclid.org/euclid.ba/1457383100

Digital Object Identifier
doi:10.1214/16-BA997

Mathematical Reviews number (MathSciNet)
MR3597573

Zentralblatt MATH identifier
1384.62118

Keywords
adaptive estimation unbiased risk minimiser maximum likelihood oracle parameters

Citation

Serra, Paulo; Krivobokova, Tatyana. Adaptive Empirical Bayesian Smoothing Splines. Bayesian Anal. 12 (2017), no. 1, 219--238. doi:10.1214/16-BA997. https://projecteuclid.org/euclid.ba/1457383100


Export citation

References

  • Babenko, A. and Belitser, E. (2010). “Oracle convergence rate of posterior under projection prior and Bayesian model selection.” Mathematical Methods of Statistics, 219–245.
  • Belitser, E. and Enikeeva, F. (2008). “Empirical Bayesian test of the smoothness.” Mathematical Methods of Statistics, 17(1): 1–18.
  • Belitser, E. and Ghosal, S. (2003). “Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution.” Annals of Statistics, 31(2): 536–559.
  • Belitser, E. and Levit, B. (2003). “On the empirical Bayes approach to adaptive filtering.” Mathematical Methods of Statistics, 12(2): 131–154.
  • de Jonge, R. and van Zanten, J. (2010). “Adaptive nonparametric Bayesian inference using location-scale mixture priors.” Annals of Statistics, 3300–3320.
  • de Jonge, R. and van Zanten, J. H. (2012). “Adaptive estimation of multivariate functions using conditionally Gaussian tensor-product spline priors.” Electronic Journal of Statistics, 6: 1984–2001.
  • Genovese, C. and Wasserman, L. (2008). “Adaptive confidence bands.” Annals of Statistics, 36(2): 875–905.
  • Ghosal, S., Ghosh, J., and van der Vaart, A. (2000). “Convergence rates of posterior distributions.” Annals of Statistics, 28: 500–531.
  • Ghosal, S., Lember, J., and van der Vaart, A. (2008). “Nonparametric Bayesian model selection and averaging.” Electronic Journal of Statistics, 2: 63–89.
  • Ghosal, S. and van der Vaart, A. (2007). “Convergence rates of posterior distributions for non-i.i.d. observations.” Annals of Statistics, 35(1): 192–223.
  • Giné, E. and Nickl, R. (2010). “Confidence bands in density estimation.” Annals of Statistics, 38(2): 1122–1170.
  • Johnstone, I. and Silverman, B. (2005). “Empirical Bayes selection of wavelet thresholds.” Annals of Statistics, 1700–1752.
  • Kimeldorf, G. and Wahba, G. (1970). “A correspondence between Bayesian estimation on stochastic processes and smoothing by splines.” Annals of Mathematical Statistics, 41: 495–502.
  • Knapik, B., van der Vaart, A., and van Zanten, J. (2011). “Bayesian inverse problems with Gaussian priors.” Annals of Statistics, 2626–2657.
  • Knapik, B. T., Szabó, B., van der Vaart, A., and van Zanten, J. (2013). “Bayes procedures for adaptive inference in inverse problems for the white noise model.” Probability Theory and Related Fields, 1–43.
  • Kohn, R. and Ansley, C. F. (1987). “A new algorithm for spline smoothing based on smoothing a stochastic process.” SIAM Journal on Scientific and Statistical Computing, 8: 33–48.
  • Kotz, S. and Nadarajah, S. (2004). Multivariate t-distributions and their applications. Cambridge University Press.
  • Krivobokova, T. (2013). “Smoothing parameter selection in two frameworks for penalized splines.” Journal of the Royal Statistical Society. Series B, 75: 725–741.
  • Low, M. (1997). “On nonparametric confidence intervals.” Annals of Statistics, 25(6): 2547–2554.
  • Lukas, M. (1998). “Assessing regularised solutions.” Journal of the Australian Mathematical Society, Series B, 30: 24–42.
  • McAuliffe, J., Blei, D., and Jordan, M. (2006). “Nonparametric empirical Bayes for the Dirichlet process mixture model.” Statistics and Computing, 5–14.
  • Picard, D. and Tribouley, K. (2000). “Adaptive confidence interval for pointwise curve estimation.” Annals of Statistics, 28(1): 298–335.
  • Robbins, H. (1955). “An empirical Bayes approach to Statistics.” In Proc. 3rd Berkeley Symp. on Math. Statist. and Prob., volume 1, 157–164. Berkeley: Univ. of California Press.
  • Shen, W. and Ghosal, S. (2015). “Adaptive Bayesian procedures using random series priors.” Scandinavian Journal of Statistics, 43: 1194–1213.
  • Shen, X. and Wasserman, L. (2001). “Rates of convergence of posterior distributions.” Annals of Statistics, 687–714.
  • Serra, P. and Krivobokova, T. (2016). “Supplementary materials: adaptive empirical Bayesian smoothing splines”.” Bayesian Analysis.
  • Speckman, P. (1985). “Spline smoothing and optimal rates of convergence in nonparametric regression models.” Annals of Statistics, 13: 970–983.
  • Speckman, P. and Sun, D. (2003). “Fully Bayesian spline smoothing and intrinsic autoregressive priors.” Biometrika, 90(2): 289–302.
  • Szabó, B., van der Vaart, A., and van Zanten, H. (2014). “Frequentist coverage of adaptive nonparametric Bayesian credible sets.” Annals of Statistics, 43(4): 1391–1428.
  • Unser, M. and Blu, T. (2000). “Fractional splines and wavelets.” SIAM Review, 42(1): 43–67.
  • van der Vaart, A. and van Zanten, J. (2008). “Rates of contraction of posterior distributions based on Gaussian process priors.” Annals of Statistics, 36(3): 1435–1463.
  • van der Vaart, A. W. and van Zanten, J. H. (2009). “Adaptive Bayesian Estimation Using A Gaussian Random Field With Inverse Gamma Bandwidth.” Annals of Statistics, 37(5B): 2655–2675.
  • Wahba, G. (1985). “A comparison of GCV and GML for choosing the smoothing parameter in the generalised spline smoothing problem.” The Annals of Statistics, 1378–1402.
  • Wahba, G. (1990). Spline models for observational data, volume 59. SIAM.
  • Yue, Y. R., Simpson, D., Lindgren, F., and Rue, H. (2014). “Bayesian Adaptive Smoothing Splines Using Stochastic Differential Equations.” Bayesian Analysis, 397–424.
  • Zhang, C.-H. (2005). “General empirical Bayes wavelet methods and exactly adaptive minimax estimation.” Annals of Statistics, 54–100.

Supplemental materials