The Annals of Statistics

Local and global asymptotic inference in smoothing spline models

Abstract

This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. Statist. Math. 37 (1966) 577–580]. Equipped with this tool, we develop four interconnected procedures for inference: (i) pointwise confidence interval; (ii) local likelihood ratio testing; (iii) simultaneous confidence band; (iv) global likelihood ratio testing. In particular, our confidence intervals are proved to be asymptotically valid at any point in the support, and they are shorter on average than the Bayesian confidence intervals proposed by Wahba [J. R. Stat. Soc. Ser. B Stat. Methodol. 45 (1983) 133–150] and Nychka [J. Amer. Statist. Assoc. 83 (1988) 1134–1143]. We also discuss a version of the Wilks phenomenon arising from local/global likelihood ratio testing. It is also worth noting that our simultaneous confidence bands are the first ones applicable to general quasi-likelihood models. Furthermore, issues relating to optimality and efficiency are carefully addressed. As a by-product, we discover a surprising relationship between periodic and nonperiodic smoothing splines in terms of inference.

Article information

Source
Ann. Statist., Volume 41, Number 5 (2013), 2608-2638.

Dates
First available in Project Euclid: 19 November 2013

https://projecteuclid.org/euclid.aos/1384871347

Digital Object Identifier
doi:10.1214/13-AOS1164

Mathematical Reviews number (MathSciNet)
MR3161439

Zentralblatt MATH identifier
1293.62107

Citation

Shang, Zuofeng; Cheng, Guang. Local and global asymptotic inference in smoothing spline models. Ann. Statist. 41 (2013), no. 5, 2608--2638. doi:10.1214/13-AOS1164. https://projecteuclid.org/euclid.aos/1384871347

References

• [1] Banerjee, M. (2007). Likelihood based inference for monotone response models. Ann. Statist. 35 931–956.
• [2] Bickel, P. J. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. Ann. Statist. 1 1071–1095.
• [3] Birkhoff, G. D. (1908). Boundary value and expansion problems of ordinary linear differential equations. Trans. Amer. Math. Soc. 9 373–395.
• [4] Chen, J. C. (1994). Testing goodness of fit of polynomial models via spline smoothing techniques. Statist. Probab. Lett. 19 65–76.
• [5] Claeskens, G. and Van Keilegom, I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Ann. Statist. 31 1852–1884.
• [6] Cox, D., Koh, E., Wahba, G. and Yandell, B. S. (1988). Testing the (parametric) null model hypothesis in (semiparametric) partial and generalized spline models. Ann. Statist. 16 113–119.
• [7] Cox, D. D. and O’Sullivan, F. (1990). Asymptotic analysis of penalized likelihood and related estimators. Ann. Statist. 18 1676–1695.
• [8] Davis, P. J. (1963). Interpolation and Approximation. Blaisdell, New York.
• [9] Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29 153–193.
• [10] Fan, J. and Zhang, J. (2004). Sieve empirical likelihood ratio tests for nonparametric functions. Ann. Statist. 32 1858–1907.
• [11] Fan, J. and Zhang, W. (2000). Simultaneous confidence bands and hypothesis testing in varying-coefficient models. Scand. J. Stat. 27 715–731.
• [12] Genovese, C. and Wasserman, L. (2008). Adaptive confidence bands. Ann. Statist. 36 875–905.
• [13] Gu, C. (2002). Smoothing Spline ANOVA Models. Springer, New York.
• [14] Gu, C. and Qiu, C. (1993). Smoothing spline density estimation: Theory. Ann. Statist. 21 217–234.
• [15] Hall, P. (1991). Edgeworth expansions for nonparametric density estimators, with applications. Statistics 22 215–232.
• [16] Hall, P. (1992). Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Statist. 20 675–694.
• [17] Härdle, W. (1989). Asymptotic maximal deviation of $M$-smoothers. J. Multivariate Anal. 29 163–179.
• [18] Herrmann, E. (1997). Local bandwidth choice in kernel regression estimation. J. Comput. Graph. Statist. 6 35–54.
• [19] Ingster, Y. I. (1993). Asymptotically minimax hypothesis testing for nonparametric alternatives I–III. Math. Methods Statist. 2 85–114; 3 171–189; 4, 249–268.
• [20] Jayasuriya, B. R. (1996). Testing for polynomial regression using nonparametric regression techniques. J. Amer. Statist. Assoc. 91 1626–1631.
• [21] Ke, C. and Wang, Y. (2002). ASSIST: A suite of S-plus functions implementing spline smoothing techniques. Preprint.
• [22] Krivobokova, T., Kneib, T. and Claeskens, G. (2010). Simultaneous confidence bands for penalized spline estimators. J. Amer. Statist. Assoc. 105 852–863.
• [23] Liu, A. and Wang, Y. (2004). Hypothesis testing in smoothing spline models. J. Stat. Comput. Simul. 74 581–597.
• [24] Mammen, E. and van de Geer, S. (1997). Penalized quasi-likelihood estimation in partial linear models. Ann. Statist. 25 1014–1035.
• [25] McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman & Hall, London.
• [26] Messer, K. and Goldstein, L. (1993). A new class of kernels for nonparametric curve estimation. Ann. Statist. 21 179–195.
• [27] Neumann, M. H. and Polzehl, J. (1998). Simultaneous bootstrap confidence bands in nonparametric regression. J. Nonparametr. Stat. 9 307–333.
• [28] Nychka, D. (1988). Bayesian confidence intervals for smoothing splines. J. Amer. Statist. Assoc. 83 1134–1143.
• [29] Nychka, D. (1995). Splines as local smoothers. Ann. Statist. 23 1175–1197.
• [30] Ramil Novo, L. A. and González Manteiga, W. (2000). $F$ tests and regression analysis of variance based on smoothing spline estimators. Statist. Sinica 10 819–837.
• [31] Rice, J. and Rosenblatt, M. (1983). Smoothing splines: Regression, derivatives and deconvolution. Ann. Statist. 11 141–156.
• [32] Shang, Z. (2010). Convergence rate and Bahadur type representation of general smoothing spline M-estimates. Electron. J. Stat. 4 1411–1442.
• [33] Shang, Z. and Cheng, G. (2013). Supplement to “Local and global asymptotic inference in smoothing spline models.” DOI:10.1214/13-AOS1164SUPP.
• [34] Shao, J. (2003). Mathematical Statistics, 2nd ed. Springer, New York.
• [35] Silverman, B. W. (1984). Spline smoothing: The equivalent variable kernel method. Ann. Statist. 12 898–916.
• [36] Stone, M. H. (1926). A comparison of the series of Fourier and Birkhoff. Trans. Amer. Math. Soc. 28 695–761.
• [37] Sun, J., Loader, C. and McCormick, W. P. (2000). Confidence bands in generalized linear models. Ann. Statist. 28 429–460.
• [38] Sun, J. and Loader, C. R. (1994). Simultaneous confidence bands for linear regression and smoothing. Ann. Statist. 22 1328–1345.
• [39] Utreras, F. I. (1988). Boundary effects on convergence rates for Tikhonov regularization. J. Approx. Theory 54 235–249.
• [40] Wahba, G. (1983). Bayesian “confidence intervals” for the cross-validated smoothing spline. J. R. Stat. Soc. Ser. B Stat. Methodol. 45 133–150.
• [41] Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia, PA.
• [42] Wang, Y. (2011). Smoothing Splines: Methods and Applications. Monographs on Statistics and Applied Probability 121. Chapman & Hall/CRC Press, Boca Raton, FL.
• [43] Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models, and the Gauss–Newton method. Biometrika 61 439–447.
• [44] Zhang, W. and Peng, H. (2010). Simultaneous confidence band and hypothesis test in generalised varying-coefficient models. J. Multivariate Anal. 101 1656–1680.
• [45] Zhou, S., Shen, X. and Wolfe, D. A. (1998). Local asymptotics for regression splines and confidence regions. Ann. Statist. 26 1760–1782.

Supplemental materials

• Supplementary material: Supplement to “Local and global asymptotic inference in smoothing spline models”. The supplementary materials contain all the proofs of the theoretical results in the present paper.