From finite sample to asymptotics: A geometric bridge for selection criteria in spline regression
S. C. Kou
Source: Ann. Statist. Volume 32, Number 6
(2004), 2444-2468.
Abstract
This paper studies, under the setting of spline regression, the connection between finite-sample properties of selection criteria and their asymptotic counterparts, focusing on bridging the gap between the two. We introduce a bias-variance decomposition of the prediction error, using which it is shown that in the asymptotics the bias term dominates the variability term, providing an explanation of the gap. A geometric exposition is provided for intuitive understanding. The theoretical and geometric results are illustrated through a numerical example.
First Page:
Show
Hide
Keywords: C_p; generalized maximum likelihood; extended exponential criterion; geometry; bias; variability; curvature
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aos/1107794875
Digital Object Identifier: doi:10.1214/009053604000000841
Mathematical Reviews number (MathSciNet): MR2153991
Zentralblatt MATH identifier: 1076.62039
References
Abramowitz, M. and Stegun, I. (1972). Handbook of Mathematical Functions, 10th printing. National Bureau of Standards, Washington.
Zentralblatt MATH: 0543.33001
Bowman, A. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations. Oxford Univ. Press, New York.
Zentralblatt MATH: 0889.62027
Demmler, A. and Reinsch, C. (1975). Oscillation matrices with spline smoothing. Numer. Math. 24 375--382.
Mathematical Reviews (MathSciNet): MR395161
Digital Object Identifier: doi:10.1007/BF01437406
Zentralblatt MATH: 0297.65002
Efron, B. (1975). Defining the curvature of a statistical problem (with application to second order efficiency) (with discussion). Ann. Statist. 3 1189--1242.
Mathematical Reviews (MathSciNet): MR428531
Efron, B. (2001). Selection criteria for scatterplot smoothers. Ann. Statist. 29 470--504.
Mathematical Reviews (MathSciNet): MR1863966
Digital Object Identifier: doi:10.1214/aos/1009210549
Project Euclid: euclid.aos/1009210549
Zentralblatt MATH: 1012.62040
Eubank, R. (1988). Spline Smoothing and Nonparametric Regression. Dekker, New York.
Mathematical Reviews (MathSciNet): MR934016
Zentralblatt MATH: 0702.62036
Fan, J. (2000). Prospects of nonparametric modeling. J. Amer. Statist. Assoc. 95 1296--1300.
Mathematical Reviews (MathSciNet): MR1825280
Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1383587
Zentralblatt MATH: 0873.62037
Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2, 2nd ed. Wiley, New York.
Zentralblatt MATH: 0219.60003
Green, P. and Silverman, B. (1994). Nonparametric Regression and Generalized Linear Models. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1270012
Zentralblatt MATH: 0832.62032
Gu, C. (1998). Model indexing and smoothing parameter selection in nonparametric function estimation (with discussion). Statist. Sinica 8 607--646.
Mathematical Reviews (MathSciNet): MR1651500
Hall, P. and Johnstone, I. (1992). Empirical functionals and efficient smoothing parameter selection (with discussion). J. Roy. Statist. Soc. Ser. B 54 475--530.
Mathematical Reviews (MathSciNet): MR1160479
Härdle, W. (1990). Applied Nonparametric Regression. Cambridge Univ. Press.
Mathematical Reviews (MathSciNet): MR1161622
Härdle, W., Hall, P. and Marron, J. (1988). How far are automatically chosen regression smoothing parameters from their optimum (with discussion)? J. Amer. Statist. Assoc. 83 86--101.
Mathematical Reviews (MathSciNet): MR941001
Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1082147
Zentralblatt MATH: 0747.62061
Hurvich, C., Simonoff, J. and Tsai, C. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 271--294.
Mathematical Reviews (MathSciNet): MR1616041
Digital Object Identifier: doi:10.1111/1467-9868.00125
Zentralblatt MATH: 0909.62039
Jones, M., Marron, J. and Sheather, S. (1996). A brief survey of bandwidth selection for density estimation. J. Amer. Statist. Assoc. 91 401--407.
Mathematical Reviews (MathSciNet): MR1394097
Kohn, R., Ansley, C. and Tharm, D. (1991). The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Amer. Statist. Assoc. 86 1042--1050.
Mathematical Reviews (MathSciNet): MR1146351
Kou, S. C. (2003). On the efficiency of selection criteria in spline regression. Probab. Theory Related Fields 127 153--176.
Mathematical Reviews (MathSciNet): MR2013979
Digital Object Identifier: doi:10.1007/s00440-003-0277-z
Zentralblatt MATH: 1027.62021
Kou, S. C. and Efron, B. (2002). Smoothers and the $C_p$, generalized maximum likelihood and extended exponential criteria: A geometric approach. J. Amer. Statist. Assoc. 97 766--782.
Mathematical Reviews (MathSciNet): MR1941408
Digital Object Identifier: doi:10.1198/016214502388618582
Zentralblatt MATH: 1048.62044
Li, K.-C. (1986). Asymptotic optimality of $C_L$ and generalized cross-validation in ridge regression with application to spline smoothing. Ann. Statist. 14 1101--1112.
Mathematical Reviews (MathSciNet): MR856808
JSTOR: links.jstor.org
Li, K.-C. (1987). Asymptotic optimality for $C_p$, $C_L$, cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15 958--975.
Mathematical Reviews (MathSciNet): MR902239
JSTOR: links.jstor.org
Rosenblatt, M. (1991). Stochastic Curve Estimation. IMS, Hayward, CA.
Zentralblatt MATH: 1163.62318
Silverman, B. (1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with discussion). J. Roy. Statist. Soc. Ser. B 47 1--52.
Mathematical Reviews (MathSciNet): MR805063
Simonoff, J. (1996). Smoothing Methods in Statistics. Springer, New York.
Mathematical Reviews (MathSciNet): MR1391963
Zentralblatt MATH: 0859.62035
Speckman, P. (1983). Efficient nonparametric regression with cross-validated smoothing splines. Unpublished manuscript.
Speckman, P. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression models. Ann. Statist. 13 970--983.
Mathematical Reviews (MathSciNet): MR803752
JSTOR: links.jstor.org
Speckman, P. and Sun, D. (2001). Asymptotic properties of smoothing parameter selection in spline regression. Preprint.
Stein, M. (1990). A comparison of generalized cross validation and modified maximum likelihood for estimating the parameters of a stochastic process. Ann. Statist. 18 1139--1157.
Mathematical Reviews (MathSciNet): MR1062702
JSTOR: links.jstor.org
Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Statist. 13 1378--1402.
Mathematical Reviews (MathSciNet): MR811498
JSTOR: links.jstor.org
Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
Mathematical Reviews (MathSciNet): MR1045442
Zentralblatt MATH: 0813.62001
Wecker, W. and Ansley, C. (1983). The signal extraction approach to nonlinear regression and spline smoothing. J. Amer. Statist. Assoc. 78 81--89.
Mathematical Reviews (MathSciNet): MR696851
