Estimation of regression functions from bounded, independent and identically distributed data is considered. Motivated by Vapnik's principle of structural risk minimization, a data-dependent choice of the smoothing parameter of multivariate smoothing spline estimates is proposed. The corresponding smoothing spline estimates automatically adapt to the unknown smoothness of the regression function and their L2 errors achieve the optimal rate of convergence up to a logarithmic factor. The result is valid without any regularity conditions on the distribution of the design.
References
[1] Baraud, Y. (1997) Model selection for regression on random design. Submitted.
[2] Barron, A.R., Birgé, L. and Massart, P. (1999) Risk bounds for model selection via penalization. Probab. Theory Related Fields, 113, 301-413.
[3] Devroye, L., Györfi, L. and Lugosi, G. (1996) A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag.
[4] Devroye, L.P. and Wagner, T.J. (1980) Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist., 8, 231-239.
[5] Duchon, J. (1976) Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique, 10, 5-12.
[6] Eubank, R.L. (1988) Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker.
[7] Kohler, M. (1998) Nonparametric regression function estimation using interaction least squares splines and complexity regularization. Metrika, 47, 147-163. Abstract can also be found in the ISI/STMA publication
[8] Kohler, M. (2000) Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J. Statist. Plann. Inference, 89, 1-23. Abstract can also be found in the ISI/STMA publication
[9] Kohler, M. and Krzyzÿak, A. (2001) Nonparametric regression estimation using penalized least-squares. IEEE Trans. Inform. Theory, 47, 3054-3058. Abstract can also be found in the ISI/STMA publication
[10] Krzyzÿak, A. and Linder, T. (1998) Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks, 9, 247-256.
[11] Stone, C.J. (1977) Consistent nonparametric regression. Ann. Statist., 5, 595-645.
[12] Stone, C.J. (1982) Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10, 1040-1053.
[13] van de Geer, S. (1987) A new approach to least-squares estimation, with applications. Ann. Statist., 15, 587-602.
[14] van de Geer, S. (1988) Regression Analysis and Empirical Processes, CWI Tract 45. Amsterdam: Centrum voor Wiskunde en Informatica.
[15] van de Geer, S. (1990) Estimating a regression function. Ann. Statist., 18, 907-924. Abstract can also be found in the ISI/STMA publication
[16] van de Geer, S. (2000) Applications of Empirical Process Theory. Cambridge: Cambridge University Press.
[17] van de Geer, S. (2001) Least squares estimation with complexity penalties. Math. Meth. Statist., 10, 355-374.
[18] Vapnik, V.N. (1998) Statistical Learning Theory. New York: Wiley.
[19] Vapnik, V.N. and Chervonenkis, A.Y. (1974) The Theory of Pattern Recognition. Moscow: Nauka.
[20] Wahba, G. (1990) Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics.
