Log in :: RSS/Atom Feeds
Bernoulli

Application of structural risk minimization to multivariate smoothing spline regression estimates

Michael Kohler, Adam Krzyzak, and Dominik Schäfer

Source: Bernoulli Volume 8, Number 4 (2002), 475-489.

Abstract

Estimation of regression functions from bounded, independent and identically distributed data is considered. Motivated by Vapnik's principle of structural risk minimization, a data-dependent choice of the smoothing parameter of multivariate smoothing spline estimates is proposed. The corresponding smoothing spline estimates automatically adapt to the unknown smoothness of the regression function and their L2 errors achieve the optimal rate of convergence up to a logarithmic factor. The result is valid without any regularity conditions on the distribution of the design.

Keywords: empirical process theory; rate of convergence; regression estimate; smoothing splines; structural risk minimization

Full-text: Open access

Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.bj/1078681380
Mathematical Reviews number (MathSciNet): MR2003e:62069
Zentralblatt MATH identifier: 1003.62035

References

[1] Baraud, Y. (1997) Model selection for regression on random design. Submitted.
[2] Barron, A.R., Birgé, L. and Massart, P. (1999) Risk bounds for model selection via penalization. Probab. Theory Related Fields, 113, 301-413.
Zentralblatt MATH: 0946.62036
Digital Object Identifier: doi:10.1007/s004400050210
[3] Devroye, L., Györfi, L. and Lugosi, G. (1996) A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag.
[4] Devroye, L.P. and Wagner, T.J. (1980) Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist., 8, 231-239.
Mathematical Reviews (MathSciNet): MR560725
Zentralblatt MATH: 0431.62025
Digital Object Identifier: doi:10.1214/aos/1176344949
Project Euclid: euclid.aos/1176344949
[5] Duchon, J. (1976) Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique, 10, 5-12.
[6] Eubank, R.L. (1988) Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker.
Mathematical Reviews (MathSciNet): MR934016
[7] Kohler, M. (1998) Nonparametric regression function estimation using interaction least squares splines and complexity regularization. Metrika, 47, 147-163. Abstract can also be found in the ISI/STMA publication
URL: Link to item
Mathematical Reviews (MathSciNet): MR1622144
Digital Object Identifier: doi:10.1007/BF02742869
[8] Kohler, M. (2000) Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J. Statist. Plann. Inference, 89, 1-23. Abstract can also be found in the ISI/STMA publication
URL: Link to item
Mathematical Reviews (MathSciNet): MR1794410
Zentralblatt MATH: 0982.62035
Digital Object Identifier: doi:10.1016/S0378-3758(99)00215-3
[9] Kohler, M. and Krzyzÿak, A. (2001) Nonparametric regression estimation using penalized least-squares. IEEE Trans. Inform. Theory, 47, 3054-3058. Abstract can also be found in the ISI/STMA publication
URL: Link to item
Digital Object Identifier: doi:10.1109/18.998089
[10] Krzyzÿak, A. and Linder, T. (1998) Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks, 9, 247-256.
Digital Object Identifier: doi:10.1109/72.661120
[11] Stone, C.J. (1977) Consistent nonparametric regression. Ann. Statist., 5, 595-645.
Mathematical Reviews (MathSciNet): MR443204
Zentralblatt MATH: 0366.62051
Digital Object Identifier: doi:10.1214/aos/1176343886
Project Euclid: euclid.aos/1176343886
[12] Stone, C.J. (1982) Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10, 1040-1053.
Mathematical Reviews (MathSciNet): MR673642
Zentralblatt MATH: 0511.62048
Digital Object Identifier: doi:10.1214/aos/1176345969
Project Euclid: euclid.aos/1176345969
[13] van de Geer, S. (1987) A new approach to least-squares estimation, with applications. Ann. Statist., 15, 587-602.
Mathematical Reviews (MathSciNet): MR888427
Zentralblatt MATH: 0625.62046
Digital Object Identifier: doi:10.1214/aos/1176350362
Project Euclid: euclid.aos/1176350362
[14] van de Geer, S. (1988) Regression Analysis and Empirical Processes, CWI Tract 45. Amsterdam: Centrum voor Wiskunde en Informatica.
Mathematical Reviews (MathSciNet): MR937256
Zentralblatt MATH: 0641.62041
[15] van de Geer, S. (1990) Estimating a regression function. Ann. Statist., 18, 907-924. Abstract can also be found in the ISI/STMA publication
URL: Link to item
Mathematical Reviews (MathSciNet): MR1056343
Zentralblatt MATH: 0709.62040
Digital Object Identifier: doi:10.1214/aos/1176347632
Project Euclid: euclid.aos/1176347632
[16] van de Geer, S. (2000) Applications of Empirical Process Theory. Cambridge: Cambridge University Press.
Mathematical Reviews (MathSciNet): MR1739079
Zentralblatt MATH: 0953.62049
[17] van de Geer, S. (2001) Least squares estimation with complexity penalties. Math. Meth. Statist., 10, 355-374.
Mathematical Reviews (MathSciNet): MR1867165
Zentralblatt MATH: 1005.62043
[18] Vapnik, V.N. (1998) Statistical Learning Theory. New York: Wiley.
Mathematical Reviews (MathSciNet): MR1641250
[19] Vapnik, V.N. and Chervonenkis, A.Y. (1974) The Theory of Pattern Recognition. Moscow: Nauka.
Mathematical Reviews (MathSciNet): MR474638
[20] Wahba, G. (1990) Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics.
Mathematical Reviews (MathSciNet): MR1045442
Zentralblatt MATH: 0813.62001

2010 © Bernoulli Society for Mathematical Statistics and Probability