## Bernoulli

• Bernoulli
• Volume 8, Number 4 (2002), 475-489.

### Application of structural risk minimization to multivariate smoothing spline regression estimates

#### Abstract

Estimation of regression functions from bounded, independent and identically distributed data is considered. Motivated by Vapnik's principle of structural risk minimization, a data-dependent choice of the smoothing parameter of multivariate smoothing spline estimates is proposed. The corresponding smoothing spline estimates automatically adapt to the unknown smoothness of the regression function and their $L^2$ errors achieve the optimal rate of convergence up to a logarithmic factor. The result is valid without any regularity conditions on the distribution of the design.

#### Article information

Source
Bernoulli, Volume 8, Number 4 (2002), 475-489.

Dates
First available in Project Euclid: 7 March 2004

https://projecteuclid.org/euclid.bj/1078681380

Mathematical Reviews number (MathSciNet)
MR2003e:62069

Zentralblatt MATH identifier
1003.62035

#### Citation

Kohler, Michael; Krzyzak, Adam; Schäfer, Dominik. Application of structural risk minimization to multivariate smoothing spline regression estimates. Bernoulli 8 (2002), no. 4, 475--489. https://projecteuclid.org/euclid.bj/1078681380

#### References

• [1] Baraud, Y. (1997) Model selection for regression on random design. Submitted.
• [2] Barron, A.R., Birgé, L. and Massart, P. (1999) Risk bounds for model selection via penalization. Probab. Theory Related Fields, 113, 301-413.
• [3] Devroye, L., Györfi, L. and Lugosi, G. (1996) A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag.
• [4] Devroye, L.P. and Wagner, T.J. (1980) Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist., 8, 231-239.
• [5] Duchon, J. (1976) Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique, 10, 5-12.
• [6] Eubank, R.L. (1988) Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker.
• [7] Kohler, M. (1998) Nonparametric regression function estimation using interaction least squares splines and complexity regularization. Metrika, 47, 147-163. Abstract can also be found in the ISI/STMA publication
• [8] Kohler, M. (2000) Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J. Statist. Plann. Inference, 89, 1-23. Abstract can also be found in the ISI/STMA publication
• [9] Kohler, M. and Krzyzÿak, A. (2001) Nonparametric regression estimation using penalized least-squares. IEEE Trans. Inform. Theory, 47, 3054-3058. Abstract can also be found in the ISI/STMA publication
• [10] Krzyzÿak, A. and Linder, T. (1998) Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks, 9, 247-256.
• [11] Stone, C.J. (1977) Consistent nonparametric regression. Ann. Statist., 5, 595-645.
• [12] Stone, C.J. (1982) Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10, 1040-1053.
• [13] van de Geer, S. (1987) A new approach to least-squares estimation, with applications. Ann. Statist., 15, 587-602.
• [14] van de Geer, S. (1988) Regression Analysis and Empirical Processes, CWI Tract 45. Amsterdam: Centrum voor Wiskunde en Informatica.
• [15] van de Geer, S. (1990) Estimating a regression function. Ann. Statist., 18, 907-924. Abstract can also be found in the ISI/STMA publication
• [16] van de Geer, S. (2000) Applications of Empirical Process Theory. Cambridge: Cambridge University Press.
• [17] van de Geer, S. (2001) Least squares estimation with complexity penalties. Math. Meth. Statist., 10, 355-374.
• [18] Vapnik, V.N. (1998) Statistical Learning Theory. New York: Wiley.
• [19] Vapnik, V.N. and Chervonenkis, A.Y. (1974) The Theory of Pattern Recognition. Moscow: Nauka.
• [20] Wahba, G. (1990) Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics.