• Bernoulli
  • Volume 8, Number 4 (2002), 475-489.

Application of structural risk minimization to multivariate smoothing spline regression estimates

Michael Kohler, Adam Krzyzak, and Dominik Schäfer

Full-text: Open access


Estimation of regression functions from bounded, independent and identically distributed data is considered. Motivated by Vapnik's principle of structural risk minimization, a data-dependent choice of the smoothing parameter of multivariate smoothing spline estimates is proposed. The corresponding smoothing spline estimates automatically adapt to the unknown smoothness of the regression function and their $L^2$ errors achieve the optimal rate of convergence up to a logarithmic factor. The result is valid without any regularity conditions on the distribution of the design.

Article information

Bernoulli Volume 8, Number 4 (2002), 475-489.

First available in Project Euclid: 7 March 2004

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

empirical process theory rate of convergence regression estimate smoothing splines structural risk minimization


Kohler, Michael; Krzyzak, Adam; Schäfer, Dominik. Application of structural risk minimization to multivariate smoothing spline regression estimates. Bernoulli 8 (2002), no. 4, 475--489.

Export citation


  • [1] Baraud, Y. (1997) Model selection for regression on random design. Submitted.
  • [2] Barron, A.R., Birgé, L. and Massart, P. (1999) Risk bounds for model selection via penalization. Probab. Theory Related Fields, 113, 301-413.
  • [3] Devroye, L., Györfi, L. and Lugosi, G. (1996) A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag.
  • [4] Devroye, L.P. and Wagner, T.J. (1980) Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist., 8, 231-239.
  • [5] Duchon, J. (1976) Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique, 10, 5-12.
  • [6] Eubank, R.L. (1988) Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker.
  • [7] Kohler, M. (1998) Nonparametric regression function estimation using interaction least squares splines and complexity regularization. Metrika, 47, 147-163. Abstract can also be found in the ISI/STMA publication
  • [8] Kohler, M. (2000) Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J. Statist. Plann. Inference, 89, 1-23. Abstract can also be found in the ISI/STMA publication
  • [9] Kohler, M. and Krzyzÿak, A. (2001) Nonparametric regression estimation using penalized least-squares. IEEE Trans. Inform. Theory, 47, 3054-3058. Abstract can also be found in the ISI/STMA publication
  • [10] Krzyzÿak, A. and Linder, T. (1998) Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks, 9, 247-256.
  • [11] Stone, C.J. (1977) Consistent nonparametric regression. Ann. Statist., 5, 595-645.
  • [12] Stone, C.J. (1982) Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10, 1040-1053.
  • [13] van de Geer, S. (1987) A new approach to least-squares estimation, with applications. Ann. Statist., 15, 587-602.
  • [14] van de Geer, S. (1988) Regression Analysis and Empirical Processes, CWI Tract 45. Amsterdam: Centrum voor Wiskunde en Informatica.
  • [15] van de Geer, S. (1990) Estimating a regression function. Ann. Statist., 18, 907-924. Abstract can also be found in the ISI/STMA publication
  • [16] van de Geer, S. (2000) Applications of Empirical Process Theory. Cambridge: Cambridge University Press.
  • [17] van de Geer, S. (2001) Least squares estimation with complexity penalties. Math. Meth. Statist., 10, 355-374.
  • [18] Vapnik, V.N. (1998) Statistical Learning Theory. New York: Wiley.
  • [19] Vapnik, V.N. and Chervonenkis, A.Y. (1974) The Theory of Pattern Recognition. Moscow: Nauka.
  • [20] Wahba, G. (1990) Spline Models for Observational Data. Philadelphia: Society of Industrial and Applied Mathematics.