Open Access
October 2006 Component selection and smoothing in multivariate nonparametric regression
Yi Lin, Hao Helen Zhang
Ann. Statist. 34(5): 2272-2297 (October 2006). DOI: 10.1214/009053606000000722

Abstract

We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The “COSSO” is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.

Citation

Download Citation

Yi Lin. Hao Helen Zhang. "Component selection and smoothing in multivariate nonparametric regression." Ann. Statist. 34 (5) 2272 - 2297, October 2006. https://doi.org/10.1214/009053606000000722

Information

Published: October 2006
First available in Project Euclid: 23 January 2007

zbMATH: 1106.62041
MathSciNet: MR2291500
Digital Object Identifier: 10.1214/009053606000000722

Subjects:
Primary: 62G05 , 62J07
Secondary: 62G20

Keywords: machine learning , method of regularization , Model selection , Nonparametric classification , Nonparametric regression , Smoothing spline ANOVA

Rights: Copyright © 2006 Institute of Mathematical Statistics

Vol.34 • No. 5 • October 2006
Back to Top