Variable Selection in Nonparametric Regression with Continuous Covariates

Abstract

In a nonparametric regression setup where the covariates are continuous, the problem of estimating the number of covariates will be discussed in this paper. The kernel method is used to estimate the regression function and the selection criterion is based on minimizing the cross-validation estimate of the mean squared prediction error. We consider choosing both the bandwidth and the number of covariates based on the data. Unlike the case of linear regression, it turns out that the selection is consistent and efficient even when the true model has only a finite number of covariates. In addition, we also observe the curse of dimensionality at work.