Open Access
February 2008 Rodeo: Sparse, greedy nonparametric regression
John Lafferty, Larry Wasserman
Ann. Statist. 36(1): 28-63 (February 2008). DOI: 10.1214/009053607000000811

Abstract

We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method—called rodeo (regularization of derivative expectation operator)—conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that the rodeo applied to local linear smoothing avoids the curse of dimensionality, achieving near optimal minimax rates of convergence in the number of relevant variables, as if these variables were isolated in advance.

Citation

Download Citation

John Lafferty. Larry Wasserman. "Rodeo: Sparse, greedy nonparametric regression." Ann. Statist. 36 (1) 28 - 63, February 2008. https://doi.org/10.1214/009053607000000811

Information

Published: February 2008
First available in Project Euclid: 1 February 2008

zbMATH: 1132.62026
MathSciNet: MR2387963
Digital Object Identifier: 10.1214/009053607000000811

Subjects:
Primary: 62G08
Secondary: 62G20

Keywords: bandwidth estimation , local linear smoothing , minimax rates of convergence , Nonparametric regression , Sparsity , Variable selection

Rights: Copyright © 2008 Institute of Mathematical Statistics

Vol.36 • No. 1 • February 2008
Back to Top