Abstract
Multivariate adaptive regression splines (MARS) is a popular method for nonparametric regression introduced by Friedman in 1991. MARS fits simple nonlinear and non-additive functions to regression data. We propose and study a natural lasso variant of the MARS method. Our method is based on least squares estimation over a convex class of functions obtained by considering infinite-dimensional linear combinations of functions in the MARS basis and imposing a variation based complexity constraint. Our estimator can be computed via finite-dimensional convex optimization, although it is defined as a solution to an infinite-dimensional optimization problem. Under a few standard design assumptions, we prove that our estimator achieves a rate of convergence that depends only logarithmically on dimension and thus avoids the usual curse of dimensionality to some extent. We also show that our method is naturally connected to nonparametric estimation techniques based on smoothness constraints. We implement our method with a cross-validation scheme for the selection of the involved tuning parameter and compare it to the usual MARS method in various simulation and real data settings.
Funding Statement
The first author was supported by NSF Grant DMS-2023505, NSF CAREER Grant DMS-1654589, and NSF Grant DMS-2210504.
The third author was supported by NSF CAREER Grant DMS-1654589 and NSF Grant DMS-2210504.
Acknowledgments
We are immensely grateful to the anonymous referees for their constructive comments and suggestions, which significantly improved the quality of the paper. We also thank Prof. Trevor Hastie for helpful comments and discussion.
Citation
Dohyeong Ki. Billy Fang. Adityanand Guntuboyina. "MARS via LASSO." Ann. Statist. 52 (3) 1102 - 1126, June 2024. https://doi.org/10.1214/24-AOS2384
Information