Electronic Journal of Statistics

Forward stagewise regression and the monotone lasso

Trevor Hastie, Jonathan Taylor, Robert Tibshirani, and Guenther Walther

Full-text: Open access

Abstract

We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron, Hastie, Johnstone & Tibshirani (2004) it is proved that the least angle regression algorithm, with a small modification, solves the lasso regression problem. Here we give an analogous result for incremental forward stagewise regression, showing that it solves a version of the lasso problem that enforces monotonicity. One consequence of this is as follows: while lasso makes optimal progress in terms of reducing the residual sum-of-squares per unit increase in L1-norm of the coefficient β, forward stage-wise is optimal per unit L1 arc-length traveled along the coefficient path. We also study a condition under which the coefficient paths of the lasso are monotone, and hence the different algorithms coincide. Finally, we compare the lasso and forward stagewise procedures in a simulation study involving a large number of correlated predictors.

Article information

Source
Electron. J. Statist., Volume 1 (2007), 1-29.

Dates
First available in Project Euclid: 27 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1177687773

Digital Object Identifier
doi:10.1214/07-EJS004

Mathematical Reviews number (MathSciNet)
MR2312144

Zentralblatt MATH identifier
1306.62176

Subjects
Primary: 62J99: None of the above, but in this section
Secondary: 62J07: Ridge regression; shrinkage estimators

Keywords
regression lasso stagewise

Citation

Hastie, Trevor; Taylor, Jonathan; Tibshirani, Robert; Walther, Guenther. Forward stagewise regression and the monotone lasso. Electron. J. Statist. 1 (2007), 1--29. doi:10.1214/07-EJS004. https://projecteuclid.org/euclid.ejs/1177687773


Export citation