Open Access
December 2007 Pathwise coordinate optimization
Jerome Friedman, Trevor Hastie, Holger Höfling, Robert Tibshirani
Ann. Appl. Stat. 1(2): 302-332 (December 2007). DOI: 10.1214/07-AOAS131

Abstract

We consider “one-at-a-time” coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the L1-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the “fused lasso,” however, so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance on some image smoothing problems.

Citation

Download Citation

Jerome Friedman. Trevor Hastie. Holger Höfling. Robert Tibshirani. "Pathwise coordinate optimization." Ann. Appl. Stat. 1 (2) 302 - 332, December 2007. https://doi.org/10.1214/07-AOAS131

Information

Published: December 2007
First available in Project Euclid: 30 November 2007

MathSciNet: MR2415737
zbMATH: 1378.90064
Digital Object Identifier: 10.1214/07-AOAS131

Keywords: Convex optimization , Coordinate descent , Lasso

Rights: Copyright © 2007 Institute of Mathematical Statistics

Vol.1 • No. 2 • December 2007
Back to Top