The Annals of Applied Statistics

Pathwise coordinate optimization

Jerome Friedman, Trevor Hastie, Holger Höfling, and Robert Tibshirani

Full-text: Open access

Abstract

We consider “one-at-a-time” coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the L1-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the “fused lasso,” however, so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance on some image smoothing problems.

Article information

Source
Ann. Appl. Stat. Volume 1, Number 2 (2007), 302-332.

Dates
First available in Project Euclid: 30 November 2007

Permanent link to this document
http://projecteuclid.org/euclid.aoas/1196438020

Digital Object Identifier
doi:10.1214/07-AOAS131

Zentralblatt MATH identifier
05226935

Mathematical Reviews number (MathSciNet)
MR2415737

Citation

Friedman, Jerome; Hastie, Trevor; Höfling, Holger; Tibshirani, Robert. Pathwise coordinate optimization. Ann. Appl. Stat. 1 (2007), no. 2, 302--332. doi:10.1214/07-AOAS131. http://projecteuclid.org/euclid.aoas/1196438020.


Export citation