The Annals of Statistics
- Ann. Statist.
- Volume 35, Number 3 (2007), 1012-1030.
Piecewise linear regularized solution paths
We consider the generic regularized optimization problem β̂(λ)=arg minβ L(y, Xβ)+λJ(β). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407–499] have shown that for the LASSO—that is, if L is squared error loss and J(β)=‖β‖1 is the ℓ1 norm of β—the optimal coefficient path is piecewise linear, that is, ∂β̂(λ)/∂λ is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the LASSO for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen and van de Geer’s locally adaptive regression splines.
Ann. Statist., Volume 35, Number 3 (2007), 1012-1030.
First available in Project Euclid: 24 July 2007
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62F35: Robustness and adaptive procedures 62G08: Nonparametric regression 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]
Rosset, Saharon; Zhu, Ji. Piecewise linear regularized solution paths. Ann. Statist. 35 (2007), no. 3, 1012--1030. doi:10.1214/009053606000001370. https://projecteuclid.org/euclid.aos/1185303996