The Annals of Applied Statistics
- Ann. Appl. Stat.
- Volume 4, Number 2 (2010), 1056-1080.
Feature selection guided by structural information
In generalized linear regression problems with an abundant number of features, lasso-type regularization which imposes an ℓ1-constraint on the regression coefficients has become a widely established technique. Deficiencies of the lasso in certain scenarios, notably strongly correlated design, were unmasked when Zou and Hastie [J. Roy. Statist. Soc. Ser. B 67 (2005) 301–320] introduced the elastic net. In this paper we propose to extend the elastic net by admitting general nonnegative quadratic constraints as a second form of regularization. The generalized ridge-type constraint will typically make use of the known association structure of features, for example, by using temporal- or spatial closeness.
We study properties of the resulting “structured elastic net” regression estimation procedure, including basic asymptotics and the issue of model selection consistency. In this vein, we provide an analog to the so-called “irrepresentable condition” which holds for the lasso. Moreover, we outline algorithmic solutions for the structured elastic net within the generalized linear model family. The rationale and the performance of our approach is illustrated by means of simulated and real world data, with a focus on signal regression.
Ann. Appl. Stat. Volume 4, Number 2 (2010), 1056-1080.
First available in Project Euclid: 3 August 2010
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Slawski, Martin; zu Castell, Wolfgang; Tutz, Gerhard. Feature selection guided by structural information. Ann. Appl. Stat. 4 (2010), no. 2, 1056--1080. doi:10.1214/09-AOAS302. https://projecteuclid.org/euclid.aoas/1280842152.
- Supplementary material: Supplement to “Feature Selection guided by Structural Information”. The supplement contains proof of all statements of the main article.