Open Access
Translator Disclaimer
June 2010 Feature selection guided by structural information
Martin Slawski, Wolfgang zu Castell, Gerhard Tutz
Ann. Appl. Stat. 4(2): 1056-1080 (June 2010). DOI: 10.1214/09-AOAS302

Abstract

In generalized linear regression problems with an abundant number of features, lasso-type regularization which imposes an 1-constraint on the regression coefficients has become a widely established technique. Deficiencies of the lasso in certain scenarios, notably strongly correlated design, were unmasked when Zou and Hastie [J. Roy. Statist. Soc. Ser. B 67 (2005) 301–320] introduced the elastic net. In this paper we propose to extend the elastic net by admitting general nonnegative quadratic constraints as a second form of regularization. The generalized ridge-type constraint will typically make use of the known association structure of features, for example, by using temporal- or spatial closeness.

We study properties of the resulting “structured elastic net” regression estimation procedure, including basic asymptotics and the issue of model selection consistency. In this vein, we provide an analog to the so-called “irrepresentable condition” which holds for the lasso. Moreover, we outline algorithmic solutions for the structured elastic net within the generalized linear model family. The rationale and the performance of our approach is illustrated by means of simulated and real world data, with a focus on signal regression.

Citation

Download Citation

Martin Slawski. Wolfgang zu Castell. Gerhard Tutz. "Feature selection guided by structural information." Ann. Appl. Stat. 4 (2) 1056 - 1080, June 2010. https://doi.org/10.1214/09-AOAS302

Information

Published: June 2010
First available in Project Euclid: 3 August 2010

zbMATH: 1194.62092
MathSciNet: MR2758433
Digital Object Identifier: 10.1214/09-AOAS302

Keywords: Elastic net , generalized linear model , Lasso , Model selection , p ≫ n , Random fields , regularization , signal regression , Sparsity

Rights: Copyright © 2010 Institute of Mathematical Statistics

JOURNAL ARTICLE
25 PAGES


SHARE
Vol.4 • No. 2 • June 2010
Back to Top