Open Access
Translator Disclaimer
June 2013 A lasso for hierarchical interactions
Jacob Bien, Jonathan Taylor, Robert Tibshirani
Ann. Statist. 41(3): 1111-1141 (June 2013). DOI: 10.1214/13-AOS1096

Abstract

We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting “saved” by the hierarchy constraint.

We distinguish between parameter sparsity—the number of nonzero coefficients—and practical sparsity—the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.

Citation

Download Citation

Jacob Bien. Jonathan Taylor. Robert Tibshirani. "A lasso for hierarchical interactions." Ann. Statist. 41 (3) 1111 - 1141, June 2013. https://doi.org/10.1214/13-AOS1096

Information

Published: June 2013
First available in Project Euclid: 13 June 2013

zbMATH: 1292.62109
MathSciNet: MR3113805
Digital Object Identifier: 10.1214/13-AOS1096

Subjects:
Primary: 62J07

Keywords: convexity , hierarchical sparsity , interactions , Lasso , regularized regression

Rights: Copyright © 2013 Institute of Mathematical Statistics

JOURNAL ARTICLE
31 PAGES


SHARE
Vol.41 • No. 3 • June 2013
Back to Top