Open Access
Translator Disclaimer
February 2017 A lava attack on the recovery of sums of dense and sparse signals
Victor Chernozhukov, Christian Hansen, Yuan Liao
Ann. Statist. 45(1): 39-76 (February 2017). DOI: 10.1214/16-AOS1434

Abstract

Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of nonzero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small nonzero parameters. We consider a generalization of these two basic models, termed here a “sparse $+$ dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein’s unbiased estimator for lava’s prediction risk. A simulation example compares the performance of lava to lasso, ridge and elastic net in a regression example using data-dependent penalty parameters and illustrates lava’s improved performance relative to these benchmarks.

Citation

Download Citation

Victor Chernozhukov. Christian Hansen. Yuan Liao. "A lava attack on the recovery of sums of dense and sparse signals." Ann. Statist. 45 (1) 39 - 76, February 2017. https://doi.org/10.1214/16-AOS1434

Information

Received: 1 March 2015; Revised: 1 December 2015; Published: February 2017
First available in Project Euclid: 21 February 2017

zbMATH: 06710505
MathSciNet: MR3611486
Digital Object Identifier: 10.1214/16-AOS1434

Subjects:
Primary: 62J07
Secondary: 62J05

Keywords: high-dimensional models , nonsparse signal recovery , Penalization , shrinkage

Rights: Copyright © 2017 Institute of Mathematical Statistics

JOURNAL ARTICLE
38 PAGES


SHARE
Vol.45 • No. 1 • February 2017
Back to Top