## The Annals of Statistics

- Ann. Statist.
- Volume 45, Number 1 (2017), 39-76.

### A lava attack on the recovery of sums of dense and sparse signals

Victor Chernozhukov, Christian Hansen, and Yuan Liao

#### Abstract

Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of nonzero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small nonzero parameters. We consider a generalization of these two basic models, termed here a “sparse $+$ dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein’s unbiased estimator for lava’s prediction risk. A simulation example compares the performance of lava to lasso, ridge and elastic net in a regression example using data-dependent penalty parameters and illustrates lava’s improved performance relative to these benchmarks.

#### Article information

**Source**

Ann. Statist. Volume 45, Number 1 (2017), 39-76.

**Dates**

Received: March 2015

Revised: December 2015

First available in Project Euclid: 21 February 2017

**Permanent link to this document**

https://projecteuclid.org/euclid.aos/1487667617

**Digital Object Identifier**

doi:10.1214/16-AOS1434

**Zentralblatt MATH identifier**

06710505

**Subjects**

Primary: 62J07: Ridge regression; shrinkage estimators

Secondary: 62J05: Linear regression

**Keywords**

High-dimensional models penalization shrinkage nonsparse signal recovery

#### Citation

Chernozhukov, Victor; Hansen, Christian; Liao, Yuan. A lava attack on the recovery of sums of dense and sparse signals. Ann. Statist. 45 (2017), no. 1, 39--76. doi:10.1214/16-AOS1434. https://projecteuclid.org/euclid.aos/1487667617

#### Supplemental materials

- Supplement to “A lava attack on the recovery of sums of dense and sparse signals”. This supplementary material contains additional simulation results and omitted proofs.Digital Object Identifier: doi:10.1214/16-AOS1434SUPP