## Abstract

We consider the problem of recovering a *k*-sparse signal $\text{}{\mathit{\beta}}_{0}\in {\mathbb{R}}^{p}$ from noisy observations $\mathbf{y}=\mathbf{X}\text{}{\mathit{\beta}}_{0}+\mathbf{w}\in {\mathbb{R}}^{n}$. One of the most popular approaches is the ${l}_{1}$-regularized least squares, also known as LASSO. We analyze the mean square error of LASSO in the case of random designs in which each row of **X** is drawn from distribution $N(0,\text{}\mathrm{\Sigma})$ with general $\text{}\mathrm{\Sigma}$. We first derive the asymptotic risk of LASSO for $\mathbf{w}\ne 0$ in the limit of $n,p\to \infty $ with $n\u2215p\to \mathit{\delta}\in [0,\infty )$. We then examine conditions on *n*, *p*, and *k* for LASSO to exactly reconstruct $\text{}{\mathit{\beta}}_{0}$ in the noiseless case $\mathbf{w}=0$. A phase boundary ${\mathit{\delta}}_{c}=\mathit{\delta}(\mathit{\u03f5})$ is precisely established in the phase space defined by $0\le \mathit{\delta},\mathit{\u03f5}\le 1$, where $\mathit{\u03f5}=k\u2215p$. Above this boundary, LASSO perfectly recovers $\text{}{\mathit{\beta}}_{0}$ with high probability. Below this boundary, LASSO fails to recover $\text{}{\mathit{\beta}}_{0}$ with high probability. While the values of the non-zero elements of $\text{}{\mathit{\beta}}_{0}$ do not have any effect on the phase transition curve, our analysis shows that ${\mathit{\delta}}_{c}$ does depend on the signed pattern of the nonzero values of $\text{}{\mathit{\beta}}_{0}$ for general $\text{}\mathrm{\Sigma}\ne {\mathbf{I}}_{p\times p}$. This is in sharp contrast to the previous phase transition results derived in i.i.d. case with $\text{}\mathrm{\Sigma}={\mathbf{I}}_{p\times p}$ where ${\mathit{\delta}}_{c}$ is completely determined by *ϵ* regardless of the distribution of $\text{}{\mathit{\beta}}_{0}$. Underlying our formalism is a recently developed efficient algorithm called approximate message passing (AMP) algorithm. We generalize the state evolution of AMP from i.i.d. case to general case with $\text{}\mathrm{\Sigma}\ne {\mathbf{I}}_{p\times p}$. Extensive computational experiments confirm that our theoretical predictions are consistent with simulation results on moderate size system.

## Funding Statement

This research is supported in part by Division of Mathematical Sciences (National Science Foundation) Grant DMS-1916411.

## Acknowledgments

The author thanks the editor, associate editor, and two referees for many helpful comments and suggestions which led to a much improved presentation.

## Citation

Hanwen Huang. "LASSO risk and phase transition under dependence." Electron. J. Statist. 16 (2) 6512 - 6552, 2022. https://doi.org/10.1214/22-EJS2092

## Information