Abstract
Sign consistency of the Lasso requires the stringent irrepresentable condition. This paper examines whether preconditioning can circumvent this condition. Let $\mathbf{X}\in\mathbb{R}^{n\times p}$ and $Y\in\mathbb{R}^{n}$ satisfy the standard linear regression equation. Instead of computing the Lasso with $(\mathbf{X},Y)$, preconditioning first left multiplies by $F\in\mathbb{R}^{n\times n}$ and then computes the Lasso with $(F\mathbf{X},FY)$.
While others have proposed preconditioning for other purposes, we provide the first results that show $F\mathbf{X}$ can satisfy the irrepresentable condition even when $\mathbf{X}$ fails to satisfy the condition. Preconditioning the Lasso creates a new estimator that is sign consistent in a wider variety of settings. Importantly, left multiplying the regression equation by $F$ does not change $\beta$, the vector of unknown coefficients. However, left multiplying this equation by $F$ often inflates the variance of the errors. We propose a class of preconditioners to balance these costs and benefits.
Citation
Jinzhu Jia. Karl Rohe. "Preconditioning the Lasso for sign consistency." Electron. J. Statist. 9 (1) 1150 - 1172, 2015. https://doi.org/10.1214/15-EJS1029
Information