Preconditioning the Lasso for sign consistency

Abstract

Sign consistency of the Lasso requires the stringent irrepresentable condition. This paper examines whether preconditioning can circumvent this condition. Let $\mathbf{X}\in\mathbb{R}^{n\times p}$ and $Y\in\mathbb{R}^{n}$ satisfy the standard linear regression equation. Instead of computing the Lasso with $(\mathbf{X},Y)$, preconditioning first left multiplies by $F\in\mathbb{R}^{n\times n}$ and then computes the Lasso with $(F\mathbf{X},FY)$.

While others have proposed preconditioning for other purposes, we provide the first results that show $F\mathbf{X}$ can satisfy the irrepresentable condition even when $\mathbf{X}$ fails to satisfy the condition. Preconditioning the Lasso creates a new estimator that is sign consistent in a wider variety of settings. Importantly, left multiplying the regression equation by $F$ does not change $\beta$, the vector of unknown coefficients. However, left multiplying this equation by $F$ often inflates the variance of the errors. We propose a class of preconditioners to balance these costs and benefits.