De-biasing the lasso with degrees-of-freedom adjustment

Abstract

This paper studies schemes to de-bias the Lasso in sparse linear regression with Gaussian design where the goal is to estimate and construct confidence intervals for a low-dimensional projection of the unknown coefficient vector in a preconceived direction ${\mathit{a}}_0$. Our analysis reveals that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy nominal coverage and asymptotic efficiency in a full range of the level of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by the Lasso. The degrees-of-freedom adjustment (a) preserves the success of de-biasing methodologies in regimes where previous proposals were successful, and (b) repairs the nominal coverage and provides efficiency in regimes where previous proposals produce spurious inferences and provably fail to achieve the nominal coverage. Hence our theoretical and simulation results call for the implementation of this degrees-of-freedom adjustment in de-biasing methodologies.

Let ${\mathit{s}}_0$ denote the number of nonzero coefficients of the true coefficient vector and Σ the population Gram matrix. The unadjusted de-biasing scheme may fail to achieve the nominal coverage as soon as ${\mathit{s}}_0⋙{\mathit{n}}^{2/3}$ if Σ is known. If Σ is unknown, the degrees-of-freedom adjustment grants efficiency for the contrast in a general direction ${\mathit{a}}_0$ when

$$\frac{{\mathit{s}}_{0}log\mathit{p}}{\mathit{n}}+min\{\frac{{\mathit{s}}_{\mathrm{\Omega }}log\mathit{p}}{\mathit{n}},\frac{\Vert {\mathrm{\Sigma }}^{-1}{\mathit{a}}_0{\Vert }_1\sqrt{log\mathit{p}}}{\Vert {\mathrm{\Sigma }}^{-1/2}{\mathit{a}}_0{\Vert }_2\sqrt{\mathit{n}}}\}+\frac{min({\mathit{s}}_{\mathrm{\Omega }},{\mathit{s}}_0)log\mathit{p}}{\sqrt{\mathit{n}}}\to 0$$

where ${\mathit{s}}_{\mathrm{\Omega }}=\Vert {\mathrm{\Sigma }}^{-1}{\mathit{a}}_0{\Vert }_0$. The dependence in ${\mathit{s}}_0,{\mathit{s}}_{\mathrm{\Omega }}$ and $\Vert {\mathrm{\Sigma }}^{-1}{\mathit{a}}_0{\Vert }_1$ is optimal and closes a gap in previous upper and lower bounds. Our construction of the estimated score vector provides a novel methodology to handle dense directions ${\mathit{a}}_0$.

Beyond the degrees-of-freedom adjustment, our proof techniques yield a sharp ${\ell }_{\infty }$ error bound for the Lasso which is of independent interest.