Translator Disclaimer
April 2021 Necessary and sufficient conditions for variable selection consistency of the LASSO in high dimensions
Soumendra N. Lahiri
Author Affiliations +
Ann. Statist. 49(2): 820-844 (April 2021). DOI: 10.1214/20-AOS1979

Abstract

This paper investigates conditions for variable selection consistency of the LASSO in high dimensional regression models and gives necessary and sufficient conditions for the same, potentially allowing the model dimension p to grow arbitrarily fast as a function of the sample size n. These conditions require both upper and lower bounds on the growth rate of the penalty parameter. It turns out that a variant of the irrepresentable Condition (IRC) of (J. Mach. Learn. Res. 7 (2006) 2541–2563), herein called the lower irrepresentable Condition (or LIRC), is determined by the lower bound considerations while the upper bound considerations lead to a new condition, called the upper irrepresentable Condition (or UIRC) in this paper. It is shown that the LIRC together with the UIRC is necessary and sufficient for the variable selection consistency of the LASSO, thereby settling a conjecture of (J. Mach. Learn. Res. 7 (2006) 2541–2563). Further, it is shown that under some mild regularity conditions, the penalty parameter must necessarily tend to infinity at a certain minimal rate to ensure variable selection consistency of the LASSO and that the corresponding LASSO estimators of the nonzero regression parameters cannot be n-consistent (even for individual parameters). Thus, under fairly general conditions, the LASSO with a single choice of the penalty parameter cannot achieve both variable selection consistency and n-consistency simultaneously.

Citation

Download Citation

Soumendra N. Lahiri. "Necessary and sufficient conditions for variable selection consistency of the LASSO in high dimensions." Ann. Statist. 49 (2) 820 - 844, April 2021. https://doi.org/10.1214/20-AOS1979

Information

Received: 1 June 2016; Revised: 1 May 2020; Published: April 2021
First available in Project Euclid: 2 April 2021

Digital Object Identifier: 10.1214/20-AOS1979

Subjects:
Primary: 62E20
Secondary: 62J05

Rights: Copyright © 2021 Institute of Mathematical Statistics

JOURNAL ARTICLE
25 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.49 • No. 2 • April 2021
Back to Top