Abstract
Variable selection properties of procedures utilizing penalized-likelihood estimates is a central topic in the study of high-dimensional linear regression problems. Existing literature emphasizes the quality of ranking of the variables by such procedures as reflected in the receiver operating characteristic curve or in prediction performance. Specifically, recent works have harnessed modern theory of approximate message-passing (AMP) to obtain, in a particular setting, exact asymptotic predictions of the type I, type II error tradeoff for selection procedures that rely on -regularized estimators.
In practice, effective ranking by itself is often not sufficient because some calibration for Type I error is required. In this work, we study theoretically the power of selection procedures that similarly rank the features by the size of an -regularized estimator, but further use Model-X knockoffs to control the false discovery rate in the realistic situation where no prior information about the signal is available. In analyzing the power of the resulting procedure, we extend existing results in AMP theory to handle the pairing between original variables and their knockoffs. This is used to derive exact asymptotic predictions for power. We apply the general results to compare the power of the knockoffs versions of Lasso and thresholded-Lasso selection, and demonstrate that in the i.i.d. covariate setting under consideration, tuning by cross-validation on the augmented design matrix is nearly optimal. We further demonstrate how the techniques allow to analyze also the Type S error, and a corresponding notion of power, when selections are supplemented with a decision on the sign of the coefficient.
Funding Statement
A. W. is partially supported by ISF via grant 039-9325.
W. J. S. is partially supported by NSF via grant CCF-1934876, and by the Wharton Dean’s Research Fund.
M. B. is supported by the Polish National Center of Science via grant 2016/23/B/ST1/00454.
R. F. B. is supported by NSF via grant DMS-1654076, and by the Office of Naval Research via grant N00014-20-1-2337.
E. C. is partially supported by NSF via grants DMS-1712800 and DMS-1934578.
Citation
Asaf Weinstein. Weijie J. Su. Małgorzata Bogdan. Rina Foygel Barber. Emmanuel J. Candès. "A power analysis for model-X knockoffs with -regularized statistics." Ann. Statist. 51 (3) 1005 - 1029, June 2023. https://doi.org/10.1214/23-AOS2274
Information