Open Access
Translator Disclaimer
October 2013 Calibrating nonconvex penalized regression in ultra-high dimension
Lan Wang, Yongdai Kim, Runze Li
Ann. Statist. 41(5): 2505-2536 (October 2013). DOI: 10.1214/13-AOS1159

Abstract

We investigate high-dimensional nonconvex penalized regression, where the number of covariates may grow at an exponential rate. Although recent asymptotic theory established that there exists a local minimum possessing the oracle property under general conditions, it is still largely an open problem how to identify the oracle estimator among potentially multiple local minima. There are two main obstacles: (1) due to the presence of multiple minima, the solution path is nonunique and is not guaranteed to contain the oracle estimator; (2) even if a solution path is known to contain the oracle estimator, the optimal tuning parameter depends on many unknown factors and is hard to estimate. To address these two challenging issues, we first prove that an easy-to-calculate calibrated CCCP algorithm produces a consistent solution path which contains the oracle estimator with probability approaching one. Furthermore, we propose a high-dimensional BIC criterion and show that it can be applied to the solution path to select the optimal tuning parameter which asymptotically identifies the oracle estimator. The theory for a general class of nonconvex penalties in the ultra-high dimensional setup is established when the random errors follow the sub-Gaussian distribution. Monte Carlo studies confirm that the calibrated CCCP algorithm combined with the proposed high-dimensional BIC has desirable performance in identifying the underlying sparsity pattern for high-dimensional data analysis.

Citation

Download Citation

Lan Wang. Yongdai Kim. Runze Li. "Calibrating nonconvex penalized regression in ultra-high dimension." Ann. Statist. 41 (5) 2505 - 2536, October 2013. https://doi.org/10.1214/13-AOS1159

Information

Published: October 2013
First available in Project Euclid: 5 November 2013

zbMATH: 1281.62106
MathSciNet: MR3127873
Digital Object Identifier: 10.1214/13-AOS1159

Subjects:
Primary: 62J05
Secondary: 62J07

Keywords: high-dimensional regression , Lasso , MCP , penalized least squares , SCAD , Variable selection

Rights: Copyright © 2013 Institute of Mathematical Statistics

JOURNAL ARTICLE
32 PAGES


SHARE
Vol.41 • No. 5 • October 2013
Back to Top