Translator Disclaimer
2013 Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression
Hanzhong Liu, Bin Yu
Electron. J. Statist. 7: 3124-3169 (2013). DOI: 10.1214/14-EJS875

Abstract

We study the asymptotic properties of Lasso+mLS and Lasso+ Ridge under the sparse high-dimensional linear regression model: Lasso selecting predictors and then modified Least Squares (mLS) or Ridge estimating their coefficients. First, we propose a valid inference procedure for parameter estimation based on parametric residual bootstrap after Lasso+ mLS and Lasso+Ridge. Second, we derive the asymptotic unbiasedness of Lasso+mLS and Lasso+Ridge. More specifically, we show that their biases decay at an exponential rate and they can achieve the oracle convergence rate of $s/n$ (where $s$ is the number of nonzero regression coefficients and $n$ is the sample size) for mean squared error (MSE). Third, we show that Lasso+mLS and Lasso+Ridge are asymptotically normal. They have an oracle property in the sense that they can select the true predictors with probability converging to $1$ and the estimates of nonzero parameters have the same asymptotic normal distribution that they would have if the zero parameters were known in advance. In fact, our analysis is not limited to adopting Lasso in the selection stage, but is applicable to any other model selection criteria with exponentially decay rates of the probability of selecting wrong models.

Citation

Download Citation

Hanzhong Liu. Bin Yu. "Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression." Electron. J. Statist. 7 3124 - 3169, 2013. https://doi.org/10.1214/14-EJS875

Information

Published: 2013
First available in Project Euclid: 15 January 2014

zbMATH: 1281.62158
MathSciNet: MR3151764
Digital Object Identifier: 10.1214/14-EJS875

Subjects:
Primary: 62F12, 62F40
Secondary: 62J07

Rights: Copyright © 2013 The Institute of Mathematical Statistics and the Bernoulli Society

JOURNAL ARTICLE
46 PAGES


SHARE
Back to Top