Open Access
April 2014 Pivotal estimation via square-root Lasso in nonparametric regression
Alexandre Belloni, Victor Chernozhukov, Lie Wang
Ann. Statist. 42(2): 757-788 (April 2014). DOI: 10.1214/14-AOS1204


We propose a self-tuning $\sqrt{\mathrm {Lasso}} $ method that simultaneously resolves three important practical problems in high-dimensional regression analysis, namely it handles the unknown scale, heteroscedasticity and (drastic) non-Gaussianity of the noise. In addition, our analysis allows for badly behaved designs, for example, perfectly collinear regressors, and generates sharp bounds even in extreme cases, such as the infinite variance case and the noiseless case, in contrast to Lasso. We establish various nonasymptotic bounds for $\sqrt{\mathrm {Lasso}} $ including prediction norm rate and sparsity. Our analysis is based on new impact factors that are tailored for bounding prediction norm. In order to cover heteroscedastic non-Gaussian noise, we rely on moderate deviation theory for self-normalized sums to achieve Gaussian-like results under weak conditions. Moreover, we derive bounds on the performance of ordinary least square (ols) applied to the model selected by $\sqrt{\mathrm {Lasso}} $ accounting for possible misspecification of the selected model. Under mild conditions, the rate of convergence of ols post $\sqrt{\mathrm {Lasso}} $ is as good as $\sqrt{\mathrm {Lasso}} $’s rate. As an application, we consider the use of $\sqrt{\mathrm {Lasso}} $ and ols post $\sqrt{\mathrm {Lasso}} $ as estimators of nuisance parameters in a generic semiparametric problem (nonlinear moment condition or $Z$-problem), resulting in a construction of $\sqrt{n}$-consistent and asymptotically normal estimators of the main parameters.


Download Citation

Alexandre Belloni. Victor Chernozhukov. Lie Wang. "Pivotal estimation via square-root Lasso in nonparametric regression." Ann. Statist. 42 (2) 757 - 788, April 2014.


Published: April 2014
First available in Project Euclid: 20 May 2014

zbMATH: 1321.62030
MathSciNet: MR3210986
Digital Object Identifier: 10.1214/14-AOS1204

Primary: 62G05 , 62G08
Secondary: 62G35

Keywords: $\sqrt{n}$-consistency and asymptotic normality after model selection , $Z$-estimation problem , generic semiparametric problem , Model selection , non-Gaussian heteroscedastic , nonlinear instrumental variable , Pivotal , square-root Lasso

Rights: Copyright © 2014 Institute of Mathematical Statistics

Vol.42 • No. 2 • April 2014
Back to Top