Electronic Journal of Statistics

Error bounds for the convex loss Lasso in linear models

Mark Hannay and Pierre-Yves Deléamont

Full-text: Open access


In this paper we investigate error bounds for convex loss functions for the Lasso in linear models, by first establishing a gap in the theory with respect to the existing error bounds. Then, under the compatibility condition, we recover bounds for the absolute value estimation error and the squared prediction error under mild conditions, which appear to be far more appropriate than the existing bounds for the convex loss Lasso. Interestingly, asymptotically the only difference between the new bounds of the convex loss Lasso and the classical Lasso is a term solely depending on a well-known expression in the robust statistics literature appearing multiplicatively in the bounds. We show that this result holds whether or not the scale parameter needs to be estimated jointly with the regression coefficients. Finally, we use the ratio to optimize our bounds in terms of minimaxity.

Article information

Electron. J. Statist. Volume 11, Number 2 (2017), 2832-2875.

Received: January 2017
First available in Project Euclid: 8 August 2017

Permanent link to this document

Digital Object Identifier

Primary: 62F35: Robustness and adaptive procedures
Secondary: 62J07: Ridge regression; shrinkage estimators

Robust Lasso high dimensions error bounds joint scale and location estimation

Creative Commons Attribution 4.0 International License.


Hannay, Mark; Deléamont, Pierre-Yves. Error bounds for the convex loss Lasso in linear models. Electron. J. Statist. 11 (2017), no. 2, 2832--2875. doi:10.1214/17-EJS1304. https://projecteuclid.org/euclid.ejs/1502157624

Export citation


  • [1] Bühlmann, P. and van de Geer, S. (2011)., Statistics for High-dimensional Data: Methods, Theory and Applications. Heidelberg: Springer.
  • [2] Fan, J., Fan, Y. and Barut, E. (2014). Adaptive robust variable selection., Ann. Statist. 42 324–351.
  • [3] Gao, X. and Huang, J. (2010). Asymptotic analysis of high-dimensional LAD regression with Lasso., Statist. Sinica 20 1485–1506.
  • [4] Greenshtein, E. (2006). Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint., Ann. Statist. 34 2367–2386.
  • [5] Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of overparametrization., Bernoulli 10 971–988.
  • [6] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables., J. Amer. Statist. Assoc. 58 13–30.
  • [7] Huber, P. J. (1964). Robust estimation of a location parameter., Ann. Math. Statist. 35 73–101.
  • [8] Huber, P. J. and Ronchetti, E. M. (2009)., Robust Statistics. New York: Wiley.
  • [9] Koltchinskii, V. (2011)., Oracle inequalities in empirical risk minimization and sparse recovery problems. Heidelberg: Springer.
  • [10] Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through the Huber’s criterion and adaptive lasso penalty., Electron. J. Stat. 5 1015–1053.
  • [11] Rosset, S. and Zhu, J. (2007). Piecewise linear regularized solution paths., Ann. Statist. 35 1012–1030.
  • [12] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso., J. Roy. Statist. Soc. Ser. B 58 267–288.
  • [13] Sun, T. and Zhang, C. H. (2012). Scaled sparse linear regression., Biometrika 99 879–898.
  • [14] van de Geer, S. (2007). The deterministic Lasso. In, JSM Proceedings, 2007 140. American Statistical Association.
  • [15] van de Geer, S. (2008). High-dimensional generalized linear models and the lasso., Ann. Statist. 36 614–645.
  • [16] van de Geer, S. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso., Electron. J. Stat. 3 1360–1392.
  • [17] Wang, H., Li, G. and Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the LAD-Lasso., J. Bus. Econom. Statist. 25 347–355.