Electronic Journal of Statistics

Error bounds for the convex loss Lasso in linear models

Mark Hannay and Pierre-Yves Deléamont

Full-text: Open access

Abstract

In this paper we investigate error bounds for convex loss functions for the Lasso in linear models, by first establishing a gap in the theory with respect to the existing error bounds. Then, under the compatibility condition, we recover bounds for the absolute value estimation error and the squared prediction error under mild conditions, which appear to be far more appropriate than the existing bounds for the convex loss Lasso. Interestingly, asymptotically the only difference between the new bounds of the convex loss Lasso and the classical Lasso is a term solely depending on a well-known expression in the robust statistics literature appearing multiplicatively in the bounds. We show that this result holds whether or not the scale parameter needs to be estimated jointly with the regression coefficients. Finally, we use the ratio to optimize our bounds in terms of minimaxity.

Article information

Source
Electron. J. Statist. Volume 11, Number 2 (2017), 2832-2875.

Dates
Received: January 2017
First available in Project Euclid: 8 August 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1502157624

Digital Object Identifier
doi:10.1214/17-EJS1304

Zentralblatt MATH identifier
1373.62369

Subjects
Primary: 62F35: Robustness and adaptive procedures
Secondary: 62J07: Ridge regression; shrinkage estimators

Keywords
Robust Lasso high dimensions error bounds joint scale and location estimation

Rights
Creative Commons Attribution 4.0 International License.

Citation

Hannay, Mark; Deléamont, Pierre-Yves. Error bounds for the convex loss Lasso in linear models. Electron. J. Statist. 11 (2017), no. 2, 2832--2875. doi:10.1214/17-EJS1304. https://projecteuclid.org/euclid.ejs/1502157624


Export citation

References

  • [1] Bühlmann, P. and van de Geer, S., (2011).Statistics for High-dimensional Data: Methods, Theory and Applications.Heidelberg: Springer.
  • [2] Fan, J., Fan, Y. and Barut, E. (2014). Adaptive robust variable, selection.Ann. Statist.42324–351.
  • [3] Gao, X. and Huang, J. (2010). Asymptotic analysis of high-dimensional LAD regression with, Lasso.Statist. Sinica201485–1506.
  • [4] Greenshtein, E. (2006). Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$, constraint.Ann. Statist.342367–2386.
  • [5] Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of, overparametrization.Bernoulli10971–988.
  • [6] Hoeffding, W. (1963). Probability inequalities for sums of bounded random, variables.J. Amer. Statist. Assoc.5813–30.
  • [7] Huber, P. J. (1964). Robust estimation of a location, parameter.Ann. Math. Statist.3573–101.
  • [8] Huber, P. J. and Ronchetti, E. M., (2009).Robust Statistics.New York: Wiley.
  • [9] Koltchinskii, V., (2011).Oracle inequalities in empirical risk minimization and sparse recovery problems.Heidelberg: Springer.
  • [10] Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through the Huber’s criterion and adaptive lasso, penalty.Electron. J. Stat.51015–1053.
  • [11] Rosset, S. and Zhu, J. (2007). Piecewise linear regularized solution, paths.Ann. Statist.351012–1030.
  • [12] Tibshirani, R. (1996). Regression shrinkage and selection via the, lasso.J. Roy. Statist. Soc. Ser. B58267–288.
  • [13] Sun, T. and Zhang, C. H. (2012). Scaled sparse linear, regression.Biometrika99879–898.
  • [14] van de Geer, S. (2007). The deterministic Lasso., InJSM Proceedings, 2007 140. American Statistical Association.
  • [15] van de Geer, S. (2008). High-dimensional generalized linear models and the, lasso.Ann. Statist.36614–645.
  • [16] van de Geer, S. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the, Lasso.Electron. J. Stat.31360–1392.
  • [17] Wang, H., Li, G. and Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the, LAD-Lasso.J. Bus. Econom. Statist.25347–355.