Electronic Journal of Statistics

On the asymptotic properties of the group lasso estimator for linear models

Yuval Nardi and Alessandro Rinaldo

Full-text: Open access

Abstract

We establish estimation and model selection consistency, prediction and estimation bounds and persistence for the group-lasso estimator and model selector proposed by Yuan and Lin (2006) for least squares problems when the covariates have a natural grouping structure. We consider the case of a fixed-dimensional parameter space with increasing sample size and the double asymptotic scenario where the model complexity changes with the sample size.

Article information

Source
Electron. J. Statist., Volume 2 (2008), 605-633.

Dates
First available in Project Euclid: 30 July 2008

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1217450797

Digital Object Identifier
doi:10.1214/08-EJS200

Mathematical Reviews number (MathSciNet)
MR2426104

Zentralblatt MATH identifier
1320.62167

Subjects
Primary: 62J05: Linear regression
Secondary: 62F12: Asymptotic properties of estimators

Keywords
Least Squares Sparsity Group-Lasso Model Selection Oracle Inequalities Persistence

Citation

Nardi, Yuval; Rinaldo, Alessandro. On the asymptotic properties of the group lasso estimator for linear models. Electron. J. Statist. 2 (2008), 605--633. doi:10.1214/08-EJS200. https://projecteuclid.org/euclid.ejs/1217450797


Export citation

References

  • Bach, F. (2007). Consistency of the group Lasso and multiple kernel learning, to appear in, Journal of Machine Learning.
  • Bickel, P.J., Ritov, Y. and Tsybakov, A. B. (2007). A Simultaneous analysis of Lasso and Dantzig selector. Submitted to, Annals of Statistics.
  • Bunea, F., Tsybakov, A. B. and Wegkamp, M. H. (2007). Aggregation for Gaussian regression, The Annals of Statistics, 35(4), 1674–1697.
  • Bunea, F., Tsybakov, A. and Wegkamp, M. (2007b). Sparsity oracle inequalities for the lasso, Electronic Journal of Statistics, 1, 169–194.
  • Cavalier, L., Golubev, G. K., Picard, D. and Tsybakov, A. B. (2002). Oracle inequalities for inverse problems, Annals of Statistics, 30 843–874.
  • Dahinden, C., Parmiggiani, G., Emerick, M.C. and Bühlmann, P. (2006). Sparse Contingency Tables and High-Dimensional Log-Linealr Models for Alternative Splicing in Full-Length cDNA Libraries, Research Report 132, Swiss Federal Institute of, Technology.
  • Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression, Annals of Statistics, 32, 407–499.
  • Fan, J. and Li, R. (2001). Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96(456), 1348–1360.
  • Fan J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters, Annals of Statistics, 32(3), 928–961.
  • Geyer, C. (1994). On the asymptotics of constrained-M estimation, Annals of Statistics,, 22, 1993–2010.
  • Gilbert, A. C. and Strauss, M. J. (2006). Algorithms for Simultaneous Sparse Approximation Part II: Convex Relaxation, Signal Processing, 86, 572–588
  • Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional predictor selection and the virtue of overparametrization, Bernoulli, 10, 971–988.
  • Greenshtein, E. (2006). Best subset selection, persistence in high-dimensional statistical learning and optimization under, 1 constraint, Annals of Statistics, 34(5), 2367–2386.
  • Kim, Y., Kim, J. and Kim, Y. (2006). Blockwise sparse regression., Statistica Sinica, 16(2).
  • Knight, K. and Fu, W. (2000). Asymptotics for Lasso-type estimators, Annals of Statistics, 28(5), 1356–1378.
  • Koltchinskii, V. (2005). Sparsity in Penalized Empirical Risk Minimization, manuscript.
  • Ledoux, M. and Talagrand, M. (1991). Probability in Banach spaces: isoperimetry and processes., Springer-Verlag.
  • Lounici, K. (2008). Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators, Electronic Journal of Statistics, 2, 90–102.
  • Massart, P. (2007). Concentration Inequalities and Model Selection, Lecture Notes in Mathematics, Vol. 1896, Springer.
  • Meier, L., van der Geer, S. and Bühlmann, P. (2006). The Group Lasso for Logistic Regression, Journal of the Royal Statistical Society, Series B, 70(1), 53–7.
  • Meinshausen, N. and Bühlmann, P. (2006). High dimensional graphs and variable selection with the lasso, Annals of Statistics, 34(3), 1436–1462.
  • Meinshausen, N. and Yu, B. (2006). Lasso-type recovery of sparse representations for high-dimensional data, to appear in the, Annals of Statistics.
  • Nardi, N. and Rinaldo, A. (2007). The Log-linear Group-Lasso Estimator and Its Asymptotic Properties, manuscript.
  • Osborne, M.R., Presnell, B. and Turlach, B.A. (2000). On the LASSO and its Dual, Journal of Computational and Graphical Statistics, 9(2), 319–337.
  • Portnoy, S. (1988). Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity, Annals of Statistics, 16(1), 356–366.
  • Ravikumar, P., Lafferty, J., Liu, H. and Wasserman, L. (2007). Sparse Additive Models, manuscript.
  • Rinaldo (2006). Computing Maximum Likelihood Estimates in Log-Linear Models, Technical report, Department of Statistics, Carnegie Mellon, University.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58(1), 267–288.
  • van de Geer, S.A. (2007). Oracle inequalities and Regularization, in Lectures on Empirical Processes, European Mathematical, Society.
  • van der Vaart, A.W. and Wellner, J.A. (1998)., Weak Convergence and Empirical Processes, Springer.
  • Wainwright, M. J. (2006). Sharp thresholds for high-dimensional and noisy recovery of sparsity, Technical Report 708, Department of Statistics, UC, Berkeley.
  • Wainwright, M. J. (2007). Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting, Technical Report, UC Berkeley, Department of, Statistics.
  • Yuan, M. and Lin Y. (2006). Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society, B, 68(1), 49–67.
  • Yuan, M. and Lin Y. (2006). On the non-negative garrotte estimator, Journal of the Royal Statistical Society, B, 69(2), 143–161.
  • Zhang, T. (2007). Some Sharp Performance Bounds for Least Squares Regression with, L1 Regularization, manuscript.
  • Zhang, H. and Huang, J. (2007). The sparsity and bias of the Lasso selection in high-dimensional linear regression, to appear in the, Annals of Statistics.
  • Zhao, P., Rocha, G. and Yu, B. (2008). Grouped and hierarchical model selection through composite absolute penalties, manuscript.
  • Zhao, P. and Yu, B. (2006). On Model Selection Consistency of Lasso, Journal of Machine Learning Research, 7, 2541–2563.
  • Zhou, S., Lafferty, J. and Wasserman, L. (2007). Compressed Regression, manuscript.
  • Zhou, N. and Zhu, J. (2007). Group Variable Selection via Hierarchical Lasso and Its Oracle Property, manuscript.
  • Zou, H. (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101(476), 1418–1429.
  • H. Zou and T. Hastie. Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society, Series B, 67(2):301–320, 2005.