The Annals of Statistics

Asymptotics for lasso-type estimators

Wenjiang Fu and Keith Knight

Full-text: Open access

Abstract

We consider the asymptotic behavior ofregression estimators that minimize the residual sum of squares plus a penalty proportional to $\sum|\beta_j|^{\gamma}$. for some $\gamma > 0$. These estimators include the Lasso as a special case when $\gamma = 1$. Under appropriate conditions, we show that the limiting distributions can have positive probability mass at 0 when the true value of the parameter is 0.We also consider asymptotics for “nearly singular” designs.

Article information

Source
Ann. Statist., Volume 28, Number 5 (2000), 1356-1378.

Dates
First available in Project Euclid: 12 March 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1015957397

Digital Object Identifier
doi:10.1214/aos/1015957397

Mathematical Reviews number (MathSciNet)
MR1805787

Zentralblatt MATH identifier
1105.62357

Subjects
Primary: 62J05: Linear regression 62J07: Ridge regression; shrinkage estimators
Secondary: 62E20: Asymptotic distribution theory 60F05: Central limit and other weak theorems

Keywords
Penalized regression Lasso shrinkage estimation epi-convergence in distribution

Citation

Knight, Keith; Fu, Wenjiang. Asymptotics for lasso-type estimators. Ann. Statist. 28 (2000), no. 5, 1356--1378. doi:10.1214/aos/1015957397. https://projecteuclid.org/euclid.aos/1015957397


Export citation

References

  • Anderson, P. K. and Gill, R. D. (1982). Cox's regression model for counting processes: a large sample study. Ann. Statist. 10 1100-1120.
  • Besag, J. (1986). On the statistical analysis ofdirty pictures (with discussion). J. Roy. Statist. Soc. Ser. B 48 259-302.
  • Chen, S. S., Donoho, D. L. and Saunders, M. A. (1999). Atomic decomposition by basis pursuit. SIAM J. Scientific Computing 20 33-61.
  • Fan, J. and Li, R. (1999). Variable selection via penalized likelihood. Unpublished manuscript.
  • Fiacco, A. V. and McCormick, G. P. (1990). Nonlinear Programming: Sequential Unconstrained Minimization Techniques. SIAM, Philadelphia.
  • Frank, I. E. and Friedman, J. H. (1993). A statistical view ofsome chemometrics regression tools (with discussion). Technometrics 35 109-148.
  • Fu, W. J. (1998). Penalized regressions: the Bridge versus the Lasso. J. Comput. Graph. Statist. 7 397-416.
  • Fu, W. J. (1999). Estimating the effective trends in age-period-cohort studies. Unpublished manuscript.
  • Geyer, C. J. (1994). On the asymptotics ofconstrained M-estimation. Ann. Statist. 22 1993-2010.
  • Geyer, C. J. (1996). On the asymptotics ofconvex stochastic optimization. Unpublished manuscript.
  • Kim, J. and Pollard, D. (1990). Cube root asymptotics. Ann. Statistics 18 191-219.
  • Kupper, L. L., Janis, J. M., Karmous, A. and Greenberg, B. G. (1985). Statistical age-periodcohort analysis: a review and critique. J. Chronic Disease 38 811-830.
  • Linhart, H. and Zucchini, W. (1986). Model Selection. Wiley, New York.
  • Osborne, M. R., Presnell, B. and Turlach, B. A. (1998). On the Lasso and its dual. Research Report 98/1, Dept. Statistics, Univ. Adelaide.
  • Pflug, G. Ch. (1995). Asymptotic stochastic programs. Math. Oper. Res. 20 769-789.
  • Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators. Econometric Theory 7 186-199.
  • Sardy, S., Bruce, A. G. and Tseng, P. (1998). Block coordinate relaxation methods for nonparametric signal denoising with wavelet dictionaries. Technical Report, Dept. Mathematics, EPFL, Lausanne.
  • Srivastava, M. S. (1971). On fixed width confidence bounds for regression parameters. Ann. Math. Statist. 42 1403-1411.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267-288.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes with Applications to Statistics. Springer, New York.