The Annals of Statistics

Perturbation bootstrap in adaptive Lasso

Debraj Das, Karl Gregory, and S. N. Lahiri

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

The Adaptive Lasso (Alasso) was proposed by Zou [J. Amer. Statist. Assoc. 101 (2006) 1418–1429] as a modification of the Lasso for the purpose of simultaneous variable selection and estimation of the parameters in a linear regression model. Zou [J. Amer. Statist. Assoc. 101 (2006) 1418–1429] established that the Alasso estimator is variable-selection consistent as well as asymptotically Normal in the indices corresponding to the nonzero regression coefficients in certain fixed-dimensional settings. In an influential paper, Minnier, Tian and Cai [J. Amer. Statist. Assoc. 106 (2011) 1371–1382] proposed a perturbation bootstrap method and established its distributional consistency for the Alasso estimator in the fixed-dimensional setting. In this paper, however, we show that this (naive) perturbation bootstrap fails to achieve second-order correctness in approximating the distribution of the Alasso estimator. We propose a modification to the perturbation bootstrap objective function and show that a suitably Studentized version of our modified perturbation bootstrap Alasso estimator achieves second-order correctness even when the dimension of the model is allowed to grow to infinity with the sample size. As a consequence, inferences based on the modified perturbation bootstrap will be more accurate than the inferences based on the oracle Normal approximation. We give simulation studies demonstrating good finite-sample properties of our modified perturbation bootstrap method as well as an illustration of our method on a real data set.

Article information

Source
Ann. Statist., Volume 47, Number 4 (2019), 2080-2116.

Dates
Received: November 2016
Revised: February 2018
First available in Project Euclid: 21 May 2019

Permanent link to this document
https://projecteuclid.org/euclid.aos/1558425640

Digital Object Identifier
doi:10.1214/18-AOS1741

Mathematical Reviews number (MathSciNet)
MR3953445

Zentralblatt MATH identifier
07082280

Subjects
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62G09: Resampling methods 62E20: Asymptotic distribution theory

Keywords
Alasso naive perturbation bootstrap modified perturbation bootstrap second-order correctness oracle

Citation

Das, Debraj; Gregory, Karl; Lahiri, S. N. Perturbation bootstrap in adaptive Lasso. Ann. Statist. 47 (2019), no. 4, 2080--2116. doi:10.1214/18-AOS1741. https://projecteuclid.org/euclid.aos/1558425640


Export citation

References

  • Bhattacharya, R. N. and Ghosh, J. K. (1978). On the validity of the formal Edgeworth expansion. Ann. Statist. 6 434–451.
  • Bhattacharya, R. N. and Ranga Rao, R. (1986). Normal Approximation and Asymptotic Expansions. John Wiley & Sons, New York.
  • Boyd, S. and Vandenberghe, L. (2004). Convex Optimization. Cambridge Univ. Press, Cambridge.
  • Bühlmann, P., Kalisch, M. and Meier, L. (2014). High-dimensional statistics with a view towards applications in biology. Annu. Rev. Statist. Appl. 1 255–278.
  • Camponovo, L. (2015). On the validity of the pairs bootstrap for Lasso estimators. Biometrika 102 981–987.
  • Caner, M. and Fan, Q. (2010). The adaptive Lasso method for instrumental variable selection. Working paper, North Carolina State Univ., Raleigh, NC.
  • Chatterjee, S. and Bose, A. (2005). Generalized bootstrap for estimating equations. Ann. Statist. 33 414–436.
  • Chatterjee, A. and Lahiri, S. N. (2010). Asymptotic properties of the residual bootstrap for Lasso estimators. Proc. Amer. Math. Soc. 138 4497–4509.
  • Chatterjee, A. and Lahiri, S. N. (2011). Bootstrapping Lasso estimators. J. Amer. Statist. Assoc. 106 608–625.
  • Chatterjee, A. and Lahiri, S. N. (2013). Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap. Ann. Statist. 41 1232–1259.
  • Das, D., Gregory, K. and Lahiri, S. N. (2019). Supplement to “Perturbation bootstrap in adaptive Lasso.” DOI:10.1214/18-AOS1741SUPP.
  • Das, D. and Lahiri, S. N. (2019). Second order correctness of perturbation bootstrap M-estimator of multiple linear regression parameter. Bernoulli 25 654–682.
  • Dezeure, R., Bühlmann, P. and Zhang, C.-H. (2017). High-dimensional simultaneous inference with the bootstrap. TEST 26 685–719. Available at arXiv:1606.03940.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33 1–22.
  • Fuk, D. H. and Nagaev, S. V. (1971). Probabilistic inequalities for sums of independent random variables. Teor. Veroyatn. Primen. 16 660–675.
  • Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • Hall, P., Lee, Y. K., Park, B. U. and Paul, D. (2009). Tie-respecting bootstrap methods for estimating distributions of sets and functions of eigenvalues. Bernoulli 15 380–401.
  • Jin, Z., Ying, Z. and Wei, L. J. (2001). A simple resampling method by perturbing the minimand. Biometrika 88 381–390.
  • Knight, K. and Fu, W. (2000). Asymptotics for Lasso-type estimators. Ann. Statist. 28 1356–1378.
  • Minnier, J., Tian, L. and Cai, T. (2011). A perturbation method for inference on regularized regression estimates. J. Amer. Statist. Assoc. 106 1371–1382.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • Turnbull, H. W. (1930). A matrix form of Taylor’s theorem. Proc. Edinb. Math. Soc. (2) 33 33–54.
  • van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Statist. 42 1166–1202.
  • Wang, X. and Song, L. (2011). Adaptive Lasso variable selection for the accelerated failure models. Comm. Statist. Theory Methods 40 4372–4386.
  • Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. J. Roy. Statist. Soc. Ser. B 76 217–242.
  • Zhou, Q. M., Song, P. X.-K. and Thompson, M. E. (2012). Information ratio test for model misspecification in quasi-likelihood inference. J. Amer. Statist. Assoc. 107 205–213.
  • Zou, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.

Supplemental materials

  • Supplement to “Perturbation bootstrap in adaptive Lasso”. Details of the proofs and additional simulation studies are provided.