The Annals of Statistics

Globally adaptive quantile regression with ultra-high dimensional data

Qi Zheng, Limin Peng, and Xuming He

Full-text: Open access

Abstract

Quantile regression has become a valuable tool to analyze heterogeneous covaraite-response associations that are often encountered in practice. The development of quantile regression methodology for high-dimensional covariates primarily focuses on the examination of model sparsity at a single or multiple quantile levels, which are typically prespecified ad hoc by the users. The resulting models may be sensitive to the specific choices of the quantile levels, leading to difficulties in interpretation and erosion of confidence in the results. In this article, we propose a new penalization framework for quantile regression in the high-dimensional setting. We employ adaptive $L_{1}$ penalties, and more importantly, propose a uniform selector of the tuning parameter for a set of quantile levels to avoid some of the potential problems with model selection at individual quantile levels. Our proposed approach achieves consistent shrinkage of regression quantile estimates across a continuous range of quantiles levels, enhancing the flexibility and robustness of the existing penalized quantile regression methods. Our theoretical results include the oracle rate of uniform convergence and weak convergence of the parameter estimators. We also use numerical studies to confirm our theoretical findings and illustrate the practical utility of our proposal.

Article information

Source
Ann. Statist., Volume 43, Number 5 (2015), 2225-2258.

Dates
Received: September 2014
Revised: April 2015
First available in Project Euclid: 16 September 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1442364151

Digital Object Identifier
doi:10.1214/15-AOS1340

Mathematical Reviews number (MathSciNet)
MR3396984

Zentralblatt MATH identifier
1327.62424

Subjects
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62H12: Estimation

Keywords
Ultra-high dimensional data varying covariate effects adaptive penalized quantile regression model selection oracle property

Citation

Zheng, Qi; Peng, Limin; He, Xuming. Globally adaptive quantile regression with ultra-high dimensional data. Ann. Statist. 43 (2015), no. 5, 2225--2258. doi:10.1214/15-AOS1340. https://projecteuclid.org/euclid.aos/1442364151


Export citation

References

  • Bassett, G. Jr. and Koenker, R. (1982). An empirical quantile function for linear models with iid errors. J. Amer. Statist. Assoc. 77 407–415.
  • Belloni, A. and Chernozhukov, V. (2011). $\ell_{1}$-penalized quantile regression in high-dimensional sparse models. Ann. Statist. 39 82–130.
  • Chen, J. and Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika 95 759–771.
  • Fan, J., Fan, Y. and Barut, E. (2014). Adaptive robust variable selection. Ann. Statist. 42 324–351.
  • Fan, J. and Lv, J. (2011). Nonconcave penalized likelihood with NP-dimensionality. IEEE Trans. Inform. Theory 57 5467–5484.
  • Fan, Y. and Tang, C. Y. (2013). Tuning parameter selection in high dimensional penalized likelihood. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 531–552.
  • He, X. and Shao, Q.-M. (2000). On parameters of increasing dimensions. J. Multivariate Anal. 73 120–135.
  • Huang, J., Ma, S. and Zhang, C.-H. (2008). Adaptive Lasso for sparse high-dimensional regression models. Statist. Sinica 18 1603–1618.
  • Kim, Y., Choi, H. and Oh, H.-S. (2008). Smoothly clipped absolute deviation on high dimensions. J. Amer. Statist. Assoc. 103 1665–1673.
  • Knight, K. and Fu, W. (2000). Asymptotics for lasso-type estimators. Ann. Statist. 28 1356–1378.
  • Koenker, R. (2005). Quantile Regression. Econometric Society Monographs 38. Cambridge Univ. Press, Cambridge.
  • Koenker, R. and Bassett, G. Jr. (1978). Regression quantiles. Econometrica 46 33–50.
  • Koenker, R. W. and D’Orey, V. (1987). Algorithm as 229: Computing regression quantiles. J. Roy. Statist. Soc. Ser. C 36 383–393.
  • Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Ergebnisse der Mathematik und Ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)] 23. Springer, Berlin.
  • Li, Y., Liu, Y. and Zhu, J. (2007). Quantile regression in reproducing kernel Hilbert spaces. J. Amer. Statist. Assoc. 102 255–268.
  • Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37 3498–3528.
  • Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436–1462.
  • Nishii, R. (1984). Asymptotic properties of criteria for selection of variables in multiple regression. Ann. Statist. 12 758–765.
  • Peng, L., Xu, J. and Kutner, N. (2014). Shrinkage estimation of varying covariate effects based on quantile regression. Stat. Comput. 24 853–869.
  • Portnoy, S. (1991). Asymptotic behavior of the number of regression quantile breakpoints. SIAM J. Sci. Statist. Comput. 12 867–883.
  • Qian, J. and Peng, L. (2010). Censored quantile regression with partially functional effects. Biometrika 97 839–850.
  • Rocha, G., Wang, X. and Yu, B. (2009). Asymptotic distribution and sparsistency for l1-penalized parametric M-estimators with applications to linear svm and logistic regression. Available at arXiv:0908.1940.
  • Scheetz, T. E., Kim, K.-Y. A., Swiderski, R. E., Philp, A. R., Braun, T. A., Knudtson, K. L., Dorrance, A. M., DiBona, G. F., Huang, J., Casavant, T. L., Sheffield, V. C. and Stone, E. M. (2006). Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proc. Natl. Acad. Sci. USA 103 14429–14434.
  • Talagrand, M. (2005). The Generic Chaining: Upper and Lower Bounds of Stochastic Processes. Springer, Berlin.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • Wang, H., Li, B. and Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 671–683.
  • Wang, H., Li, R. and Tsai, C.-L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94 553–568.
  • Wang, L., Wu, Y. and Li, R. (2012). Quantile regression for analyzing heterogeneity in ultra-high dimension. J. Amer. Statist. Assoc. 107 214–222.
  • Wang, T. and Zhu, L. (2011). Consistent tuning parameter selection in high dimensional sparse linear regression. J. Multivariate Anal. 102 1141–1151.
  • Welsh, A. H. (1989). On $M$-processes and $M$-estimation. Ann. Statist. 17 337–361.
  • Wu, Y. and Liu, Y. (2009). Variable selection in quantile regression. Statist. Sinica 19 801–817.
  • Yang, Y. and He, X. (2012). Bayesian empirical likelihood for quantile regression. Ann. Statist. 40 1102–1131.
  • Zhang, C.-H. and Huang, J. (2008). The sparsity and bias of the LASSO selection in high-dimensional linear regression. Ann. Statist. 36 1567–1594.
  • Zhang, Y., Li, R. and Tsai, C.-L. (2010). Regularization parameter selections via generalized information criterion. J. Amer. Statist. Assoc. 105 312–323.
  • Zheng, Q., Gallagher, C. and Kulasekera, K. B. (2013). Adaptive penalized quantile regression for high dimensional data. J. Statist. Plann. Inference 143 1029–1038.
  • Zheng, Q., Peng, L. and He, X. (2015). Supplement to “Globally adaptive quantile regression with ultra-high dimensional data.” DOI:10.1214/15-AOS1340SUPP.
  • Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.
  • Zou, H. and Yuan, M. (2008). Composite quantile regression and the oracle model selection theory. Ann. Statist. 36 1108–1126.

Supplemental materials

  • Supplement to “Globally adaptive quantile regression with ultra-high dimensional data”. Due to space constraints, additional simulation results, the proofs of technical lemmas and corollaries, justification for the proposed grid approximation, and sample codes are relegated to the supplement [Zheng, Peng and He (2015)].