Electronic Journal of Statistics

Principal quantile regression for sufficient dimension reduction with heteroscedasticity

Chong Wang, Seung Jun Shin, and Yichao Wu

Full-text: Open access

Abstract

Sufficient dimension reduction (SDR) is a successful tool for reducing data dimensionality without stringent model assumptions. In practice, data often display heteroscedasticity which is of scientific importance in general but frequently overlooked since a primal goal of most existing statistical methods is to identify conditional mean relationship among variables. In this article, we propose a new SDR method called principal quantile regression (PQR) that efficiently tackles heteroscedasticity. PQR can naturally be extended to a nonlinear version via kernel trick. Asymptotic properties are established and an efficient solution path-based algorithm is provided. Numerical examples based on both simulated and real data demonstrate the PQR’s advantageous performance over existing SDR methods. PQR still performs very competitively even for the case without heteroscedasticity.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 2114-2140.

Dates
Received: March 2017
First available in Project Euclid: 13 July 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1531468823

Digital Object Identifier
doi:10.1214/18-EJS1432

Subjects
Primary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]

Keywords
Heteroscedasticity kernel quantile regression principal quantile regression sufficient dimension reduction

Rights
Creative Commons Attribution 4.0 International License.

Citation

Wang, Chong; Shin, Seung Jun; Wu, Yichao. Principal quantile regression for sufficient dimension reduction with heteroscedasticity. Electron. J. Statist. 12 (2018), no. 2, 2114--2140. doi:10.1214/18-EJS1432. https://projecteuclid.org/euclid.ejs/1531468823


Export citation

References

  • [1] Bura, E. and Pfeiffer, C. A. (2008). On the distribution of the left singular vectors of a random matrix and its applications., Statistics and Probability Letters 78 2275–2280.
  • [2] Cook, R. D. (1994). On the interpretation of regression plots., Journal of the American Statistical Association 89 177–189.
  • [3] Cook, R. (1998a). Regression Graphics: Ideas for Studying Regressions Through Graphics. 1998., USA: A Wiley-Interscience Pubilcation CrossRef Google Scholar.
  • [4] Cook, R. D. (1998b). Principal Hessian directions revisited., Journal of the American Statistical Association 93 84–94.
  • [5] Cook, R. D. (2007). Fisher Lecture: Dimension Reduction in Regression., Statistical Science 22 1–26.
  • [6] Cook, R. D. and Ni, L. (2005). Sufficient dimension reduction via inverse regression., Journal of the American Statistical Association 100.
  • [7] Cook, R. D. and Ni, L. (2006). Using intraslice covariances for improved estimation of the central subspace in regression., Biometrika 65–74.
  • [8] Cook, R. D. and Weisberg, S. (1991). Discussion of “Sliced inverse regression for dimension reduction”., Journal of the American Statistical Association 86 28–33.
  • [9] Hall, P. and Li, K. C. (1993). On almost linearity of low-dimensional projections from high-dimensional data., The Annals of Statistics 21 867–889.
  • [10] Harrison, D. and Rubinfeld, D. L. (1978). Hedonic housing prices and the demand for clean air., Journal of environmental economics and management 5 81–102.
  • [11] Helland, I. S. (1990). Partial least squares regression and statistical models., Scandinavian Journal of Statistics 2 97–114.
  • [12] Kimeldorf, G. and Wahba, G. (1971). Some results on Tchebycheffian spline functions., Journal of Mathematical Analysis and Applications 33 82–95.
  • [13] Kong, E. and Xia, Y. (2014). An adaptive composite quantile approach to dimension reduction., The Annals of Statistics 42 1657–1688.
  • [14] Lee, K.-Y., Li, B. and Chiaromonte, F. (2013). A general theory for nonlinear sufficient dimension reduction: Formulation and estimation., The Annals of Statistics 41 221–249.
  • [15] Li, K. C. (1991a). Sliced inverse regression for dimension reduction (with discussion)., Journal of the American Statistical Association 86 316–342.
  • [16] Li, K.-C. (1991b). Sliced inverse regression for dimension reduction., Journal of the American Statistical Association 86 316–327.
  • [17] Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: another application of Stein’s lemma., Journal of the American Statistical Association 87 1025–1039.
  • [18] Li, B., Artemiou, A. and Li, L. (2011). Principal support vector machines for linear and nonlinear sufficient dimension reduction., The Annals of Statistics 39 3182–3210.
  • [19] Li, B. and Dong, Y. (2009). dimension reduction for nonelliptically distributed predictors., The Annals of Statistics 37 1272–1298.
  • [20] Li, K. C. and Duan, N. (1989). Regression analysis under link violation., The Annals of Statistics 17 1009–1052.
  • [21] Li, Y., Liu, Y. and Zhu, J. (2007). Quantile regression in reproducing kernel Hilbert spaces., Journal of the American Statistical Association 102 255–268.
  • [22] Li, B. and Wang, S. (2007). On directional regression for dimension reduction., Journal of the American Statistical Association 102 997–1008.
  • [23] Li, B., Zha, H. and Chiaromonte, F. (2005). Contour regression: a general approach to dimension reduction., The Annals of statistics 1580–1616.
  • [24] Newey, W. K. and McFadden, D. (1994). Large sample estimation and hypothesis testing. In, Hnadbook of Economietrics IV 2113–2245.
  • [25] Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators., Econometric Theory 7 186–199.
  • [26] Schölkopf, B., Smola, A. and Müller, K.-R. (1997). Kernel principal component analysis. In, Artificial Neural Networks–ICANN’97 583–588. Springer.
  • [27] Shin, S. J., Zhang, H. H. and Wu, Y. (2017). A nonparametric survival function estimator via censored kernel quantile regressions., Statistica Sinica 27 457-478.
  • [28] Shin, S. J., Wu, Y., Zhang, H. H. and Liu, Y. (2017). Principal Weighted Support Vector Machines for Sufficient Dimension Reduction in Binary Classification., Biometrika 104 67-81.
  • [29] Székely, G. J., Rizzo, M. L. and Bakirov, N. K. (2007). Measuring and testing dependence by correlation of distances., The Annals of Statistics 35 2769–2794.
  • [30] Takeuchi, I., Nomura, K. and Kanamori, T. (2009). Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression., Neural Computation 21 533–559.
  • [31] Takeuchi, I., Le, Q. V., Sears, T. D. and Somolar, A. J. (2006). Nonparametric quantile estimation., Journal of machine learning research 7 1231–1264.
  • [32] Van der Vaart, A. W. (2000)., Asymptotic statistics 3. Cambridge university press.
  • [33] Vapnik, V. (1996)., The Nature of Statistical Learning Theory. Cambridge University Press: New York.
  • [34] Wu, H. M. (2008). Kernel sliced inverse regression with applications to classification., Journal of Computational and Graphical Statistics 17 590–610.
  • [35] Wu, Q., Liang, F. and Mukherjee, S. (2013). Kernel sliced inverse regression: regularization and consistency., Abstract and Applied Analysis 2013 1–11. Article ID 540725.
  • [36] Xia, Y., Tong, H., Li, W. and Zhu, L.-X. (2002). An adaptive estimation of dimension reduction space., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64 363–410.
  • [37] Yeh, Y. R., Huang, S. Y. and Lee, Y. Y. (2009). Nonlinear dimension reduction with kernel sliced inverse regression., IEEE Transactions on Knowledge and Data Engineering 21 1590–1603.
  • [38] Yin, X., Li, B. and Cook, R. D. (2008). Successive direction extraction for estimating the central subspace in a multiple-index regression., Journal of Multivariate Analysis 99 1733–1757.