The Annals of Statistics

Dimension reduction for nonelliptically distributed predictors

Bing Li and Yuexiao Dong

Full-text: Open access

Abstract

Sufficient dimension reduction methods often require stringent conditions on the joint distribution of the predictor, or, when such conditions are not satisfied, rely on marginal transformation or reweighting to fulfill them approximately. For example, a typical dimension reduction method would require the predictor to have elliptical or even multivariate normal distribution. In this paper, we reformulate the commonly used dimension reduction methods, via the notion of “central solution space,” so as to circumvent the requirements of such strong assumptions, while at the same time preserve the desirable properties of the classical methods, such as $\sqrt{n}$-consistency and asymptotic normality. Imposing elliptical distributions or even stronger assumptions on predictors is often considered as the necessary tradeoff for overcoming the “curse of dimensionality,” but the development of this paper shows that this need not be the case. The new methods will be compared with existing methods by simulation and applied to a data set.

Article information

Source
Ann. Statist. Volume 37, Number 3 (2009), 1272-1298.

Dates
First available in Project Euclid: 10 April 2009

Permanent link to this document
http://projecteuclid.org/euclid.aos/1239369022

Digital Object Identifier
doi:10.1214/08-AOS598

Zentralblatt MATH identifier
1160.62050

Mathematical Reviews number (MathSciNet)
MR2509074

Subjects
Primary: 62H12: Estimation 62G08: Nonparametric regression 62G09: Resampling methods

Keywords
Canonical correlation central solution spaces kernel inverse regression inverse regression sliced inverse regression parametric inverse regression

Citation

Li, Bing; Dong, Yuexiao. Dimension reduction for nonelliptically distributed predictors. Ann. Statist. 37 (2009), no. 3, 1272--1298. doi:10.1214/08-AOS598. http://projecteuclid.org/euclid.aos/1239369022.


Export citation

References

  • Allen, D. M. (1974). The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16 125–127.
  • Bellman, R. (1961). Adaptive Control Processes: A Guided Tour. Princeton Univ. Press.
  • Bickel, P., Klaassen, C. A. J., Ritov, Y. and Wellner, J. (1993). Efficient and Adaptive Inference in Semi-Parametric Models. Johns Hopkins Univ. Press, Baltimore.
  • Bura, E. and Cook, R. D. (2001). Estimating the structural dimension of regressions via parametric inverse regression. J. Roy. Statist. Soc. Ser. B 63 393–410.
  • Chiaromonte, F. and Cook, R. D. (2001). Sufficient dimension reduction and graphics in regression. Ann. Inst. Statist. Math. 54 768–795.
  • Cook, R. D. (1994). Using dimension-reduction subspaces to identify important inputs in models of physical systems. In Amer. Statist. Assoc. Proceedings of the Section on Physical and Engineering Sciences. Amer. Statist. Assoc., Washington, DC.
  • Cook, R. D. (1996). Graphics for regressions with a binary response. J. Amer. Statist. Assoc. 91 983–992.
  • Cook, R. D. (1998). Regression Graphics: Ideas for Studying Regressions Through Graphics. Wiley, New York.
  • Cook, R. D. (2007). Fisher lecture: Dimension reduction for regression (with discussion). Statist. Sci. 22 1–26.
  • Cook, R. D. and Nachtsheim, C. J. (1994). Reweighting to achieve elliptically contoured covariates in regression. J. Amer. Statist. Assoc. 89 592–599.
  • Cook, R. D. and Ni, L. (2005). Sufficient dimension reduction via inverse regression: A minimum discrepancy approach. J. Amer. Statist. Assoc. 100 410–428.
  • Cook, R. D. and Ni, L. (2006). Using intra-slice covariances for improved estimation of central subspace in regression. Biometrika 93 65–74.
  • Cook, R. D. and Weisberg, S. (1991). Discussion of “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 328–332.
  • Dawid, A. P. (1979). Conditional independence in statistical theory (with discussion). J. Roy. Statist. Soc. Ser. B 41 1–31.
  • Eaton, M. L. (1986). A characterization of spherical distributions. J. Multivariate Anal. 34 439–446.
  • Fernholz, L. T. (1983). Von Mises Calculus for Statistical Functionals. Springer, New York.
  • Ferre, L. and Yao, A. F. (2005). Smooth function inverse regression. Statist. Sinica 15 665–683.
  • Fung, K. F., He, X., Liu, L. and Shi, P. (2002). Dimension reduction based on canonical correlation. Statist. Sinica 12 1093–1113.
  • Hall, W. J. and Mathiason, D. J. (1990). On large-sample estimation and testing in parametric models. Internat. Statist. Rev. 58 77–97.
  • Li, B. and Wang, S. (2007). On directional regression for dimension reduction. J. Amer. Statist. Assoc. 102 997–1008.
  • Li, K. C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316–342.
  • Li, K. C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein’s lemma. J. Amer. Statist. Assoc. 87 1025–1039.
  • Li, K. C. and Duan, N. (1989). Regression analysis under link violation. Ann. Statist. 17 1009–1052.
  • McCullagh, P. (1987). Tensor Methods in Statistics. Chapman and Hall, London.
  • Nelder, J. A. and Mead, R. (1965). A simplex method for function minimization. Comput. J. 7 308–313.
  • Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. J. Roy. Statist. Soc. Ser. B 36 111–147.
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press.
  • von Mises, R. (1947). On the asymptotic distribution of differentiable statistical functions. Ann. Math. Statist. 18 309–348.
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L. X. (2002). An adaptive estimation of optimal regression subspace. J. Roy. Statist. Soc. Ser. B 64 363–410.
  • Yin, X., Li, B. and Cook, R. D. (2008). Successive direction extraction for estimating the central subspace in a multiple-index regression. J. Multivariate Anal. 99 1733–1757.
  • Zhu, L.-X. and Fang, K.-T. (1996). Asymptotics for kernel estimate of sliced inverse regression. Ann. Statist. 3 1053–1068.