The Annals of Statistics

A constructive approach to the estimation of dimension reduction directions

Yingcun Xia

Full-text: Open access


In this paper we propose two new methods to estimate the dimension-reduction directions of the central subspace (CS) by constructing a regression model such that the directions are all captured in the regression mean. Compared with the inverse regression estimation methods [e.g., J. Amer. Statist. Assoc. 86 (1991) 328–332, J. Amer. Statist. Assoc. 86 (1991) 316–342, J. Amer. Statist. Assoc. 87 (1992) 1025–1039], the new methods require no strong assumptions on the design of covariates or the functional relation between regressors and the response variable, and have better performance than the inverse regression estimation methods for finite samples. Compared with the direct regression estimation methods [e.g., J. Amer. Statist. Assoc. 84 (1989) 986–995, Ann. Statist. 29 (2001) 1537–1566, J. R. Stat. Soc. Ser. B Stat. Methodol. 64 (2002) 363–410], which can only estimate the directions of CS in the regression mean, the new methods can detect the directions of CS exhaustively. Consistency of the estimators and the convergence of corresponding algorithms are proved.

Article information

Ann. Statist., Volume 35, Number 6 (2007), 2654-2690.

First available in Project Euclid: 22 January 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G09: Resampling methods 62H05: Characterization and structure theory

Conditional density function convergence of algorithm double-kernel smoothing efficient dimension reduction root-n consistency


Xia, Yingcun. A constructive approach to the estimation of dimension reduction directions. Ann. Statist. 35 (2007), no. 6, 2654--2690. doi:10.1214/009053607000000352.

Export citation


  • Ando, T. (1987). Totally positive matrices. Linear Algebra Appl. 90 165–219.
  • Bai, Z. D., Miao, B. Q. and Rao, C. R. (1991). Estimation of directions of arrival of signals: Asymptotic results. In Advances in Spectrum Analysis and Array Processing (S. Haykin, ed.) 1 327–347. Prentice Hall, Englewood Cliffs, NJ.
  • Chen, C.-H. and Li, K.-C. (1998). Can SIR be as popular as multiple linear regression? Statist. Sinica 8 289–316.
  • Chow, Y. S. and Teicher, H. (1978). Probability Theory. Independence, Interchangeability, Martingales. Springer, New York.
  • Chung, K. L. (1968). A Course in Probability Theory. Harcourt, Brace and World, New York.
  • Cook, R. D. (1998). Regression Graphics. Wiley, New York.
  • Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30 455–474.
  • Cook, R. D. and Weisberg, S. (1991). Comment on “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 328–332.
  • de la Peña, V. H. (1999). A general class of exponential inequalities for martingales and ratios. Ann. Probab. 27 537–564.
  • Delecroix, M., Hristache, M. and Patilea, V. (2005). On semiparametric $M$-estimation in single-index regression. J. Statist. Plann. Inference 136 730–769.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
  • Fan, J., Yao, Q. and Tong, H. (1996). Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 189–206.
  • Härdle, W., Hall, P. and Ichimura, H. (1993). Optimal smoothing in single-index models. Ann. Statist. 21 157–178.
  • Härdle, W. and Stoker, T. M. (1989). Investigating smooth multiple regression by the method of average derivatives. J. Amer. Statist. Assoc. 84 986–995.
  • Härdle, W. and Tsybakov, A. B. (1991). Comment on “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 333–335.
  • Hoeffding, W. (1961). The strong law of large numbers for U-statistics. Mimeo Report No. 302, Inst. Statist., Univ. North Carolina.
  • Horowitz, J. L. and Härdle, W. (1996). Direct semiparametric estimation of single-index models with discrete covariates. J. Amer. Statist. Assoc. 91 1632–1640.
  • Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537–1566.
  • Hristache, M., Juditsky, A. and Spokoiny, V. (2001). Direct estimation of the index coefficient in a single-index model. Ann. Statist. 29 595–623.
  • Li, B., Zha, H. and Chiaromonte, F. (2005). Contour regression: A general approach to dimension reduction. Ann. Statist. 33 1580–1616.
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316–342.
  • Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein's lemma. J. Amer. Statist. Assoc. 87 1025–1039.
  • Lue, H.-H. (2004). Principal Hessian directions for regression with measurement error. Biometrika 91 409–423.
  • Mack, Y. P. and Silverman, B. W. (1982). Weak and strong uniform consistency of kernel regression estimates. Z. Wahrsch. Verw. Gebiete 61 405–415.
  • Samarov, A. M. (1993). Exploring regression structure using nonparametric functional estimation. J. Amer. Statist. Assoc. 88 836–847.
  • Scott, D. W. (1992). Multivariate Density Estimation: Theory, Practice and Visualization. Wiley, New York.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
  • World Health Organization (2003). Reports on a WHO/HEI working group. Bonn, Germany.
  • Xia, Y. (2006). Asymptotic distributions for two estimators of the single-index model. Econometric Theory 22 1112–1137.
  • Xia, Y. (2006). A constructive approach to the estimation of dimension reduction directions. Technical report, Dept. Statistics and Applied Probability, National Univ. Singapore.
  • Xia, Y., Tong, H. and Li, W. K. (2002). Single-index volatility models and estimation. Statist. Sinica 12 785–799.
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L. (2002). An adaptive estimation of dimension reduction space (with discussion). J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 363–410.
  • Yin, X. and Cook, R. D. (2002). Dimension reduction for the conditional $k$th moment in regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 159–175.
  • Yin, X. and Cook, R. D. (2005). Direction estimation in single-index regressions. Biometrika 92 371–384.