Annals of Statistics

A constructive approach to the estimation of dimension reduction directions

Yingcun Xia

Full-text: Open access

Enhanced PDF (467 KB)

Abstract

In this paper we propose two new methods to estimate the dimension-reduction directions of the central subspace (CS) by constructing a regression model such that the directions are all captured in the regression mean. Compared with the inverse regression estimation methods [e.g., J. Amer. Statist. Assoc. 86 (1991) 328–332, J. Amer. Statist. Assoc. 86 (1991) 316–342, J. Amer. Statist. Assoc. 87 (1992) 1025–1039], the new methods require no strong assumptions on the design of covariates or the functional relation between regressors and the response variable, and have better performance than the inverse regression estimation methods for finite samples. Compared with the direct regression estimation methods [e.g., J. Amer. Statist. Assoc. 84 (1989) 986–995, Ann. Statist. 29 (2001) 1537–1566, J. R. Stat. Soc. Ser. B Stat. Methodol. 64 (2002) 363–410], which can only estimate the directions of CS in the regression mean, the new methods can detect the directions of CS exhaustively. Consistency of the estimators and the convergence of corresponding algorithms are proved.

Article information

Source
Ann. Statist., Volume 35, Number 6 (2007), 2654-2690.

Dates
First available in Project Euclid: 22 January 2008

Permanent link to this document
https://projecteuclid.org/euclid.aos/1201012976

Digital Object Identifier
doi:10.1214/009053607000000352

Mathematical Reviews number (MathSciNet)
MR2382662

Zentralblatt MATH identifier
1360.62196

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G09: Resampling methods 62H05: Characterization and structure theory

Keywords
Conditional density function convergence of algorithm double-kernel smoothing efficient dimension reduction root-n consistency

Citation

Xia, Yingcun. A constructive approach to the estimation of dimension reduction directions. Ann. Statist. 35 (2007), no. 6, 2654--2690. doi:10.1214/009053607000000352. https://projecteuclid.org/euclid.aos/1201012976


Export citation

References

  • Ando, T. (1987). Totally positive matrices. Linear Algebra Appl. 90 165–219.
    Mathematical Reviews (MathSciNet): MR0884118
    Zentralblatt MATH: 0613.15014
    Digital Object Identifier: doi:10.1016/0024-3795(87)90313-2
  • Bai, Z. D., Miao, B. Q. and Rao, C. R. (1991). Estimation of directions of arrival of signals: Asymptotic results. In Advances in Spectrum Analysis and Array Processing (S. Haykin, ed.) 1 327–347. Prentice Hall, Englewood Cliffs, NJ.
  • Chen, C.-H. and Li, K.-C. (1998). Can SIR be as popular as multiple linear regression? Statist. Sinica 8 289–316.
    Mathematical Reviews (MathSciNet): MR1624402
    Zentralblatt MATH: 0897.62069
  • Chow, Y. S. and Teicher, H. (1978). Probability Theory. Independence, Interchangeability, Martingales. Springer, New York.
    Mathematical Reviews (MathSciNet): MR0513230
    Zentralblatt MATH: 0399.60001
  • Chung, K. L. (1968). A Course in Probability Theory. Harcourt, Brace and World, New York.
    Mathematical Reviews (MathSciNet): MR0229268
  • Cook, R. D. (1998). Regression Graphics. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1645673
    Zentralblatt MATH: 0903.62001
  • Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30 455–474.
    Mathematical Reviews (MathSciNet): MR1902895
    Zentralblatt MATH: 1012.62035
    Digital Object Identifier: doi:10.1214/aos/1021379861
    Project Euclid: euclid.aos/1021379861
  • Cook, R. D. and Weisberg, S. (1991). Comment on “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 328–332.
    Mathematical Reviews (MathSciNet): MR1137117
    Zentralblatt MATH: 0742.62044
    Digital Object Identifier: doi:10.1080/01621459.1991.10475035
  • de la Peña, V. H. (1999). A general class of exponential inequalities for martingales and ratios. Ann. Probab. 27 537–564.
    Mathematical Reviews (MathSciNet): MR1681153
    Zentralblatt MATH: 0942.60004
    Digital Object Identifier: doi:10.1214/aop/1022677271
    Project Euclid: euclid.aop/1022677271
  • Delecroix, M., Hristache, M. and Patilea, V. (2005). On semiparametric $M$-estimation in single-index regression. J. Statist. Plann. Inference 136 730–769.
    Mathematical Reviews (MathSciNet): MR2181975
    Zentralblatt MATH: 1077.62027
    Digital Object Identifier: doi:10.1016/j.jspi.2004.09.006
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
    Mathematical Reviews (MathSciNet): MR1383587
    Zentralblatt MATH: 0873.62037
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1964455
    Zentralblatt MATH: 1014.62103
  • Fan, J., Yao, Q. and Tong, H. (1996). Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 189–206.
    Mathematical Reviews (MathSciNet): MR1399164
    Zentralblatt MATH: 0865.62026
    Digital Object Identifier: doi:10.1093/biomet/83.1.189
  • Härdle, W., Hall, P. and Ichimura, H. (1993). Optimal smoothing in single-index models. Ann. Statist. 21 157–178.
    Mathematical Reviews (MathSciNet): MR1212171
    Zentralblatt MATH: 0770.62049
    Digital Object Identifier: doi:10.1214/aos/1176349020
    Project Euclid: euclid.aos/1176349020
  • Härdle, W. and Stoker, T. M. (1989). Investigating smooth multiple regression by the method of average derivatives. J. Amer. Statist. Assoc. 84 986–995.
    Mathematical Reviews (MathSciNet): MR1134488
    Zentralblatt MATH: 0703.62052
  • Härdle, W. and Tsybakov, A. B. (1991). Comment on “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 333–335.
    Mathematical Reviews (MathSciNet): MR1137117
    Zentralblatt MATH: 0742.62044
    Digital Object Identifier: doi:10.1080/01621459.1991.10475035
  • Hoeffding, W. (1961). The strong law of large numbers for U-statistics. Mimeo Report No. 302, Inst. Statist., Univ. North Carolina.
  • Horowitz, J. L. and Härdle, W. (1996). Direct semiparametric estimation of single-index models with discrete covariates. J. Amer. Statist. Assoc. 91 1632–1640.
    Mathematical Reviews (MathSciNet): MR1439104
    Zentralblatt MATH: 0881.62037
    Digital Object Identifier: doi:10.1080/01621459.1996.10476732
  • Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537–1566.
    Mathematical Reviews (MathSciNet): MR1891738
    Zentralblatt MATH: 1043.62052
    Project Euclid: euclid.aos/1015345954
  • Hristache, M., Juditsky, A. and Spokoiny, V. (2001). Direct estimation of the index coefficient in a single-index model. Ann. Statist. 29 595–623.
    Mathematical Reviews (MathSciNet): MR1865333
    Zentralblatt MATH: 1012.62043
    Digital Object Identifier: doi:10.1214/aos/1009210681
  • Li, B., Zha, H. and Chiaromonte, F. (2005). Contour regression: A general approach to dimension reduction. Ann. Statist. 33 1580–1616.
    Mathematical Reviews (MathSciNet): MR2166556
    Zentralblatt MATH: 1078.62033
    Digital Object Identifier: doi:10.1214/009053605000000192
    Project Euclid: euclid.aos/1123250223
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316–342.
    Mathematical Reviews (MathSciNet): MR1137117
    Zentralblatt MATH: 0742.62044
    Digital Object Identifier: doi:10.1080/01621459.1991.10475035
  • Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein's lemma. J. Amer. Statist. Assoc. 87 1025–1039.
    Mathematical Reviews (MathSciNet): MR1209564
    Zentralblatt MATH: 0765.62003
    Digital Object Identifier: doi:10.1080/01621459.1992.10476258
  • Lue, H.-H. (2004). Principal Hessian directions for regression with measurement error. Biometrika 91 409–423.
    Mathematical Reviews (MathSciNet): MR2081310
    Zentralblatt MATH: 1079.62066
    Digital Object Identifier: doi:10.1093/biomet/91.2.409
  • Mack, Y. P. and Silverman, B. W. (1982). Weak and strong uniform consistency of kernel regression estimates. Z. Wahrsch. Verw. Gebiete 61 405–415.
    Mathematical Reviews (MathSciNet): MR0679685
    Zentralblatt MATH: 0495.62046
    Digital Object Identifier: doi:10.1007/BF00539840
  • Samarov, A. M. (1993). Exploring regression structure using nonparametric functional estimation. J. Amer. Statist. Assoc. 88 836–847.
    Mathematical Reviews (MathSciNet): MR1242934
    Zentralblatt MATH: 0790.62035
    Digital Object Identifier: doi:10.1080/01621459.1993.10476348
  • Scott, D. W. (1992). Multivariate Density Estimation: Theory, Practice and Visualization. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1191168
    Zentralblatt MATH: 0850.62006
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
    Mathematical Reviews (MathSciNet): MR0848134
    Zentralblatt MATH: 0617.62042
  • World Health Organization (2003). Reports on a WHO/HEI working group. Bonn, Germany.
  • Xia, Y. (2006). Asymptotic distributions for two estimators of the single-index model. Econometric Theory 22 1112–1137.
    Mathematical Reviews (MathSciNet): MR2328530
    Zentralblatt MATH: 1170.62323
    Digital Object Identifier: doi:10.1017/S0266466606060531
  • Xia, Y. (2006). A constructive approach to the estimation of dimension reduction directions. Technical report, Dept. Statistics and Applied Probability, National Univ. Singapore.
    Mathematical Reviews (MathSciNet): MR2382662
    Zentralblatt MATH: 05241119
    Digital Object Identifier: doi:10.1214/009053607000000352
    Project Euclid: euclid.aos/1201012976
  • Xia, Y., Tong, H. and Li, W. K. (2002). Single-index volatility models and estimation. Statist. Sinica 12 785–799.
    Mathematical Reviews (MathSciNet): MR1929964
    Zentralblatt MATH: 1002.62082
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L. (2002). An adaptive estimation of dimension reduction space (with discussion). J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 363–410.
    Mathematical Reviews (MathSciNet): MR1924297
    Zentralblatt MATH: 1091.62028
    Digital Object Identifier: doi:10.1111/1467-9868.03411
  • Yin, X. and Cook, R. D. (2002). Dimension reduction for the conditional $k$th moment in regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 159–175.
    Mathematical Reviews (MathSciNet): MR1904698
    Zentralblatt MATH: 1067.62042
    Digital Object Identifier: doi:10.1111/1467-9868.00330
  • Yin, X. and Cook, R. D. (2005). Direction estimation in single-index regressions. Biometrika 92 371–384.
    Mathematical Reviews (MathSciNet): MR2201365
    Zentralblatt MATH: 1094.62054
    Digital Object Identifier: doi:10.1093/biomet/92.2.371

The Institute of Mathematical Statistics

Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more