The Annals of Statistics

Contour regression: A general approach to dimension reduction

Bing Li, Hongyuan Zha, and Francesca Chiaromonte

Full-text: Open access

Abstract

We propose a novel approach to sufficient dimension reduction in regression, based on estimating contour directions of small variation in the response. These directions span the orthogonal complement of the minimal space relevant for the regression and can be extracted according to two measures of variation in the response, leading to simple and general contour regression (SCR and GCR) methodology. In comparison with existing sufficient dimension reduction techniques, this contour-based methodology guarantees exhaustive estimation of the central subspace under ellipticity of the predictor distribution and mild additional assumptions, while maintaining $\sqrt{n}$-consistency and computational ease. Moreover, it proves robust to departures from ellipticity. We establish population properties for both SCR and GCR, and asymptotic properties for SCR. Simulations to compare performance with that of standard techniques such as ordinary least squares, sliced inverse regression, principal Hessian directions and sliced average variance estimation confirm the advantages anticipated by the theoretical analyses. We demonstrate the use of contour-based methods on a data set concerning soil evaporation.

Article information

Source
Ann. Statist. Volume 33, Number 4 (2005), 1580-1616.

Dates
First available in Project Euclid: 5 August 2005

Permanent link to this document
http://projecteuclid.org/euclid.aos/1123250223

Digital Object Identifier
doi:10.1214/009053605000000192

Mathematical Reviews number (MathSciNet)
MR2166556

Zentralblatt MATH identifier
1078.62033

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G09: Resampling methods 62H05: Characterization and structure theory

Keywords
Central subspace empirical directions PCA nonparametric regression data visualization

Citation

Li, Bing; Zha, Hongyuan; Chiaromonte, Francesca. Contour regression: A general approach to dimension reduction. The Annals of Statistics 33 (2005), no. 4, 1580--1616. doi:10.1214/009053605000000192. http://projecteuclid.org/euclid.aos/1123250223.


Export citation

References

  • Chiaromonte, F. and Cook, R. D. (2002). Sufficient dimension reduction and graphics in regression. Ann. Inst. Statist. Math. 54 768--795.
  • Chiaromonte, F., Cook, R. D. and Li, B. (2002). Sufficient dimension reduction in regressions with categorical predictors. Ann. Statist. 30 475--497.
  • Cook, R. D. (1994). Using dimension reduction subspaces to identify important inputs in models of physical systems. In Proc. Section on Physical and Engineering Sciences 18--25. Amer. Statist. Assoc., Alexandria, VA.
  • Cook, R. D. (1998). Regression Graphics. Wiley, New York.
  • Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30 455--474.
  • Cook, R. D. and Nachtsheim, C. J. (1994). Reweighting to achieve elliptically contoured covariates in regression. J. Amer. Statist. Assoc. 89 592--599.
  • Cook, R. D. and Weisberg, S. (1991). Discussion of ``Sliced inverse regression for dimension reduction,'' by K.-C. Li. J. Amer. Statist. Assoc. 86 328--332.
  • Dawid, A. P. (1979). Conditional independence in statistical theory (with discussion). J. Roy. Statist. Soc. Ser. B 41 1--31.
  • Duan, N. and Li, K.-C. (1991). Slicing regression: A link-free regression method. Ann. Statist. 19 505--530.
  • Eaton, M. L. and Tyler, D. (1994). The asymptotic distribution of singular values with application to canonical correlations and correspondence analysis. J. Multivariate Anal. 50 238--264.
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196--216.
  • Freund, R. J. (1979). Multicollinearity etc., some ``new'' examples. ASA Proc. Statistical Computing Section 111--112. Amer. Statist. Assoc., Washington.
  • Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537--1566.
  • Li, B., Cook, R. D. and Chiaromonte, F. (2003). Dimension reduction for the conditional mean in regressions with categorical predictors. Ann. Statist. 31 1636--1668.
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316--342.
  • Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein's lemma. J. Amer. Statist. Assoc. 87 1025--1039.
  • Li, K.-C. and Duan, N. (1989). Regression analysis under link violation. Ann. Statist. 17 1009--1052.
  • Nadaraya, E. A. (1964). On estimating regression. Theory Probab. Appl. 9 141--142.
  • Peters, B. C., Jr., Redner, R. and Decell, H. P., Jr. (1978). Characterizations of linear sufficient statistics. Sankhyā Ser. A 40 303--309.
  • Serfling, R. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
  • Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595--645.
  • Watson, G. S. (1964). Smooth regression analysis. Sankhyā Ser. A 26 359--372.
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L.-X. (2002). An adaptive estimation of dimension reduction space. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 363--410.