Contour regression: A general approach to dimension reduction

Bing Li, Hongyuan Zha, and Francesca Chiaromonte
Source: Ann. Statist. Volume 33, Number 4 (2005), 1580-1616.

Abstract

We propose a novel approach to sufficient dimension reduction in regression, based on estimating contour directions of small variation in the response. These directions span the orthogonal complement of the minimal space relevant for the regression and can be extracted according to two measures of variation in the response, leading to simple and general contour regression (SCR and GCR) methodology. In comparison with existing sufficient dimension reduction techniques, this contour-based methodology guarantees exhaustive estimation of the central subspace under ellipticity of the predictor distribution and mild additional assumptions, while maintaining -consistency and computational ease. Moreover, it proves robust to departures from ellipticity. We establish population properties for both SCR and GCR, and asymptotic properties for SCR. Simulations to compare performance with that of standard techniques such as ordinary least squares, sliced inverse regression, principal Hessian directions and sliced average variance estimation confirm the advantages anticipated by the theoretical analyses. We demonstrate the use of contour-based methods on a data set concerning soil evaporation.

First Page:
Primary Subjects: 62G08
Secondary Subjects: 62G09, 62H05
Full-text: Open access

Permanent link to this document: http://projecteuclid.org/euclid.aos/1123250223
Digital Object Identifier: doi:10.1214/009053605000000192
Mathematical Reviews number (MathSciNet): MR2166556
Zentralblatt MATH identifier: 1078.62033

References

Chiaromonte, F. and Cook, R. D. (2002). Sufficient dimension reduction and graphics in regression. Ann. Inst. Statist. Math. 54 768--795.
Mathematical Reviews (MathSciNet): MR1954046
Digital Object Identifier: doi:10.1023/A:1022411301790
Zentralblatt MATH: 1047.62066
Chiaromonte, F., Cook, R. D. and Li, B. (2002). Sufficient dimension reduction in regressions with categorical predictors. Ann. Statist. 30 475--497.
Mathematical Reviews (MathSciNet): MR1902896
Digital Object Identifier: doi:10.1214/aos/1021379862
Project Euclid: euclid.aos/1021379862
Zentralblatt MATH: 1012.62036
Cook, R. D. (1994). Using dimension reduction subspaces to identify important inputs in models of physical systems. In Proc. Section on Physical and Engineering Sciences 18--25. Amer. Statist. Assoc., Alexandria, VA.
Cook, R. D. (1998). Regression Graphics. Wiley, New York.
Mathematical Reviews (MathSciNet): MR1645673
Zentralblatt MATH: 0903.62001
Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30 455--474.
Mathematical Reviews (MathSciNet): MR1902895
Digital Object Identifier: doi:10.1214/aos/1021379861
Project Euclid: euclid.aos/1021379861
Zentralblatt MATH: 1012.62035
Cook, R. D. and Nachtsheim, C. J. (1994). Reweighting to achieve elliptically contoured covariates in regression. J. Amer. Statist. Assoc. 89 592--599.
Cook, R. D. and Weisberg, S. (1991). Discussion of Sliced inverse regression for dimension reduction,'' by K.-C. Li. J. Amer. Statist. Assoc. 86 328--332.
Mathematical Reviews (MathSciNet): MR1137117
Dawid, A. P. (1979). Conditional independence in statistical theory (with discussion). J. Roy. Statist. Soc. Ser. B 41 1--31.
Mathematical Reviews (MathSciNet): MR535541
Duan, N. and Li, K.-C. (1991). Slicing regression: A link-free regression method. Ann. Statist. 19 505--530.
Mathematical Reviews (MathSciNet): MR1105834
Eaton, M. L. and Tyler, D. (1994). The asymptotic distribution of singular values with application to canonical correlations and correspondence analysis. J. Multivariate Anal. 50 238--264.
Mathematical Reviews (MathSciNet): MR1293045
Digital Object Identifier: doi:10.1006/jmva.1994.1041
Zentralblatt MATH: 0805.62020
Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196--216.
Mathematical Reviews (MathSciNet): MR1212173
Freund, R. J. (1979). Multicollinearity etc., some new'' examples. ASA Proc. Statistical Computing Section 111--112. Amer. Statist. Assoc., Washington.
Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537--1566.
Mathematical Reviews (MathSciNet): MR1891738
Project Euclid: euclid.aos/1015345954
Li, B., Cook, R. D. and Chiaromonte, F. (2003). Dimension reduction for the conditional mean in regressions with categorical predictors. Ann. Statist. 31 1636--1668.
Mathematical Reviews (MathSciNet): MR2012828
Digital Object Identifier: doi:10.1214/aos/1065705121
Project Euclid: euclid.aos/1065705121
Zentralblatt MATH: 1042.62037
Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316--342.
Mathematical Reviews (MathSciNet): MR1137117
Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein's lemma. J. Amer. Statist. Assoc. 87 1025--1039.
Mathematical Reviews (MathSciNet): MR1209564
Li, K.-C. and Duan, N. (1989). Regression analysis under link violation. Ann. Statist. 17 1009--1052.
Mathematical Reviews (MathSciNet): MR1015136
Nadaraya, E. A. (1964). On estimating regression. Theory Probab. Appl. 9 141--142.
Peters, B. C., Jr., Redner, R. and Decell, H. P., Jr. (1978). Characterizations of linear sufficient statistics. Sankhyā Ser. A 40 303--309.
Mathematical Reviews (MathSciNet): MR589284
Serfling, R. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
Mathematical Reviews (MathSciNet): MR595165
Zentralblatt MATH: 0538.62002
Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595--645.
Mathematical Reviews (MathSciNet): MR443204
Watson, G. S. (1964). Smooth regression analysis. Sankhyā Ser. A 26 359--372.
Mathematical Reviews (MathSciNet): MR185765
Xia, Y., Tong, H., Li, W. K. and Zhu, L.-X. (2002). An adaptive estimation of dimension reduction space. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 363--410.
Mathematical Reviews (MathSciNet): MR1924297
Digital Object Identifier: doi:10.1111/1467-9868.03411
Zentralblatt MATH: 1091.62028