Principal Fitted Components for Dimension Reduction in Regression
R. Dennis Cook and Liliana Forzani
Source: Statist. Sci. Volume 23, Number 4 (2008), 485-501.
Abstract
We provide a remedy for two concerns that have dogged the use of principal components in regression: (i) principal components are computed from the predictors alone and do not make apparent use of the response, and (ii) principal components are not invariant or equivariant under full rank linear transformation of the predictors. The development begins with principal fitted components [Cook, R. D. (2007). Fisher lecture: Dimension reduction in regression (with discussion). Statist. Sci. 22 1–26] and uses normal models for the inverse regression of the predictors on the response to gain reductive information for the forward regression of interest. This approach includes methodology for testing hypotheses about the number of components and about conditional independencies among the predictors.
Keywords: Central subspace; dimension reduction; inverse regression; principal components
Full-text: Access denied (no subscription detected)
We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber.
If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ss/1242049391
Digital Object Identifier: doi:10.1214/08-STS275
Mathematical Reviews number (MathSciNet):
MR2530547
References
Anderson, T. W. (1969). Statistical inference for covariance matrices with linear structure. In Multivariate Analysis II (P. Krishnia, ed.) 55–66. Academic Press, New York.
Mathematical Reviews (MathSciNet):
MR256509
Anderson, T. W. (1971). An Introduction to Multivariate Statistical Analysis. Wiley, New York.
Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006). Prediction by supervised principal components. J. Amer. Statist. Assoc. 101 119–137.
Mathematical Reviews (MathSciNet):
MR2252436
Digital Object Identifier: doi:10.1198/016214505000000628
Burnham, K. and Anderson, D. (2002). Model Selection and Multimodel Inference. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR1919620
Bura, E. and Cook, R. D. (2001). Estimating the structural dimension of regressions via parametric inverse regression. J. Roy. Statist. Soc. Ser. B 63 393–410.
Mathematical Reviews (MathSciNet):
MR1841422
Digital Object Identifier: doi:10.1111/1467-9868.00292
JSTOR: links.jstor.org
Bura, E. and Pfeiffer, R. M. (2003). Graphical methods for class prediction using dimension reduction techniques on DNA microarray data. Bioinformatics 19 1252–1258.
Chikuse, Y. (2003). Statistics on Special Manifolds. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1960435
Cook, R. D. (1994). Using dimension-reduction subspaces to identify important inputs in models of physical systems. In Proceedings of the Section on Physical and Engineering Sciences 18–25. Amer. Statist. Assoc., Alexandria, VA.
Cook, R. D. (1998). Regression Graphics. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR1645673
Cook, R. D. (2007). Fisher lecture: Dimension reduction in regression (with discussion). Statist. Sci. 22 1–26.
Mathematical Reviews (MathSciNet):
MR2408655
Digital Object Identifier: doi:10.1214/088342306000000682
Project Euclid: euclid.ss/1185975631
Cook, R. D. and Forzani, L. (2009). Likelihood-based sufficient dimension reduction. J. Amer. Statist. Assoc. To appear.
Cook, R. D., Li, B. and Chiaromonte, F. (2007). Dimension reduction in regression without matrix inversion. Biometrika 94 569–584.
Mathematical Reviews (MathSciNet):
MR2410009
Digital Object Identifier: doi:10.1093/biomet/asm038
Cook, R. D. and Ni, L. (2006). Using intraslice covariances for improved estimation of the central subspace in regression. Biometrika 93 65–74.
Mathematical Reviews (MathSciNet):
MR2277740
Digital Object Identifier: doi:10.1093/biomet/93.1.65
Cox, D. R. (1968). Notes on some aspects of regression analysis. J. Roy. Statist. Soc. Ser. A 131 265–279.
Eaton, M. (1983). Multivariate Statistics. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR716321
Fearn, T. (1983). A misuse of ridge regression in the calibration of a near infrared reflectance instrument. J. Appl. Statist. 32 73–79.
Franklin N. L., Pinchbeck P. H. and Popper F. (1956). A statistical approach to catalyst development, part 1: The effects of process variables on the vapor phase oxidation of naphthalene. Transactions of the Institute of Chemical Engineers 34 280–293.
Helland, I. S. (1990). Partial least squares regression and statistical models. Scand. J. Statist. 17 97–114.
Mathematical Reviews (MathSciNet):
MR1085924
Helland, I. S. and Almøy, T. (1994). Comparison of prediction methods when only a few components are relevant. J. Amer. Statist. Assoc. 89 583–591.
Li, K. C. (1991). Sliced inverse regression for dimension reduction with discussion. J. Amer. Statist. Assoc. 86 316–342.
Mathematical Reviews (MathSciNet):
MR1137117
Digital Object Identifier: doi:10.2307/2290563
JSTOR: links.jstor.org
Li, L. and Li, H. (2004). Dimension reduction for microarrays with application to censored survival data. Bioinformatics 20 3406–3412.
Meinshausen, N. (2006). relaxo: Relaxed lasso. R package version 0.1-1. Available at http://www.stat.berkeley.edu/~nicolai.
Mathematical Reviews (MathSciNet):
MR2409990
Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR652932
Rogers, G. S. and Young, D. L. (1977). Explicit maximum likelihood estimators for certain patterned covariance matrices. Comm. Statist. Theory Methods A6 121–133.
Mathematical Reviews (MathSciNet):
MR436430
Digital Object Identifier: doi:10.1080/03610927708827477
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
Mathematical Reviews (MathSciNet):
MR1379242
JSTOR: links.jstor.org
