Statistical Science

Principal Fitted Components for Dimension Reduction in Regression

R. Dennis Cook and Liliana Forzani

Full-text: Open access


We provide a remedy for two concerns that have dogged the use of principal components in regression: (i) principal components are computed from the predictors alone and do not make apparent use of the response, and (ii) principal components are not invariant or equivariant under full rank linear transformation of the predictors. The development begins with principal fitted components [Cook, R. D. (2007). Fisher lecture: Dimension reduction in regression (with discussion). Statist. Sci. 22 1–26] and uses normal models for the inverse regression of the predictors on the response to gain reductive information for the forward regression of interest. This approach includes methodology for testing hypotheses about the number of components and about conditional independencies among the predictors.

Article information

Statist. Sci. Volume 23, Number 4 (2008), 485-501.

First available in Project Euclid: 11 May 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Central subspace dimension reduction inverse regression principal components


Cook, R. Dennis; Forzani, Liliana. Principal Fitted Components for Dimension Reduction in Regression. Statist. Sci. 23 (2008), no. 4, 485--501. doi:10.1214/08-STS275.

Export citation


  • Anderson, T. W. (1969). Statistical inference for covariance matrices with linear structure. In Multivariate Analysis II (P. Krishnia, ed.) 55–66. Academic Press, New York.
  • Anderson, T. W. (1971). An Introduction to Multivariate Statistical Analysis. Wiley, New York.
  • Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006). Prediction by supervised principal components. J. Amer. Statist. Assoc. 101 119–137.
  • Burnham, K. and Anderson, D. (2002). Model Selection and Multimodel Inference. Wiley, New York.
  • Bura, E. and Cook, R. D. (2001). Estimating the structural dimension of regressions via parametric inverse regression. J. Roy. Statist. Soc. Ser. B 63 393–410.
  • Bura, E. and Pfeiffer, R. M. (2003). Graphical methods for class prediction using dimension reduction techniques on DNA microarray data. Bioinformatics 19 1252–1258.
  • Chikuse, Y. (2003). Statistics on Special Manifolds. Springer, New York.
  • Cook, R. D. (1994). Using dimension-reduction subspaces to identify important inputs in models of physical systems. In Proceedings of the Section on Physical and Engineering Sciences 18–25. Amer. Statist. Assoc., Alexandria, VA.
  • Cook, R. D. (1998). Regression Graphics. Wiley, New York.
  • Cook, R. D. (2007). Fisher lecture: Dimension reduction in regression (with discussion). Statist. Sci. 22 1–26.
  • Cook, R. D. and Forzani, L. (2009). Likelihood-based sufficient dimension reduction. J. Amer. Statist. Assoc. To appear.
  • Cook, R. D., Li, B. and Chiaromonte, F. (2007). Dimension reduction in regression without matrix inversion. Biometrika 94 569–584.
  • Cook, R. D. and Ni, L. (2006). Using intraslice covariances for improved estimation of the central subspace in regression. Biometrika 93 65–74.
  • Cox, D. R. (1968). Notes on some aspects of regression analysis. J. Roy. Statist. Soc. Ser. A 131 265–279.
  • Eaton, M. (1983). Multivariate Statistics. Wiley, New York.
  • Fearn, T. (1983). A misuse of ridge regression in the calibration of a near infrared reflectance instrument. J. Appl. Statist. 32 73–79.
  • Franklin N. L., Pinchbeck P. H. and Popper F. (1956). A statistical approach to catalyst development, part 1: The effects of process variables on the vapor phase oxidation of naphthalene. Transactions of the Institute of Chemical Engineers 34 280–293.
  • Helland, I. S. (1990). Partial least squares regression and statistical models. Scand. J. Statist. 17 97–114.
  • Helland, I. S. and Almøy, T. (1994). Comparison of prediction methods when only a few components are relevant. J. Amer. Statist. Assoc. 89 583–591.
  • Li, K. C. (1991). Sliced inverse regression for dimension reduction with discussion. J. Amer. Statist. Assoc. 86 316–342.
  • Li, L. and Li, H. (2004). Dimension reduction for microarrays with application to censored survival data. Bioinformatics 20 3406–3412.
  • Meinshausen, N. (2006). relaxo: Relaxed lasso. R package version 0.1-1. Available at
  • Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York.
  • Rogers, G. S. and Young, D. L. (1977). Explicit maximum likelihood estimators for certain patterned covariance matrices. Comm. Statist. Theory Methods A6 121–133.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.