Open Access
Translator Disclaimer
October, 1965 An Optimal Property of Principal Components
J. N. Darroch
Ann. Math. Statist. 36(5): 1579-1582 (October, 1965). DOI: 10.1214/aoms/1177699920

Abstract

Let $x' = (x_1, x_2, \cdots, x_p)$ be a random vector and let $E\lbrack x\rbrack = 0, E\lbrack xx'\rbrack = \Sigma = (\sigma_{ij})$ where we assume that $\Sigma$ is non-singular. Further let \[ \Sigma = T\Lambda T' = (t_1, t_2, \cdots, t_p) \left[ \begin{array}{c|c|c|c|c|c|c}\lambda_1 & 0 & \cdot & \cdot & \cdot & 0 \\ 0 & \lambda_2 & \cdot & \cdot & \cdot & 0 \\ \vdots & \vdots & \vdots & \vdots \\0 & 0 & \cdot & \cdot & \cdot & \lambda_p\end{array} \right] \] \[ \left[ \begin{array}{|c}t'_1 \\ t'_2 \\ \vdots \\ t'_p\end{array}\right] \] where $TT' = I$ and where we suppose that the eigenvalues of $\Sigma$ are in order of decreasing magnitude, that is $\lambda_1 \geqq \lambda_2 \geqq \cdots \geqq \lambda_p > 0$. The principal components of $x$, namely $u_1 = t'_1 x, u_2 = t'_2 x, \cdots, u_p = t_p' x$ were introduced by Hotelling (1933) who characterised them by certain optimal properties. Since then Girshick (1936), Anderson (1958) and Kullback (1959) have characterised the principal components by slightly different sets of optimal properties. Thus Anderson shows that $u_1$ is the linear function $\alpha_1' x$ having maximum variance subject to $\alpha_1' \alpha_1 = 1; u_2$ is the linear function $\alpha_2'x$ which is uncorrelated with $u_1$ and has maximum variance subject to $\alpha_2'\alpha_2 = 1$; and so on. The above mentioned characterisations have two properties in common; they introduce the principal components one by one and, more importantly, the optimal properties hold only with-in the class of linear functions of $x_1, x_2, \cdots, x_p$. In the following theorem the first $k$ principal components are characterized by an optimal property within the class of all random variables.

Citation

Download Citation

J. N. Darroch. "An Optimal Property of Principal Components." Ann. Math. Statist. 36 (5) 1579 - 1582, October, 1965. https://doi.org/10.1214/aoms/1177699920

Information

Published: October, 1965
First available in Project Euclid: 27 April 2007

zbMATH: 0141.34401
MathSciNet: MR183073
Digital Object Identifier: 10.1214/aoms/1177699920

Rights: Copyright © 1965 Institute of Mathematical Statistics

JOURNAL ARTICLE
4 PAGES


SHARE
Vol.36 • No. 5 • October, 1965
Back to Top