Open Access
December 2008 Finite sample approximation results for principal component analysis: A matrix perturbation approach
Boaz Nadler
Ann. Statist. 36(6): 2791-2817 (December 2008). DOI: 10.1214/08-AOS618

Abstract

Principal component analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size n, and those of the limiting population PCA as n→∞. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample PCA and population PCA under a spiked covariance model. In addition, we also consider the relation between finite sample PCA and the asymptotic results in the joint limit p, n→∞, with p/n=c. We present a matrix perturbation view of the “phase transition phenomenon,” and a simple linear-algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit. Moreover, our analysis also applies for finite p, n where we show that although there is no sharp phase transition as in the infinite case, either as a function of noise level or as a function of sample size n, the eigenvector of sample PCA may exhibit a sharp “loss of tracking,” suddenly losing its relation to the (true) eigenvector of the population PCA matrix. This occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise, whose eigenvector points in a random direction.

Citation

Download Citation

Boaz Nadler. "Finite sample approximation results for principal component analysis: A matrix perturbation approach." Ann. Statist. 36 (6) 2791 - 2817, December 2008. https://doi.org/10.1214/08-AOS618

Information

Published: December 2008
First available in Project Euclid: 5 January 2009

zbMATH: 1168.62058
MathSciNet: MR2485013
Digital Object Identifier: 10.1214/08-AOS618

Subjects:
Primary: 62E17 , 62H25
Secondary: 15A42

Keywords: matrix perturbation , phase transition , Principal Component Analysis , Random matrix theory , spiked covariance model

Rights: Copyright © 2008 Institute of Mathematical Statistics

Vol.36 • No. 6 • December 2008
Back to Top