Open Access
Translator Disclaimer
2020 On the predictive potential of kernel principal components
Ben Jones, Andreas Artemiou, Bing Li
Electron. J. Statist. 14(1): 1-23 (2020). DOI: 10.1214/19-EJS1655

Abstract

We give a probabilistic analysis of a phenomenon in statistics which, until recently, has not received a convincing explanation. This phenomenon is that the leading principal components tend to possess more predictive power for a response variable than lower-ranking ones despite the procedure being unsupervised. Our result, in its most general form, shows that the phenomenon goes far beyond the context of linear regression and classical principal components — if an arbitrary distribution for the predictor $X$ and an arbitrary conditional distribution for $Y\vert X$ are chosen then any measureable function $g(Y)$, subject to a mild condition, tends to be more correlated with the higher-ranking kernel principal components than with the lower-ranking ones. The “arbitrariness” is formulated in terms of unitary invariance then the tendency is explicitly quantified by exploring how unitary invariance relates to the Cauchy distribution. The most general results, for technical reasons, are shown for the case where the kernel space is finite dimensional. The occurency of this tendency in real world databases is also investigated to show that our results are consistent with observation.

Citation

Download Citation

Ben Jones. Andreas Artemiou. Bing Li. "On the predictive potential of kernel principal components." Electron. J. Statist. 14 (1) 1 - 23, 2020. https://doi.org/10.1214/19-EJS1655

Information

Received: 1 August 2019; Published: 2020
First available in Project Euclid: 3 January 2020

zbMATH: 07147381
MathSciNet: MR4047592
Digital Object Identifier: 10.1214/19-EJS1655

Subjects:
Primary: 60K35 , 60K35
Secondary: 60K35

Keywords: Cauchy distribution , Dimension reduction , kernel principal components , Nonparametric regression , unitary invariance

JOURNAL ARTICLE
23 PAGES


SHARE
Vol.14 • No. 1 • 2020
Back to Top