Covariance discriminative power of kernel clustering methods

Abstract

Let $x_1,\cdots ,x_n$ be independent observations of size p, each of them belonging to one of c distinct classes. We assume that observations within the class a are characterized by their distribution $N(0,\frac{1}{p}{C}_a)$ where here $C_1,\cdots ,C_c$ are some non-negative definite $p\times p$ matrices. This paper studies the asymptotic behavior of the symmetric matrix ${\tilde{\mathrm{\Phi }}}_{kl}=\sqrt{p}\left({(x_k^T{x}_l)}^2{\mathit{\delta }}_{k\ne l}\right)$ when p and n grow to infinity with $\frac{n}{p}\to c_0$. Particularly, we prove that, if the class covariance matrices are sufficiently close in a certain sense, the matrix $\tilde{\mathrm{\Phi }}$ behaves like a low-rank perturbation of a Wigner matrix, presenting possibly some isolated eigenvalues outside the bulk of the semi-circular law. We carry out a careful analysis of some of the isolated eigenvalues of $\tilde{\mathrm{\Phi }}$ and their associated eigenvectors and illustrate how these results can help understand spectral clustering methods that use $\tilde{\mathrm{\Phi }}$ as a kernel matrix.