Concentration inequalities and moment bounds for sample covariance operators

Abstract

Let $X,X_{1},\dots,X_{n},\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $\Sigma$:

\[\Sigma:E^{\ast}\mapsto E,\qquad\Sigma u=\mathbb{E}\langle X,u\rangle X,\qquad u\in E^{\ast}.\] The sample covariance operator $\hat{\Sigma}:E^{\ast}\mapsto E$ is defined as

\[\hat{\Sigma}u:=n^{-1}\sum_{j=1}^{n}\langle X_{j},u\rangle X_{j},\qquad u\in E^{\ast}.\] The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\Vert \hat{\Sigma}-\Sigma\Vert $ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that

\[\mathbb{E}\Vert \hat{\Sigma}-\Sigma\Vert \asymp\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\vee \frac{{\mathbf{r}}(\Sigma)}{n}),\] where

\[{\mathbf{r}}(\Sigma):=\frac{(\mathbb{E}\Vert X\Vert )^{2}}{\Vert \Sigma\Vert }.\] Moreover, it is proved that, under the assumption that ${\mathbf{r}}(\Sigma)\leq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

\[\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{t}{n}}\vee \frac{t}{n}),\] where $M$ is either the median, or the expectation of $\Vert \hat{\Sigma}-\Sigma\Vert $. On the other hand, under the assumption that ${\mathbf{r}}(\Sigma)\geq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

\[\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\sqrt{\frac{t}{n}}\vee \frac{t}{n}).\]