## Bernoulli

• Bernoulli
• Volume 23, Number 1 (2017), 110-133.

### Concentration inequalities and moment bounds for sample covariance operators

#### Abstract

Let $X,X_{1},\dots,X_{n},\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $\Sigma$:

$\Sigma:E^{\ast}\mapsto E,\qquad\Sigma u=\mathbb{E}\langle X,u\rangle X,\qquad u\in E^{\ast}.$ The sample covariance operator $\hat{\Sigma}:E^{\ast}\mapsto E$ is defined as

$\hat{\Sigma}u:=n^{-1}\sum_{j=1}^{n}\langle X_{j},u\rangle X_{j},\qquad u\in E^{\ast}.$ The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\Vert \hat{\Sigma}-\Sigma\Vert$ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that

$\mathbb{E}\Vert \hat{\Sigma}-\Sigma\Vert \asymp\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\vee \frac{{\mathbf{r}}(\Sigma)}{n}),$ where

${\mathbf{r}}(\Sigma):=\frac{(\mathbb{E}\Vert X\Vert )^{2}}{\Vert \Sigma\Vert }.$ Moreover, it is proved that, under the assumption that ${\mathbf{r}}(\Sigma)\leq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

$\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{t}{n}}\vee \frac{t}{n}),$ where $M$ is either the median, or the expectation of $\Vert \hat{\Sigma}-\Sigma\Vert$. On the other hand, under the assumption that ${\mathbf{r}}(\Sigma)\geq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

$\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\sqrt{\frac{t}{n}}\vee \frac{t}{n}).$

#### Article information

Source
Bernoulli Volume 23, Number 1 (2017), 110-133.

Dates
Revised: March 2015
First available in Project Euclid: 27 September 2016

https://projecteuclid.org/euclid.bj/1475001350

Digital Object Identifier
doi:10.3150/15-BEJ730

Mathematical Reviews number (MathSciNet)
MR3556768

Zentralblatt MATH identifier
1366.60057

#### Citation

Koltchinskii, Vladimir; Lounici, Karim. Concentration inequalities and moment bounds for sample covariance operators. Bernoulli 23 (2017), no. 1, 110--133. doi:10.3150/15-BEJ730. https://projecteuclid.org/euclid.bj/1475001350

#### References

• [1] Adamczak, R. (2014). A note on the Hanson–Wright inequality for random vectors with dependencies. Preprint. Available at arXiv:1409.8457.
• [2] Ahlswede, R. and Winter, A. (2002). Strong converse for identification via quantum channels. IEEE Trans. Inform. Theory 48 569–579.
• [3] Bednorz, W. (2014). Concentration via chaining method and its applications. Preprint. Available at arXiv:1405.0676v2.
• [4] Dirksen, S. (2015). Tail bounds via generic chaining. Electron. J. Probab. 20 no. 53, 29.
• [5] Klartag, B. and Mendelson, S. (2005). Empirical processes and random projections. J. Funct. Anal. 225 229–245.
• [6] Koltchinskii, V. (2011). Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math. 2033. Heidelberg: Springer.
• [7] Kwapień, S. and Szymański, B. (1980). Some remarks on Gaussian measures in Banach spaces. Probab. Math. Statist. 1 59–65.
• [8] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces. Isoperimetry and Processes. Berlin: Springer.
• [9] Lounici, K. (2014). High-dimensional covariance matrix estimation with missing observations. Bernoulli 20 1029–1058.
• [10] Lust-Piquard, F. and Pisier, G. (1991). Noncommutative Khintchine and Paley inequalities. Ark. Mat. 29 241–260.
• [11] Mendelson, S. (2010). Empirical processes with a bounded $\psi_{1}$ diameter. Geom. Funct. Anal. 20 988–1027.
• [12] Mendelson, S. (2012). Oracle inequalities and the isomorphic method. Unpublished manuscript.
• [13] Rudelson, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60–72.
• [14] Talagrand, M. (2005). The Generic Chaining. Berlin: Springer.
• [15] Tropp, J.A. (2012). User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12 389–434.
• [16] Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing (Y. Eldar and G. Kutyniok, eds.) 210–268. Cambridge: Cambridge Univ. Press.