Bernoulli

  • Bernoulli
  • Volume 23, Number 1 (2017), 110-133.

Concentration inequalities and moment bounds for sample covariance operators

Vladimir Koltchinskii and Karim Lounici

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Let $X,X_{1},\dots,X_{n},\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $\Sigma$:

\[\Sigma:E^{\ast}\mapsto E,\qquad\Sigma u=\mathbb{E}\langle X,u\rangle X,\qquad u\in E^{\ast}.\] The sample covariance operator $\hat{\Sigma}:E^{\ast}\mapsto E$ is defined as

\[\hat{\Sigma}u:=n^{-1}\sum_{j=1}^{n}\langle X_{j},u\rangle X_{j},\qquad u\in E^{\ast}.\] The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\Vert \hat{\Sigma}-\Sigma\Vert $ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that

\[\mathbb{E}\Vert \hat{\Sigma}-\Sigma\Vert \asymp\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\vee \frac{{\mathbf{r}}(\Sigma)}{n}),\] where

\[{\mathbf{r}}(\Sigma):=\frac{(\mathbb{E}\Vert X\Vert )^{2}}{\Vert \Sigma\Vert }.\] Moreover, it is proved that, under the assumption that ${\mathbf{r}}(\Sigma)\leq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

\[\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{t}{n}}\vee \frac{t}{n}),\] where $M$ is either the median, or the expectation of $\Vert \hat{\Sigma}-\Sigma\Vert $. On the other hand, under the assumption that ${\mathbf{r}}(\Sigma)\geq n$, for all $t\geq1$, with probability at least $1-e^{-t}$

\[\vert \Vert \hat{\Sigma}-\Sigma\Vert -M\vert \lesssim\Vert \Sigma\Vert (\sqrt{\frac{{\mathbf{r}}(\Sigma)}{n}}\sqrt{\frac{t}{n}}\vee \frac{t}{n}).\]

Article information

Source
Bernoulli Volume 23, Number 1 (2017), 110-133.

Dates
Received: August 2014
Revised: March 2015
First available in Project Euclid: 27 September 2016

Permanent link to this document
https://projecteuclid.org/euclid.bj/1475001350

Digital Object Identifier
doi:10.3150/15-BEJ730

Mathematical Reviews number (MathSciNet)
MR3556768

Zentralblatt MATH identifier
1366.60057

Keywords
concentration inequalities effective rank moment bounds sample covariance

Citation

Koltchinskii, Vladimir; Lounici, Karim. Concentration inequalities and moment bounds for sample covariance operators. Bernoulli 23 (2017), no. 1, 110--133. doi:10.3150/15-BEJ730. https://projecteuclid.org/euclid.bj/1475001350


Export citation

References

  • [1] Adamczak, R. (2014). A note on the Hanson–Wright inequality for random vectors with dependencies. Preprint. Available at arXiv:1409.8457.
  • [2] Ahlswede, R. and Winter, A. (2002). Strong converse for identification via quantum channels. IEEE Trans. Inform. Theory 48 569–579.
  • [3] Bednorz, W. (2014). Concentration via chaining method and its applications. Preprint. Available at arXiv:1405.0676v2.
  • [4] Dirksen, S. (2015). Tail bounds via generic chaining. Electron. J. Probab. 20 no. 53, 29.
  • [5] Klartag, B. and Mendelson, S. (2005). Empirical processes and random projections. J. Funct. Anal. 225 229–245.
  • [6] Koltchinskii, V. (2011). Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math. 2033. Heidelberg: Springer.
  • [7] Kwapień, S. and Szymański, B. (1980). Some remarks on Gaussian measures in Banach spaces. Probab. Math. Statist. 1 59–65.
  • [8] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces. Isoperimetry and Processes. Berlin: Springer.
  • [9] Lounici, K. (2014). High-dimensional covariance matrix estimation with missing observations. Bernoulli 20 1029–1058.
  • [10] Lust-Piquard, F. and Pisier, G. (1991). Noncommutative Khintchine and Paley inequalities. Ark. Mat. 29 241–260.
  • [11] Mendelson, S. (2010). Empirical processes with a bounded $\psi_{1}$ diameter. Geom. Funct. Anal. 20 988–1027.
  • [12] Mendelson, S. (2012). Oracle inequalities and the isomorphic method. Unpublished manuscript.
  • [13] Rudelson, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60–72.
  • [14] Talagrand, M. (2005). The Generic Chaining. Berlin: Springer.
  • [15] Tropp, J.A. (2012). User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12 389–434.
  • [16] Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing (Y. Eldar and G. Kutyniok, eds.) 210–268. Cambridge: Cambridge Univ. Press.