Robust sub-Gaussian estimation of a mean vector in nearly linear time

Abstract

We construct an algorithm for estimating the mean of a heavy-tailed random variable when given an adversarial corrupted sample of N independent observations. The only assumption we make on the distribution of the noncorrupted (or informative) data is the existence of a covariance matrix Σ, unknown to the statistician. Our algorithm outputs $\hat{\mathit{\mu }}$, which is robust to the presence of $|\mathcal{O}|$ adversarial outliers and satisfies

$$(1)\Vert \hat{\mathit{\mu }}-\mathit{\mu }{\Vert }_2\lesssim \sqrt{\frac{Tr(\mathrm{\Sigma })}{\mathit{N}}}+\sqrt{\frac{\Vert \mathrm{\Sigma }{\Vert }_{\mathrm{op}}\mathit{K}}{\mathit{N}}}$$

with probability at least $1-exp(-{\mathit{c}}_0\mathit{K})-exp(-{\mathit{c}}_1\mathit{u})$, and runtime $\tilde{\mathcal{O}}(\mathit{N}\mathit{d}+\mathit{u}\mathit{K}\mathit{d})$ where $\mathit{K}\in \{600|\mathcal{O}|,\dots ,\mathit{N}\}$ and $\mathit{u}\in {\mathbb{N}}^{\ast }$ are two parameters of the algorithm. The algorithm is fully data-dependent and does not use (1) in its construction, which combines recently developed tools for median-of-means estimators and covering semidefinite programming. We also show that this algorithm can automatically adapt to the number of outliers (adaptive choice of K) and that it satisfies the same bound in expectation.