Electronic Journal of Statistics

A general decision theory for Huber’s $\epsilon$-contamination model

Mengjie Chen, Chao Gao, and Zhao Ren

Full-text: Open access


Today’s data pose unprecedented challenges to statisticians. It may be incomplete, corrupted or exposed to some unknown source of contamination. We need new methods and theories to grapple with these challenges. Robust estimation is one of the revived fields with potential to accommodate such complexity and glean useful information from modern datasets. Following our recent work on high dimensional robust covariance matrix estimation, we establish a general decision theory for robust statistics under Huber’s $\epsilon$-contamination model. We propose a solution using Scheffé estimate to a robust two-point testing problem that leads to the construction of robust estimators adaptive to the proportion of contamination. Applying the general theory, we construct robust estimators for nonparametric density estimation, sparse linear regression and low-rank trace regression. We show that these new estimators achieve the minimax rate with optimal dependence on the contamination proportion. This testing procedure, Scheffé estimate, also enjoys an optimal rate in the exponent of the testing error, which may be of independent interest.

Article information

Electron. J. Statist., Volume 10, Number 2 (2016), 3752-3774.

Received: March 2016
First available in Project Euclid: 6 December 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C20: Minimax procedures 62G35: Robustness
Secondary: 62H12: Estimation

Robust statistics robust testing minimax rate density estimation sparse linear regression trace regression


Chen, Mengjie; Gao, Chao; Ren, Zhao. A general decision theory for Huber’s $\epsilon$-contamination model. Electron. J. Statist. 10 (2016), no. 2, 3752--3774. doi:10.1214/16-EJS1216. https://projecteuclid.org/euclid.ejs/1480993453

Export citation


  • [1] Birgé, L. (1984). Sur un theórème de minimax et son application aux tests., Probability and Mathematical Statistics 3 259–282.
  • [2] Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise., The Annals of Statistics 24 2384–2398.
  • [3] Candes, E. J. and Plan, Y. (2011). Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements., Information Theory, IEEE Transactions on 57 2342–2359.
  • [4] Chen, M., Gao, C. and Ren, Z. (2015). Robust covariance and scatter matrix estimation via matrix depth., arXiv preprint arXiv:1506.00691.
  • [5] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms., Applied and Computational Harmonic Analysis 1 54–81.
  • [6] Devroye, L. and Lugosi, G. (2012)., Combinatorial Methods in Density Estimation. Springer Science & Business Media.
  • [7] Donoho, D. L. (1994). Statistical estimation and optimal recovery., The Annals of Statistics 22 238–270.
  • [8] Donoho, D. L. and Liu, R. C. (1991). Geometrizing rates of convergence, III., The Annals of Statistics 19 668–701.
  • [9] Hoffmann, M., Rousseau, J. and Schmidt-Hieber, J. (2015). On adaptive posterior concentration rates., The Annals of Statistics 43 2259–2295.
  • [10] Huber, P. J. (1964). Robust estimation of a location parameter., The Annals of Mathematical Statistics 35 73–101.
  • [11] Huber, P. J. (1965). A robust version of the probability ratio test., The Annals of Mathematical Statistics 36 1753–1758.
  • [12] Huber, P. J. and Strassen, V. (1973). Minimax tests and the Neyman-Pearson lemma for capacities., The Annals of Statistics 1 251–263.
  • [13] Koltchinskii, V., Lounici, K. and Tsybakov, A. B. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., The Annals of Statistics 39 2302–2329.
  • [14] Le Cam, L. (1973). Convergence of estimates under dimensionality restrictions., The Annals of Statistics 1 38–53.
  • [15] Neyman, J. and Pearson, E. (1933). On the problem of the most efficient tests of statistical hypotheses., Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 231 289–337.
  • [16] Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise., The Annals of Statistics 24 2399–2430.
  • [17] Pinsker, M. S. (1980). Optimal filtering of square-integrable signals in Gaussian noise., Problemy Peredachi Informatsii 16 52–68.
  • [18] Pollard, D. (1990). Empirical Processes: Theory and Applications. In, NSF-CBMS Regional Conference Series in Probability and Statistics i–86. JSTOR.
  • [19] Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax rates of estimation for high-dimensional linear regression over $\ell_q$-balls., IEEE Transactions on Information Theory 57 6976–6994.
  • [20] Recht, B., Fazel, M. and Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization., SIAM review 52 471–501.
  • [21] Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices., The Annals of Statistics 39 887–930.
  • [22] Rudelson, M. and Zhou, S. (2013). Reconstruction from anisotropic random measurements., IEEE Transactions on Information Theory 59 3434–3447.
  • [23] Tsybakov, A. B. (2008)., Introduction to Nonparametric Estimation. Springer Science & Business Media.
  • [24] Yang, Y. and Barron, A. (1999). Information-theoretic determination of minimax rates of convergence., The Annals of Statistics 27 1564–1599.
  • [25] Yatracos, Y. G. (1985). Rates of convergence of minimum distance estimators and Kolmogorov’s entropy., The Annals of Statistics 13 768–774.