The Annals of Statistics

A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing

Wen-Xin Zhou, Koushiki Bose, Jianqing Fan, and Han Liu

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Heavy-tailed errors impair the accuracy of the least squares estimate, which can be spoiled by a single grossly outlying observation. As argued in the seminal work of Peter Huber in 1973 [Ann. Statist. 1 (1973) 799–821], robust alternatives to the method of least squares are sorely needed. To achieve robustness against heavy-tailed sampling distributions, we revisit the Huber estimator from a new perspective by letting the tuning parameter involved diverge with the sample size. In this paper, we develop nonasymptotic concentration results for such an adaptive Huber estimator, namely, the Huber estimator with the tuning parameter adapted to sample size, dimension and the variance of the noise. Specifically, we obtain a sub-Gaussian-type deviation inequality and a nonasymptotic Bahadur representation when noise variables only have finite second moments. The nonasymptotic results further yield two conventional normal approximation results that are of independent interest, the Berry–Esseen inequality and Cramér-type moderate deviation. As an important application to large-scale simultaneous inference, we apply these robust normal approximation results to analyze a dependence-adjusted multiple testing procedure for moderately heavy-tailed data. It is shown that the robust dependence-adjusted procedure asymptotically controls the overall false discovery proportion at the nominal level under mild moment conditions. Thorough numerical results on both simulated and real datasets are also provided to back up our theory.

Article information

Source
Ann. Statist., Volume 46, Number 5 (2018), 1904-1931.

Dates
Received: June 2016
Revised: April 2017
First available in Project Euclid: 17 August 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1534492823

Digital Object Identifier
doi:10.1214/17-AOS1606

Mathematical Reviews number (MathSciNet)
MR3845005

Zentralblatt MATH identifier
06964320

Subjects
Primary: 62F03: Hypothesis testing 62F35: Robustness and adaptive procedures
Secondary: 62J05: Linear regression 62E17: Approximations to distributions (nonasymptotic)

Keywords
Approximate factor model Bahadur representation false discovery proportion heavy-tailed data Huber loss large-scale multiple testing $M$-estimator

Citation

Zhou, Wen-Xin; Bose, Koushiki; Fan, Jianqing; Liu, Han. A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing. Ann. Statist. 46 (2018), no. 5, 1904--1931. doi:10.1214/17-AOS1606. https://projecteuclid.org/euclid.aos/1534492823


Export citation

References

  • Ahn, S. C. and Horenstein, A. R. (2013). Eigenvalue ratio test for the number of factors. Econometrica 81 1203–1227.
  • Barbe, P. and Bertail, P. (1995). The Weighted Bootstrap. Lecture Notes in Statistics 98. Springer, New York.
  • Barras, L., Scaillet, O. and Wermers, R. (2010). False discoveries in mutual fund performance: Measuring luck in estimated alphas. J. Finance 65 179–216.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B. Stat. Methodol. 57 289–300.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
  • Carhart, M. M. (1997). On persistence in mutual fund performance. J. Finance 52 57–82.
  • Catoni, O. (2012). Challenging the empirical mean and empirical variance: A deviation study. Ann. Inst. Henri Poincaré Probab. Stat. 48 1148–1185.
  • Chang, J., Shao, Q.-M. and Zhou, W.-X. (2016). Cramér-type moderate deviations for Studentized two-sample $U$-statistics with applications. Ann. Statist. 44 1931–1956.
  • Clarke, S. and Hall, P. (2009). Robustness of multiple testing procedures against dependence. Ann. Statist. 37 332–358.
  • Cont, R. (2001). Empirical properties of asset returns: Stylized facts and statistical issues. Quant. Finance 1 223–236.
  • Delaigle, A., Hall, P. and Jin, J. (2011). Robustness and accuracy of methods for high dimensional data analysis based on Student’s $t$-statistic. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 283–301.
  • Desai, K. H. and Storey, J. D. (2012). Cross-dimensional inference of dependent high-dimensional data. J. Amer. Statist. Assoc. 107 135–151.
  • Efron, B. (2004). Large-scale simultaneous hypothesis testing: The choice of a null hypothesis. J. Amer. Statist. Assoc. 99 96–104.
  • Efron, B. (2007). Correlation and large-scale simultaneous significance testing. J. Amer. Statist. Assoc. 102 93–103.
  • Efron, B., Tibshirani, R., Storey, J. D. and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151–1160.
  • Fama, E. F. (1963). Mandelbrot and the stable paretian hypothesis. J. Bus. 36 420–429.
  • Fan, J., Hall, P. and Yao, Q. (2007). To how many simultaneous hypothesis tests can normal, Student’s $t$ or bootstrap calibration be applied? J. Amer. Statist. Assoc. 102 1282–1288.
  • Fan, J. and Han, X. (2017). Estimation of the false discovery proportion with unknown dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol. 79 1143–1164.
  • Fan, J., Han, X. and Gu, W. (2012). Estimating false discovery proportion under arbitrary covariance dependence. J. Amer. Statist. Assoc. 107 1019–1035.
  • Fan, J., Li, Q. and Wang, Y. (2017). Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 79 247–265.
  • Finkenstadt, B. and Rootzeén, H. (2003). Extreme Values in Finance, Telecommunications and the Environment. Chapman & Hall, New York.
  • Friguet, C., Kloareg, M. and Causeur, D. (2009). A factor model approach to multiple testing under dependence. J. Amer. Statist. Assoc. 104 1406–1415.
  • Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035–1061.
  • He, X. and Shao, Q.-M. (1996). A general Bahadur representation of $M$-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608–2630.
  • He, X. and Shao, Q.-M. (2000). On parameters of increasing dimensions. J. Multivariate Anal. 73 120–135.
  • Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Stat. 35 73–101.
  • Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799–821.
  • Joly, E. and Lugosi, G. (2016). Robust estimation of $U$-statistics. Stochastic Process. Appl. 126 3760–3773.
  • Kosorok, M. R. and Ma, S. (2007). Marginal asymptotics for the “large $p$, small $n$” paradigm: With applications to microarray data. Ann. Statist. 35 1456–1486.
  • Langaas, M., Lindqvist, B. H. and Ferkingstad, E. (2005). Estimating the proportion of true null hypotheses, with application to DNA microarray data. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 555–572.
  • Leek, J. T. and Storey, J. D. (2008). A general framework for multiple testing dependence. Proc. Natl. Acad. Sci. USA 105 18718–18723.
  • Lehmann, E. L. and Romano, J. P. (2005). Generalizations of the familywise error rate. Ann. Statist. 33 1138–1154.
  • Linnik, Ju. V. (1961). On the probability of large deviations for the sums of independent variables. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II 289–306. Univ. California Press, Berkeley, CA.
  • Liu, W. and Shao, Q.-M. (2010). Cramér-type moderate deviation for the maximum of the periodogram with application to simultaneous tests in gene expression time series. Ann. Statist. 38 1913–1935.
  • Liu, W. and Shao, Q.-M. (2014). Phase transition and regularized bootstrap in large-scale $t$-tests with false discovery rate control. Ann. Statist. 42 2003–2025.
  • Mammen, E. (1989). Asymptotics with increasing dimension for robust regression with applications to the bootstrap. Ann. Statist. 17 382–400.
  • Mandelbrot, B. (1963). The variation of certain speculative prices. J. Bus. 36 394–419.
  • Meinshausen, N. and Rice, J. (2006). Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. Ann. Statist. 34 373–393.
  • Nemirovsky, A. S. and Yudin, D. B. (1983). Problem Complexity and Method Efficiency in Optimization. Wiley, New York.
  • Oberthuer, A., Berthold, F., Warnat, P., Hero, B., Kahlert, Y., Spitz, R., Ernestus, K., König, R., Haas, S., Eils, R., Schwab, M., Brors, B., Westermann, F. and Fischer, M. (2006). Customized oligonucleotide microarray gene expression based classification of neuroblastoma patients outperforms current clinical risk stratification. J. Clin. Oncol. 24 5070–5078.
  • Portnoy, S. (1985). Asymptotic behavior of $M$ estimators of $p$ regression parameters when $p^{2}/n$ is large. II. Normal approximation. Ann. Statist. 13 1403–1417.
  • Schwartzman, A. and Lin, X. (2011). The effect of correlation in false discovery rate estimation. Biometrika 98 199–214.
  • Shi, L., et al. (MAQC Consortium) (2010). The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat. Biotechnol. 28 827–841.
  • Spokoiny, V. and Zhilova, M. (2015). Bootstrap confidence sets under model misspecification. Ann. Statist. 43 2653–2675.
  • Storey, J. D. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 479–498.
  • Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B. Stat. Methodol. 66 187–205.
  • Sun, W. and Cai, T. T. (2009). Large-scale multiple testing under dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 393–424.
  • Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge Univ. Press, Cambridge.
  • Yohai, V. J. and Maronna, R. A. (1979). Asymptotic behavior of $M$-estimators for the linear model. Ann. Statist. 7 258–268.
  • Zhilova, M. (2016). Non-classical Berry-Esseen inequality and accuracy of the weighted bootstrap. Available at arXiv:1611.02686.
  • Zhou, W.-X., Bose, K., Fan, J. and Liu, H. (2018). Supplement to “A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing.” DOI:10.1214/17-AOS1606SUPP.

Supplemental materials

  • Supplement to “A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing”. This supplemental material contains the proofs for the theoretical results in the main text and additional simulation results.