Annals of Statistics

A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing

Wen-Xin Zhou, Koushiki Bose, Jianqing Fan, and Han Liu

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Heavy-tailed errors impair the accuracy of the least squares estimate, which can be spoiled by a single grossly outlying observation. As argued in the seminal work of Peter Huber in 1973 [Ann. Statist. 1 (1973) 799–821], robust alternatives to the method of least squares are sorely needed. To achieve robustness against heavy-tailed sampling distributions, we revisit the Huber estimator from a new perspective by letting the tuning parameter involved diverge with the sample size. In this paper, we develop nonasymptotic concentration results for such an adaptive Huber estimator, namely, the Huber estimator with the tuning parameter adapted to sample size, dimension and the variance of the noise. Specifically, we obtain a sub-Gaussian-type deviation inequality and a nonasymptotic Bahadur representation when noise variables only have finite second moments. The nonasymptotic results further yield two conventional normal approximation results that are of independent interest, the Berry–Esseen inequality and Cramér-type moderate deviation. As an important application to large-scale simultaneous inference, we apply these robust normal approximation results to analyze a dependence-adjusted multiple testing procedure for moderately heavy-tailed data. It is shown that the robust dependence-adjusted procedure asymptotically controls the overall false discovery proportion at the nominal level under mild moment conditions. Thorough numerical results on both simulated and real datasets are also provided to back up our theory.

Article information

Source
Ann. Statist., Volume 46, Number 5 (2018), 1904-1931.

Dates
Received: June 2016
Revised: April 2017
First available in Project Euclid: 17 August 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1534492823

Digital Object Identifier
doi:10.1214/17-AOS1606

Mathematical Reviews number (MathSciNet)
MR3845005

Zentralblatt MATH identifier
06964320

Subjects
Primary: 62F03: Hypothesis testing 62F35: Robustness and adaptive procedures
Secondary: 62J05: Linear regression 62E17: Approximations to distributions (nonasymptotic)

Keywords
Approximate factor model Bahadur representation false discovery proportion heavy-tailed data Huber loss large-scale multiple testing $M$-estimator

Citation

Zhou, Wen-Xin; Bose, Koushiki; Fan, Jianqing; Liu, Han. A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing. Ann. Statist. 46 (2018), no. 5, 1904--1931. doi:10.1214/17-AOS1606. https://projecteuclid.org/euclid.aos/1534492823


Export citation

References

  • Ahn, S. C. and Horenstein, A. R. (2013). Eigenvalue ratio test for the number of factors. Econometrica 81 1203–1227.
    Zentralblatt MATH: 1274.62403
    Digital Object Identifier: doi:10.3982/ECTA8968
  • Barbe, P. and Bertail, P. (1995). The Weighted Bootstrap. Lecture Notes in Statistics 98. Springer, New York.
    Zentralblatt MATH: 0826.62030
  • Barras, L., Scaillet, O. and Wermers, R. (2010). False discoveries in mutual fund performance: Measuring luck in estimated alphas. J. Finance 65 179–216.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B. Stat. Methodol. 57 289–300.
    Zentralblatt MATH: 0809.62014
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
    Zentralblatt MATH: 1041.62061
    Digital Object Identifier: doi:10.1214/aos/1013699998
    Project Euclid: euclid.aos/1013699998
  • Carhart, M. M. (1997). On persistence in mutual fund performance. J. Finance 52 57–82.
  • Catoni, O. (2012). Challenging the empirical mean and empirical variance: A deviation study. Ann. Inst. Henri Poincaré Probab. Stat. 48 1148–1185.
    Zentralblatt MATH: 1282.62070
    Digital Object Identifier: doi:10.1214/11-AIHP454
    Project Euclid: euclid.aihp/1353098444
  • Chang, J., Shao, Q.-M. and Zhou, W.-X. (2016). Cramér-type moderate deviations for Studentized two-sample $U$-statistics with applications. Ann. Statist. 44 1931–1956.
    Mathematical Reviews (MathSciNet): MR3546439
    Digital Object Identifier: doi:10.1214/15-AOS1375
    Project Euclid: euclid.aos/1473685264
  • Clarke, S. and Hall, P. (2009). Robustness of multiple testing procedures against dependence. Ann. Statist. 37 332–358.
    Zentralblatt MATH: 1155.62031
    Digital Object Identifier: doi:10.1214/07-AOS557
    Project Euclid: euclid.aos/1232115937
  • Cont, R. (2001). Empirical properties of asset returns: Stylized facts and statistical issues. Quant. Finance 1 223–236.
  • Delaigle, A., Hall, P. and Jin, J. (2011). Robustness and accuracy of methods for high dimensional data analysis based on Student’s $t$-statistic. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 283–301.
  • Desai, K. H. and Storey, J. D. (2012). Cross-dimensional inference of dependent high-dimensional data. J. Amer. Statist. Assoc. 107 135–151.
    Mathematical Reviews (MathSciNet): MR2949347
    Zentralblatt MATH: 1261.62048
    Digital Object Identifier: doi:10.1080/01621459.2011.645777
  • Efron, B. (2004). Large-scale simultaneous hypothesis testing: The choice of a null hypothesis. J. Amer. Statist. Assoc. 99 96–104.
    Zentralblatt MATH: 1089.62502
    Digital Object Identifier: doi:10.1198/016214504000000089
  • Efron, B. (2007). Correlation and large-scale simultaneous significance testing. J. Amer. Statist. Assoc. 102 93–103.
    Zentralblatt MATH: 1284.62340
    Digital Object Identifier: doi:10.1198/016214506000001211
  • Efron, B., Tibshirani, R., Storey, J. D. and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151–1160.
    Zentralblatt MATH: 1073.62511
    Digital Object Identifier: doi:10.1198/016214501753382129
  • Fama, E. F. (1963). Mandelbrot and the stable paretian hypothesis. J. Bus. 36 420–429.
  • Fan, J., Hall, P. and Yao, Q. (2007). To how many simultaneous hypothesis tests can normal, Student’s $t$ or bootstrap calibration be applied? J. Amer. Statist. Assoc. 102 1282–1288.
    Zentralblatt MATH: 1332.62063
    Digital Object Identifier: doi:10.1198/016214507000000969
  • Fan, J. and Han, X. (2017). Estimation of the false discovery proportion with unknown dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol. 79 1143–1164.
    Zentralblatt MATH: 1373.62272
    Digital Object Identifier: doi:10.1111/rssb.12204
  • Fan, J., Han, X. and Gu, W. (2012). Estimating false discovery proportion under arbitrary covariance dependence. J. Amer. Statist. Assoc. 107 1019–1035.
    Mathematical Reviews (MathSciNet): MR3010891
    Zentralblatt MATH: 06104668
    Digital Object Identifier: doi:10.1080/01621459.2012.713878
  • Fan, J., Li, Q. and Wang, Y. (2017). Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 79 247–265.
  • Finkenstadt, B. and Rootzeén, H. (2003). Extreme Values in Finance, Telecommunications and the Environment. Chapman & Hall, New York.
  • Friguet, C., Kloareg, M. and Causeur, D. (2009). A factor model approach to multiple testing under dependence. J. Amer. Statist. Assoc. 104 1406–1415.
    Zentralblatt MATH: 1205.62071
    Digital Object Identifier: doi:10.1198/jasa.2009.tm08332
  • Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035–1061.
    Zentralblatt MATH: 1092.62065
    Digital Object Identifier: doi:10.1214/009053604000000283
    Project Euclid: euclid.aos/1085408494
  • He, X. and Shao, Q.-M. (1996). A general Bahadur representation of $M$-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608–2630.
    Zentralblatt MATH: 0867.62012
    Digital Object Identifier: doi:10.1214/aos/1032181172
    Project Euclid: euclid.aos/1032181172
  • He, X. and Shao, Q.-M. (2000). On parameters of increasing dimensions. J. Multivariate Anal. 73 120–135.
    Zentralblatt MATH: 0948.62013
    Digital Object Identifier: doi:10.1006/jmva.1999.1873
  • Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Stat. 35 73–101.
    Zentralblatt MATH: 0136.39805
    Digital Object Identifier: doi:10.1214/aoms/1177703732
    Project Euclid: euclid.aoms/1177703732
  • Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799–821.
    Zentralblatt MATH: 0289.62033
    Digital Object Identifier: doi:10.1214/aos/1176342503
    Project Euclid: euclid.aos/1176342503
  • Joly, E. and Lugosi, G. (2016). Robust estimation of $U$-statistics. Stochastic Process. Appl. 126 3760–3773.
    Zentralblatt MATH: 1386.60074
    Digital Object Identifier: doi:10.1016/j.spa.2016.04.021
  • Kosorok, M. R. and Ma, S. (2007). Marginal asymptotics for the “large $p$, small $n$” paradigm: With applications to microarray data. Ann. Statist. 35 1456–1486.
    Zentralblatt MATH: 1123.62005
    Digital Object Identifier: doi:10.1214/009053606000001433
    Project Euclid: euclid.aos/1188405618
  • Langaas, M., Lindqvist, B. H. and Ferkingstad, E. (2005). Estimating the proportion of true null hypotheses, with application to DNA microarray data. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 555–572.
    Zentralblatt MATH: 1095.62037
    Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00515.x
  • Leek, J. T. and Storey, J. D. (2008). A general framework for multiple testing dependence. Proc. Natl. Acad. Sci. USA 105 18718–18723.
    Zentralblatt MATH: 1359.62202
    Digital Object Identifier: doi:10.1073/pnas.0808709105
  • Lehmann, E. L. and Romano, J. P. (2005). Generalizations of the familywise error rate. Ann. Statist. 33 1138–1154.
    Zentralblatt MATH: 1072.62060
    Digital Object Identifier: doi:10.1214/009053605000000084
    Project Euclid: euclid.aos/1120224098
  • Linnik, Ju. V. (1961). On the probability of large deviations for the sums of independent variables. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II 289–306. Univ. California Press, Berkeley, CA.
  • Liu, W. and Shao, Q.-M. (2010). Cramér-type moderate deviation for the maximum of the periodogram with application to simultaneous tests in gene expression time series. Ann. Statist. 38 1913–1935.
    Zentralblatt MATH: 1202.62127
    Digital Object Identifier: doi:10.1214/09-AOS774
    Project Euclid: euclid.aos/1271271282
  • Liu, W. and Shao, Q.-M. (2014). Phase transition and regularized bootstrap in large-scale $t$-tests with false discovery rate control. Ann. Statist. 42 2003–2025.
    Zentralblatt MATH: 1305.62213
    Digital Object Identifier: doi:10.1214/14-AOS1249
    Project Euclid: euclid.aos/1410440632
  • Mammen, E. (1989). Asymptotics with increasing dimension for robust regression with applications to the bootstrap. Ann. Statist. 17 382–400.
    Mathematical Reviews (MathSciNet): MR981457
    Zentralblatt MATH: 0674.62017
    Digital Object Identifier: doi:10.1214/aos/1176347023
    Project Euclid: euclid.aos/1176347023
  • Mandelbrot, B. (1963). The variation of certain speculative prices. J. Bus. 36 394–419.
  • Meinshausen, N. and Rice, J. (2006). Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. Ann. Statist. 34 373–393.
    Zentralblatt MATH: 1091.62059
    Digital Object Identifier: doi:10.1214/009053605000000741
    Project Euclid: euclid.aos/1146576267
  • Nemirovsky, A. S. and Yudin, D. B. (1983). Problem Complexity and Method Efficiency in Optimization. Wiley, New York.
  • Oberthuer, A., Berthold, F., Warnat, P., Hero, B., Kahlert, Y., Spitz, R., Ernestus, K., König, R., Haas, S., Eils, R., Schwab, M., Brors, B., Westermann, F. and Fischer, M. (2006). Customized oligonucleotide microarray gene expression based classification of neuroblastoma patients outperforms current clinical risk stratification. J. Clin. Oncol. 24 5070–5078.
  • Portnoy, S. (1985). Asymptotic behavior of $M$ estimators of $p$ regression parameters when $p^{2}/n$ is large. II. Normal approximation. Ann. Statist. 13 1403–1417.
    Zentralblatt MATH: 0601.62026
    Digital Object Identifier: doi:10.1214/aos/1176349744
    Project Euclid: euclid.aos/1176349744
  • Schwartzman, A. and Lin, X. (2011). The effect of correlation in false discovery rate estimation. Biometrika 98 199–214.
    Zentralblatt MATH: 1215.62071
    Digital Object Identifier: doi:10.1093/biomet/asq075
  • Shi, L., et al. (MAQC Consortium) (2010). The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat. Biotechnol. 28 827–841.
  • Spokoiny, V. and Zhilova, M. (2015). Bootstrap confidence sets under model misspecification. Ann. Statist. 43 2653–2675.
    Zentralblatt MATH: 1327.62179
    Digital Object Identifier: doi:10.1214/15-AOS1355
    Project Euclid: euclid.aos/1444222088
  • Storey, J. D. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 479–498.
    Zentralblatt MATH: 1090.62073
    Digital Object Identifier: doi:10.1111/1467-9868.00346
  • Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B. Stat. Methodol. 66 187–205.
    Zentralblatt MATH: 1061.62110
    Digital Object Identifier: doi:10.1111/j.1467-9868.2004.00439.x
  • Sun, W. and Cai, T. T. (2009). Large-scale multiple testing under dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 393–424.
    Zentralblatt MATH: 1248.62005
    Digital Object Identifier: doi:10.1111/j.1467-9868.2008.00694.x
  • Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge Univ. Press, Cambridge.
  • Yohai, V. J. and Maronna, R. A. (1979). Asymptotic behavior of $M$-estimators for the linear model. Ann. Statist. 7 258–268.
    Mathematical Reviews (MathSciNet): MR520237
    Zentralblatt MATH: 0408.62027
    Digital Object Identifier: doi:10.1214/aos/1176344610
    Project Euclid: euclid.aos/1176344610
  • Zhilova, M. (2016). Non-classical Berry-Esseen inequality and accuracy of the weighted bootstrap. Available at arXiv:1611.02686.
    arXiv: 1611.02686
  • Zhou, W.-X., Bose, K., Fan, J. and Liu, H. (2018). Supplement to “A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing.” DOI:10.1214/17-AOS1606SUPP.

Supplemental materials

  • Supplement to “A new perspective on robust $M$-estimation: Finite sample theory and applications to dependence-adjusted multiple testing”. This supplemental material contains the proofs for the theoretical results in the main text and additional simulation results.
    Digital Object Identifier: doi:10.1214/17-AOS1606SUPP
    Supplemental files are immediately available to subscribers. Non-subscribers gain access to supplemental files with the purchase of the article.

The Institute of Mathematical Statistics

Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more