previous :: next

### The control of the false discovery rate in multiple testing under dependency

Yoav Benjamini and Daniel Yekutieli
Source: Ann. Statist. Volume 29, Number 4 (2001), 1165-1188.

#### Abstract

Benjamini and Hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems. A simple procedure was given there as an FDR controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate. We prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses. This condition for positive dependency is general enough to cover many problems of practical interest, including the comparisons of many treatments with a single control, multivariate normal test statistics with positive correlation matrix and multivariate $t$. Furthermore, the test statistics may be discrete, and the tested hypotheses composite without posing special difficulties. For all other forms of dependency, a simple conservative modification of the procedure controls the false discovery rate. Thus the range of problems for which a procedure with proven FDR control can be offered is greatly increased.

First Page:
Primary Subjects: 62J15, 62G30, 47N30
Full-text: Open access

Permanent link to this document: http://projecteuclid.org/euclid.aos/1013699998
Mathematical Reviews number (MathSciNet): MR1869245
Digital Object Identifier: doi:10.1214/aos/1013699998
Zentralblatt MATH identifier: 01829051

### References

Abramovich, F. and Benjamini, Y. (1996). Adaptive thresholding of wavelet coefficients. Comput. Statist. Data Anal. 22 351-361.
Mathematical Reviews (MathSciNet): MR97e:62052
Barinaga, M. (1994). From fruit flies, rats, mice: evidence of genetic influence. Science 264 1690-1693.
Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300.
Mathematical Reviews (MathSciNet): MR96d:62143
Benjamini, Y. and Hochberg, Y. (1997). Multiple hypotheses testing with weights. Scand. J. Statist. 24 407-418.
Mathematical Reviews (MathSciNet): MR99d:62046
Digital Object Identifier: doi:10.1111/1467-9469.00072
Benjamini, Y. and Hochberg, Y. (2000). The adaptive control of the false discovery rate in multiple hypotheses testing. J. Behav. Educ. Statist. 25 60-83.
Benjamini, Y., Hochberg, Y. and Kling, Y. (1993). False discovery rate control in pairwise comparisons. Working Paper 93-2, Dept. Statistics and O.R., Tel Aviv Univ.
Benjamini, Y., Hochberg, Y. and Kling, Y. (1997). False discovery rate control in multiple hypotheses testing using dependent test statistics. Research Paper 97-1, Dept. Statistics and O.R., Tel Aviv Univ.
Benjamini, Y. and Wei, L. (1999). A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. J. Statist. Plann. Inference 82 163-170.
Mathematical Reviews (MathSciNet): MR1736441
Zentralblatt MATH: 1063.62558
Digital Object Identifier: doi:10.1016/S0378-3758(99)00040-3
Chang, C. K., Rom, D. M. and Sarkar, S. K. (1996). A modified Bonferroni procedure for repeated significance testing. Technical Report 96-01, Temple Univ.
Eaton, M. L. (1986). Lectures on topics in probability inequalities. CWI Tract 35.
Mathematical Reviews (MathSciNet): MR889671
Zentralblatt MATH: 0622.60024
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800-803.
Mathematical Reviews (MathSciNet): MR90d:62037
Zentralblatt MATH: 0661.62067
Digital Object Identifier: doi:10.1093/biomet/75.4.800
Hochberg, Y. and Hommel, G. (1998). Step-up multiple testing procedures. Encyclopedia Statist. Sci. (Supp.) 2.
Hochberg, Y. and Rom, D. (1995). Extensions of multiple testing procedures based on Simes' test. J. Statist. Plann. Inference 48 141-152.
Mathematical Reviews (MathSciNet): MR97c:62038
Zentralblatt MATH: 0851.62054
Digital Object Identifier: doi:10.1016/0378-3758(95)00005-T
Hochberg, Y. and Tamhane, A. (1987). Multiple Comparison Procedures. Wiley, New York.
Mathematical Reviews (MathSciNet): MR89c:62125
Holland, P. W. and Rosenbaum, P. R. (1986). Conditional association and unidimensionality in monotone latent variable models. Ann. Statist. 14 1523-1543.
Mathematical Reviews (MathSciNet): MR88g:62115
Zentralblatt MATH: 0625.62102
Digital Object Identifier: doi:10.1214/aos/1176350174
Project Euclid: euclid.aos/1176350174
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist 6 65-70.
Mathematical Reviews (MathSciNet): MR81i:62042
Hommel, G. (1988). A stage-wise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75 383-386.
Hsu, J. (1996). Multiple Comparisons Procedures. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1629127
Karlin, S. and Rinott, Y. (1980). Classes of orderings of measures and related correlation inequalities I. Multivariate totally positive distributions. J. Multivariate Statist. 10 467-498.
Zentralblatt MATH: 0469.60006
Mathematical Reviews (MathSciNet): MR599685
Digital Object Identifier: doi:10.1016/0047-259X(80)90065-2
Karlin, S. and Rinott, Y. (1981). Total positivity properties of absolute value multinormal variable with applications to confidence interval estimates and related probabilistic inequalities. Ann. Statist. 9 1035-1049.
Mathematical Reviews (MathSciNet): MR83c:62081
Zentralblatt MATH: 0477.62035
Digital Object Identifier: doi:10.1214/aos/1176345583
Project Euclid: euclid.aos/1176345583
Lander E. S. and Botstein D. (1989). Mapping Mendelian factors underlying quantitative traits using RFLP linkage maps. Genetics 121 185-190.
Lander, E. S. and Kruglyak L. (1995). Genetic dissection of complex traits: guidelines for interpreting and reporting linkage results. Nature Genetics 11 241-247.
Lehmann, E. L. (1966). Some concepts of dependence. Ann. Math. Statist. 37 1137-1153.
Mathematical Reviews (MathSciNet): MR34:2101
Zentralblatt MATH: 0146.40601
Digital Object Identifier: doi:10.1214/aoms/1177699260
Project Euclid: euclid.aoms/1177699260
Needleman, H., Gunnoe, C., Leviton, A., Reed, R., Presie, H., Maher, C. and Barret, P. (1979). Deficits in psychologic and classroom performance of children with elevated dentine lead levels. New England J. Medicine 300 689-695.
Paterson, A. H. G., Powles, T. J., Kanis, J. A., McCloskey, E., Hanson, J. and Ashley, S. (1993). Double-blind controlled trial of oral clodronate in patients with bone metastases from breastcancer. J. Clinical Oncology 1 59-65.
Rosenbaum, P. R. (1984). Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika 49 425-436.
Mathematical Reviews (MathSciNet): MR86f:62187
Zentralblatt MATH: 0569.62097
Digital Object Identifier: doi:10.1007/BF02306030
Sarkar, T. K. (1969). Some lower bounds of reliability. Technical Report, 124, Dept. Operation Research and Statistics, Stanford Univ.
Sarkar, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: a proof of Simes' conjecture. Ann. Statist. 26 494-504.
Mathematical Reviews (MathSciNet): MR99m:62024
Zentralblatt MATH: 0929.62065
Digital Object Identifier: doi:10.1214/aos/1028144846
Project Euclid: euclid.aos/1028144846
Sarkar, S. K. and Chang, C. K. (1997). The Simes method for multiple hypotheses testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601-1608.
Mathematical Reviews (MathSciNet): MR1615269
Zentralblatt MATH: 0912.62079
Digital Object Identifier: doi:10.2307/2965431
Seeger, (1968). A note on a method for the analysis of significances en mass. Technometrics 10 586-593. Sen, P. K. (1999a). Some remarks on Simes-type multiple tests of significance. J. Statist. Plann. Inference, 82 139-145. Sen, P. K. (1999b). Multiple comparisons in interim analysis. J. Statist. Plann. Inference 82 5-23.
Shaffer, J. P. (1995). Multiple hypotheses-testing. Ann. Rev. Psychol. 46 561-584.
Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754.
Mathematical Reviews (MathSciNet): MR88d:62043
Zentralblatt MATH: 0613.62067
Digital Object Identifier: doi:10.1093/biomet/73.3.751
Steel, R. G. D. and Torrie, J. H. (1980). Principles and Procedures of Statistics: A Biometrical Approach, 2nd ed. McGraw-Hill, New York.
Tamhane, A. C. (1996). Multiple comparisons. In Handbook of Statistics (S. Ghosh and C. R. Rao, eds.) 13 587-629. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet): MR1492580
Tamhane, A. C. and Dunnett, C. W. (1999). Stepwise multiple test procedures with biometric applications. J. Statist. Plann. Inference 82 55-68.
Mathematical Reviews (MathSciNet): MR1736432
Zentralblatt MATH: 0979.62049
Digital Object Identifier: doi:10.1016/S0378-3758(99)00031-2
Troendle, J. (2000). Stepwise normal theory tests procedures controlling the false discovery rate. J. Statist. Plann. Inference 84 139-158.
Mathematical Reviews (MathSciNet): MR1747501
Zentralblatt MATH: 1131.62310
Digital Object Identifier: doi:10.1016/S0378-3758(99)00145-7
Wassmer, G., Reitmer, P., Kieser, M. and Lehmacher, W. (1999). Procedures for testing multiple endpoints in clinical trials: an overview. J. Statist. Plann. Inference 82 69-81.
Mathematical Reviews (MathSciNet): MR1736433
Zentralblatt MATH: 0979.62090
Digital Object Identifier: doi:10.1016/S0378-3758(99)00032-4
Weller, J. I., Song, J. Z., Heyen, D. W., Lewin, H. A. and Ron, M. (1998). A new approach to the problem of multiple comparison in the genetic dissection of complex traits. Genetics 150 1699-1706.
Westfall, P. H. and Young, S. S. (1993). Resampling Based Multiple Testing, Wiley, New York.
Williams, V. S. L., Jones, L. V. and Tukey, J. W. (1999). Controlling error in multiple comparisons, with special attention to the National Assessment of Educational Progress. J. Behav. Educ. Statist. 24 42-69.
Yekutieli, D. and Benjamini, Y. (1999). A resampling based false discovery rate controlling multiple test procedure. J. Statist. Plann. Inference 82 171-196.
Mathematical Reviews (MathSciNet): MR1736442
Zentralblatt MATH: 1063.62563
Digital Object Identifier: doi:10.1016/S0378-3758(99)00041-5
previous :: next