The Annals of Statistics

Some Results on False Discovery Rate in Stepwise multiple testing procedures

Sanat K. Sarkar

Full-text: Open access


The concept of false discovery rate (FDR) has been receiving increasing attention by researchers in multiple hypotheses testing. This paper produces some theoretical results on the FDR in the context of stepwise multiple testing procedures with dependent test statistics. It was recently shown by Benjamini and Yekutieli that the Benjamini–Hochberg step-up procedure controls the FDR when the test statistics are positively dependent in a certain sense. This paper strengthens their work by showing that the critical values of that procedure can be used in a much more general stepwise procedure under similar positive dependency. It is also shown that the FDR-controlling Benjamini–Liu step-down procedure originally developed for independent test statistics works even when the test statistics are positively dependent in some sense. An explicit expression for the FDR of a generalized stepwise procedure and an upper bound to the FDR of a step-down procedure are obtained in terms of probability distributions of ordered components of dependent random variables before establishing the main results.

Article information

Ann. Statist., Volume 30, Number 1 (2002), 239-257.

First available in Project Euclid: 5 March 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62H15 62H99

Generalized step-up–step-down procedure Benjamini–Liu step-down procedure positive regression dependency on subset multivariate totally positive of order 2


Sarkar, Sanat K. Some Results on False Discovery Rate in Stepwise multiple testing procedures. Ann. Statist. 30 (2002), no. 1, 239--257. doi:10.1214/aos/1015362192.

Export citation


  • ABRAMOVICH, F. and BENJAMINI, Y. (1996). Adaptive thresholding of wavelets coefficients. Comput. Statist. Data Anal. 22 351-361.
  • ABRAMOVICH, F., BENJAMINI, Y., DONOHO, D. L. and JOHNSTONE, I. M. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
  • BASFORD, K. E. and TUKEY, J. W. (1997). Graphical profiles as an aid to understanding plant breeding experiments. J. Statist. Plann. Inference 57 93-107.
  • BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300.
  • BENJAMINI, Y. and LIU, W. (1999). A step-down multiple hypotheses testing procedures that controls the false discovery rate under independence. J. Statist. Plann. Inference 82 163- 170.
  • BENJAMINI, Y. and YEKUTIELI, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1188.
  • DRIGALENKO, E. I. and ELSTON, R. C. (1997). False discoveries in genome scanning. Gen. Epidem. 14 779-784.
  • FINNER, H. (1999). Stepwise multiple test procedures and control of directional errors. Ann. Statist. 27 274-289.
  • FINNER, H. and ROTERS, M. (1998). Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics. Ann. Statist. 26 505-524.
  • FINNER, H. and ROTERS, M. (1999). Asymptotic comparisons of the critical values of step-down and step-up multiple comparison procedures. J. Statist. Plann. Inference 79 11-30.
  • KARLIN, S. and RINOTT, Y. (1980). Classes of orderings of measures and related correlation inequalities I. J. Multivariate Anal. 10 467-498.
  • LIU, W. (1996). Multiple tests of a non-hierarchical finite family of hypotheses. J. Roy. Statist. Soc. Ser. B 58 455-461.
  • SARKAR, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: a proof of the Simes conjecture. Ann. Statist. 26 494-504.
  • SARKAR, S. K. and CHANG, C.-K. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601-1608.
  • SIMES, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754.
  • TAMHANE, A. C., LIU, W. and DUNNETT, C. W. (1998). A generalized step-up-down multiple test procedure. Canad. J. Statist. 26 353-363.
  • WILLIAMS, V. S. L., JONES, L. V. and TUKEY, J. W. (1999). Controlling error in multiple comparisons with examples from state-to-state differences in educational achievement. J. Edu. Behav. Statist. 24 42-69.