Electronic Journal of Statistics

Inequalities for the false discovery rate (FDR) under dependence

Philipp Heesen and Arnold Janssen

Full-text: Open access


Inequalities are key tools to prove FDR control of a multiple test. The present paper studies upper and lower bounds for the FDR under various dependence structures of $p$-values, namely independence, reverse martingale dependence and positive regression dependence on the subset (PRDS) of true null hypotheses. The inequalities are based on exact finite sample formulas which are also of interest for independent uniformly distributed $p$-values under the null. As applications the asymptotic worst case FDR of step up and step down tests coming from an non-decreasing rejection curve is established. In addition, new step up tests are established and necessary conditions for the FDR control are discussed. The reverse martingale models yield sharper FDR results than the PRDS models. Already in certain multivariate normal dependence models the familywise error rate of the Benjamini Hochberg step up test can be different from the desired level $\alpha$. The second part of the paper is devoted to adaptive step up tests under dependence. The well-known Storey estimator is modified so that the corresponding step up test has finite sample control for various block wise dependent $p$-values. These results may be applied to dependent genome data. Within each chromosome the $p$-values may be reverse martingale dependent while the chromosomes are independent.

Article information

Electron. J. Statist., Volume 9, Number 1 (2015), 679-716.

First available in Project Euclid: 2 April 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G10: Hypothesis testing
Secondary: 62G20: Asymptotic properties

False discovery rate (FDR) inequalities multiple hypotheses testing PRDS reverse martingale dependence blockwise dependence adaptive Benjamini Hochberg methods p-values Storey test


Heesen, Philipp; Janssen, Arnold. Inequalities for the false discovery rate (FDR) under dependence. Electron. J. Statist. 9 (2015), no. 1, 679--716. doi:10.1214/15-EJS1016. https://projecteuclid.org/euclid.ejs/1427990069

Export citation


  • [1] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach in multiple testing., J. Roy. Stat. Soc. B 57 289–300.
  • [2] Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Ann. Stat. 10 1165–1188.
  • [3] Benjamini, Y., Krieger, A. M. and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate., Biometrika 93(3) 491–507.
  • [4] Blanchard, G. and Roquain, E. (2009). Adaptive false discovery rate control under independence and dependence., J. Mach. Learn. Res. 29 2837–2871.
  • [5] Blanchard, G. and Roquain, E. (2008). Two simple sufficient conditions for FDR control., Electron. J. Stat. 2 963–992.
  • [6] Efron, B., Tibshirani, R. and Storey, D. (2001). Empirical Bayes analysis of a microarray experiment., J. Am. Stat. Soc. 96 1151–1160.
  • [7] Finner, H., Gontscharuk, V., Dickhaus, T. (2012). False discovery rate control of step-up-down tests with special emphasis on the asymptotically optimal rejections curve., Scand. J. Stat. 39 382–397.
  • [8] Finner, H. and Gontscharuk, V. (2009). Controlling the familywise error rate with plug-in estimator for the proportion of true null hypotheses., J. R. Stat. Soc. B 71 1031–1048.
  • [9] Finner, H., Dickhaus, T. and Roters, M. (2009). On the false discovery rate and an asymptotically optimal rejection curve., Ann. Stat. 37 596–618.
  • [10] Finner, H. and Roters, M. (2001). On the false discovery rate and expected type I errors., Biometrical J. 43 985–1005.
  • [11] Gavrilov, Y., Benjamini, Y. and Sarkar, S. K. (2009). An adaptive step-down procedure with proven FDR control under independence., Ann. Stat. 37 619–629.
  • [12] Gontscharuk, V. (2010). Asymptotic and exact results on FWER and FDR in multiple hypothesis testing. Diss. PhD thesis, Heinrich-Heine-Universität Düsseldorf., http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=16990.
  • [13] Guo, W., Rao, M. B. (2008). On optimality of the Benjamini-Hochberg procedure for the false discovery rate., Statist. Probab. Lett. 78(14) 2024–2030.
  • [14] Guo, W. and Sarkar, S. K. (2013). Adaptive Controls of FWER and FDR Under Block Dependence., Unpublished manuscript. http://web.njit.edu/~wguo/Guo & Sarkar 2012.pdf.
  • [15] Heesen, P. and Janssen, A. (2014). Dynamic adaptive multiple tests with finite sample FDR control., arXiv:1410.6296v1, October 2014.
  • [16] Heesen, P. (2014). Adaptive step up tests for the false discovery rate (FDR) under independence and dependence. Ph.D. thesis, Heinrich-Heine-Universität Düsseldorf, Germany., http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=33047.
  • [17] Liang, K. and Nettleton, D. (2012). Adaptive and dynamic adaptive procedures for false discovery rate control and estimation., J. Roy. Stat. Soc. B 74(1) 163–182.
  • [18] Marshall, A. W. and Olkin, I. (1967). A multivariate exponential distribution., J. Amer. Stat. Assoc. 62(317) 30–44.
  • [19] Meskaldji, D. E., Thiran, J.-P. and Morgenthaler, S. (2013). A comprehensive error rate for multiple testing., arXiv:1112.4519v4, July 2013.
  • [20] Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures., Ann. Statist. 30 239–257.
  • [21] Sarkar, S. K. (2008). On methods controlling the false discovery rate., Sankhya Series A, 70 135–168.
  • [22] Scheer, M. (2013). Controlling the Number of False Rejections in Multiple Hypotheses Testing. Diss. PhD thesis, Heinrich- Heine-Universität Düsseldorf., http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=23691.
  • [23] Storey, J. D. (2002). A direct approach to false discovery rates., J. Roy. Stat. Soc. B 64 479–498.
  • [24] Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach., J. Roy. Stat. Soc. B 66 187–205.
  • [25] Zeisel, A., Zuk, O. and Domany, E. (2011). FDR control with adaptive procedures and FDR monotonicity., Ann. Appl. Stat. 5(2A) 943–968.