Electronic Journal of Statistics

On improving some adaptive BH procedures controlling the FDR under dependence

Li He and Sanat K. Sarkar

Full-text: Open access


Blanchard and Roquain (2009) presented for the first time methods of adapting the Benjamini-Hochberg (BH) method to data through an estimate of the proportion of true null hypotheses that continue to control the false discovery rate (FDR) under positive dependence in a non-asymptotic setting. However, they are often too conservative to provide a real improvement of the BH method. To obtain adaptive BH methods with proven FDR control improving the original BH method in more situations than what are seen in Blanchard and Roquain (2009), we propose alternative versions of the Blanchard-Roquain methods under some additional assumptions allowing explicit use of pairwise correlations whenever they are available. We offer numerical evidence of improved performances of the proposed alternatives in two scenarios involving test statistics satisfying the positive dependence conditions assumed for the main results.

Article information

Electron. J. Statist., Volume 7 (2013), 2683-2701.

First available in Project Euclid: 11 November 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons
Secondary: 62G10: Hypothesis testing

False discovery rate multiple testing adaptive BH procedures positive dependence pairwise correlations


He, Li; Sarkar, Sanat K. On improving some adaptive BH procedures controlling the FDR under dependence. Electron. J. Statist. 7 (2013), 2683--2701. doi:10.1214/13-EJS855. https://projecteuclid.org/euclid.ejs/1384179699

Export citation


  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing., Journal of the Royal Statistical Society. Series B (Methodological), pages 289–300.
  • Benjamini, Y. and Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics., Journal of Educational and Behavioral Statistics, 25(1):60–83.
  • Benjamini, Y., Krieger, A. M., and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate., Biometrika, 93(3):491–507.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Annals of Statistics, pages 1165–1188.
  • Blanchard, G. and Roquain, E. (2009). Adaptive false discovery rate control under independence and dependence., The Journal of Machine Learning Research, 10:2837–2871.
  • Chen, X. and Doerge, R. (2012). Estimating the proportion of nonzero normal means under certain strong covariance, dependence.
  • Dudoit, S., Shaffer, J. P., and Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments., Statistical Science, pages 71–103.
  • Finner, H. and Gontscharuk, V. (2009). Controlling the familywise error rate with plug-in estimator for the proportion of true null hypotheses., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(5):1031–1048.
  • Finos, L. and Farcomeni, A. (2011). k-fwer control without p-value adjustment, with application to detection of genetic determinants of multiple sclerosis in italian twins., Biometrics, 67(1):174–181.
  • Friguet, C. and Causeur, D. (2011). Estimation of the proportion of true null hypotheses in high-dimensional data under dependence., Computational Statistics & Data Analysis, 55(9):2665–2676.
  • Gavrilov, Y., Benjamini, Y., and Sarkar, S. K. (2009). An adaptive step-down procedure with proven fdr control under independence., The Annals of Statistics, 37(2):619–629.
  • Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control., The Annals of Statistics, 32(3):1035–1061.
  • Guo, W. (2009). A note on adaptive bonferroni and holm procedures under dependence., Biometrika, 96(4):1012–1018.
  • Guo, W. and Rao, M. B. (2010). On stepwise control of the generalized familywise error rate., Electronic Journal of Statistics, 4:472–485.
  • Han, X., Gu, W., and Fan, J. (2010). Control of the false discovery rate under arbitrary covariance dependence. arXiv preprint, arXiv:1012.4397.
  • Hochberg, Y. and Benjamini, Y. (1990). More powerful procedures for multiple significance testing., Statistics in Medicine, 9(7):811–818.
  • Holm, S. (1979). A simple sequentially rejective multiple test procedure., Scandinavian Journal of Statistics, pages 65–70.
  • Jin, J. (2008). Proportion of non-zero normal means: universal oracle equivalences and uniformly consistent estimators., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(3):461–493.
  • Jin, J. and Cai, T. T. (2007). Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons., Journal of the American Statistical Association, 102(478):495–506.
  • Karlin, S. and Rinott, Y. (1980). Classes of orderings of measures and related correlation inequalities. i. multivariate totally positive distributions., Journal of Multivariate Analysis, 10(4):467–498.
  • Langaas, M., Lindqvist, B. H., and Ferkingstad, E. (2005). Estimating the proportion of true null hypotheses, with application to DNA microarray data., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(4):555–572.
  • Leek, J. T. and Storey, J. D. (2008). A general framework for multiple testing dependence., Proceedings of the National Academy of Sciences, 105(48):18718–18723.
  • Lehmann, E. and Romano, J. P. (2005). Generalizations of the familywise error rate., The Annals of Statistics, 33(3):1138–1154.
  • Liang, K. and Nettleton, D. (2012). Adaptive and dynamic adaptive procedures for false discovery rate control and estimation., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(1):163–182.
  • Liu, F. and Sarkar, S. K. (2011). A new adaptive method to control the false discovery rate., Recent Advances in Biostatistics: False Discovery Rates, Survival Analysis, and Related Topics. Series in Biostatistics, 4:3–26.
  • Pounds, S. and Morris, S. W. (2003). Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values., Bioinformatics, 19(10):1236–1242.
  • Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures., The Annals of Statistics, 30(1):239–257.
  • Sarkar, S. K. (2004). Fdr-controlling stepwise procedures and their false negatives rates., Journal of Statistical Planning and Inference, 125(1):119–137.
  • Sarkar, S. K. (2008a). Generalizing simes’ test and hochberg’s stepup procedure., The Annals of Statistics, pages 337–363.
  • Sarkar, S. K. (2008b). On methods controlling the false discovery rate., Sankhyā: The Indian Journal of Statistics, Series A (2008–), pages 135–168.
  • Sarkar, S. K., Guo, W., and Finner, H. (2012). On adaptive procedures controlling the familywise error rate., Journal of Statistical Planning and Inference, 142(1):65–78.
  • Schwartzman, A. (2008). Empirical null and false discovery rate inference for exponential, families.
  • Schweder, T. and Spjotvoll, E. (1982). Plots of p-values to evaluate many tests simultaneously., Biometrika, pages 493–502.
  • Storey, J. D., Taylor, J. E., and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 66(1):187–205.
  • Storey, J. D. and Tibshirani, R. (2003). Statistical significance for genome-wide studies., Proceedings of the National Academy of Sciences of the United States of America, 100:9440–9445.