Electronic Journal of Statistics

Optimal weighting for false discovery rate control

Etienne Roquain and Mark A. van de Wiel

Full-text: Open access


How to weigh the Benjamini-Hochberg procedure? In the context of multiple hypothesis testing, we propose a new step-wise procedure that controls the false discovery rate (FDR) and we prove it to be more powerful than any weighted Benjamini-Hochberg procedure. Both finite-sample and asymptotic results are presented. Moreover, we illustrate good performance of our procedure in simulations and a genomics application. This work is particularly useful in the case of heterogeneous p-value distributions.

Article information

Electron. J. Statist., Volume 3 (2009), 678-711.

First available in Project Euclid: 13 July 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons
Secondary: 62G10: Hypothesis testing

False discovery rate multiple testing p-value weighting power maximization


Roquain, Etienne; van de Wiel, Mark A. Optimal weighting for false discovery rate control. Electron. J. Statist. 3 (2009), 678--711. doi:10.1214/09-EJS430. https://projecteuclid.org/euclid.ejs/1247490949

Export citation


  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing., J. Roy. Statist. Soc. Ser. B, 57(1):289–300.
  • Benjamini, Y., Krieger, A.M., and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate., Biometrika, 93(3):491–507.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Ann. Statist., 29(4):1165–1188.
  • Blanchard, G. and Roquain, E. (2008). Two simple sufficient conditions for FDR control., Electron. J. Stat., 2:963–992.
  • Blanchard, G. and Roquain, E. (2009). Adaptive false discovery rate control under independence and dependence., J. Mach. Learn. Res. To appear.
  • Efron, B., Tibshirani, R., Storey, J.D., and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment., J. Amer. Statist. Assoc., 96(456):1151–1160.
  • Finner, H., Dickhaus, R., and Roters, M. (2009). On the false discovery rate and an asymptotically optimal rejection curve., Ann. Statist., 37(2):596–618.
  • Gavrilov, Y., Benjamini, Y., and Sarkar, S.K. (2009). An adaptive step-down procedure with proven fdr control under independence., Ann. Statist., 37(2):619–629.
  • Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control., Ann. Statist., 32(3):1035–1061.
  • Genovese, C.R., Roeder, K., and Wasserman, L. (2006). False discovery control with, p-value weighting. Biometrika, 93(3):509–524.
  • Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables., J. Amer. Statist. Assoc., 58:13–30.
  • Holm, S. (1979). A simple sequentially rejective multiple test procedure., Scand. J. Statist., 6(2):65–70.
  • Hyman, E., Kauraniemi, P., Hautaniemi, S., Wolf, M., Mousses, S., Rozenblum, E., Ringnr, M., Sauter, G., Monni, O., Elkahloun, A., Kallioniemi, A., and Kallioniemi, O. (2002). Impact of dna amplification on gene expression patterns in breast cancer., Cancer research, 62(21):6240–5.
  • Kim, K.I. and van de Wiel, M. (2008). Effects of dependence in high-dimensional multiple testing problems., BMC Bioinformatics, 9(1):114.
  • Lehmann, E.L., Romano, J.P., and Shaffer, J.P. (2005). On optimality of stepdown and stepup multiple test procedures., Ann. Statist., 33(3):1084–1108.
  • Muris, J., Ylstra, B., Cillessen, S., Ossenkoppele, G., Kluin-Nelemans, J., Eijk, P., Nota, B., Tijssen, M., de Boer, W., van de Wiel, M., van den Ijssel, P., Jansen, P., de Bruin, P., van Krieken, J., Meijer, G., Meijer, C., and Oudejans, J. (2007). Profiling of apoptosis genes allows for clinical stratification of primary nodal diffuse large B-cell lymphomas., Br. J. Haematol., 136:38–47.
  • Picard, F., Robin, S., Lebarbier, E., and Daudin, J. (2007). A segmentation/clustering model for the analysis of array CGH data., Biometrics, 63:758–766.
  • Romano, J.P., Shaikh, A.M., and Wolf, M. (2008). Control of the false discovery rate under dependence using the bootstrap and subsampling., TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, 17(3):417–442.
  • Roquain, E. (2007)., Exceptional motifs in heterogeneous sequences. Contributions to theory and methodology of multiple testing. PhD thesis, Université Paris XI.
  • Roquain, E. and van de Wiel, M. (2008). Multi-weighting for FDR control. ArXiV preprint, math.ST/0807.4081v1.
  • Rubin, D., Dudoit, S., and van der Laan, M. (2006). A method to increase the power of multiple testing procedures through sample splitting., Stat. Appl. Genet. Mol. Biol., 5:Art. 19, 20 pp. (electronic).
  • Sarkar, S.K. (2002). Some results on false discovery rate in stepwise multiple testing procedures., Ann. Statist., 30(1):239–257.
  • Storey, J.D. (2002). A direct approach to false discovery rates., J. R. Stat. Soc. Ser. B Stat. Methodol., 64(3):479–498.
  • Storey, J.D. (2003). The positive false discovery rate: a Bayesian interpretation and the, q-value. Ann. Statist., 31(6):2013–2035.
  • Storey, J.D. (2007). The optimal discovery procedure: a new approach to simultaneous significance testing., J. R. Stat. Soc. Ser. B Stat. Methodol., 69(3):347–368.
  • van der Vaart, A.W. (1998)., Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge.
  • Wasserman, L. and Roeder, K. (2006). Weighted hypothesis testing. Technical report, Dept. of statistics, Carnegie Mellon, University.