The Annals of Statistics

On the Benjamini–Hochberg method

J. A. Ferreira and A. H. Zwinderman

Full-text: Open access


We investigate the properties of the Benjamini–Hochberg method for multiple testing and of a variant of Storey’s generalization of it, extending and complementing the asymptotic and exact results available in the literature. Results are obtained under two different sets of assumptions and include asymptotic and exact expressions and bounds for the proportion of rejections, the proportion of incorrect rejections out of all rejections and two other proportions used to quantify the efficacy of the method.

Article information

Ann. Statist. Volume 34, Number 4 (2006), 1827-1849.

First available in Project Euclid: 3 November 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons 62G30: Order statistics; empirical distribution functions 60F05: Central limit and other weak theorems

Multiple testing goodness of fit empirical distributions false discovery rate


Ferreira, J. A.; Zwinderman, A. H. On the Benjamini–Hochberg method. Ann. Statist. 34 (2006), no. 4, 1827--1849. doi:10.1214/009053606000000425.

Export citation


  • Abramovich, F., Benjamini, Y., Donoho, D. and Johnstone, I. M. (2006). Adapting to known sparsity by controlling the false discovery rate. Ann. Statist. 34 584--653.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289--300.
  • Benjamini, Y. and Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. J. Educational and Behavioral Statistics 25 60--83.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependence. Ann. Statist. 29 1165--1188.
  • Delongchamp, R., Bowyer, J., Chen, J. and Kodell, R. (2004). Multiple-testing strategy for analyzing cDNA array data on gene expression. Biometrics 60 774--782.
  • Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962--994.
  • Dudoit, S., Schaffer, J. and Boldrick, J. (2003). Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 71--103.
  • Fan, J., Tam, P., Vande Woude, G. and Ren, Y. (2004). Normalization and analysis of cDNA microarrays using within-array replications applied to neuroblastoma cell response to a cytokine. Proc. Natl. Acad. Sci. USA 101 1135--1140.
  • Genovese, C. and Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 499--517.
  • Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035--1061.
  • Karlin, S. and Taylor, H. (1981). A Second Course in Stochastic Processes. Academic Press, New York.
  • McLachlan, G., Do, K.-A. and Ambroise, C. (2004). Analyzing Microarray Gene Expression Data. Wiley, New York.
  • Nakas, C., Yiannoutsos, C., Bosch, R. and Moyssiadis, C. (2003). Assessment of diagnostic markers by goodness-of-fit tests. Statistics in Medicine 22 2503--2513.
  • Reiner, A., Yekutieli, D. and Benjamini, Y. (2003). Identifying differentially expressed genes using false discovery rate controlling procedures. Bioinformatics 19 368--375.
  • Sarkar, S. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239--257.
  • Sarkar, S. (2004). FDR-controlling stepwise procedures and their false negative rates. J. Statist. Plann. Inference 125 119--137.
  • Sarkar, S. (2006). False discovery and false non-discovery rates in single-step multiple testing procedures. Ann. Statist. 34 394--415.
  • Storey, J. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 479--498.
  • Storey, J., Taylor, J. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach J. R. Stat. Soc. Ser. B Stat. Methodol. 66 187--205.
  • Tsai, C.-A., Hsueh, H.-M. and Chen, J. (2003). Estimation of false discovery rates in multiple testing: Application to gene microarray data. Biometrics 59 1071--1081.
  • Tucker, H. (1953). A generalization of the Glivenko--Cantelli theorem. Ann. Math. Statist. 30 828--830.
  • Tusher, V., Tibshirani, R. and Chu, G. (2001). Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. USA 98 5116--5121.