The Annals of Statistics

On a generalized false discovery rate

Sanat K. Sarkar and Wenge Guo

Full-text: Open access


The concept of k-FWER has received much attention lately as an appropriate error rate for multiple testing when one seeks to control at least k false rejections, for some fixed k≥1. A less conservative notion, the k-FDR, has been introduced very recently by Sarkar [Ann. Statist. 34 (2006) 394–415], generalizing the false discovery rate of Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300]. In this article, we bring newer insight to the k-FDR considering a mixture model involving independent p-values before motivating the developments of some new procedures that control it. We prove the k-FDR control of the proposed methods under a slightly weaker condition than in the mixture model. We provide numerical evidence of the proposed methods’ superior power performance over some k-FWER and k-FDR methods. Finally, we apply our methods to a real data set.

Article information

Ann. Statist., Volume 37, Number 3 (2009), 1545-1565.

First available in Project Euclid: 10 April 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons
Secondary: 62H99: None of the above, but in this section

Average power gene expression generalized FDR generalized FWER multiple hypothesis testing oracle k-FDR procedure stepup procedures


Sarkar, Sanat K.; Guo, Wenge. On a generalized false discovery rate. Ann. Statist. 37 (2009), no. 3, 1545--1565. doi:10.1214/08-AOS617.

Export citation


  • [1] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
  • [2] Benjamini, Y., Krieger, A. M. and Yekutieli, D. (2006). Adaptive linear step-up false discovery rate controlling procedures. Biometrika 93 491–507.
  • [3] Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
  • [4] Block, H. W., Savits, T. H. and Shaked, M. (1985). A concept of negative dependence using stochastic ordering. Statist. Probab. Lett. 3 81–86.
  • [5] Dudoit, S., van der Laan, M. and Pollard, K. (2004). Multiple testing: Part I. Single-step procedures for control of general type I error rates. Statist. Appl. Genet. Mol. Biol. 3 Article 13.
  • [6] Guo, W. and Romano, J. P. (2007). A generalized Sidak–Holm procedure and control of generalized error rates under independence. Statist. Appl. Genet. Mol. Biol. 6 Article 3.
  • [7] Hedenfalk, I., Duggan, D., Chen, Y., Radmacher, M., Bittner, M., Simon, R., Meltzer, P., Gusterson, B., Esteller, M., Raffeld, M., Yakhini, Z., Ben-Dor, A., Dougherty, E., Kononen, J., Bubendorf, L., Fehrle, W., Pittaluga, S., Gruvberger, S., Loman, N., Johannsson, O., Olsson, H., Wilfond, B., Sauter, G., Kallioniemi, O., Borg, A. and Trent, J. (2001). Gene-expression profiles in hereditary breast cancer. New Eng. J. Med. 344 539–548.
  • [8] Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800–802.
  • [9] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65–70.
  • [10] Jin, J. and Cai, T. (2007). Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons. J. Amer. Statist. Assoc. 102 495–506.
  • [11] Karlin, S. and Rinott, Y. (1980). Classes of orderings of measures and related correlation inequalities I: Multivariate totally positive distributions. J. Multivariate Anal. 10 467–498.
  • [12] Korn, E., Troendle, T., McShane, L. and Simon, R. (2004). Controlling the number of false discoveries: Application to high-dimensional genomic data. J. Statist. Plann. Inference 124 379–398.
  • [13] Lehmann, E. L. and Romano, J. P. (2005). Generalizations of the familywise error rate. Ann. Statist. 33 1138–1154.
  • [14] Meinshausen, N. and Rice, J. (2006). Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. Ann. Statist. 34 373–393.
  • [15] Romano, J. P. and Shaikh, A. M. (2006). Step-up procedures for control of generalizations of the familywise error rate. Ann. Statist. 34 1850–1873.
  • [16] Romano, J. P. and Wolf, M. (2007). Control of generalized error rates in multiple testing. Ann. Statist. 35 1378–1408.
  • [17] Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239–257.
  • [18] Sarkar, S. K. (2006). False discovery and false nondiscovery rates in single-step multiple testing procedures. Ann. Statist. 34 394–415.
  • [19] Sarkar, S. K. (2007). Step-up procedures controlling generalized FWER and generalized FDR. Ann. Statist. 35 2405–2420.
  • [20] Sarkar, S. K. (2008). Generalizing Simes’ test and Hochberg’s stepup procedure. Ann. Statist. 36 337–363.
  • [21] Sarkar, S. K. (2008). Two-stage stepup procedures controlling FDR. J. Statist. Plann. Inference 138 1072–1084.
  • [22] Sarkar, S. K. and Guo, W. (2006). Procedures controlling generalized false discovery rate. Available at
  • [23] Storey, J. D. (2002). A direct approach to false discovery rates. J. Roy. Statist. Soc. Ser. B 64 479–498.
  • [24] Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. Roy. Statist. Soc. Ser. B 66 187–205.
  • [25] Storey, J. D. and Tibshirani, R. (2003). Statistical significance for genomewide studies. Proc. Natl. Acad. Sci. USA 100 9440–9445.
  • [26] Sun, W. and Cai, T. (2007). Oracle and adaptive compound decision rules for false discovery rate control. J. Amer. Statist. Assoc. 102 901–912.
  • [27] van der Laan, M., Dodoit, S. and Pollard, K. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Stat. Appl. Genet. Mol. Biol. 3 Article 15.
  • [28] Yekutieli, D. and Benjamini, Y. (1999). Resampling based false discovery rate controlling multiple testing procedure for correlated test statistics. J. Statist. Plann. Inference 82 171–196.