The Annals of Statistics

A stochastic process approach to false discovery control

Christopher Genovese and Larry Wasserman

Full-text: Open access


This paper extends the theory of false discovery rates (FDR) pioneered by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300]. We develop a framework in which the False Discovery Proportion (FDP)—the number of false rejections divided by the number of rejections—is treated as a stochastic process. After obtaining the limiting distribution of the process, we demonstrate the validity of a class of procedures for controlling the False Discovery Rate (the expected FDP). We construct a confidence envelope for the whole FDP process. From these envelopes we derive confidence thresholds, for controlling the quantiles of the distribution of the FDP as well as controlling the number of false discoveries. We also investigate methods for estimating the p-value distribution.

Article information

Ann. Statist., Volume 32, Number 3 (2004), 1035-1061.

First available in Project Euclid: 24 May 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62H15: Hypothesis testing 62G10: Hypothesis testing

Multiple testing p-values false discovery rate


Genovese, Christopher; Wasserman, Larry. A stochastic process approach to false discovery control. Ann. Statist. 32 (2004), no. 3, 1035--1061. doi:10.1214/009053604000000283.

Export citation


  • Abramovich, F., Benjamini, Y., Donoho, D. and Johnstone, I. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
  • Benjamini, Y. and Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. J. Educational and Behavioral Statistics 25 60–83.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
  • Efron, B., Tibshirani, R., Storey, J. and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151–1160.
  • Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I errors. Ann. Statist. 30 220–238.
  • Genovese, C. R. and Wasserman, L. (2002). Operating characteristics and extensions of the FDR procedure. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 499–518.
  • Havránek, T. and Chytil, M. (1983). Mechanizing hypotheses formation–-a way for computerized exploratory data analysis? Bull. Internat. Statist. Inst. 50 104–121.
  • Halperin, M., Lan, K. K. G. and Hamdy, M. I. (1988). Some implications of an alternative definition of the multiple comparison problem. Biometrika 75 773–778.
  • Hengartner, N. W. and Stark, P. B. (1995). Finite-sample confidence envelopes for shape-restricted densities. Ann. Statist. 23 525–550.
  • Hochberg, Y. and Benjamini, Y. (1990). More powerful procedures for multiple significance testing. Statistics in Medicine 9 811–818.
  • Hommel, G. and Hoffman, T. (1987). Controlled uncertainty. In Multiple Hypothesis Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154–161. Springer, Heidelberg.
  • Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239–257.
  • Schweder, T. and Spjøtvoll, E. (1982). Plots of $p$-values to evaluate many tests simultaneously. Biometrika 69 493–502.
  • Storey, J. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 479–498.
  • Storey, J. (2003). The positive false discovery rate: A Bayesian interpretation and the $q$-value. Ann. Statist. 31 2013–2035.
  • Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation, and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 187–205.
  • Swanepoel, J. W. H. (1999). The limiting behavior of a modified maximal symmetric $2s$-spacing with applications. Ann. Statist. 27 24–35.
  • van der Vaart, A. (1998). Asymptotic Statistics. Cambridge Univ. Press.