Institute of Mathematical Statistics Lecture Notes - Monograph Series

On stepdown control of the false discovery proportion

Joseph P. Romano and Azeem M. Shaikh

Full-text: Open access


Consider the problem of testing multiple null hypotheses. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate ($FWER$), the probability of even one false rejection. However, if $s$ is large, control of the $FWER$ is so stringent that the ability of a procedure which controls the $FWER$ to detect false null hypotheses is limited. Consequently, it is desirable to consider other measures of error control. We will consider methods based on control of the false discovery proportion ($FDP$) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg (1995) controls $E (FDP)$. Here, we construct methods such that, for any $\gamma$ and $\alpha$, $P \{ FDP > \gamma \} \le \alpha$. Based on $p$-values of individual tests, we consider stepdown procedures that control the $FDP$, without imposing dependence assumptions on the joint distribution of the $p$-values. A greatly improved version of a method given in Lehmann and Romano is derived and generalized to provide a means by which any sequence of nondecreasing constants can be rescaled to ensure control of the $FDP$. We also provide a stepdown procedure that controls the $FDR$ under a dependence assumption.

Chapter information

Javier Rojo, ed., Optimality: The Second Erich L. Lehmann Symposium (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2006), 33-50

First available in Project Euclid: 28 November 2007

Permanent link to this document

Digital Object Identifier

Primary: 62J15: Paired and multiple comparisons

familywise error rate multiple testing $p$-value stepdown procedure

Copyright © 2006, Institute of Mathematical Statistics


Romano, Joseph P.; Shaikh, Azeem M. On stepdown control of the false discovery proportion. Optimality, 33--50, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2006. doi:10.1214/074921706000000383.

Export citation