Open Access
June 2005 Generalizations of the familywise error rate
E. L. Lehmann, Joseph P. Romano
Ann. Statist. 33(3): 1138-1154 (June 2005). DOI: 10.1214/009053605000000084

Abstract

Consider the problem of simultaneously testing null hypotheses H1,…,Hs. The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, particularly if s is large, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which we call the k-FWER. We derive both single-step and stepdown procedures that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the k-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] controls E(FDP). Here, we construct methods such that, for any γ and α, P{FDP>γ}≤α. Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of p-values, while the second is more conservative but holds without any dependence assumptions.

Citation

Download Citation

E. L. Lehmann. Joseph P. Romano. "Generalizations of the familywise error rate." Ann. Statist. 33 (3) 1138 - 1154, June 2005. https://doi.org/10.1214/009053605000000084

Information

Published: June 2005
First available in Project Euclid: 1 July 2005

zbMATH: 1072.62060
MathSciNet: MR2195631
Digital Object Identifier: 10.1214/009053605000000084

Subjects:
Primary: 62J15
Secondary: 62G10

Keywords: familywise error rate , multiple testing , p-value , stepdown procedure

Rights: Copyright © 2005 Institute of Mathematical Statistics

Vol.33 • No. 3 • June 2005
Back to Top