On stepwise control of the generalized familywise error rate

Abstract

A classical approach for dealing with a multiple testing problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of at least one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of a procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In this article, a unified approach is presented for deriving the k-FWER controlling procedures. We first generalize the well-known closure principle in the context of the FWER to the case of controlling the k-FWER. Then, we discuss how to derive the k-FWER controlling stepup procedures based on marginal p-values using this principle. We show that, under certain conditions, generalized closed testing procedures can be reduced to stepup procedures, and any stepup procedure is equivalent to a generalized closed testing procedure. Finally, we generalize the well-known Hommel procedure in two directions, and show that any generalized Hommel procedure is equivalent to a generalized closed testing procedure with the same critical values.