Generalizing Simes’ test and Hochberg’s stepup procedure

Abstract

In a multiple testing problem where one is willing to tolerate a few false rejections, procedure controlling the familywise error rate (FWER) can potentially be improved in terms of its ability to detect false null hypotheses by generalizing it to control the k-FWER, the probability of falsely rejecting at least k null hypotheses, for some fixed k>1. Simes’ test for testing the intersection null hypothesis is generalized to control the k-FWER weakly, that is, under the intersection null hypothesis, and Hochberg’s stepup procedure for simultaneous testing of the individual null hypotheses is generalized to control the k-FWER strongly, that is, under any configuration of the true and false null hypotheses. The proposed generalizations are developed utilizing joint null distributions of the k-dimensional subsets of the p-values, assumed to be identical. The generalized Simes’ test is proved to control the k-FWER weakly under the multivariate totally positive of order two (MTP₂) condition [J. Multivariate Analysis 10 (1980) 467–498] of the joint null distribution of the p-values by generalizing the original Simes’ inequality. It is more powerful to detect k or more false null hypotheses than the original Simes’ test when the p-values are independent. A stepdown procedure strongly controlling the k-FWER, a version of generalized Holm’s procedure that is different from and more powerful than [Ann. Statist. 33 (2005) 1138–1154] with independent p-values, is derived before proposing the generalized Hochberg’s procedure. The strong control of the k-FWER for the generalized Hochberg’s procedure is established in situations where the generalized Simes’ test is known to control its k-FWER weakly.