Multiple hypothesis testing on composite nulls using constrained p-values

Abstract

Multiple hypothesis testing often encounters composite nulls and intractable alternative distributions. In this case, using p-values that are defined as maximum significance levels over all null distributions (“p_max”) often leads to very conservative testing. We propose constructing p-values via maximization under linear constraints imposed by data’s empirical distribution, and show that these p-values allow the false discovery rate (FDR) to be controlled with substantially more power than p_max.