On the existence of powerful p-values and e-values for composite hypotheses

Abstract

Given a composite null $\mathcal{P}$ and composite alternative $\mathcal{Q}$, when and how can we construct a p-value whose distribution is exactly uniform under the null, and stochastically smaller than uniform under the alternative? Similarly, when and how can we construct an e-value whose expectation exactly equals one under the null, but its expected logarithm under the alternative is positive? We answer these basic questions, and other related ones, when $\mathcal{P}$ and $\mathcal{Q}$ are convex polytopes (in the space of probability measures). We prove that such constructions are possible if and only if $\mathcal{Q}$ does not intersect the span of $\mathcal{P}$. If the p-value is allowed to be stochastically larger than uniform under $\mathit{P}\in \mathcal{P}$, and the e-value can have expectation at most one under $\mathit{P}\in \mathcal{P}$, then it is achievable whenever $\mathcal{P}$ and $\mathcal{Q}$ are disjoint. More generally, even when $\mathcal{P}$ and $\mathcal{Q}$ are not polytopes, we characterize the existence of a bounded nontrivial e-variable whose expectation exactly equals one under any $\mathit{P}\in \mathcal{P}$. The proofs utilize recently developed techniques in simultaneous optimal transport. A key role is played by coarsening the filtration: sometimes, no such p-value or e-value exists in the richest data filtration, but it does exist in some reduced filtration, and our work provides the first general characterization of this phenomenon. We also provide an iterative construction that explicitly constructs such processes, and under certain conditions it finds the one that grows fastest under a specific alternative Q. We discuss implications for the construction of composite nonnegative (super)martingales, and end with some conjectures and open problems.