Consistency of invariance-based randomization tests

Abstract

Invariance-based randomization tests—such as permutation tests, rotation tests, or sign changes—are an important and widely used class of statistical methods. They allow drawing inferences under weak assumptions on the data distribution. Most work focuses on their type I error control properties, while their consistency properties are much less understood.

We develop a general framework to study the consistency of invariance-based randomization tests, assuming the data is drawn from a signal-plus-noise model. We allow the transforms (e.g., permutations or rotations) to be general compact topological groups, such as rotation groups, acting by linear group representations. We study test statistics with a generalized subadditivity property.

We apply our framework to a number of fundamental and highly important problems in statistics, including sparse vector detection, testing for low-rank matrices in noise, sparse detection in linear regression, and two-sample testing. Comparing with minimax lower bounds we develop, we find perhaps surprisingly that in some cases, randomization tests detect signals at the minimax optimal rate.