Open Access
April 2013 Exact and asymptotically robust permutation tests
EunYi Chung, Joseph P. Romano
Ann. Statist. 41(2): 484-507 (April 2013). DOI: 10.1214/13-AOS1090


Given independent samples from $P$ and $Q$, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is $P=Q$. On the other hand, when comparing or testing particular parameters $\theta$ of $P$ and $Q$, such as their means or medians, permutation tests need not be level $\alpha$, or even approximately level $\alpha$ in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability $\alpha$ in finite samples when the underlying distributions are identical. The ideas are broadly applicable and special attention is given to the $k$-sample problem of comparing general parameters, whereby a permutation test is constructed which is exact level $\alpha$ under the hypothesis of identical distributions, but has asymptotic rejection probability $\alpha$ under the more general null hypothesis of equality of parameters. A Monte Carlo simulation study is performed as well. A quite general theory is possible based on a coupling construction, as well as a key contiguity argument for the multinomial and multivariate hypergeometric distributions.


Download Citation

EunYi Chung. Joseph P. Romano. "Exact and asymptotically robust permutation tests." Ann. Statist. 41 (2) 484 - 507, April 2013.


Published: April 2013
First available in Project Euclid: 16 April 2013

zbMATH: 1267.62064
MathSciNet: MR3099111
Digital Object Identifier: 10.1214/13-AOS1090

Primary: 62E20
Secondary: 62G10

Keywords: Behrens–Fisher Problem , coupling , Permutation test

Rights: Copyright © 2013 Institute of Mathematical Statistics

Vol.41 • No. 2 • April 2013
Back to Top