In this paper, we investigate local permutation tests for testing conditional independence between two random vectors X and Y given Z. The local permutation test determines the significance of a test statistic by locally shuffling samples, which share similar values of the conditioning variables Z, and it forms a natural extension of the usual permutation approach for unconditional independence testing. Despite its simplicity and empirical support, the theoretical underpinnings of the local permutation test remain unclear. Motivated by this gap, this paper aims to establish theoretical foundations of local permutation tests with a particular focus on binning-based statistics. We start by revisiting the hardness of conditional independence testing and provide an upper bound for the power of any valid conditional independence test, which holds when the probability of observing “collisions” in Z is small. This negative result naturally motivates us to impose additional restrictions on the possible distributions under the null and alternate. To this end, we focus our attention on certain classes of smooth distributions and identify provably tight conditions under which the local permutation method is universally valid, that is, it is valid when applied to any (binning-based) test statistic. To complement this result on type I error control, we also show that in some cases, a binning-based statistic calibrated via the local permutation method can achieve minimax optimal power. We also introduce a double-binning permutation strategy, which yields a valid test over less smooth null distributions than the typical single-binning method without compromising much power. Finally, we present simulation results to support our theoretical findings.
This work was partially supported by funding from the NSF Grants DMS-1713003, DMS-2113684 and CIF-1763734, as well as Amazon AI and a Google Research Scholar Award to SB.
IK acknowledges support from the Yonsei University Research Fund of 2021-22-0332 as well as support from the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2022R1A4A1033384).
IK also acknowledges support from EPSRC Grant EP/N031938/1.
The authors would like to thank the reviewers for their constructive comments and suggestions. The authors are also grateful to Richard J. Samworth, Rajen D. Shah and Anton Rask Lundborg for their helpful discussion on Theorem 1.
Ilmun Kim. Matey Neykov. Sivaraman Balakrishnan. Larry Wasserman. "Local permutation tests for conditional independence." Ann. Statist. 50 (6) 3388 - 3414, December 2022. https://doi.org/10.1214/22-AOS2233