Abstract
Given a set of incomplete observations, we study the nonparametric problem of testing whether data are Missing Completely At Random (MCAR). Our first contribution is to characterise precisely the set of alternatives that can be distinguished from the MCAR null hypothesis. This reveals interesting and novel links to the theory of Fréchet classes (in particular, compatible distributions) and linear programming, that allow us to propose MCAR tests that are consistent against all detectable alternatives. We define an incompatibility index as a natural measure of ease of detectability, establish its key properties and show how it can be computed exactly in some cases and bounded in others. Moreover, we prove that our tests can attain the minimax separation rate according to this measure, up to logarithmic factors. Our methodology does not require any complete cases to be effective, and is available in the package .
Funding Statement
The first author was supported by Engineering and Physical Sciences Research Council (EPSRC) New Investigator Award EP/W016117/1.
The second author was supported by EPSRC Programme grant EP/N031938/1, EPSRC Fellowship EP/P031447/1 and European Research Council Advanced grant 101019498.
Acknowledgments
We thank Danat Duisenbekov and Sean Jaffe for their assistance in speeding up the computational algorithms, as well as the anonymous reviewers for their constructive comments, which helped to improve the paper.
Citation
Thomas B. Berrett. Richard J. Samworth. "Optimal nonparametric testing of Missing Completely At Random and its connections to compatibility." Ann. Statist. 51 (5) 2170 - 2193, October 2023. https://doi.org/10.1214/23-AOS2326
Information