- Statist. Sci.
- Volume 32, Number 3 (2017), 331-345.
A Paradox from Randomization-Based Causal Inference
Under the potential outcomes framework, causal effects are defined as comparisons between potential outcomes under treatment and control. To infer causal effects from randomized experiments, Neyman proposed to test the null hypothesis of zero average causal effect (Neyman’s null), and Fisher proposed to test the null hypothesis of zero individual causal effect (Fisher’s null). Although the subtle difference between Neyman’s null and Fisher’s null has caused a lot of controversies and confusions for both theoretical and practical statisticians, a careful comparison between the two approaches has been lacking in the literature for more than eighty years. We fill this historical gap by making a theoretical comparison between them and highlighting an intriguing paradox that has not been recognized by previous researchers. Logically, Fisher’s null implies Neyman’s null. It is therefore surprising that, in actual completely randomized experiments, rejection of Neyman’s null does not imply rejection of Fisher’s null for many realistic situations, including the case with constant causal effect. Furthermore, we show that this paradox also exists in other commonly-used experiments, such as stratified experiments, matched-pair experiments and factorial experiments. Asymptotic analyses, numerical examples and real data examples all support this surprising phenomenon. Besides its historical and theoretical importance, this paradox also leads to useful practical implications for modern researchers.
Statist. Sci. Volume 32, Number 3 (2017), 331-345.
First available in Project Euclid: 1 September 2017
Permanent link to this document
Digital Object Identifier
Ding, Peng. A Paradox from Randomization-Based Causal Inference. Statist. Sci. 32 (2017), no. 3, 331--345. doi:10.1214/16-STS571. https://projecteuclid.org/euclid.ss/1504253116
- Understanding Ding's Apparent Paradox.
- Randomization-Based Tests for "No Treatment Effects".
- Inference from Randomized (Factorial) Experiments.
- An Apparent Paradox Explained.
- Rejoinder: A Paradox from Randomization-Based Causal Inference.
- Supplementary Material. Appendix A.1 gives two useful lemmas for randomized experiments. Appendix A.2 gives the proofs of all the theorems and corollaries in the main text. Appendix A.3 comments on the regression-based causal inference, and establishes a new connection between Rao’s score test and the FRT. Appendix A.4 shows more details about generating Figure 2 in the main text. Appendix A.5 discusses the behaviors of the FRT using the Kolmogorov–Smirnov and Wilcoxon–Mann–Whitney statistics.