## Statistical Science

- Statist. Sci.
- Volume 32, Number 3 (2017), 331-345.

### A Paradox from Randomization-Based Causal Inference

#### Abstract

Under the potential outcomes framework, causal effects are defined as comparisons between potential outcomes under treatment and control. To infer causal effects from randomized experiments, Neyman proposed to test the null hypothesis of zero average causal effect (Neyman’s null), and Fisher proposed to test the null hypothesis of zero individual causal effect (Fisher’s null). Although the subtle difference between Neyman’s null and Fisher’s null has caused a lot of controversies and confusions for both theoretical and practical statisticians, a careful comparison between the two approaches has been lacking in the literature for more than eighty years. We fill this historical gap by making a theoretical comparison between them and highlighting an intriguing paradox that has not been recognized by previous researchers. Logically, Fisher’s null implies Neyman’s null. It is therefore surprising that, in actual completely randomized experiments, rejection of Neyman’s null does not imply rejection of Fisher’s null for many realistic situations, including the case with constant causal effect. Furthermore, we show that this paradox also exists in other commonly-used experiments, such as stratified experiments, matched-pair experiments and factorial experiments. Asymptotic analyses, numerical examples and real data examples all support this surprising phenomenon. Besides its historical and theoretical importance, this paradox also leads to useful practical implications for modern researchers.

#### Article information

**Source**

Statist. Sci. Volume 32, Number 3 (2017), 331-345.

**Dates**

First available in Project Euclid: 1 September 2017

**Permanent link to this document**

https://projecteuclid.org/euclid.ss/1504253116

**Digital Object Identifier**

doi:10.1214/16-STS571

**Keywords**

Average null hypothesis Fisher randomization test potential outcome randomized experiment repeated sampling property sharp null hypothesis

#### Citation

Ding, Peng. A Paradox from Randomization-Based Causal Inference. Statist. Sci. 32 (2017), no. 3, 331--345. doi:10.1214/16-STS571. https://projecteuclid.org/euclid.ss/1504253116

#### See also

- Understanding Ding's Apparent Paradox. Digital Object Identifier: doi:10.1214/16-STS582
- Randomization-Based Tests for "No Treatment Effects". Digital Object Identifier: doi:10.1214/16-STS590
- Inference from Randomized (Factorial) Experiments. Digital Object Identifier: doi:10.1214/16-STS600
- An Apparent Paradox Explained. Digital Object Identifier: doi:10.1214/17-STS610
- Rejoinder: A Paradox from Randomization-Based Causal Inference. Digital Object Identifier: doi:10.1214/17-STS571REJ

#### Supplemental materials

- Supplementary Material. Appendix A.1 gives two useful lemmas for randomized experiments. Appendix A.2 gives the proofs of all the theorems and corollaries in the main text. Appendix A.3 comments on the regression-based causal inference, and establishes a new connection between Rao’s score test and the FRT. Appendix A.4 shows more details about generating Figure 2 in the main text. Appendix A.5 discusses the behaviors of the FRT using the Kolmogorov–Smirnov and Wilcoxon–Mann–Whitney statistics.Digital Object Identifier: doi:10.1214/16-STS571SUPPSupplemental files are immediately available to subscribers. Non-subscribers gain access to supplemental files with the purchase of the article.