Open Access
Translator Disclaimer
September 2018 Posterior Predictive p-Values with Fisher Randomization Tests in Noncompliance Settings: Test Statistics vs Discrepancy Measures
Laura Forastiere, Fabrizia Mealli, Luke Miratrix
Bayesian Anal. 13(3): 681-701 (September 2018). DOI: 10.1214/17-BA1062


In randomized experiments with noncompliance, one might wish to focus on compliers rather than on the overall sample. In this vein, Rubin (1998) argued that testing for the complier average causal effect and averaging permutation-based p-values over the posterior distribution of the compliance types could increase power as compared to general intent-to-treat tests. The general scheme is a repeated two-step process: impute missing compliance types and conduct a permutation test with the completed data. In this paper, we explore this idea further, comparing the use of discrepancy measures—which depend on unknown but imputed parameters—to classical test statistics and contrasting different approaches for imputing the unknown compliance types. We also examine consequences of model misspecification in the imputation step, and discuss to what extent this additional modeling undercuts the advantage of permutation tests being model independent. We find that, especially for discrepancy measures, modeling choices can impact both power and validity. In particular, imputing missing compliance types under the null can radically reduce power, but not doing so can jeopardize validity. Fortunately, using covariates predictive of compliance type in the imputation can mitigate these results. We also compare this overall approach to Bayesian model-based tests, that is, tests that are directly derived from posterior credible intervals, under both correct and incorrect model specification.


Download Citation

Laura Forastiere. Fabrizia Mealli. Luke Miratrix. "Posterior Predictive p-Values with Fisher Randomization Tests in Noncompliance Settings: Test Statistics vs Discrepancy Measures." Bayesian Anal. 13 (3) 681 - 701, September 2018.


Published: September 2018
First available in Project Euclid: 17 August 2017

zbMATH: 06989963
MathSciNet: MR3807862
Digital Object Identifier: 10.1214/17-BA1062

Keywords: complier average causal effects (CACE) , noncompliance , permutation testing , posterior predictive p-values (PPPV) , Principal stratification


Vol.13 • No. 3 • September 2018
Back to Top