Statistical Science

A Generalized Approach to Power Analysis for Local Average Treatment Effects

Kirk Bansak

Full-text: Open access

Enhanced PDF (332 KB)

Abstract

This study introduces a new approach to power analysis in the context of estimating a local average treatment effect (LATE), where the study subjects exhibit noncompliance with treatment assignment. As a result of distributional complications in the LATE context, compared to the simple ATE context, there is currently no standard method of power analysis for the LATE. Moreover, existing methods and commonly used substitutes—which include instrumental variable (IV), intent-to-treat (ITT) and scaled ATE power analyses—require specifying generally unknown variance terms and/or rely upon strong and unrealistic assumptions, thus providing unreliable guidance on the power of tests of the LATE. This study develops a new approach that uses standardized effect sizes to place bounds on the power for the most commonly used estimator of the LATE, the Wald IV estimator, whereby variance terms and distributional parameters need not be specified nor assumed. Instead, in addition to the effect size, sample size and error tolerance parameters, the only other parameter that must be specified by the researcher is the compliance rate. Additional conditions can also be introduced to further narrow the bounds on the power calculation. The result is a generalized approach to power analysis in the LATE context that is simple to implement.

Article information

Source
Statist. Sci., Volume 35, Number 2 (2020), 254-271.

Dates
First available in Project Euclid: 3 June 2020

Permanent link to this document
https://projecteuclid.org/euclid.ss/1591171230

Digital Object Identifier
doi:10.1214/19-STS732

Mathematical Reviews number (MathSciNet)
MR4106604

Keywords
Experimental design local average treatment effects noncompliance principal stratification statistical power

Citation

Bansak, Kirk. A Generalized Approach to Power Analysis for Local Average Treatment Effects. Statist. Sci. 35 (2020), no. 2, 254--271. doi:10.1214/19-STS732. https://projecteuclid.org/euclid.ss/1591171230


Export citation

References

  • Abadie, A., Angrist, J. and Imbens, G. (2002). Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica 70 91–117.
    Zentralblatt MATH: 1104.62331
    Digital Object Identifier: doi:10.1111/1468-0262.00270
  • Angrist, J. D. (1990). Lifetime earnings and the Vietnam era draft lottery: Evidence from Social Security administrative records. Am. Econ. Rev. 80 313–336.
  • Angrist, J. D. and Imbens, G. W. (1995). Two-stage least squares estimation of average causal effects in models with variable treatment intensity. J. Amer. Statist. Assoc. 90 431–442.
    Zentralblatt MATH: 0925.62541
    Digital Object Identifier: doi:10.1080/01621459.1995.10476535
  • Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables. J. Amer. Statist. Assoc. 91 444–455.
    Zentralblatt MATH: 0897.62130
    Digital Object Identifier: doi:10.1080/01621459.1996.10476902
  • Angrist, J. D. and Pischke, J.-S. (2009). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton Univ. Press, Princeton, NJ.
    Zentralblatt MATH: 1159.62090
  • Baiocchi, M., Cheng, J. and Small, D. S. (2014). Instrumental variable methods for causal inference. Stat. Med. 33 2297–2340.
  • Bansak, K. (2020). Supplement to “A Generalized Approach to Power Analysis for Local Average Treatment Effects.” https://doi.org/10.1214/19-STS732SUPP.
  • Bloom, H. S. (1995). Minimum detectable effects: A simple way to report the statistical power of experimental designs. Evaluation Review 19 547–556.
  • Bloom, H. S. (2006). The core analytics of randomized experiments for social research. Technical report, MDRC.
  • Bloom, H. S., Orr, L. L., Bell, S. H., Cave, G., Doolittle, F., Lin, W. and Bos, J. M. (1997). The benefits and costs of JTPA Title II-A programs: Key findings from the National Job Training Partnership Act Study. J. Hum. Resour. 32 549–576.
  • Brion, M.-J. A., Shakhbazov, K. and Visscher, P. M. (2013). Calculating statistical power in Mendelian randomization studies. Int. J. Epidemiol. 42 1497–1501.
  • Cohen, J. (1962). The statistical power of abnormal-social psychological research. J. Abnorm. Soc. Psychol. 65 145–153.
  • Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, NJ.
    Zentralblatt MATH: 0747.62110
  • Duflo, E., Glennerster, R. and Kremer, M. (2007). Using randomization in development economics research: A toolkit. Technical report, Centre for Economic Policy Research, London.
  • Freeman, G., Cowling, B. J. and Schooling, C. M. (2013). Power and sample size calculations for Mendelian randomization studies using one genetic instrument. Int. J. Epidemiol. 42 1157–1163.
  • Gerber, A. S. and Green, D. P. (2000). The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. Am. Polit. Sci. Rev. 94 653–663.
  • Gerber, A. S. and Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. W. W. Norton & Company, New York.
  • Hahn, J., Todd, P. and Van der Klaauw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica 69 201–209.
  • Hainmueller, J., Hangartner, D. and Pietrantuono, G. (2016). Naturalization fosters the long-term political integration of immigrants. Proc. Natl. Acad. Sci. USA 112 12651–12656.
  • Hirano, K., Imbens, G. W., Rubin, D. B. and Zhou, X.-H. (2000). Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics 1 69–88.
    Zentralblatt MATH: 0972.62104
    Digital Object Identifier: doi:10.1093/biostatistics/1.1.69
  • Imbens, G. W. (2014a). Instrumental variables: An econometrician’s perspective. Statist. Sci. 29 323–358.
    Zentralblatt MATH: 1331.62471
    Digital Object Identifier: doi:10.1214/14-STS480
    Project Euclid: euclid.ss/1411437513
  • Imbens, G. (2014b). Rejoinder: “Instrumental variables: An econometrician’s perspective” [MR3264546; MR3264547; MR3264548; MR3264549; MR3264545]. Statist. Sci. 29 375–379.
    Zentralblatt MATH: 1331.62471
    Digital Object Identifier: doi:10.1214/14-STS480
    Project Euclid: euclid.ss/1411437513
  • Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica 62 467–475.
    Zentralblatt MATH: 0800.90648
    Digital Object Identifier: doi:10.2307/2951620
  • Imbens, G. W. and Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. Ann. Statist. 25 305–327.
    Zentralblatt MATH: 0877.62005
    Digital Object Identifier: doi:10.1214/aos/1034276631
    Project Euclid: euclid.aos/1034276631
  • Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge Univ. Press, New York.
    Zentralblatt MATH: 1355.62002
  • Jo, B. (2002). Statistical power in randomized intervention studies with noncompliance. Psychol. Methods 7 178–193.
  • Kitagawa, T. (2014). Instrumental variables before and LATEr [discussion of MR3264545]. Statist. Sci. 29 359–362.
    Zentralblatt MATH: 1331.62475
    Digital Object Identifier: doi:10.1214/14-STS494
    Project Euclid: euclid.ss/1411437514
  • Lipsey, M. W. (1990). Design Sensitivity: Statistical Power for Experimental Research. Sage Publications, Newbury Park, CA.
  • Neyman, J. (1923). Sur les applications de la théorie des probabilités aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych 10 1–51.
  • Pierce, B. L., Ahsan, H. and VanderWeele, T. J. (2011). Power and instrument strength requirements for Mendelian randomization studies using multiple genetic variants. Int. J. Epidemiol. 40 740–752.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688.
  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34–58.
    Zentralblatt MATH: 0383.62021
    Digital Object Identifier: doi:10.1214/aos/1176344064
    Project Euclid: euclid.aos/1176344064
  • Rubin, D. B. (1980). Comment: Randomization analysis of experimental data: The Fisher randomization test. J. Amer. Statist. Assoc. 75 591–593.
  • Rubin, D. B. (1990). Comment on J. Neyman and causal inference in experiments and observational studies: “On the application of probability theory to agricultural experiments. Essay on principles. Section 9” [Ann. Agric. Sci. 10 (1923), 1–51]. Statist. Sci. 5 472–480.
    Zentralblatt MATH: 0955.01559
    Digital Object Identifier: doi:10.1214/ss/1177012032
    Project Euclid: euclid.ss/1177012032
  • Swanson, S. A. and Hernán, M. A. (2014). Think globally, act globally: An epidemiologist’s perspective on instrumental variable estimation [discussion of MR3264545]. Statist. Sci. 29 371–374.
    Zentralblatt MATH: 1331.62487
    Digital Object Identifier: doi:10.1214/14-STS491
    Project Euclid: euclid.ss/1411437517
  • Tsang, R., Colley, L. and Lynd, L. D. (2009). Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials. J. Clin. Epidemiol. 62 609–616.
  • Tversky, A. and Kahneman, D. (1971). The belief in the law of small numbers. Psychol. Bull. 76 105–110.
  • Wang, X., Jiang, Y., Zhang, N. R. and Small, D. S. (2018). Sensitivity analysis and power for instrumental variable studies. Biometrics 74 1150–1160.

Supplemental materials

  • Supplement to “A Generalized Approach to Power Analysis for Local Average Treatment Effects”. The supplementary materials contain the proofs of all propositions along with additional results referenced in the main text.
    Digital Object Identifier: doi:10.1214/19-STS732SUPP
    Supplemental PDF (391 KB)

The Institute of Mathematical Statistics

Statistical Science

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more