The Annals of Applied Statistics

Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects

Trang Quynh Nguyen, Cyrus Ebnesajjad, Stephen R. Cole, and Elizabeth A. Stuart

Full-text: Access denied (no subscription detected) We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

In the presence of treatment effect heterogeneity, the average treatment effect (ATE) in a randomized controlled trial (RCT) may differ from the average effect of the same treatment if applied to a target population of interest. If all treatment effect moderators are observed in the RCT and in a dataset representing the target population, then we can obtain an estimate for the target population ATE by adjusting for the difference in the distribution of the moderators between the two samples. This paper considers sensitivity analyses for two situations: (1) where we cannot adjust for a specific moderator $V$ observed in the RCT because we do not observe it in the target population; and (2) where we are concerned that the treatment effect may be moderated by factors not observed even in the RCT, which we represent as a composite moderator $U$. In both situations, the outcome is not observed in the target population. For situation (1), we offer three sensitivity analysis methods based on (i) an outcome model, (ii) full weighting adjustment and (iii) partial weighting combined with an outcome model. For situation (2), we offer two sensitivity analyses based on (iv) a bias formula and (v) partial weighting combined with a bias formula. We apply methods (i) and (iii) to an example where the interest is to generalize from a smoking cessation RCT conducted with participants of alcohol/illicit drug use treatment programs to the target population of people who seek treatment for alcohol/illicit drug use in the US who are also cigarette smokers. In this case a treatment effect moderator is observed in the RCT but not in the target population dataset.

Article information

Source
Ann. Appl. Stat. Volume 11, Number 1 (2017), 225-247.

Dates
Received: October 2015
Revised: November 2016
First available in Project Euclid: 8 April 2017

Permanent link to this document
http://projecteuclid.org/euclid.aoas/1491616879

Digital Object Identifier
doi:10.1214/16-AOAS1001

Keywords
Sensitivity analysis generalization treatment effect heterogeneity unobserved moderator unobserved effect modifier

Citation

Nguyen, Trang Quynh; Ebnesajjad, Cyrus; Cole, Stephen R.; Stuart, Elizabeth A. Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects. Ann. Appl. Stat. 11 (2017), no. 1, 225--247. doi:10.1214/16-AOAS1001. http://projecteuclid.org/euclid.aoas/1491616879.


Export citation

References

  • Arah, O. A., Chiba, Y. and Greenland, S. (2008). Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. Ann. Epidemiol. 18 637–646.
  • Cole, S. R. and Stuart, E. A. (2010). Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. Am. J. Epidemiol. 172 107–115.
  • Cornfield, J., Haenszel, W., Hammond, E. C., Lilienfeld, A. M., Shimkin, M. B. and Wynder, E. L. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. J. Natl. Cancer Inst. 22 173–203.
  • Ding, P. and VanderWeele, T. J. (2014). Generalized Cornfield conditions for the risk difference. Biometrika 101 971–977.
    Zentralblatt MATH: 1306.62225
    Digital Object Identifier: doi:10.1093/biomet/asu030
  • Ding, P. and VanderWeele, T. J. (2015). The differential geometry of homogeneity spaces ccross effect scales. Available at arXiv:1510.08534v2.
    arXiv: arXiv:1510.08534v2
  • Ding, P. and VanderWeele, T. J. (2016). Sensitivity analysis without assumptions. Epidemiology 27 368–377.
  • Gastwirth, J. L., Krieger, A. M. and Rosenbaum, P. R. (1998). Dual and simultaneous sensitivity analysis for matched pairs. Biometrika 85 907–920.
    Zentralblatt MATH: 0921.62053
    Digital Object Identifier: doi:10.1093/biomet/85.4.907
  • Greenhouse, J. B., Kaizar, E. E., Kelleher, K., Seltman, H. and Gardner, W. (2008). Generalizing from clinical trial data: A case study. The risk of suicidality among pediatric antidepressant users. Stat. Med. 27 1801–1813.
  • Greenland, S. (1996). Basic methods for sensitivity analysis of biases. Int. J. Epidemiol. 25 1107–1116.
  • Hong, G. (2010). Ratio of mediator probability weighting for estimating natural direct and indirect effects. Proc. Am. Stat. Assoc. Biom. Sect. 2401–2415.
  • Kern, H. L., Stuart, E. A., Hill, J. L. and Green, D. P. (2016). Assessing methods for generalizing experimental impact estimates to target samples. J. Res. Educ. Eff. 9 103–127.
  • Nguyen, T. Q., Ebnesajjad, C., Cole, S. R. and Stuart, E. A. (2017). Supplement to “Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects.” DOI:10.1214/16-AOAS1001SUPPA, DOI:10.1214/16-AOAS1001SUPPB.
    Zentralblatt MATH: 06725403
  • Olsen, R. B., Orr, L. L., Bell, S. H. and Stuart, E. A. (2013). External validity in policy evaluations that choose sites purposively. J. Policy Anal. Manage. 32 107–121.
  • Reid, M. S., Fallon, B., Sonne, S., Flammino, F., Nunes, E. V., Jiang, H., Kuoniotis, E., Lima, J., Brady, R., Burgess, C., Arfken, C., Pihlgren, E., Giordano, L., Starosta, A., Robison, J. and Rotrosen, J. (2008). Smoking cessation treatment in community-based substance abuse rehabilitation programs. J. Subst. Abuse Treat. 35 68–77.
  • Rosenbaum, P. R. (1987). Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika 74 13–26.
    Zentralblatt MATH: 0605.62130
    Digital Object Identifier: doi:10.1093/biomet/74.1.13
  • Rosenbaum, P. R. and Rubin, D. B. (1983a). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J. R. Stat. Soc. Ser. B. Stat. Methodol. 45 212–218.
  • Rosenbaum, P. R. and Rubin, D. B. (1983b). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41–55.
    Mathematical Reviews (MathSciNet): MR742974
    Zentralblatt MATH: 0522.62091
    Digital Object Identifier: doi:10.1093/biomet/70.1.41
  • Schneeweiss, S. (2006). Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol. Drug Saf. 15 291–303.
  • Stuart, E. A., Bradshaw, C. P. and Leaf, P. J. (2015). Assessing the generalizability of randomized trial results to target populations. Prev. Sci. 16 475–485.
  • Stuart, E. A. and Rhodes, A. (2016). Generalizing treatment effect estimates from sample to population: A case study in the difficulties of finding sufficient data. Eval. Rev.
  • Stuart, E. A., Cole, S. R., Bradshaw, C. P. and Leaf, P. J. (2011). The use of propensity scores to assess the generalizability of results from randomized trials. J. Roy. Statist. Soc. Ser. A 174 369–386.
  • Susukida, R., Crum, R. M., Stuart, E. A., Ebnesajjad, C. and Mojtabai, R. (2016). Assessing sample representativeness in randomized controlled trials: Application to the National Institute of Drug Abuse Clinical Trials Network. Addiction 111 1226–1234.
  • Tipton, E. (2013). Improving generalizations from experiments using propensity score subclassification: Assumptions, properties, and contexts. J. Educ. Behav. Stat. 38 239–266.
  • Vanderweele, T. J. and Arah, O. A. (2011). Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology 22 42–52.
  • Weisberg, H. I., Hayden, V. C. and Pontes, V. P. (2009). Selection criteria and generalizability within the counterfactual framework: Explaining the paradox of antidepressant-induced suicidality? Clin. Trials 6 109–118.

Supplemental materials

  • Simulation study. R-code for the simulation study.
    Digital Object Identifier: doi:10.1214/16-AOAS1001SUPPA
    Supplemental files available for subscribers.
  • Data example. R-code for implementing the outcome-model-based and weighted-outcome-model-based methods on the data example.
    Digital Object Identifier: doi:10.1214/16-AOAS1001SUPPB
    Supplemental files available for subscribers.

The Institute of Mathematical Statistics

The Annals of Applied Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more