Statistical Science

Defining and Estimating Intervention Effects for Groups that will Develop an Auxiliary Outcome

Marshall M. Joffe, Dylan Small, and Chi-Yuan Hsu

Full-text: Open access


It has recently become popular to define treatment effects for subsets of the target population characterized by variables not observable at the time a treatment decision is made. Characterizing and estimating such treatment effects is tricky; the most popular but naive approach inappropriately adjusts for variables affected by treatment and so is biased. We consider several appropriate ways to formalize the effects: principal stratification, stratification on a single potential auxiliary variable, stratification on an observed auxiliary variable and stratification on expected levels of auxiliary variables. We then outline identifying assumptions for each type of estimand. We evaluate the utility of these estimands and estimation procedures for decision making and understanding causal processes, contrasting them with the concepts of direct and indirect effects. We motivate our development with examples from nephrology and cancer screening, and use simulated data and real data on cancer screening to illustrate the estimation methods.

Article information

Statist. Sci. Volume 22, Number 1 (2007), 74-97.

First available: 1 August 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Joffe, Marshall M.; Small, Dylan; Hsu, Chi-Yuan. Defining and Estimating Intervention Effects for Groups that will Develop an Auxiliary Outcome. Statistical Science 22 (2007), no. 1, 74--97. doi:10.1214/088342306000000655.

Export citation


  • Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables (with discussion). J. Amer. Statist. Assoc. 91 444--472.
  • Baker, S. G. and Lindeman, K. S. (1994). The paired availability design: A proposal for evaluating epidural analgesia during labor. Stat. Med. 13 2269--2278.
  • Balke, A. and Pearl, J. (1997). Bounds on treatment effects from studies with imperfect compliance. J. Amer. Statist. Assoc. 92 1171--1176.
  • Barratt, A. L., Irwig, L. M., Glasziou, P. P., Salkeld, G. P. and Houssami, N. (2002). Benefits, harms and costs of screening mammography in women 70 years and over: A systematic review. Medical J. Australia 176 266--271.
  • Besarab, A., Bolton, W. K., Browne, J. K., Egrie, J. C., Nissenson, A. R., Okamoto, D. M., Schwab, S. J. and Goodkin, D. A. (1998). The effects of normal as compared with low hematocrit values in patients with cardiac disease who are receiving hemodialysis and epoetin. New England J. Medicine 339 584--590.
  • Brett, J., Bankhead, C., Henderson, B., Watson, E. and Austoker, J. (2005). The psychologoical impact of mammographic screening. A systematic review. Psycho-Oncology 14 917--938.
  • Cheng, J. and Small, D. (2006). Bounds on causal effects in three-arm trials with noncompliance. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 815--836.
  • Copas, J. and Li, H. (1997). Inference for nonrandom samples (with discussion). J. Roy. Statist. Soc. Ser. B 59 55--95.
  • Cox, D. R. and Oakes, D. (1984). Analysis of Survival Data. Chapman and Hall, London.
  • Efron, B. and Feldman, D. (1991). Compliance as an explanatory variable in clinical trials (with discussion). J. Amer. Statist. Assoc. 86 9--26.
  • Eknoyan, G., Beck, G. J., Cheung, A. K., Daugirdas, J., Greene, T., Kusek, J. W., Allon, M., Bailey, J., Delmez, J. A., Depner, T. A., Dwyer, J. T., Levey, A. S., Levin, N. W., Milford, E., Ornt, D. B., Rocco, M. V., Schulman, G., Schwab, S. J., Teehan, B. P. and Toto, R., for the Hemodialysis (HEMO) Study Group (2002). Effect of dialysis dose and membrane flux in maintenance hemodialysis. New England J. Medicine 347 2010--2019.
  • Feldman, H. I., Appel, L. J., Chertow, G. M., Cifelli, D., Cizman, B., Daugirdas, J., Fink, J. C., Franklin-Becker, E. D., Go, A. S., Hamm, L. L., He, J., Hostetter, T., Hsu, C.-Y., Jamerson, K., Joffe, M., Kusek, J. W., Landis, J. R., Lash, J. P., Miller, E. R., Mohler, E. R., Muntner, P., Ojo, A. O., Rahman, M., Townsend, R. R. and Wright, J. T. (2003). The chronic renal insufficiency cohort (CRIC) study: Design and methods. J. Amer. Soc. Nephrology 14 S148--S153.
  • Frangakis, C. E. and Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics 58 21--29.
  • Gilbert, P. B., Bosch, R. J. and Hudgens, M. G. (2003). Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine trials. Biometrics 59 531--541.
  • Greenland, S. (2003). Quantifying biases in causal models: Classical confounding vs collider-stratification bias. Epidemiology 14 300--306.
  • Greenland, S. and Robins, J. M. (1986). Identifiability, exchangeability and epidemiological confounding. Internat. J. Epidemiology 15 413--419.
  • Hernán, M. A., Hernández-Diáz, S. and Robins, J. M. (2004). A structural approach to selection bias. Epidemiology 15 615--625.
  • Hill, J., Waldfogel, J. and Brooks-Gunn, J. (2002). Differential effects of high-quality child care. J. Policy Analysis and Management 21 601--627.
  • Imbens, G. W. and Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. Ann. Statist. 25 305--327.
  • Joffe, M. M. (1994). Estimation of the effect of a treatment on a failure-time outcome using structural nested failure-time models and G-estimation. Ph.D. dissertation, Univ. California, Los Angeles.
  • Joffe, M. M. (2001). Using information on realized effects to determine prospective causal effects. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 759--774.
  • Joffe, M. M. (2001). Administrative and artificial censoring in censored regression models. Stat. Med. 20 2287--2304.
  • Joffe, M. M. and Brensinger, C. (2003). Weighting in instrumental variables and G-estimation. Stat. Med. 22 1285--1303.
  • Joffe, M. M., Ten Have, T. R. and Brensinger, C. (2003). The compliance score as a regressor in randomized trials. Biostatistics 4 327--340.
  • Kalbfleisch, J. D. and Prentice, R. L. (2002). The Statistical Analysis of Failure Time Data, 2nd ed. Wiley, Hoboken, NJ.
  • Little, R. J. A. (1995). Modeling the drop-out mechanism in repeated-measures studies. J. Amer. Statist. Assoc. 90 1112--1121.
  • Manski, C. F. (1990). Nonparametric bounds on treatment effects. American Economic Review Papers and Proceedings 80 319--323.
  • Mark, S. D. and Robins, J. M. (1993a). A method for the analysis of randomized trials with compliance information: An application to the multiple risk factor intervention trial. Controlled Clinical Trials 14 79--97.
  • Mark, S. D. and Robins, J. M. (1993b). Estimating the causal effect of smoking cessation in the presence of confounding factors using a rank preserving structural failure time model. Stat. Med. 12 1605--1628.
  • Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statist. Sci. 5 465--472.
  • Pagan, A. (1984). Econometric issues in the analysis of regressions with generated regressors. Internat. Econom. Rev. 25 221--247.
  • Paniagua, R., Amato, D., Vonesh, E., Correa-Rotter, R., Ramos, A., Moran, J. and Mujais, S. (2002). Effects of increased peritoneal clearances on mortality rates in peritoneal dialysis: ADEMEX, a prospective, randomized, controlled trial. J. Amer. Soc. Nephrology 13 1307--1320.
  • Pearl, J. (1995). Causal diagrams for empirical research (with discussion). Biometrika 82 669--710.
  • Pearl, J. (2000). Causality: Models, Reasoning and Inference. Cambridge Univ. Press.
  • Pearl, J. (2001). Direct and indirect effects. In Proc. Seventeenth Conference on Uncertainty in Artificial Intelligence 411--420. Morgan Kaufmann, San Francisco.
  • Robins, J. M. (1986). A new approach to causal inference in mortality studies with a sustained exposure period---application to control of the healthy worker survivor effect. Math. Modelling 7 1393--1512.
  • Robins, J. M. (1989). The analysis of randomized and nonrandomized AIDS treatment trials using a new approach to causal inference in longitudinal studies. In Health Service Research Methodology: A Focus on AIDS (L. Sechrest, H. Freeman and A. Mulley, eds.) 113--159. NCHSR, U.S. Public Health Service, Washington.
  • Robins, J. M. (1992). Estimation of the time-dependent accelerated failure time model in the presence of confounding factors. Biometrika 79 321--334.
  • Robins, J. M. (1994). Correcting for non-compliance in randomized trials using structural nested mean models. Comm. Statist.---Theory Methods 23 2379--2412.
  • Robins, J. M. (1999). Testing and estimation of direct effects by reparameterizing directed acyclic graphs with structural nested models. In Computation, Causation and Discovery (C. Glymour and G. Cooper, eds.) 349--405. AAAI Press, Menlo Park, CA.
  • Robins, J. M. (2003). Semantics of causal DAG models and the identification of direct and indirect effects. In Highly Structured Stochastic Systems (P. J. Green, N. L. Hjort and S. Richardson, eds.). Oxford Univ. Press, New York.
  • Robins, J. M., Blevins, D., Ritter, G. and Wulfsohn, M. (1992). G-estimation of the effect of prophylaxis therapy for pneumocystis carinii pneumonia on the survival of AIDS patients. Epidemiology 3 319--336.
  • Robins, J. M. and Greenland, S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology 3 143--155.
  • Robins, J. M. and Greenland, S. (1994). Adjusting for differential rates of prophylaxis therapy for PCP in high- versus low-dose AZT treatment arms in an AIDS randomized trial. J. Amer. Statist. Assoc. 89 737--749.
  • Robins, J. and Rotnitzky, A. (2004). Estimation of treatment effects in randomized trials with non-compliance and a dichotomous outcome using structural mean models. Biometrika 91 763--783.
  • Robins, J. M., Rotnitzky, A. and Scharfstein, D. O. (2000). Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical Models in Epidemiology, the Environment and Clinical Trials (E. Halloran and D. Berry, eds.) 1--94. Springer, New York.
  • Rosenbaum, P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. J. Roy. Statist. Soc. Ser. A 147 656--666.
  • Rosenbaum, P. R. (2002). Covariance adjustment in randomized experiments and observational studies (with discussion). Statist. Sci. 17 286--327.
  • Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41--55.
  • Rothman, K. J. and Greenland, S., eds. (1998). Modern Epidemiology, 2nd ed. Lippincott-Raven, Philadelphia.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educational Psychology 66 688--701.
  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34--58.
  • Rubin, D. B. (2004). Direct and indirect causal effects via potential outcomes. Scand. J. Statist. 31 161--170.
  • Shapiro, S., Venet, W., Strax, P. and Venet, L. (1988). Periodic Screening for Breast Cancer. The Health Insurance Plan Project and Its Sequelae, 1963--1986. Johns Hopkins Univ. Press, Baltimore.
  • Sommer, A. and Zeger, S. L. (1991). On estimating efficacy from clinical trials. Stat. Med. 10 45--52.
  • Teicher, H. (1963). Identifiability of finite mixtures. Ann. Math. Statist. 34 1265--1269.
  • Ten Have, T. R., Elliott, M. R., Joffe, M., Zanutto, E. and Datto, C. (2004). Causal models for randomized physician encouragement trials in treating primary care depression. J. Amer. Statist. Assoc. 99 16--25.
  • Thompson, W. D. (1991). Effect modification and the limits of biological inference from epidemiologic data. J. Clinical Epidemiology 44 221--232.
  • van der Laan, M. J. and Petersen, M. L. (2004). Estimation of direct and indirect causal effects in longitudinal studies. Berkeley Electronic Press. Div. Biostatistics Working Paper Series, Univ. California, Berkeley. Available at
  • Vansteelandt, S. and Goetghebeur, E. J. (2003). Causal inference with generalized structural mean models. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 817--835.
  • White, I. R. and Goetghebeur, E. J. (1998). Clinical trials comparing two treatment policies: Which aspects of the treatment policies make a difference. Stat. Med. 17 319--339.
  • Zhang, J. L. and Rubin, D. B. (2003). Estimation of causal effects via principal stratification when some outcomes are truncated by ``death.'' J. Educational and Behavioral Statistics 28 353--368.