The Role of a Second Control Group in an Observational Study

Abstract

An observational study is an attempt to estimate the effects of a treatment when subjects are not randomly assigned to treatment or control. The possibility of using more than one control group has often been briefly mentioned in general discussions of observational studies, and many observational studies have used two control groups. Here, the limited and dispersed literature on this subject is reviewed, and the topic is developed in several directions by using a formal notation for observational studies. The value of a second control group depends on the supplementary information that is available about unobserved biases that are suspected to exist. A second control group provides a test of the assumption that conventional adjustments for observed covariates suffice in estimating treatment effects. Under the best of circumstances, this test is consistent and unbiased, and its power exceeds the probability of falsely detecting a treatment effect. Indeed, under the best of circumstances, two control groups can yield consistent and unbiased estimates of bounds on the treatment effect when conventional adjustments fail. In contrast, however, in the worst of circumstances, a second control group can be of little value.