The Annals of Applied Statistics

Inferring causal impact using Bayesian structural time-series models

Kay H. Brodersen, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L. Scott

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

An important problem in econometrics and marketing is to infer the causal impact that a designed market intervention has exerted on an outcome metric over time. This paper proposes to infer causal impact on the basis of a diffusion-regression state-space model that predicts the counterfactual market response in a synthetic control that would have occurred had no intervention taken place. In contrast to classical difference-in-differences schemes, state-space models make it possible to (i) infer the temporal evolution of attributable impact, (ii) incorporate empirical priors on the parameters in a fully Bayesian treatment, and (iii) flexibly accommodate multiple sources of variation, including local trends, seasonality and the time-varying influence of contemporaneous covariates. Using a Markov chain Monte Carlo algorithm for posterior inference, we illustrate the statistical properties of our approach on simulated data. We then demonstrate its practical utility by estimating the causal effect of an online advertising campaign on search-related site visits. We discuss the strengths and limitations of state-space models in enabling causal attribution in those settings where a randomised experiment is unavailable. The CausalImpact R package provides an implementation of our approach.

Article information

Source
Ann. Appl. Stat. Volume 9, Number 1 (2015), 247-274.

Dates
First available in Project Euclid: 28 April 2015

Permanent link to this document
https://projecteuclid.org/euclid.aoas/1430226092

Digital Object Identifier
doi:10.1214/14-AOAS788

Mathematical Reviews number (MathSciNet)
MR3341115

Zentralblatt MATH identifier
06446568

Keywords
Causal inference counterfactual synthetic control observational difference in differences econometrics advertising market research

Citation

Brodersen, Kay H.; Gallusser, Fabian; Koehler, Jim; Remy, Nicolas; Scott, Steven L. Inferring causal impact using Bayesian structural time-series models. Ann. Appl. Stat. 9 (2015), no. 1, 247--274. doi:10.1214/14-AOAS788. https://projecteuclid.org/euclid.aoas/1430226092


Export citation

References

  • Abadie, A. (2005). Semiparametric difference-in-differences estimators. Rev. Econom. Stud. 72 1–19.
  • Abadie, A., Diamond, A. and Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. J. Amer. Statist. Assoc. 105 493–505.
  • Abadie, A. and Gardeazabal, J. (2003). The economic costs of conflict: A case study of the basque country. Amer. Econ. Rev. 93 113–132.
  • Angrist, J. D. and Krueger, A. B. (1999). Empirical strategies in labor economics. Handbook of Labor Economics 3 1277–1366.
  • Angrist, J. D. and Pischke, J.-S. (2008). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton Univ. Press, Princeton, NJ.
  • Antonakis, J., Bendahan, S., Jacquart, P. and Lalive, R. (2010). On making causal claims: A review and recommendations. Leadersh. Q. 21 1086–1120.
  • Ashenfelter, O. and Card, D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67 648–660.
  • Ataman, M. B., Mela, C. F. and Van Heerde, H. J. (2008). Building brands. Mark. Sci. 27 1036–1054.
  • Athey, S. and Imbens, G. W. (2002). Identification and inference in nonlinear difference-in-differences models. Working Paper 280, National Bureau of Economic Research, Cambridge, MA.
  • Banerjee, S., Kauffman, R. J. and Wang, B. (2007). Modeling Internet firm survival using Bayesian dynamic models with time-varying coefficients. Electron. Commer. Res. Appl. 6 332–342.
  • Belloni, A., Chernozhukov, V., Fernandez-Val, I. and Hansen, C. (2013). Program evaluation with high-dimensional data. CeMMAP Working Paper CWP77/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies, London.
  • Berndt, E. R. (1991). The Practice of Econometrics: Classic and Contemporary. Addison-Wesley, Reading, MA.
  • Bertrand, M., Duflo, E. and Mullainathan, S. (2002). How much should we trust differences-in-differences estimates? Working Paper 8841, National Bureau of Economic Research, Cambridge, MA.
  • Brady, H. E. (2002). Models of causal inference: Going beyond the Neyman-Rubin-Holland theory. In Annual Meetings of the Political Methodology Group, Boston, MA.
  • Brodersen, K. H., Daunizeau, J., Mathys, C., Chumbley, J. R., Buhmann, J. M. and Stephan, K. E. (2013). Variational Bayesian mixed-effects inference for classification studies. Neuroimage 76 345–361.
  • Camillo, F. and d’Attoma, I. (2010). A new data mining approach to estimate causal effects of policy interventions. Expert Syst. Appl. 37 171–181.
  • Campbell, D. T., Stanley, J. C. and Gage, N. L. (1963). Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin, Boston.
  • Card, D. and Krueger, A. B. (1993). Minimum wages and employment: A case study of the fast food industry in New Jersey and Pennsylvania. Technical report, National Bureau of Economic Research, Cambridge, MA.
  • Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541–553.
  • Chan, D., Ge, R., Gershony, O., Hesterberg, T. and Lambert, D. (2010). Evaluating online ad campaigns in a pipeline: Causal models at scale. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 7–16. ACM, New York.
  • Chipman, H., George, E. I. and McCulloch, R. E. (2001). The practical implementation of Bayesian model selection. In Model Selection. Institute of Mathematical Statistics Lecture Notes—Monograph Series 38 65–134. IMS, Beachwood, OH.
  • Claveau, F. (2012). The Russo–Williamson Theses in the social sciences: Causal inference drawing on two types of evidence. Stud. Hist. Philos. Biol. Biomed. Sci. 43 806–813.
  • Cox, D. and Wermuth, N. (2001). Causal inference and statistical fallacies. In International Encyclopedia of the Social & Behavioral Sciences (Neil J. Smelser and P. B. Baltes, eds.) 1554–1561. Pergamon, Oxford.
  • Danaher, P. J. and Rust, R. T. (1996). Determining the optimal return on investment for an advertising campaign. European J. Oper. Res. 95 511–521.
  • de Jong, P. and Shephard, N. (1995). The simulation smoother for time series models. Biometrika 82 339–350.
  • Donald, S. G. and Lang, K. (2007). Inference with difference-in-differences and other panel data. Rev. Econ. Stat. 89 221–233.
  • Durbin, J. and Koopman, S. J. (2002). A simple and efficient simulation smoother for state space time series analysis. Biometrika 89 603–615.
  • Frühwirth-Schnatter, S. (1994). Data augmentation and dynamic linear models. J. Time Series Anal. 15 183–202.
  • George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881–889.
  • George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339–374.
  • Ghosh, J. and Clyde, M. A. (2011). Rao-Blackwellization for Bayesian variable selection and model averaging in linear and binary regression: A novel data augmentation approach. J. Amer. Statist. Assoc. 106 1041–1052.
  • Hansen, C. B. (2007a). Asymptotic properties of a robust variance matrix estimator for panel data when $T$ is large. J. Econometrics 141 597–620.
  • Hansen, C. B. (2007b). Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects. J. Econometrics 140 670–694.
  • Heckman, J. J. and Vytlacil, E. J. (2007). Econometric evaluation of social programs, Part I: Causal models, structural models and econometric policy evaluation. In Handbook of Econometrics 6, Part B (J. J. Heckman and E. E. Leamer, eds.) 4779–4874. Elsevier, Amsterdam.
  • Hitchcock, C. (2004). Do all and only causes raise the probabilities of effects? In Causation and Counterfactuals. MIT Press, Cambridge.
  • Hoover, K. D. (2012). Economic theory and causal inference. In Philosophy of Economics 13 (U. Mäki, ed.) 89–113. Elsevier, Amsterdam.
  • Kleinberg, S. and Hripcsak, G. (2011). A review of causal inference for biomedical informatics. J. Biomed. Inform. 44 1102–1112.
  • Leeflang, P. S., Bijmolt, T. H., van Doorn, J., Hanssens, D. M., van Heerde, H. J., Verhoef, P. C. and Wieringa, J. E. (2009). Creating lift versus building the base: Current trends in marketing dynamics. Int. J. Res. Mark. 26 13–20.
  • Lester, R. A. (1946). Shortcomings of marginal analysis for wage-employment problems. Amer. Econ. Rev. 36 63–82.
  • Lewis, R. A., Rao, J. M. and Reiley, D. H. (2011). Here, there, and everywhere: Correlated online behaviors can lead to overestimates of the effects of advertising. In Proceedings of the 20th International Conference on World Wide Web. WWW’11 157–166. ACM, New York.
  • Lewis, R. A. and Reiley, D. H. (2011). Does retail advertising work? Technical report.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of $g$ priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410–423.
  • Mathys, C., Daunizeau, J., Friston, K. J. and Stephan, K. E. (2011). A Bayesian foundation for individual learning under uncertainty. Front. Human Neurosci. 5 39.
  • Meyer, B. D. (1995). Natural and quasi-experiments in economics. J. Bus. Econom. Statist. 13 151.
  • Morgan, S. L. and Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge Univ. Press, Cambridge.
  • Nakajima, J. and West, M. (2013). Bayesian analysis of latent threshold dynamic models. J. Bus. Econom. Statist. 31 151–164.
  • Polson, N. G. and Scott, S. L. (2011). Data augmentation for support vector machines. Bayesian Anal. 6 1–23.
  • Robinson, G., McNulty, J. E. and Krasno, J. S. (2009). Observing the counterfactual? The search for political experiments in nature. Polit. Anal. 17 341–357.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688.
  • Rubin, D. B. (2008). Statistical inference for causal effects, with emphasis on applications in epidemiology and medical statistics. In Epidemiology and Medical Statistics (J. P. Miller, C. R. Rao and D. C. Rao, eds.). Handbook of Statist. 27 28–63. Elsevier, Amsterdam.
  • Rubin, D. B. and Waterman, R. P. (2006). Estimating the causal effects of marketing interventions using propensity score methodology. Statist. Sci. 21 206–222.
  • Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587–2619.
  • Scott, S. L. and Varian, H. R. (2014). Predicting the present with Bayesian structural time series. International Journal of Mathematical Modeling and Optimization 5 4–23.
  • Seggie, S. H., Cavusgil, E. and Phelan, S. E. (2007). Measurement of return on marketing investment: A conceptual framework and the future of marketing metrics. Ind. Mark. Manage. 36 834–841.
  • Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Wadsworth Cengage Learning, Seattle, WA.
  • Solon, G. (1984). Estimating Autocorrelations in Fixed-Effects Models. National Bureau of Economic Research, Cambridge, MA.
  • Stewart, D. W. (2009). Marketing accountability: Linking marketing actions to financial results. J. Bus. Res. 62 636–643.
  • Takada, H. and Bass, F. M. (1998). Multiple time series analysis of competitive marketing behavior. J. Bus. Res. 43 97–107.
  • Vaver, J. and Koehler, J. (2011). Measuring ad effectiveness using geo experiments. Technical report, Google Inc.
  • Vaver, J. and Koehler, J. (2012). Periodic measurement of advertising effectiveness using multiple-test-period geo experiments. Technical report, Google Inc.
  • West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models, 2nd ed. Springer, New York.
  • Winship, C. and Morgan, S. L. (1999). The estimation of causal effects from observational data. Annu. Rev. Sociol. 659–706.
  • Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with $g$-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (P. K. Goel and A. Zellner, eds.). Stud. Bayesian Econometrics Statist. 6 233–243. North-Holland, Amsterdam.