The Annals of Statistics

Quantifying causal influences

Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, and Bernhard Schölkopf

Full-text: Open access


Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question.

Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution.

Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.

Article information

Ann. Statist. Volume 41, Number 5 (2013), 2324-2358.

First available in Project Euclid: 5 November 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62-09: Graphical methods 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]

Causality Bayesian networks information flow transfer entropy


Janzing, Dominik; Balduzzi, David; Grosse-Wentrup, Moritz; Schölkopf, Bernhard. Quantifying causal influences. Ann. Statist. 41 (2013), no. 5, 2324--2358. doi:10.1214/13-AOS1145.

Export citation


  • [1] Amari, S. and Nagaoka, H. (1993). Methods of Information Geometry. Oxford Univ. Press, New York.
  • [2] Avin, C., Shpitser, I. and Pearl, J. (2005). Identifiability of path-specific effects. In Proceedings of the International Joint Conference in Artificial Intelligence, Edinburgh, Scotland 357–363. Professional Book Center, Denver.
  • [3] Ay, N. and Krakauer, D. (2007). Geometric robustness and biological networks. Theory in Biosciences 125 93–121.
  • [4] Ay, N. and Polani, D. (2008). Information flows in causal networks. Adv. Complex Syst. 11 17–41.
  • [5] Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley, New York.
  • [6] Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37 424–38.
  • [7] Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equations models. In Sociological Methodology (C. Clogg, ed.) 449–484. American Sociological Association, Washington, DC.
  • [8] Hoyer, P., Janzing, D., Mooij, J., Peters, J. and Schölkopf, B. (2009). Nonlinear causal discovery with additive noise models. In Advances in Neural Information Processing Systems 21: 22nd Annual Conference on Neural Information Processing Systems 2008 (D. Koller, D. Schuurmans, Y. Bengio and L. Bottou, eds.) 689–696. Curran Associates, Red Hook, NY.
  • [9] Janzing, D., Balduzzi, D., Grosse-Wentrup, M. and Schölkopf, B. (2013). Supplement to “Quantifying causal influences.” DOI:10.1214/13-AOS1145SUPP.
  • [10] Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series 17. Oxford Univ. Press, New York.
  • [11] Lewontin, R. C. (1974). Annotation: The analysis of variance and the analysis of causes. American Journal Human Genetics 26 400–411.
  • [12] Massey, J. (1990). Causality, feedback and directed information. In Proc. 1990 Intl. Symp. on Info. Th. and Its Applications. Waikiki, Hawaii.
  • [13] Northcott, R. (2008). Can ANOVA measure causal strength? The Quaterly Review of Biology 83 47–55.
  • [14] Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, Cambridge.
  • [15] Pearl, J. (2001). Direct and indirect effects. In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI2001) 411–420. Morgan Kaufmann, San Francisco, CA.
  • [16] Pérez-Cruz, F. (2009). Estimation of information theoretic measures for continuous random variables. In Advances in Neural Information Processing Systems 21: 22nd Annual Conference on Neural Information Processing Systems 2008 (D. Koller, D. Schuurmans, Y. Bengio and L. Bottou, eds.) 1257–1264. Curran Associates, Red Hook, NY.
  • [17] Peters, J., Janzing, D. and Schölkopf, B. (2011). Causal inference on discrete data using additive noise models. IEEE Transac. Patt. Analysis and Machine Int. 33 2436–2450.
  • [18] Peters, J., Mooij, J., Janzing, D. and Schölkopf, B. (2001). Identifiability of causal graphs using functional models. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011) 589–598. AUAI Press, Corvallis, OR. Available at
  • [19] Robins, J. M. and Greenland, S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology 3 143–155.
  • [20] Schreiber, T. (2000). Measuring information transfer. Phys. Rev. Lett. 85 461–464.
  • [21] Spirtes, P., Glymour, C. and Scheines, R. (1993). Causation, Prediction, and Search. Lecture Notes in Statistics 81. Springer, New York.
  • [22] Touchette, H. and Lloyd, S. (2004). Information-theoretic approach to the study of control systems. Phys. A 331 140–172.
  • [23] Zhang, K. and Hyvärinen, A. (2009). On the identifiability of the post-nonlinear causal model. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada 647–655. AUAI Press, Arlington, VA.

Supplemental materials

  • Supplementary material: Supplement to “Quantifying causal influences”. Three supplementary sections: (1) Generating an i.i.d. copy via random permutations; (2) Another option to define causal strength; and (3) The problem of defining total influence.