The Annals of Statistics

Quantifying causal influences

Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, and Bernhard Schölkopf

Full-text: Open access

Enhanced PDF (444 KB)

Abstract

Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question.

Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution.

Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.

Article information

Source
Ann. Statist., Volume 41, Number 5 (2013), 2324-2358.

Dates
First available in Project Euclid: 5 November 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1383661266

Digital Object Identifier
doi:10.1214/13-AOS1145

Mathematical Reviews number (MathSciNet)
MR3127868

Zentralblatt MATH identifier
1281.62030

Subjects
Primary: 62-09: Graphical methods 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]

Keywords
Causality Bayesian networks information flow transfer entropy

Citation

Janzing, Dominik; Balduzzi, David; Grosse-Wentrup, Moritz; Schölkopf, Bernhard. Quantifying causal influences. Ann. Statist. 41 (2013), no. 5, 2324--2358. doi:10.1214/13-AOS1145. https://projecteuclid.org/euclid.aos/1383661266


Export citation

References

  • [1] Amari, S. and Nagaoka, H. (1993). Methods of Information Geometry. Oxford Univ. Press, New York.
  • [2] Avin, C., Shpitser, I. and Pearl, J. (2005). Identifiability of path-specific effects. In Proceedings of the International Joint Conference in Artificial Intelligence, Edinburgh, Scotland 357–363. Professional Book Center, Denver.
    Mathematical Reviews (MathSciNet): MR2192340
    Digital Object Identifier: doi:10.1016/j.artint.2005.10.003
  • [3] Ay, N. and Krakauer, D. (2007). Geometric robustness and biological networks. Theory in Biosciences 125 93–121.
  • [4] Ay, N. and Polani, D. (2008). Information flows in causal networks. Adv. Complex Syst. 11 17–41.
    Mathematical Reviews (MathSciNet): MR2400125
    Digital Object Identifier: doi:10.1142/S0219525908001465
  • [5] Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1122806
  • [6] Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37 424–38.
  • [7] Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equations models. In Sociological Methodology (C. Clogg, ed.) 449–484. American Sociological Association, Washington, DC.
  • [8] Hoyer, P., Janzing, D., Mooij, J., Peters, J. and Schölkopf, B. (2009). Nonlinear causal discovery with additive noise models. In Advances in Neural Information Processing Systems 21: 22nd Annual Conference on Neural Information Processing Systems 2008 (D. Koller, D. Schuurmans, Y. Bengio and L. Bottou, eds.) 689–696. Curran Associates, Red Hook, NY.
  • [9] Janzing, D., Balduzzi, D., Grosse-Wentrup, M. and Schölkopf, B. (2013). Supplement to “Quantifying causal influences.” DOI:10.1214/13-AOS1145SUPP.
  • [10] Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series 17. Oxford Univ. Press, New York.
    Mathematical Reviews (MathSciNet): MR1419991
  • [11] Lewontin, R. C. (1974). Annotation: The analysis of variance and the analysis of causes. American Journal Human Genetics 26 400–411.
  • [12] Massey, J. (1990). Causality, feedback and directed information. In Proc. 1990 Intl. Symp. on Info. Th. and Its Applications. Waikiki, Hawaii.
  • [13] Northcott, R. (2008). Can ANOVA measure causal strength? The Quaterly Review of Biology 83 47–55.
  • [14] Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR1744773
  • [15] Pearl, J. (2001). Direct and indirect effects. In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI2001) 411–420. Morgan Kaufmann, San Francisco, CA.
  • [16] Pérez-Cruz, F. (2009). Estimation of information theoretic measures for continuous random variables. In Advances in Neural Information Processing Systems 21: 22nd Annual Conference on Neural Information Processing Systems 2008 (D. Koller, D. Schuurmans, Y. Bengio and L. Bottou, eds.) 1257–1264. Curran Associates, Red Hook, NY.
  • [17] Peters, J., Janzing, D. and Schölkopf, B. (2011). Causal inference on discrete data using additive noise models. IEEE Transac. Patt. Analysis and Machine Int. 33 2436–2450.
  • [18] Peters, J., Mooij, J., Janzing, D. and Schölkopf, B. (2001). Identifiability of causal graphs using functional models. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011) 589–598. AUAI Press, Corvallis, OR. Available at http://uai.sis.pitt.edu/papers/11/p589-peters.pdf.
  • [19] Robins, J. M. and Greenland, S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology 3 143–155.
  • [20] Schreiber, T. (2000). Measuring information transfer. Phys. Rev. Lett. 85 461–464.
  • [21] Spirtes, P., Glymour, C. and Scheines, R. (1993). Causation, Prediction, and Search. Lecture Notes in Statistics 81. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1227558
  • [22] Touchette, H. and Lloyd, S. (2004). Information-theoretic approach to the study of control systems. Phys. A 331 140–172.
    Mathematical Reviews (MathSciNet): MR2046360
    Digital Object Identifier: doi:10.1016/j.physa.2003.09.007
  • [23] Zhang, K. and Hyvärinen, A. (2009). On the identifiability of the post-nonlinear causal model. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada 647–655. AUAI Press, Arlington, VA.

Supplemental materials

  • Supplementary material: Supplement to “Quantifying causal influences”. Three supplementary sections: (1) Generating an i.i.d. copy via random permutations; (2) Another option to define causal strength; and (3) The problem of defining total influence.
    Digital Object Identifier: doi:10.1214/13-AOS1145SUPP
    Supplemental PDF (219 KB)

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more