The Annals of Statistics

Margins of discrete Bayesian networks

Robin J. Evans

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Bayesian network models with latent variables are widely used in statistics and machine learning. In this paper, we provide a complete algebraic characterization of these models when the observed variables are discrete and no assumption is made about the state-space of the latent variables. We show that it is algebraically equivalent to the so-called nested Markov model, meaning that the two are the same up to inequality constraints on the joint probabilities. In particular, these two models have the same dimension, differing only by inequality constraints for which there is no general description. The nested Markov model is therefore the closest possible description of the latent variable model that avoids consideration of inequalities. A consequence of this is that the constraint finding algorithm of Tian and Pearl [In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (2002) 519–527] is complete for finding equality constraints.

Latent variable models suffer from difficulties of unidentifiable parameters and nonregular asymptotics; in contrast the nested Markov model is fully identifiable, represents a curved exponential family of known dimension, and can easily be fitted using an explicit parameterization.

Article information

Source
Ann. Statist., Volume 46, Number 6A (2018), 2623-2656.

Dates
Received: January 2017
Revised: August 2017
First available in Project Euclid: 7 September 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1536307228

Digital Object Identifier
doi:10.1214/17-AOS1631

Mathematical Reviews number (MathSciNet)
MR3851750

Zentralblatt MATH identifier
06968594

Subjects
Primary: 62H99: None of the above, but in this section 62F12: Asymptotic properties of estimators

Keywords
Algebraic statistics Bayesian network latent variable model nested Markov model Verma constraint

Citation

Evans, Robin J. Margins of discrete Bayesian networks. Ann. Statist. 46 (2018), no. 6A, 2623--2656. doi:10.1214/17-AOS1631. https://projecteuclid.org/euclid.aos/1536307228


Export citation

References

  • Čencov, N. N. (1982). Statistical Decision Rules and Optimal Inference. Translations of Mathematical Monographs 53. Amer. Math. Soc., Providence, RI. Translation from the Russian edited by Lev J. Leifman.
  • Allman, E. S., Matias, C. and Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. Ann. Statist. 37 3099–3132.
    Mathematical Reviews (MathSciNet): MR2549554
    Zentralblatt MATH: 1191.62003
    Digital Object Identifier: doi:10.1214/09-AOS689
    Project Euclid: euclid.aos/1250515381
  • Anandkumar, A., Hsu, D., Javanmard, A. and Kakade, S. (2013). Learning linear Bayesian networks with latent variables. In Proceedings of the 30th International Conference on Machine Learning 28 249–257.
  • Basu, S., Pollack, R. and Roy, M.-F. (1996). Algorithms in Real Algebraic Geometry, Springer, Berlin.
    Zentralblatt MATH: 1031.14028
  • Bishop, C. M. (2007). Pattern Recognition and Machine Learning. Springer, Berlin.
    Zentralblatt MATH: 1107.68072
  • Cox, D., Little, J. and O’Shea, D. (2007). Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3rd ed. Springer, New York.
    Zentralblatt MATH: 1118.13001
  • Darwiche, A. (2009). Modeling and Reasoning with Bayesian Networks. Cambridge Univ. Press, Cambridge.
    Zentralblatt MATH: 1231.68003
  • Dawid, A. P. (2002). Influence diagrams for causal modelling and inference. Int. Stat. Rev. 70 161–189.
    Zentralblatt MATH: 1215.62002
    Digital Object Identifier: doi:10.1111/j.1751-5823.2002.tb00354.x
  • Drton, M. (2009). Likelihood ratio tests and singularities. Ann. Statist. 37 979–1012.
    Zentralblatt MATH: 1196.62020
    Digital Object Identifier: doi:10.1214/07-AOS571
    Project Euclid: euclid.aos/1236693157
  • Evans, R. J. (2016). Graphs for margins of Bayesian networks. Scand. J. Stat. 43 625–648.
    Zentralblatt MATH: 06628975
    Digital Object Identifier: doi:10.1111/sjos.12194
  • Evans, R. J. (2018). Supplement to “Margins of discrete Bayesian networks.” DOI:10.1214/17-AOS1631SUPP.
  • Evans, R. J. and Richardson, T. S. (2010). Maximum likelihood fitting of acyclic directed mixed graphs to binary data. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence 177–184.
  • Evans, R. J. and Richardson, T. S. (2014). Markovian acyclic directed mixed graphs for discrete data. Ann. Statist. 42 1452–1482.
    Zentralblatt MATH: 1302.62148
    Digital Object Identifier: doi:10.1214/14-AOS1206
    Project Euclid: euclid.aos/1407420005
  • Evans, R. J. and Richardson, T. S. (2018). Smooth identifiable supermodels of discrete DAG models with latent variables. Bernoulli. To appear. Available at arXiv:1511.06813.
    arXiv: 1511.06813
  • Fox, C. J., Käufl, A. and Drton, M. (2015). On the causal interpretation of acyclic mixed graphs under multivariate normality. Linear Algebra Appl. 473 93–113.
    Zentralblatt MATH: 1314.05085
    Digital Object Identifier: doi:10.1016/j.laa.2014.02.032
  • Garcia, L. D., Stillman, M. and Sturmfels, B. (2005). Algebraic geometry of Bayesian networks. J. Symbolic Comput. 39 331–355.
    Zentralblatt MATH: 1126.68102
    Digital Object Identifier: doi:10.1016/j.jsc.2004.11.007
  • Gill, R. D. (2014). Statistics, causality and Bell’s theorem. Statist. Sci. 29 512–528.
    Zentralblatt MATH: 1331.81048
    Digital Object Identifier: doi:10.1214/14-STS490
    Project Euclid: euclid.ss/1421330545
  • Henson, J., Lal, R. and Pusey, M. F. (2014). Theory-independent limits on correlations from generalized Bayesian networks. New J. Phys. 16 113043.
  • Kass, R. E. and Vos, P. W. (1997). Geometrical Foundations of Asymptotic Inference. Wiley, New York.
    Zentralblatt MATH: 0880.62005
  • Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series 17. Oxford Univ. Press, New York.
  • Neyman, J. (1923). Sur les applications de la théorie des probabilités aux experiences agricoles: Essai des principes. Rocz. Nauk Rol. 10 1–51. In Polish; English translation by D. Dabrowska and T. Speed in Statist. Sci. 5 463–472 (1990).
  • Pearl, J. (2009). Causality: Models, Reasoning, and Inference, 2nd ed. Cambridge Univ. Press, Cambridge.
    Zentralblatt MATH: 1188.68291
  • Provan, J. S. and Billera, L. J. (1980). Decompositions of simplicial complexes related to diameters of convex polyhedra. Math. Oper. Res. 5 576–594.
    Zentralblatt MATH: 0457.52005
    Digital Object Identifier: doi:10.1287/moor.5.4.576
  • Richardson, T. S. (2003). Markov properties for acyclic directed mixed graphs. Scand. J. Stat. 30 145–157.
    Zentralblatt MATH: 1035.60005
    Digital Object Identifier: doi:10.1111/1467-9469.00323
  • Richardson, T. S., Evans, R. J. and Robins, J. M. (2011). Transparent parametrizations of models for potential outcomes. In Bayesian Statistics 9 569–610. Oxford Univ. Press, Oxford.
  • Richardson, T. S. and Spirtes, P. (2002). Ancestral graph Markov models. Ann. Statist. 30 962–1030.
    Mathematical Reviews (MathSciNet): MR1926166
    Zentralblatt MATH: 1033.60008
    Digital Object Identifier: doi:10.1214/aos/1031689015
    Project Euclid: euclid.aos/1031689015
  • Richardson, T. S., Evans, R. J., Robins, J. M. and Shpitser, I. (2017). Nested Markov properties for acyclic directed mixed graphs. Available at arXiv:1701.06686.
    arXiv: 1701.06686
  • Robins, J. (1986). A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Math. Model. 7 1393–1512.
    Zentralblatt MATH: 0614.62136
    Digital Object Identifier: doi:10.1016/0270-0255(86)90088-6
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688–701.
  • Shpitser, I., Evans, R. J., Richardson, T. S. and Robins, J. M. (2013). Sparse nested Markov models with log-linear parameters. In 29th Conference on Uncertainty in Artificial Intelligence 576–585.
  • Silva, R. and Ghahramani, Z. (2009). The hidden life of latent variables: Bayesian learning with mixed graph models. J. Mach. Learn. Res. 10 1187–1238.
    Zentralblatt MATH: 1235.68191
  • Spirtes, P., Glymour, C. and Scheines, R. (2000). Causation, Prediction, and Search, 2nd ed. MIT Press, Cambridge, MA.
    Zentralblatt MATH: 0806.62001
  • Tian, J. and Pearl, J. (2002). On the testable implications of causal models with hidden variables. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence 519–527.
  • ver Steeg, G. and Galstyan, A. (2011). A sequence of relaxations constraining hidden variable models. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence 717–726.
  • Verma, T. S. and Pearl, J. (1990). Equivalence and synthesis of causal models. In Proceedings of the 6th Conference on Uncertainty in Artificial Intelligence 255–268.

Supplemental materials

  • Supplement to “Margins of discrete Bayesian networks”. Technical proofs and some additional examples are contained in the supplement.
    Digital Object Identifier: doi:10.1214/17-AOS1631SUPP
    Supplemental files are immediately available to subscribers. Non-subscribers gain access to supplemental files with the purchase of the article.

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more