## Electronic Journal of Statistics

- Electron. J. Statist.
- Volume 10, Number 2 (2016), 3490-3515.

### Priors on exchangeable directed graphs

Diana Cai, Nathanael Ackerman, and Cameron Freer

**Full-text: Open access**

#### Abstract

Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable *undirected* graphs, which extends to the directed case via measurable objects known as *digraphons*. Using digraphons, we first show how to construct models for exchangeable directed graphs, including special cases such as tournaments, linear orderings, directed acyclic graphs, and partial orderings. We then show how to construct priors on digraphons via the *infinite relational digraphon model* (di-IRM), a new Bayesian nonparametric block model for exchangeable directed graphs, and demonstrate inference on synthetic data.

#### Article information

**Source**

Electron. J. Statist., Volume 10, Number 2 (2016), 3490-3515.

**Dates**

Received: January 2016

First available in Project Euclid: 16 November 2016

**Permanent link to this document**

https://projecteuclid.org/euclid.ejs/1479287229

**Digital Object Identifier**

doi:10.1214/16-EJS1185

**Mathematical Reviews number (MathSciNet)**

MR3572857

**Zentralblatt MATH identifier**

1355.60041

**Subjects**

Primary: 60G09: Exchangeability 05C20: Directed graphs (digraphs), tournaments

Secondary: 62F15: Bayesian inference 62G05: Estimation

**Keywords**

Graphon digraphon directed graph exchangeable graph nonparametric prior network model block model

#### Citation

Cai, Diana; Ackerman, Nathanael; Freer, Cameron. Priors on exchangeable directed graphs. Electron. J. Statist. 10 (2016), no. 2, 3490--3515. doi:10.1214/16-EJS1185. https://projecteuclid.org/euclid.ejs/1479287229

#### References

- E. M. Airoldi, D. M. Blei, S. E. Fienberg, and E. P. Xing. Mixed membership stochastic blockmodels.,
*J. Machine Learning Res.*, 9 :1981–2014, 2008. - D. J. Aldous. Representations for partially exchangeable arrays of random variables.,
*J. Multivariate Anal.*, 11(4):581–598, 1981.Mathematical Reviews (MathSciNet): MR637937

Digital Object Identifier: doi:10.1016/0047-259X(81)90099-3 - D. J. Aldous. Exchangeability and related topics. In,
*École d’été de probabilités de Saint-Flour, XIII —1983*, volume 1117 of*Lecture Notes in Math.*, pages 1–198. Springer, Berlin, 1985. - N. Alon and A. Shapira. Testing subgraphs in directed graphs.,
*J. Comput. System Sci.*, 69(3):353–382, 2004. - A. Aroskar and J. Cummings. Limits, regularity and removal for finite structures.,
*To appear in J. Symb. Logic. ArXiv e-print 1412.8084*, 2014. - A. Aroskar.,
*Limits, regularity and removal for relational and weighted structures*. PhD thesis, Carnegie Mellon University, 2012. - T. Austin. On exchangeable random variables and the statistics of large graphs and hypergraphs.,
*Probab. Surv.*, 5:80–145, 2008. - C. Blundell, K. A. Heller, and J. M. Beck. Modelling reciprocating relationships with Hawkes processes. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 26*, pages 2609–2617, 2012. - C. Borgs, J. T. Chayes, H. Cohn, and S. Ganguly. Consistent nonparametric estimation for heavy-tailed sparse graphs.,
*ArXiv e-print 1508.06675v2*, 2015. - C. Borgs, J. T. Chayes, H. Cohn, and N. Holden. Sparse exchangeable graphs and their limits via graphon processes.,
*ArXiv e-print 1601.07134*, 2016. - R. A. Bradley and M. E. Terry. Rank analysis of incomplete block designs. I. The method of paired comparisons.,
*Biometrika*, 39:324–345, 1952. - D. Cai, T. Campbell, and T. Broderick. Edge-exchangeable graphs and sparsity. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 29*, 2016. - F. Caron and E. B. Fox. Sparse graphs using exchangeable random measures.,
*ArXiv e-print 1401.1137*, 2014. - S. H. Chan and E. Airoldi. A consistent histogram estimator for exchangeable graph models. In,
*Proc. 31st Int. Conf. Mach. Learn. (ICML)*, pages 208–216, 2014. - S. Chatterjee and S. Mukherjee. On estimation in tournaments and graphs under monotonicity constraints.,
*ArXiv e-print 1603.04556*, 2016. - S. Chatterjee. Matrix estimation by universal singular value thresholding.,
*Ann. Statist.*, 43(1):177–214, 2015. - F. R. K. Chung and R. L. Graham. Quasi-random tournaments.,
*J. Graph Theory*, 15(2):173–198, 1991. - K. J. Compton. The computational complexity of asymptotic problems. I. Partial orders.,
*Inform. and Comput.*, 78(2):108–123, 1988. - H. Crane and W. Dempsey. Edge exchangeable models for network data.,
*ArXiv e-print 1603.04571*, 2016. - P. Diaconis and S. Janson. Graph limits and exchangeable random graphs.,
*Rend. Mat. Appl. (7)*, 28(1):33–61, 2008.Mathematical Reviews (MathSciNet): MR2463439 - W. Fu, L. Song, and E. P. Xing. Dynamic mixed membership blockmodel for evolving networks. In,
*Proc. 26th Int. Conf. Mach. Learn. (ICML)*, pages 329–336, 2009. - P. S. Gill and T. B. Swartz. Bayesian analysis of directed graphs data with applications to social networks.,
*J. Roy. Statist. Soc. Ser. C*, 53(2):249–260, 2004. - E. Glasner and B. Weiss. Minimal actions of the group $\mathbbS(\mathbbZ)$ of permutations of the integers.,
*Geom. Funct. Anal.*, 12(5):964–988, 2002. - C. Heaukulani and Z. Ghahramani. Dynamic probabilistic models for latent feature propagation in social networks. In,
*Proc. 30th Int. Conf. Mach. Learn. (ICML)*, pages 275–283, 2013. - T. Herlau and M. Schmidt. Completely random measures for modelling block-structured sparse networks. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 29*, 2016. - J. Hladký, A. Máthé, V. Patel, and O. Pikhurko. Poset limits can be totally ordered.,
*Trans. Amer. Math. Soc.*, 367(6) :4319–4337, 2015. - P. D. Hoff, A. E. Raftery, and M. S. Handcock. Latent space approaches to social network analysis.,
*J. Amer. Statist. Assoc.*, 97(460) :1090–1098, 2002. - P. W. Holland and S. Leinhardt. An exponential family of probability distributions for directed graphs.,
*J. Amer. Statist. Assoc.*, 76(373):33–65, 1981. - P. W. Holland, K. B. Laskey, and S. Leinhardt. Stochastic blockmodels: first steps.,
*Social Networks*, 5(2):109–137, 1983. - D. N. Hoover. Relations on probability spaces and arrays of random variables. Preprint, Institute for Advanced Study, Princeton, NJ, 1979.
- S. Janson. Poset limits and exchangeable random posets.,
*Combinatorica*, 31(5):529–563, 2011.Mathematical Reviews (MathSciNet): MR2886098

Digital Object Identifier: doi:10.1007/s00493-011-2591-x - O. Kallenberg. Multivariate sampling and the estimation problem for exchangeable arrays.,
*J. Theoret. Probab.*, 12(3):859–883, 1999. - C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In,
*Proc. 21st Nat. Conf. Artificial Intelligence (AAAI-06)*, 2006. - C. Kemp, T. L. Griffiths, and J. B. Tenenbaum. Discovering latent classes in relational data. AI Memo 2004-019, Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 2004.
- M. Kim and J. Leskovec. Nonparametric multi-group membership model for dynamic networks. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 27*, pages 1385–1393, 2013. - D. J. Kleitman and B. L. Rothschild. Asymptotic enumeration of partial orders on a finite set.,
*Trans. Amer. Math. Soc.*, 205:205–220, 1975. - I. Kolossváry and B. Ráth. Multigraph limits and exchangeability.,
*Acta Math. Hungar.*, 130(1–2):1–34, 2011. - D. Kunszenti-Kovács, L. Lovász, and B. Szegedy. Multigraph limits, unbounded kernels, and Banach space decorated graphs.,
*ArXiv e-print 1406.7846*, 2014. - J. R. Lloyd, P. Orbanz, Z. Ghahramani, and D. M. Roy. Random function priors for exchangeable arrays with applications to graphs and relational data. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 25*, pages 1007–1015, 2012. - J. R. Lloyd, P. Orbanz, Z. Ghahramani, and D. M. Roy. Exchangeable databases and their functional representation. In,
*NIPS Workshop on Frontiers of Network Analysis: Methods, Models, and Application*, 2013. - L. Lovász.,
*Large networks and graph limits*, volume 60 of*Amer. Math. Soc. Colloq. Publ.*Amer. Math. Soc., Providence, RI, 2012. - L. Lovász and B. Szegedy. Limits of dense graph sequences.,
*J. Combin. Theory Ser. B*, 96(6):933–957, 2006. - K. T. Miller, T. L. Griffiths, and M. I. Jordan. Nonparametric latent feature models for link prediction. In,
*Adv. Neural Inform. Process. Syst. (NIPS) 22*, pages 1276–1284, 2009. - K. Nowicki and T. A. B. Snijders. Estimation and prediction for stochastic blockstructures.,
*J. Amer. Statist. Assoc.*, 96(455) :1077–1087, 2001. - D. Offner.,
*Extremal problems on the hypercube*. PhD thesis, Carnegie Mellon University, 2009. - P. Orbanz and D. M. Roy. Bayesian models of graphs, arrays and other exchangeable random structures.,
*IEEE Trans. Pattern Anal. Mach. Intell.*, 37(2):437–461, 2015. - K. Palla, D. A. Knowles, and Z. Ghahramani. An infinite latent attribute model for network data. In,
*Proc. 29th Int. Conf. Mach. Learn. (ICML)*, 2012. - J. Pitman and M. Yor. The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator.,
*Ann. Probab.*, 25(2):855–900, 1997. - A. Roverato and G. Consonni. Compatible prior distributions for directed acyclic graph models.,
*J. R. Stat. Soc. Ser. B Stat. Methodol.*, 66(1):47–61, 2004. - E. Thörnblad. Decomposition of tournament limits.,
*ArXiv e-print 1604.04271*, 2016. - V. Veitch and D. M. Roy. The class of random graphs arising from exchangeable random measures.,
*ArXiv e-print 1512.03099*, 2015. - Y. J. Wang and G. Y. Wong. Stochastic blockmodels for directed graphs.,
*J. Amer. Statist. Assoc.*, 82(397):8–19, 1987. - S. Wasserman and K. Faust.,
*Social network analysis: methods and applications*. Structural Analysis in the Social Sciences. Cambridge University Press, 1994. - G. Y. Wong. Bayesian models for directed graphs.,
*J. Amer. Statist. Assoc.*, 82(397):140–148, 1987.Mathematical Reviews (MathSciNet): MR883341

Digital Object Identifier: doi:10.1080/01621459.1987.10478406 - Z. Xu, V. Tresp, S. Yu, K. Yu, and H. Kriegel. Fast inference in infinite hidden relational models. In,
*Proc. of Mining and Learning with Graphs (MLG 2007)*, 2007. - E. Zermelo. Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung.,
*Math. Z.*, 29(1):436–460, 1929.

#### The Institute of Mathematical Statistics and the Bernoulli Society

### More like this

- Wishart distributions for decomposable covariance graph models

Khare, Kshitij and Rajaratnam, Bala, The Annals of Statistics, 2011 - Learning Markov Equivalence Classes of Directed Acyclic Graphs: An Objective Bayes Approach

Castelletti, Federico, Consonni, Guido, Della Vedova, Marco L., and Peluso, Stefano, Bayesian Analysis, 2018 - Markov properties for mixed graphs

Sadeghi, Kayvan and Lauritzen, Steffen, Bernoulli, 2014

- Wishart distributions for decomposable covariance graph models

Khare, Kshitij and Rajaratnam, Bala, The Annals of Statistics, 2011 - Learning Markov Equivalence Classes of Directed Acyclic Graphs: An Objective Bayes Approach

Castelletti, Federico, Consonni, Guido, Della Vedova, Marco L., and Peluso, Stefano, Bayesian Analysis, 2018 - Markov properties for mixed graphs

Sadeghi, Kayvan and Lauritzen, Steffen, Bernoulli, 2014 - A characterization of Markov equivalence classes for acyclic digraphs

Andersson, Steen A., Madigan, David, and Perlman, Michael D., The Annals of Statistics, 1997 - Finitely Constrained Classes of Homogeneous Directed Graphs

Latka, Brenda J., Journal of Symbolic Logic, 1994 - Inference From a Knockout Tournament

Hartigan, J. A., The Annals of Mathematical Statistics, 1968 - Googling the Brain: Discovering Hierarchical and Asymmetric Network Structures, with Applications in Neuroscience

Crofts, Jonathan J. and Higham, Desmond J., Internet Mathematics, 2011 - Separation and Completeness Properties for Amp Chain Graph
Markov Models

Levitz, Michael, Madigan, David, and Perlman, Michael D., The Annals of Statistics, 2001 - Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs

He, Yangbo, Jia, Jinzhu, and Yu, Bin, The Annals of Statistics, 2013 - Tracking network dynamics: A survey using graph distances

Donnat, Claire and Holmes, Susan, The Annals of Applied Statistics, 2018