Electronic Journal of Statistics

Exchangeable trait allocations

Trevor Campbell, Diana Cai, and Tamara Broderick

Full-text: Open access

Abstract

Trait allocations are a class of combinatorial structures in which data may belong to multiple groups and may have different levels of belonging in each group. Often the data are also exchangeable, i.e., their joint distribution is invariant to reordering. In clustering—a special case of trait allocation—exchangeability implies the existence of both a de Finetti representation and an exchangeable partition probability function (EPPF), distributional representations useful for computational and theoretical purposes. In this work, we develop the analogous de Finetti representation and exchangeable trait probability function (ETPF) for trait allocations, along with a characterization of all trait allocations with an ETPF. Unlike previous feature allocation characterizations, our proofs fully capture single-occurrence “dust” groups. We further introduce a novel constrained version of the ETPF that we use to establish an intuitive connection between the probability functions for clustering, feature allocations, and trait allocations. As an application of our general theory, we characterize the distribution of all edge-exchangeable graphs, a class of recently-developed models that captures realistic sparse graph sequences.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 2290-2322.

Dates
Received: September 2016
First available in Project Euclid: 25 July 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1532484331

Digital Object Identifier
doi:10.1214/18-EJS1455

Mathematical Reviews number (MathSciNet)
MR3832093

Zentralblatt MATH identifier
06917477

Keywords
Trait allocation exchangeability paintbox probability function partition feature allocation graph vertex allocation edge exchangeability

Rights
Creative Commons Attribution 4.0 International License.

Citation

Campbell, Trevor; Cai, Diana; Broderick, Tamara. Exchangeable trait allocations. Electron. J. Statist. 12 (2018), no. 2, 2290--2322. doi:10.1214/18-EJS1455. https://projecteuclid.org/euclid.ejs/1532484331


Export citation

References

  • Ackerman, N. (2015). Representations of aut(M)-invariant measures: part 1., arXiv:1509.06170.
  • Aldous, D. (1981). Representations for partially exchangeable arrays of random variables., Journal of Multivariate Analysis 11 581–598.
  • Aldous, D. (1985)., Exchangeability and related topics. École d’été de probabilités de Saint-Flour, XIII. Springer, Berlin.
  • Borgs, C., Chayes, J., Cohn, H. and Holden, N. (2018). Sparse exchangeable graphs and their limits via graphon processes., Journal of Machine Learning Research 18 1–71.
  • Broderick, T. and Cai, D. (2015). Edge-exchangeable graphs and sparsity. In, NIPS Workshop on Networks in the Social and Information Sciences.
  • Broderick, T., Pitman, J. and Jordan, M. (2013). Feature allocations, probability functions, and paintboxes., Bayesian Analysis 8 801–836.
  • Broderick, T. and Steorts, R. (2014). Variational Bayes for merging noisy databases. In, NIPS Workshop on Advances in Variational Inference.
  • Broderick, T., Wilson, A. and Jordan, M. (2018). Posteriors, conjugacy, and exponential families for completely random measures., Bernoulli 24 3181–3221.
  • Broderick, T., Mackey, L., Paisley, J. and Jordan, M. (2015). Combinatorial clustering and the beta negative binomial process., IEEE transactions on pattern analysis and machine intelligence 37 290–306.
  • Bustamante, C., Wakeley, J., Sawyer, S. and Hartl, D. (2001). Directional selection and the site-frequency spectrum., Genetics 159 1779–1788.
  • Cai, D., Campbell, T. and Broderick, T. (2016). Edge-exchangeable graphs and sparsity. In, Advances in Neural Information Processing Systems.
  • Caron, F. and Fox, E. (2017). Sparse graphs using exchangeable random measures., Journal of the Royal Statistical Society Series B 79 1295–1366.
  • Clauset, A., Shalizi, C. R. and Newman, M. (2009). Power-law distributions in empirical data., SIAM Review 51 661–703.
  • Crane, H. and Dempsey, W. (2015). A framework for statistical network modeling., arXiv:1509.08185.
  • Crane, H. and Dempsey, W. (2016a). Edge exchangeable models for network data., arXiv:1603.04571v3.
  • Crane, H. and Dempsey, W. (2016b). Relational Exchangeability., arXiv: 1607.06762v1.
  • Crane, H. and Towsner, H. (2015). Relatively exchangeable structures., arXiv:1509.06733.
  • de Finetti, B. (1931). Funzione caratteristica di un fenomeno aleatorio., Atti della R. Academia Nazionale dei Lincei, Serie 6. 4 251–299. In Italian.
  • Doshi-Velez, F. and Williamson, S. (2017). Restricted Indian buffet processes., Statistical Computing 27 1205–1223.
  • Dummit, D. and Foote, R. (2004)., Abstract algebra, 3rd ed. Wiley, Hoboken.
  • Escobar, M. (1994). Estimating normal means with a Dirichlet process prior., Journal of the American Statistical Association 89 268–277.
  • Escobar, M. and West, M. (1995). Bayesian density estimation and inference using mixtures., Journal of the American Statistical Association 90 577–588.
  • Goldenberg, A., Zheng, A., Fienberg, S. and Airoldi, E. (2010). A survey of statistical network models., Foundations and Trends in Machine Learning 2 129–233.
  • Griffiths, T. and Ghahramani, Z. (2005). Infinite latent feature models and the Indian buffet process. In, Advances in Neural Information Processing Systems.
  • Herlau, T. and Schmidt, M. (2016). Completely random measures for modelling block-structured sparse networks. In, Advances in Neural Information Processing Systems.
  • Hewitt, E. and Savage, L. (1955). Symmetric measures on Cartesian products., Transactions of the American Mathematical Society 80 470–501.
  • Hoover, D. (1979). Relations on probability spaces and arrays of random variables. Preprint, Institute for Advanced Study, Princeton, University.
  • Ishwaran, H. and James, L. (2001). Gibbs sampling methods for stick-breaking priors., Journal of the American Statistical Association 96 161–173.
  • Ishwaran, H. and James, L. (2003). Generalized weighted Chinese restaurant processes for species sampling mixture models., Statistica Sinica 13.
  • James, L. (2017). Bayesian Poisson calculus for latent feature modeling via generalized Indian buffet process priors., Annals of Statistics 45 2016–2045.
  • Jordan, M. (2010). Bayesian nonparametric learning: expressive priors for intelligent systems. In, Heuristics, probability and causality: a tribute to Judea Pearl (R. Dechter, H. Geffner and J. Halpern, eds.) College Publications.
  • Kallenberg, O. (1997)., Foundations of modern probability, 1st ed. Probability and its applications. Springer, New York.
  • Kingman, J. F. C. (1978). The representation of partition structures., Journal of the London Mathematical Society 2 374–380.
  • Lee, J., Quintana, F., Müller, P. and Trippa, L. (2013). Defining predictive probability functions for species sampling models., Statistical Science 28 209–222.
  • Lloyd, J. R., Orbanz, P., Ghahramani, Z. and Roy, D. (2012). Random function priors for exchangeable arrays with applications to graphs and relational data. In, Advances in Neural Information Processing Systems.
  • Miller, J., Betancourt, B., Zaidi, A., Wallach, H. and Steorts, R. (2016). Microclustering: when the cluster sizes grow sublinearly with the size of the data set., Advances in Neural Information Processing Systems.
  • Mitzenmacher, M. (2003). A brief history of generative models for power law and lognormal distributions., Internet Mathematics 1 226–251.
  • Newman, M. (2005). Power laws, Pareto distributions and Zipf’s law., Contemporary physics 46 323–351.
  • Palla, K., Caron, F. and Teh, Y. W. (2016). Bayesian nonparametrics for sparse dynamic networks., arXiv:1607.01624.
  • Pitman, J. (1995). Exchangeable and partially exchangeable random partitions., Probability Theory and Related Fields 102 145–158.
  • Pitman, J. and Yor, M. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator., The Annals of Probability 855–900.
  • Ross, N. (2011). Fundamentals of Stein’s method., Probability Surveys 8 210–293.
  • Roy, D. (2014). The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof., arXiv:1501.00208.
  • Roychowdhury, A. and Kulis, B. (2015). Gamma processes, stick-breaking, and variational inference. In, International Conference on Artificial Intelligence and Statistics.
  • Thibaux, R. and Jordan, M. (2007). Hierarchical beta processes and the Indian buffet process. In, International Conference on Artificial Intelligence and Statistics.
  • Titsias, M. (2008). The infinite gamma-Poisson feature model. In, Advances in Neural Information Processing Systems.
  • Veitch, V. and Roy, D. (2015). The class of random graphs arising from exchangeable random measures., arXiv:1512.03099.
  • Wallach, H., Jensen, S., Dicker, L. and Heller, K. (2010). an alternative prior process for nonparametric Bayesian clustering. In, International Conference on Artificial Intelligence and Statistics.
  • Williamson, S. (2016). Nonparametric network models for link prediction., Journal of Machine Learning Research 17 1–21.
  • Williamson, S., MacEachern, S. and Xing, E. (2013). Restricting exchangeable nonparametric distributions. In, Advances in Neural Information Processing Systems.
  • Zhou, M. (2014). Beta-negative binomial process and exchangeable random partitions for mixed-membership modeling. In, Advances in Neural Information Processing Systems.
  • Zhou, M., Hannah, L., Dunson, D. and Carin, L. (2012). Beta-negative binomial process and Poisson factor analysis. In, International Conference on Artificial Intelligence and Statistics.