Bernoulli

  • Bernoulli
  • Volume 22, Number 4 (2016), 2301-2324.

The combinatorial structure of beta negative binomial processes

Creighton Heaukulani and Daniel M. Roy

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We characterize the combinatorial structure of conditionally-i.i.d. sequences of negative binomial processes with a common beta process base measure. In Bayesian nonparametric applications, such processes have served as models for latent multisets of features underlying data. Analogously, random subsets arise from conditionally-i.i.d. sequences of Bernoulli processes with a common beta process base measure, in which case the combinatorial structure is described by the Indian buffet process. Our results give a count analogue of the Indian buffet process, which we call a negative binomial Indian buffet process. As an intermediate step toward this goal, we provide a construction for the beta negative binomial process that avoids a representation of the underlying beta process base measure. We describe the key Markov kernels needed to use a NB-IBP representation in a Markov Chain Monte Carlo algorithm targeting a posterior distribution.

Article information

Source
Bernoulli, Volume 22, Number 4 (2016), 2301-2324.

Dates
Received: June 2014
Revised: March 2015
First available in Project Euclid: 3 May 2016

Permanent link to this document
https://projecteuclid.org/euclid.bj/1462297682

Digital Object Identifier
doi:10.3150/15-BEJ729

Mathematical Reviews number (MathSciNet)
MR3498030

Zentralblatt MATH identifier
1358.60069

Keywords
Bayesian nonparametrics Indian buffet process latent feature models multisets

Citation

Heaukulani, Creighton; Roy, Daniel M. The combinatorial structure of beta negative binomial processes. Bernoulli 22 (2016), no. 4, 2301--2324. doi:10.3150/15-BEJ729. https://projecteuclid.org/euclid.bj/1462297682


Export citation

References

  • [1] Barndorff-Nielsen, O. and Yeo, G.F. (1969). Negative binomial processes. J. Appl. Probab. 6 633–647.
  • [2] Broderick, T., Jordan, M.I. and Pitman, J. (2013). Cluster and feature modeling from combinatorial stochastic processes. Statist. Sci. 28 289–312.
  • [3] Broderick, T., Mackey, L., Paisley, J. and Jordan, M.I. (2014). Combinatorial clustering and the beta-negative binomial process. IEEE Trans. Pattern Anal. Mach. Intell. 37 290–306. Special issue on Bayesian nonparametrics.
  • [4] Broderick, T., Pitman, J. and Jordan, M.I. (2013). Feature allocations, probability functions, and paintboxes. Bayesian Anal. 8 801–836.
  • [5] Canny, J. (2004). Gap: A factor model for discrete data. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, United Kingdom.
  • [6] Devroye, L. (1992). Random variate generation for the digamma and trigamma distributions. J. Stat. Comput. Simul. 43 197–216.
  • [7] Ghahramani, Z., Griffiths, T.L. and Sollich, P. (2007). Bayesian nonparametric latent feature models. In Bayesian Statistics 8. Oxford Sci. Publ. 201–226. Oxford: Oxford Univ. Press.
  • [8] Gopalan, P., Ruiz, F.J.R., Ranganath, R. and Blei, D.M. (2014). Bayesian nonparametric Poisson factorization for recommendation systems. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, Reykjavik, Iceland.
  • [9] Grégoire, G. (1984). Negative binomial distributions for point processes. Stochastic Process. Appl. 16 179–188.
  • [10] Griffiths, T.L. and Ghahramani, Z. (2006). Infinite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems 19, Vancouver, Canada.
  • [11] Hjort, N.L. (1990). Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist. 18 1259–1294.
  • [12] Kallenberg, O. (2002). Foundations of Modern Probability, 2nd ed. Probability and Its Applications (New York). New York: Springer.
  • [13] Kim, Y. (1999). Nonparametric Bayesian estimators for counting processes. Ann. Statist. 27 562–588.
  • [14] Kingman, J.F.C. (1967). Completely random measures. Pacific J. Math. 21 59–78.
  • [15] Kozubowski, T.J. and Podgórski, K. (2009). Distributional properties of the negative binomial Lévy process. Probab. Math. Statist. 29 43–71.
  • [16] Lo, A.Y. (1982). Bayesian nonparametric statistical inference for Poisson point processes. Z. Wahrsch. Verw. Gebiete 59 55–66.
  • [17] Meeds, E., Ghahramani, Z., Neal, R.M. and Roweis, S.T. (2007). Modeling dyadic data with binary latent factors. In Advances in Neural Information Processing Systems 20, Vancouver, Canada.
  • [18] Neal, R.M. (2003). Slice sampling. Ann. Statist. 31 705–767. With discussions and a rejoinder by the author.
  • [19] Paisley, J., Zaas, A., Woods, C.W., Ginsburg, G.S. and Carin, L. (2010). A stick-breaking construction of the beta process. In Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel.
  • [20] Roy, D.M. (2014). The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof. Preprint. Available at arXiv:1501.00208.
  • [21] Sibuya, M. (1979). Generalized hypergeometric, digamma and trigamma distributions. Ann. Inst. Statist. Math. 31 373–390.
  • [22] Sudderth, E.B., Torralba, A., Freeman, W.T. and Willsky, A.S. (2005). Describing visual scenes using transformed Dirichlet processes. In Advances in Neural Information Processing Systems 18, Vancouver, Canada.
  • [23] Teh, Y.W., Görür, D. and Ghahramani, Z. (2007). Stick-breaking construction for the Indian buffet process. In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico.
  • [24] Thibaux, R. and Jordan, M.I. (2007). Hierarchical beta processes and the Indian buffet process. In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico.
  • [25] Thibaux, R.J. (2008). Nonparametric Bayesian models for machine learning. Ph.D. thesis, EECS Department, Univ. California, Berkeley.
  • [26] Titsias, M. (2007). The infinite gamma-Poisson feature model. In Advances in Neural Information Processing Systems 20.
  • [27] Wolpert, R.L. and Ickstadt, K. (1998). Poisson/gamma random field models for spatial statistics. Biometrika 85 251–267.
  • [28] Zhou, M., Hannah, L., Dunson, D. and Carin, L. (2012). Beta-negative binomial process and Poisson factor analysis. In Proceedings of the 29th International Conference on Machine Learning, Edinburgh, United Kingdom.
  • [29] Zhou, M., Madrid, O. and Scott, J.G. (2014). Priors for random count matrices derived from a family of negative binomial processes. Preprint. Available at arXiv:1404.3331v2.