Bayesian Analysis

Gibbs-type Indian Buffet Processes

Creighton Heaukulani and Daniel M. Roy

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-parameter) Indian buffet process, corresponding to the Pitman–Yor process. Asymptotic behavior of the Gibbs-type partitions, such as power laws holding for the number of latent clusters, translates into analogous characteristics for this class of Gibbs-type feature allocation models. Despite containing several different distinct subclasses, the properties of Gibbs-type partitions allow us to develop a black-box procedure for posterior inference within any subclass of models. Through numerical experiments, we compare and contrast a few of these subclasses and highlight the utility of varying power-law behaviors in the latent features.

Article information

Source
Bayesian Anal., Advance publication (2018), 28 pages.

Dates
First available in Project Euclid: 19 June 2019

Permanent link to this document
https://projecteuclid.org/euclid.ba/1560909812

Digital Object Identifier
doi:10.1214/19-BA1166

Keywords
feature allocation partition combinatorial stochastic processes completely random measure Bayesian nonparametrics

Rights
Creative Commons Attribution 4.0 International License.

Citation

Heaukulani, Creighton; Roy, Daniel M. Gibbs-type Indian Buffet Processes. Bayesian Anal., advance publication, 19 June 2019. doi:10.1214/19-BA1166. https://projecteuclid.org/euclid.ba/1560909812


Export citation

References

  • Broderick, T., Jordan, M. I., and Pitman, J. (2012). “Beta Processes, Stick-Breaking and Power Laws.” Bayesian Analysis, 7(2): 439–476.
  • Broderick, T., Pitman, J., and Jordan, M. I. (2013). “Feature allocations, probability functions, and paintboxes.” Bayesian Analysis, 8(4): 801–836.
  • Charalambides, C. A. (2005). Combinatorial methods in discrete distributions. John Wiley & Sons.
  • De Blasi, P., Favaro, S., Lijoi, A., Mena, R., Prünster, I., and Ruggiero, M. (2014). “Are Gibbs-type priors the most natural generalization of the Dirichlet process?” IEEE Transactions on Pattern Analysis and Machine Intelligence. Special issue on Bayesian nonparametrics.
  • Doshi-Velez, F., Miller, K. T., Gael, J. V., and Teh, Y. W. (2009). “Variational inference for the Indian buffet process.” In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics.
  • Favaro, S., Lomeli, M., Nipoti, B., and Teh, Y. W. (2014). “On the stick-breaking representation of $\sigma $-stable Poisson–Kingman models.” Electronic Journal of Statistics, 8(1): 1063–1085.
  • Favaro, S. and Walker, S. G. (2013). “Slice sampling $\sigma $-stable Poisson–Kingman mixture models.” Journal of Computational and Graphical Statistics, 22(4): 830–847.
  • Ghahramani, Z., Griffiths, T. L., and Sollich, P. (2007). “Bayesian nonparametric latent feature models.” Bayesian Statistics, 8: 201–226. See also the discussion and rejoinder.
  • Gnedin, A. (2010). “A species sampling model with finitely many types.” Electronic Communications in Probability, 15: 79–88.
  • Gnedin, A. and Pitman, J. (2006). “Exchangeable Gibbs partitions and Stirling triangles.” Journal of Mathematical Sciences, 138(3): 5674–5685.
  • Griffiths, T. L. and Ghahramani, Z. (2006). “Infinite latent feature models and the Indian buffet process.” In Advances in Neural Information Processing Systems 19. Creighton Heaukulani and Daniel M. Roy
  • Heaukulani, C. and Roy, D. M. (2019). “Supplementary Material: Gibbs-type Indian buffet processes.” Bayesian Analysis.
  • Hjort, N. L. (1990). “Nonparametric Bayes estimators based on beta processes in models for life history data.” The Annals of Statistics, 18(3): 1259–1294.
  • Ishwaran, H. and James, L. F. (2001). “Gibbs sampling methods for stick-breaking priors.” Journal of the American Statistical Association, 96(453).
  • James, L. F. (2017). “Bayesian Poisson calculus for latent feature modeling via generalized Indian Buffet Process priors.” The Annals of Statistics, 45(5).
  • Kallenberg, O. (2002). Foundations of Modern Probability. New York: Springer, 2nd edition.
  • Kim, Y. (1999). “Nonparametric Bayesian estimators for counting processes.” The Annals of Statistics, 27(2): 562–588.
  • Kingman, J. F. C. (1975). “Random discrete distributions.” Journal of the Royal Statistical Society, Series B, 37(1): 1–22.
  • Kingman, J. F. C. (1978). “The representation of partition structures.” Journal of the London Mathematical Society, 2(2): 374–380.
  • Korwar, R. M. and Hollander, M. (1973). “Contributions to the theory of Dirichlet processes.” The Annals of Probability, 1(4): 705–711.
  • Lijoi, A., Mena, R. H., and Prünster, I. (2005). “Hierarchical mixture modeling with normalized inverse-Gaussian priors.” Journal of the American Statistical Association, 100(472): 1278–1291.
  • Meeds, E., Ghahramani, Z., Neal, R. M., and Roweis, S. T. (2007). “Modeling dyadic data with binary latent factors.” In Advances in Neural Information Processing Systems 20.
  • Neal, R. M. (2003). “Slice Sampling.” The Annals of Statistics, 31(3): 705–741.
  • Paisley, J., Carin, L., and Blei, D. M. (2011). “Variational inference for stick-breaking beta process priors.” In Proceedings of the 28th International Conference on Machine Learning.
  • Paisley, J., Zaas, A., Woods, C. W., Ginsburg, G. S., and Carin, L. (2010). “A stick-breaking construction of the beta process.” In Proceedings of the 27th International Conference on Machine Learning.
  • Perman, M., Pitman, J., and Yor, M. (1992). “Size-biased sampling of Poisson point processes and excursions.” Probability Theory and Related Fields, 92(1): 21–39.
  • Pitman, J. (1995). “Exchangeable and partially exchangeable random partitions.” Probability theory and related fields, 102(2): 145–158.
  • Pitman, J. (2002). Combinatorial stochastic processes. Springer. Presented as a lecture course at the 32nd Summer School on Probability Theory held in Saint-Flour, July 2002. Available online.
  • Pitman, J. (2003). “Poisson–Kingman partitions.” In Statistics and science: a Festschrift for Terry Speed, 1–34. Institute of Mathematical Statistics.
  • Pitman, J. and Yor, M. (1997). “The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator.” The Annals of Probability, 25(2): 855–900.
  • Roy, D. M. (2014). “The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof.” arXiv preprint 1501.00208 [math.PR] (version 1).
  • Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.” Statistica Sinica, 4: 639–650.
  • Teh, Y. W. and Görür, D. (2009). “Indian Buffet Processes with Power-law Behavior.” In Advances in Neural Information Processing Systems 22.
  • Teh, Y. W., Görür, D., and Ghahramani, Z. (2007). “Stick-breaking Construction for the Indian Buffet Process.” In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics.
  • Thibaux, R. and Jordan, M. I. (2007). “Hierarchical Beta Processes and the Indian Buffet Process.” In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics.

Supplemental materials