Bayesian Analysis

Gibbs-type Indian Buffet Processes

Abstract

We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-parameter) Indian buffet process, corresponding to the Pitman–Yor process. Asymptotic behavior of the Gibbs-type partitions, such as power laws holding for the number of latent clusters, translates into analogous characteristics for this class of Gibbs-type feature allocation models. Despite containing several different distinct subclasses, the properties of Gibbs-type partitions allow us to develop a black-box procedure for posterior inference within any subclass of models. Through numerical experiments, we compare and contrast a few of these subclasses and highlight the utility of varying power-law behaviors in the latent features.

Article information

Source
Bayesian Anal., Volume 15, Number 3 (2020), 683-710.

Dates
First available in Project Euclid: 19 June 2019

https://projecteuclid.org/euclid.ba/1560909812

Digital Object Identifier
doi:10.1214/19-BA1166

Citation

Heaukulani, Creighton; Roy, Daniel M. Gibbs-type Indian Buffet Processes. Bayesian Anal. 15 (2020), no. 3, 683--710. doi:10.1214/19-BA1166. https://projecteuclid.org/euclid.ba/1560909812

References

• Broderick, T., Jordan, M. I., and Pitman, J. (2012). “Beta Processes, Stick-Breaking and Power Laws.” Bayesian Analysis, 7(2): 439–476.
• Broderick, T., Pitman, J., and Jordan, M. I. (2013). “Feature allocations, probability functions, and paintboxes.” Bayesian Analysis, 8(4): 801–836.
• Charalambides, C. A. (2005). Combinatorial methods in discrete distributions. John Wiley & Sons.
• De Blasi, P., Favaro, S., Lijoi, A., Mena, R., Prünster, I., and Ruggiero, M. (2014). “Are Gibbs-type priors the most natural generalization of the Dirichlet process?” IEEE Transactions on Pattern Analysis and Machine Intelligence. Special issue on Bayesian nonparametrics.
• Doshi-Velez, F., Miller, K. T., Gael, J. V., and Teh, Y. W. (2009). “Variational inference for the Indian buffet process.” In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics.
• Favaro, S., Lomeli, M., Nipoti, B., and Teh, Y. W. (2014). “On the stick-breaking representation of $\sigma$-stable Poisson–Kingman models.” Electronic Journal of Statistics, 8(1): 1063–1085.
• Favaro, S. and Walker, S. G. (2013). “Slice sampling $\sigma$-stable Poisson–Kingman mixture models.” Journal of Computational and Graphical Statistics, 22(4): 830–847.
• Ghahramani, Z., Griffiths, T. L., and Sollich, P. (2007). “Bayesian nonparametric latent feature models.” Bayesian Statistics, 8: 201–226. See also the discussion and rejoinder.
• Gnedin, A. (2010). “A species sampling model with finitely many types.” Electronic Communications in Probability, 15: 79–88.
• Gnedin, A. and Pitman, J. (2006). “Exchangeable Gibbs partitions and Stirling triangles.” Journal of Mathematical Sciences, 138(3): 5674–5685.
• Griffiths, T. L. and Ghahramani, Z. (2006). “Infinite latent feature models and the Indian buffet process.” In Advances in Neural Information Processing Systems 19. Creighton Heaukulani and Daniel M. Roy
• Heaukulani, C. and Roy, D. M. (2019). “Supplementary Material: Gibbs-type Indian buffet processes.” Bayesian Analysis.
• Hjort, N. L. (1990). “Nonparametric Bayes estimators based on beta processes in models for life history data.” The Annals of Statistics, 18(3): 1259–1294.
• Ishwaran, H. and James, L. F. (2001). “Gibbs sampling methods for stick-breaking priors.” Journal of the American Statistical Association, 96(453).
• James, L. F. (2017). “Bayesian Poisson calculus for latent feature modeling via generalized Indian Buffet Process priors.” The Annals of Statistics, 45(5).
• Kallenberg, O. (2002). Foundations of Modern Probability. New York: Springer, 2nd edition.
• Kim, Y. (1999). “Nonparametric Bayesian estimators for counting processes.” The Annals of Statistics, 27(2): 562–588.
• Kingman, J. F. C. (1975). “Random discrete distributions.” Journal of the Royal Statistical Society, Series B, 37(1): 1–22.
• Kingman, J. F. C. (1978). “The representation of partition structures.” Journal of the London Mathematical Society, 2(2): 374–380.
• Korwar, R. M. and Hollander, M. (1973). “Contributions to the theory of Dirichlet processes.” The Annals of Probability, 1(4): 705–711.
• Lijoi, A., Mena, R. H., and Prünster, I. (2005). “Hierarchical mixture modeling with normalized inverse-Gaussian priors.” Journal of the American Statistical Association, 100(472): 1278–1291.
• Meeds, E., Ghahramani, Z., Neal, R. M., and Roweis, S. T. (2007). “Modeling dyadic data with binary latent factors.” In Advances in Neural Information Processing Systems 20.
• Neal, R. M. (2003). “Slice Sampling.” The Annals of Statistics, 31(3): 705–741.
• Paisley, J., Carin, L., and Blei, D. M. (2011). “Variational inference for stick-breaking beta process priors.” In Proceedings of the 28th International Conference on Machine Learning.
• Paisley, J., Zaas, A., Woods, C. W., Ginsburg, G. S., and Carin, L. (2010). “A stick-breaking construction of the beta process.” In Proceedings of the 27th International Conference on Machine Learning.
• Perman, M., Pitman, J., and Yor, M. (1992). “Size-biased sampling of Poisson point processes and excursions.” Probability Theory and Related Fields, 92(1): 21–39.
• Pitman, J. (1995). “Exchangeable and partially exchangeable random partitions.” Probability theory and related fields, 102(2): 145–158.
• Pitman, J. (2002). Combinatorial stochastic processes. Springer. Presented as a lecture course at the 32nd Summer School on Probability Theory held in Saint-Flour, July 2002. Available online.
• Pitman, J. (2003). “Poisson–Kingman partitions.” In Statistics and science: a Festschrift for Terry Speed, 1–34. Institute of Mathematical Statistics.
• Pitman, J. and Yor, M. (1997). “The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator.” The Annals of Probability, 25(2): 855–900.
• Roy, D. M. (2014). “The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof.” arXiv preprint 1501.00208 [math.PR] (version 1).
• Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.” Statistica Sinica, 4: 639–650.
• Teh, Y. W. and Görür, D. (2009). “Indian Buffet Processes with Power-law Behavior.” In Advances in Neural Information Processing Systems 22.
• Teh, Y. W., Görür, D., and Ghahramani, Z. (2007). “Stick-breaking Construction for the Indian Buffet Process.” In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics.
• Thibaux, R. and Jordan, M. I. (2007). “Hierarchical Beta Processes and the Indian Buffet Process.” In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics.