The Annals of Statistics

Bayesian Poisson calculus for latent feature modeling via generalized Indian Buffet Process priors

Lancelot F. James

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Statistical latent feature models, such as latent factor models, are models where each observation is associated with a vector of latent features. A general problem is how to select the number/types of features, and related quantities. In Bayesian statistical machine learning, one seeks (nonparametric) models where one can learn such quantities in the presence of observed data. The Indian Buffet Process (IBP), devised by Griffiths and Ghahramani (2005), generates a (sparse) latent binary matrix with columns representing a potentially unbounded number of features and where each row corresponds to an individual or object. Its generative scheme is cast in terms of customers entering sequentially an Indian Buffet restaurant and selecting previously sampled dishes as well as new dishes. Dishes correspond to latent features shared by individuals. The IBP has been applied to a wide range of statistical problems. Recent works have demonstrated the utility of generalizations to nonbinary matrices. The purpose of this work is to describe a unified mechanism for construction, Bayesian analysis, and practical sampling of broad generalizations of the IBP that generate (sparse) matrices with general entries. An adaptation of the Poisson partition calculus is employed to handle the complexities, including combinatorial aspects, of these models. Our work reveals a spike and slab characterization, and also presents a general framework for multivariate extensions. We close by highlighting a multivariate IBP with condiments, and the role of a stable-Beta Dirichlet multivariate prior.

Article information

Source
Ann. Statist., Volume 45, Number 5 (2017), 2016-2045.

Dates
Received: April 2015
Revised: November 2015
First available in Project Euclid: 31 October 2017

Permanent link to this document
https://projecteuclid.org/euclid.aos/1509436826

Digital Object Identifier
doi:10.1214/16-AOS1517

Mathematical Reviews number (MathSciNet)
MR3718160

Zentralblatt MATH identifier
06821117

Subjects
Primary: 60C05: Combinatorial probability 60G09: Exchangeability
Secondary: 60G57: Random measures 60E99: None of the above, but in this section

Keywords
Bayesian statistical machine learning Indian buffet process nonparametric latent feature models Poisson process calculus spike and slab priors

Citation

James, Lancelot F. Bayesian Poisson calculus for latent feature modeling via generalized Indian Buffet Process priors. Ann. Statist. 45 (2017), no. 5, 2016--2045. doi:10.1214/16-AOS1517. https://projecteuclid.org/euclid.aos/1509436826


Export citation

References

  • [1] Archambeau, C., Lakshminarayanan, B. and Bouchard, G. (2015). Latent IBP compound Dirichlet allocation. IEEE Trans. Pattern Anal. Mach. Intell. 37 321–333.
  • [2] Arratia, R., Barbour, A. D. and Tavaré, S. (2003). Logarithmic Combinatorial Structures: A Probabilistic Approach. European Mathematical Society (EMS), Zürich.
  • [3] Barndorff-Nielsen, O. E., Pedersen, J. and Sato, K. (2001). Multivariate subordination, self-decomposability and stability. Adv. in Appl. Probab. 33 160–187.
  • [4] Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2015). Central limit theorems for an Indian buffet model with random weights. Ann. Appl. Probab. 25 523–547.
  • [5] Broderick, T., Jordan, M. I. and Pitman, J. (2013). Cluster and feature modeling from combinatorial stochastic processes. Statist. Sci. 28 289–312.
  • [6] Broderick, T., Mackey, L., Paisley, J. and Jordan, M. I. (2015). Combinatorial clustering and the beta negative binomial process. IEEE Trans. Pattern Anal. Mach. Intell. 37 290–306.
  • [7] Broderick, T., Pitman, J. and Jordan, M. I. (2013). Feature allocations, probability functions, and paintboxes. Bayesian Anal. 8 801–836.
  • [8] Broderick, T., Wilson, A. C. and Jordan, M. I. (2017). Posteriors, conjugacy, and exponential families for completely random measures. Bernoulli. To appear. DOI:10.3150/16-BEJ855.
  • [9] Caron, F. (2012). Bayesian nonparametric models for bipartite graphs. In Neural Information Processing Systems (NIPS 2012), Lake Tahoe, CA.
  • [10] Doksum, K. (1974). Tailfree and neutral random probabilities and their posterior distributions. Ann. Probab. 2 183–201.
  • [11] Gershman, S. J., Frazier, P. I. and Blei, D. M. (2015). Distance dependent infinite latent feature models. IEEE Trans. Pattern Anal. Mach. Intell. 37 334–345.
  • [12] Ghahramani, Z., Griffiths, T. L. and Sollich, P. (2007). Bayesian nonparametric latent feature models. In Bayesian Statistics 8. 201–226. Oxford Univ. Press, Oxford.
  • [13] Gnedin, A. and Pitman, J. (2005). Regenerative composition structures. Ann. Probab. 33 445–479.
  • [14] Görur, D., Jäkel, F. and Rasmussen, C. E. (2006). A choice model with infinitely many latent features. In Proceedings of the 23rd International Conference on Machine Learning 361–368. ACM, New York.
  • [15] Griffiths, T. L. and Ghahramani, Z. (2006). Infinite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems 18 (NIPS-2005).
  • [16] Griffiths, T. L. and Ghahramani, Z. (2011). The Indian buffet process: An introduction and review. J. Mach. Learn. Res. 12 1185–1224.
  • [17] Heaukulani, C. and Roy, D. M. (2016). The combinatorial structure of beta negative binomial processes. Bernoulli 22 2301–2324.
  • [18] Hjort, N. L. (1990). Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist. 18 1259–1294.
  • [19] Hoff, P. D., Raftery, A. E. and Handcock, M. S. (2002). Latent space approaches to social network analysis. J. Amer. Statist. Assoc. 97 1090–1098.
  • [20] Ishwaran, H. and James, L. F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96 161–173.
  • [21] Ishwaran, H. and James, L. F. (2004). Computational methods for multiplicative intensity models using weighted gamma processes: Proportional hazards, marked point processes, and panel count data. J. Amer. Statist. Assoc. 99 175–190.
  • [22] Ishwaran, H. and Rao, J. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. Ann. Statist. 33 730–773.
  • [23] James, L. F. (2002). Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics. Unpublished manuscript. Available at arXiv:math.PR/0205093.
  • [24] James, L. F. (2003). Bayesian calculus for gamma processes with applications to semiparametric intensity models. Sankhyā 65 179–206.
  • [25] James, L. F. (2005). Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages. Ann. Statist. 33 1771–1799.
  • [26] James, L. F. (2006). Poisson calculus for spatial neutral to the right processes. Ann. Statist. 34 416–440.
  • [27] James, L. F. (2014). Poisson latent feature calculus for generalized Indian buffet processes. Preprint. Available at arXiv:1411.2936.
  • [28] James, L. F., Orbanz, P. and Teh, Y. W. (2015). Scaled subordinators and generalizations of the Indian buffet process. Preprint. Available at arXiv:1510.07309.
  • [29] James, L. F., Roynette, B. and Yor, M. (2008). Generalized gamma convolutions, Dirichlet means, Thorin measures, with explicit examples. Probab. Surv. 5 346–415.
  • [30] Kang, H. and Choi, S. (2013). Bayesian multi-subject common spatial patterns with Indian buffet process priors. In ICASSP 3347–3351. IEEE, New York.
  • [31] Kim, Y. (1999). Nonparametric Bayesian estimators for counting processes. Ann. Statist. 27 562–588.
  • [32] Kim, Y., James, L. and Weissbach, R. (2012). Bayesian analysis of multistate event history data: Beta-Dirichlet process prior. Biometrika 99 127–140.
  • [33] Kim, Y. and Lee, J. (2001). On posterior consistency of survival models. Ann. Statist. 29 666–686.
  • [34] Knowles, D. and Ghahramani, Z. (2011). Nonparametric Bayesian sparse factor models with application to gene expression modeling. Ann. Appl. Stat. 5 1534–1552.
  • [35] Lee, J., Müller, P., Sengupta, S., Gulukota, K. and Ji, Y. (2014). Bayesian inference for tumor subclones accounting for sequencing and structural variants. Preprint. Available at arXiv:1409.7158 [stat.ME].
  • [36] Lo, A. Y. (1982). Bayesian nonparametric statistical inference for Poisson point processes. Z. Wahrsch. Verw. Gebiete 59 55–66.
  • [37] Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates. I. Density estimates. Ann. Statist. 12 351–357.
  • [38] Lo, A. Y. and Weng, C. S. (1989). On a class of Bayesian nonparametric estimates: II. Hazard rate estimates. Ann. Inst. Statist. Math. 41 227–245.
  • [39] Miller, K. T., Griffiths, T. L. and Jordan, M. I. (2009). Nonparametric latent feature models for link prediction. In Advances in Neural Information Processing Systems (NIPS) 22.
  • [40] Orbanz, P. and Roy, D. M. (2015). Bayesian models of graphs, arrays and other exchangeable random structures. IEEE Trans. Pattern Anal. Mach. Intell. 37 437–461.
  • [41] Paisley, J. and Carin, L. (2009). Nonparametric factor analysis with beta process priors. In International Conference on Machine Learning (ICML), Montreal, Canada.
  • [42] Palla, K., Knowles, D. and Ghahramani, Z. (2012). An infinite latent attribute model for network data. In ICML 2012.
  • [43] Pitman, J. (1996). Some developments of the Blackwell–MacQueen urn scheme. In Statistics, Probability and Game Theory. Institute of Mathematical Statistics Lecture Notes—Monograph Series 30 245–267. IMS, Hayward, CA.
  • [44] Pitman, J. (1997). Partition structures derived from Brownian motion and stable subordinators. Bernoulli 3 79–96.
  • [45] Pitman, J. (2003). Poisson–Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed. Institute of Mathematical Statistics Lecture Notes—Monograph Series 40 1–34. IMS, Beachwood, OH.
  • [46] Pitman, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Math. 1875. Springer, Berlin. Lectures from the 32nd Summer School on Probability Theory held in Saint-Flour, July 7–24, 2002; With a foreword by Jean Picard.
  • [47] Rai, P. and Daume, H. (2009). The infinite hierarchical factor regression model. In Advances in Neural Information Processing Systems 1321–1328.
  • [48] Teh, Y. W. and Görür, D. (2009). Indian buffet processes with power-law behavior. In NIPS 2009.
  • [49] Thibaux, R. (2008). Nonparametric Bayesian Models for Machine Learning. Doctoral dissertation, Univ. California, Berkeley.
  • [50] Thibaux, R. and Jordan, M. I. (2007). Hierarchical beta processes and the Indian buffet process. In International Conference on Artificial Intelligence and Statistics 564–571.
  • [51] Titsias, M. K. (2008). The infinite gamma-Poisson feature model. In Advances in Neural Information Processing Systems.
  • [52] Williamson, S., Wang, C., Heller, K. and Blei, D. (2010). The IBP compound Dirichlet process and its application to focused topic modeling. In The 27th International Conference on Machine Learning (ICML 2010).
  • [53] Zhou, M., Hannah, L., Dunson, D. and Carin, L. (2012). Beta-negative binomial process and Poisson factor analysis. AISTATS. Preprint. Available at arXiv:1112.3605.
  • [54] Zhou, M., Madrid-Padilla, O-H. and Scott, J. G. (2016). Priors for random count matrices derived from a family of negative binomial processes. J. Amer. Statist. Assoc. 111 1144–1156.