## Statistical Science

### Sufficientness Postulates for Gibbs-Type Priors and Hierarchical Generalizations

#### Abstract

A fundamental problem in Bayesian nonparametrics consists of selecting a prior distribution by assuming that the corresponding predictive probabilities obey certain properties. An early discussion of such a problem, although in a parametric framework, dates back to the seminal work by English philosopher W. E. Johnson, who introduced a noteworthy characterization for the predictive probabilities of the symmetric Dirichlet prior distribution. This is typically referred to as Johnson’s “sufficientness” postulate. In this paper, we review some nonparametric generalizations of Johnson’s postulate for a class of nonparametric priors known as species sampling models. In particular, we revisit and discuss the “sufficientness” postulate for the two parameter Poisson–Dirichlet prior within the more general framework of Gibbs-type priors and their hierarchical generalizations.

#### Article information

Source
Statist. Sci., Volume 32, Number 4 (2017), 487-500.

Dates
First available in Project Euclid: 28 November 2017

https://projecteuclid.org/euclid.ss/1511838024

Digital Object Identifier
doi:10.1214/17-STS619

Mathematical Reviews number (MathSciNet)
MR3730518

Zentralblatt MATH identifier
1383.62079

#### Citation

Bacallado, S.; Battiston, M.; Favaro, S.; Trippa, L. Sufficientness Postulates for Gibbs-Type Priors and Hierarchical Generalizations. Statist. Sci. 32 (2017), no. 4, 487--500. doi:10.1214/17-STS619. https://projecteuclid.org/euclid.ss/1511838024

#### References

• [1] Antoniak, C. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist. 2 1152–1174.
• [2] Arbel, J., Favaro, S., Nipoti, B. and Teh, Y. W. (2017). Bayesian nonparametric inference for discovery probabilities: Credible intervals and large sample asymptotics. Statist. Sinica 27 839–858.
• [3] Bacallado, S., Battiston, M., Favaro, S. and Trippa, L. (2017). Supplement to “Sufficientness postulates for Gibbs-type priors and hierarchical generalizations.” DOI:10.1214/17-STS619SUPP.
• [4] Bacallado, S., Favaro, S. and Trippa, L. (2013). Bayesian nonparametric analysis of reversible Markov chains. Ann. Statist. 41 870–896.
• [5] Bacallado, S., Favaro, S. and Trippa, L. (2015). Looking-backward probabilities for Gibbs-type exchangeable random partitions. Bernoulli 21 1–37.
• [6] Bacallado, S., Favaro, S. and Trippa, L. (2015). Bayesian nonparametric inference for shared species richness in multiple populations. J. Statist. Plann. Inference 166 14–23.
• [7] Battiston, M., Favaro, S., Roy, D. M. and Teh, Y. W. (2016). A characterization of product-form exchangeable feature probability functions. Preprint. Available at arXiv:1607.02066.
• [8] Bertoin, J. (2006). Random Fragmentation and Coagulation Processes. Cambridge Studies in Advanced Mathematics 102. Cambridge Univ. Press, Cambridge.
• [9] Blackwell, D. and MacQueen, J. B. (1973). Ferguson distributions via Pólya urn schemes. Ann. Statist. 1 353–355.
• [10] Broderick, T., Pitman, J. and Jordan, M. I. (2013). Feature allocations, probability functions, and paintboxes. Bayesian Anal. 8 801–836.
• [11] Caron, F. (2012). Bayesian nonparametric models for bipartite graphs. In NIPS’12. Proc. 25th International Conference on Neural Information Processing Systems 2051–2059. Curran Associates Inc., Red Hook, NY.
• [12] Caron, F. and Fox, E. B. (2015). Sparse graphs with exchangeable random measures. Preprint. Available at arXiv:1401.1137.
• [13] Caron, F., Teh, Y. W. and Murphy, T. B. (2014). Bayesian nonparametric Plackett–Luce models for the analysis of preferences for college degree programmes. Ann. Appl. Stat. 8 1145–1181.
• [14] Chen, C., Ding, N. and Buntine, W. (2012). Dependent hierarchical normalized random measures for dynamic topic modeling. In Proc. 29th International Conference on Machine Learning (ICML-12) 895–902.
• [15] Chen, C., Rao, V. A., Buntine, W. and Teh, Y. W. (2013). Dependent normalized random measures. In Proc. 30th International Conference on Machine Learning (ICML-13) 969–977.
• [16] Crane, H. (2016). The ubiquitous Ewens sampling formula. Statist. Sci. 31 1–19.
• [17] de Finetti, B. (1938). Sur la condition d’“équivalence partielle.” In VI Colloque Gèneve: “Act. Sc. Ind.” Herman, Paris.
• [18] De Blasi, P., Favaro, S., Lijoi, A., Mena, R. H., Prünster, I. and Ruggiero, M. (2015). Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. 37 212–229.
• [19] Engen, S. (1978). Stochastic Abundance Models: With Emphasis on Biological Communities and Species Diversity. Chapman & Hall, London.
• [20] Ewens, W. J. (1972). The sampling theory of selectively neutral alleles. Theor. Popul. Biol. 3 87–112. Erratum, ibid. 3 240 (1972); erratum, ibid. 3 376 (1972).
• [21] Favaro, S., Lijoi, A., Mena, R. H. and Prünster, I. (2009). Bayesian non-parametric inference for species variety with a two-parameter Poisson–Dirichlet process prior. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 993–1008.
• [22] Favaro, S., Lijoi, A. and Prünster, I. (2012). A new estimator of the discovery probability. Biometrics 68 1188–1196.
• [23] Favaro, S. and Walker, S. G. (2013). Slice sampling $\sigma$-stable Poisson–Kingman mixture models. J. Comput. Graph. Statist. 22 830–847.
• [24] Feng, S. and Hoppe, F. M. (1998). Large deviation principles for some random combinatorial structures in population genetics and Brownian motion. Ann. Appl. Probab. 8 975–994.
• [25] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209–230.
• [26] Fisher, R. A., Corbet, A. S. and Williams, C. B. (1943). The relation between the number of species and the number of individuals in a random sample of an animal population. J. Anim. Ecol. 12 42–58.
• [27] Gnedin, A. and Pitman, J. (2006). Exchangeable Gibbs partitions and Stirling triangles. J. Math. Sci. 138 5674–5685.
• [28] Good, I. J. (1965). The Estimation of Probabilities. An Essay on Modern Bayesian Methods. Research Monograph 30. MIT Press, Cambridge, MA.
• [29] Heaukulani, C. and Roy, D. M. (2015). Gibbs-type Indian buffet processes. Preprint. Available at arXiv:1512.02543.
• [30] Herlau, T. (2015). Completely random measures for modeling block-structured sparse networks. Preprint. Available at arXiv:1507.02925.
• [31] Hjort, N. L. (2000). Bayesian analysis for a generalised Dirichlet process prior Technical report, Matematisk Institutt, Universitetet i Oslo.
• [32] Ho, M., James, L. F. and Lau, J. W. (2007). Gibbs partitions (EPPF’s) derived from a stable subordinator are Fox H and Meijer G transforms. Preprint. Available at arXiv:0708.0619.
• [33] Hoppe, F. M. (1984). Pólya-like urns and the Ewens’ sampling formula. J. Math. Biol. 20 91–94.
• [34] Ishwaran, H. and James, L. F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96 161–173.
• [35] James, L. F. (2002). Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics. Preprint. Available at arXiv:math/0205093.
• [36] James, L. F. (2013). Stick-breaking $\operatorname{PG}(\alpha,\zeta)$-generalized Gamma processes. Preprint. Available at arXiv:1308.6570.
• [37] Jo, S., Lee, J., Müller, P., Quintana, F. A. and Trippa, L. (2017). Dependent species sampling models for spatial density estimation. Bayesian Anal. 12 379–406.
• [38] Johnson, W. E. (1932). Probability: The deductive and inductive problems. Mind 41 409–423.
• [39] Kingman, J. F. C. (1978). The representation of partition structures. J. Lond. Math. Soc. (2) 18 374–380.
• [40] Korwar, R. M. and Hollander, M. (1973). Contributions to the theory of Dirichlet processes. Ann. Probab. 1 705–711.
• [41] Lee, J., Quintana, F. A., Müller, P. and Trippa, L. (2013). Defining predictive probability functions for species sampling models. Statist. Sci. 28 209–222.
• [42] Lijoi, A., Mena, R. H. and Prünster, I. (2005). Hierarchical mixture modeling with normalized inverse-Gaussian priors. J. Amer. Statist. Assoc. 100 1278–1291.
• [43] Lijoi, A., Mena, R. H. and Prünster, I. (2007). Bayesian nonparametric estimation of the probability of discovering new species. Biometrika 94 769–786.
• [44] Lijoi, A., Mena, R. H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian non-parametric mixture models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 69 715–740.
• [45] Lijoi, A. and Prünster, I. (2010). Models beyond the Dirichlet process. In Bayesian Nonparametrics. Camb. Ser. Stat. Probab. Math. 28 80–136. Cambridge Univ. Press, Cambridge.
• [46] Lijoi, A., Prünster, I. and Walker, S. G. (2008). Investigating nonparametric priors with Gibbs structure. Statist. Sinica 18 1653–1668.
• [47] Lijoi, A., Prünster, I. and Walker, S. G. (2008). Bayesian nonparametric estimators derived from conditional Gibbs structures. Ann. Appl. Probab. 18 1519–1547.
• [48] Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates. I. Density estimates. Ann. Statist. 12 351–357.
• [49] Lo, A. Y. (1991). A characterization of the Dirichlet process. Statist. Probab. Lett. 12 185–187.
• [50] Lomelí, M., Favaro, S. and Teh, Y. W. (2017). A marginal sampler for $\sigma$-stable Poisson–Kingman mixture models. J. Comput. Graph. Statist. 26 44–53.
• [51] Perman, M., Pitman, J. and Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probab. Theory Related Fields 92 21–39.
• [52] Pitman, J. (1995). Exchangeable and partially exchangeable random partitions. Probab. Theory Related Fields 102 145–158.
• [53] Pitman, J. (1996). Some developments of the Blackwell–MacQueen urn scheme. In Statistics, Probability and Game Theory. Institute of Mathematical Statistics Lecture Notes—Monograph Series 30 245–267. IMS, Hayward, CA.
• [54] Pitman, J. (2003). Poisson–Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed. Institute of Mathematical Statistics Lecture Notes—Monograph Series 40 1–34. IMS, Beachwood, OH.
• [55] Pitman, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Math. 1875. Springer, Berlin. Lectures from the 32nd Summer School on Probability Theory held in Saint-Flour, July 7–24, 2002, with a foreword by Jean Picard.
• [56] Pitman, J. and Yor, M. (1997). The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25 855–900.
• [57] Prünster, I. (2002). Random probability measures derived from increasing additive processes and their application to Bayesian statistics Ph.D. thesis, Univ. Pavia.
• [58] Quintana, F. A. and Iglesias, P. L. (2003). Bayesian clustering and product partition models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 65 557–574.
• [59] Ravel, J., Gajer, P., Abdo, Z., Schneider, G. M., Koenig, S. S., McCulle, S. L., Karlebach, S., Gorle, R., Russell, J., Tacket, C. O., Brotman, R. M., Davis, C. C., Ault, K., Peralta, L. and Forney, L. J. (2011). Vaginal microbiome of reproductive-age women. Proc. Natl. Acad. Sci. USA 108 4680–4687.
• [60] Regazzini, E. (1978). Intorno ad alcune questioni relative alla definizione del premio secondo la teoria della credibilià. G. Ist. Ital. Attuari 41 77–89.
• [61] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of normalized random measures with independent increments. Ann. Statist. 31 560–585.
• [62] Rolles, S. W. W. (2003). How edge-reinforced random walk arises naturally. Probab. Theory Related Fields 126 243–260.
• [63] Roy, D. M. (2014). The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof. Preprint. Available at arXiv:1501.00208.
• [64] Sato, K. (1999). Lévy Processes and Infinitely Divisible Distributions. Cambridge Studies in Advanced Mathematics 68. Cambridge Univ. Press, Cambridge. Translated from the 1990 Japanese original, revised by the author.
• [65] Teh, Y. W. (2006). A hierarchical Bayesian language model based on Pitman–Yor processes. In Proceedings of the 21st International Conference on Computational Linguistic.
• [66] Teh, Y. W. and Görür (2010). Indian buffet processes with power-law behavior. In NIPS’09. Proc. 22nd International Conference on Neural Information Processing Systems 1838–1846. Curran Associates Inc., Red Hook, NY.
• [67] Teh, Y. W. and Jordan, M. I. (2010). Hierarchical Bayesian nonparametric models with applications. In Bayesian Nonparametrics. Camb. Ser. Stat. Probab. Math. 28 158–207. Cambridge Univ. Press, Cambridge.
• [68] Teh, Y. W., Jordan, M. I., Beal, M. J. and Blei, D. M. (2006). Hierarchical Dirichlet processes. J. Amer. Statist. Assoc. 101 1566–1581.
• [69] Walker, S. and Muliere, P. (1999). A characterization of a neutral to the right prior via an extension of Johnson’s sufficientness postulate. Ann. Statist. 27 589–599.
• [70] Watterson, G. A. (1976). The stationary distribution of the infinitely-many neutral alleles diffusion model. J. Appl. Probab. 13 639–651.
• [71] Zabell, S. L. (1982). W. E. Johnson’s “sufficientness” postulate. Ann. Statist. 10 1090–1099 (1 plate).
• [72] Zabell, S. L. (1992). Predicting the unpredictable. Synthese 90 205–232.
• [73] Zabell, S. L. (1995). Characterizing Markov exchangeable sequences. J. Theoret. Probab. 8 175–178.
• [74] Zabell, S. L. (1997). The continuum of inductive methods revisited. In The Cosmos of Science: Essays in Exploration (J. Earman and J. D. Norton, eds.) Univ. Pittsburgh Press, Pittsburgh, PA.
• [75] Zabell, S. L. (2005). Symmetry and Its Discontents: Essays on the History of Inductive Probability. Cambridge Univ. Press, New York.

#### Supplemental materials

• Supplement to “Sufficientness Postulates for Gibbs-Type Priors and Hierarchical Generalizations”. Online supplementary material includes the proofs of Proposition 1, 2, 3, 4 and 5, and the derivation of the Pólya-like urn scheme for Gibbs-type species sampling models.