Looking-backward probabilities for Gibbs-type exchangeable random partitions

Sergio Bacallado, Stefano Favaro, and Lorenzo Trippa

Full-text: Open access


Gibbs-type random probability measures and the exchangeable random partitions they induce represent the subject of a rich and active literature. They provide a probabilistic framework for a wide range of theoretical and applied problems that are typically referred to as species sampling problems. In this paper, we consider the class of looking-backward species sampling problems introduced in Lijoi et al. (Ann. Appl. Probab. 18 (2008) 1519–1547) in Bayesian nonparametrics. Specifically, given some information on the random partition induced by an initial sample from a Gibbs-type random probability measure, we study the conditional distributions of statistics related to the old species, namely those species detected in the initial sample and possibly re-observed in an additional sample. The proposed results contribute to the analysis of conditional properties of Gibbs-type exchangeable random partitions, so far focused mainly on statistics related to those species generated by the additional sample and not already detected in the initial sample.

Article information

Bernoulli, Volume 21, Number 1 (2015), 1-37.

First available in Project Euclid: 17 March 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Bayesian nonparametrics conditional random partitions Ewens–Pitman sampling model Gibbs-type exchangeable random partitions looking-backward probabilities species diversity species sampling problems


Bacallado, Sergio; Favaro, Stefano; Trippa, Lorenzo. Looking-backward probabilities for Gibbs-type exchangeable random partitions. Bernoulli 21 (2015), no. 1, 1--37. doi:10.3150/13-BEJ559.

Export citation


  • [1] Arratia, R., Barbour, A.D. and Tavaré, S. (2003). Logarithmic Combinatorial Structures: A Probabilistic Approach. EMS Monographs in Mathematics. Zürich: European Mathematical Society (EMS).
  • [2] Charalambides, C.A. (2005). Combinatorial Methods in Discrete Distributions. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley.
  • [3] De Blasi, P., Favaro, S., Lijoi, A., Mena, R.H., Prünster, I. and Ruggiero, M. (2014). Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. To appear.
  • [4] De Blasi, P., Lijoi, A. and Prünster, I. (2013). An asymptotic analysis of a class of discrete nonparametric priors. Statist. Sinica 23 1299–1321.
  • [5] Ewens, W.J. (1972). The sampling theory of selectively neutral alleles. Theoret. Population Biology 3 87–112; erratum, ibid. 3 (1972), 240, 376.
  • [6] Ewens, W.J. and Tavaré, S. (1998). The Ewens sampling formula. In Encyclopedia of Statistical Sciences. A Wiley-Interscience Publication 2 update (S. Kotz, C.B. Read and L.D. Banks, eds.) 230–234. New York: Wiley.
  • [7] Favaro, S., Lijoi, A. and Prünster, I. (2012). A new estimator of the discovery probability. Biometrics 68 1188–1196.
  • [8] Favaro, S., Lijoi, A. and Prünster, I. (2013). Conditional formulae for Gibbs-type exchangeable random partitions. Ann. Appl. Probab. 23 1721–2160.
  • [9] Gnedin, A. (2010). A species sampling model with finitely many types. Electron. Commun. Probab. 15 79–88.
  • [10] Gnedin, A. and Pitman, J. (2005). Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325 83–102, 244–245.
  • [11] Griffiths, R.C. and Spanò, D. (2007). Record indices and age-ordered frequencies in exchangeable Gibbs partitions. Electron. J. Probab. 12 1101–1130.
  • [12] Kingman, J.F.C. (1978). The representation of partition structures. J. London Math. Soc. (2) 18 374–380.
  • [13] Lijoi, A., Mena, R.H. and Prünster, I. (2007). Bayesian nonparametric estimation of the probability of discovering new species. Biometrika 94 769–786.
  • [14] Lijoi, A., Mena, R.H. and Prünster, I. (2007). A Bayesian nonparametric method for prediction in EST analysis. BMC Bioinformatics 8 339.
  • [15] Lijoi, A. and Prünster, I. (2010). Models beyond the Dirichlet process. In Bayesian Nonparametrics (N.L. Hjort, C.C. Holmes, P. Müller and S.G. Walker, eds.). Camb. Ser. Stat. Probab. Math. 80–136. Cambridge: Cambridge Univ. Press.
  • [16] Lijoi, A., Prünster, I. and Walker, S.G. (2008). Bayesian nonparametric estimators derived from conditional Gibbs structures. Ann. Appl. Probab. 18 1519–1547.
  • [17] Pitman, J. (1995). Exchangeable and partially exchangeable random partitions. Probab. Theory Related Fields 102 145–158.
  • [18] Pitman, J. (2003). Poisson–Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed (D.R. Goldstein, ed.). Institute of Mathematical Statistics Lecture Notes – Monograph Series 40 1–34. Beachwood, OH: IMS.
  • [19] Pitman, J. (2006). Combinatorial Stochastic Processes. Lectures from the 32nd Summer School on Probability Theory held in Saint-Flour, July 724, 2002. Lecture Notes in Math. 1875. Berlin: Springer.
  • [20] Pitman, J. and Yor, M. (1997). The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25 855–900.
  • [21] Teh, Y.W. and Jordan, M.I. (2010). Hierarchical Bayesian nonparametric models with applications. In Bayesian Nonparametrics (N.L. Hjort, C.C. Holmes, P. Müller and S.G. Walker, eds.). Camb. Ser. Stat. Probab. Math. 158–207. Cambridge: Cambridge Univ. Press.
  • [22] Teh, Y.W., Jordan, M.I., Beal, M.J. and Blei, D.M. (2006). Hierarchical Dirichlet processes. J. Amer. Statist. Assoc. 101 1566–1581.