Bernoulli

  • Bernoulli
  • Volume 20, Number 3 (2014), 1260-1291.

Bayesian inference with dependent normalized completely random measures

Antonio Lijoi, Bernardo Nipoti, and Igor Prünster

Full-text: Open access

Abstract

The proposal and study of dependent prior processes has been a major research focus in the recent Bayesian nonparametric literature. In this paper, we introduce a flexible class of dependent nonparametric priors, investigate their properties and derive a suitable sampling scheme which allows their concrete implementation. The proposed class is obtained by normalizing dependent completely random measures, where the dependence arises by virtue of a suitable construction of the Poisson random measures underlying the completely random measures. We first provide general distributional results for the whole class of dependent completely random measures and then we specialize them to two specific priors, which represent the natural candidates for concrete implementation due to their analytic tractability: the bivariate Dirichlet and normalized $\sigma$-stable processes. Our analytical results, and in particular the partially exchangeable partition probability function, form also the basis for the determination of a Markov Chain Monte Carlo algorithm for drawing posterior inferences, which reduces to the well-known Blackwell–MacQueen Pólya urn scheme in the univariate case. Such an algorithm can be used for density estimation and for analyzing the clustering structure of the data and is illustrated through a real two-sample dataset example.

Article information

Source
Bernoulli, Volume 20, Number 3 (2014), 1260-1291.

Dates
First available in Project Euclid: 11 June 2014

Permanent link to this document
https://projecteuclid.org/euclid.bj/1402488940

Digital Object Identifier
doi:10.3150/13-BEJ521

Mathematical Reviews number (MathSciNet)
MR3217444

Zentralblatt MATH identifier
1309.60048

Keywords
completely random measure dependent Poisson processes Dirichlet process generalized Polýa urn scheme infinitely divisible vector normalized $\sigma$-stable process partially exchangeable random partition

Citation

Lijoi, Antonio; Nipoti, Bernardo; Prünster, Igor. Bayesian inference with dependent normalized completely random measures. Bernoulli 20 (2014), no. 3, 1260--1291. doi:10.3150/13-BEJ521. https://projecteuclid.org/euclid.bj/1402488940


Export citation

References

  • [1] Antoniak, C.E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist. 2 1152–1174.
  • [2] Bailey, W.N. (1964). Generalized Hypergeometric Series. Cambridge Tracts in Mathematics and Mathematical Physics, No. 32. New York: Stechert-Hafner, Inc.
  • [3] Barrientos, A.F., Jara, A. and Quintana, F.A. (2012). On the support of MacEachern’s dependent Dirichlet processes and extensions. Bayesian Anal. 7 277–309.
  • [4] Cifarelli, D.M. and Regazzini, E. (1978). Problemi statistici non parametrici in condizioni di scambiabilità parziale. Quaderni Istituto Matematica Finanziaria, Università di Torino Serie III, 12. English translation. Available at: http://www.unibocconi.it/wps/allegatiCTP/CR-Scamb-parz[1].20080528.135739.pdf.
  • [5] Constantine, G.M. and Savits, T.H. (1996). A multivariate Faà di Bruno formula with applications. Trans. Amer. Math. Soc. 348 503–520.
  • [6] Daley, D.J. and Vere-Jones, D. (1988). An Introduction to the Theory of Point Processes. Springer Series in Statistics. New York: Springer.
  • [7] de Finetti, B. (1938). Sur la condition d’equivalence partielle. In Actualités Scientifiques et Industrielles, 739 5–18. Paris: Herman.
  • [8] Dunson, D.B. (2010). Nonparametric Bayes applications to biostatistics. In Bayesian Nonparametrics (N.L. Hjort, C.C. Holmes, P. Müller and S.G. Walker, eds.). Camb. Ser. Stat. Probab. Math. 223–273. Cambridge: Cambridge Univ. Press.
  • [9] Epifani, I. and Lijoi, A. (2010). Nonparametric priors for vectors of survival functions. Statist. Sinica 20 1455–1484.
  • [10] Escobar, M.D. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577–588.
  • [11] Ewens, W.J. (1972). The sampling theory of selectively neutral alleles. Theoret. Population Biology 3 87–112.
  • [12] Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209–230.
  • [13] Gelfand, A.E. and Kottas, A. (2002). A computational approach for full nonparametric Bayesian inference under Dirichlet process mixture models. J. Comput. Graph. Statist. 11 289–305.
  • [14] Gradshteyn, I.S. and Ryzhik, I.M. (2007). Table of Integrals, Series, and Products, 7th ed. Amsterdam: Elsevier/Academic Press.
  • [15] Griffiths, R.C. and Milne, R.K. (1978). A class of bivariate Poisson processes. J. Multivariate Anal. 8 380–395.
  • [16] Ishwaran, H. and James, L.F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96 161–173.
  • [17] James, L.F. (2005). Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages. Ann. Statist. 33 1771–1799.
  • [18] James, L.F., Lijoi, A. and Prünster, I. (2006). Conjugacy as a distinctive feature of the Dirichlet process. Scand. J. Stat. 33 105–120.
  • [19] James, L.F., Lijoi, A. and Prünster, I. (2009). Posterior analysis for normalized random measures with independent increments. Scand. J. Stat. 36 76–97.
  • [20] Kingman, J.F.C. (1993). Poisson Processes. Oxford Studies in Probability 3. New York: The Clarendon Press Oxford Univ. Press.
  • [21] Kolossiatis, M., Griffin, J.E. and Steel, M.F.J. (2013). On Byesian nonparametric modelling of two correlated distributions. Statistics and Computing 23 1–15.
  • [22] Leisen, F. and Lijoi, A. (2011). Vectors of two-parameter Poisson–Dirichlet processes. J. Multivariate Anal. 102 482–495.
  • [23] Lijoi, A., Mena, R.H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian non-parametric mixture models. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 715–740.
  • [24] Lijoi, A. and Prünster, I. (2010). Models beyond the Dirichlet process. In Bayesian Nonparametrics (N.L. Hjort, C.C. Holmes, P. Müller and S.G. Walker, eds.). Camb. Ser. Stat. Probab. Math. 80–136. Cambridge: Cambridge Univ. Press.
  • [25] MacEachern, S.N. (1994). Estimating normal means with a conjugate style Dirichlet process prior. Comm. Statist. Simulation Comput. 23 727–741.
  • [26] MacEachern, S.N. (1999). Dependent nonparametric processes. In ASA Proceedings of the Section on Bayesian Statistical Science Alexandria, VA: American Statistical Association.
  • [27] MacEachern, S.N. (2000). Dependent Dirichlet processes. Technical report, Ohio State Univ.
  • [28] Müller, P., Quintana, F. and Rosner, G. (2004). A method for combining inference across related nonparametric Bayesian models. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 735–749.
  • [29] Müller, P. and Quintana, F.A. (2004). Nonparametric Bayesian data analysis. Statist. Sci. 19 95–110.
  • [30] Nipoti, B. (2011). Dependent completely random measures and statistical applications. Ph.D. thesis, Dept. Mathematics, Univ. Pavia.
  • [31] Olkin, I. and Liu, R. (2003). A bivariate beta distribution. Statist. Probab. Lett. 62 407–412.
  • [32] Orbanz, P. (2011). Projective limit random probabilities on Polish spaces. Electron. J. Stat. 5 1354–1373.
  • [33] Pitman, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Math. 1875. Berlin: Springer.
  • [34] Prünster, I. (2002). Random probability measures derived from increasing additive processes and their application to Bayesian statistics. Ph.D thesis, Univ. Pavia.
  • [35] Rao, V.A. and Teh, Y.W. (2009). Spatial normalized Gamma processes. In Advances in Neural Information Processing Systems 22. NIPS Foundation. Available at http://books.nips.cc/papers/files/nips22/NIPS2009_0744.pdf.
  • [36] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of normalized random measures with independent increments. Ann. Statist. 31 560–585.
  • [37] Teh, Y.W. and Jordan, M.I. (2010). Hierarchical Bayesian nonparametric models with applications. In Bayesian Nonparametrics (N.L. Hjort, C.C. Holmes, P. Müller and S.G. Walker, eds.). Camb. Ser. Stat. Probab. Math. 158–207. Cambridge: Cambridge Univ. Press.
  • [38] West, M., Müller, P. and Escobar, M.D. (1994). Hierarchical priors and mixture models, with application in regression and density estimation. In Aspects of Uncertainty. Wiley Ser. Probab. Math. Statist. Probab. Math. Statist. 363–386. Chichester: Wiley.