Bayesian Analysis

Modelling and Computation Using NCoRM Mixtures for Density Regression

Jim Griffin and Fabrizio Leisen

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

Normalized compound random measures are flexible nonparametric priors for related distributions. We consider building general nonparametric regression models using normalized compound random measure mixture models. Posterior inference is made using a novel pseudo-marginal Metropolis-Hastings sampler for normalized compound random measure mixture models. The algorithm makes use of a new general approach to the unbiased estimation of Laplace functionals of compound random measures (which includes completely random measures as a special case). The approach is illustrated on problems of density regression.

Article information

Source
Bayesian Anal. (2017), 20 pages.

Dates
First available in Project Euclid: 26 October 2017

Permanent link to this document
https://projecteuclid.org/euclid.ba/1508983454

Digital Object Identifier
doi:10.1214/17-BA1072

Keywords
dependent random measures mixture models multivariate Lévy measures pseudo-marginal samplers Poisson estimator

Rights
Creative Commons Attribution 4.0 International License.

Citation

Griffin, Jim; Leisen, Fabrizio. Modelling and Computation Using NCoRM Mixtures for Density Regression. Bayesian Anal., advance publication, 26 October 2017. doi:10.1214/17-BA1072. https://projecteuclid.org/euclid.ba/1508983454


Export citation

References

  • Andrieu and Vihola (2016). “Establishing some order amongst exact approximations of MCMCs.”Annals of Applied Probability, 26: 2661–2696.
  • Andrieu, C. and Roberts, G. O. (2009). “The pseudo-marginal approach for efficient Monte Carlo computations.”Annals of Statistics, 37: 697–725.
  • Arbel, J. and Prünster, I. (2016). “A moment-matching Ferguson & Klass algorithm.”Statistics and Computing, 27: 3–17.
  • Atchadé, Y. F. and Rosenthal, J. S. (2005). “On Adaptive Markov Chain Monte Carlo Algorithms.”Bernoulli, 11: 815–828.
  • Brix, A. (1999). “Generalised gamma measures and shot-noise Cox processes.”Advances in Applied Probability, 31: 929–953.
  • Chen, C., Rao, V. A., Buntine, W., and Teh, Y. W. (2013). “Dependent Normalized Random Measures.” InProceedings of the International Conference on Machine Learning.
  • De Iorio, M., Müller, P., Rosner, G. L., and MacEachern, S. N. (2004). “An ANOVA model for dependent random measures.”Journal of the American Statistical Association, 99: 205–215.
  • Doucet, A., Pitt, M., Deligiannidis, G., and Kohn, R. (2015). “Efficient Implementation of Markov chain Monte Carlo when Using an Unbiased Likelihood Estimator.”Biometrika, 102: 295–313.
  • Dunson, D. B. (2010). “Nonparametric Bayes applications to biostatistics.” InBayesian Nonparametrics, 223–273. Cambridge University Press.
  • Favaro, S. and Teh, Y. W. (2013). “MCMC for Normalized Random Measure Mixture Models.”Statistical Science, 28: 335–359.
  • Fearnhead, P., Papaspiliopoulos, O., Roberts, G. O., and Stuart, A. (2010). “Random-weight particle filtering of continuous time processes.”Journal of the Royal Statistical Society, Series B, 74: 497–512.
  • Foti, N. and Williamson, S. (2012). “Slice sampling normalized kernel-weighted completely random measure mixture models.” InAdvances in Neural Information Processing Systems 25, 2240–2248.
  • Griffin, J. E. (2011). “The Ornstein-Uhlenback Dirichlet process and other time-varying processes for Bayesian nonparametric inference.”Journal of Statistical Planning and Inference, 141: 3648–3664.
  • Griffin, J. E., Kolossiatis, M., and Steel, M. F. J. (2013). “Comparing Distributions By Using Dependent Normalized Random-Measure Mixtures.”Journal of the Royal Statistical Society, Series B, 75: 499–529.
  • Griffin, J. and Leisen, F. (2017). “Appendix of “Modelling and computation using NCoRM mixtures for density regression”.”Bayesian Analysis.
  • Griffin, J. E. and Leisen, F. (2017). “Compound Random Measures and their use in Bayesian nonparametrics.”Journal of the Royal Statistical Society, Series B, 79: 525–545.
  • Griffin, J. E. and Walker, S. G. (2011). “Posterior simulation of normalized random measure mixtures.”Journal of Computational and Graphical Statistics, 20: 241–259.
  • James, L. F., Lijoi, A., and Prünster, I. (2009). “Posterior Analysis for Normalized Random Measures with Independent Increments.”Scandinavian Journal of Statistics, 36: 76–97.
  • Leisen, F. and Lijoi, A. (2011). “Vectors of Poisson-Dirichlet processes.”Journal of Multivariate Analysis, 102: 482–495.
  • Leisen, F., Lijoi, A., and Spano, D. (2013). “A Vector of Dirichlet processes.”Electronic Journal of Statistics, 7: 62–90.
  • Lijoi, A. and Nipoti, B. (2014). “A class of hazard rate mixtures for combining survival data from different experiments.”Journal of the American Statistical Association, 109: 802–814.
  • Lijoi, A., Nipoti, B., and Prünster, I. (2014a). “Bayesian inference with dependent normalized completely random measures.”Bernoulli, 20: 1260–1291.
  • Lijoi, A., Nipoti, B., and Prünster, I. (2014b). “Dependent mixture models: clustering and borrowing information.”Computational Statistics and Data Analysis, 71: 417–433.
  • Lijoi, A. and Prünster, I. (2010). “Models beyond the Dirichlet Process.” InBayesian Nonparametrics, 80–136. Cambridge University Press.
  • Lyne, A.-M., Girolami, M., Strathmann, H., Simpson, D., and Atchade, Y. (2015). “On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods.”Statistical Science, 30: 443–467.
  • MacEachern, S. N. (1999). “Dependent nonparametric processes.” InASA Proceedings of the Section on Bayesian Statistical Science.
  • Müller, P. and Rosner, G. (1997). “A Bayesian population model with hierarchical mixture priors applied to blood count data.”Journal of the American Statistical Association, 92: 1279–1292.
  • Neal, R. M. (2000). “Markov chain sampling methods for Dirichlet process mixture models.”Journal of Computational and Graphical Statistics, 9: 249–265.
  • Papaspiliopoulos, O. (2009). “A methodological framework for Monte Carlo probabilistic inference for diffusion processes.” InBayesian Time Series Models, 91–112. Cambridge University Press.
  • Ranganath, R. and Blei, D. M. (2017). “Correlated Random Measures.”Journal of the American Statistical Association, to appear.
  • Regazzini, E., Lijoi, A., and Prünster, I. (2003). “Distributional results for means of normalized random measures with independent increments.”Annals of Statistics, 31: 560–585.
  • Rhee, C.-H. and Glynn, P. W. (2015). “Unbiased estimation with square root convergence for SDE models.”Operations Research, 63: 1026–1043.
  • Rodriguez, A. and Dunson, D. B. (2011). “Nonparametric Bayesian models through probit stick-breaking processes.”Bayesian Analysis, 6: 145–178.
  • Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.”Statistica Sinica, 4: 639–650.
  • Silverman, B. W. (1985). “Some aspects of the spline smoothing approach to non-parametric curve fitting.”Journal of the Royal Statistical Society, Series B, 47: 1–52.
  • Todeschini, A. and Caron, F. (2016). “Exchangeable random measures for sparse and modular graphs with overlapping communities.” ArXiv: 1602.0211.
  • Yu, Y. and Meng, X.-L. (2011). “To Center or Not to Center: That is Not the Question – An Ancillarity-Sufficiency Interweaving Strategy (ASIS) for Boosting MCMC Efficiency.”Journal of Computational and Graphical Statistics, 20: 531–570.
  • Zhu, W. and Leisen, F. (2015). “A multivariate extension of a vector of Poisson-Dirichlet processes.”Journal of Nonparametric Statistics, 27: 89–105.

Supplemental materials