Electronic Journal of Statistics

Conjugacy properties of time-evolving Dirichlet and gamma random measures

Omiros Papaspiliopoulos, Matteo Ruggiero, and Dario Spanò

Full-text: Open access

Abstract

We extend classic characterisations of posterior distributions under Dirichlet process and gamma random measures priors to a dynamic framework. We consider the problem of learning, from indirect observations, two families of time-dependent processes of interest in Bayesian nonparametrics: the first is a dependent Dirichlet process driven by a Fleming–Viot model, and the data are random samples from the process state at discrete times; the second is a collection of dependent gamma random measures driven by a Dawson–Watanabe model, and the data are collected according to a Poisson point process with intensity given by the process state at discrete times. Both driving processes are diffusions taking values in the space of discrete measures whose support varies with time, and are stationary and reversible with respect to Dirichlet and gamma priors respectively. A common methodology is developed to obtain in closed form the time-marginal posteriors given past and present data. These are shown to belong to classes of finite mixtures of Dirichlet processes and gamma random measures for the two models respectively, yielding conjugacy of these classes to the type of data we consider. We provide explicit results on the parameters of the mixture components and on the mixing weights, which are time-varying and drive the mixtures towards the respective priors in absence of further data. Explicit algorithms are provided to recursively compute the parameters of the mixtures. Our results are based on the projective properties of the signals and on certain duality properties of their projections.

Article information

Source
Electron. J. Statist., Volume 10, Number 2 (2016), 3452-3489.

Dates
Received: December 2015
First available in Project Euclid: 16 November 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1479287228

Digital Object Identifier
doi:10.1214/16-EJS1194

Mathematical Reviews number (MathSciNet)
MR3572856

Zentralblatt MATH identifier
1353.62092

Subjects
Primary: 62M05: Markov processes: estimation 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11]
Secondary: 62G05: Estimation 60J60: Diffusion processes [See also 58J65] 60G57: Random measures

Keywords
Bayesian nonparametrics Dawson–Watanabe process Dirichlet process duality Fleming–Viot process gamma random measure

Citation

Papaspiliopoulos, Omiros; Ruggiero, Matteo; Spanò, Dario. Conjugacy properties of time-evolving Dirichlet and gamma random measures. Electron. J. Statist. 10 (2016), no. 2, 3452--3489. doi:10.1214/16-EJS1194. https://projecteuclid.org/euclid.ejs/1479287228


Export citation

References

  • Antoniak, C. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems., Ann. Statist. 2, 1152–1174.
  • Barndorff-Nielsen, O. and Shephard, N. (2001). Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial economics., J. Roy. Statist. Soc. Ser. B 63, 167–241.
  • Beal, M. J., Ghahramani, Z. and Rasmussen, C. E. (2002). The infinite hidden Markov model., Advances in Neural Information Processing Systems 14, 577–585.
  • Blackwell, D. (1973). Discreteness of Ferguson selections., Ann. Statist. 1, 356–358.
  • Blackwell, D. and MacQueen, J. B. (1973). Ferguson distributions via Pólya urn schemes., Ann. Statist. 1, 353–355.
  • Caron, F., Davy. M. and Doucet, A. (2007). Generalized Pólya urn for time-varying Dirichlet process mixtures., Proc. 23rd Conf. on Uncertainty in Artificial Intelligence, Vancouver.
  • Caron, F., Davy, M., Doucet, A., Duflos, E. and Vanheeghe, P. (2008). Bayesian inference for linear dynamic models with Dirichlet process mixtures., IEEE Trans. Sig. Proc. 56, 71–84.
  • Caron, F., Neiswanger, W., Wood, F., Doucet, A. and Davy, M. (2016). Generalized Pólya urn for time-varying Pitman–Yor processes., J. Mach. Learn. Res., in press.
  • Caron, F. and Teh, Y. W. (2012). Bayesian nonparametric models for ranked data., Neural Information Processing Systems (NIPS 2012), Lake Tahoe, USA, 2012.
  • Cox, J. C., Ingersoll, J. E. and Ross, S. A. (1985). A theory of the term structure of interest rates., Econometrica 53, 385–407.
  • Chaleyat-Maurel, M. and Genon-Catalot, V. (2006). Computable infinite-dimensional filters with applications to discretized diffusion processes., Stoch. Proc. Appl. 116, 1447–1467.
  • Chaleyat-Maurel, M. and Genon-Catalot, V. (2009). Filtering the Wright–Fisher diffusion., ESAIM Probab. Stat. 13, 197–217.
  • Daley, D. J. and Vere-Jones (2008)., An introduction to the theory of point processes, Vol. 2. Springer, New York.
  • Dawson, D. A. (1993)., Measure-valued Markov processes. Ecole d’Eté de Probabilités de Saint Flour XXI. Lecture Notes in Mathematics 1541. Springer, Berlin.
  • Dawson, D. A. (2010)., Introductory lectures on stochastic population systems. Technical Report Series 451, Laboratory for Research in Statistics and Probability, Carleton University.
  • Dawson, D. A. and Hochberg, K. J. (1982). Wandering random measures in the Fleming–Viot model., Ann. Probab. 10, 554–580.
  • Dunson, D. B. (2006). Bayesian dynamic modeling of latent trait distributions., Biostatistics 7, 551–568.
  • Etheridge, A. M. (2009)., Some mathematical models from population genetics. École d’été de Probabilités de Saint-Flour XXXIX. Lecture Notes in Math. 2012. Springer.
  • Etheridge, A. M. (2000)., An introduction to superprocesses. University Lecture Series, 20. American Mathematical Society, Providence, RI.
  • Ethier, S. N. and Griffiths, R. C. (1993). The transition function of a Fleming–Viot process., Ann. Probab. 21, 1571–1590.
  • Ethier, S. N. and Griffiths, R. C. (1993b). The transition function of a measure-valued branching diffusion with immigration. In, Stochastic Processes. A Festschrift in Honour of Gopinath Kallianpur (S. Cambanis, J. Ghosh, R. L. Karandikar and P. K. Sen, eds.), 71–79. Springer, New York.
  • Ethier, S. N. and Kurtz, T. G. (1993). Fleming–Viot processes in population genetics., SIAM J. Control Optim. 31, 345–386.
  • Favaro, S., Ruggiero, M. and Walker, S. G. (2009). On a Gibbs sampler based random process in Bayesian nonparametrics., Electron. J. Statist. 3, 1556–1566.
  • Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems., Ann. Statist. 1, 209–230.
  • Gassiat, E. and Rousseau, J. (2016). Nonparametric finite translation hidden Markov models and extensions., Bernoulli 22, 193–212.
  • Ghosal, S. (2010). The Dirichlet process, related priors and posterior asymptotics. In Bayesian Nonparametrics (N. L. Hjort, C. C. Holmes, P. Müller and S. G. Walker, eds.). Cambridge Univ. Press, Cambridge
  • Griffin, J. E. (2011). The Ornstein-Uhlenbeck Dirichlet Process and other time-varying processes for Bayesian nonparametric inference., J. Stat. Plan. Infer. 141, 3648–3664.
  • Griffin, J. E. and Steel, M. F. J. (2006). Order-based dependent Dirichlet processes., JASA 473, 179–194.
  • Griffiths, R. C. and Spanò, D. (2010). Diffusion processes and coalescent trees. In, Probability and Mathematical Genetics: Papers in Honour of Sir John Kingman (Bingham, N. H. and Goldie, C. M., eds.). London Mathematical Society Lecture Notes Series, Cambridge University Press.
  • Gutierrez, L., Mena, R. H. and Ruggiero, M. (2016). A time dependent Bayesian nonparametric model for air quality analysis., Comput. Statist. Data Anal. 95, 161–175.
  • Jansen, S. and Kurt, N. (2014). On the notion(s) of duality for Markov processes., Probab. Surveys. 11, 59–120.
  • Johnson, N. L., Kotz, S. and Balakrishnan, N. (1997)., Discrete multivariate distributions. John Wiley & Sons, New York.
  • Kawazu, K. and Watanabe, S. (1971). Branching processes with immigration and related limit theorems., Theory Probab. Appl. 16, 36–54.
  • Konno, N. and Shiga, T. (1988). Stochastic differential equations for some measure valued diffusions., Probab. Th. Rel. Fields 79, 201–225.
  • Li, Z. (2011)., Measure-valued branching Markov processes. Springer, Heidelberg.
  • Lo, A. Y. (1982). Bayesian nonparametric statistical inference for Poisson point process., Z. Wahrsch. Verw. Gebiete 59, 55–66.
  • MacEachern, S. N. (1999). Dependent Nonparametric Processes. In, ASA Proceedings of the Section on Bayesian Statistical Science. American Statist. Assoc., Alexandria, VA.
  • MacEachern, S. N. (2000). Dependent Dirichlet processes., Tech. Rep., Ohio State University.
  • Mena, R. H. and Ruggiero, M. (2016). Dynamic density estimation with diffusive Dirichlet mixtures., Bernoulli 22, 901–926.
  • Mena, R. H., Ruggiero, M. and Walker, S. G. (2011). Geometric stick-breaking processes for continuous-time Bayesian nonparametric modeling., J. Statist. Plann. Inf. 141, 3217–3230.
  • Papaspiliopoulos, O. and Roberts, G. O. (2008). Retrospective mcmc for dirichlet process hierarchical models., Biometrika 95, 169–186.
  • Papaspiliopoulos, O. and Ruggiero, M. (2014). Optimal filtering and the dual process., Bernoulli 20, 1999–2019.
  • Rodriguez, A. and ter Horst, E. (2008). Bayesian dynamic density estimation., Bayes. Anal. 3, 339–366.
  • Ruggiero, M. and Walker, S. G. (2009a). Bayesian nonparametric construction of the Fleming–Viot process with fertility selection., Statist. Sinica 19, 707–720.
  • Ruggiero, M. and Walker, S. G. (2009b). Countable representation for infinite-dimensional diffusions derived from the two-parameter Poisson–Dirichlet process., Elect. Comm. Probab. 14, 501–517.
  • Stepleton, T., Ghahramani, Z., Gordon, G., and Lee, T.-S. (2009). The block diagonal infinite hidden Markov model., Journal of Machine Learning Research 5, 544–551.
  • Sethuraman, J. (1994). A constructive definition of the Dirichlet process prior., Statist. Sinica 2, 639–650.
  • Shiga, T. (1990). A stochastic equation based on a Poisson system for a class of measure-valued diffusion processes., J. Math. Kyoto Univ. 30, 245–279.
  • Spanò, D. and Lijoi, A. (2016). Canonical correlations for dependent gamma processes., arXiv:1601.06079.
  • Tavaré, S. (1984). Line-of-descent and genealogical processes, and their applications in population genetic models., Theoret. Population Biol. 26, 119–164.
  • Van Gael, V., Saatci, Y., Teh, Y. W. and Ghahramani, Z. (2008). Beam sampling for the infinite hidden Markov model. In, Proceedings of the 25th International Conference on Machine Learning.
  • Walker, S. G. (2007). Sampling the dirichlet mixture model with slices., Comm. Statist. Sim. Comput. 36, 45–54.
  • Walker, S. G., Hatjispyros S. J. and Nicoleris, T. (2007). A Fleming–Viot process and Bayesian nonparametrics., Ann. Appl. Probab. 17, 67–80.
  • Yau, C., Papaspiliopoulos, O., Roberts, G. O. and Holmes, C. (2011). Bayesian non-parametric hidden Markov models with applications in genomics., J. Roy. Statist. Soc. Ser. B 73, 37–57.
  • Zhang, A., Zhu, J. and Zhang, B. (2014). Max-margin infinite hidden Markov models. In, Proceedings of the 31st International Conference on Machine Learning.