Bayesian Analysis

Joining and Splitting Models with Markov Melding

Robert J. B. Goudie, Anne M. Presanis, David Lunn, Daniela De Angelis, and Lorenz Wernisch

Full-text: Open access


Analysing multiple evidence sources is often feasible only via a modular approach, with separate submodels specified for smaller components of the available evidence. Here we introduce a generic framework that enables fully Bayesian analysis in this setting. We propose a generic method for forming a suitable joint model when joining submodels, and a convenient computational algorithm for fitting this joint model in stages, rather than as a single, monolithic model. The approach also enables splitting of large joint models into smaller submodels, allowing inference for the original joint model to be conducted via our multi-stage algorithm. We motivate and demonstrate our approach through two examples: joining components of an evidence synthesis of A/H1N1 influenza, and splitting a large ecology model.

Article information

Bayesian Anal., Volume 14, Number 1 (2019), 81-109.

First available in Project Euclid: 14 April 2018

Permanent link to this document

Digital Object Identifier

model integration Markov combination Bayesian melding evidence synthesis

Creative Commons Attribution 4.0 International License.


Goudie, Robert J. B.; Presanis, Anne M.; Lunn, David; De Angelis, Daniela; Wernisch, Lorenz. Joining and Splitting Models with Markov Melding. Bayesian Anal. 14 (2019), no. 1, 81--109. doi:10.1214/18-BA1104.

Export citation


  • Ades, A. E. and Sutton, A. J. (2006). “Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 169(1): 5–35.
  • Albert, I., Espié, E., de Valk, H., and Denis, J.-B. (2011). “A Bayesian evidence synthesis for estimating Campylobacteriosis prevalence.” Risk Analysis, 31(7): 1141–1155.
  • Bardenet, R., Doucet, A., and Holmes, C. (2017). “On Markov chain Monte Carlo methods for tall data.” Journal of Machine Learning Research, 18(47): 1–43.
  • Besbeas, P., Freeman, S. N., Morgan, B. J. T., and Catchpole, E. A. (2002). “Integrating mark–recapture–recovery and census data to estimate animal abundance and demographic parameters.” Biometrics, 58(3): 540–547.
  • Birrell, P. J., De Angelis, D., Wernisch, L., Tom, B. D., Roberts, G. O., and Pebody, R. G. (2016). “Efficient real-time monitoring of an emerging influenza epidemic: how feasible?” arXiv:1608.05292.
  • Brooks, S. P., King, R., and Morgan, B. J. T. (2004). “A Bayesian approach to combining animal abundance and demographic data.” Animal Biodiversity and Conservation, 27(1): 515–529.
  • Clemen, R. T. and Winkler, R. L. (1999). “Combining probability distributions from experts in risk analysis.” Risk Analysis, 19(2): 187–203.
  • Commenges, D. and Hejblum, B. P. (2012). “Evidence synthesis through a degradation model applied to myocardial infarction.” Lifetime Data Analysis, 19(1): 1–18.
  • Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper Markov laws in the statistical analysis of decomposable graphical models.” Annals of Statistics, 21(3): 1272–1317.
  • Department of Health (2011). “Department of Health Winter Watch.”
  • Doucet, A., de Freitas, N., and Gordon, N. (eds.) (2013). Sequential Monte Carlo Methods in Practice. New York: Springer Science & Business Media.
  • Draper, D. (1995). “Assessment and propagation of model uncertainty.” Journal of the Royal Statistical Society: Series B (Methodological), 57(1): 45–97.
  • Durante, F. and Sempi, C. (2010). “Copula theory: an introduction.” In Jaworski, P., Durante, F., Härdle, W. K., and Rychlik, T. (eds.), Copula Theory and its Applications, 3–31. Berlin: Springer-Verlag.
  • Eddy, D. M., Hasselblad, V., and Shachter, R. (1992). Meta-Analysis by the Confidence Profile Method. London: Academic Press.
  • Gåsemyr, J. and Natvig, B. (2009). “Extensions of a conflict measure of inconsistencies in Bayesian hierarchical models.” Scandinavian Journal of Statistics, 36(4): 822–838.
  • Goudie, R. J. B., Hovorka, R., Murphy, H. R., and Lunn, D. (2015). “Rapid model exploration for complex hierarchical data: application to pharmacokinetics of insulin aspart.” Statistics in Medicine, 34(23): 3144–3158.
  • Goudie, R. J. B., Presanis, A. M., Lunn, D., De Angelis, D., and Wernisch, L. (2019). “Supplementary Material for: “Joining and splitting models with Markov melding”.” Bayesian Analysis.
  • Green, P. J., Hjort, N. L., and Richardson, S. (2003). “Introducing highly structured stochastic systems.” In Green, P. J., Hjort, N. L., and Richardson, S. (eds.), Highly Structured Stochastic Systems, 1–12. Oxford: Oxford University Press.
  • Hasselblad, V., Eddy, D. M., and Kotchmar, D. J. (1992). “Synthesis of environmental evidence: nitrogen dioxide epidemiology studies.” Journal of the Air & Waste Management Association, 42(5): 662–671.
  • Henderson, D. J. and Parmeter, C. F. (2015). Applied Nonparametric Econometrics. New York: Cambridge University Press.
  • Hinton, G. E. (2002). “Training products of experts by minimizing contrastive divergence.” Neural Computation, 14(8): 1771–1800.
  • Huang, Z. and Gelman, A. (2005). “Sampling for Bayesian computation with large datasets.” Working paper.
  • Jackson, C. H., Best, N. G., and Richardson, S. (2009). “Bayesian graphical models for regression on multiple data sets with different variables.” Biostatistics, 10(2): 335–351.
  • Jackson, C. H., Jit, M., Sharples, L. D., and De Angelis, D. (2015). “Calibration of complex models through Bayesian evidence synthesis.” Medical Decision Making, 35(2): 148–161.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford: Clarendon Press.
  • Leonelli, M. (2015). “Bayesian decision support in complex modular systems: an algebraic and graphical approach.” Ph.D. thesis, University of Warwick, UK.
  • Liang, L.-J. and Weiss, R. E. (2007). “A hierarchical semiparametric regression model for combining HIV-1 phylogenetic analyses using iterative reweighting algorithms.” Biometrics, 63(3): 733–741.
  • Lindley, D. V., Tversky, A., and Brown, R. V. (1979). “On the reconciliation of probability assessments.” Journal of the Royal Statistical Society: Series A (General), 142(2): 146–180.
  • Liu, F., Bayarri, M. J., and Berger, J. O. (2009). “Modularization in Bayesian analysis, with emphasis on analysis of computer models.” Bayesian Analysis, 4(1): 119–150.
  • Lunn, D., Barrett, J., Sweeting, M., and Thompson, S. (2013a). “Fully Bayesian hierarchical modelling in two stages, with application to meta-analysis.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 62(4): 551–572.
  • Lunn, D., Jackson, C., Best, N., Thomas, A., and Spiegelhalter, D. (2013b). The BUGS Book: A Practical Introduction to Bayesian Analysis. Boca Raton: CRC Press.
  • Massa, M. S. and Lauritzen, S. L. (2010). “Combining statistical models.” In Viana, M. A. G. and Wynn, H. P. (eds.), Contemporary Mathematics: Algebraic Methods in Statistics and Probability II, 239–260.
  • Massa, M. S. and Riccomagno, E. (2017). “Algebraic representations of Gaussian Markov combinations.” Bernoulli, 23(1): 626–644.
  • Minsker, S., Srivastava, S., Lin, L., and Dunson, D. B. (2017). “Robust and scalable Bayes via a median of subset posterior measures.” Journal of Machine Learning Research, 18(124): 1–40.
  • Moran, E. V. and Clark, J. S. (2011). “Estimating seed and pollen movement in a monoecious plant: a hierarchical Bayesian approach integrating genetic and ecological data.” Molecular Ecology, 20(6): 1248–1262.
  • Müller, P. (1991). “A generic approach to posterior integration and Gibbs sampling.” Technical Report 91-09, Purdue University.
  • Neiswanger, W., Wang, C., and Xing, E. P. (2014). “Asymptotically exact, embarrassingly parallel MCMC.” In Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14), 623–632. Corvallis, Oregon: AUAI Press.
  • Neuenschwander, B., Branson, M., and Spiegelhalter, D. J. (2009). “A note on the power prior.” Statistics in Medicine, 28(28): 3562–3566.
  • O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., and Rakow, T. (2006). Uncertain Judgements: Eliciting Experts’ Probabilities. Chichester: John Wiley & Sons.
  • Plummer, M. (2015a). “Cuts in Bayesian graphical models.” Statistics and Computing, 25(1): 37–43.
  • Plummer, M. (2015b). “JAGS Version 4.0.1 user manual.”
  • Poole, D. and Raftery, A. E. (2000). “Inference for deterministic simulation models: The Bayesian melding approach.” Journal of the American Statistical Association, 95(452): 1244–1255.
  • Presanis, A. M., Ohlssen, D., Cui, K., Rosinska, M., and De Angelis, D. (2016). “Conflict diagnostics for evidence synthesis in a multiple testing framework.” arXiv:1702.07304.
  • Presanis, A. M., Ohlssen, D., Spiegelhalter, D. J., and De Angelis, D. (2013). “Conflict diagnostics in directed acyclic graphs, with applications in Bayesian evidence synthesis.” Statistical Science, 28(3): 376–397.
  • Presanis, A. M., Pebody, R. G., Birrell, P. J., Tom, B. D. M., Green, H. K., Durnall, H., Fleming, D., and De Angelis, D. (2014). “Synthesising evidence to estimate pandemic (2009) A/H1N1 influenza severity in 2009–2011.” Annals of Applied Statistics, 8(4): 2378–2403.
  • Public Health England (2014). “Sources of UK flu data: influenza surveillance in the UK.”
  • Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods. New York: Springer.
  • Scott, S. L., Blocker, A. W., Bonassi, F. V., Chipman, H. A., George, E. I., and McCulloch, R. E. (2016). “Bayes and big data: The consensus Monte Carlo algorithm.” International Journal of Management Science and Engineering Management, 11(2): 78–88.
  • Shubin, M., Lebedev, A., Lyytikäinen, O., and Auranen, K. (2016). “Revealing the true incidence of pandemic A(H1N1)pdm09 influenza in Finland during the first two seasons—An analysis based on a dynamic transmission model.” PLOS Computational Biology, 12(3): e1004803.
  • Sugiyama, M., Suzuki, T., and Kanamori, T. (2012). Density Ratio Estimation in Machine Learning. New York: Cambridge University Press.
  • Tom, J. A., Sinsheimer, J. S., and Suchard, M. A. (2010). “Reuse, recycle, reweigh: combating influenza through efficient sequential Bayesian computation for massive data.” Annals of Applied Statistics, 4(4): 1722–1748.
  • Turner, R. M., Spiegelhalter, D. J., Smith, G. C. S., and Thompson, S. G. (2009). “Bias modelling in evidence synthesis.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1): 21–47.
  • Wang, X. and Dunson, D. B. (2013). “Parallel MCMC via Weierstrass Sampler.” arXiv:1312.4605.
  • Welton, N. J., Cooper, N. J., Ades, A. E., Lu, G., and Sutton, A. J. (2008). “Mixed treatment comparison with multiple outcomes reported inconsistently across trials: Evaluation of antivirals for treatment of influenza A and B.” Statistics in Medicine, 27(27): 5620–5639.
  • Welton, N. J., Sutton, A. J., Cooper, N. J., Abrams, K. R., and Ades., A. (2012). Evidence Synthesis for Decision Making in Healthcare. Chichester: John Wiley & Sons.

Supplemental materials