Bayesian Analysis

Joining and Splitting Models with Markov Melding

Robert J. B. Goudie, Anne M. Presanis, David Lunn, Daniela De Angelis, and Lorenz Wernisch

Full-text: Open access


Analysing multiple evidence sources is often feasible only via a modular approach, with separate submodels specified for smaller components of the available evidence. Here we introduce a generic framework that enables fully Bayesian analysis in this setting. We propose a generic method for forming a suitable joint model when joining submodels, and a convenient computational algorithm for fitting this joint model in stages, rather than as a single, monolithic model. The approach also enables splitting of large joint models into smaller submodels, allowing inference for the original joint model to be conducted via our multi-stage algorithm. We motivate and demonstrate our approach through two examples: joining components of an evidence synthesis of A/H1N1 influenza, and splitting a large ecology model.

Article information

Bayesian Anal., Volume 14, Number 1 (2019), 81-109.

First available in Project Euclid: 14 April 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

model integration Markov combination Bayesian melding evidence synthesis

Creative Commons Attribution 4.0 International License.


Goudie, Robert J. B.; Presanis, Anne M.; Lunn, David; De Angelis, Daniela; Wernisch, Lorenz. Joining and Splitting Models with Markov Melding. Bayesian Anal. 14 (2019), no. 1, 81--109. doi:10.1214/18-BA1104.

Export citation


  • Ades, A. E. and Sutton, A. J. (2006). “Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 169(1): 5–35.
  • Albert, I., Espié, E., de Valk, H., and Denis, J.-B. (2011). “A Bayesian evidence synthesis for estimating Campylobacteriosis prevalence.” Risk Analysis, 31(7): 1141–1155.
  • Bardenet, R., Doucet, A., and Holmes, C. (2017). “On Markov chain Monte Carlo methods for tall data.” Journal of Machine Learning Research, 18(47): 1–43.
  • Besbeas, P., Freeman, S. N., Morgan, B. J. T., and Catchpole, E. A. (2002). “Integrating mark–recapture–recovery and census data to estimate animal abundance and demographic parameters.” Biometrics, 58(3): 540–547.
  • Birrell, P. J., De Angelis, D., Wernisch, L., Tom, B. D., Roberts, G. O., and Pebody, R. G. (2016). “Efficient real-time monitoring of an emerging influenza epidemic: how feasible?” arXiv:1608.05292.
  • Brooks, S. P., King, R., and Morgan, B. J. T. (2004). “A Bayesian approach to combining animal abundance and demographic data.” Animal Biodiversity and Conservation, 27(1): 515–529.
  • Clemen, R. T. and Winkler, R. L. (1999). “Combining probability distributions from experts in risk analysis.” Risk Analysis, 19(2): 187–203.
  • Commenges, D. and Hejblum, B. P. (2012). “Evidence synthesis through a degradation model applied to myocardial infarction.” Lifetime Data Analysis, 19(1): 1–18.
  • Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper Markov laws in the statistical analysis of decomposable graphical models.” Annals of Statistics, 21(3): 1272–1317.
  • Department of Health (2011). “Department of Health Winter Watch.”
  • Doucet, A., de Freitas, N., and Gordon, N. (eds.) (2013). Sequential Monte Carlo Methods in Practice. New York: Springer Science & Business Media.
  • Draper, D. (1995). “Assessment and propagation of model uncertainty.” Journal of the Royal Statistical Society: Series B (Methodological), 57(1): 45–97.
  • Durante, F. and Sempi, C. (2010). “Copula theory: an introduction.” In Jaworski, P., Durante, F., Härdle, W. K., and Rychlik, T. (eds.), Copula Theory and its Applications, 3–31. Berlin: Springer-Verlag.
  • Eddy, D. M., Hasselblad, V., and Shachter, R. (1992). Meta-Analysis by the Confidence Profile Method. London: Academic Press.
  • Gåsemyr, J. and Natvig, B. (2009). “Extensions of a conflict measure of inconsistencies in Bayesian hierarchical models.” Scandinavian Journal of Statistics, 36(4): 822–838.
  • Goudie, R. J. B., Hovorka, R., Murphy, H. R., and Lunn, D. (2015). “Rapid model exploration for complex hierarchical data: application to pharmacokinetics of insulin aspart.” Statistics in Medicine, 34(23): 3144–3158.
  • Goudie, R. J. B., Presanis, A. M., Lunn, D., De Angelis, D., and Wernisch, L. (2019). “Supplementary Material for: “Joining and splitting models with Markov melding”.” Bayesian Analysis.
  • Green, P. J., Hjort, N. L., and Richardson, S. (2003). “Introducing highly structured stochastic systems.” In Green, P. J., Hjort, N. L., and Richardson, S. (eds.), Highly Structured Stochastic Systems, 1–12. Oxford: Oxford University Press.
  • Hasselblad, V., Eddy, D. M., and Kotchmar, D. J. (1992). “Synthesis of environmental evidence: nitrogen dioxide epidemiology studies.” Journal of the Air & Waste Management Association, 42(5): 662–671.
  • Henderson, D. J. and Parmeter, C. F. (2015). Applied Nonparametric Econometrics. New York: Cambridge University Press.
  • Hinton, G. E. (2002). “Training products of experts by minimizing contrastive divergence.” Neural Computation, 14(8): 1771–1800.
  • Huang, Z. and Gelman, A. (2005). “Sampling for Bayesian computation with large datasets.” Working paper.
  • Jackson, C. H., Best, N. G., and Richardson, S. (2009). “Bayesian graphical models for regression on multiple data sets with different variables.” Biostatistics, 10(2): 335–351.
  • Jackson, C. H., Jit, M., Sharples, L. D., and De Angelis, D. (2015). “Calibration of complex models through Bayesian evidence synthesis.” Medical Decision Making, 35(2): 148–161.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford: Clarendon Press.
  • Leonelli, M. (2015). “Bayesian decision support in complex modular systems: an algebraic and graphical approach.” Ph.D. thesis, University of Warwick, UK.
  • Liang, L.-J. and Weiss, R. E. (2007). “A hierarchical semiparametric regression model for combining HIV-1 phylogenetic analyses using iterative reweighting algorithms.” Biometrics, 63(3): 733–741.
  • Lindley, D. V., Tversky, A., and Brown, R. V. (1979). “On the reconciliation of probability assessments.” Journal of the Royal Statistical Society: Series A (General), 142(2): 146–180.
  • Liu, F., Bayarri, M. J., and Berger, J. O. (2009). “Modularization in Bayesian analysis, with emphasis on analysis of computer models.” Bayesian Analysis, 4(1): 119–150.
  • Lunn, D., Barrett, J., Sweeting, M., and Thompson, S. (2013a). “Fully Bayesian hierarchical modelling in two stages, with application to meta-analysis.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 62(4): 551–572.
  • Lunn, D., Jackson, C., Best, N., Thomas, A., and Spiegelhalter, D. (2013b). The BUGS Book: A Practical Introduction to Bayesian Analysis. Boca Raton: CRC Press.
  • Massa, M. S. and Lauritzen, S. L. (2010). “Combining statistical models.” In Viana, M. A. G. and Wynn, H. P. (eds.), Contemporary Mathematics: Algebraic Methods in Statistics and Probability II, 239–260.
  • Massa, M. S. and Riccomagno, E. (2017). “Algebraic representations of Gaussian Markov combinations.” Bernoulli, 23(1): 626–644.
  • Minsker, S., Srivastava, S., Lin, L., and Dunson, D. B. (2017). “Robust and scalable Bayes via a median of subset posterior measures.” Journal of Machine Learning Research, 18(124): 1–40.
  • Moran, E. V. and Clark, J. S. (2011). “Estimating seed and pollen movement in a monoecious plant: a hierarchical Bayesian approach integrating genetic and ecological data.” Molecular Ecology, 20(6): 1248–1262.
  • Müller, P. (1991). “A generic approach to posterior integration and Gibbs sampling.” Technical Report 91-09, Purdue University.
  • Neiswanger, W., Wang, C., and Xing, E. P. (2014). “Asymptotically exact, embarrassingly parallel MCMC.” In Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14), 623–632. Corvallis, Oregon: AUAI Press.
  • Neuenschwander, B., Branson, M., and Spiegelhalter, D. J. (2009). “A note on the power prior.” Statistics in Medicine, 28(28): 3562–3566.
  • O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., and Rakow, T. (2006). Uncertain Judgements: Eliciting Experts’ Probabilities. Chichester: John Wiley & Sons.
  • Plummer, M. (2015a). “Cuts in Bayesian graphical models.” Statistics and Computing, 25(1): 37–43.
  • Plummer, M. (2015b). “JAGS Version 4.0.1 user manual.”
  • Poole, D. and Raftery, A. E. (2000). “Inference for deterministic simulation models: The Bayesian melding approach.” Journal of the American Statistical Association, 95(452): 1244–1255.
  • Presanis, A. M., Ohlssen, D., Cui, K., Rosinska, M., and De Angelis, D. (2016). “Conflict diagnostics for evidence synthesis in a multiple testing framework.” arXiv:1702.07304.
  • Presanis, A. M., Ohlssen, D., Spiegelhalter, D. J., and De Angelis, D. (2013). “Conflict diagnostics in directed acyclic graphs, with applications in Bayesian evidence synthesis.” Statistical Science, 28(3): 376–397.
  • Presanis, A. M., Pebody, R. G., Birrell, P. J., Tom, B. D. M., Green, H. K., Durnall, H., Fleming, D., and De Angelis, D. (2014). “Synthesising evidence to estimate pandemic (2009) A/H1N1 influenza severity in 2009–2011.” Annals of Applied Statistics, 8(4): 2378–2403.
  • Public Health England (2014). “Sources of UK flu data: influenza surveillance in the UK.”
  • Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods. New York: Springer.
  • Scott, S. L., Blocker, A. W., Bonassi, F. V., Chipman, H. A., George, E. I., and McCulloch, R. E. (2016). “Bayes and big data: The consensus Monte Carlo algorithm.” International Journal of Management Science and Engineering Management, 11(2): 78–88.
  • Shubin, M., Lebedev, A., Lyytikäinen, O., and Auranen, K. (2016). “Revealing the true incidence of pandemic A(H1N1)pdm09 influenza in Finland during the first two seasons—An analysis based on a dynamic transmission model.” PLOS Computational Biology, 12(3): e1004803.
  • Sugiyama, M., Suzuki, T., and Kanamori, T. (2012). Density Ratio Estimation in Machine Learning. New York: Cambridge University Press.
  • Tom, J. A., Sinsheimer, J. S., and Suchard, M. A. (2010). “Reuse, recycle, reweigh: combating influenza through efficient sequential Bayesian computation for massive data.” Annals of Applied Statistics, 4(4): 1722–1748.
  • Turner, R. M., Spiegelhalter, D. J., Smith, G. C. S., and Thompson, S. G. (2009). “Bias modelling in evidence synthesis.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1): 21–47.
  • Wang, X. and Dunson, D. B. (2013). “Parallel MCMC via Weierstrass Sampler.” arXiv:1312.4605.
  • Welton, N. J., Cooper, N. J., Ades, A. E., Lu, G., and Sutton, A. J. (2008). “Mixed treatment comparison with multiple outcomes reported inconsistently across trials: Evaluation of antivirals for treatment of influenza A and B.” Statistics in Medicine, 27(27): 5620–5639.
  • Welton, N. J., Sutton, A. J., Cooper, N. J., Abrams, K. R., and Ades., A. (2012). Evidence Synthesis for Decision Making in Healthcare. Chichester: John Wiley & Sons.

Supplemental materials