Statistical Science

Comparison and Assessment of Epidemic Models

Gavin J. Gibson, George Streftaris, and David Thong

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Model criticism is a growing focus of research in stochastic epidemic modelling, following the successful addressing of model fitting and parameter estimation via powerful computationally intensive statistical methods in recent decades. In this paper, we consider a variety of stochastic representations of epidemic outbreaks, with emphasis on individual-based continuous-time models, and review the range of model comparison and assessment approaches currently applied. We highlight some of the factors that can serve to impede checking and criticism of epidemic models such as lack of replication, partial observation of processes, lack of prior knowledge on parameters in competing models, the nonnested nature of models to be compared, and computational challenges. Based on a wide selection of approaches as reported in the literature, we argue that assessment and comparison of stochastic epidemic models is complex and often, by necessity, idiosyncratic to specific applications. We particularly advocate following the advice of Box [J. Amer. Statist. Assoc. 71 (1976) 791–799] to be selective regarding the model inadequacies for which one tests and, moreover, to be open to the blending of classical and Bayesian ideas in epidemic model criticism, rather than adhering to a single philosophy.

Article information

Statist. Sci., Volume 33, Number 1 (2018), 19-33.

First available in Project Euclid: 2 February 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Epidemic models model comparison model criticism Bayesian methods classical methods


Gibson, Gavin J.; Streftaris, George; Thong, David. Comparison and Assessment of Epidemic Models. Statist. Sci. 33 (2018), no. 1, 19--33. doi:10.1214/17-STS615.

Export citation


  • [1] Alharthi, M. (2015). Bayesian model assessment for stochastic epidemic models, Ph.D. thesis, Univ. Nottingham.
  • [2] Bailey, N. T. J. (1975). The Mathematical Theory of Infectious Diseases and Its Applications, 2nd ed. Hafner Press, New York.
  • [3] Bjørnstad, O. N. and Falck, W. (2001). Nonparametric spatial covariance functions: Estimation and testing. Environ. Ecol. Stat. 8 53–70.
  • [4] Blum, M. and Tran, V. (2010). HIV with contact tracing: A case study in approximate Bayesian computation. Biostatistics 11 644–660.
  • [5] Box, G. E. P. (1976). Science and statistics. J. Amer. Statist. Assoc. 71 791–799.
  • [6] Box, G. E. P. (1980). Sampling and Bayes’ inference in scientific modelling and robustness. J. Roy. Statist. Soc. Ser. A 143 383–430.
  • [7] Boys, R. J. and Giles, P. R. (2007). Bayesian inference for stochastic epidemic models with time-inhomogeneous removal rates. J. Math. Biol. 55 223–247.
  • [8] Celeux, G., Forbes, F., Robert, C. P. and Titterington, D. M. (2006). Deviance information criteria for missing data models. Bayesian Anal. 1 651–673.
  • [9] Chis Ster, I., Singh, B. K. and Ferguson, N. M. (2009). Epidemiological inference for partially observed epidemics: The example of the 2001 foot and mouth epidemic in Great Britain. Epidemics 1 21–34.
  • [10] Clancy, D. and O’Neill, P. D. (2007). Exact Bayesian inference and model selection for stochastic models of epidemics among a community of households. Scand. J. Stat. 34 259–274.
  • [11] Cook, A. R., Gibson, G. J., Gottwald, T. and Gilligan, C. A. (2008). Constructing the effect of alternative intervention strategies in historic epidemics. J. R. Soc. Interface 5 1203–1213.
  • [12] Dawid, A. P. and Stone, M. (1982). The functional-model basis of fiducial inference. Ann. Statist. 10 1054–1074.
  • [13] Deeth, L., Deardon, R. and Gillis, D. (2015). Model choice using the Deviance Information Criterion for latent conditional individual-level models of infectious disease spread. Epidemiologic Methods 4 47–68.
  • [14] De Angelis, D., Presanis, A. M., Birrell, P. J., Tomba, G. S. and House, T. (2015). Four key challenges in infectious disease modelling using data from multiple sources. Epidemics 10 83–87.
  • [15] Draper, D. (1995). Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B. Stat. Methodol. 57 45–97.
  • [16] Finkenstädt, B. F. and Grenfell, B. T. (2000). Time series modelling of childhood diseases: A dynamical systems approach. J. R. Stat. Soc. Ser. C. Appl. Stat. 49 187–205.
  • [17] Gibson, G. J., Otten, W., Filipe, J. A. N., Cook, A., Marion, G. and Gilligan, C. A. (2006). Bayesian estimation for percolation models of disease spread in plant populations. Stat. Comput. 16 391–402.
  • [18] Gibson, G. J. and Renshaw, E. (2001). Likelihood estimation for stochastic compartmental models using Markov chain methods. Stat. Comput. 11 347–358.
  • [19] Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning. In A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues (G. Keren and C. Lewis, eds.). Lawrence Erlbaum Associates, Hillsdale, NJ.
  • [20] Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 711–732.
  • [21] Jewell, C. P., Keeling, M. J. and Roberts, G. O. (2008). Predicting undetected infections during the 2007 foot and mouth disease outbreak. J. R. Soc. Interface 6 1145–1151.
  • [22] Jewell, C. P., Kypraios, T., Neal, P. and Roberts, G. O. (2009). Bayesian analysis for emerging infectious diseases. Bayesian Anal. 4 465–496.
  • [23] Jombart, T., Didelot, X., Cauchemez, S., Viboud, F. C. and Ferguson, N. (2014). Bayesian reconstruction of disease outbreaks by combining epidemiologic and genomic data. PLoS Comput. Biol. 10 e1003457.
  • [24] Keeling, M. J., Woolhouse, M. E. J., Shaw, D. J., Matthews, L., Chase-Topping, M., Haydon, D. T., Cornell, S. J., Kappey, J., Wilesmith, J. and Grenfell, B. T. (2001). Dynamics of the 2001 UK Foot and Mouth Epidemic: Stochastic dispersal in a heterogeneous landscape. Science 294 813–817.
  • [25] King, A. A., Ionides, E. L., Pascual, M. and Bouma, M. J. (2008). Inapparent infections and cholera dynamics. Nature 454 877–880.
  • [26] Knock, E. S. and O’Neill, P. D. (2014). Bayesian model choice for epidemic models with two levels of mixing. Biostatistics 15 46–59.
  • [27] Lau, M. S. Y., Dalziel, B. D., Funk, S., McClelland, A., Tiffany, A., Riley, S., Metcalf, C., Jessica, E. and Grenfell, B. T. (2017). Spatial and temporal dynamics of superspreading events in the 2014–2015 West Africa Ebola epidemic. Proc. Natl. Acad. Sci. USA 114 2337–2342.
  • [28] Lau, M. S. Y., Marion, G., Streftaris, G. and Gibson, G. J. (2014). New model diagnostics for spatio-temporal systems in ecology and epidemiology. J. R. Soc. Interface 11 20131093. DOI:10.1098/rsif.2013.1093.
  • [29] Lau, M. S. Y., Marion, G., Streftaris, G. and Gibson, G. J. (2015). A systematic Bayesian integration of epidemiological and genetic data. PLoS Comput. Biol. 11. e1004633. DOI:10.1371/journal.pcbi.1004633.
  • [30] McKinley, T., Cook, A. R. and Deardon, R. (2009). Inference in epidemic models without likelihoods. Int. J. Biostat. 5 Art. 24, 39.
  • [31] Meng, X.-L. (1994). Posterior predictive $p$-values. Ann. Statist. 22 1142–1160.
  • [32] Meng, X.-L. and Vaida, F. (2006). Comment on article by Celeux et al. [MR2282197]. Bayesian Anal. 1 687–698.
  • [33] Monto, A. S., Koopman, J. S. and Longini, I. M. (1985). Tecumseh study of illness: XIII, influenza infection and disease, 1976–1981. Amer. J. Epidemiol. 121 811–822.
  • [34] Morelli, M. J., Thébaud, G., Chadœuf, J., King, D. P., Haydon, D. T. and Soubeyrand, S. (2012). A Bayesian inference framework to reconstruct transmission trees using epidemiological and genetic data. PLoS Comput. Biol. 8 e1002768, 14.
  • [35] Morton, A. and Finkenstädt, B. F. (2005). Discrete time modelling of disease incidence time series by using Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. C. Appl. Stat. 54 575–594.
  • [36] Neal, P. J. and Roberts, G. O. Statistical inference and model selection for the 1861 Hagelloch measles epidemic. Biostatistics 5 249–261.
  • [37] Neri, F. M., Cook, A. R., Gibson, G. J., Gottwald, T. R. and Gilligan, C. A. (2014). Bayesian analysis for inference of an emerging epidemic: Citrus canker in urban landscapes. PLoS Comput. Biol. 10 e1003587. DOI:10.1371/journal.pcbi.1003587.
  • [38] O’Neill, P. D. and Becker, N. G. (2001). Inference for an epidemic when susceptibility varies. Biostatistics 2 99–108.
  • [39] O’Neill, P. D. and Roberts, G. O. (1999). Bayesian inference for partially observed epidemics. J. R. Stat. Soc., A 162 121–129.
  • [40] Papaspiliopoulos, O., Roberts, G. O. and Sköld, M. (2007). A general framework for the parametrization of hierarchical models. Statist. Sci. 22 59–73.
  • [41] Parry, M., Gibson, G. J., Parnell, S., Gottwald, T. R., Irey, M. S., Gast, T. C. and Gilligan, C. A. (2014). Bayesian inference for an emerging arboreal epidemic in the presence of control. Proc. Natl. Acad. Sci. USA 111 6258–6262.
  • [42] Sellke, T. (1983). On the asymptotic distribution of the size of a stochastic epidemic. J. Appl. Probab. 20 390–394.
  • [43] Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 583–639.
  • [44] Streftaris, G. and Gibson, G. J. (2004). Bayesian analysis of experimental epidemics of foot-and-mouth disease. Proc. R. Soc. Lond., B Biol. Sci. 271 1111–1117.
  • [45] Streftaris, G. and Gibson, G. J. (2012). Non-exponential tolerance to infection in epidemic systems—modelling, inference and assessment. Biostatistics 13 580–593.
  • [46] Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. J. Amer. Statist. Assoc. 82 528–550.
  • [47] Tildesley, M. J., Deardon, R., Savill, N. J., Bessell, P. R., Brooks, S. P., Woolhouse, M. E. J., Grenfell, B. T. and Keeling, M. J. (2008). Accuracy of models for the 2001 foot-and-mouth epidemic. Proc. R. Soc. Lond., B Biol. Sci. 275 1459–1468.
  • [48] Verdinelli, I. and Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage–Dickey density ratio. J. Amer. Statist. Assoc. 90 614–618.
  • [49] Ypma, R., Bataille, A., Stegeman, A., Koch, G., Wallinga, J. and Van Ballegooijen, W. (2012). Unravelling transmission trees of infectious diseases by combining genetic and epidemiological data. Proc. R. Soc. Lond., B Biol. Sci. 279 444–450.