Annals of Applied Statistics

Efficient real-time monitoring of an emerging influenza pandemic: How feasible?

Paul J. Birrell, Lorenz Wernisch, Brian D. M. Tom, Leonhard Held, Gareth O. Roberts, Richard G. Pebody, and Daniela De Angelis

Full-text: Open access


A prompt public health response to a new epidemic relies on the ability to monitor and predict its evolution in real time as data accumulate. The 2009 A/H1N1 outbreak in the UK revealed pandemic data as noisy, contaminated, potentially biased and originating from multiple sources. This seriously challenges the capacity for real-time monitoring. Here, we assess the feasibility of real-time inference based on such data by constructing an analytic tool combining an age-stratified SEIR transmission model with various observation models describing the data generation mechanisms. As batches of data become available, a sequential Monte Carlo (SMC) algorithm is developed to synthesise multiple imperfect data streams, iterate epidemic inferences and assess model adequacy amidst a rapidly evolving epidemic environment, substantially reducing computation time in comparison to standard MCMC, to ensure timely delivery of real-time epidemic assessments. In application to simulated data designed to mimic the 2009 A/H1N1 epidemic, SMC is shown to have additional benefits in terms of assessing predictive performance and coping with parameter nonidentifiability.

Article information

Ann. Appl. Stat., Volume 14, Number 1 (2020), 74-93.

Received: June 2018
Revised: March 2019
First available in Project Euclid: 16 April 2020

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Sequential Monte Carlo resample-move real-time inference pandemic influenza SEIR transmission model


Birrell, Paul J.; Wernisch, Lorenz; Tom, Brian D. M.; Held, Leonhard; Roberts, Gareth O.; Pebody, Richard G.; De Angelis, Daniela. Efficient real-time monitoring of an emerging influenza pandemic: How feasible?. Ann. Appl. Stat. 14 (2020), no. 1, 74--93. doi:10.1214/19-AOAS1278.

Export citation


  • Ahrens, H. (1976). Multivariate variance-covariance components (MVCC) and generalized intraclass correlation coefficient (GICC). Biom. J. 18 527–533.
  • Banterle, M., Grazian, C., Lee, A. and Robert, C. P. (2019). Accelerating Metropolis–Hastings algorithms by Delayed Acceptance. Foundations of Data Science 1 103–128.
  • Bettencourt, L. M. A. and Ribeiro, R. M. (2008). Real time Bayesian estimation of the epidemic potential of emerging infectious diseases. PLoS ONE 3 e2185.
  • Bierkens, J., Fearnhead, P. and Roberts, G. (2019). The Zig-Zag process and super-efficient sampling for Bayesian analysis of big data. Ann. Stat. 47 1288–1320.
  • Birrell, P. J., Ketsetzis, G., Gay, N. G., Cooper, B. S., Presanis, A. M., Harris, R. J., Charlett, A., Zhang, X.-S., White, P. et al. (2011). Bayesian modelling to unmask and predict the influenza A/H1N1pdm dynamics in London. Proc. Natn. Acad. Sci. USA 108 18238–18243.
  • Birrell, P. J., Wernisch, L., Tom, B. D. M., Held, L., Roberts, G. O., Pebody, R. G. and De Angelis, D. (2020). Supplement to “Efficient real-time monitoring of an emerging influenza pandemic: How feasible?.”
  • Blei, D. M., Kucukelbir, A. and McAuliffe, J. D. (2017). Variational inference: A review for statisticians. J. Amer. Statist. Assoc. 112 859–877.
  • Camacho, A., Kucharski, A., Aki-Sawyerr, Y., White, M. A., Flasche, S., Baguelin, M., Pollington, T., Carney, J. R., Glover, R. et al. (2015). Temporal changes in Ebola transmission in Sierra Leone and implications for control requirements: A real-time modelling study. PLoS Curr 7.
  • Carpenter, J., Clifford, P. and Fearnhead, P. (1999). Improved particle filter for nonlinear problems. IEE Proc. Radar Sonar Navig. 146 2+.
  • Cauchemez, S., Boëlle, P. Y., Thomas, G. and Valleron, A. J. (2006). Estimating in real time the efficacy of measures to control emerging communicable diseases. Am. J. Epidemiol. 164 591–597.
  • Chopin, N. (2002). A sequential particle filter method for static models. Biometrika 89 539–551.
  • Czado, C., Gneiting, T. and Held, L. (2009). Predictive model assessment for count data. Biometrics 65 1254–1261.
  • Dawid, A. P. (1984). Statistical theory. The prequential approach. J. Roy. Statist. Soc. Ser. A 147 278–292.
  • Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 411–436.
  • Donner, A. and Koval, J. J. (1980). The estimation of intraclass correlation in the analysis of family data. Biometrics 36 19–25.
  • Doucet, A. and Johansen, A. M. (2011). A tutorial on particle filtering and smoothing: Fifteen years later. In The Oxford Handbook of Nonlinear Filtering 656–704. Oxford Univ. Press, Oxford.
  • Dukic, V., Lopes, H. F. and Polson, N. G. (2012). Tracking epidemics with Google Flu Trends data and a state-space SEIR model. J. Amer. Statist. Assoc. 107 1410–1426.
  • Dureau, J., Kalogeropoulos, K. and Baguelin, M. (2013). Capturing the time-varying drivers of an epidemic using stochastic dynamical systems. Biostatistics 14 541–555.
  • Farah, M., Birrell, P., Conti, S. and De Angelis, D. (2014). Bayesian emulation and calibration of a dynamic epidemic model for A/H1N1 influenza. J. Amer. Statist. Assoc. 109 1398–1411.
  • Fearnhead, P. (2002). Markov chain Monte Carlo, sufficient statistics, and particle filters. J. Comput. Graph. Statist. 11 848–862.
  • Fearnhead, P. and Taylor, B. M. (2013). An adaptive sequential Monte Carlo sampler. Bayesian Anal. 8 411–438.
  • Funk, S., Camacho, A., Kucharski, A. J., Eggo, R. M. and Edmunds, W. J. (2018). Real-time forecasting of infectious disease dynamics with a stochastic semi-mechanistic model. Epidemics 22 56–61.
  • Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: The 23rd Symposium on the Interface 156–163. Interface Foundation of North America, Fairfax Station, VA.
  • Gilks, W. R. and Berzuini, C. (2001). Following a moving target—Monte Carlo inference for dynamic Bayesian models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 63 127–146.
  • Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 123–214.
  • Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359–378.
  • Gordon, N. J., Salmond, D. J. and Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc-F 140 107–113.
  • Held, L., Meyer, S. and Bracher, J. (2017). Probabilistic forecasting in infectious disease epidemiology: The 13th Armitage lecture. Stat. Med. 36 3443–3460.
  • Jasra, A., Stephens, D. A. and Holmes, C. C. (2007). On population-based simulation for static inference. Stat. Comput. 17 263–279.
  • Jasra, A., Stephens, D. A., Doucet, A. and Tsagaris, T. (2011). Inference for Lévy-driven stochastic volatility models via adaptive sequential Monte Carlo. Scand. J. Stat. 38 1–22.
  • Jewell, C. P., Kypraios, T., Christley, R. M. and Roberts, G. O. (2009). A novel approach to real-time risk prediction for emerging infectious diseases: A case study in Avian Influenza H5N1. Prev. vet. med. 91 19–28.
  • Kantas, N., Beskos, A. and Jasra, A. (2014). Sequential Monte Carlo methods for high-dimensional inverse problems: A case study for the Navier–Stokes equations. SIAM/ASA J. Uncertain. Quantificat. 2 464–489.
  • Konishi, S., Khatri, C. G. and Rao, C. R. (1991). Inferences on multivariate measures of interclass and intraclass correlations in familial data. J. Roy. Statist. Soc. Ser. B 53 649–659.
  • Liang, F. and Wong, W. H. (2001). Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Amer. Statist. Assoc. 96 653–666.
  • Liu, J. S. and Chen, R. (1995). Blind deconvolution via sequential imputations. J. Amer. Statist. Assoc. 90 567–576.
  • Liu, J. S. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. J. Amer. Statist. Assoc. 93 1032–1044.
  • Meester, R., de Koning, J., de Jong, M. C. M. and Diekmann, O. (2002). Modeling and real-time prediction of classical swine fever epidemics. Biometrics 58 178–184.
  • Neal, R. M. (1996). Sampling from multimodal distributions using tempered transitions. Stat. Comput. 6 353–366.
  • Nemeth, C., Fearnhead, P. and Mihaylova, L. (2014). Sequential Monte Carlo methods for state and parameter estimation in abruptly changing environments. IEEE Trans. Signal Process. 62 1245–1255.
  • Ong, J. B. S., Chen, M. I.-C., Cook, A. R., Chyi, H., Lee, V. J., Pin, R. T., Ananth, P. and Gan, L. (2010). Real-time epidemic monitoring and forecasting of H1N1-2009 using influenza-like illness from general practice and family doctor clinics in Singapore. PLoS ONE 5 e10036.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Optimal scaling for various Metropolis–Hastings algorithms. Statist. Sci. 16 351–367.
  • Schuster, I., Strathmann, H., Paige, B. and Sejdinovic, D. (2017). Kernel sequential Monte Carlo. In Machine Learning and Knowledge Discovery in Databases (M. Ceci, J. Hollmén, L. Todorovski, C. Vens and S. Džeroski, eds.) 390–409. Springer, Cham.
  • Scientific Pandemic Influenza Advisory Committee: Subgroup On Modelling (2011). Modelling Summary. SPI-M-O Committee document (Accessed 4 February, 2016).
  • Seillier-Moiseiwitsch, F. and Dawid, A. P. (1993). On testing the validity of sequential probability forecasts. J. Amer. Statist. Assoc. 88 355–359.
  • Shaman, J. and Karspeck, A. (2012). Forecasting seasonal outbreaks of influenza. Proc. Natn. Acad. Sci. USA 109 20425–20430.
  • Sherlock, C., Fearnhead, P. and Roberts, G. O. (2010). The random walk Metropolis: Linking theory and practice through a case study. Statist. Sci. 25 172–190.
  • Shubin, M., Lebedev, A., Lyytikäinen, O. and Auranen, K. (2016). Revealing the true incidence of pandemic A(H1N1)pdm09 influenza in Finland during the first two seasons—an analysis based on a dynamic transmission model. PLoS Comput. Biol. 12 1–3.
  • Skvortsov, A. and Ristic, B. (2012). Monitoring and prediction of an epidemic outbreak using syndromic observations. Math. Biosci. 240 12–19.
  • Sokal, R. R. and Rohlf, F. (1981). Biometry, 2nd ed. 668. WH Feeman and Company, New York.
  • te Beest, D. E., Birrell, P. J., Wallinga, J., Angelis, D. D. and van Boven, M. (2015). Joint modelling of serological and hospitalization data reveals that high levels of pre-existing immunity and school holidays shaped the influenza A pandemic of 2009 in the Netherlands. J. R. Soc. Interface 12.
  • Viboud, C., Sun, K., Gaffey, R., Ajelli, M., Fumanelli, L., Merler, S., Zhang, Q., Chowell, G., Simonsen, L. et al. (2018). The RAPIDD Ebola forecasting challenge: Synthesis and lessons learnt. Epidemics 22 13–21.
  • Wallinga, J. and Teunis, P. (2004). Different epidemic curves for severe acute respiratory syndrome reveal similar impacts of control measures. Am. J. Epidemiol. 160 509–516.
  • West, M. (1993). Mixtures models, Monte Carlo, Bayesian updating and dynamic models. Computer Science and Statistics 24 325–333.
  • Whiteley, N., Johansen, A. M. and Godsill, S. (2011). Monte Carlo filtering of piecewise deterministic processes. J. Comput. Graph. Statist. 20 119–139.
  • Wu, J. T., Cowling, B. J., Lau, E. H. Y., Ip, D. K. M., Ho, L. M., Tsang, T., Chuang, S. K., Leung, P. Y., Lo, S. V. et al. (2010). School closure and mitigation of pandemic (H1N1) 2009, Hong Kong. Emerg. Infec. Dis. 16 538–541.

Supplemental materials

  • Efficient real-time monitoring of an emerging influenza epidemic: How feasible? Web appendix. Additional supporting tables, plots and mathematical detail omitted from the main manuscript for brevity.