Bayesian Analysis

Exact and Approximate Bayesian Inference for Low Integer-Valued Time Series Models with Intractable Likelihoods

Christopher C. Drovandi, Anthony N. Pettitt, and Roy A. McCutchan

Full-text: Open access


In this paper we develop a likelihood-free simulation methodology in order to obtain Bayesian inference for models for low integer-valued time series data that have computationally demanding likelihood functions. The algorithm fits within the framework of particle Markov chain Monte Carlo (PMCMC) methods and uses a so-called alive particle filter. The particle filter requires only model simulations and, in this regard, our approach has connections with approximate Bayesian computation (ABC). However, an advantage of using the PMCMC approach in this setting is that simulated data can be matched with data observed one-at-a-time, rather than attempting to match on the full dataset simultaneously or on a low-dimensional non-sufficient summary statistic, which is common practice in ABC. For low integer-valued time series data, we find that it is often computationally feasible to match simulated data with observed data exactly. The alive particle filter uses negative binomial sampling in order to maintain a fixed number of particles. The algorithm creates an unbiased estimate of the likelihood, resulting in exact posterior inferences when included in an MCMC algorithm. In cases where exact matching is computationally prohibitive, a tolerance is introduced as in ABC. This paper further develops the alive particle filter by introducing auxiliary variables so that partially observed and/or non-Markovian models can be accommodated. We demonstrate that Bayesian model choice problems involving such models can be handled with this approach. The methodology is illustrated on a wide variety of models for simulated and real low-count time series data involving a rich set of applications.

Article information

Bayesian Anal., Volume 11, Number 2 (2016), 325-352.

First available in Project Euclid: 20 April 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

approximate Bayesian computation branching process INARMA model Markov process particle filter particle Markov chain Monte Carlo pseudo-marginal methods


Drovandi, Christopher C.; Pettitt, Anthony N.; McCutchan, Roy A. Exact and Approximate Bayesian Inference for Low Integer-Valued Time Series Models with Intractable Likelihoods. Bayesian Anal. 11 (2016), no. 2, 325--352. doi:10.1214/15-BA950.

Export citation


  • Andrieu, C., Doucet, A., and Holenstein, R. (2010). “Particle Markov chain Monte Carlo methods.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3): 269–342.
  • Andrieu, C. and Roberts, G. O. (2009). “The pseudo-marginal approach for efficient Monte Carlo computations.” The Annals of Statistics, 37(2): 697–725.
  • Barthelmé, S. and Chopin, N. (2014). “Expectation propagation for likelihood-free inference.” Journal of the American Statistical Association, 109(505): 315–333.
  • Blum, M. G. B., Nunes, M. A., Prangle, D., and Sisson, S. A. (2013). “A comparative review of dimension reduction methods in approximate Bayesian computation.” Statistical Science, 28: 189–208.
  • Chopin, N., Jacob, P. E., and Papaspiliopoulos, O. (2013). “SMC$^{2}$: an efficient algorithm for sequential analysis of state space models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75(3): 397–426.
  • Cui, Y. and Lund, R. (2009). “A new look at time series of counts.” Biometrika, 96(4): 781–792.
  • Drovandi, C. C. and Pettitt, A. N. (2008). “Multivariate Markov process models for the transmission of Methicillin-resistant Staphylococcus aureus in a hospital ward.” Biometrics, 64(3): 851–859.
  • — (2011). “Using approximate Bayesian computation to estimate transmission rates of nosocomial pathogens.” Statistical Communications in Infectious Diseases, 3(1): 2.
  • Drovandi, C. C., Pettitt, A. N., Henderson, R. D., and McCombe, P. A. (2014). “Marginal reversible jump Markov chain Monte Carlo with application to motor unit number estimation.” Computational Statistics & Data Analysis, 72: 128–146.
  • Drovandi, C. C., Pettitt, A. N., and McCutchan, R. A. (2015). “Supplementary Material for Exact and Approximate Bayesian Inference for Low Integer-Valued Time Series Models with Intractable Likelihoods.” Bayesian Analysis.
  • Eduarda Silva, M. and Pereira, I. (2012). “Detection of additive outliers in Poisson Integer-valued autoregressive time series.” arXiv:1204.6516v1.
  • Enciso-Mora, V., Neal, P., and Subba Rao, T. (2009). “Efficient order selection algorithms for integer-valued ARMA processes.” Journal of Time Series Analysis, 30(1): 1–18.
  • Fearnhead, P., Giagos, V., and Sherlock, C. (2014). “Inference for reaction networks using the linear noise approximation.” Biometrics, 70(2): 457–466.
  • Golightly, A. and Wilkinson, D. J. (2005). “Bayesian inference for stochastic kinetic models using a diffusion approximation.” Biometrics, 61(3): 781–788.
  • González, M., Gutiérrez, C., Martínez, R., and Inés, M. (2013). “Bayesian inference for controlled branching processes through MCMC and ABC methodologies.” Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 107(2): 459–473.
  • Gordon, N. J., Salmond, D. J., and Smith, A. F. M. (1993). “Novel approach to nonlinear/non-Gaussian Bayesian state estimation.” In: Radar and Signal Processing, IEE Proceedings F, volume 140, 107–113.
  • Green, P. J. (1995). “Reversible jump Markov chain Monte Carlo computation and Bayesian model determination.” Biometrika, 82(4): 711–732.
  • Greenwood, M. (1931). “On the statistical measure of infectiousness.” The Journal of Hygiene, (3): 336–351.
  • Hastie, D. I. and Green, P. J. (2012). “Model choice using reversible jump Markov chain Monte Carlo.” Statistica Neerlandica, 66(3): 309–338.
  • He, D., Ionides, E. L., and King, A. A. (2010). “Plug-and-play inference for disease dynamics: measles in large and small populations as a case study.” Journal of the Royal Society Interface, 7(43): 271–283.
  • Holenstein, R. (2009). “Particle Markov chain Monte Carlo.” Ph.D. thesis, The University Of British Columbia.
  • Jasra, A., Lee, A., Yau, C., and Zhang, X. (2013). “The alive particle filter.” arXiv:1304.0151.
  • Jazi, M. A., Jones, G., and Lai, C.-D. (2012). “First-order integer valued AR processes with zero inflated Poisson innovations.” Journal of Time Series Analysis, 33(6): 954–963.
  • Le Gland, F. and Oudjane, N. (2006). “A Sequential Particle Algorithm that Keeps the Particle System Alive.” In: Stochastic Hybrid Systems: Theory and Safety Critical Applications, volume 337 of Lecture Notes in Control and Information Sciences, Springer, 351–389.
  • Lee, X. J., Drovandi, C. C., and Pettitt, A. N. (2015). “Model choice problems using approximate Bayesian computation with applications to pathogen transmission data sets.” Biometrics, 71(1): 198–207.
  • Marjoram, P., Molitor, J., Plagnol, V., and Tavaré, S. (2003). “Markov chain Monte Carlo without likelihoods.” Proceedings of the National Academy of Sciences, 100(26): 15324–15328.
  • Martin, V. L., Tremayne, A. R., and Jung, R. C. (2014). “Efficient method of moments estimators for integer time series models.” Journal of Time Series Analysis, 35(6): 491–516.
  • McBryde, E. S., Pettitt, A. N., and McElwain, D. L. S. (2007). “A stochastic mathematical model of Methicillin-resistant Staphylococcus aureus transmission in an intensive care unit: predicting the impact of interventions.” Journal of Theoretical Biology, 245(3): 470–481.
  • McKinley, T. J., Ross, J. V., Deardon, R., and Cook, A. R. (2014). “Simulation-based Bayesian inference for epidemic models.” Computational Statistics & Data Analysis, 71: 434–447.
  • Moler, C. and van Loan, C. (2003). “Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later.” SIAM Review, 45(1): 3–49.
  • Neal, P. and Subba Rao, T. (2007). “MCMC for integer-valued ARMA processes.” Journal of Time Series Analysis, 28(1): 92–110.
  • Pedeli, X. and Karlis, D. (2011). “A bivariate INAR(1) process with application.” Statistical Modelling, 11(4): 325–349.
  • Pericchi, L. R. (2005). “Model selection and hypothesis testing based on objective probabilities and Bayes factors.” In: Bayesian thinking: modeling and computation, Elsevier B. V., 115–149.
  • Persing, A. and Jasra, A. (2014). “Twisting the alive particle filter.” Methodology and Computing in Applied Probability, in press.
  • Plummer, M., Best, N., Cowles, K., and Vines, K. (2006). “CODA: Convergence Diagnosis and Output Analysis for MCMC.” R News, 6(1): 7–11.
  • Sidje, R. B. (1998). “Expokit: a software package for computing matrix exponentials.” ACM Transactions on Mathematical Software (TOMS), 24(1): 130–156.
  • Tran, M.-N., Scharth, M., Pitt, M. K., and Kohn, R. (2013). “Importance sampling squared for Bayesian inference in latent variable models.” arXiv:1309.3339v3.
  • White, S. R., Kypraios, T., and Preston, S. P. (2015). “Piecewise approximate Bayesian computation: fast inference for discretely observed Markov models using a factorised posterior distribution.” Statistics and Computing, 25(2): 289–301.
  • Wood, S. N. (2010). “Statistical inference for noisy nonlinear ecological dynamic systems.” Nature, 466(7310): 1102–1104.

Supplemental materials