Bayesian Analysis

Bayesian Inference for Diffusion-Driven Mixed-Effects Models

Gavin A. Whitaker, Andrew Golightly, Richard J. Boys, and Chris Sherlock

Full-text: Open access


Stochastic differential equations (SDEs) provide a natural framework for modelling intrinsic stochasticity inherent in many continuous-time physical processes. When such processes are observed in multiple individuals or experimental units, SDE driven mixed-effects models allow the quantification of both between and within individual variation. Performing Bayesian inference for such models using discrete-time data that may be incomplete and subject to measurement error is a challenging problem and is the focus of this paper. We extend a recently proposed MCMC scheme to include the SDE driven mixed-effects framework. Fundamental to our approach is the development of a novel construct that allows for efficient sampling of conditioned SDEs that may exhibit nonlinear dynamics between observation times. We apply the resulting scheme to synthetic data generated from a simple SDE model of orange tree growth, and real data on aphid numbers recorded under a variety of different treatment regimes. In addition, we provide a systematic comparison of our approach with an inference scheme based on a tractable approximation of the SDE, that is, the linear noise approximation.

Article information

Bayesian Anal., Volume 12, Number 2 (2017), 435-463.

First available in Project Euclid: 23 May 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

stochastic differential equation mixed-effects Markov chain Monte Carlo modified innovation scheme linear noise approximation

Creative Commons Attribution 4.0 International License.


Whitaker, Gavin A.; Golightly, Andrew; Boys, Richard J.; Sherlock, Chris. Bayesian Inference for Diffusion-Driven Mixed-Effects Models. Bayesian Anal. 12 (2017), no. 2, 435--463. doi:10.1214/16-BA1009.

Export citation


  • Aït-Sahalia, Y. (2002). “Maximum likelihood estimation of discretely sampled diffusions: a closed-form approximation approach.” Econometrica, 70(1): 223–262.
  • Aït-Sahalia, Y. (2008). “Closed-form likelihood expansions for multivariate diffusions.” The Annals of Statistics, 36(2): 906–937.
  • Berglund, M., Sunnåker, M., Adiels, M., Jirstrand, M., and Wennberg, B. (2011). “Investigations of a compartmental model for leucine kinetics using non-linear mixed effects models with ordinary and stochastic differential equations.” Mathematical Medicine and Biology, 29(4): 361–384.
  • Beskos, A., Kalogeropoulos, K., and Pazos, E. (2013). “Advanced MCMC methods for sampling on diffusion pathspace.” Stochastic Processes and their Applications, 123(4): 1415–1453.
  • Beskos, A., Papaspiliopoulos, O., Roberts, G. O., and Fearnhead, P. (2006). “Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68: 333–382.
  • Chib, S., Pitt, M. K., and Shephard, N. (2004). “Likelihood based inference for diffusion driven models.” Technical Report 2004-W20, Nuffield College, Oxford.
  • Clark, J. (1990). “The simulation of pinned diffusions.” In Proceedings of the 29th IEEE Conference on Decision and Control, 1990, 1418–1420. IEEE.
  • Delyon, B. and Hu, Y. (2006). “Simulation of conditioned diffusion and application to parameter estimation.” Stochastic Processes and Their Applications, 116: 1660–1675.
  • Donnet, S., Foulley, J., and Samson, A. (2010). “Bayesian analysis of growth curves using mixed models defined by stochastic differential equations.” Biometrics, 66(3): 733–741.
  • Durham, G. B. and Gallant, A. R. (2002). “Numerical techniques for maximum likelihood estimation of continuous-time diffusion processes.” Journal of Business and Economic Statistics, 20: 297–338.
  • Elerian, O., Chib, S., and Shephard, N. (2001). “Likelihood inference for discretely observed nonlinear diffusions.” Econometrica, 69: 959–993.
  • Eraker, B. (2001). “MCMC analysis of diffusion models with application to finance.” Journal of Business and Economic Statistics, 19(2): 177–191.
  • Fearnhead, P., Giagos, V., and Sherlock, C. (2014). “Inference for reaction networks using the linear noise approximation.” Biometrics, 70(2): 457–466.
  • Ferm, L., Lötstedt, P., and Hellander, A. (2008). “A hierarchy of approximations of the master equation scaled by a size parameter.” Journal of Scientific Computing, 34(2): 127–151.
  • Finkenstädt, B., Woodcock, D., Komorowski, M., Harper, C., Davis, J., White, M., and Rand, D. A. (2013). “Quantifying intrinsic and extrinsic noise in gene transcription using the linear noise approximation: an application to single cell data.” Annals of Applied Statistics, 7: 1960–1982.
  • Fuchs, C. (2013). Inference for diffusion processes with applications in Life Sciences. Heidelberg: Springer.
  • Gillespie, C. S. and Golightly, A. (2010). “Bayesian inference for generalized stochastic population growth models with application to aphids.” Journal of the Royal Statistical Society – Series C (Applied Statistics), 59(2): 341–357.
  • Golightly, A. and Wilkinson, D. J. (2006). “Bayesian sequential inference for nonlinear multivariate diffusions.” Statistics and Computing, 16: 323–338.
  • Golightly, A. and Wilkinson, D. J. (2008). “Bayesian inference for nonlinear multivariate diffusion models observed with error.” Computational Statistics & Data Analysis, 52(3): 1674–1693.
  • Golightly, A. and Wilkinson, D. J. (2010). “Markov chain Monte Carlo algorithms for SDE parameter estimation.” In Lawrence, N. D., Girolami, M., Rattray, M., and Sanguinetti, G. (eds.), Learning and inference in Computational Systems Biology, Computational Molecular Biology. MIT Press.
  • Golightly, A. and Wilkinson, D. J. (2011). “Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo.” Interface Focus, 1(6): 807–820.
  • Klim, S., Mortensen, S. B., Kristensen, N. R., Overgaard, R. V., and Madsen, H. (2009). “Population stochastic modelling (PSM) – An R package for mixed-effects models based on stochastic differential equations.” Computer Methods and Programs in Biomedicine, 94(3): 279–289.
  • Komorowski, M., Finkenstädt, B., Harper, C., and Rand, D. (2009). “Bayesian inference of biochemical kinetic parameters using the linear noise approximation.” BMC Bioinformatics, 10(1): 343.
  • Kou, S. C., Olding, B. P., Lysy, M., and Liu, J. S. (2012). “A multiresolution method for parameter estimation of diffusion processes.” Journal of the American Statistical Association, 107: 1558–1574.
  • Kurtz, T. G. (1970). “Solutions of ordinary differential equations as limits of pure jump Markov processes.” Journal of Applied Probability, 7(1): 49–58.
  • Lindström, E. (2012). “A regularized bridge sampler for sparsely sampled diffusions.” Statistics and Computing, 22(1): 615–623.
  • Matis, J. H., Kiffe, T. R., Matis, T. I., and Stevenson, D. E. (2006). “Application of population growth models based on Cumulative size to Pecan Aphids.” Journal of Agricultural, Biological, and Environmental Statistics, 11: 425–449.
  • Matis, T. I., Parajulee, M. N., Matis, J. H., and Shrestha, R. B. (2008). “A mechanistic model based analysis of cotton aphid population dynamics data.” Agricultural and Forest Entomology, 10(4): 355–362.
  • Overgaard, R. V., Jonsson, N., Tornøe, C. W., and Madsen, H. (2005). “Non-linear mixed-effects models with stochastic differential equations: implementation of an estimation algorithm.” Journal of Pharmacokinetics and Pharmacodynamics, 32(1): 85–107.
  • Papaspiliopoulos, O. and Roberts, G. O. (2012). “Importance sampling techniques for estimation of diffusion models.” In Statistical Methods for Stochastic Differential Equations, Monographs on Statistics and Applied Probability, 311–337. Chapman and Hall.
  • Papaspiliopoulos, O., Roberts, G. O., and Stramer, O. (2013). “Data augmentation for diffusions.” Journal of Computational and Graphical Statistics, 22: 665–688.
  • Pedersen, A. R. (1995). “A new approach to maximum likelihood estimation for stochastic differential equations based on discrete observations.” Scandinavian Journal of Statistics, 22(1): 55–71.
  • Picchini, U. and Ditlevsen, S. (2011). “Practical estimation of high dimensional stochastic differential mixed-effects models.” Computational Statistics and Data Analysis, 55(3): 1426–1444.
  • Picchini, U., Gaetano, S., and Ditlevsen, S. (2010). “Stochastic differential mixed-effects models.” Scandinavian Journal of Statistics, 37: 67–90.
  • Roberts, G. O. and Stramer, O. (2001). “On inference for partially observed nonlinear diffusion models using the Metropolis–Hastings algorithm.” Biometrika, 88(3): 603–621.
  • Schauer, M., van der Meulen, F., and van Zanten, H. (2014). “Guided proposals for simulating multi-dimensional diffusion bridges.” Available from
  • Sermaidis, G., Papaspiliopoulos, O., Roberts, G. O., Beskos, A., and Fearnhead, P. (2013). “Markov chain Monte Carlo for exact inference for diffusions.” Scandinavian Journal of Statistics, 40: 294–321.
  • Stramer, O. and Bognar, M. (2011). “Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach.” Bayesian Analysis, 6(2): 231–258.
  • Stramer, O., Bognar, M., and Scheider, P. (2010). “Bayesian inference for discretely sampled Markov processes with closed-form likelihood expansions.” Journal of Financial Econometrics, 8: 450–480.
  • Stramer, O. and Yan, J. (2007). “Asymptotics of an efficient Monte Carlo estimation for the transition density of diffusion processes.” Methodology and Computing in Applied Probability, 9(4): 483–496.
  • Tornøe, C. W., Overgaard, R. V., Agerso, H., Nielsen, H. A., Madsen, H., and Jonsson, E. N. (2005). “Stochastic differential equations in NONMEM: implementation, application, and comparison with ordinary differential equations.” Pharmaceutical Research, 22(8): 1247–1258.
  • Whitaker G. A., Golightly A., Boys R. J., and Sherlock C. (2016). Supplementary material for “Bayesian inference for diffusion-driven mixed-effects models”. Bayesian Analysis.

Supplemental materials