Bayesian Analysis

Bayesian Inference for Diffusion-Driven Mixed-Effects Models

Gavin A. Whitaker, Andrew Golightly, Richard J. Boys, and Chris Sherlock

Full-text: Open access

Enhanced PDF (735 KB)

Abstract

Stochastic differential equations (SDEs) provide a natural framework for modelling intrinsic stochasticity inherent in many continuous-time physical processes. When such processes are observed in multiple individuals or experimental units, SDE driven mixed-effects models allow the quantification of both between and within individual variation. Performing Bayesian inference for such models using discrete-time data that may be incomplete and subject to measurement error is a challenging problem and is the focus of this paper. We extend a recently proposed MCMC scheme to include the SDE driven mixed-effects framework. Fundamental to our approach is the development of a novel construct that allows for efficient sampling of conditioned SDEs that may exhibit nonlinear dynamics between observation times. We apply the resulting scheme to synthetic data generated from a simple SDE model of orange tree growth, and real data on aphid numbers recorded under a variety of different treatment regimes. In addition, we provide a systematic comparison of our approach with an inference scheme based on a tractable approximation of the SDE, that is, the linear noise approximation.

Article information

Source
Bayesian Anal. Volume 12, Number 2 (2017), 435-463.

Dates
First available in Project Euclid: 23 May 2016

Permanent link to this document
http://projecteuclid.org/euclid.ba/1464035697

Digital Object Identifier
doi:10.1214/16-BA1009

Keywords
stochastic differential equation mixed-effects Markov chain Monte Carlo modified innovation scheme linear noise approximation

Rights
Creative Commons Attribution 4.0 International License.

Citation

Whitaker, Gavin A.; Golightly, Andrew; Boys, Richard J.; Sherlock, Chris. Bayesian Inference for Diffusion-Driven Mixed-Effects Models. Bayesian Anal. 12 (2017), no. 2, 435--463. doi:10.1214/16-BA1009. http://projecteuclid.org/euclid.ba/1464035697.


Export citation

References

  • Aït-Sahalia, Y. (2002). “Maximum likelihood estimation of discretely sampled diffusions: a closed-form approximation approach.” Econometrica, 70(1): 223–262.
    Zentralblatt MATH: 1104.62323
    Digital Object Identifier: doi:10.1111/1468-0262.00274
  • Aït-Sahalia, Y. (2008). “Closed-form likelihood expansions for multivariate diffusions.” The Annals of Statistics, 36(2): 906–937.
    Zentralblatt MATH: 1246.62180
    Digital Object Identifier: doi:10.1214/009053607000000622
    Project Euclid: euclid.aos/1205420523
  • Berglund, M., Sunnåker, M., Adiels, M., Jirstrand, M., and Wennberg, B. (2011). “Investigations of a compartmental model for leucine kinetics using non-linear mixed effects models with ordinary and stochastic differential equations.” Mathematical Medicine and Biology, 29(4): 361–384.
    Zentralblatt MATH: 06115893
  • Beskos, A., Kalogeropoulos, K., and Pazos, E. (2013). “Advanced MCMC methods for sampling on diffusion pathspace.” Stochastic Processes and their Applications, 123(4): 1415–1453.
    Zentralblatt MATH: 1276.60078
    Digital Object Identifier: doi:10.1016/j.spa.2012.12.001
  • Beskos, A., Papaspiliopoulos, O., Roberts, G. O., and Fearnhead, P. (2006). “Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68: 333–382.
    Zentralblatt MATH: 1100.62079
    Digital Object Identifier: doi:10.1111/j.1467-9868.2006.00552.x
  • Chib, S., Pitt, M. K., and Shephard, N. (2004). “Likelihood based inference for diffusion driven models.” Technical Report 2004-W20, Nuffield College, Oxford.
  • Clark, J. (1990). “The simulation of pinned diffusions.” In Proceedings of the 29th IEEE Conference on Decision and Control, 1990, 1418–1420. IEEE.
  • Delyon, B. and Hu, Y. (2006). “Simulation of conditioned diffusion and application to parameter estimation.” Stochastic Processes and Their Applications, 116: 1660–1675.
    Zentralblatt MATH: 1107.60046
    Digital Object Identifier: doi:10.1016/j.spa.2006.04.004
  • Donnet, S., Foulley, J., and Samson, A. (2010). “Bayesian analysis of growth curves using mixed models defined by stochastic differential equations.” Biometrics, 66(3): 733–741.
    Mathematical Reviews (MathSciNet): MR2758209
    Zentralblatt MATH: 1203.62187
    Digital Object Identifier: doi:10.1111/j.1541-0420.2009.01342.x
  • Durham, G. B. and Gallant, A. R. (2002). “Numerical techniques for maximum likelihood estimation of continuous-time diffusion processes.” Journal of Business and Economic Statistics, 20: 297–338.
  • Elerian, O., Chib, S., and Shephard, N. (2001). “Likelihood inference for discretely observed nonlinear diffusions.” Econometrica, 69: 959–993.
    Zentralblatt MATH: 1017.62068
    Digital Object Identifier: doi:10.1111/1468-0262.00226
  • Eraker, B. (2001). “MCMC analysis of diffusion models with application to finance.” Journal of Business and Economic Statistics, 19(2): 177–191.
  • Fearnhead, P., Giagos, V., and Sherlock, C. (2014). “Inference for reaction networks using the linear noise approximation.” Biometrics, 70(2): 457–466.
    Zentralblatt MATH: 06319576
  • Ferm, L., Lötstedt, P., and Hellander, A. (2008). “A hierarchy of approximations of the master equation scaled by a size parameter.” Journal of Scientific Computing, 34(2): 127–151.
    Mathematical Reviews (MathSciNet): MR2373035
    Zentralblatt MATH: 1136.65007
    Digital Object Identifier: doi:10.1007/s10915-007-9179-z
  • Finkenstädt, B., Woodcock, D., Komorowski, M., Harper, C., Davis, J., White, M., and Rand, D. A. (2013). “Quantifying intrinsic and extrinsic noise in gene transcription using the linear noise approximation: an application to single cell data.” Annals of Applied Statistics, 7: 1960–1982.
  • Fuchs, C. (2013). Inference for diffusion processes with applications in Life Sciences. Heidelberg: Springer.
    Zentralblatt MATH: 1276.62051
  • Gillespie, C. S. and Golightly, A. (2010). “Bayesian inference for generalized stochastic population growth models with application to aphids.” Journal of the Royal Statistical Society – Series C (Applied Statistics), 59(2): 341–357.
  • Golightly, A. and Wilkinson, D. J. (2006). “Bayesian sequential inference for nonlinear multivariate diffusions.” Statistics and Computing, 16: 323–338.
  • Golightly, A. and Wilkinson, D. J. (2008). “Bayesian inference for nonlinear multivariate diffusion models observed with error.” Computational Statistics & Data Analysis, 52(3): 1674–1693.
    Zentralblatt MATH: 05564596
  • Golightly, A. and Wilkinson, D. J. (2010). “Markov chain Monte Carlo algorithms for SDE parameter estimation.” In Lawrence, N. D., Girolami, M., Rattray, M., and Sanguinetti, G. (eds.), Learning and inference in Computational Systems Biology, Computational Molecular Biology. MIT Press.
  • Golightly, A. and Wilkinson, D. J. (2011). “Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo.” Interface Focus, 1(6): 807–820.
  • Klim, S., Mortensen, S. B., Kristensen, N. R., Overgaard, R. V., and Madsen, H. (2009). “Population stochastic modelling (PSM) – An R package for mixed-effects models based on stochastic differential equations.” Computer Methods and Programs in Biomedicine, 94(3): 279–289.
  • Komorowski, M., Finkenstädt, B., Harper, C., and Rand, D. (2009). “Bayesian inference of biochemical kinetic parameters using the linear noise approximation.” BMC Bioinformatics, 10(1): 343.
  • Kou, S. C., Olding, B. P., Lysy, M., and Liu, J. S. (2012). “A multiresolution method for parameter estimation of diffusion processes.” Journal of the American Statistical Association, 107: 1558–1574.
    Mathematical Reviews (MathSciNet): MR3036416
    Zentralblatt MATH: 1258.62089
    Digital Object Identifier: doi:10.1080/01621459.2012.720899
  • Kurtz, T. G. (1970). “Solutions of ordinary differential equations as limits of pure jump Markov processes.” Journal of Applied Probability, 7(1): 49–58.
    Mathematical Reviews (MathSciNet): MR254917
    Zentralblatt MATH: 0191.47301
    Digital Object Identifier: doi:10.1017/S0021900200026929
  • Lindström, E. (2012). “A regularized bridge sampler for sparsely sampled diffusions.” Statistics and Computing, 22(1): 615–623.
  • Matis, J. H., Kiffe, T. R., Matis, T. I., and Stevenson, D. E. (2006). “Application of population growth models based on Cumulative size to Pecan Aphids.” Journal of Agricultural, Biological, and Environmental Statistics, 11: 425–449.
  • Matis, T. I., Parajulee, M. N., Matis, J. H., and Shrestha, R. B. (2008). “A mechanistic model based analysis of cotton aphid population dynamics data.” Agricultural and Forest Entomology, 10(4): 355–362.
  • Overgaard, R. V., Jonsson, N., Tornøe, C. W., and Madsen, H. (2005). “Non-linear mixed-effects models with stochastic differential equations: implementation of an estimation algorithm.” Journal of Pharmacokinetics and Pharmacodynamics, 32(1): 85–107.
  • Papaspiliopoulos, O. and Roberts, G. O. (2012). “Importance sampling techniques for estimation of diffusion models.” In Statistical Methods for Stochastic Differential Equations, Monographs on Statistics and Applied Probability, 311–337. Chapman and Hall.
    Zentralblatt MATH: 1272.65012
  • Papaspiliopoulos, O., Roberts, G. O., and Stramer, O. (2013). “Data augmentation for diffusions.” Journal of Computational and Graphical Statistics, 22: 665–688.
    Mathematical Reviews (MathSciNet): MR3173736
    Digital Object Identifier: doi:10.1080/10618600.2013.783484
  • Pedersen, A. R. (1995). “A new approach to maximum likelihood estimation for stochastic differential equations based on discrete observations.” Scandinavian Journal of Statistics, 22(1): 55–71.
    Zentralblatt MATH: 0827.62087
  • Picchini, U. and Ditlevsen, S. (2011). “Practical estimation of high dimensional stochastic differential mixed-effects models.” Computational Statistics and Data Analysis, 55(3): 1426–1444.
    Zentralblatt MATH: 1328.65014
    Digital Object Identifier: doi:10.1016/j.csda.2010.10.003
  • Picchini, U., Gaetano, S., and Ditlevsen, S. (2010). “Stochastic differential mixed-effects models.” Scandinavian Journal of Statistics, 37: 67–90.
    Mathematical Reviews (MathSciNet): MR2675946
    Zentralblatt MATH: 1192.62179
    Digital Object Identifier: doi:10.1111/j.1467-9469.2009.00665.x
  • Roberts, G. O. and Stramer, O. (2001). “On inference for partially observed nonlinear diffusion models using the Metropolis–Hastings algorithm.” Biometrika, 88(3): 603–621.
    Zentralblatt MATH: 0985.62066
    Digital Object Identifier: doi:10.1093/biomet/88.3.603
  • Schauer, M., van der Meulen, F., and van Zanten, H. (2014). “Guided proposals for simulating multi-dimensional diffusion bridges.” Available from http://arxiv.org/abs/1311.3606.
  • Sermaidis, G., Papaspiliopoulos, O., Roberts, G. O., Beskos, A., and Fearnhead, P. (2013). “Markov chain Monte Carlo for exact inference for diffusions.” Scandinavian Journal of Statistics, 40: 294–321.
    Zentralblatt MATH: 1328.65015
    Digital Object Identifier: doi:10.1111/j.1467-9469.2012.00812.x
  • Stramer, O. and Bognar, M. (2011). “Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach.” Bayesian Analysis, 6(2): 231–258.
    Zentralblatt MATH: 1330.60092
    Digital Object Identifier: doi:10.1214/11-BA608
  • Stramer, O., Bognar, M., and Scheider, P. (2010). “Bayesian inference for discretely sampled Markov processes with closed-form likelihood expansions.” Journal of Financial Econometrics, 8: 450–480.
  • Stramer, O. and Yan, J. (2007). “Asymptotics of an efficient Monte Carlo estimation for the transition density of diffusion processes.” Methodology and Computing in Applied Probability, 9(4): 483–496.
    Zentralblatt MATH: 1134.60369
    Digital Object Identifier: doi:10.1007/s11009-006-9006-2
  • Tornøe, C. W., Overgaard, R. V., Agerso, H., Nielsen, H. A., Madsen, H., and Jonsson, E. N. (2005). “Stochastic differential equations in NONMEM: implementation, application, and comparison with ordinary differential equations.” Pharmaceutical Research, 22(8): 1247–1258.
  • Whitaker G. A., Golightly A., Boys R. J., and Sherlock C. (2016). Supplementary material for “Bayesian inference for diffusion-driven mixed-effects models”. Bayesian Analysis.

Supplemental materials

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more