Electronic Journal of Statistics

Bayesian two-step estimation in differential equation models

Prithwish Bhaumik and Subhashis Ghosal

Full-text: Open access


Ordinary differential equations (ODEs) are used to model dynamic systems appearing in engineering, physics, biomedical sciences and many other fields. These equations contain an unknown vector of parameters of physical significance, say $\boldsymbol{\theta}$ which has to be estimated from the noisy data. Often there is no closed form analytic solution of the equations and hence we cannot use the usual non-linear least squares technique to estimate the unknown parameters. The two-step approach to solve this problem involves fitting the data nonparametrically and then estimating the parameter by minimizing the distance between the nonparametrically estimated derivative and the derivative suggested by the system of ODEs. The statistical aspects of this approach have been studied under the frequentist framework. We consider this two-step estimation under the Bayesian framework. The response variable is allowed to be multidimensional and the true mean function of it is not assumed to be in the model. We induce a prior on the regression function using a random series based on the B-spline basis functions. We establish the Bernstein-von Mises theorem for the posterior distribution of the parameter of interest. Interestingly, even though the posterior distribution of the regression function based on splines converges at a rate slower than $n^{-1/2}$, the parameter vector $\boldsymbol{\theta}$ is nevertheless estimated at $n^{-1/2}$ rate.

Article information

Electron. J. Statist., Volume 9, Number 2 (2015), 3124-3154.

Received: August 2014
First available in Project Euclid: 25 January 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J02: General nonlinear regression 62G08: Nonparametric regression 62G20: Asymptotic properties 62F15: Bayesian inference

Ordinary differential equation Bayesian inference spline smoothing Bernstein-von Mises theorem


Bhaumik, Prithwish; Ghosal, Subhashis. Bayesian two-step estimation in differential equation models. Electron. J. Statist. 9 (2015), no. 2, 3124--3154. doi:10.1214/15-EJS1099. https://projecteuclid.org/euclid.ejs/1453730083

Export citation


  • R. M. Anderson and R. M. May (1992)., Infectious Diseases of Humans: Dynamics and Control. Oxford University Press.
  • Y. Bard (1974)., Nonlinear Parameter Estimation. Academic Press New York.
  • D. Bontemps (2011). Bernstein-von mises theorems for gaussian regression with increasing number of regressors., The Annals of Statistics 39, 2557–2584.
  • N. J. Brunel (2008). Parameter estimation of ode’s via nonparametric estimators., Electronic Journal of Statistics 2, 1242–1267.
  • N. J. Brunel, Q. Clairon, and F. d’Alché Buc (2014). Parametric estimation of ordinary differential equations with orthogonality conditions., Journal of the American Statistical Association 109, 173–185.
  • D. Campbell and R. J. Steele (2012). Smooth functional tempering for nonlinear differential equation models., Statistics and Computing 22, 429–443.
  • D. A. Campbell (2007)., Bayesian Collocation Tempering and Generalized Profiling for Estimation of Parameters from Differential Equation Models. ProQuest.
  • T. Chen, H. L. He, G. M. Church, et al. (1999). Modeling gene expression with differential equations. In, Pacific Symposium on Biocomputing, Volume 4, pp. 4.
  • O. Chkrebtii, D. A. Campbell, M. A. Girolami, and B. Calderhead (2013). Bayesian uncertainty quantification for differential equations., arXiv preprint arXiv:1306.2365.
  • I. Dattner and S. Gugushvili (2015). Accelerated least squares estimation for systems of ordinary differential equations., arXiv preprint arXiv:1503.07973.
  • C. De Boor (1978)., A Practical Guide to Splines, Volume 27. Springer-Verlag, New York.
  • T. A. Dean and S. S. Singh (2011). Asymptotic behaviour of approximate bayesian estimators., arXiv preprint arXiv:1105.3655.
  • v. B. Domselaar and P. Hemker (1975). Nonlinear parameter estimation in initial value problems., Stichting Mathematisch Centrum. Numerieke Wiskunde, 1–49.
  • J. Gabrielsson and D. Weiner (2006)., Pharmacokinetic and Pharmacodynamic Data Analysis: Concepts and Applications. Swedish Pharmaceutical Press.
  • A. Gelman, F. Bois, and J. Jiang (1996). Physiological pharmacokinetic analysis using population modeling and informative prior distributions., Journal of the American Statistical Association 91, 1400–1412.
  • M. Girolami (2008). Bayesian inference for differential equations., Theoretical Computer Science 408, 4–16.
  • S. Gugushvili and C. A. Klaassen (2012). $\sqrtn$-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing, Bernoulli 18, 1061–1098.
  • E. Hairer, S. Nørsett, and G. Wanner (1993)., Solving Ordinary Differential Equations 1: Nonstiff Problems. Springer-Verlag, New York, Inc.
  • J. Henderson and G. Michailidis (2014). Network reconstruction using nonparametric additive ode models., PloS one 9(4), 94003.
  • J. Jaeger (2012)., Functional estimation in systems defined by differential equations using Bayesian smoothing methods. Ph. D. thesis, Université Catholique de Louvain.
  • B. Kleijn and A. van der Vaart (2012). The Bernstein-von Mises theorem under misspecification., Electronic Journal of Statistics 6, 354–381.
  • K. Levenberg (1944). A method for the solution of certain problems in least squares., Quarterly of Applied Mathematics 2, 164–168.
  • D. W. Marquardt (1963). An algorithm for least-squares estimation of nonlinear parameters., Journal of the Society for Industrial & Applied Mathematics 11, 431–441.
  • R. M. Mattheij and J. Molenaar (2002). Ordinary differential equations in theory and practice. Reprint of (1996) original., Classics in Applied Mathematics.
  • M. Nowak and R. M. May (2000)., Virus Dynamics: Mathematical Principles of Immunology and Virology. Oxford University Press.
  • X. Qi and H. Zhao (2010). Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations., The Annals of Statistics 38, 435–481.
  • J. O. Ramsay, G. Hooker, D. Campbell, and J. Cao (2007). Parameter estimation for differential equations: a generalized smoothing approach., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 69, 741–796.
  • S. Rogers, R. Khanin, and M. Girolami (2007). Bayesian model-based inference of transcription factor activity., BMC Bioinformatics 8(Suppl 2), S2.
  • W.-H. Steeb (2006)., Problems and Solutions in Introductory and Advanced Matrix Calculus. World Scientific.
  • J. Varah (1982). A spline least squares method for numerical parameter estimation in differential equations., SIAM Journal on Scientific and Statistical Computing 3, 28–46.
  • E. O. Voit and J. Almeida (2004). Decoupling dynamical systems for pathway identification from metabolic profiles., Bioinformatics 20, 1670–1681.
  • H. Wu, H. Xue, and A. Kumar (2012). Numerical discretization-based estimation methods for ordinary differential equation models via penalized spline smoothing with applications in biomedical research., Biometrics 68, 344–352.
  • H. Xue, H. Miao, and H. Wu (2010). Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error., The Annals of Statistics 38, 2351.
  • S. Zhou, X. Shen, and D. Wolfe (1998). Local asymptotics for regression splines and confidence regions., The Annals of Statistics 26, 1760–1782.
  • S. Zhou and D. A. Wolfe (2000). On derivative estimation in spline regression., Statistica Sinica 10, 93–108.