Electronic Journal of Statistics

A tracking approach to parameter estimation in linear ordinary differential equations

Nicolas J. B. Brunel and Quentin Clairon

Full-text: Open access


Ordinary Differential Equations are widespread tools to model chemical, physical, biological process but they usually rely on parameters which are of critical importance in terms of dynamic and need to be estimated directly from the data. Classical statistical approaches (nonlinear least squares, maximum likelihood estimator) can give unsatisfactory results because of computational difficulties and ill-posed statistical problem. New estimation methods that use some nonparametric devices have been proposed to circumvent these issues. We present a new estimator that shares properties with Two-Step estimators and Generalized Smoothing (introduced by Ramsay et al. [37]). Our estimation method relies on a relaxation and penalization scheme to regularize the inverse problem. We introduce a perturbed model and we use optimal control theory for constructing a criterion that aims at minimizing the discrepancy between data and the original model. Here, we focus on the case of linear Ordinary Differential Equations as our criterion has a closed-form expression that permits a detailed analysis. Our approach avoids the use of a nonparametric estimator of the derivative, which is one of the main causes of inaccuracy in Two-Step estimators. Regarding the theoretical asymptotic behavior of our estimator, we show its consistency and that we reach the parametric $\sqrt{n}$-rate when regression splines are used in the first step. We consider the estimation of two models possessing sloppy parameters, which usually makes the estimation of ODE models an ill-posed problem in applications [20, 41] and shows the efficiency of the Tracking estimator. Quite interestingly, our relaxation scheme makes the estimator robust to some kind of model misspecification, as shown in simulations.

Article information

Electron. J. Statist., Volume 9, Number 2 (2015), 2903-2949.

Received: September 2014
First available in Project Euclid: 5 January 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Parameter estimation ordinary differential equations optimal control Riccati differential equation smoothing plug-in property asymptotic statistics


Brunel, Nicolas J. B.; Clairon, Quentin. A tracking approach to parameter estimation in linear ordinary differential equations. Electron. J. Statist. 9 (2015), no. 2, 2903--2949. doi:10.1214/15-EJS1086. https://projecteuclid.org/euclid.ejs/1452004955

Export citation


  • [1] R. Bellman and K.J. Astrom. On structural identifiability., Mathematical Biosciences, 7:329–339, 1970.
  • [2] E. Blayo, E. Cosme, M. Nodet, and A. Vidart. Introduction to data assimilation., 2011.
  • [3] C. De Boor., A practical guide to Splines, volume 27 of Applied Mathematical Sciences. Spinger, 2001.
  • [4] H. Brezis., Functional Analysis. Dunod, 1983.
  • [5] N. J.-B. Brunel. Parameter estimation of ode’s via nonparametric estimators., Electronic Journal of Statistics, 2 :1242–1267, 2008.
  • [6] N. J.-B. Brunel, Q. Clairon, and F. D’Alche-Buc. Parameter estimation of ordinary differential equations with orthogonality conditions., JASA, 109(205):173–185, 2014.
  • [7] B. Calderhead and M. Girolami. Estimating Bayes factors via thermodynamic integration and population MCMC., Computational Statistics & Data Analysis, 53(12) :4028–4045, October 2009.
  • [8] D.A. Campbell and O. Chkrebtii. Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates., Mathematical Biosciences, 2013.
  • [9] F. Clarke., Functional Analysis, Calculus of Variations and Optimal Control. Graduate Texts in Mathematics. Springer-Verlag London, 2013.
  • [10] G. Hooker, D.A. Campbell, and K.B. McAuley. Parameter estimation in differential equation models with constrained states., Journal of Chemometrics, 26:322–332, 2011.
  • [11] H.W. Engl, C. Flamm, P. Kügler, J. Lu, S. Müller, and P. Schuster. Inverse problems in systems biology., Inverse Problems, 25(12), 2009.
  • [12] J. Brynjarsdottir and A. O’Hagan. Learning about physical parameters: The importance of model discrepancy., Inverse Problems, 30:24, 2014.
  • [13] C.P. Fall, E.S. Marland, J.M. Wagner, and J.J. Tyson, editors., Computational Cell Biology. Interdisciplinary applied mathematics. Springer, 2002.
  • [14] R.E. Fuguitt and J.E. Hawkins. Rate of Thermal Isomerization of a-Pinene in the Liquid Phase., J.A.C.S, 319(39), 1947.
  • [15] A. Gelman, F. Bois, and J. Jiang. Physiological pharmacokinetic analysis using population modeling and informative prior distributions., Journal of the American Statistical Association, 91, 1996.
  • [16] O. Ghasemi, M. Lindsey, T. Yang, N. Nguyen, Y. Huang, and Y. Jin. Bayesian parameter estimation for nonlinear modelling of biological pathways., BMC Systems Biology, 5, 2011.
  • [17] M. Girolami and B. Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. volume 73, pages 1–37, 2011.
  • [18] A. Goldbeter., Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour. Cambridge University Press, 1997.
  • [19] S. Gugushvili and C.A.J. Klaassen. Root-n-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing., Bernoulli, to appear, 2011.
  • [20] J.J. Waterfall, F.P. Casey, K.S. Brown, C.R. Myers, R.N. Gutenkunst, and J.P. Sethna. Universally sloppy parameter sensitivities in systems biology models., PLoS Computational Biology, 3:e189, 2007.
  • [21] J. Happel, I. Suzuki, P. Kokayeff, and V. Fthenakis. Multiple isotope tracing of methanation over nickel catalyst., Journal of Catalysis, 65:59–77, 1980.
  • [22] G. Hooker. Forcing function diagnostics for nonlinear dynamics., Biometrics, 65:928–936, 2009.
  • [23] G. Hooker and S. Ellner. Goodness of fit in nonlinear dynamics: Mis-specified rates or mis-specified states? Technical report, Cornell University, 2013., arXiv:1312.0294.
  • [24] G. Hooker and S.P Ellner. Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?, Annals of Applied Statistics, 9(2):754–776, 2015.
  • [25] Y. Huang and H. Wu. A bayesian approach for estimating antiviral efficacy in hiv dynamic models., Journal of Applied Statistics, 33:155–174, 2006.
  • [26] B. Hipszer, T.V. Apanasovich I. Chervoneva, B. Freydin, and J.I. Joseph. Estimation of nonlinear differential equation model for glucose-insulin dynamics in type i diabetic patients using generalized smoothing., Annals of Applied Statistics(submitted), 2014.
  • [27] D.E. Kirk., Optimal Control Theory: An Introduction. Dover Publication, 1998.
  • [28] H.L. Koul. Weighted empiricals and linear models., Hayward, CA: Institute of Mathematical Statistics, 21:105–175, 1992.
  • [29] R.V. Gamkrelidze, L.S. Pontryagin, V.G. Boltyanskii, and E.F. Mischenko., The Mathematical Theory of Optimal Processes. Wiley-Interscience, 1962.
  • [30] Z. Li, M.R. Osborne, and T. Prvan. Parameter estimation of ordinary differential equations., IMA Journal of Numerical Analysis, 25:264–285, 2005.
  • [31] H. Liang and H. Wu. Parameter estimation for differential equation models using a framework of measurement error in regression models., Journal of the American Statistical Association, 103(484) :1570–1583, December 2008.
  • [32] H. Miao, X. Xia, A.S. Perelson, and H. Wu. On identifiability of nonlinear ode models and applications in viral dynamics., SIAM Review, 53:3–39, 2011.
  • [33] A.A. Milyutin and N.P. Osmolovskii., Calculus of Variation and Optimal control. Mathematical Monographs. American mathematical society, 1998.
  • [34] H.P. Mirsky, A.C. Liu, D.K. Welsh, S.A. Kay, and F.J. Doyle III. A model of the cell-autonomous mammalian circadian clock., PNAS, 106(27) :11107–11112, July 2009.
  • [35] W.K. Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79:147–168, 1997.
  • [36] Xin Qi and Hongyu Zhao. Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations., The Annals of Statistics, 1:435–481, 2010.
  • [37] J.O. Ramsay, G. Hooker, J. Cao, and D. Campbell. Parameter estimation for differential equations: A generalized smoothing approach., Journal of the Royal Statistical Society (B), 69:741–796, 2007.
  • [38] M. Rodriguez-Fernandez, J.A. Egea, and J.R. Banga. Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems., BioMed Central, 2006.
  • [39] D. Ruppert, M.P. Wand, and R.J. Carroll., Semiparametric regression. Cambridge series on statistical and probabilistic mathematics. Cambridge University Press, 2003.
  • [40] E. Sontag., Mathematical Control Theory: Deterministic finite-dimensional systems. Springer-Verlag (New-York), 1998.
  • [41] C. Tonsing, J. Timmer, and C. Kreutz. Cause and cure of sloppiness in ordinary differential equation models., Physical Review, 90 :023303, 2014.
  • [42] R. Tuo and C.F.J. Wu. Efficient calibration for imperfect computer models., Annals of Statistics, 2015.
  • [43] A.W. van der Vaart., Asymptotic Statistics. Cambridge Series in Statistical and Probabilities Mathematics. Cambridge University Press, 1998.
  • [44] J.M. Varah. A spline least squares method for numerical parameter estimation in differential equations., SIAM J.sci. Stat. Comput., 3(1):28–46, 1982.