Open Access
2015 A tracking approach to parameter estimation in linear ordinary differential equations
Nicolas J. B. Brunel, Quentin Clairon
Electron. J. Statist. 9(2): 2903-2949 (2015). DOI: 10.1214/15-EJS1086
Abstract

Ordinary Differential Equations are widespread tools to model chemical, physical, biological process but they usually rely on parameters which are of critical importance in terms of dynamic and need to be estimated directly from the data. Classical statistical approaches (nonlinear least squares, maximum likelihood estimator) can give unsatisfactory results because of computational difficulties and ill-posed statistical problem. New estimation methods that use some nonparametric devices have been proposed to circumvent these issues. We present a new estimator that shares properties with Two-Step estimators and Generalized Smoothing (introduced by Ramsay et al. [37]). Our estimation method relies on a relaxation and penalization scheme to regularize the inverse problem. We introduce a perturbed model and we use optimal control theory for constructing a criterion that aims at minimizing the discrepancy between data and the original model. Here, we focus on the case of linear Ordinary Differential Equations as our criterion has a closed-form expression that permits a detailed analysis. Our approach avoids the use of a nonparametric estimator of the derivative, which is one of the main causes of inaccuracy in Two-Step estimators. Regarding the theoretical asymptotic behavior of our estimator, we show its consistency and that we reach the parametric $\sqrt{n}$-rate when regression splines are used in the first step. We consider the estimation of two models possessing sloppy parameters, which usually makes the estimation of ODE models an ill-posed problem in applications [20, 41] and shows the efficiency of the Tracking estimator. Quite interestingly, our relaxation scheme makes the estimator robust to some kind of model misspecification, as shown in simulations.

References

1.

[1] R. Bellman and K.J. Astrom. On structural identifiability., Mathematical Biosciences, 7:329–339, 1970.[1] R. Bellman and K.J. Astrom. On structural identifiability., Mathematical Biosciences, 7:329–339, 1970.

2.

[2] E. Blayo, E. Cosme, M. Nodet, and A. Vidart. Introduction to data assimilation., 2011.[2] E. Blayo, E. Cosme, M. Nodet, and A. Vidart. Introduction to data assimilation., 2011.

3.

[3] C. De Boor., A practical guide to Splines, volume 27 of Applied Mathematical Sciences. Spinger, 2001.[3] C. De Boor., A practical guide to Splines, volume 27 of Applied Mathematical Sciences. Spinger, 2001.

4.

[4] H. Brezis., Functional Analysis. Dunod, 1983. MR697382[4] H. Brezis., Functional Analysis. Dunod, 1983. MR697382

5.

[5] N. J.-B. Brunel. Parameter estimation of ode’s via nonparametric estimators., Electronic Journal of Statistics, 2 :1242–1267, 2008. MR2471285 06165733 10.1214/07-EJS132 euclid.ejs/1229975381 [5] N. J.-B. Brunel. Parameter estimation of ode’s via nonparametric estimators., Electronic Journal of Statistics, 2 :1242–1267, 2008. MR2471285 06165733 10.1214/07-EJS132 euclid.ejs/1229975381

6.

[6] N. J.-B. Brunel, Q. Clairon, and F. D’Alche-Buc. Parameter estimation of ordinary differential equations with orthogonality conditions., JASA, 109(205):173–185, 2014. MR3180555 10.1080/01621459.2013.841583[6] N. J.-B. Brunel, Q. Clairon, and F. D’Alche-Buc. Parameter estimation of ordinary differential equations with orthogonality conditions., JASA, 109(205):173–185, 2014. MR3180555 10.1080/01621459.2013.841583

7.

[7] B. Calderhead and M. Girolami. Estimating Bayes factors via thermodynamic integration and population MCMC., Computational Statistics & Data Analysis, 53(12) :4028–4045, October 2009. MR2744303[7] B. Calderhead and M. Girolami. Estimating Bayes factors via thermodynamic integration and population MCMC., Computational Statistics & Data Analysis, 53(12) :4028–4045, October 2009. MR2744303

8.

[8] D.A. Campbell and O. Chkrebtii. Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates., Mathematical Biosciences, 2013. MR3132050 1283.92025 10.1016/j.mbs.2013.03.011[8] D.A. Campbell and O. Chkrebtii. Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates., Mathematical Biosciences, 2013. MR3132050 1283.92025 10.1016/j.mbs.2013.03.011

9.

[9] F. Clarke., Functional Analysis, Calculus of Variations and Optimal Control. Graduate Texts in Mathematics. Springer-Verlag London, 2013. MR3026831 1277.49001[9] F. Clarke., Functional Analysis, Calculus of Variations and Optimal Control. Graduate Texts in Mathematics. Springer-Verlag London, 2013. MR3026831 1277.49001

10.

[10] G. Hooker, D.A. Campbell, and K.B. McAuley. Parameter estimation in differential equation models with constrained states., Journal of Chemometrics, 26:322–332, 2011.[10] G. Hooker, D.A. Campbell, and K.B. McAuley. Parameter estimation in differential equation models with constrained states., Journal of Chemometrics, 26:322–332, 2011.

11.

[11] H.W. Engl, C. Flamm, P. Kügler, J. Lu, S. Müller, and P. Schuster. Inverse problems in systems biology., Inverse Problems, 25(12), 2009.[11] H.W. Engl, C. Flamm, P. Kügler, J. Lu, S. Müller, and P. Schuster. Inverse problems in systems biology., Inverse Problems, 25(12), 2009.

12.

[12] J. Brynjarsdottir and A. O’Hagan. Learning about physical parameters: The importance of model discrepancy., Inverse Problems, 30:24, 2014. MR3274591[12] J. Brynjarsdottir and A. O’Hagan. Learning about physical parameters: The importance of model discrepancy., Inverse Problems, 30:24, 2014. MR3274591

13.

[13] C.P. Fall, E.S. Marland, J.M. Wagner, and J.J. Tyson, editors., Computational Cell Biology. Interdisciplinary applied mathematics. Springer, 2002. MR1911592[13] C.P. Fall, E.S. Marland, J.M. Wagner, and J.J. Tyson, editors., Computational Cell Biology. Interdisciplinary applied mathematics. Springer, 2002. MR1911592

14.

[14] R.E. Fuguitt and J.E. Hawkins. Rate of Thermal Isomerization of a-Pinene in the Liquid Phase., J.A.C.S, 319(39), 1947.[14] R.E. Fuguitt and J.E. Hawkins. Rate of Thermal Isomerization of a-Pinene in the Liquid Phase., J.A.C.S, 319(39), 1947.

15.

[15] A. Gelman, F. Bois, and J. Jiang. Physiological pharmacokinetic analysis using population modeling and informative prior distributions., Journal of the American Statistical Association, 91, 1996.[15] A. Gelman, F. Bois, and J. Jiang. Physiological pharmacokinetic analysis using population modeling and informative prior distributions., Journal of the American Statistical Association, 91, 1996.

16.

[16] O. Ghasemi, M. Lindsey, T. Yang, N. Nguyen, Y. Huang, and Y. Jin. Bayesian parameter estimation for nonlinear modelling of biological pathways., BMC Systems Biology, 5, 2011.[16] O. Ghasemi, M. Lindsey, T. Yang, N. Nguyen, Y. Huang, and Y. Jin. Bayesian parameter estimation for nonlinear modelling of biological pathways., BMC Systems Biology, 5, 2011.

17.

[17] M. Girolami and B. Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. volume 73, pages 1–37, 2011. MR2814492 10.1111/j.1467-9868.2010.00765.x[17] M. Girolami and B. Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. volume 73, pages 1–37, 2011. MR2814492 10.1111/j.1467-9868.2010.00765.x

18.

[18] A. Goldbeter., Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour. Cambridge University Press, 1997.[18] A. Goldbeter., Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour. Cambridge University Press, 1997.

19.

[19] S. Gugushvili and C.A.J. Klaassen. Root-n-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing., Bernoulli, to appear, 2011. MR2948913 10.3150/11-BEJ362 euclid.bj/1340887014 [19] S. Gugushvili and C.A.J. Klaassen. Root-n-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing., Bernoulli, to appear, 2011. MR2948913 10.3150/11-BEJ362 euclid.bj/1340887014

20.

[20] J.J. Waterfall, F.P. Casey, K.S. Brown, C.R. Myers, R.N. Gutenkunst, and J.P. Sethna. Universally sloppy parameter sensitivities in systems biology models., PLoS Computational Biology, 3:e189, 2007. MR2369325 10.1371/journal.pcbi.0030189[20] J.J. Waterfall, F.P. Casey, K.S. Brown, C.R. Myers, R.N. Gutenkunst, and J.P. Sethna. Universally sloppy parameter sensitivities in systems biology models., PLoS Computational Biology, 3:e189, 2007. MR2369325 10.1371/journal.pcbi.0030189

21.

[21] J. Happel, I. Suzuki, P. Kokayeff, and V. Fthenakis. Multiple isotope tracing of methanation over nickel catalyst., Journal of Catalysis, 65:59–77, 1980.[21] J. Happel, I. Suzuki, P. Kokayeff, and V. Fthenakis. Multiple isotope tracing of methanation over nickel catalyst., Journal of Catalysis, 65:59–77, 1980.

22.

[22] G. Hooker. Forcing function diagnostics for nonlinear dynamics., Biometrics, 65:928–936, 2009. MR2649866 10.1111/j.1541-0420.2008.01172.x[22] G. Hooker. Forcing function diagnostics for nonlinear dynamics., Biometrics, 65:928–936, 2009. MR2649866 10.1111/j.1541-0420.2008.01172.x

23.

[23] G. Hooker and S. Ellner. Goodness of fit in nonlinear dynamics: Mis-specified rates or mis-specified states? Technical report, Cornell University, 2013., arXiv:1312.0294.[23] G. Hooker and S. Ellner. Goodness of fit in nonlinear dynamics: Mis-specified rates or mis-specified states? Technical report, Cornell University, 2013., arXiv:1312.0294.

24.

[24] G. Hooker and S.P Ellner. Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?, Annals of Applied Statistics, 9(2):754–776, 2015. MR3371334 10.1214/15-AOAS828 euclid.aoas/1437397110 [24] G. Hooker and S.P Ellner. Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?, Annals of Applied Statistics, 9(2):754–776, 2015. MR3371334 10.1214/15-AOAS828 euclid.aoas/1437397110

25.

[25] Y. Huang and H. Wu. A bayesian approach for estimating antiviral efficacy in hiv dynamic models., Journal of Applied Statistics, 33:155–174, 2006. MR2223142 1106.62121 10.1080/02664760500250552[25] Y. Huang and H. Wu. A bayesian approach for estimating antiviral efficacy in hiv dynamic models., Journal of Applied Statistics, 33:155–174, 2006. MR2223142 1106.62121 10.1080/02664760500250552

26.

[26] B. Hipszer, T.V. Apanasovich I. Chervoneva, B. Freydin, and J.I. Joseph. Estimation of nonlinear differential equation model for glucose-insulin dynamics in type i diabetic patients using generalized smoothing., Annals of Applied Statistics(submitted), 2014. MR3262538 06333780 10.1214/13-AOAS706 euclid.aoas/1404229518 [26] B. Hipszer, T.V. Apanasovich I. Chervoneva, B. Freydin, and J.I. Joseph. Estimation of nonlinear differential equation model for glucose-insulin dynamics in type i diabetic patients using generalized smoothing., Annals of Applied Statistics(submitted), 2014. MR3262538 06333780 10.1214/13-AOAS706 euclid.aoas/1404229518

27.

[27] D.E. Kirk., Optimal Control Theory: An Introduction. Dover Publication, 1998.[27] D.E. Kirk., Optimal Control Theory: An Introduction. Dover Publication, 1998.

28.

[28] H.L. Koul. Weighted empiricals and linear models., Hayward, CA: Institute of Mathematical Statistics, 21:105–175, 1992. MR1218395 0998.62501[28] H.L. Koul. Weighted empiricals and linear models., Hayward, CA: Institute of Mathematical Statistics, 21:105–175, 1992. MR1218395 0998.62501

29.

[29] R.V. Gamkrelidze, L.S. Pontryagin, V.G. Boltyanskii, and E.F. Mischenko., The Mathematical Theory of Optimal Processes. Wiley-Interscience, 1962. MR166037[29] R.V. Gamkrelidze, L.S. Pontryagin, V.G. Boltyanskii, and E.F. Mischenko., The Mathematical Theory of Optimal Processes. Wiley-Interscience, 1962. MR166037

30.

[30] Z. Li, M.R. Osborne, and T. Prvan. Parameter estimation of ordinary differential equations., IMA Journal of Numerical Analysis, 25:264–285, 2005. MR2126204 1070.65061 10.1093/imanum/drh016[30] Z. Li, M.R. Osborne, and T. Prvan. Parameter estimation of ordinary differential equations., IMA Journal of Numerical Analysis, 25:264–285, 2005. MR2126204 1070.65061 10.1093/imanum/drh016

31.

[31] H. Liang and H. Wu. Parameter estimation for differential equation models using a framework of measurement error in regression models., Journal of the American Statistical Association, 103(484) :1570–1583, December 2008. MR2504205 1286.62039 10.1198/016214508000000797[31] H. Liang and H. Wu. Parameter estimation for differential equation models using a framework of measurement error in regression models., Journal of the American Statistical Association, 103(484) :1570–1583, December 2008. MR2504205 1286.62039 10.1198/016214508000000797

32.

[32] H. Miao, X. Xia, A.S. Perelson, and H. Wu. On identifiability of nonlinear ode models and applications in viral dynamics., SIAM Review, 53:3–39, 2011. MR2785878 1215.34015 10.1137/090757009[32] H. Miao, X. Xia, A.S. Perelson, and H. Wu. On identifiability of nonlinear ode models and applications in viral dynamics., SIAM Review, 53:3–39, 2011. MR2785878 1215.34015 10.1137/090757009

33.

[33] A.A. Milyutin and N.P. Osmolovskii., Calculus of Variation and Optimal control. Mathematical Monographs. American mathematical society, 1998. MR1641590[33] A.A. Milyutin and N.P. Osmolovskii., Calculus of Variation and Optimal control. Mathematical Monographs. American mathematical society, 1998. MR1641590

34.

[34] H.P. Mirsky, A.C. Liu, D.K. Welsh, S.A. Kay, and F.J. Doyle III. A model of the cell-autonomous mammalian circadian clock., PNAS, 106(27) :11107–11112, July 2009.[34] H.P. Mirsky, A.C. Liu, D.K. Welsh, S.A. Kay, and F.J. Doyle III. A model of the cell-autonomous mammalian circadian clock., PNAS, 106(27) :11107–11112, July 2009.

35.

[35] W.K. Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79:147–168, 1997. MR1457700 0873.62049 10.1016/S0304-4076(97)00011-0[35] W.K. Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79:147–168, 1997. MR1457700 0873.62049 10.1016/S0304-4076(97)00011-0

36.

[36] Xin Qi and Hongyu Zhao. Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations., The Annals of Statistics, 1:435–481, 2010. MR2589327 1181.62156 10.1214/09-AOS724 euclid.aos/1262271620 [36] Xin Qi and Hongyu Zhao. Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations., The Annals of Statistics, 1:435–481, 2010. MR2589327 1181.62156 10.1214/09-AOS724 euclid.aos/1262271620

37.

[37] J.O. Ramsay, G. Hooker, J. Cao, and D. Campbell. Parameter estimation for differential equations: A generalized smoothing approach., Journal of the Royal Statistical Society (B), 69:741–796, 2007. MR2368570 10.1111/j.1467-9868.2007.00610.x[37] J.O. Ramsay, G. Hooker, J. Cao, and D. Campbell. Parameter estimation for differential equations: A generalized smoothing approach., Journal of the Royal Statistical Society (B), 69:741–796, 2007. MR2368570 10.1111/j.1467-9868.2007.00610.x

38.

[38] M. Rodriguez-Fernandez, J.A. Egea, and J.R. Banga. Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems., BioMed Central, 2006.[38] M. Rodriguez-Fernandez, J.A. Egea, and J.R. Banga. Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems., BioMed Central, 2006.

39.

[39] D. Ruppert, M.P. Wand, and R.J. Carroll., Semiparametric regression. Cambridge series on statistical and probabilistic mathematics. Cambridge University Press, 2003. MR1998720[39] D. Ruppert, M.P. Wand, and R.J. Carroll., Semiparametric regression. Cambridge series on statistical and probabilistic mathematics. Cambridge University Press, 2003. MR1998720

40.

[40] E. Sontag., Mathematical Control Theory: Deterministic finite-dimensional systems. Springer-Verlag (New-York), 1998. MR1640001[40] E. Sontag., Mathematical Control Theory: Deterministic finite-dimensional systems. Springer-Verlag (New-York), 1998. MR1640001

41.

[41] C. Tonsing, J. Timmer, and C. Kreutz. Cause and cure of sloppiness in ordinary differential equation models., Physical Review, 90 :023303, 2014.[41] C. Tonsing, J. Timmer, and C. Kreutz. Cause and cure of sloppiness in ordinary differential equation models., Physical Review, 90 :023303, 2014.

42.

[42] R. Tuo and C.F.J. Wu. Efficient calibration for imperfect computer models., Annals of Statistics, 2015. MR3405596 10.1214/15-AOS1314 euclid.aos/1444222077 [42] R. Tuo and C.F.J. Wu. Efficient calibration for imperfect computer models., Annals of Statistics, 2015. MR3405596 10.1214/15-AOS1314 euclid.aos/1444222077

43.

[43] A.W. van der Vaart., Asymptotic Statistics. Cambridge Series in Statistical and Probabilities Mathematics. Cambridge University Press, 1998. MR1652247 0910.62001[43] A.W. van der Vaart., Asymptotic Statistics. Cambridge Series in Statistical and Probabilities Mathematics. Cambridge University Press, 1998. MR1652247 0910.62001

44.

[44] J.M. Varah. A spline least squares method for numerical parameter estimation in differential equations., SIAM J.sci. Stat. Comput., 3(1):28–46, 1982. MR651865 10.1137/0903003[44] J.M. Varah. A spline least squares method for numerical parameter estimation in differential equations., SIAM J.sci. Stat. Comput., 3(1):28–46, 1982. MR651865 10.1137/0903003
Copyright © 2015 The Institute of Mathematical Statistics and the Bernoulli Society
Nicolas J. B. Brunel and Quentin Clairon "A tracking approach to parameter estimation in linear ordinary differential equations," Electronic Journal of Statistics 9(2), 2903-2949, (2015). https://doi.org/10.1214/15-EJS1086
Received: 1 September 2014; Published: 2015
Vol.9 • No. 2 • 2015
Back to Top