The Annals of Statistics

Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error

Hongqi Xue, Hongyu Miao, and Hulin Wu

Full-text: Open access


This article considers estimation of constant and time-varying coefficients in nonlinear ordinary differential equation (ODE) models where analytic closed-form solutions are not available. The numerical solution-based nonlinear least squares (NLS) estimator is investigated in this study. A numerical algorithm such as the Runge–Kutta method is used to approximate the ODE solution. The asymptotic properties are established for the proposed estimators considering both numerical error and measurement error. The B-spline is used to approximate the time-varying coefficients, and the corresponding asymptotic theories in this case are investigated under the framework of the sieve approach. Our results show that if the maximum step size of the p-order numerical algorithm goes to zero at a rate faster than n−1/(p∧4), the numerical error is negligible compared to the measurement error. This result provides a theoretical guidance in selection of the step size for numerical evaluations of ODEs. Moreover, we have shown that the numerical solution-based NLS estimator and the sieve NLS estimator are strongly consistent. The sieve estimator of constant parameters is asymptotically normal with the same asymptotic co-variance as that of the case where the true ODE solution is exactly known, while the estimator of the time-varying parameter has the optimal convergence rate under some regularity conditions. The theoretical results are also developed for the case when the step size of the ODE numerical solver does not go to zero fast enough or the numerical error is comparable to the measurement error. We illustrate our approach with both simulation studies and clinical data on HIV viral dynamics.

Article information

Ann. Statist., Volume 38, Number 4 (2010), 2351-2387.

First available in Project Euclid: 11 July 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G02 60G20: Generalized stochastic processes
Secondary: 60G08 62P10: Applications to biology and medical sciences

Nonlinear least squares ordinary differential equation Runge–Kutta algorithm sieve approach spline smoothing time-varying parameter


Xue, Hongqi; Miao, Hongyu; Wu, Hulin. Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. Ann. Statist. 38 (2010), no. 4, 2351--2387. doi:10.1214/09-AOS784.

Export citation


  • Adams, B. M. (2005). Non-parametric parameter estimation and clinical data fitting with a model of HIV infection. Ph.D. thesis, North Carolina State Univ.
  • Anderson, R. M. and May, R. M. (1991). Infectious Diseases of Humans: Dynamics and Control. Oxford Univ. Press, Oxford.
  • Audoly, S., Bellu, G., D’Angio, L., Saccomani, M. P. and Cobelli, C. (2001). Global identifiability of nonlinear models of biological systems. IEEE Trans. Biomed. Eng. 48 55–65.
  • Babenko, V. F., Kofanov, V. A. and Pichugov, S. A. (1996). Multivariate inequalities of Kolmogorov type and their applications. In Multivariate Approximation and Splines (G. Nuraberger, J. W. Schmidt and G. Walz, eds.) 1–12. Birkhauser, Basel.
  • Bard, Y. (1974). Nonlinear Parameter Estimation. Academic Press, New York.
  • Bellman, R. and Åström, K. J. (1970). On structural identifiability. Math. Biosci. 7 329–339.
  • Benson, M. (1979). Parameter fitting in dynamic models. Ecol. Mod. 6 97–115.
  • Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Univ. Press, Baltimore.
  • Brookmeyer, R. and Gail, M. H. (1994). AIDS Epidemiology: A Quantitative Approach. Monographs in Epidemiology and Biostatistics 23. Oxford Univ. Press, New York.
  • Brunel, N. (2008). Parameter estimation of ODE’s via nonparametric estimators. Electron. J. Statist. 2 1242–1267.
  • Burnham, K. P. and Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociol. Methods Res. 33 261.
  • Chappel, M. J. and Godfrey, K. R. (1992). Structural identifiability of the parameters of a nonlinear batch reactor model. Math. Biosci. 108 245–251.
  • Chen, J. and Wu, H. (2008). Efficient local estimation for time-varying coefficients in deterministic dynamic models with applications to HIV-1 dynamics. J. Amer. Statist. Assoc. 103 369–384.
  • Chen, T., He, H. L. and Church, G. M. (1999). Modeling gene expression with differential equations. Pac. Symp. Biocomput. 29–40.
  • Cobelli, C., Lepschy, A. and Jacur, R. (1979). Identifiability of compartmental systems and related structural proerties. Math. Biosci. 44 1–18.
  • Daley, D. J. and Gani, J. (1999). Epidemic Modeling. Cambridge Univ. Press, Cambridge.
  • de Boor, C. (1978). A Practical Guide to Splines. Springer, New York.
  • Delgado, M. A. (1992). Semiparametric generalized least squares in the multivariate nonlinear regression model. Econometric Theory 8 203–222.
  • Donnet, S. and Samson, A. (2007). Estimation of parameters in incomplete data models defined by dynamical systems. J. Statist. Plann. Inference 137 2815–2831.
  • Englezos, P. and Kalogerakis, N. (2001). Applied Parameter Estimation for Chemical Engineers. Dekker, New York.
  • Grenander, U. (1981). Abstract Inference. Wiley, New York.
  • Hairer, E., Nørsett, S. P. and Wanner, G. (1993). Solving Ordinary Differential Equations I: Nonstiff Problems, 2nd ed. Springer, Berlin.
  • He, X., Fung, W. K. and Zhu, Z. Y. (2002). Estimation in a semiparametric model for longitudinal data with unspecified dependence structure. Biometrika 89 579–590.
  • Heckman, N. E. and Ramsay, J. O. (2000). Penalized regression with model-based penalties. Canad. J. Statist. 28 241–258.
  • Ho, D. D., Neumann, A. U., Perelson, A. S. et al. (1995). Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection. Nature 373 123–126.
  • Huang, J. (1996). Efficient estimation for the proportinal hazards model with interval censoring. Ann. Statist. 24 540–568.
  • Huang, J. (1999). Efficient estimation of the partly linear additive Cox model. Ann. Statist. 27 1536–1563.
  • Huang, J. and Rossini, A. J. (1997). Sieve estimation for the proportional-odds failure-time regression model with interval censoring. J. Amer. Statist. Assoc. 92 960–967.
  • Huang, J. Z. (2003). Local asymptotics for polynomial spline regression. Ann. Statist. 31 1600–1635.
  • Huang, J. Z., Zhang, L. and Zhou, L. (2007). Efficient estimation in marginal partially linear models for longitudinal/clustered data using plines. Scand. J. Statist. 34 451–477.
  • Huang, Y., Liu, D. and Wu, H. (2006). Hierarchical Bayesian methods for estimation of parameters in a longitudinal HIV dynamic system. Biometrics 62 413–423.
  • Huang, Y., Rosenkranz, S. L. and Wu, H. (2003). Modeling HIV dynamics and antiviral response with consideration of time-varying drug exposures, adherence and phenotypic sensitivity. Math. Biosci. 184 165–186.
  • Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proc. Fifth Berkeley Symp. Math. Statist. Probab. 221–233. Univ. California Press, Berkeley.
  • Jansson, B. and Revesz, L. (1975). Analysis of the growth of tumor cell populations. Math. Biosci. 19 131–154.
  • Jeffrey, A. M. and Xia, X. (2005). Identifiability of HIV/AIDS model. In Deterministic and Stochastic Models of AIDS Epidemics and HIV Infections with Intervention (W. Y. Tan and H. Wu, eds.) 255–286. World Scientific, Singapore.
  • Jennerich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Ann. Math. Statist. 40 633–643.
  • Jolliffe, I. T. (1972). Discarding variables in a principal component analyssis. I: Artificial data. J. Roy. Statist. Soc. Ser. C Appl. Statist. 21 160–172.
  • Joshi, M., Seidel-Morgenstern, A. and Kremling, A. (2006). Exploiting the boostrap method for quantifying parameter confidence intervals in dynamic systems. Metabolic Engineering 8 447–455.
  • Kolchin, E. (1973). Differential Algebra and Algebraic Groups. Academic Press, New York.
  • Kutta, W. (1901). Beitrag zur näherungsweisen itegration totaler differentialgleichungen. Zeitschr. Math. Phys. 46 435–453.
  • Li, L., Brown, M. B., Lee, K. H. and Gupta, S. (2002). Estimation and inference for a spline-enhanced pupulation pharmacokinetic model. Biometrics 58 601–611.
  • Li, Z., Osborne, M. R. and Pravan, T. (2005). Parameter estimation of ordinary differential equations. IMA J. Numer. Anal. 25 264–285.
  • Liang, H. and Wu, H. (2008). Parameter estimation for differential equation models using a framework of measurement error in regression models. J. Amer. Statist. Assoc. 103 1570–1583.
  • Ljung, L. and Glad, T. (1994). On global identifiability for arbitrary model parametrizations. Automatica 30 265–276.
  • Ma, S. and Kosorok, M. R. (2005). Robust semiparametric M-estimation and the weighted bootstrap. J. Multivariate Anal. 96 190–217.
  • Malinvaud, E. (1970). The consistancy of nonlinear regressions. Ann. Math. Statist. 41 956–969.
  • Mattheij, R. and Molenaar, J. (2002). Ordinary Differential Equations in Theory and Practice. SIAM, Philadelphia.
  • Miao, H., Dykes, C., Demeter, L. M., Cavenaugh, J., Park, S. Y., Perelson, A. S. and Wu, H. (2008). Modeling and estimation of kinetic parameters and replicative fitness of HIV-1 from flow-cytometry-based growth competition experiments. Bull. Math. Biol. 70 1749–1771.
  • Michelson, S. and Leith, J. T. (1997). Tumor heterogeneity and growth control. In Tumor Heterogeneity and Growth Control (J. A. Adam and N. Bellomo, eds.) 295–326. Birkhäuser, Boston.
  • Moles, C. G., Banga, J. R. and Keller, K. (2004). Solving nonconvex climate control problems: Pitfalls and algorithm performances. Appl. Soft. Comput. 5 35–44.
  • Nowak, M. A. and May, R. M. (2000). Virus Dynamics: Mathematical Principles of Immunology and Virology. Oxford Univ. Press, Oxford.
  • Ollivier, F. (1990). Le problème de l’identifiabilité globale: Étude thé orique, méthodes effectives et bornes de complexité. Ph.D. thesis, École Polytechnique, Paris, France.
  • Pakes, A. and Pollard, D. (1989). Simulation and the asymptotics of optimization estimators. Econometrica 57 1027–1057.
  • Perelson, A. S., Essunger, P., Cao, Y., Vesanen, M., Hurley, A., Saksela, K., Markowitz, M. and Ho, D. D. (1997). Decay characteristics of HIV-1-infected compartments during combination therapy. Nature 387 188–191.
  • Perelson, A. S. and Nelson, P. W. (1999). Mathematical analysis of HIV-1 dynamics in vivo. SIAM Rev. 41 3–44.
  • Perelson, A. S., Neumann, A. U., Markowitz, M., Leonard, J. M. and Ho, D. D. (1996). HIV-1 dynamics in vivo: Virion clearance rate, infected cell life-span, and viral generation time. Science 271 1582–1586.
  • Pohjanpalo, H. (1978). System identifiability based on the power series expansion of the solution. Math. Biosci. 41 21–33.
  • Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York.
  • Pollard, D. (1985). New ways to prove central limit theorems. Econometric Theory 1 295–314.
  • Pollard, D. (1990). Empirical Processes Theory and Applications. IMS, Hayward, CA.
  • Poyton, A. A., Varziri, M. S., McAuley, K. B., McLellen, P. J. and Ramsay, J. O. (2006). Parameter estimation in continuous-time dynamic models using principal differential analysis. Computers and Chemical Engineering 30 698–708.
  • Putter, H., Heisterkamp, S. H., Lange, J. M. and de Wolf, F. (2002). A Bayesian approach to parameter estimation in HIV dynamical models. Stat. Med. 21 2199–2214.
  • Quaiser, T. and Mönnigmann, M. (2009). Systematic identifiability testing for unambiguous mechanistic modeling—application to JAK-STAT, MAP kinase, and NF-κB signaling pathway models. BMC Sys. Bio. 3 50.
  • Ramsay, J. O. (1996). Principal Differential Analysis: Data Reduction by Differential Operators. J. Roy. Statist. Soc. Ser. B 58 495–508.
  • Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter estimation for differential equations: A generalized smoothing approach (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 69 741–796.
  • Ritt, J. F. (1950). Differential Algebra. Amer. Math. Soc., Providence, RI.
  • Rodriguez-Fernandez, M., Egea, J. A. and Banga, J. R. (2006). Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinformatics 7 1–18.
  • Runge, C. (1895). Ueber die numerische Auflösung von Differentialgleichungen. Math. Ann. 46 167–178.
  • Schick, A. (1986). On asymptotically efficient estimation in semiparametric models. Ann. Statist. 14 1139–1151.
  • Schumaker, L. L. (1981). Spline Functions. Wiley, New York.
  • Shen, X. (1997). On methods of sieves and penalization. Ann. Statist. 25 2555–2591.
  • Shen, X. and Wong, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580–615.
  • Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040–1053.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
  • Storn, R. and Price, K. (1997). Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11 341–359.
  • Swartz, J. and Bremermann, H. (1975). Discussion of parameter estimation in biological modeling: Algorithms for estimation and evaluation of the estimates. J. Math. Biol. 1 241–275.
  • Tan, W. Y. and Wu, H. (2005). Deterministic and Stochastic Models of AIDS Epidemics and HIV Infections With Intervention. World Scientific, Singapore.
  • Thomaseth, K., Alexandra, K. W., Bernhard, L. et al. (1996). Integrated mathematical model to assess β-cell activity during the oral glucose test. Amer. J. Phisiol. 270 E522–E531.
  • Vajda, S., Rabitz, H., Walter, E. and Lecourtier, Y. (1989). Qualitative and quantitative identifiability analysis of nonlinear chemical kinetiv-models. Chem. Eng. Commun. 83 191–219.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • van Domselaar, B. and Hemker, P. W. (1975). Nonlinear parameter estimation in initial value problmes. Report NW18/75, Math. Centrum, Amsterdam.
  • Varah, J. M. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM J. Sci. Comput. 3 28–46.
  • Varziri, M. S., Poyton, A. A., McAuley, K. B., McLellen, P. J. and Ramsay, J. O. (2008). Selecting optimal weighting factors in iPDA for parameter estimation in continuous-time dynamic models. Comp. Chem. Eng. 32 3011–3022.
  • Walter, E. (1987). Identifiability of Parameteric Models. Pergamon Press, Oxford.
  • Wei, X., Ghosh, S. K., Taylor, M. E. et al. (1995). Viral dynamics in human immunodeficiency virus type 1 infection. Nature 373 117–122.
  • Wellner, J. A. and Zhang, Y. (2007). Two likelihood-based semiparametric estimation methods for panel count data with covariates. Ann. Statist. 35 2106–2142.
  • Wu, C. F. (1981). Asymptotic theory of nonlinear least squares estiamtion. Ann. Statist. 9 501–513.
  • Wu, H. (2005). Statistical methods for HIV dynamic studies in AIDS clinical trials. Stat. Methods Med. Res. 14 1–22.
  • Wu, H., Huang, Y., Acosta, E. P. et al. (2005). Modeling long-term HIV dynamics and antiretroviral response: Effects of drug potency, pharmacokinetics, adherence, and drug resistance. JAIDS 39 272–283.
  • Wu, H., Zhu, H., Miao, H. and Perelson, A. S. (2008). Identifiability and statistical estimation of dynamic parameters in HIV/AIDS dynamic models. Bull. Math. Biol. 70 785–799.
  • Xia, X. (2003). Estimation of HIV/AIDS parameters. Automatica J. IFAC 39 1983–1988.
  • Xia, X. and Moog, C. H. (2003). Identifiability of nonlinear systems with applications to HIV/AIDS models. IEEE Trans. Automat. Control 48 330–336.
  • Xue, H., Lam, K. F. and Li, G. (2004). Sieve maximum likelihood estimator for semiparametric regression models with current status data. J. Amer. Statist. Assoc. 99 346–356.