The Annals of Applied Statistics

Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?

Giles Hooker and Stephen P. Ellner

Full-text: Access denied (no subscription detected) We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


This paper introduces diagnostic tests for the nature of lack of fit in ordinary differential equation models (ODEs) proposed for data. We present a hierarchy of three possible sources of lack of fit: unaccounted-for stochastic variation, misspecification of functional forms in rate equations, and omission of dynamic variables in the description of the system. We represent lack of fit by allowing a parameter vector to vary over time, and propose generic testing procedures that do not rely on specific alternative models. Instead, different sources for lack of fit are characterized in terms of nonparametric relationships among latent variables. The tests are carried out through a combination of residual bootstrap and permutation methods. We demonstrate the effectiveness of these tests on simulated data and on real data from laboratory ecological experiments and electro-cardiogram data.

Article information

Ann. Appl. Stat. Volume 9, Number 2 (2015), 754-776.

Received: December 2013
Revised: December 2014
First available in Project Euclid: 20 July 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Differential equation diagnostics goodness of fit attractor reconstruction bootstrap


Hooker, Giles; Ellner, Stephen P. Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?. Ann. Appl. Stat. 9 (2015), no. 2, 754--776. doi:10.1214/15-AOAS828.

Export citation


  • Abarbanel, H. D. I. (1996). Analysis of Observed Chaotic Data. Springer, New York.
  • Arora, N. and Biegler, L. T. (2004). A trust region SQP algorithm for equality constrained parameter estimation with simple parameter bounds. Comput. Optim. Appl. 28 51–86.
  • Bates, D. M. and Watts, D. G. (1988). Nonlinear Regression Analysis and Its Applications. Wiley, New York.
  • Becks, L., Ellner, S. P., Jones, L. E. and Hairston, N. G. (2010). Reduction of adaptive genetic diversity radically alters eco-evolutionary community dynamics. Ecol. Lett. 13 989–997.
  • Bellman, R. and Roth, R. S. (1971). The use of splines with unknown end points in the identification of systems. J. Math. Anal. Appl. 34 26–33.
  • Bock, H. G. (1983). Recent advances in parameter identification techniques for ODE. In Numerical Treatment of Inverse Problems in Differential and Integral Equations (Heidelberg, 1982) (P. Deuflhard and E. Harrier, eds.). Progr. Sci. Comput. 2 95–121. Birkhäuser, Boston, MA.
  • Dattner, I. and Klaassen, C. A. J. (2013). Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters. Preprint. Available at arXiv:1305.4126.
  • Ellner, S. P., Seifu, Y. and Smith, R. H. (2002). Fitting population dynamic models to time-series data by gradient matching. Ecology 83 2256–2270.
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
  • Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 123–214.
  • Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Peng M. G.B., C.-K. and Stanley, H. E. (2000). Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation 101 e215–e220.
  • Golightly, A. and Wilkinson, D. J. (2011). Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo. Interface Focus 1 1–14.
  • Gouriéroux, C. and Monfort, A. (1997). Simulation-Based Econometric Methods. Oxford Univ. Press, Oxford.
  • Hamilton, J. D. (1994). Time Series Analysis. Princeton Univ. Press, Princeton, NJ.
  • Hiltunen, T., Hairston, N. G. Jr., Hooker, G., Jones, L. E. and Ellner, S. P. (2014). AUG. A newly discovered role of evolution in previously published consumer-resource dynamics. Ecology Letters 17 915–923.
  • Hooker, G. (2009). Forcing function diagnostics for nonlinear dynamics. Biometrics 65 928–936.
  • Hooker, G. and Ellner, S. P. (2015). Supplement to “Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?” DOI:10.1214/15-AOAS828SUPP.
  • Hooker, G., Lin, K. K. and Rogers, B. (2015). Control theory and experimental design in diffusion processes. Under review. Journal on Uncertainty Quantification.
  • Ionides, E. L., Bretó, C. and King, A. A. (2006). Inference for nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA 103 18438–18443.
  • Kantz, H. and Schreiber, T. (2005). Nonlinear Time Series Analysis, Cambridge Univ. Press, Cambridge.
  • Moody, G. B. and Mark, R. G. (2001). The impact of the MIT-BIH arrhythmia database. IEEE Eng. Med. Biol. Mag. 20 45–50.
  • Pascual, M. and Ellner, S. P. (2000). Linking ecological patterns to environmental forcing via nonlinear time series models. Ecology 81 2767–2780.
  • Ramsay, J. O., Hooker, G. and Graves, S. (2009). Functional Data Analysis in R and Matlab. Springer, New York.
  • Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter estimation for differential equations: A generalized smoothing approach. J. R. Stat. Soc. Ser. B. Stat. Methodol. 69 741–796.
  • Ratmann, O., Andrieu, C., Wiuf, C. and Richardson, S. (2009). Model criticism based on likelihood-free inference, with an application to protein network evolution. Proc. Nat. Acad. Sci. USA 106 10576–10581.
  • Reuman, D. C., Desharnais, R. A., Costantino, R. F., Ahmad, O. S. and Cohen, J. E. (2006). Power spectra reveal the influence of stochasticity on nonlinear population dynamics. Proceedings of the National Academies of Sciences 103 18660–18665.
  • Rössler, O. E. (1976). An equation for continuous chaos. Physics Letters 57A(5) 397–398.
  • Srivastava, R. K. (2014). An exact two-sample test in high dimensions using random projections. Preprint. Available at arXiv:1405.1792.
  • Stark, J., Broomhead, D. S., Davies, M. E. and Huke, J. (1997). Takens embedding theorems for forced and stochastic systems. Nonlinear. Anal. 30 5303–5314.
  • Takens, F. (1981). Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence, Warwick 1980 (Coventry, 1979/1980). Lecture Notes in Math. 898 366–381. Springer, Berlin.
  • Thorbergsson, L. and Hooker, G. (2013). Experimental design for partially observed Markov decision processes. Preprint. Available at arXiv:1209.4019.
  • Tien, J. H. and Guckenheimer, J. (2008). Parameter estimation for bursting neural models. J. Comput. Neurosci. 24 358–373.
  • van der Pol, B. (1927). On relaxation-oscillations. The London, Edinburgh and Dublin Philosophical Magazine and Journal of Science 2 978–992.
  • Varah, J. M. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM J. Sci. Statist. Comput. 3 28–46.
  • Wilson, H. R. (1999). Spikes, Decisions, and Actions: The Dynamical Foundations of Neuroscience. Oxford Univ. Press, New York.
  • Wood, S. N. (2010). Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466 1102–U113.
  • Wood, S. (2013). mgcv: Mixed GAM Computation Vehicle with GCV/AIC/REML smoothness estimation. R package version 1.7-27.
  • Wu, H., Xue, H. and Kumar, A. (2012). Numerical discretization-based estimation methods for ordinary differential equation models via penalized spline smoothing with applications in biomedical research. Biometrics 68 344–352.
  • Yoshida, T., Jones, L. E., Ellner, S. P., Fussmann, G. F. and Hairston, N. G. (2003). Rapid evolution drives ecological dynamics in a predator–prey system. Nature 424 303–306.

Supplemental materials

  • Supplementary material for “Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?”. This appendix provides supporting material which includes the following: details of the chemostat models used to generate data for Section 6 and background material on the generalized profiling methods of Ramsay et al. (2007), along with simulation experiments using this method instead of gradient matching.