Bayesian Analysis

A Role for Symmetry in the Bayesian Solution of Differential Equations

Junyang Wang, Jon Cockayne, and Chris. J. Oates

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access


The interpretation of numerical methods, such as finite difference methods for differential equations, as point estimators suggests that formal uncertainty quantification can also be performed in this context. Competing statistical paradigms can be considered and Bayesian probabilistic numerical methods (PNMs) are obtained when Bayesian statistical principles are deployed. Bayesian PNM have the appealing property of being closed under composition, such that uncertainty due to different sources of discretisation in a numerical method can be jointly modelled and rigorously propagated. Despite recent attention, no exact Bayesian PNM for the numerical solution of ordinary differential equations (ODEs) has been proposed. This raises the fundamental question of whether exact Bayesian methods for (in general nonlinear) ODEs even exist. The purpose of this paper is to provide a positive answer for a limited class of ODE. To this end, we work at a foundational level, where a novel Bayesian PNM is proposed as a proof-of-concept. Our proposal is a synthesis of classical Lie group methods, to exploit underlying symmetries in the gradient field, and non-parametric regression in a transformed solution space for the ODE. The procedure is presented in detail for first and second order ODEs and relies on a certain strong technical condition – existence of a solvable Lie algebra – being satisfied. Numerical illustrations are provided.

Article information

Bayesian Anal., Advance publication (2020), 29 pages.

First available in Project Euclid: 16 October 2019

Permanent link to this document

Digital Object Identifier

probabilistic numerics ordinary differential equations Lie groups

Creative Commons Attribution 4.0 International License.


Wang, Junyang; Cockayne, Jon; Oates, Chris. J. A Role for Symmetry in the Bayesian Solution of Differential Equations. Bayesian Anal., advance publication, 16 October 2019. doi:10.1214/19-BA1183.

Export citation


  • Abdulle, A. and Garegnani, G. (2018). “Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration.” arXiv:1801.01340.
  • Baumann, G. (2013). Symmetry Analysis of Differential Equations with Mathematica®. Springer Science & Business Media.
  • Bluman, G. and Anco, S. (2002). Symmetry and Integration Methods for Differential Equations. Springer.
  • Briol, F.-X., Oates, C. J., Girolami, M., Osborne, M. A., and Sejdinovic, D. (2019). “Probabilistic integration: A role in statistical computation? (with discussion and rejoinder).” Statistical Science, 34(1): 1–42.
  • Chabiniok, R., Wang, V. Y., Hadjicharalambous, M., Asner, L., Lee, J., Sermesant, M., Kuhl, E., Young, A. A., Moireau, P., Nash, M. P., et al. (2016). “Multiphysics and multiscale modelling, data–model fusion and integration of organ physiology in the clinic: ventricular cardiac mechanics.” Interface Focus, 6(2): 20150083.
  • Chang, J. T. and Pollard, D. (1997). “Conditioning as disintegration.” Statistica Neerlandica, 51(3): 287–317.
  • Chkrebtii, O. and Campbell, D. (2019). “Adaptive step-size selection for state-space based probabilistic differential equation solvers.” Statistics and Computing. To appear.
  • Chkrebtii, O., Campbell, D. A., Girolami, M. A., and Calderhead, B. (2016). “Bayesian uncertainty quantification for differential equations (with discussion).” Bayesian Analysis, 11(4): 1239–1267.
  • Cockayne, J., Oates, C., Sullivan, T., and Girolami, M. (2019). “Bayesian probabilistic numerical methods.” SIAM Review. To appear.
  • Conrad, P. R., Girolami, M., Särkkä, S., Stuart, A., and Zygalakis, K. (2017). “Statistical analysis of differential equations: introducing probability measures on numerical solutions.” Statistics and Computing, 27(4): 1065–1082.
  • Diaconis, P. (1988). “Bayesian numerical analysis.” Statistical Decision Theory and Related Topics IV(1), 1988.
  • Estep, D. (1995). “A posteriori error bounds and global error control for approximation of ordinary differential equations.” SIAM Journal on Numerical Analysis, 32(1): 1–48.
  • Gelman, A. and Shalizi, C. R. (2013). “Philosophy and the practice of Bayesian statistics.” British Journal of Mathematical and Statistical Psychology, 66(1): 8–38.
  • Hawkins, T. (2012). Emergence of the theory of Lie groups: An essay in the history of mathematics 1869–1926. Springer Science & Business Media.
  • Hennig, P., Osborne, M. A., and Girolami, M. (2015). “Probabilistic numerics and uncertainty in computations.” Proceedings of the Royal Society A, 471(2179): 20150142.
  • Higham, N. J. (2002). Accuracy and Stability of Numerical Algorithms. SIAM.
  • Karvonen, T., Oates, C. J., and Särkkä, S. (2018). “A Bayes-Sard cubature method.” In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPS2018), 5882–5893.
  • Kersting, H. and Hennig, P. (2016). “Active uncertainty calibration in Bayesian ODE solvers.” In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016), 309–318.
  • Kersting, H., Sullivan, T., and Hennig, P. (2018). “Convergence rates of Gaussian ODE filters.” arXiv:1807.09737.
  • Larkin, F. (1972). “Gaussian measure in Hilbert space and applications in numerical analysis.” The Rocky Mountain Journal of Mathematics, 2: 379–421.
  • Larkin, F. M. (1974). “Probabilistic error estimates in spline interpolation and quadrature.” In Information Processing 74 (Proc. IFIP Congress, Stockholm, 1974), 605–609.
  • Lie, H., Stuart, A., and Sullivan, T. (2019). “Strong convergence rates of probabilistic integrators for ordinary differential equations.” Statistics and Computing. To appear.
  • Lie, H. C., Sullivan, T., and Teckentrup, A. L. (2018). “Random forward models and log-likelihoods in Bayesian inverse problems.” SIAM/ASA Journal on Uncertainty Quantification, 6(4): 1600–1629.
  • López-Lopera, A. F., Bachoc, F., Durrande, N., and Roustant, O. (2018). “Finite-dimensional Gaussian approximation with linear inequality constraints.” SIAM/ASA Journal of Uncertainty Quantification, 6(3): 1224–1255.
  • Oates, C. J. and Sullivan, T. J. (2019). “A modern retrospective on probabilistic numerics.” Statistics and Computing. To appear.
  • Perilla, J. R., Goh, B. C., Cassidy, C. K., Liu, B., Bernardi, R. C., Rudack, T., Yu, H., Wu, Z., and Schulten, K. (2015). “Molecular dynamics simulations of large macromolecular complexes.” Current Opinion in Structural Biology, 31: 64–74.
  • Schober, M., Duvenaud, D. K., and Hennig, P. (2014). “Probabilistic ODE solvers with Runge-Kutta means.” In Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS 2014), 739–747.
  • Schober, M., Särkkä, S., and Hennig, P. (2019). “A probabilistic model for the numerical solution of initial value problems.” Statistics and Computing, 29: 99–122.
  • Skilling, J. (1992). “Bayesian solution of ordinary differential equations.” In Maximum Entropy and Bayesian Methods, 23–37. Springer.
  • Teymur, O., Lie, H. C., Sullivan, T., and Calderhead, B. (2018). “Implicit Probabilistic Integrators for ODEs.” In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPS 2018).
  • Teymur, O., Zygalakis, K., and Calderhead, B. (2016). “Probabilistic linear multistep methods.” In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), 4321–4328.
  • Traub, J. and Woźniakowski (1992). “Perspectives on Information-Based Complexity.” Bulletin of the American Mathematical Society, 26(1): 29–52.
  • Tronarp, F., Kersting, H., Särkkä, S., and Hennig, P. (2019). “Probabilistic solutions to ordinary differential equations as non-linear Bayesian filtering: A new perspective.” Statistics and Computing. To appear.
  • Wang, J., Cockayne, J., and Oates, C. J. (2019). “Supplementary material.” Bayesian Analysis.
  • Wedi, N. P. (2014). “Increasing horizontal resolution in numerical weather prediction and climate simulations: illusion or panacea?” Philosophical Transactions of the Royal Society A, 372(2018): 20130289.

Supplemental materials