The Annals of Applied Statistics

Bayesian inference of high-dimensional, cluster-structured ordinary differential equation models with applications to brain connectivity studies

Tingting Zhang, Qiannan Yin, Brian Caffo, Yinge Sun, and Dana Boatman-Reich

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We build a new ordinary differential equation (ODE) model for the directional interaction, also called effective connectivity, among brain regions whose activities are measured by intracranial electrocorticography (ECoG) data. In contrast to existing ODE models that focus on effective connectivity among only a few large anatomic brain regions and that rely on strong prior belief of the existence and strength of the connectivity, the proposed high-dimensional ODE model, motivated by statistical considerations, can be used to explore connectivity among multiple small brain regions. The new model, called the modular and indicator-based dynamic directional model (MIDDM), features a cluster structure, which consists of modules of densely connected brain regions, and uses indicators to differentiate significant and void directional interactions among brain regions. We develop a unified Bayesian framework to quantify uncertainty in the assumed ODE model, identify clusters, select strongly connected brain regions, and make statistical comparison between brain networks across different experimental trials. The prior distributions in the Bayesian model for MIDDM parameters are carefully designed such that the ensuing joint posterior distributions for ODE state functions and the MIDDM parameters have well-defined and easy-to-simulate posterior conditional distributions. To further speed up the posterior simulation, we employ parallel computing schemes in Markov chain Monte Carlo steps. We show that the proposed Bayesian approach outperforms an existing optimization-based ODE estimation method. We apply the proposed method to an auditory electrocorticography dataset and evaluate brain auditory network changes across trials and different auditory stimuli.

Article information

Ann. Appl. Stat., Volume 11, Number 2 (2017), 868-897.

Received: June 2016
Revised: January 2017
First available in Project Euclid: 20 July 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Bayesian inference ODE models cluster structure directional brain networks network edge selection


Zhang, Tingting; Yin, Qiannan; Caffo, Brian; Sun, Yinge; Boatman-Reich, Dana. Bayesian inference of high-dimensional, cluster-structured ordinary differential equation models with applications to brain connectivity studies. Ann. Appl. Stat. 11 (2017), no. 2, 868--897. doi:10.1214/17-AOAS1021.

Export citation


  • Aertsen, A. and Preissl, H. (1991). Dynamics of activity and connectivity in physiological neuronal networks. In Nonlinear Dynamics and Neuronal Networks (H. Schuster, ed.) 281–302. VCH publishers Inc, New York.
  • Anderson, J. (2005). Learning in sparsely connected and sparsely coded system. Ersatz Brain Project Working Note.
  • Bard, Y. (1974). Nonlinear Parameter Estimation. Academic Press, New York.
  • Bhaumik, P. and Ghosal, S. (2014). Bayesian estimation in differential equation models. Preprint. Available at arXiv:1403.0609.
  • Biegler, L., Damiano, J. and Blau, G. (1986). Nonlinear parameter estimation: A case study comparison. AIChE J. 32 29–45.
  • Boatman-Reich, D., Franaszczuk, P. J., Korzeniewska, A., Caffo, B., Ritzl, E. K., Colwell, S. and Crone, N. E. (2010). Quantifying auditory event-related responses in multichannel human intracranial recordings. Front. Comput. Neurosci. 4 4.
  • Bressler, S. and Ding, M. (2002). Event-related potentials. In The Handbook of Brain Theory and Neural Networks 412–415. Wiley, New York.
  • Brown, P. J., Vannucci, M. and Fearn, T. (1998). Multivariate Bayesian variable selection and prediction. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 627–641.
  • Brunel, N. J.-B. (2008). Parameter estimation of ODE’s via nonparametric estimators. Electron. J. Stat. 2 1242–1267.
  • Bullmore, E. and Sporns, O. (2009). Complex brain networks: Graph theoretical analysis of structural and functional systems. Nat. Rev., Neurosci. 10 186–198.
  • Caffo, B., Peng, R., Dominici, F., Louis, T. A. and Zeger, S. (2011). Parallel MCMC for analyzing distributed lag models with systematic missing data for an application in environmental epidemiology. In Handbook of Markov Chain Monte Carlo (S. Brooks, A. Gelman, G. Jones and X. Meng, eds.) 493–511. CRC Press, Boca Raton, FL.
  • Calderhead, B., Girolami, M. and Lawrence, N. (2008). Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. Adv. Neural Inf. Process. Syst. 22.
  • Campbell, D. A. (2007). Bayesian Collocation Tempering and Generalized Profiling for Estimation of Parameters from Differential Equation Models. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–McGill University (Canada).
  • Cao, J., Huang, J. Z. and Wu, H. (2012). Penalized nonlinear least squares estimation of time-varying parameters in ordinary differential equations. J. Comput. Graph. Statist. 21 42–56.
  • Catani, M., DellAcqua, F., Vergani, F., Malik, F., Hodge, H., Roy, P., Valabregue, R. and Thiebaut de Schotten, M. (2012). Short frontal lobe connections of the human brain. Cortex 48 273–291.
  • Cervenka, M. C., Franaszczuk, P. J., Crone, N. E., Hong, B., Caffo, B. S., Bhatt, P., Lenz, F. A. and Boatman-Reich, D. (2013). Reliability of early cortical auditory gamma-band responses. Clin. Neurophysiol. 124 70–82.
  • Cheung, S., Oliver, T., Prudencio, E., Prudhomme, S. and Moser, R. (2011). Bayesian uncertainty analysis with applications to turbulence modeling. Reliab. Eng. Syst. Saf. 96 1137–1149.
  • Chkrebtii, O. A., Campbell, D. A., Calderhead, B. and Girolami, M. A. (2016). Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11 1239–1267.
  • Conrad, P., Girolami, M., Särkkä, S., Stuart, A. and Zygalakis, K. (2015). Probability Measures for Numerical Solutions of Differential Equations.
  • Daunizeau, J., David, O. and Stephan, K. (2011). Dynamic causal modelling: A critical review of the biophysical and statistical foundations. NeuroImage 58 312–322.
  • David, O. and Friston, K. (2003). A neural mass model for MEG/EEG: Coupling and neuronal dynamics. NeuroImage 20 1743–1755.
  • David, O., Kiebel, S., Harrison, L., Mattout, J., Kilner, J. and Friston, K. (2006). Dynamic causal modelling of evoked responses in EEG and MEG. NeuroImage 30 1255–1272.
  • Deuflhard, P. and Bornemann, F. (2002). Scientific Computing with Ordinary Differential Equations. Springer, New York.
  • Dunson, D. B., Herring, A. H. and Engel, S. M. (2008). Bayesian selection and clustering of polymorphisms in functionally related genes. J. Amer. Statist. Assoc. 103 534–546.
  • Durka, P., Ircha, D., Neuper, C. and Pfurtscheller, G. (2001). Time-frequency microstructure of event-related electro-encephalogram eesynchronisation and synchronisation. Med. Biol. Eng. Comput. 39 315–3211.
  • Eliades, S., Crone, N., Anderson, W., Ramadoss, D., Lenz, F. and Boatman-Reich, D. (2014). Adaptation of high-gamma responses in human auditory association cortex. J. Neurophysiol. 112 2147–2163.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Földiák, P. and Young, M. P. (1995). Sparse coding in the primate cortex. In The Handbook of Brain Theory and Neural Networks 895–898. MIT Press, Cambridge.
  • Franaszczuk, P. J. and Bergey, G. K. (1998). Application of the directed transfer function method to mesial and lateral onset temporal lobe seizures. Brain Topogr. 11 13–21.
  • Friston, K. (2009). Causal modelling and brain connectivity in functional magnetic resonance imaging. PLoS Biology 7 33.
  • Friston, K., Harrison, L. and Penny, W. (2003). Dynamic causal modelling. NeuroImage 19 1273–1302.
  • Garrido, M., Kilner, J., Kiebel, S., Stephan, K., Baldeweg, T. and Friston, K. (2009). Comparative frequency analysis of single EEG-evoked potential records. NeuroImage 48 269–279.
  • Gelman, A., Bois, F. and Jiang, J. (1996). Physiological pharamacokinetic analysis using population modeling and informative prior distributions. J. Amer. Statist. Assoc. 91 1400–1412.
  • Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL.
  • George, E. and McCulloch, R. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881–889.
  • George, E. and McCulloch, R. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339–373.
  • Girolami, M. (2008). Bayesian inference for differential equations. Theoret. Comput. Sci. 408 4–16.
  • Graner, F. and Glazier, J. A. (1992). Simulation of biological cell sorting using a two-dimensional extended Potts model. Phys. Rev. Lett. 69 2013–2016.
  • Hemker, P. (1972). Numerical methods for differential equations in system simulations and in parameter estimation. Analysis and Simulation of Biochemical Systems 59–80.
  • Herrmann, B., Henry, M. and Obleser, J. (2013). Frequency-specific adaptation in human auditory cortex depends on the spectral variance in the acoustic stimulation. J. Neurophysiol. 109 2086–2096.
  • Herrmann, B., Schlichting, N. and Obleser, J. (2014). Dynamic range adaptation to spectral stimulus statistics in human auditory cortex. J. Neurosci. 34 327–331.
  • Huang, Y., Liu, D. and Wu, H. (2006). Hierarchical Bayesian methods for estimation of parameters in a longitudinal HIV dynamic system. Biometrics 62 413–423.
  • Huang, Y. and Wu, H. (2006). A Bayesian approach for estimating antiviral efficacy in HIV dynamic models. J. Appl. Stat. 33 155–174.
  • Ishwaran, H. and Rao, J. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. Ann. Statist. 33 730–773.
  • Kennedy, M. C. and O’Hagan, A. (2001). Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 63 425–464.
  • Kiebel, S., David, O. and Friston, K. (2006). Dynamic causal modelling of evoked responses in EEG/MEG with lead-field parameterization. NeuroImage 30 1273–1284.
  • Kim, S., Tadesse, M. G. and Vannucci, M. (2006). Variable selection in clustering via Dirichlet process mixture models. Biometrika 93 877–893.
  • Li, Z., Osborne, M. R. and Prvan, T. (2005). Parameter estimation of ordinary differential equations. IMA J. Numer. Anal. 25 264–285.
  • Lu, T., Liang, H., Li, H. and Wu, H. (2011). High-dimensional ODEs coupled with mixed-effects modeling techniques for dynamic gene regulatory network identification. J. Amer. Statist. Assoc. 106 1242–1258.
  • Mattheij, R. and Molenaar, J. (2002). Ordinary Differential Equations in Theory and Practice. Classics in Applied Mathematics 43. SIAM, Philadelphia, PA.
  • Micheloyannis, S. (2012). Graph-based network analysis in schizophrenia. World J. Psychiatry 2 1–12.
  • Miller, A. (2002). Subset Selection in Regression, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL.
  • Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N., Chklovskii, D. and Alon, U. (2002). Network motifs: Simple building blocks of complex networks. Science 298 824–827.
  • Milo, R., Itzkovitz, S., Kashtan, N., Levitt, R., Shen-Orr, S., Ayzenshtat, I., Sheffer, M. and Alon, U. (2004). Superfamilies of evolved and designed networks. Science 303 1538–1542.
  • Näätänen, R., Paavilainen, P., Rinne, T. and Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clin. Neurophysiol. 118 2544–2590.
  • Newman, M. E. J. (2006). Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103 8577–8696.
  • Oliver, T. and Moser, R. (2011). Bayesian uncertainty quantification applied to RANS turbulence models. Int. J. Mod. Phys. Conf. Ser. 318 042032.
  • Olshausen, B. and Field, D. (2004). Sparse coding of sensor inputs. Current Opinions in Neurobiology 14 481–487.
  • Park, H.-J. and Friston, K. (2013). Structural and functional brain networks: From connections to cognition. Science 342 1238411.
  • Potts, R. B. (1952). Some generalized order-disorder transformations. Math. Proc. Cambridge Philos. Soc. 48 106–109.
  • Poyton, A., Varziri, M., McAuley, K., McLellan, P. and Ramsay, J. (2006). Parameter estimation in continuous dynamic models using principal differential analysis. Computational Chemical Engineering 30 698–708.
  • Qi, X. and Zhao, H. (2010). Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations. Ann. Statist. 38 435–481.
  • Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter estimation for differential equations: A generalized smoothing approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 741–796.
  • Reiss, P. T. and Ogden, R. T. (2007). Functional principal component regression and functional partial least squares. J. Amer. Statist. Assoc. 102 984–996.
  • Reiss, P. T. and Ogden, R. T. (2009). Smoothing parameter selection for a class of semiparametric linear models. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 505–523.
  • Schönwiesner, M., Novitski, N., Pakarinen, S., Carlson, S., Tervaniemi, M. and Näätänen, R. (2007). Heschl’s gyrus, posterior superior temporal gyrus, and mid-ventrolateral prefrontal cortex have different roles in the detection of acoustic changes. J. Neurophysiol. 97 2075–2082.
  • Sinai, A., Crone, N., Wied, H., Franaszczuk, P., Miglioretti, D. and Boatman-Reich, D. (2009). Intracranial mapping of auditory perception: Event-related responses and electrocortical stimulation. Clin. Neurophysiol. 120 140–149.
  • Sporns, O. (2011). Networks of the Brain. MIT Press, Cambridge, MA.
  • Stuart, A. M. (2010). Inverse problems: A Bayesian perspective. Acta Numer. 19 451–559.
  • Tadesse, M. G., Sha, N. and Vannucci, M. (2005). Bayesian variable selection in clustering high-dimensional data. J. Amer. Statist. Assoc. 100 602–617.
  • Theo, H. and Mike, E. (2004). Mapping multiple QTL using linkage disequilibrium and linkage analysis information and multitrait data. Genet. Sel. Evol 36 261–279.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • van Dyk, D. A. and Park, T. (2008). Partially collapsed Gibbs samplers: Theory and methods. J. Amer. Statist. Assoc. 103 790–796.
  • Varah, J. M. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM J. Sci. Statist. Comput. 3 28–46.
  • Voit, E. (2000). Computational Analysis of Biochemical Systems: A Practical Guide for Biochemists and Molecular Biologists. Cambridge Univ. Press, Cambridge.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia, PA.
  • Wang, H. and Leng, C. (2008). A note on adaptive group lasso. Comput. Statist. Data Anal. 52 5277–5286.
  • Wu, H., Lu, T., Xue, H. and Liang, H. (2014a). Sparse additive ordinary differential equations for dynamic gene regulatory network modeling. J. Amer. Statist. Assoc. 109 700–716.
  • Wu, S., Xue, H., Wu, Y. and Wu, H. (2014b). Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening. Statist. Sinica 24 1365–1387.
  • Xue, H., Miao, H. and Wu, H. (2010). Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. Ann. Statist. 38 2351–2387.
  • Yi, N., George, V. and Allison, D. B. (2003). Stochastic search variable selection for identifying multiple quantitative trait loci. Genetics 164 1129–1138.
  • Yuan, M. and Lin, Y. (2005). Efficient empirical Bayes variable selection and estimation in linear models. J. Amer. Statist. Assoc. 100 1215–1225.
  • Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 49–67.
  • Zhang, T., Wu, J., Li, F., Caffo, B. and Boatman-Reich, D. (2015). A dynamic directional model for effective brain connectivity using electrocorticographic (ECoG) time series. J. Amer. Statist. Assoc. 110 93–106.
  • Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.
  • Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 301–320.