Bayesian Analysis

Searching Multiregression Dynamic Models of Resting-State fMRI Networks Using Integer Programming

Lilia Costa, Jim Smith, Thomas Nichols, James Cussens, Eugene P. Duff, and Tamar R. Makin

Full-text: Open access


A Multiregression Dynamic Model (MDM) is a class of multivariate time series that represents various dynamic causal processes in a graphical way. One of the advantages of this class is that, in contrast to many other Dynamic Bayesian Networks, the hypothesised relationships accommodate conditional conjugate inference. We demonstrate for the first time how straightforward it is to search over all possible connectivity networks with dynamically changing intensity of transmission to find the Maximum a Posteriori Probability (MAP) model within this class. This search method is made feasible by using a novel application of an Integer Programming algorithm. The efficacy of applying this particular class of dynamic models to this domain is shown and more specifically the computational efficiency of a corresponding search of 11-node Directed Acyclic Graph (DAG) model space. We proceed to show how diagnostic methods, analogous to those defined for static Bayesian Networks, can be used to suggest embellishment of the model class to extend the process of model selection. All methods are illustrated using simulated and real resting-state functional Magnetic Resonance Imaging (fMRI) data.

Article information

Bayesian Anal., Volume 10, Number 2 (2015), 441-478.

First available in Project Euclid: 2 February 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Multiregression Dynamic Model Bayesian Network Integer Program Algorithm Model Selection Functional magnetic resonance imaging (fMRI)


Costa, Lilia; Smith, Jim; Nichols, Thomas; Cussens, James; Duff, Eugene P.; Makin, Tamar R. Searching Multiregression Dynamic Models of Resting-State fMRI Networks Using Integer Programming. Bayesian Anal. 10 (2015), no. 2, 441--478. doi:10.1214/14-BA913.

Export citation


  • Achterberg, T. (2007). “Constraint Integer Programming.” Ph.D. thesis, Technische Universität Berlin.
  • Ali, R. A., Richardson, T. S., and Spirtes, P. (2009). “Markov equivalence for ancestral graphs.” The Annals of Statistics, 37(5B): 2808–2837.
  • Allen, E. A., Damaraju, E., Plis, S. M., Erhardt, E. B., Eichele, T., and Calhoun, V. D. (2014). “Tracking whole-brain connectivity dynamics in the resting state.” Cerebral cortex, 24: 663–676.
  • Anacleto, O., Queen, C., and Albers, C. J. (2013). “Multivariate forecasting of road traffic flows in the presence of heteroscedasticity and measurement errors.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 62(2): 251–270.
  • Arnhold, J., Grassberger, P., Lehnertz, K., and Elger, C. (1999). “A robust method for detecting interdependences: application to intracranially recorded EEG.” Physica D: Nonlinear Phenomena, 134(4): 419–430.
  • Baba, K., Shibata, R., and Sibuya, M. (2004). “Partial correlation and conditional correlation as measures of conditional independence.” Australian & New Zealand Journal of Statistics, 46(4): 657–664.
  • Bartlett, M. and Cussens, J. (2013). “Advances in Bayesian Network Learning Using Integer Programming.” In Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence (UAI 2013), 182–191. AUAI Press.
  • Benjamini, Y. and Hochberg, Y. (1995). “Controlling the false discovery rate: a practical and powerful approach to multiple testing.” Journal of the Royal Statistical Society. Series B (Methodological), 289–300.
  • Bhattacharya, S. and Maitra, R. (2011). “A nonstationary nonparametric Bayesian approach to dynamically modeling effective connectivity in functional magnetic resonance imaging experiments.” The Annals of Applied Statistics, 5(2B): 1183–1206.
  • Bhattacharya, S., Ringo Ho, M.-H., and Purkayastha, S. (2006). “A Bayesian approach to modeling dynamic effective connectivity with fMRI data.” Neuroimage, 30(3): 794–812.
  • Chang, C. and Glover, G. H. (2010). “Time–frequency dynamics of resting-state brain connectivity measured with fMRI.” Neuroimage, 50(1): 81–98.
  • Chang, C., Thomason, M. E., and Glover, G. H. (2008). “Mapping and correction of vascular hemodynamic latency in the BOLD signal.” Neuroimage, 43(1): 90–102.
  • Chickering, D. M. (2003). “Optimal structure identification with greedy search.” The Journal of Machine Learning Research, 3: 507–554.
  • Consonni, G. and La Rocca, L. (2011). “Moment priors for Bayesian model choice with applications to directed acyclic graphs.” Bayesian Statistics 9, 9: 119.
  • Cowell, R. G. (2013). “A simple greedy algorithm for reconstructing pedigrees.” Theoretical Population Biology, 83: 55–63.
  • Cowell, R. G., Dawid, A. P., Lauritzen, S. L., and Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. New York: Springer-Verlag.
  • Cribben, I., Haraldsdottir, R., Atlas, L. Y., Wager, T. D., and Lindquist, M. A. (2012). “Dynamic connectivity regression: determining state-related changes in brain connectivity.” Neuroimage, 61(4): 907–920.
  • Cussens, J. (2010). “Maximum likelihood pedigree reconstruction using integer programming.” In Proceedings of the Workshop on Constraint Based Methods for Bioinformatics (WCB-10). Edinburgh.
  • — (2011). “Bayesian Network Learning with Cutting Planes.” In Cozman, F. G. and Pfeffer, A. (eds.), Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), 153–160. Barcelona: AUAI Press. URL
  • Daunizeau, J., Friston, K., and Kiebel, S. (2009). “Variational Bayesian identification and prediction of stochastic nonlinear dynamic causal models.” Physica D: Nonlinear Phenomena, 238(21): 2089–2118.
  • Dauwels, J., Vialatte, F., Musha, T., and Cichocki, A. (2010). “A comparative study of synchrony measures for the early diagnosis of Alzheimer’s disease based on EEG.” NeuroImage, 49(1): 668–693.
  • David, O., Guillemain, I., Saillet, S., Reyt, S., Deransart, C., Segebarth, C., and Depaulis, A. (2008). “Identifying neural drivers with functional MRI: an electrophysiological validation.” PLoS Biology, 6(12): e315.
  • Duff, E., Makin, T., Madugula, S., Smith, S. M., and Woolrich, M. W. (2013). “Utility of Partial Correlation for Characterising Brain Dynamics: MVPA-based Assessment of Regularisation and Network Selection.” In Pattern Recognition in Neuroimaging (PRNI), 2013 International Workshop on, 58–61. IEEE.
  • Durbin, J. and Koopman, S. J. (2012). Time series analysis by state space methods. 38. Oxford University Press.
  • Friston, K. J. (2011). “Functional and effective connectivity: a review.” Brain Connectivity, 1(1): 13–36.
  • Friston, K. J., Harrison, L., and Penny, W. (2003). “Dynamic causal modelling.” Neuroimage, 19(4): 1273–1302.
  • Frühwirth-Schnatter, S. (2006). Finite Mixture and Markov Switching Models: Modeling and Applications to Random Processes. Springer.
  • Ge, T., Kendrick, K. M., and Feng, J. (2009). “A novel extended Granger causal model approach demonstrates brain hemispheric differences during face recognition learning.” PLoS computational biology, 5(11): e1000570.
  • Granger, C. W. (1969). “Investigating causal relations by econometric models and cross-spectral methods.” Econometrica: Journal of the Econometric Society, 37(3): 424–438.
  • Harrison, J. and West, M. (1991). “Dynamic linear model diagnostics.” Biometrika, 78(4): 797–808.
  • Havlicek, M., Jan, J., Brazdil, M., and Calhoun, V. D. (2010). “Dynamic Granger causality based on Kalman filter for evaluation of functional network connectivity in fMRI data.” Neuroimage, 53(1): 65–77.
  • Heard, N. A., Holmes, C. C., and Stephens, D. A. (2006). “A quantitative study of gene regulation involved in the immune response of anopheline mosquitoes: An application of Bayesian hierarchical clustering of curves.” Journal of the American Statistical Association, 101(473): 18–29.
  • Heckerman, D. (1998). A tutorial on learning with Bayesian networks. Springer.
  • Jaakkola, T., Sontag, D., Globerson, A., and Meila, M. (2010). “Learning Bayesian Network Structure using LP Relaxations.” In Proceedings of 13th International Conference on Artificial Intelligence and Statistics (AISTATS 2010), volume 9, 358–365. Journal of Machine Learning Research Workshop and Conference Proceedings.
  • Jeffreys, H. (1998). The theory of probability. Oxford University Press.
  • Kalisch, M. and Bühlmann, P. (2008). “Robustification of the PC-algorithm for Directed Acyclic Graphs.” Journal of Computational and Graphical Statistics, 17(4): 773–789.
  • Korb, K. B. and Nicholson, A. E. (2003). Bayesian artificial intelligence. cRc Press.
  • Lauritzen, S. L. (1996). Graphical models. Oxford University Press.
  • Leonardi, N., Richiardi, J., Gschwind, M., Simioni, S., Annoni, J.-M., Schluep, M., Vuilleumier, P., and Van De Ville, D. (2013). “Principal components of functional connectivity: a new approach to study dynamic brain connectivity during rest.” NeuroImage, 83: 937–950.
  • Li, B., Daunizeau, J., Stephan, K. E., Penny, W., Hu, D., and Friston, K. (2011). “Generalised filtering and stochastic DCM for fMRI.” Neuroimage, 58(2): 442–457.
  • Marrelec, G., Krainik, A., Duffau, H., Pélégrini-Issac, M., Lehéricy, S., Doyon, J., and Benali, H. (2006). “Partial correlation for functional brain interactivity investigation in functional MRI.” Neuroimage, 32(1): 228–237.
  • Meek, C. (1995). “Causal inference and causal explanation with background knowledge.” In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, 403–410. Morgan Kaufmann Publishers Inc.
  • — (1997). “Graphical Models: Selecting causal and statistical models.” Ph.D. thesis, Carnegie Mellon University.
  • Patel, R. S., Bowman, F. D., and Rilling, J. K. (2006). “A Bayesian approach to determining connectivity of the human brain.” Human brain mapping, 27(3): 267–276.
  • Pearl, J. (2000). Causality: models, reasoning and inference, volume 29. Cambridge University Press.
  • — (2009). “Causal inference in statistics: An overview.” Statistics Surveys, 3: 96–146.
  • Penny, W., Ghahramani, Z., and Friston, K. (2005). “Bilinear dynamical systems.” Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1457): 983–993.
  • Pereda, E., Quiroga, R. Q., and Bhattacharya, J. (2005). “Nonlinear multivariate analysis of neurophysiological signals.” Progress in neurobiology, 77(1): 1–37.
  • Petris, G., Petrone, S., and Campagnoli, P. (2009). Dynamic linear models with R. Springer.
  • Poldrack, R. A., Mumford, J. A., and Nichols, T. E. (2011). Handbook of functional MRI data analysis. Cambridge University Press.
  • Queen, C. M. and Albers, C. J. (2009). “Intervention and causality: forecasting traffic flows using a dynamic Bayesian network.” Journal of the American Statistical Association, 104(486): 669–681.
  • Queen, C. M. and Smith, J. Q. (1993). “Multiregression dynamic models.” Journal of the Royal Statistical Society. Series B (Methodological), 55(4): 849–870.
  • Queen, C. M., Wright, B. J., and Albers, C. J. (2008). “Forecast covariances in the linear multiregression dynamic model.” Journal of Forecasting, 27(2): 175–191.
  • Quian Quiroga, R., Kraskov, A., Kreuz, T., and Grassberger, P. (2002). “Performance of different synchronization measures in real data: a case study on electroencephalographic signals.” Physical Review E, 65(4): 041903.
  • Ramsey, J. D., Hanson, S. J., Hanson, C., Halchenko, Y. O., Poldrack, R. A., and Glymour, C. (2010). “Six problems for causal inference from fMRI.” Neuroimage, 49(2): 1545–1558.
  • Roebroeck, A., Formisano, E., and Goebel, R. (2011). “The identification of interacting networks in the brain using fMRI: model selection, causality and deconvolution.” Neuroimage, 58(2): 296–302.
  • Ryali, S., Supekar, K., Chen, T., and Menon, V. (2011). “Multivariate dynamical systems models for estimating causal interactions in fMRI.” Neuroimage, 54(2): 807–823.
  • Salimi-Khorshidi, G., Douaud, G., Beckmann, C. F., Glasser, M. F., Griffanti, L., and Smith, S. M. (2014). “Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers.” NeuroImage, 90: 449–468.
  • Schwarz, G. et al. (1978). “Estimating the dimension of a model.” The annals of statistics, 6(2): 461–464.
  • Shimizu, S., Hoyer, P. O., Hyvärinen, A., and Kerminen, A. (2006). “A linear non-Gaussian acyclic model for causal discovery.” The Journal of Machine Learning Research, 7: 2003–2030.
  • Smith, J. (1985). “Diagnostic checks of non-standard time series models.” Journal of Forecasting, 4(3): 283–291.
  • Smith, J. F., Pillai, A., Chen, K., and Horwitz, B. (2010). “Identification and validation of effective connectivity networks in functional magnetic resonance imaging using switching linear dynamic systems.” Neuroimage, 52(3): 1027–1040.
  • — (2011a). “Effective connectivity modeling for fMRI: six issues and possible solutions using linear dynamic systems.” Frontiers in systems neuroscience, 5(104).
  • Smith, S. M., Bandettini, P. A., Miller, K. L., Behrens, T., Friston, K. J., David, O., Liu, T., Woolrich, M. W., and Nichols, T. E. (2012). “The danger of systematic bias in group-level FMRI-lag-based causality estimation.” Neuroimage, 59(2): 1228–1229.
  • Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., and Woolrich, M. W. (2011b). “Network modelling methods for FMRI.” Neuroimage, 54(2): 875–891.
  • Spirtes, P., Glymour, C. N., and Scheines, R. (2000). Causation, prediction, and search, volume 81. MIT press.
  • Steinsky, B. (2003). “Enumeration of labelled chain graphs and labelled essential directed acyclic graphs.” Discrete Mathematics, 270(1): 267–278.
  • Stephan, K. E., Kasper, L., Harrison, L. M., Daunizeau, J., den Ouden, H. E., Breakspear, M., and Friston, K. J. (2008). “Nonlinear dynamic causal models for fMRI.” Neuroimage, 42(2): 649–662.
  • Stephan, K. E., Penny, W. D., Moran, R. J., den Ouden, H. E., Daunizeau, J., and Friston, K. J. (2010). “Ten simple rules for dynamic causal modeling.” Neuroimage, 49(4): 3099–3109.
  • Valdés-Sosa, P. A., Roebroeck, A., Daunizeau, J., and Friston, K. (2011). “Effective connectivity: influence, causality and biophysical modeling.” Neuroimage, 58(2): 339–361.
  • West, M. and Harrison, P. J. (1997). Bayesian Forecasting and Dynamic Models. New York: Springer-Verlag, 2nd edition.
  • Wolsey, L. A. (1998). Integer Programming. John Wiley.
  • Zhang, J., Li, X., Li, C., Lian, Z., Huang, X., Zhong, G., Zhu, D., Li, K., Jin, C., Hu, X., et al. (2014). “Inferring functional interaction and transition patterns via dynamic Bayesian variable partition models.” Human brain mapping, 35: 3314–3331.