Bayesian Analysis

Efficient Bayesian Regularization for Graphical Model Selection

Suprateek Kundu, Bani K. Mallick, and Veera Baladandayuthapani

Full-text: Open access


There has been an intense development in the Bayesian graphical model literature over the past decade; however, most of the existing methods are restricted to moderate dimensions. We propose a novel graphical model selection approach for large dimensional settings where the dimension increases with the sample size, by decoupling model fitting and covariance selection. First, a full model based on a complete graph is fit under a novel class of mixtures of inverse–Wishart priors, which induce shrinkage on the precision matrix under an equivalence with Cholesky-based regularization, while enabling conjugate updates. Subsequently, a post-fitting model selection step uses penalized joint credible regions to perform model selection. This allows our methods to be computationally feasible for large dimensional settings using a combination of straightforward Gibbs samplers and efficient post-fitting inferences. Theoretical guarantees in terms of selection consistency are also established. Simulations show that the proposed approach compares favorably with competing methods, both in terms of accuracy metrics and computation times. We apply this approach to a cancer genomics data example.

Article information

Bayesian Anal., Volume 14, Number 2 (2019), 449-476.

First available in Project Euclid: 11 July 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

covariance selection Cholesky-based regularization joint penalized credible regions shrinkage priors selection consistency

Creative Commons Attribution 4.0 International License.


Kundu, Suprateek; Mallick, Bani K.; Baladandayuthapani, Veera. Efficient Bayesian Regularization for Graphical Model Selection. Bayesian Anal. 14 (2019), no. 2, 449--476. doi:10.1214/17-BA1086.

Export citation


  • Ambros, V. (2004). “The functions of animal microRNAs.” Nature, 431, 350–355.
  • Atay-Kayis, A. and Massam, H. (2005). “A Monte-Carlo Method for Computing the Marginal Likelihood in Nondecomposable Gaussian Graphical Models.” Biometrika, 92, 317–335.
  • Baladandayuthapani, V., Ji. Y., Talluri, R., Nieto-Barajas, L. E. and Morris, J. S. (2010). “Bayesian Random Segmentation Models to Identify Shared Copy Number Aberrations for Array CGH Data.” Journal of American Statistical Association, 105, 1358–1375.
  • Bondell, H. D. and Reich, B. J. (2012). “Consistent high-dimensional Bayesian variable selection via penalized credible regions.” Journal of the American Statistical Association, 107, 1610–1624.
  • Bickel, P. J. and Levina, E. (2008). “Covariance regularization by thresholding”. Annals of Statistics, 36(6), 2577–2604.
  • Cai, T. and Liu, W. (2012). “Adaptive Thresholding for Sparse Covariance Matrix Estimation”. Journal of the American Statistical Association, 106, 672–684.
  • Cancer Genome Atlas Research Network (2008). “Comprehensive genomic characterization defines human glioblastoma genes and core pathways.” Nature, 455, 1061-8.
  • Carvalho, C. M., Polson, N. G., and Scott, J. G. (2009). “Handling sparsity via the horseshoe.” Journal of Machine Learning Research W&CP, 5, 73–80.
  • Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97, 465–480.
  • Carvalho, C. M. and Scott, J. G. (2009). “Objective Bayesian model selection in Gaussian graphical models.” Biometrika, 96(3), 497–512.
  • Chan, J. C. and Jeliazkov, I. (2009). “Estimation of Restricted Covariance Matrices.” Journal of the Computational and Graphical Statistics, 18(2), 457–480.
  • Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper markov Laws in the Statistical Analysis of Decomposable Graphical Models.” Annals of Statistics, 21, 1272–1317.
  • Dellaportas, P., Giudici, P. and Roberts, G. (2003). “Bayesian inference for non-decomposable graphical Gaussian models.” Sankhyā, 65, 43–55.
  • Delfino K. R., Serão, N. V., Southey, B. R., Rodriguez-Zas, S. L. (2011). “Therapy-, gender- and race-specific microRNA markers, target genes and networks related to glioblastoma recurrence and survival.” Cancer Genomics Proteonomics, 8, 173–183.
  • Dempster, A. P. (1972). “Covariance Selection.” Biometrics. 28, 157–175.
  • Diaconis, P. and Ylvisaker, D. (1979). “Conjugate Priors for Exponential Families.” Annals of Statistics, 7, 269–281.
  • Dong, H., Luo, L., Hong, S., Siu, H., Xiao, Y., Jin, L., Chen, R., and Xiong, M. (2010). “Integrated analysis of mutations, miRNA and mRNA expression in glioblastoma.” BMC Systems Biology, 4, 163.
  • Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). “Least angle regression.” The Annals of Statistics, 32, 407–499.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2008). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9, 432–441.
  • Fitch, M. A., Jones, M. B., and Massam, H. (2014). “The Performance of Covariance Selection Methods That Consider Decomposable Models Only.” Bayesian Analysis, 9, 659–684.
  • Fouskakis, D.; Ntzoufras, I.; Draper, D. (2009). “Bayesian variable selection using cost-adjusted BIC, with application to cost-effective measurement of quality of health care.” Annals Applied Statistics 3, 663–690.
  • Frühwirth-Schnatter, S. and Tüchler, R. (2008). “Bayesian parsimonious covariance estimation for hierarchical linear mixed models.” Statistics and Computing, 18, 1–13.
  • George, E. I. and McCulloch, R. (1993). “Variable Selection via Gibbs Sampling.” Journal of the American Statistical Association, 88, 881–889.
  • Giudici, P. and Green, P. J. (1999). “Decomposable Graphical Gaussian Model Determination.” Biometrika, 86, 785–801.
  • Green, P. J. and Thomas, A. (2013). “Sampling decomposable graphs using a Markov chain on junction trees.” Biometrika, 100, 91–110.
  • Hahn, P. R. and Carvalho, C. M. (2015). “Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective.” Journal of the American Statistical Association, 110, 435–448.
  • Herranz, H. and Cohen, S. M. (2010). “MicroRNAs and gene regulatory networks: managing the impact of noise in biological systems.” Genes and Development, 24, 1339–44.
  • Huang, A. and Wand, M. P. (2013). “Simple marginally non-informative prior distributions for covariance matrices.” Bayesian Analysis, 8, 439–452.
  • Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in Stochastic Computation for High-dimensional Graphical Models.” Statistical Science, 20, 388–400.
  • Kundu, S. and Dunson, D. B. (2014). “Bayes variable selection in semi-parametric linear models.” Journal of the American Statistical Association, 109, 437–447.
  • Kundu, S., Mallick, B. K., and Baladandayuthapani, V. (2018). “Supplementary Materials for “Efficient Bayesian Regularization for Graphical Model Selection”.” Bayesian Analysis.
  • Lee, S. T., Chu, K., Oh, H. J., Im, W. S., Lim, J. Y., Kim, S. K., Park, C. K., Jung, K. H., Lee, S. K., Kim, M., and Roh, J. K. (2011). “Let-7 microRNA inhibits the proliferation of human glioblastoma cells”, Journal of Neuro-Oncology, 102, 19–24.
  • Lenkoski, A. and Dobra, A. (2011). “Computational aspects related to inference in Gaussian graphical models with the G-Wishart prior.” Journal of Computational and Graphical Statistics, 20, 140–157.
  • Lewis, B. P., Burge, C. B., Bartel, D. P. (2005). “Conserved Seed Pairing, Often Flanked by Adenosines, Indicates that Thousands of Human Genes are MicroRNA Targets.” Cell, 120, 15–20.
  • Lv, J. and Fan, Y. (2009). “A unified approach to model selection and sparse recovery using regularized least squares.” Annals of Statistics, 37, 3498–3528.
  • Meinshausen, N. and Bühlmann, P. (2006). “High-dimensional Graphs and Variable Selection with the Lasso”, Annals of Statistics, 34, 1436–1462.
  • Mohammadi, A. and Wit, E. C. (2015). “Bayesian Structure Learning in Sparse Gaussian Graphical Models.” Bayesian Analysis, 10, 109–138.
  • Monti, R. P., Hellyer, P., Sharp, D., Leech, R., Anagnostopoulos, C., Montana, G. (2014). “Estimating time-varying brain connectivity networks from functional MRI time series.” NeuroImage, 103, 427–443.
  • Morris, J. S., Brown, P. J., Herrick, R. C., Baggerly, K. A. and Coombes, K. R. (2008). “Bayesian analysis of mass spectrometry proteomic data using wavelet-based functional mixed models.” Biometrics, 64, 479–489.
  • Newton, M. A., Noueiry, A., Sarkar, D. and Ahlquist, P. (2004). “Detecting differential gene expression with a semiparametric hierarchical mixture method.” Biostatistics, 5, 155–176.
  • Peng, J., Wang, P., Zhou, N., and Zhu, J. (2009). “Partial Correlation Estimation by Joint Sparse Regression Models.” Journal of the American Statistical Association, 104, 735–746.
  • Polson, N. G. and Scott, J. (2011). “Shrink globally, act locally: sparse Bayesian regularization and prediction.” In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.), pages 501–538. Oxford University Press, New York.
  • Pourahmadi, M. (1999). “Joint mean-covariance models with applications to longitudinal data: unconstrained parameterisation.” Biometrika, 86, 677–690.
  • Roverato, A. (2000). “Cholesky decomposition of a hyper inverse Wishart matrix.” Biometrika, 87, 99–112.
  • Scott, J. G. and Carvalho, C. M. (2008). “Feature-Inclusion Stochastic Search for Gaussian Graphical Models.” Journal of Computational and Graphical Statistics, 17, 790–808.
  • Smith, M. and Kohn, R. (2002). “Parsimonious covariance matrix estimation for longitudinal data.” Journal of the American Statistical Association, 97, 1141–1153.
  • Tang, W., Duan, J., Zhang, J. G., and Wang, Y. P. (2013). “Subtyping glioblastoma by combining miRNA and mRNA expression data using compressed sensing-based approach.” EURASIP Journal on Bioinformatics and Systems Biology, 2.
  • Tibshirani, R. J. (2013). “The lasso problem and uniqueness.” Electronic Journal of Statistics, 7, 1456–1490.
  • Wang, H. (2012). “Bayesian Graphical Lasso Models and Efficient Posterior Computation.” Bayesian Analysis, 7, 771–790.
  • Wang, H. and West, M. (2009). “Bayesian analysis of matrix normal graphical models.” Biometrika, 96, 821–834.
  • Wong, A. J., Ruppert, J. M., Bigner, S. H., Grzeschik, C. H., Humphrey, P. A., Bigner, D. S., and Vogelstein, B. (1992). “Structural alterations of the epidermal growth factor receptor gene in human gliomas.” Proceedings of the National Academy of Sciences of the United States of America, 89, 2965–2969.
  • Wong, F., Carter, C., and Kohn, R. (2003). “Efficient Estimation of Covariance Selection Models.” Biometrika, 90, 809–830.
  • Wu, W. B. and Pourahmadi, M. (2003). “Nonparametric estimation of large covariance matrices of longitudinal data.” Biometrika, 90, 831–44.
  • Yuan, M. and Lin, Y. (2007). “Model selection and estimation in the Gaussian graphical model.” Biometrika, 94, 19–35.
  • Zou, H. and Li, R. (2008). “One-step sparse estimates in nonconcave penalized likelihood models (with discussion).” Annals of Statistics, 36, 1509–1566.

Supplemental materials