Bayesian Analysis

Bayesian Graphical Lasso Models and Efficient Posterior Computation

Hao Wang

Full-text: Open access

Abstract

Recently, the graphical lasso procedure has become popular in estimating Gaussian graphical models. In this paper, we introduce a fully Bayesian treatment of graphical lasso models. We first investigate the graphical lasso prior that has been relatively unexplored. Using data augmentation, we develop a simple but highly efficient block Gibbs sampler for simulating covariance matrices. We then generalize the Bayesian graphical lasso to the Bayesian adaptive graphical lasso. Finally, we illustrate and compare the results from our approach to those obtained using the standard graphical lasso procedures for real and simulated data. In terms of both covariance matrix estimation and graphical structure learning, the Bayesian adaptive graphical lasso appears to be the top overall performer among a range of frequentist and Bayesian methods.

Article information

Source
Bayesian Anal., Volume 7, Number 4 (2012), 867-886.

Dates
First available in Project Euclid: 27 November 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1354024465

Digital Object Identifier
doi:10.1214/12-BA729

Mathematical Reviews number (MathSciNet)
MR3000017

Zentralblatt MATH identifier
1330.62041

Keywords
Adaptive graphical lasso Block Gibbs sampler Constrained parameter spaces Covariance matrix estimation Double-exponential distribution Graphical lasso

Citation

Wang, Hao. Bayesian Graphical Lasso Models and Efficient Posterior Computation. Bayesian Anal. 7 (2012), no. 4, 867--886. doi:10.1214/12-BA729. https://projecteuclid.org/euclid.ba/1354024465


Export citation

References

  • Andrews, D. F. and Mallows, C. L. (1974). “Scale Mixtures of Normal Distributions.” Journal of the Royal Statistical Society. Series B (Methodological), 36(1): pp. 99–102.
  • Armagan, A., Dunson, D., and Lee, J. (2012). “Generalized double Pareto shrinkage.” Statistica Sinica (forthcoming).
  • Atay-Kayis, A. and Massam, H. (2005). “A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models.” Biometrika, 92: 317–335.
  • Banerjee, O., El Ghaoui, L., and d’Aspremont, A. (2008). “Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data.” The Journal of Machine Learning Research, 9: 485–516.
  • Barnard, J., McCulloch, R., and Meng, X.-L. (2000). “Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage.” Statistica Sinica, 10(4): 1281–1311.
  • Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97(2): 465–480.
  • Daniels, M. J. and Kass, R. E. (1999). “Nonconjugate Bayesian Estimation of Covariance Matrices and Its Use in Hierarchical Models.” Journal of the American Statistical Association, 94(448): pp. 1254–1263.
  • — (2001). “Shrinkage Estimators for Covariance Matrices.” Biometrics, 57(4): 1173–1184.
  • Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper-Markov laws in the statistical analysis of decomposable graphical models.” Annals of Statistics, 21: 1272–1317.
  • Fan, J., Feng, Y., and Wu, Y. (2009). “Network exploration via the adaptive LASSO and SCAD penalties.” Annals of Applied Statistics, 3(2): 521–541.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2008). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9(3): 432–441.
  • Griffin, J. E. and Brown, P. (2010). “Inference with normal-gamma prior distributions in regression problems.” Bayesian Analysis, 5(1): 171–188.
  • Guo, J., Levina, E., Michailidis, G., and Zhu, J. (2011). “Joint estimation of multiple graphical models.” Biometrika, 98(1): 1–15.
  • Hans, C. (2009). “Bayesian lasso regression.” Biometrika, 96(4): 835–845.
  • Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in stochastic computation for high-dimensional graphical models.” Statistical Science, 20: 388–400.
  • Khondker, Z., Zhu, H., Chu, H., Lin, W., and Ibrahim, J. (2012). “Bayesian covariance lasso.” Statistics and Its Interface (forthcoming).
  • Kyung, M., Gill, J., Ghosh, M., and Casella, G. (2010). “Penalized Regression, Standard Errors, and Bayesian Lassos.” Bayesian Analysis, 5(2): 369–412.
  • Li, Q. and Lin, N. (2010). “The Bayesian Elastic Net.” Bayesian Analysis, 5(1): 151–170.
  • Liechty, J. C., Liechty, M. W., and Müller, P. (2004). “Bayesian correlation estimation.” Biometrika, 91(1): 1–14.
  • Liechty, M. W., Liechty, J. C., and Müller, P. (2009). “The Shadow Prior.” Journal of Computational and Graphical Statistics, 18(2): 368–383.
  • Marlin, B. M. and Murphy, K. P. (2009). “Sparse Gaussian graphical models with unknown block structure.” In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, 705–712. New York, NY, USA: ACM.
  • Marlin, B. M., Schmidt, M., and Murphy, K. P. (2009). “Group Sparse Priors for Covariance Estimation.” In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence.
  • Meinshausen, N. and Bühlmann, P. (2006). “High-dimensional graphs and variable selection with the lasso.” The Annals of Statistics, 34(3): 1436–1462.
  • Park, T. and Casella, G. (2008). “The Bayesian Lasso.” Journal of the American Statistical Association, 103(482): 681–686.
  • Rothman, A. J., Bickel, P. J., Levina, E., and Zhu, J. (2008). “Sparse permutation invariant covariance estimation.” Electronic Journal of Statistics, 2: 494–515.
  • Roverato, A. (2000). “Cholesky decomposition of a hyper-inverse Wishart matrix.” Biometrika, 87: 99–112.
  • Sachs, K., Perez, O., Pe’er, D., Lauffenburger, D. A., and Nolan, G. P. (2005). “Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data.” Science, 308(5721): 523–529.
  • Wang, H. and Li, S. Z. (2012). “Efficient Gaussian graphical model determination under G-Wishart prior distributions.” Electronic Journal of Statistics, 6: 168–198.
  • West, M. (1987). “On Scale Mixtures of Normal Distributions.” Biometrika, 74(3): pp. 646–648.
  • Wong, F., Carter, C., and Kohn, R. (2003). “Efficient estimation of covariance selection models.” Biometrika, 90: 809–30.
  • Yang, R. and Berger, J. O. (1994). “Estimation of a Covariance Matrix Using the Reference Prior.” The Annals of Statistics, 22(3): pp. 1195–1211.
  • Yuan, M. and Lin, Y. (2007). “Model selection and estimation in the Gaussian graphical model.” Biometrika, 94(1): 19–35.