Bayesian Analysis
- Bayesian Anal.
- Volume 7, Number 4 (2012), 867-886.
Bayesian Graphical Lasso Models and Efficient Posterior Computation
Full-text: Open access
Abstract
Recently, the graphical lasso procedure has become popular in estimating Gaussian graphical models. In this paper, we introduce a fully Bayesian treatment of graphical lasso models. We first investigate the graphical lasso prior that has been relatively unexplored. Using data augmentation, we develop a simple but highly efficient block Gibbs sampler for simulating covariance matrices. We then generalize the Bayesian graphical lasso to the Bayesian adaptive graphical lasso. Finally, we illustrate and compare the results from our approach to those obtained using the standard graphical lasso procedures for real and simulated data. In terms of both covariance matrix estimation and graphical structure learning, the Bayesian adaptive graphical lasso appears to be the top overall performer among a range of frequentist and Bayesian methods.
Article information
Source
Bayesian Anal., Volume 7, Number 4 (2012), 867-886.
Dates
First available in Project Euclid: 27 November 2012
Permanent link to this document
https://projecteuclid.org/euclid.ba/1354024465
Digital Object Identifier
doi:10.1214/12-BA729
Mathematical Reviews number (MathSciNet)
MR3000017
Zentralblatt MATH identifier
1330.62041
Keywords
Adaptive graphical lasso Block Gibbs sampler Constrained parameter spaces Covariance matrix estimation Double-exponential distribution Graphical lasso
Citation
Wang, Hao. Bayesian Graphical Lasso Models and Efficient Posterior Computation. Bayesian Anal. 7 (2012), no. 4, 867--886. doi:10.1214/12-BA729. https://projecteuclid.org/euclid.ba/1354024465
References
- Andrews, D. F. and Mallows, C. L. (1974). “Scale Mixtures of Normal Distributions.” Journal of the Royal Statistical Society. Series B (Methodological), 36(1): pp. 99–102.Mathematical Reviews (MathSciNet): MR359122
- Armagan, A., Dunson, D., and Lee, J. (2012). “Generalized double Pareto shrinkage.” Statistica Sinica (forthcoming).
- Atay-Kayis, A. and Massam, H. (2005). “A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models.” Biometrika, 92: 317–335.Mathematical Reviews (MathSciNet): MR2201362
Zentralblatt MATH: 1094.62028
Digital Object Identifier: doi:10.1093/biomet/92.2.317 - Banerjee, O., El Ghaoui, L., and d’Aspremont, A. (2008). “Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data.” The Journal of Machine Learning Research, 9: 485–516.
- Barnard, J., McCulloch, R., and Meng, X.-L. (2000). “Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage.” Statistica Sinica, 10(4): 1281–1311.
- Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97(2): 465–480.Mathematical Reviews (MathSciNet): MR2650751
Zentralblatt MATH: 05773446
Digital Object Identifier: doi:10.1093/biomet/asq017 - Daniels, M. J. and Kass, R. E. (1999). “Nonconjugate Bayesian Estimation of Covariance Matrices and Its Use in Hierarchical Models.” Journal of the American Statistical Association, 94(448): pp. 1254–1263.Mathematical Reviews (MathSciNet): MR1731487
Zentralblatt MATH: 1069.62508
Digital Object Identifier: doi:10.1080/01621459.1999.10473878 - — (2001). “Shrinkage Estimators for Covariance Matrices.” Biometrics, 57(4): 1173–1184.Mathematical Reviews (MathSciNet): MR1950425
Digital Object Identifier: doi:10.1111/j.0006-341X.2001.01173.x - Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper-Markov laws in the statistical analysis of decomposable graphical models.” Annals of Statistics, 21: 1272–1317.Mathematical Reviews (MathSciNet): MR1241267
Zentralblatt MATH: 0815.62038
Digital Object Identifier: doi:10.1214/aos/1176349260
Project Euclid: euclid.aos/1176349260 - Fan, J., Feng, Y., and Wu, Y. (2009). “Network exploration via the adaptive LASSO and SCAD penalties.” Annals of Applied Statistics, 3(2): 521–541.Mathematical Reviews (MathSciNet): MR2750671
Zentralblatt MATH: 1166.62040
Digital Object Identifier: doi:10.1214/08-AOAS215
Project Euclid: euclid.aoas/1245676184 - Friedman, J., Hastie, T., and Tibshirani, R. (2008). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9(3): 432–441.
- Griffin, J. E. and Brown, P. (2010). “Inference with normal-gamma prior distributions in regression problems.” Bayesian Analysis, 5(1): 171–188.
- Guo, J., Levina, E., Michailidis, G., and Zhu, J. (2011). “Joint estimation of multiple graphical models.” Biometrika, 98(1): 1–15.Mathematical Reviews (MathSciNet): MR2804206
Zentralblatt MATH: 1214.62058
Digital Object Identifier: doi:10.1093/biomet/asq060 - Hans, C. (2009). “Bayesian lasso regression.” Biometrika, 96(4): 835–845.Mathematical Reviews (MathSciNet): MR2564494
Zentralblatt MATH: 1179.62038
Digital Object Identifier: doi:10.1093/biomet/asp047 - Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in stochastic computation for high-dimensional graphical models.” Statistical Science, 20: 388–400.Mathematical Reviews (MathSciNet): MR2210226
Digital Object Identifier: doi:10.1214/088342305000000304
Project Euclid: euclid.ss/1137076659 - Khondker, Z., Zhu, H., Chu, H., Lin, W., and Ibrahim, J. (2012). “Bayesian covariance lasso.” Statistics and Its Interface (forthcoming).
- Kyung, M., Gill, J., Ghosh, M., and Casella, G. (2010). “Penalized Regression, Standard Errors, and Bayesian Lassos.” Bayesian Analysis, 5(2): 369–412.
- Li, Q. and Lin, N. (2010). “The Bayesian Elastic Net.” Bayesian Analysis, 5(1): 151–170.
- Liechty, J. C., Liechty, M. W., and Müller, P. (2004). “Bayesian correlation estimation.” Biometrika, 91(1): 1–14.Mathematical Reviews (MathSciNet): MR2050456
Zentralblatt MATH: 1132.62314
Digital Object Identifier: doi:10.1093/biomet/91.1.1 - Liechty, M. W., Liechty, J. C., and Müller, P. (2009). “The Shadow Prior.” Journal of Computational and Graphical Statistics, 18(2): 368–383.
- Marlin, B. M. and Murphy, K. P. (2009). “Sparse Gaussian graphical models with unknown block structure.” In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, 705–712. New York, NY, USA: ACM.
- Marlin, B. M., Schmidt, M., and Murphy, K. P. (2009). “Group Sparse Priors for Covariance Estimation.” In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence.
- Meinshausen, N. and Bühlmann, P. (2006). “High-dimensional graphs and variable selection with the lasso.” The Annals of Statistics, 34(3): 1436–1462.Mathematical Reviews (MathSciNet): MR2278363
Zentralblatt MATH: 1113.62082
Digital Object Identifier: doi:10.1214/009053606000000281
Project Euclid: euclid.aos/1152540754 - Park, T. and Casella, G. (2008). “The Bayesian Lasso.” Journal of the American Statistical Association, 103(482): 681–686.Mathematical Reviews (MathSciNet): MR2524001
Zentralblatt MATH: 05564521
Digital Object Identifier: doi:10.1198/016214508000000337 - Rothman, A. J., Bickel, P. J., Levina, E., and Zhu, J. (2008). “Sparse permutation invariant covariance estimation.” Electronic Journal of Statistics, 2: 494–515.Mathematical Reviews (MathSciNet): MR2417391
Digital Object Identifier: doi:10.1214/08-EJS176
Project Euclid: euclid.ejs/1214491853 - Roverato, A. (2000). “Cholesky decomposition of a hyper-inverse Wishart matrix.” Biometrika, 87: 99–112.Mathematical Reviews (MathSciNet): MR1766831
Zentralblatt MATH: 0974.62047
Digital Object Identifier: doi:10.1093/biomet/87.1.99 - Sachs, K., Perez, O., Pe’er, D., Lauffenburger, D. A., and Nolan, G. P. (2005). “Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data.” Science, 308(5721): 523–529.
- Wang, H. and Li, S. Z. (2012). “Efficient Gaussian graphical model determination under G-Wishart prior distributions.” Electronic Journal of Statistics, 6: 168–198.Mathematical Reviews (MathSciNet): MR2879676
Digital Object Identifier: doi:10.1214/12-EJS669
Project Euclid: euclid.ejs/1328280902 - West, M. (1987). “On Scale Mixtures of Normal Distributions.” Biometrika, 74(3): pp. 646–648.Mathematical Reviews (MathSciNet): MR909372
Zentralblatt MATH: 0648.62015
Digital Object Identifier: doi:10.1093/biomet/74.3.646 - Wong, F., Carter, C., and Kohn, R. (2003). “Efficient estimation of covariance selection models.” Biometrika, 90: 809–30.
- Yang, R. and Berger, J. O. (1994). “Estimation of a Covariance Matrix Using the Reference Prior.” The Annals of Statistics, 22(3): pp. 1195–1211.Mathematical Reviews (MathSciNet): MR1311972
Zentralblatt MATH: 0819.62013
Digital Object Identifier: doi:10.1214/aos/1176325625
Project Euclid: euclid.aos/1176325625 - Yuan, M. and Lin, Y. (2007). “Model selection and estimation in the Gaussian graphical model.” Biometrika, 94(1): 19–35.Mathematical Reviews (MathSciNet): MR2367824
Zentralblatt MATH: 1142.62408
Digital Object Identifier: doi:10.1093/biomet/asm018
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Bayesian regularized quantile regression
Lin, Nan, Li, Qing, and Xi, Ruibin, Bayesian Analysis, 2010 - Trace class Markov chains for the Normal-Gamma Bayesian shrinkage model
Zhang, Liyuan, Khare, Kshitij, and Xing, Zeren, Electronic Journal of Statistics, 2019 - Model selection and adaptive Markov chain Monte Carlo for Bayesian cointegrated {VAR}
models
Kannan, Balakrishnan, Lasscock, Ben, Mellen, Chris, and Peters, Gareth W., Bayesian Analysis, 2010
- Bayesian regularized quantile regression
Lin, Nan, Li, Qing, and Xi, Ruibin, Bayesian Analysis, 2010 - Trace class Markov chains for the Normal-Gamma Bayesian shrinkage model
Zhang, Liyuan, Khare, Kshitij, and Xing, Zeren, Electronic Journal of Statistics, 2019 - Model selection and adaptive Markov chain Monte Carlo for Bayesian cointegrated {VAR}
models
Kannan, Balakrishnan, Lasscock, Ben, Mellen, Chris, and Peters, Gareth W., Bayesian Analysis, 2010 - Cross-Fertilizing Strategies for Better EM Mountain Climbing and DA Field Exploration: A Graphical Guide Book
van Dyk, David A. and Meng, Xiao-Li, Statistical Science, 2010 - Penalized regression, standard errors, and Bayesian lassos
Casella, George, Ghosh, Malay, Gill, Jeff, and Kyung, Minjung, Bayesian Analysis, 2010 - Bayesian Quantile Regression with Mixed Discrete and Nonignorable Missing Covariates
Wang, Zhi-Qiang and Tang, Nian-Sheng, Bayesian Analysis, 2020 - Bayesian bootstrap for proportional hazards models
Kim, Yongdai and Lee, Jaeyong, Annals of Statistics, 2003 - Smoothing ℓ1-penalized estimators for high-dimensional time-course data
Meier, Lukas and Bühlmann, Peter, Electronic Journal of Statistics, 2007 - Adaptive Approximate Bayesian Computation Tolerance Selection
Simola, Umberto, Cisewski-Kehe, Jessi, Gutmann, Michael U., and Corander, Jukka, Bayesian Analysis, 2020 - The Matrix-F Prior for Estimating and Testing Covariance Matrices
Mulder, Joris and Pericchi, Luis Raúl, Bayesian Analysis, 2018