Electronic Journal of Statistics

Hierarchical Gaussian graphical models: Beyond reversible jump

Yuan Cheng and Alex Lenkoski

Full-text: Open access

Abstract

The Gaussian Graphical Model (GGM) is a popular tool for incorporating sparsity into joint multivariate distributions. The G-Wishart distribution, a conjugate prior for precision matrices satisfying general GGM constraints, has now been in existence for over a decade. However, due to the lack of a direct sampler, its use has been limited in hierarchical Bayesian contexts, relegating mixing over the class of GGMs mostly to situations involving standard Gaussian likelihoods. Recent work has developed methods that couple model and parameter moves, first through reversible jump methods and later by direct evaluation of conditional Bayes factors and subsequent resampling. Further, methods for avoiding prior normalizing constant calculations–a serious bottleneck and source of numerical instability–have been proposed. We review and clarify these developments and then propose a new methodology for GGM comparison that blends many recent themes. Theoretical developments and computational timing experiments reveal an algorithm that has limited computational demands and dramatically improves on computing times of existing methods. We conclude by developing a parsimonious multivariate stochastic volatility model that embeds GGM uncertainty in a larger hierarchical framework. The method is shown to be capable of adapting to swings in market volatility, offering improved calibration of predictive distributions.

Article information

Source
Electron. J. Statist. Volume 6 (2012), 2309-2331.

Dates
First available in Project Euclid: 30 November 2012

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1354284421

Digital Object Identifier
doi:10.1214/12-EJS746

Mathematical Reviews number (MathSciNet)
MR3020264

Zentralblatt MATH identifier
1335.62042

Keywords
Gaussian graphical models G-Wishart distribution conditional Bayes factors exchange algorithms

Citation

Cheng, Yuan; Lenkoski, Alex. Hierarchical Gaussian graphical models: Beyond reversible jump. Electron. J. Statist. 6 (2012), 2309--2331. doi:10.1214/12-EJS746. https://projecteuclid.org/euclid.ejs/1354284421


Export citation

References

  • Amestoy, P. R., Davis, T. A., and Duff, I. S. (2004). Algorithm 837: AMD, an approximate minimum degree ordering algorithm., ACM Transactions on Mathematical Software, 30:381–388.
  • Atay-Kayis, A. and Massam, H. (2005). A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models., Biometrika, 92:317–335.
  • Carvalho, C. M. and West, M. (2007). Dynamic matrix-variate graphical models., Bayesian Analysis, 2:69–98.
  • Dawid, A. P. and Lauritzen, S. L. (1993). Hyper Markov laws in the statistical analysis of decomposable graphical models., Ann. Statist., 21:1272–1317.
  • Dempster, A. P. (1972). Covariance selection., Biometrics, 28:157–175.
  • Dickey, J. M. and Gunel, E. (1978). Bayes factors from mixed probabilities., J. R. Statist. Soc. B, 40:43–46.
  • Dobra, A. and Lenkoski, A. (2011). Copula Gaussian graphical models and their application to modeling functional disability data., Annals of Applied Statistics, 5:969–993.
  • Dobra, A., Lenkoski, A., and Rodriguez, A. (2011). Bayesian inference for general Gaussian graphical models with application to multivariate lattice data., Journal of the American Statistical Association, 106:1418–1433.
  • Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction and estimation., Journal of the American Statistical Association, 102:359–378.
  • Gneiting, T., Raftery, A. E., Westveld, A. H., and Goldman, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation., Monthly Weather Review, 133:1098–1118.
  • Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination., Biometrika, 82(711-732).
  • Jacquier, E., Polson, N. G., and Rossi, P. E. (1994). Bayesian analysis of stochastic volatility models., Journal of Business and Economic Statistics, 12:371–389.
  • Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). Experiments in stochastic computation for high-dimensional graphical models., Statistical Science, 20:388–400.
  • Lenkoski, A. and Dobra, A. (2011). Computational aspects related to inference in Gaussian graphical models with the G-Wishart prior., Journal of Computational and Graphical Statistics, 20:140–157.
  • Letac, G. and Massam, H. (2007). Wishart distributions for decomposable graphs., Ann. Statist., 35:1278–323.
  • Liang, F. (2010). A double Metropolis-Hastings sampler for spatial models with intractable normalizing constants., Journal of Statistical Computing and Simulation, 80:1007–1022.
  • Mitsakakis, N., Massam, H., and Escobar, M. D. (2011). A Metropolis-Hastings based method for sampling from the G-Wishart distribution in Gaussian graphical models., Electronic Journal of Statistics, 5:18–30.
  • Murray, I., Ghahramani, Z., and MacKay, D. (2006). MCMC for doubly-intractable distributions., Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence.
  • Piccioni, M. (2000). Independence structure of natural conjugate densities to exponential families and the Gibbs sampler., Scand. J. Statist., 27:111–27.
  • Raftery, A. E., Gneiting, T., Balabdaoui, F., and Polakowski, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles., Monthly Weather Review, 133:1155–1174.
  • Rajaratnam, B., Massam, H., and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models., Ann. Statist., 36:2818–2849.
  • Rodriguez, A., Dobra, A., and Lenkoski, A. (2011). Sparse covariance estimation in heterogeneous samples., Electronic Journal of Statistics, 5:981–1014.
  • Roverato, A. (2002). Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models., Scand. J. Statist., 29:391–411.
  • Rue, H. (2001). Fast sampling of Gaussian Markov random fields., Journal of the Royal Statistical Society, Series B, 63:325–338.
  • Rue, H. and Held, L. (2005)., Gaussian Markov Random Fields. Chapman & Hall.
  • Thorarinsdottir, T. L. and Gneiting, T. (2010). Probabilistic forecasts of wind speed: Ensemble model output statistics using heteroskedastic censored regression., Journal of the Royal Statistical Society Ser. A, 173:371–388.
  • Wang, H. and Carvalho, C. M. (2010). Simulation of hyper-inverse Wishart distributions for non-decomposable graphs., Electronic Journal of Statistics, 4:1470–1475.
  • Wang, H. and Li, S. Z. (2012). Efficient Gaussian graphical model determination under G-Wishart prior distributions., Electronic Journal of Statistics, 6:168–198.
  • Wang, H., Reeson, C., and Carvalho, C. M. (2011). Dynamic financial index models: Modeling conditional dependencies via graphs., Bayesian Analysis, 6:639–664.
  • West, M. and Harrison, J. (1997)., Bayesian Forecasting and Dynamic Models. Springer.