Bayesian Analysis

Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models

Ioannis Ntzoufras, Claudia Tarantola, and Monia Lupparelli

Full-text: Open access


We introduce a novel Bayesian approach for quantitative learning for graphical log-linear marginal models. These models belong to curved exponential families that are difficult to handle from a Bayesian perspective. The likelihood cannot be analytically expressed as a function of the marginal log-linear interactions, but only in terms of cell counts or probabilities. Posterior distributions cannot be directly obtained, and Markov Chain Monte Carlo (MCMC) methods are needed. Finally, a well-defined model requires parameter values that lead to compatible marginal probabilities. Hence, any MCMC should account for this important restriction. We construct a fully automatic and efficient MCMC strategy for quantitative learning for such models that handles these problems. While the prior is expressed in terms of the marginal log-linear interactions, we build an MCMC algorithm that employs a proposal on the probability parameter space. The corresponding proposal on the marginal log-linear interactions is obtained via parameter transformation. We exploit a conditional conjugate setup to build an efficient proposal on probability parameters. The proposed methodology is illustrated by a simulation study and a real dataset.

Article information

Bayesian Anal., Volume 14, Number 3 (2019), 777-803.

First available in Project Euclid: 11 June 2019

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

graphical models marginal log-linear parameterisation Markov Chain Monte Carlo computation

Creative Commons Attribution 4.0 International License.


Ntzoufras, Ioannis; Tarantola, Claudia; Lupparelli, Monia. Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models. Bayesian Anal. 14 (2019), no. 3, 777--803. doi:10.1214/18-BA1128.

Export citation


  • Aitchison, J. and Silvey, S. D. (1958). “Maximum-Likelihood Estimation of Parameters Subject to Restraints.” The Annals of Mathematical Statistics, 29: 813–828.
  • Bartolucci, F., Scaccia, L., and Farcomeni, A. (2012). “Bayesian inference through encompassing priors and importance sampling for a class of marginal models for categorical data.” Computational Statistics & Data Analysis, 56: 4067–4080.
  • Bergsma, W. O., Croon, M. A., and Hagenaars, J. A. (2009). Marginal Models: For Dependent, Clustered, and Longitudinal Categorical Data. Springer.
  • Bergsma, W. P. and Rudas, T. (2002). “Marginal Models for Categorical Data.” The Annals of Statistics, 30: 140–159.
  • Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Multivariate Analysis, Theory and Practice. MIT press.
  • Cox, D. R. and Wermuth, N. (1993). “Linear Dependencies Represented by Chain Graphs (with discussion).” Statistical Science, 8: 204–218, 247–277.
  • Dellaportas, P. and Forster, J. J. (1999). “Markov Chain Monte Carlo Model Determination for Hierarchical and Graphical Log-Linear Models.” Biometrika, 86: 615–633.
  • Drton, M. and Richardson, T. S. (2008). “Binary models for marginal independence.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70: 287–309.
  • Evans, R. J. and Richardson, T. S. (2013). “Marginal log-linear parameters for graphical Markov models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75: 743–768.
  • Evans, R. J. (2016). “ Graphs for margins of Bayesian networks.” Scandinavian Journal of Statistics, 43: 625–648.
  • Forcina, A., Lupparelli, M., and Marchetti, G. M. (2010). “Marginal parameterizations of discrete models defined by a set of conditional independencies.” Journal of Multivariate Analysis, 101: 2519–2527.
  • Lupparelli, M. (2006). “Graphical models of marginal independence for categorical variables.” Ph.D. thesis, University of Florence.
  • Lupparelli, M., Marchetti, G., and Bergsma, W. P. (2009). “Parameterizations and Fitting of Bi-directed Graph Models to Categorical Data.” Scandinavian Journal of Statistics, 36: 559–576.
  • Muller, T. P. and Mayhall, J. T. (1971). “Analysis of contingency table data on torus mandibularis using a log linear model.” American Journal of Physical Anthropology, 34: 149–153.
  • Ntzoufras, I. and Tarantola, C. (2013). “Conjugate and conditional conjugate Bayesian analysis of discrete graphical models of marginal independence.” Computational Statistics & Data Analysis, 66: 161–177.
  • Ntzoufras, I., Tarantola, C., and Lupparelli, M. (2018). “ Supplementary Material: Electronic Appendix for the Paper Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models.” Bayesian Analysis.
  • Pearl, J. and Wermuth, N. (1994). “When can association graphs admit a causal interpretation?” In P., C. and R.W., O. (eds.), Selecting Models from Data. Lecture Notes in Statistics, volume 89, 205–214. New York, NY: Springer.
  • Plummer, M., Best, N., Cowles, K., and Vines, K. (2006). “CODA: convergence diagnosis and output analysis for MCMC.” R News, 6: 7–11. URL
  • Raftery, A. and Lewis, S. (1992). “One long run with diagnostics: Implementation strategies for Markov chain Monte Carlo.” Statistical Science, 7: 493–497.
  • Richardson, T. (2003). “Markov Properties for Acyclic Directed Mixed Graphs.” Scandinavian Journal of Statistics, 30: 145–157.
  • Rudas, T., Bergsma, W. P., and Németh, R. (2010). “Marginal log-linear parameterization of conditional independence models.” Biometrika, 97: 1006–1012.
  • Silva, R. and Ghahramani, Z. (2009a). “Factorial Mixture of Gaussians and the Marginal Independence Model.” In AISTATS09.
  • Silva, R. and Ghahramani, Z. (2009b). “The Hidden Life of Latent Variables: Bayesian Learning with Mixed Graph Models.” Journal of Machine Learning Research, 10: 1187–1238.
  • Smith, B. J. (2007). “boa: An R Package for MCMC Output Convergence Assessment and Posterior Inference.” Journal of Statistical Software, 21(11): 1–37.
  • Stan Development Team (2017). “RStan: the R interface to Stan.” R package version 2.16.2. URL

Supplemental materials

  • Supplementary Material: Electronic for the Paper “Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models”. The Supplementary Material includes details for the Jacobian calculations (Appendix A) and details for the construction of M and C matrices (Appendices B and C respectively). Finally, some additional results for the illustrated examples of Section 5 are provided in Appendix D.