Brazilian Journal of Probability and Statistics

Bayesian computation for statistical models with intractable normalizing constants

Yves F. Atchadé, Nicolas Lartillot, and Christian Robert

Full-text: Open access

Abstract

This paper deals with a computational aspect of the Bayesian analysis of statistical models with intractable normalizing constants. In the presence of intractable normalizing constants in the likelihood function, traditional MCMC methods cannot be applied. We propose here a general approach to sample from such posterior distributions that bypasses the computation of the normalizing constant. Our method can be thought as a Bayesian version of the MCMC-MLE approach of Geyer and Thompson [J. Roy. Statist. Soc. Ser. B 54 (1992) 657–699]. We illustrate our approach on examples from image segmentation and social network modeling. We study as well the asymptotic behavior of the algorithm and obtain a strong law of large numbers for empirical averages.

Article information

Source
Braz. J. Probab. Stat., Volume 27, Number 4 (2013), 416-436.

Dates
First available in Project Euclid: 9 September 2013

Permanent link to this document
https://projecteuclid.org/euclid.bjps/1378729981

Digital Object Identifier
doi:10.1214/11-BJPS174

Mathematical Reviews number (MathSciNet)
MR3105037

Zentralblatt MATH identifier
1298.62046

Keywords
Monte Carlo methods adaptive Markov Chain Monte Carlo Bayesian inference doubly-intractable distributions Ising model

Citation

Atchadé, Yves F.; Lartillot, Nicolas; Robert, Christian. Bayesian computation for statistical models with intractable normalizing constants. Braz. J. Probab. Stat. 27 (2013), no. 4, 416--436. doi:10.1214/11-BJPS174. https://projecteuclid.org/euclid.bjps/1378729981


Export citation

References

  • Andrieu, C., Moulines, É. and Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44, 283–312 (electronic).
  • Atchade, Y. F. (2006). An adaptive version for the Metropolis adjusted Langevin algorithm with a truncated drift. Methodol. Comput. Appl. Probab. 8, 235–254.
  • Atchade, Y. F. and Liu, J. S. (2010). The Wang–Landau algorithm for Monte Carlo computation in general state spaces. Statistica Sinica 20, 209–233.
  • Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. Roy. Statist. Soc. Ser. B 36, 192–236. With discussion by D. R. Cox, A. G. Hawkes, P. Clifford, P. Whittle, K. Ord, R. Mead, J. M. Hammersley and M. S. Bartlett and with a reply by the author.
  • Cappé, O., Douc, R., Guillin, A., Marin, J.-M. and Robert, C. (2008). Adaptive importance sampling in general mixture classes. Statist. Comput. 18, 447–459.
  • Gelman, A. and Meng, X.-L. (1998). Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Statist. Sci. 13, 163–185.
  • Geyer, C. J. (1994). On the convergence of Monte Carlo maximum likelihood calculations. J. Roy. Statist. Soc. Ser. B 56, 261–274.
  • Geyer, C. J. and Thompson, E. A. (1992). Constrained Monte Carlo maximum likelihood for dependent data. J. Roy. Statist. Soc. Ser. B 54, 657–699. With discussion and a reply by the authors.
  • Hurn, M., Husby, O. and Rue, H. (2003). A tutorial on image analysis. In Spatial Statistics and Computational Methods (Aalborg, 2001). Lecture Notes in Statistics 173, 87–141. New York: Springer.
  • Ibanez, M. V. and Simo, A. (2003). Parameter estimation in markov random field image modeling with imperfect observations. a comparative study. Pattern Recognition Letters 24, 2377–2389.
  • Kleinman, A., Rodrigue, N., Bonnard, C. and Philippe, H. (2006). A maximum likelihood framework for protein design. BMC Bioinformatics 7, 20–39.
  • Marin, J.-M. and Robert, C. (2007). Bayesian Core. New York: Springer Verlag.
  • Marin, J.-M., Robert, C. and Titterington, D. (2009). A Bayesian reassessment of nearest-neighbour classification. J. Amer. Statist. Assoc. 104, 263–273.
  • Møller, J., Pettitt, A. N., Reeves, R. and Berthelsen, K. K. (2006). An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93, 451–458.
  • Møller, J. and Waagepetersen, R. P. (2004). Statistical Inference and Simulation for Spatial Point Processes. Monographs on Statistics and Applied Probability 100. Boca Raton, FL: Chapman & Hall/CRC.
  • Murray, I., Ghahramani, Z. and MacKay, D. (2006). MCMC for doubly-intractable distributions. In Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence (UAI). Corvallis, OR: AUAI Press.
  • Robins, G., P., P., Kalish, Y. and Lusher, D. (2007). An introduction to exponential random graph models for social networks. Social Networks 29, 173–191.
  • Wang, F. and Landau, D. P. (2001). Efficient, multiple-range random walk algorithm to calculate the density of states. Physical Review Letters 86, 2050–2053.
  • Younes, L. (1988). Estimation and annealing for gibbsian fields. Annales de l’Institut Henri Poincaré. Probabilité et Statistiques 24, 269–294.