Bayesian Analysis

On some difficulties with a posterior probability approximation technique

Jean-Michel Marin and Christian P. Robert

Full-text: Open access

Abstract

In Scott (2002) and Congdon (2006), a new method is advanced to compute posterior probabilities of models under consideration. It is based solely on MCMC outputs restricted to single models, i.e., it is bypassing reversible jump and other model exploration techniques. While it is indeed possible to approximate posterior probabilities based solely on MCMC outputs from single models, as demonstrated by Gelfand and Dey (1994) and Bartolucci et al. (2006), we show that the proposals of Scott (2002) and Congdon (2006) are biased and advance several arguments towards this thesis, the primary one being the confusion between model-based posteriors and joint pseudo-posteriors. From a practical point of view, the bias in Scott's (2002) approximation appears to be much more severe than the one in Congdon's (2006), the latter being often of the same magnitude as the posterior probability it approximates, although we also exhibit an example where the divergence from the true posterior probability is extreme.

Article information

Source
Bayesian Anal., Volume 3, Number 2 (2008), 427-441.

Dates
First available in Project Euclid: 22 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340370554

Digital Object Identifier
doi:10.1214/08-BA316

Mathematical Reviews number (MathSciNet)
MR2407433

Zentralblatt MATH identifier
1330.62149

Keywords
Bayesian model choice posterior approximation reversible jump Markov Chain Monte Carlo (MCMC) pseudo-priors unbiasedness improperty

Citation

Robert, Christian P.; Marin, Jean-Michel. On some difficulties with a posterior probability approximation technique. Bayesian Anal. 3 (2008), no. 2, 427--441. doi:10.1214/08-BA316. https://projecteuclid.org/euclid.ba/1340370554


Export citation

References

  • Bartolucci, F., Scaccia, L., and Mira, A. (2006). "Efficient Bayes factor estimation from the reversible jump output." Biometrika, 93: 41–52.
  • Brooks, S., Giudici, P., and Roberts, G. (2003). "Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with discussion)." J. Royal Statist. Society Series B, 65(1): 3–55.
  • Carlin, B. and Chib, S. (1995). "Bayesian model choice through Markov chain Monte Carlo." J. Royal Statist. Society Series B, 57(3): 473–484.
  • Chen, C., Gerlach, R., and So, M. (2008). "Bayesian Model Selection for Heteroskedastic Models." Advances in Econometrics, 23. To appear.
  • Chen, M., Shao, Q., and Ibrahim, J. (2000). Monte Carlo Methods in Bayesian Computation. Springer-Verlag, New York.
  • Chopin, N. and Robert, C. (2007). "Contemplating Evidence: properties, extensions of, and alternatives to Nested Sampling." Technical Report 2007-46, CEREMADE, Université Paris Dauphine. ArXiv:0801.3887.
  • Congdon, P. (2006). "Bayesian model choice based on Monte Carlo estimates of posterior model probabilities." Comput. Stat. Data Analysis, 50: 346–357.
  • –- (2007). "Model weights for model choice and averaging." Statistical Methodology, 4(2): 143–157.
  • Gamerman, D. and Lopes, H. (2006). Markov Chain Monte Carlo. Chapman and Hall, New York, second edition.
  • Gelfand, A. and Dey, D. (1994). "Bayesian model choice: asymptotics and exact calculations." J. Royal Statist. Society Series B, 56: 501–514.
  • Gelman, A. and Meng, X. (1998). "Simulating normalizing constants: From importance sampling to bridge sampling to path sampling." Statist. Science, 13: 163–185.
  • Green, P. (1995). "Reversible jump MCMC" computation and Bayesian model determination. Biometrika, 82(4): 711–732.
  • Newton, M. and Raftery, A. (1994). "Approximate Bayesian inference by the weighted likelihood boostrap (with discussion)." J. Royal Statist. Society Series B, 56: 1–48.
  • Robert, C. (2001). The Bayesian Choice. Springer-Verlag, New York, second edition.
  • Scott, S. L. (2002). "Bayesian methods for hidden Markov models: recursive computing in the 21st" Century. J. American Statist. Assoc., 97: 337–351.