Electronic Journal of Statistics

On resolving the Savage–Dickey paradox

Jean-Michel Marin and Christian P. Robert

Full-text: Open access

Abstract

When testing a null hypothesis H0: θ=θ0 in a Bayesian framework, the Savage–Dickey ratio (Dickey, 1971) is known as a specific representation of the Bayes factor (O’Hagan and Forster, 2004) that only uses the posterior distribution under the alternative hypothesis at θ0, thus allowing for a plug-in version of this quantity. We demonstrate here that the Savage–Dickey representation is in fact a generic representation of the Bayes factor and that it fundamentally relies on specific measure-theoretic versions of the densities involved in the ratio, instead of being a special identity imposing some mathematically void constraints on the prior distributions. We completely clarify the measure-theoretic foundations of the Savage–Dickey representation as well as of the later generalisation of Verdinelli and Wasserman (1995). We provide furthermore a general framework that produces a converging approximation of the Bayes factor that is unrelated with the approach of Verdinelli and Wasserman (1995) and propose a comparison of this new approximation with their version, as well as with bridge sampling and Chib’s approaches.

Article information

Source
Electron. J. Statist., Volume 4 (2010), 643-654.

Dates
First available in Project Euclid: 9 July 2010

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1278682959

Digital Object Identifier
doi:10.1214/10-EJS564

Mathematical Reviews number (MathSciNet)
MR2660536

Zentralblatt MATH identifier
1329.62091

Keywords
Bayesian model choice Bayes factor bridge sampling conditional distribution hypothesis testing Savage–Dickey ratio zero measure set

Citation

Marin, Jean-Michel; Robert, Christian P. On resolving the Savage–Dickey paradox. Electron. J. Statist. 4 (2010), 643--654. doi:10.1214/10-EJS564. https://projecteuclid.org/euclid.ejs/1278682959


Export citation

References

  • Albert, J. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data., J. American Statist. Assoc., 88 669–679.
  • Billingsley, P. (1986)., Probability and Measure. 2nd ed. John Wiley, New York.
  • Chen, M., Shao, Q. and Ibrahim, J. (2000)., Monte Carlo Methods in Bayesian Computation. Springer-Verlag, New York.
  • Chib, S. (1995). Marginal likelihood from the Gibbs output., J. American Statist. Assoc., 90 1313–1321.
  • Chopin, N. and Robert, C. (2010). Properties of evidence., Biometrika. To appear.
  • Consonni, G. and Veronese, P. (2008). Compatibility of prior specifications across linear models., Statist. Science, 23 332–353.
  • Dickey, J. (1971). The weighted likelihood ratio, linear hypotheses on normal location parameters., Ann. Mathemat. Statist., 42 204–223.
  • Gelman, A. and Meng, X. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling., Statist. Science, 13 163–185.
  • Jeffreys, H. (1939)., Theory of Probability. 1st ed. The Clarendon Press, Oxford.
  • Marin, J. and Robert, C. (2010). Importance sampling methods for Bayesian discrimination between embedded models. In, Frontiers of Statistical Decision Making and Bayesian Analysis (M.-H. Chen, D. Dey, P. Müller, D. Sun and K. Ye, eds.). Springer-Verlag, New York. To appear, see arXiv:0910.2325.
  • Marin, J.-M. and Robert, C. (2007)., Bayesian Core. Springer-Verlag, New York.
  • O’Hagan, A. and Forster, J. (2004)., Kendall’s advanced theory of Statistics: Bayesian inference. Arnold, London.
  • R Development Core Team (2008)., R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org.
  • Robert, C. (2001)., The Bayesian Choice. 2nd ed. Springer-Verlag, New York.
  • Torrie, G. and Valleau, J. (1977). Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling., J. Comp. Phys., 23 187–199.
  • Verdinelli, I. and Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage–Dickey density ratio., J. American Statist. Assoc., 90 614–618.
  • Wetzels, R., Grasman, R. and Wagenmakers, E.-J. (2010). An encompassing prior generalization of the Savage-Dickey density ratio., Comput. Statist. Data Anal., 54 2094–2102.
  • Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with, g-prior distribution regression using Bayesian variable selection. In Bayesian inference and decision techniques: Essays in Honor of Bruno de Finetti. North-Holland / Elsevier, 233–243.