Source: Electron. J. Statist. Volume 4
(2010), 643-654.
When testing a null hypothesis H0: θ=θ0 in a Bayesian framework, the Savage–Dickey ratio (Dickey, 1971) is known as a specific representation of the Bayes factor (O’Hagan and Forster, 2004) that only uses the posterior distribution under the alternative hypothesis at θ0, thus allowing for a plug-in version of this quantity. We demonstrate here that the Savage–Dickey representation is in fact a generic representation of the Bayes factor and that it fundamentally relies on specific measure-theoretic versions of the densities involved in the ratio, instead of being a special identity imposing some mathematically void constraints on the prior distributions. We completely clarify the measure-theoretic foundations of the Savage–Dickey representation as well as of the later generalisation of Verdinelli and Wasserman (1995). We provide furthermore a general framework that produces a converging approximation of the Bayes factor that is unrelated with the approach of Verdinelli and Wasserman (1995) and propose a comparison of this new approximation with their version, as well as with bridge sampling and Chib’s approaches.
References
Albert, J. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data., J. American Statist. Assoc., 88 669–679.
Billingsley, P. (1986)., Probability and Measure. 2nd ed. John Wiley, New York.
Mathematical Reviews (MathSciNet):
MR830424
Chen, M., Shao, Q. and Ibrahim, J. (2000)., Monte Carlo Methods in Bayesian Computation. Springer-Verlag, New York.
Chib, S. (1995). Marginal likelihood from the Gibbs output., J. American Statist. Assoc., 90 1313–1321.
Chopin, N. and Robert, C. (2010). Properties of evidence., Biometrika. To appear.
Consonni, G. and Veronese, P. (2008). Compatibility of prior specifications across linear models., Statist. Science, 23 332–353.
Dickey, J. (1971). The weighted likelihood ratio, linear hypotheses on normal location parameters., Ann. Mathemat. Statist., 42 204–223.
Mathematical Reviews (MathSciNet):
MR309225
Gelman, A. and Meng, X. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling., Statist. Science, 13 163–185.
Jeffreys, H. (1939)., Theory of Probability. 1st ed. The Clarendon Press, Oxford.
Marin, J. and Robert, C. (2010). Importance sampling methods for Bayesian discrimination between embedded models. In, Frontiers of Statistical Decision Making and Bayesian Analysis (M.-H. Chen, D. Dey, P. Müller, D. Sun and K. Ye, eds.). Springer-Verlag, New York. To appear, see arXiv:0910.2325.
Marin, J.-M. and Robert, C. (2007)., Bayesian Core. Springer-Verlag, New York.
O’Hagan, A. and Forster, J. (2004)., Kendall’s advanced theory of Statistics: Bayesian inference. Arnold, London.
R Development Core Team (2008)., R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org.
Robert, C. (2001)., The Bayesian Choice. 2nd ed. Springer-Verlag, New York.
Torrie, G. and Valleau, J. (1977). Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling., J. Comp. Phys., 23 187–199.
Verdinelli, I. and Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage–Dickey density ratio., J. American Statist. Assoc., 90 614–618.
Wetzels, R., Grasman, R. and Wagenmakers, E.-J. (2010). An encompassing prior generalization of the Savage-Dickey density ratio., Comput. Statist. Data Anal., 54 2094–2102.
Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with, g-prior distribution regression using Bayesian variable selection. In Bayesian inference and decision techniques: Essays in Honor of Bruno de Finetti. North-Holland / Elsevier, 233–243.
Mathematical Reviews (MathSciNet):
MR881437
