Bayesian Analysis

Bayesian Model Selection Based on Proper Scoring Rules

A. Philip Dawid and Monica Musio

Full-text: Open access

Abstract

Bayesian model selection with improper priors is not well-defined because of the dependence of the marginal likelihood on the arbitrary scaling constants of the within-model prior densities. We show how this problem can be evaded by replacing marginal log-likelihood by a homogeneous proper scoring rule, which is insensitive to the scaling constants. Suitably applied, this will typically enable consistent selection of the true model.

Article information

Source
Bayesian Anal., Volume 10, Number 2 (2015), 479-499.

Dates
First available in Project Euclid: 4 February 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1423083641

Digital Object Identifier
doi:10.1214/15-BA942

Mathematical Reviews number (MathSciNet)
MR3420890

Zentralblatt MATH identifier
1335.62017

Keywords
consistent model selection homogeneous score Hyvärinen score prequential

Citation

Dawid, A. Philip; Musio, Monica. Bayesian Model Selection Based on Proper Scoring Rules. Bayesian Anal. 10 (2015), no. 2, 479--499. doi:10.1214/15-BA942. https://projecteuclid.org/euclid.ba/1423083641


Export citation

References

  • Bayarri, M. J., Berger, J. O., Forte, A., and García-Donato, G. (2012). “Criteria for Bayesian Model Choice with Application to Variable Selection.” The Annals of Statistics, 40: 1550–1577.
  • Berger, J. O. and Pericchi, L. R. (1996). “The Intrinsic Bayes Factor for Model Selection and Prediction.” Journal of the American Statistical Association, 91: 109–122.
  • Dawid, A. P. (1984). “Statistical Theory—The Prequential Approach (with Discussion).” Journal of the Royal Statistical Society, Series A, 147: 278–292.
  • — (1986). “Probability Forecasting.” In: Kotz, S., Johnson, N. L., and Read, C. B. (eds.), Encyclopedia of Statistical Sciences, volume 7, 210–218. New York: Wiley-Interscience.
  • — (1992). “Prequential Analysis, Stochastic Complexity and Bayesian Inference (with Discussion).” In: Bernardo, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M. (eds.), Bayesian Statistics 4, 109–125. Oxford: Oxford University Press.
  • — (2011). “Posterior Model Probabilities.” In: Bandyopadhyay, P. S. and Forster, M. (eds.), Philosophy of Statistics, 607–630. New York: Elsevier.
  • Dawid, A. P. and Lauritzen, S. L. (2005). “The Geometry of Decision Theory.” In Proceedings of the Second International Symposium on Information Geometry and its Applications, 22–28. University of Tokyo. 12–16 December 2005.
  • Dawid, A. P. and Musio, M. (2013). “Estimation of Spatial Processes Using Local Scoring Rules.” AStA Advances in Statistical Analysis, 97: 173–179.
  • — (2014). “Theory and Applications of Proper Scoring Rules.” Metron, 72: 169–183.
  • Dawid, A. P., Musio, M., and Ventura, L. (2015). “Minimum Scoring Rule Inference.” Scandinavian Journal of Statistics, submitted for publication. arXiv:1403.3920
  • Good, I. J. (1952). “Rational Decisions.” Journal of the Royal Statistical Society, Series B, 14: 107–114.
  • Grünwald, P. D. and Dawid, A. P. (2004). “Game Theory, Maximum Entropy, Minimum Discrepancy, and Robust Bayesian Decision Theory.” The Annals of Statistics, 32: 1367–1433.
  • Hyvärinen, A. (2005). “Estimation of Non-Normalized Statistical Models by Score Matching.” Journal of Machine Learning Research, 6: 695–709.
  • Johnson, V. E. and Rossell, D. (2012). “Bayesian Model Selection in High-Dimensional Settings.” Journal of the American Statistical Association, 107: 649–660.
  • Kabanov, Y. M., Liptser, R. S., and Shiryayev, A. N. (1977). “On the Question of Absolute Continuity and Singularity of Probability Measures.” Mathematics of the USSR. Sbornik, 33: 203–221.
  • Lindley, D. V. and Smith, A. F. M. (1972). “Bayes Estimates for the Linear Model (with Discussion).” Journal of the Royal Statistical Society, Series B, 34: 1–41.
  • Mameli, V., Musio, M., and Dawid, A. P. (2014). “Comparisons of Hyvärinen and Pairwise Estimators in Two Simple Linear Time Series Models.” arXiv:1409.3690
  • Moreno, E., Girón, F. J., and Casella, G. (2010). “Consistency of Objective Bayes Factors as the Model Dimension Grows.” The Annals of Statistics, 38: 1937–1952.
  • Musio, M. and Dawid, A. P. (2013). “Local Scoring rules: A Versatile Tool for Inference.” Paper presented at 59th World Statistics Congress, Hong Kong. http://www. statistics.gov.hk/wsc/STS019-P3-S.pdf
  • O’Hagan, A. (1995). “Fractional Bayes Factors for Model Comparison.” Journal of the Royal Statistical Society, Series B, 57: 99–138.
  • Parry, M. F. (2013). “Multidimensional Local Scoring Rules.” Paper presented at 59th World Statistics Congress, Hong Kong. http://www.statistics.gov.hk/wsc/ STS019-P2-S.pdf
  • Parry, M. F., Dawid, A. P., and Lauritzen, S. L. (2012). “Proper Local Scoring Rules.” The Annals of Statistics, 40: 561–592.
  • Skouras, K. (1998). “Absolute Continuity of Markov Chains.” Journal of Statistical Planning and Inference, 75: 1–8.
  • Stout, W. F. (1970). “A Martingale Analogue of Kolmogorov’s Law of the Iterated Logarithm.” Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 15: 279–290.

See also

  • Related item: Matthias Katzfuss, Anirban Bhattacharya (2015). Comment on Article by Dawid and Musio. Bayesian Anal. Vol. 10, Iss. 2, 501–504.
  • Related item: Christopher M. Hans, Mario Peruggia (2015). Comment on Article by Dawid and Musio. Bayesian Anal. Vol. 10, Iss. 2, 505–509.
  • Related item: C. Grazian, I. Masiani, C. P. Robert (2015). Comment on Article by Dawid and Musio. Bayesian Anal. Vol. 10, Iss. 2, 511–515.
  • Related item: A. Philip Dawid, Monica Musio (2015). Rejoinder. Bayesian Anal. Vol. 10, Iss. 2, 517–521.