Translator Disclaimer
December 2017 Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It
Peter Grünwald, Thijs van Ommen
Bayesian Anal. 12(4): 1069-1103 (December 2017). DOI: 10.1214/17-BA1085

Abstract

We empirically show that Bayesian inference can be inconsistent under misspecification in simple linear regression problems, both in a model averaging/selection and in a Bayesian ridge regression setting. We use the standard linear model, which assumes homoskedasticity, whereas the data are heteroskedastic (though, significantly, there are no outliers). As sample size increases, the posterior puts its mass on worse and worse models of ever higher dimension. This is caused by hypercompression, the phenomenon that the posterior puts its mass on distributions that have much larger KL divergence from the ground truth than their average, i.e. the Bayes predictive distribution. To remedy the problem, we equip the likelihood in Bayes’ theorem with an exponent called the learning rate, and we propose the SafeBayesian method to learn the learning rate from the data. SafeBayes tends to select small learning rates, and regularizes more, as soon as hypercompression takes place. Its results on our data are quite encouraging.

Citation

Download Citation

Peter Grünwald. Thijs van Ommen. "Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It." Bayesian Anal. 12 (4) 1069 - 1103, December 2017. https://doi.org/10.1214/17-BA1085

Information

Published: December 2017
First available in Project Euclid: 18 November 2017

zbMATH: 1384.62088
MathSciNet: MR3724979
Digital Object Identifier: 10.1214/17-BA1085

JOURNAL ARTICLE
35 PAGES


SHARE
Vol.12 • No. 4 • December 2017
Back to Top