## Bayesian Analysis

### On Posterior Concentration in Misspecified Models

#### Abstract

We investigate the asymptotic behavior of Bayesian posterior distributions under independent and identically distributed ($i.i.d.$) misspecified models. More specifically, we study the concentration of the posterior distribution on neighborhoods of $f^{\star}$, the density that is closest in the Kullback–Leibler sense to the true model $f_{0}$. We note, through examples, the need for assumptions beyond the usual Kullback–Leibler support assumption. We then investigate consistency with respect to a general metric under three assumptions, each based on a notion of divergence measure, and then apply these to a weighted $L_{1}$-metric in convex models and non-convex models.

Although a few results on this topic are available, we believe that these are somewhat inaccessible due, in part, to the technicalities and the subtle differences compared to the more familiar well-specified model case. One of our goals is to make some of the available results, especially that of Kleijn and van der Vaart (2006), more accessible. Unlike their paper, our approach does not require construction of test sequences. We also discuss a preliminary extension of the $i.i.d.$ results to the independent but not identically distributed ($i.n.i.d.$) case.

#### Article information

Source
Bayesian Anal., Volume 10, Number 4 (2015), 759-789.

Dates
First available in Project Euclid: 4 February 2015

https://projecteuclid.org/euclid.ba/1423083642

Digital Object Identifier
doi:10.1214/15-BA941

Mathematical Reviews number (MathSciNet)
MR3432239

Zentralblatt MATH identifier
1335.62022

#### Citation

Ramamoorthi, R. V.; Sriram, Karthik; Martin, Ryan. On Posterior Concentration in Misspecified Models. Bayesian Anal. 10 (2015), no. 4, 759--789. doi:10.1214/15-BA941. https://projecteuclid.org/euclid.ba/1423083642

#### References

• Berk, R. H. (1966). “Limiting behavior of posterior distribution when the model is incorrect.” The Annals of Mathematical Statistics, 37(1): 51–58.
• Bunke, O. and Milhaud, X. (1998). “Asymptotic behavior of Bayes estimates under possibly incorrect models.” The Annals of Statistics, 26(2): 617–644.
• De Blasi, P. and Walker, S. G. (2013). “Bayesian asymptotics with misspecified models.” Statistica Sinica, 23: 169–187.
• Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. New York: Springer-Verlag.
• Kleijn, B. J. K. and van der Vaart, A. W. (2006). “Misspecification in infinite-dimensional Bayesian statistics.” The Annals of Statistics, 34(2): 837–877.
• Lee, J. and MacEachern, S. N. (2011). “Consistency of Bayes estimators without the assumption that the model is correct.” Journal of Statistical Planning and Inference, 141(2): 748–757.
• Shalizi, C. R. (2009). “Dynamics of Bayesian updating with dependent data and misspecified models.” Electronic Journal of Statistics, 3: 1039–1074.
• Sion, M. (1958). “On general minimax theorems.” Pacific Journal of Mathematics, 8(1): 171–176.
• Sriram, K., Ramamoorthi, R. V., and Ghosh, P. (2013). “Posterior consistency of Bayesian quantile regression based on the misspecified asymmetric Laplace density.” Bayesian Analysis, 8(2): 479–504.
• Yu, K. and Moyeed, R. A. (2001). “Bayesian quantile regression.” Statistics and Probability Letters, 54(2): 437–447.