Bayesian Analysis

On Posterior Concentration in Misspecified Models

R. V. Ramamoorthi, Karthik Sriram, and Ryan Martin

Full-text: Open access

Abstract

We investigate the asymptotic behavior of Bayesian posterior distributions under independent and identically distributed (i.i.d.) misspecified models. More specifically, we study the concentration of the posterior distribution on neighborhoods of f, the density that is closest in the Kullback–Leibler sense to the true model f0. We note, through examples, the need for assumptions beyond the usual Kullback–Leibler support assumption. We then investigate consistency with respect to a general metric under three assumptions, each based on a notion of divergence measure, and then apply these to a weighted L1-metric in convex models and non-convex models.

Although a few results on this topic are available, we believe that these are somewhat inaccessible due, in part, to the technicalities and the subtle differences compared to the more familiar well-specified model case. One of our goals is to make some of the available results, especially that of Kleijn and van der Vaart (2006), more accessible. Unlike their paper, our approach does not require construction of test sequences. We also discuss a preliminary extension of the i.i.d. results to the independent but not identically distributed (i.n.i.d.) case.

Article information

Source
Bayesian Anal., Volume 10, Number 4 (2015), 759-789.

Dates
First available in Project Euclid: 4 February 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1423083642

Digital Object Identifier
doi:10.1214/15-BA941

Mathematical Reviews number (MathSciNet)
MR3432239

Zentralblatt MATH identifier
1335.62022

Subjects
Primary: 62C10: Bayesian problems; characterization of Bayes procedures
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Keywords
Bayesian consistency misspecified Kullback–Leibler

Citation

Ramamoorthi, R. V.; Sriram, Karthik; Martin, Ryan. On Posterior Concentration in Misspecified Models. Bayesian Anal. 10 (2015), no. 4, 759--789. doi:10.1214/15-BA941. https://projecteuclid.org/euclid.ba/1423083642


Export citation

References

  • Berk, R. H. (1966). “Limiting behavior of posterior distribution when the model is incorrect.” The Annals of Mathematical Statistics, 37(1): 51–58.
  • Bunke, O. and Milhaud, X. (1998). “Asymptotic behavior of Bayes estimates under possibly incorrect models.” The Annals of Statistics, 26(2): 617–644.
  • De Blasi, P. and Walker, S. G. (2013). “Bayesian asymptotics with misspecified models.” Statistica Sinica, 23: 169–187.
  • Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. New York: Springer-Verlag.
  • Kleijn, B. J. K. and van der Vaart, A. W. (2006). “Misspecification in infinite-dimensional Bayesian statistics.” The Annals of Statistics, 34(2): 837–877.
  • Lee, J. and MacEachern, S. N. (2011). “Consistency of Bayes estimators without the assumption that the model is correct.” Journal of Statistical Planning and Inference, 141(2): 748–757.
  • Shalizi, C. R. (2009). “Dynamics of Bayesian updating with dependent data and misspecified models.” Electronic Journal of Statistics, 3: 1039–1074.
  • Sion, M. (1958). “On general minimax theorems.” Pacific Journal of Mathematics, 8(1): 171–176.
  • Sriram, K., Ramamoorthi, R. V., and Ghosh, P. (2013). “Posterior consistency of Bayesian quantile regression based on the misspecified asymmetric Laplace density.” Bayesian Analysis, 8(2): 479–504.
  • Yu, K. and Moyeed, R. A. (2001). “Bayesian quantile regression.” Statistics and Probability Letters, 54(2): 437–447.