Electronic Journal of Statistics
- Electron. J. Statist.
- Volume 3 (2009), 1039-1074.
Dynamics of Bayesian updating with dependent data and misspecified models
Much is now known about the consistency of Bayesian updating on infinite-dimensional parameter spaces with independent or Markovian data. Necessary conditions for consistency include the prior putting enough weight on the correct neighborhoods of the data-generating distribution; various sufficient conditions further restrict the prior in ways analogous to capacity control in frequentist nonparametrics. The asymptotics of Bayesian updating with mis-specified models or priors, or non-Markovian data, are far less well explored. Here I establish sufficient conditions for posterior convergence when all hypotheses are wrong, and the data have complex dependencies. The main dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman) property of information theory. This, along with Egorov’s Theorem on uniform convergence, lets me build a sieve-like structure for the prior. The main statistical assumption, also a form of capacity control, concerns the compatibility of the prior and the data-generating process, controlling the fluctuations in the log-likelihood when averaged over the sieve-like sets. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between these results and the replicator dynamics of evolutionary theory.
Electron. J. Statist. Volume 3 (2009), 1039-1074.
First available in Project Euclid: 29 October 2009
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62C10: Bayesian problems; characterization of Bayes procedures 62G20: Asymptotic properties 62M09: Non-Markovian processes: estimation
Secondary: 60F10: Large deviations 62M05: Markov processes: estimation 92D15: Problems related to evolution 94A17: Measures of information, entropy
Shalizi, Cosma Rohilla. Dynamics of Bayesian updating with dependent data and misspecified models. Electron. J. Statist. 3 (2009), 1039--1074. doi:10.1214/09-EJS485. https://projecteuclid.org/euclid.ejs/1256822130