Abstract
Under model misspecification, it is known that Bayesian posteriors often do not properly quantify uncertainty about true or pseudo-true parameters. Even more fundamentally, misspecification leads to a lack of reproducibility in the sense that the same model will yield contradictory posteriors on independent data sets from the true distribution. To define a criterion for reproducible uncertainty quantification under misspecification, we consider the probability that two credible sets constructed from independent data sets have nonempty overlap, and we establish a lower bound on this overlap probability that holds whenever the credible sets are valid confidence sets. We prove that credible sets from the standard posterior can strongly violate this bound, indicating that it is not internally coherent under misspecification. To improve reproducibility in an easy-to-use and widely applicable way, we propose to apply bagging to the Bayesian posterior (“BayesBag”); that is, to use the average of posterior distributions conditioned on bootstrapped datasets. We motivate BayesBag from first principles based on Jeffrey conditionalization and show that the bagged posterior typically satisfies the overlap lower bound. Further, we prove a Bernstein–Von Mises theorem for the bagged posterior, establishing its asymptotic normal distribution. We demonstrate the benefits of BayesBag via simulation experiments and an application to crime rate prediction.
Funding Statement
The first author was partially supported by the National Institute of General Medical Sciences of the National Institutes of Health as part of the Joint NSF/NIGMS Mathematical Biology Program. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Acknowledgments
Thanks to Pierre Jacob for bringing P. Bühlmann’s BayesBag paper to our attention. Thanks also to Ryan Giordano and Pierre Jacob for helpful feedback on an earlier version of this paper, to Peter Grünwald, Natalia Bochkina, Mathieu Gerber, and Anthony Lee for helpful discussions, and to the Associate Editor and two referees whose comments led to substantial improvements to scope and focus of the paper.
Citation
Jonathan H. Huggins. Jeffrey W. Miller. "Reproducible parameter inference using bagged posteriors." Electron. J. Statist. 18 (1) 1549 - 1585, 2024. https://doi.org/10.1214/24-EJS2237
Information