The Annals of Statistics
- Ann. Statist.
- Volume 47, Number 1 (2019), 39-66.
Bayesian fractional posteriors
We consider the fractional posterior distribution that is obtained by updating a prior distribution via Bayes theorem with a fractional likelihood function, a usual likelihood function raised to a fractional power. First, we analyze the contraction property of the fractional posterior in a general misspecified framework. Our contraction results only require a prior mass condition on certain Kullback–Leibler (KL) neighborhood of the true parameter (or the KL divergence minimizer in the misspecified case), and obviate constructions of test functions and sieves commonly used in the literature for analyzing the contraction property of a regular posterior. We show through a counterexample that some condition controlling the complexity of the parameter space is necessary for the regular posterior to contract, rendering additional flexibility on the choice of the prior for the fractional posterior. Second, we derive a novel Bayesian oracle inequality based on a PAC-Bayes inequality in misspecified models. Our derivation reveals several advantages of averaging based Bayesian procedures over optimization based frequentist procedures. As an application of the Bayesian oracle inequality, we derive a sharp oracle inequality in multivariate convex regression problems. We also illustrate the theory in Gaussian process regression and density estimation problems.
Ann. Statist., Volume 47, Number 1 (2019), 39-66.
Received: November 2016
Revised: April 2018
First available in Project Euclid: 30 November 2018
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Bhattacharya, Anirban; Pati, Debdeep; Yang, Yun. Bayesian fractional posteriors. Ann. Statist. 47 (2019), no. 1, 39--66. doi:10.1214/18-AOS1712. https://projecteuclid.org/euclid.aos/1543568581
- Proofs of main results. All proofs and additional details pertaining to Section 4 and Section 5 are provided in the supplementary document.