We offer a general Bayes theoretic framework to derive posterior contraction rates under a hierarchical prior design: the first-step prior serves to assess the model selection uncertainty, and the second-step prior quantifies the prior belief on the strength of the signals within the model chosen from the first step. In particular, we establish non-asymptotic oracle posterior contraction rates under (i) a local Gaussianity condition on the log likelihood ratio of the statistical experiment, (ii) a local entropy condition on the dimensionality of the models, and (iii) a sufficient mass condition on the second-step prior near the best approximating signal for each model. The first-step prior can be designed generically. The posterior distribution enjoys Gaussian tail behavior and therefore the resulting posterior mean also satisfies an oracle inequality, automatically serving as an adaptive point estimator in a frequentist sense. Model mis-specification is allowed in these oracle rates.
The local Gaussianity condition serves as a unified attempt of non-asymptotic Gaussian quantification of the experiments, and can be easily verified in various experiments considered in [GvdV07a] and beyond. The general results are applied in various problems including: (i) trace regression, (ii) shape-restricted isotonic/convex regression, (iii) high-dimensional partially linear regression, (iv) covariance matrix estimation in the sparse factor model, (v) detection of non-smooth polytopal image boundary, and (vi) intensity estimation in a Poisson point process model. These new results serve either as theoretical justification of practical prior proposals in the literature, or as an illustration of the generic construction scheme of a (nearly) minimax adaptive estimator for a complicated experiment.
Supported in part by NSF Grant DMS-1566514, DMS-1916221, and by Isaac Newton Institute for Mathematical Sciences, program Statistical Scalability, EPSRC Grant Number LNAG/036 RG91310.
The author is indebted to Chao Gao, Johannes Schmidt-Hieber, and the anonymous referees for numerous comments and suggestions that lead to substantial improvements of an earlier version of the paper. The author would also like to thank Jon Wellner for constant support and continuous encouragement as this work developed.
"Oracle posterior contraction rates under hierarchical priors." Electron. J. Statist. 15 (1) 1085 - 1153, 2021. https://doi.org/10.1214/21-EJS1811