Electronic Journal of Statistics
- Electron. J. Statist.
- Volume 2 (2008), 63-89.
Nonparametric Bayesian model selection and averaging
We consider nonparametric Bayesian estimation of a probability density p based on a random sample of size n from this density using a hierarchical prior. The prior consists, for instance, of prior weights on the regularity of the unknown density combined with priors that are appropriate given that the density has this regularity. More generally, the hierarchy consists of prior weights on an abstract model index and a prior on a density model for each model index. We present a general theorem on the rate of contraction of the resulting posterior distribution as n→∞, which gives conditions under which the rate of contraction is the one attached to the model that best approximates the true density of the observations. This shows that, for instance, the posterior distribution can adapt to the smoothness of the underlying density. We also study the posterior distribution of the model index, and find that under the same conditions the posterior distribution gives negligible weight to models that are bigger than the optimal one, and thus selects the optimal model or smaller models that also approximate the true density well. We apply these result to log spline density models, where we show that the prior weights on the regularity index interact with the priors on the models, making the exact rates depend in a complicated way on the priors, but also that the rate is fairly robust to specification of the prior weights.
Electron. J. Statist. Volume 2 (2008), 63-89.
First available in Project Euclid: 1 February 2008
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62G07: Density estimation 62G20: Asymptotic properties 62C10: Bayesian problems; characterization of Bayes procedures
Secondary: 65U05 68T05: Learning and adaptive systems [See also 68Q32, 91E40]
Ghosal, Subhashis; Lember, Jüri; van der Vaart, Aad. Nonparametric Bayesian model selection and averaging. Electron. J. Statist. 2 (2008), 63--89. doi:10.1214/07-EJS090. http://projecteuclid.org/euclid.ejs/1201877208.