On Approximating Parametric Bayes Models by Nonparametric Bayes Models

Abstract

Let $\tau$ be a prior distribution over the parameter space $\Theta$ for a given parametric model $P_\theta, \theta \in \Theta$. For the sample space $\mathscr{X}$ (over which $P_\theta$'s are probability measures) belonging to a general class of topological spaces, which include the usual Euclidean spaces, it is shown that this parametric Bayes model can be approximated by a nonparametric Bayes model of the form of a mixture of Dirichlet processes prior, so that (i) the nonparametric prior assigns most of its weight to neighborhoods of the parametric model, and (ii) the Bayes rule for the nonparametric model is close to the Bayes rule for the parametric model in the no-sample case. Moreover, any prior parametric or nonparametric, may be approximated arbitrarily closely by a prior which is a mixture of Dirichlet processes. These results have implications in Bayesian inference.