An Asymptotic Expansion for Posterior Distributions

Abstract

Let $\phi$ be a real valued parameter for the exponential family having densities of the form \begin{equation*}\tag{0.1}p_\phi(x) = C(\phi) \exp \lbrack\phi R(x)\rbrack\end{equation*} with respect to a $\sigma$-finite measure $\mu$ over a Euclidean sample space. Now assume that the parameter $\phi$ has a prior density $\rho(\phi)$. The posterior density of $\phi$, given $(X_1, X_2, \cdots, X_n) = (x_1, x_2, \cdots, x_n)$, is proportional to \begin{equation*}\tag{0.2}\lbrack C(\phi)e^{\phi r}\rbrack^n\rho(\phi)\quad\text{where}\quad r = \sum^n_{i = 1} R(x_i)/n.\end{equation*} The expression (0.2) is proportional to a density function and hence defines a random variable $\phi$ whose density depends on $r$. But since $r = \sum^n_{i = 1} R(x_i)/n$, the distribution of $X$ when $\phi = \phi_0$ generates a sequence $(x_1, x_2, \cdots)$ and a sequence $(R(x_1), \frac{1}{2}\lbrack R(x_1) + R(x_2)\rbrack, \cdots)$ and ultimately an infinite sequence of posterior densities of $\phi$. It is the asymptotic form of this sequence with which we shall be concerned. Neglecting for the moment the stochastic aspect of $r$, we see that if $r$ is held fixed, perhaps at the expected value of $R(x)$, the density proportional to (0.2) is exactly of the form considered in Johnson (1966, 1967). Accordingly, after suitably centering and scaling, we obtain an asymptotic expansion having the standard normal cdf as the leading term. Closely related to this approach is the work by Bernstein and von Mises. Their results are for the Bernoulli situation, and both use the usual parameter $p$ rather than the $\phi = \log \lbrack p/(1 - p)\rbrack$ which results if the density is cast into the form (0.2). von Mises actually holds $r$ fixed as he passes to the limit. Their results, which give only the limiting normal term, are reproduced in Bernstein (1934), page 406, and von Mises (1964), Chapter VIII, Section C. von Mises also gives the multinomial generalization. A more recent work following the same line of attack is given in Gnedenko (1962), Section 65. LeCam, in two basic papers (1953) and (1958), takes into account the stochastic nature of $(x_1, x_2, \cdots)$, and his Theorem 7 (1953) and Lemma 5 (1958) show that under very general conditions, the scaled posterior distribution converges to the normal distribution for almost all sequences $(x_1, x_2, \cdots)$ with respect to the infinite product measure generated by (0.1). His conditions include a more general likelihood than ours and the case where the parameter is multidimensional. See LeCam (1953), page 278, for a discussion concerning the historical background on the problem of convergence of the scaled posterior distribution. The main theorem of this paper is given in Section 1 and the following two sections give the details needed to modify the approach of Johnson (1966, 1967) so that it works in the present situation. This theorem shows that when the observations are taken from the population having density (0.1) with $\phi = \phi_0$, not only does the centered and scaled posterior distribution converge to the normal but there exists an asymptotic expansion in powers of $n^{-\frac{1}{2}}$. Section 4 gives the first two correction terms of the expansion together with examples. Throughout this work, we will use the following notational conventions. $\Phi$ and $\varphi$ are the standard normal cdf and pdf respectively. $F_n(\cdot, r)$ is the cdf of $n^{\frac{1}{2}}\theta$ where $\theta$ is defined below by Equation (1.1).