Asymptotic Expansions Associated with Posterior Distributions

Abstract

In this paper, an extension of the investigation of Johnson (1967b) is made by giving a larger class of posterior distributions which possess asymptotic expansions having a normal distribution as a leading term. Asymptotic expansions for the related normalizing transformation and percentiles are also presented. Before asymptotic expansions were treated rigorously, LaPlace (1847) gave an expansion for certain posterior distributions. The method used in this paper is a variation of his technique. Bernstein (1934), page 406, and von Mises (1964), chapter VIII, Section C, also treat special cases of these expansions. The conditions imposed are sufficient to make the maximum likelihood estimate strongly consistent and asymptotically normal. They also include higher order derivative assumptions on the log of the likelihood. As shown by Schwartz (1966), the posterior distribution may behave well even when the maximum likelihood estimate does not. However, we have not attempted to find the weakest assumptions under which the posterior distribution has an expansion. For general conditions under which the posterior distribution converges in variation to a normal distribution with probability one see LeCam (1953) and (1958) for the independent case and Kallianpur and Borwanker (1968) for Markov processes. In Section 2, we show that with probability one, the centered and scaled posterior distribution possesses an asymptotic expansion in powers of $n^{-\frac{1}{2}}$ having the standard normal as a leading term. The number of terms in the expansion obtained is two less than the number of continuous derivatives of the log likelihood. All terms beyond the first consist of a polynomial multiplied by the standard normal density. The coefficients of the polynomial depend on the prior density $\rho$ and the likelihood. The moments of the posterior distribution are shown to possess an expansion in Section 3. The following two sections present the normalizing transformation and percentile expansions. These last three expansions also apply for the case considered by Johnson (1967b) as does the information on the form of the terms in the expansion of the posterior distribution. To simplify the already heavy notation, these results are first proved for independent identically distributed random variables. The extension of all these results to the case of certain stationary ergodic Markov processes is immediate; Section 6 presents the necessary modifications. Throughout this paper, $\Phi$ and $\varphi$ will denote the standard normal cdf and pdf respectively. Also, $\mathbf{n}$ will be assumed to range over the positive integers; thus in some cases, the order of the error term in the expansion may be kept for smaller $n$ if the bounding constant is modified.