Expansions for posterior distributions

Abstract

Suppose that ${\mathit{X}}_{\mathit{n}}$ is a sample of size n with log likelihood $\mathit{n}\mathit{l}(\mathit{\theta })$, where θ is an unknown parameter in ${\mathit{R}}^{\mathit{p}}$ having a prior distribution $\mathit{\xi }(\mathit{\theta })$. We need not assume that the sample values are independent or even stationary. Let $\hat{\mathit{\theta }}$ be the maximum likelihood estimate (MLE). We show that $\mathit{\theta }|{\mathit{X}}_{\mathit{n}}$ is asymptotically normal with mean $\hat{\mathit{\theta }}$ and covariance $-{\mathit{n}}^{-1}{\mathit{l}}_{․,․}{(\hat{\mathit{\theta }})}^{-1}$, where ${\mathit{l}}_{․,․}(\mathit{\theta })={\partial }^2\mathit{l}(\mathit{\theta })/\partial \mathit{\theta }\partial {\mathit{\theta }}^{\prime }$. In contrast, $\hat{\mathit{\theta }}|\mathit{\theta }$ is asymptotically normal with mean θ and covariance ${\mathit{n}}^{-1}{[\mathit{I}(\mathit{\theta })]}^{-1}$, where $\mathit{I}(\mathit{\theta })=-\mathit{E}[{\mathit{l}}_{․,․}(\hat{\mathit{\theta }})|\mathit{\theta }]$ is Fisher’s information. So, frequentist inference conditional on θ cannot be used to approximate Bayesian inference, except for exponential families. However, under mild conditions $-{\mathit{l}}_{․,․}(\hat{\mathit{\theta }})|\mathit{\theta }\to \mathit{I}(\mathit{\theta })$ in probability. So, Bayesian inference (that is, conditional on ${\mathit{X}}_{\mathit{n}}$) can be used to approximate frequentist inference.

For $\mathit{t}(\mathit{\theta })$ any smooth function, we obtain posterior cumulant expansions, posterior Edgeworth–Cornish–Fisher (ECF) expansions and posterior tilted Edgeworth expansions for $\mathcal{L}\mathit{t}(\mathit{\theta })|{\mathit{X}}_{\mathit{n}}$, as well as confidence regions for $\mathit{t}(\mathit{\theta })|{\mathit{X}}_{\mathit{n}}$ of high accuracy. We also give expansions for the Bayes estimate (estimator) of $\mathit{t}(\mathit{\theta })$ about $\mathit{t}(\hat{\mathit{\theta }})$, and for the maximum a posteriori estimate about $\hat{\mathit{\theta }}$, as well as their relative efficiencies with respect to squared error loss.