Antithetic coupling of two Gibbs sampler chains

Abstract

Two coupled Gibbs sampler chains, both with invariant probability density $p$, are run in parallel so that the chains are negatively correlated.We define an asymptotically unbiased estimator of the $\pi$-expectation $E(f(\mathbf(X))$ which achieves significant variance reduction with respect to the usual Gibbs sampler at comparable computational cost. The variance of the estimator based on the new algorithm is always smaller than the variance of a single Gibbs sampler chain, if $\pi$ is attractive and $f$ is monotone nondecreasing in all components of $\mathbf{X}$. For nonattractive targets $\pi$, our results are not complete: The new antithetic algorithm outperforms the standard Gibbs sampler when $\pi$ is a multivariate normal density or the Ising model. More generally, nonrigorous arguments and numerical experiments support the usefulness of the antithetically coupled Gibbs samplers also for other nonattractive models. In our experiments the variance is reduced to at least a third and the efficiency also improves significantly.