Accelerating Gaussian Diffusions

Abstract

Let $\pi(x)$ be a given probability density proportional to $\exp(-U(x))$ in a high-dimensional Euclidean space $\mathbb{R}^m$. The diffusion $dX(t) = -\nabla U(X(t))dt + \sqrt 2 dW(t)$ is often used to sample from $\pi$. Instead of $-\nabla U(x)$, we consider diffusions with smooth drift $b(x)$ and having equilibrium $\pi(x)$. First we study some general properties and then concentrate on the Gaussian case, namely, $-\nabla U(x) = Dx$ with a strictly negative-definite real matrix $D$ and $b(x) = Bx$ with a stability matrix $B$; that is, the real parts of the eigenvalues of $B$ are strictly negative. Using the rate of convergence of the covariance of $X(t)$ [or together with $EX(t)$] as the criterion, we prove that, among all such $b(x)$, the drift $Dx$ is the worst choice and that improvement can be made if and only if the eigenvalues of $D$ are not identical. In fact, the convergence rate of the covariance is $\exp(2\lambda_M(B)t)$, where $\lambda_M(B)$ is the maximum of the real parts of the eigenvalues of $B$ and the infimum of $\lambda_M(B)$ over all such $B$ is $1/m \operatorname{tr} D$. If, for example, a "circulant" drift $\bigg(\frac{\partial U}{\partial x_m} - \frac{\partial U}{\partial x_2},\frac{\partial U}{\partial x_1} - \frac{\partial U}{\partial x_3}, \cdots, \frac{\partial U}{\partial x_{m-1}} - \frac{\partial U}{\partial x_1}\bigg)$ is added to $Dx$, then for essentially all $D$, the diffusion with this modified drift has a better convergence rate.