Counterexamples for optimal scaling of Metropolis–Hastings chains with rough target densities

Abstract

For sufficiently smooth targets of product form it is known that the variance of a single coordinate of the proposal in RWM (random walk Metropolis) and MALA (Metropolis adjusted Langevin algorithm) should optimally scale as ${\mathit{n}}^{-1}$ and as ${\mathit{n}}^{-\frac{1}{3}}$ with dimension n, and that the acceptance rates should be tuned to 0.234 and 0.574. We establish counterexamples to demonstrate that smoothness assumptions of the order of ${\mathcal{C}}^1(\mathbb{R})$ for RWM and ${\mathcal{C}}^3(\mathbb{R})$ for MALA are indeed required if these scaling rates are to hold. The counterexamples identify classes of marginal targets for which these guidelines are violated, obtained by perturbing a standard normal density (at the level of the potential for RWM and the second derivative of the potential for MALA) using roughness generated by a path of fractional Brownian motion with Hurst exponent H. For such targets there is strong evidence that RWM and MALA proposal variances should optimally be scaled as ${\mathit{n}}^{-\frac{1}{\mathit{H}}}$ and as ${\mathit{n}}^{-\frac{1}{2+\mathit{H}}}$ and will then obey anomalous acceptance rate guidelines. Useful heuristics resulting from this theory are discussed. The paper develops a framework capable of tackling optimal scaling results for quite general Metropolis–Hastings algorithms (possibly depending on a random environment).