Open Access
November 2013 Optimal tuning of the hybrid Monte Carlo algorithm
Alexandros Beskos, Natesh Pillai, Gareth Roberts, Jesus-Maria Sanz-Serna, Andrew Stuart
Bernoulli 19(5A): 1501-1534 (November 2013). DOI: 10.3150/12-BEJ414


We investigate the properties of the hybrid Monte Carlo algorithm (HMC) in high dimensions. HMC develops a Markov chain reversible with respect to a given target distribution $\Pi$ using separable Hamiltonian dynamics with potential $-\log\Pi$. The additional momentum variables are chosen at random from the Boltzmann distribution, and the continuous-time Hamiltonian dynamics are then discretised using the leapfrog scheme. The induced bias is removed via a Metropolis–Hastings accept/reject rule. In the simplified scenario of independent, identically distributed components, we prove that, to obtain an $\mathcal{O}(1)$ acceptance probability as the dimension $d$ of the state space tends to $\infty$, the leapfrog step size $h$ should be scaled as $h=l\times d^{-1/4}$. Therefore, in high dimensions, HMC requires $\mathcal{O}(d^{1/4})$ steps to traverse the state space. We also identify analytically the asymptotically optimal acceptance probability, which turns out to be $0.651$ (to three decimal places). This value optimally balances the cost of generating a proposal, which decreases as $l$ increases (because fewer steps are required to reach the desired final integration time), against the cost related to the average number of proposals required to obtain acceptance, which increases as $l$ increases.


Download Citation

Alexandros Beskos. Natesh Pillai. Gareth Roberts. Jesus-Maria Sanz-Serna. Andrew Stuart. "Optimal tuning of the hybrid Monte Carlo algorithm." Bernoulli 19 (5A) 1501 - 1534, November 2013.


Published: November 2013
First available in Project Euclid: 5 November 2013

zbMATH: 1287.60090
MathSciNet: MR3129023
Digital Object Identifier: 10.3150/12-BEJ414

Keywords: Hamiltonian dynamics , high dimensions , leapfrog scheme , optimal acceptance probability , squared jumping distance

Rights: Copyright © 2013 Bernoulli Society for Mathematical Statistics and Probability

Vol.19 • No. 5A • November 2013
Back to Top