Open Access
2023 Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions
Chirag Modi, Alex Barnett, Bob Carpenter
Author Affiliations +
Bayesian Anal. Advance Publication 1-28 (2023). DOI: 10.1214/23-BA1360


The efficiency of Hamiltonian Monte Carlo (HMC) can suffer when sampling a distribution with a wide range of length scales, because the small step sizes needed for stability in high-curvature regions are inefficient elsewhere. To address this we present a delayed rejection (DR) variant: if an initial HMC trajectory is rejected, we make one or more subsequent proposals each using a step size geometrically smaller than the last. To reduce the cost of DR approaches, we extend the standard delayed rejection to a probabilistic framework wherein we do not make multiple proposals at every rejection, but allow the probability of a retry to depend on the probability of accepting the previous proposal. We test the scheme in several sampling tasks, including statistical applications and multiscale model distributions such as Neal’s funnel. Delayed rejection enables sampling multiscale distributions for which standard approaches such as HMC fail to explore the tails, and improves performance five-fold over optimally-tuned HMC as measured by effective sample size per gradient evaluation. Even for simpler distributions, delayed rejection provides increased robustness to step size misspecification.


Download Citation

Chirag Modi. Alex Barnett. Bob Carpenter. "Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions." Bayesian Anal. Advance Publication 1 - 28, 2023.


Published: 2023
First available in Project Euclid: 20 February 2023

arXiv: 2110.00610
Digital Object Identifier: 10.1214/23-BA1360

Keywords: delayed rejection , detailed balance , Hamiltonian Monte Carlo , multiscale distributions

Advance Publication
Back to Top