Brazilian Journal of Probability and Statistics
- Braz. J. Probab. Stat.
- Volume 31, Number 3 (2017), 569-617.
A brief tutorial on transformation based Markov Chain Monte Carlo and optimal scaling of the additive transformation
We consider the recently introduced Transformation-based Markov Chain Monte Carlo (TMCMC) (Stat. Methodol. 16 (2014) 100–116), a methodology that is designed to update all the parameters simultaneously using some simple deterministic transformation of a one-dimensional random variable drawn from some arbitrary distribution on a relevant support. The additive transformation based TMCMC is similar in spirit to random walk Metropolis, except the fact that unlike the latter, additive TMCMC uses a single draw from a one-dimensional proposal distribution to update the high-dimensional parameter. In this paper, we first provide a brief tutorial on TMCMC, exploring its connections and contrasts with various available MCMC methods.
Then we study the diffusion limits of additive TMCMC under various set-ups ranging from the product structure of the target density to the case where the target is absolutely continuous with respect to a Gaussian measure; we also consider the additive TMCMC within Gibbs approach for all the above set-ups. These investigations lead to appropriate scaling of the one-dimensional proposal density. We also show that the optimal acceptance rate of additive TMCMC is 0.439 under all the aforementioned set-ups, in contrast with the well-established 0.234 acceptance rate associated with optimal random walk Metropolis algorithms under the same set-ups. We also elucidate the ramifications of our results and clear advantages of additive TMCMC over random walk Metropolis with ample simulation studies and Bayesian analysis of a real, spatial dataset with which $160$ unknowns are associated.
Braz. J. Probab. Stat., Volume 31, Number 3 (2017), 569-617.
Received: March 2015
Accepted: June 2016
First available in Project Euclid: 22 August 2017
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Dey, Kushal Kr.; Bhattacharya, Sourabh. A brief tutorial on transformation based Markov Chain Monte Carlo and optimal scaling of the additive transformation. Braz. J. Probab. Stat. 31 (2017), no. 3, 569--617. doi:10.1214/16-BJPS325. https://projecteuclid.org/euclid.bjps/1503388830
- Supplement to “A brief tutorial on transformation based Markov Chain Monte Carlo and optimal scaling of the additive transformation”. Additional details are provided in this supplementary material, whose sections and figures have the prefix “S-” when referred to in this article. Briefly, in Section S-1, we provide details on computational efficiency of TMCMC. Specifically, we demonstrate with an experiment the superior computational speed of additive TMCMC in comparison with RWM, particularly in high dimensions. In Section S-2 we discuss, with appropriate experiments, the necessity of optimal scaling in additive TMCMC, while in Sections S-3 and S-4 we delve into the robustness issues associated with the scale choices of additive TMCMC and RWM. In Section S-5, we include brief discussions of adaptive versions of RWM and TMCMC. Moreover, the proofs of all our technical results are provided in Sections S-6 and S-7 of the supplement.