We study the convergence properties of the Gibbs Sampler in the context of posterior distributions arising from Bayesian analysis of conditionally Gaussian hierarchical models. We develop a multigrid approach to derive analytic expressions for the convergence rates of the algorithm for various widely used model structures, including nested and crossed random effects. Our results apply to multilevel models with an arbitrary number of layers in the hierarchy, while most previous work was limited to the two-level nested case. The theoretical results provide explicit and easy-to-implement guidelines to optimize practical implementations of the Gibbs Sampler, such as indications on which parametrization to choose (e.g. centred and non-centred), which constraint to impose to guarantee statistical identifiability, and which parameters to monitor in the diagnostic process. Simulations suggest that the results are informative also in the context of non-Gaussian distributions and more general MCMC schemes, such as gradient-based ones.
In the corrected version, we updated and corrected the references Bass and Sahu (2016a) and Bass and Sahu (2016b), which became Bass and Sahu (2017) and Bass and Sahu (2019) in the new version.
BA Webinar: https://www.youtube.com/watch?v=srfWsSKAc4E
The authors are grateful for stimulating discussions with Omiros Papaspiliopoulos and Art Owen. GZ was supported in part by an EPSRC Doctoral Prize fellowship, by the European Research Council (ERC) through StG “N-BNP” 306406 and by MIUR through the PRIN Project 2015SNS29B. GOR acknowledges support from EPSRC through grants EP/K014463/1 (i-Like), EP/D002060/1 (CRiSM), EP/R034710/1 (CoSInES), and EP/R018561/1 (Bayes for Health).
Giacomo Zanella. Gareth Roberts. "Multilevel Linear Models, Gibbs Samplers and Multigrid Decompositions (with Discussion)." Bayesian Anal. 16 (4) 1309 - 1391, December 2021. https://doi.org/10.1214/20-BA1242