Many applications in Bayesian statistics are extremely computationally intensive. However, they are often inherently parallel, making them prime targets for modern massively parallel processors. Multi-core and distributed computing is widely applied in the Bayesian community, however, very little attention has been given to fine-grain parallelisation using single instruction multiple data (SIMD) operations that are available on most modern CPUs. In this work, we practically demonstrate, using standard programming libraries, the utility of the SIMD approach for several topical Bayesian applications. Using the C programming language, we show that SIMD can improve the single-core floating point arithmetic performance by up to a factor of compared scalar C code and more than compared with optimised R code. Such improvements are multiplicative to any gains achieved through multi-core processing. We illustrate the potential of SIMD for accelerating Bayesian computations and provide the reader with techniques for exploiting modern massively parallel processing environments.
C.D. was supported by the Australian Research Council (ARC) under the Discovery Project scheme (DP200102101). S.A.S. was supported by the ARC under the Discovery Project scheme (DP160102544). C.D., S.A.S., and D.J.W. are supported by the ARC Centre of Excellence for Mathematical and Statistical Frontiers (ACEMS; CE140100049). C.D. and D.J.W. also acknowledge support from the Centre for Data Science, Queensland University of Technology. Computational resources were provided by the eResearch Office, Queensland University of Technology.
"Vector Operations for Accelerating Expensive Bayesian Computations – A Tutorial Guide." Bayesian Anal. 17 (2) 593 - 622, June 2022. https://doi.org/10.1214/21-BA1265