Divide-and-conquer Metropolis–Hastings samplers with matched samples

Abstract

Divide-and-conquer methods for scalable Bayesian inference divide the massive data into subsets, sample from the subset posterior distributions, and then combine the results. We develop an asymptotically exact recombination method by matched samples. Subset posterior densities calculated by the Metropolis–Hastings samplers are recycled for evaluating the importance weight to reduce the computational burden. Our computationally efficient aggregation algorithm features a collection of consistent estimators of expectations with respect to the full posterior distribution. Weight degeneracy of the importance sampling is resolved by the matched-sample resample-move method, which handles heterogeneous and non-overlapping subposteriors. Numeric examples and real-world mortgage data applications demonstrate excellent performance of the novel approach.