Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose “marginally augmented variational Bayes” (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions.
I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.
Open-source software to implement the models in this paper in R can be downloaded from github.com/mgoplerud/vglmer. I thank the anonymous reviewers, Michael Auslen, Naoki Egami, Shusei Eshima, Justin Grimmer, June Hwang, Kosuke Imai, Pierre Jacob, Gary King, Shiro Kuriwaki, Marc Ratkovic, Sun Young Park, Casey Petroff, Marc Ratkovic, Tyler Simko, Diana Stanescu, Dustin Tingley, Soichiro Yamauchi, and participants at PolMeth 2020 and the University of Pittsburgh’s Statistics Seminar for helpful comments on earlier versions of this paper. All remaining errors are my own.
"Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference." Bayesian Anal. Advance Publication 1 - 28, 2021. https://doi.org/10.1214/21-BA1266