Translator Disclaimer
2021 Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference
Max Goplerud
Author Affiliations +
Bayesian Anal. Advance Publication 1-28 (2021). DOI: 10.1214/21-BA1266

Abstract

Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose “marginally augmented variational Bayes” (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions.

I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.

Acknowledgments

Open-source software to implement the models in this paper in R can be downloaded from github.com/mgoplerud/vglmer. I thank the anonymous reviewers, Michael Auslen, Naoki Egami, Shusei Eshima, Justin Grimmer, June Hwang, Kosuke Imai, Pierre Jacob, Gary King, Shiro Kuriwaki, Marc Ratkovic, Sun Young Park, Casey Petroff, Marc Ratkovic, Tyler Simko, Diana Stanescu, Dustin Tingley, Soichiro Yamauchi, and participants at PolMeth 2020 and the University of Pittsburgh’s Statistics Seminar for helpful comments on earlier versions of this paper. All remaining errors are my own.

Citation

Download Citation

Max Goplerud. "Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference." Bayesian Anal. Advance Publication 1 - 28, 2021. https://doi.org/10.1214/21-BA1266

Information

Published: 2021
First available in Project Euclid: 23 April 2021

Digital Object Identifier: 10.1214/21-BA1266

JOURNAL ARTICLE
28 PAGES


SHARE
Advance Publication
Back to Top