Open Access
August 2019 Conditionally Conjugate Mean-Field Variational Bayes for Logistic Models
Daniele Durante, Tommaso Rigon
Statist. Sci. 34(3): 472-485 (August 2019). DOI: 10.1214/19-STS712

Abstract

Variational Bayes (VB) is a common strategy for approximate Bayesian inference, but simple methods are only available for specific classes of models including, in particular, representations having conditionally conjugate constructions within an exponential family. Models with logit components are an apparently notable exception to this class, due to the absence of conjugacy among the logistic likelihood and the Gaussian priors for the coefficients in the linear predictor. To facilitate approximate inference within this widely used class of models, Jaakkola and Jordan (Stat. Comput. 10 (2000) 25–37) proposed a simple variational approach which relies on a family of tangent quadratic lower bounds of the logistic log-likelihood, thus restoring conjugacy between these approximate bounds and the Gaussian priors. This strategy is still implemented successfully, but few attempts have been made to formally understand the reasons underlying its excellent performance. Following a review on VB for logistic models, we cover this gap by providing a formal connection between the above bound and a recent Pólya-gamma data augmentation for logistic regression. Such a result places the computational methods associated with the aforementioned bounds within the framework of variational inference for conditionally conjugate exponential family models, thereby allowing recent advances for this class to be inherited also by the methods relying on Jaakkola and Jordan (Stat. Comput. 10 (2000) 25–37).

Citation

Download Citation

Daniele Durante. Tommaso Rigon. "Conditionally Conjugate Mean-Field Variational Bayes for Logistic Models." Statist. Sci. 34 (3) 472 - 485, August 2019. https://doi.org/10.1214/19-STS712

Information

Published: August 2019
First available in Project Euclid: 11 October 2019

zbMATH: 07162133
MathSciNet: MR4017524
Digital Object Identifier: 10.1214/19-STS712

Keywords: EM , logistic regression , Pólya-gamma data augmentation , quadratic approximation , variational Bayes

Rights: Copyright © 2019 Institute of Mathematical Statistics

Vol.34 • No. 3 • August 2019
Back to Top