Electronic Journal of Statistics

Geometric ergodicity of Pólya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior

Xin Wang and Vivekananda Roy

Full-text: Open access

Abstract

The logistic regression model is the most popular model for analyzing binary data. In the absence of any prior information, an improper flat prior is often used for the regression coefficients in Bayesian logistic regression models. The resulting intractable posterior density can be explored by running Polson, Scott and Windle’s (2013) data augmentation (DA) algorithm. In this paper, we establish that the Markov chain underlying Polson, Scott and Windle’s (2013) DA algorithm is geometrically ergodic. Proving this theoretical result is practically important as it ensures the existence of central limit theorems (CLTs) for sample averages under a finite second moment condition. The CLT in turn allows users of the DA algorithm to calculate standard errors for posterior estimates.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 3295-3311.

Dates
Received: February 2018
First available in Project Euclid: 5 October 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1538705039

Digital Object Identifier
doi:10.1214/18-EJS1481

Subjects
Primary: 60J05: Discrete-time Markov processes on general state spaces
Secondary: 62F15: Bayesian inference

Keywords
Central limit theorem data augmentation drift condition geometric rate Markov chain posterior propriety

Rights
Creative Commons Attribution 4.0 International License.

Citation

Wang, Xin; Roy, Vivekananda. Geometric ergodicity of Pólya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior. Electron. J. Statist. 12 (2018), no. 2, 3295--3311. doi:10.1214/18-EJS1481. https://projecteuclid.org/euclid.ejs/1538705039


Export citation

References

  • Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data., Journal of the American statistical Association 88 669–679.
  • Asmussen, S. and Glynn, P. W. (2011). A new proof of convergence of MCMC via the ergodic theorem., Statistics & Probability Letters 81 1482–1485.
  • Athreya, K. B. and Roy, V. (2014). Monte Carlo methods for improper target distributions., Electronic Journal of Statistics 8 2664–2692.
  • Bernstein, D. S. (2005)., Matrix mathematics: Theory, facts, and formulas with application to linear systems theory 41. Princeton University Press.
  • Chen, M.-H. and Shao, Q.-M. (2001). Propriety of posterior distribution for dichotomous quantal response models., Proceedings of the American Mathematical Society 129 293–302.
  • Choi, H. M. and Hobert, J. P. (2013). The Pólya-Gamma Gibbs sampler for Bayesian logistic regression is uniformly ergodic., Electronic Journal of Statistics 7 2054–2064.
  • Choi, H. M. and Román, J. C. (2017). Analysis of Pólya-Gamma Gibbs sampler for Bayesian logistic analysis of variance., Electronic Journal of Statistics 11 326–337.
  • Flegal, J. M. and Jones, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo., The Annals of Statistics 38 1034–1070.
  • Frühwirth-Schnatter, S. and Frühwirth, R. (2010). Data augmentation and MCMC for binary and multinomial logit models. In, Statistical Modelling and Regression Structures 111–132. Springer.
  • Holmes, C. C. and Held, L. (2006). Bayesian auxiliary variable models for binary and multinomial regression., Bayesian analysis 1 145–168.
  • Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo., Statistical Science 16 312–334.
  • McCulloch, C. E., Searle, S. R. and Neuhaus, J. M. (2011)., Generalized, Linear, and Mixed Models. John Wiley & Sons.
  • Meyn, S. P. and Tweedie, R. L. (1993)., Markov chains and stochastic stability. Springer.
  • Olver, F. W. J., Lozier, D. W., F., B. R. and Clark, C. W. (2010)., NIST Handbook of Mathematical Functions. Cambridge University Press.
  • Polson, N. G., Scott, J. G. and Windle, J. (2013). Bayesian inference for logistic models using Pólya-Gamma latent variables., Journal of the American statistical Association 108 1339–1349.
  • Roberts, G. O. and Rosenthal, J. S. (1997). Geometric ergodicity and hybrid Markov chains., Electron. Comm. Probab 2 13–25.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes., Scandinavian Journal of Statistics 28 489–504.
  • Roy, V. and Hobert, J. P. (2007). Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression., Journal of the Royal Statistical Society: Series B 69 607–623.
  • Roy, V. and Hobert, J. P. (2010). On Monte Carlo methods for Bayesian multivariate regression models with heavy-tailed errors., Journal of Multivariate Analysis 101 1190–1202.