August 2022 Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks
Theodore Papamarkou, Jacob Hinkle, M. Todd Young, David Womble
Author Affiliations +
Statist. Sci. 37(3): 425-442 (August 2022). DOI: 10.1214/21-STS840

Abstract

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a nonconverged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research avenue.

Funding Statement

Research sponsored by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U.S. Department of Energy under contract DE-AC05-00OR22725.

Acknowledgement

The first author would like to thank Google for the provision of free credit on Google Cloud Platform.

Citation

Download Citation

Theodore Papamarkou. Jacob Hinkle. M. Todd Young. David Womble. "Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks." Statist. Sci. 37 (3) 425 - 442, August 2022. https://doi.org/10.1214/21-STS840

Information

Published: August 2022
First available in Project Euclid: 21 June 2022

MathSciNet: MR4444376
zbMATH: 07569970
Digital Object Identifier: 10.1214/21-STS840

Keywords: Bayesian inference , Bayesian neural networks , convergence diagnostics , Markov chain Monte Carlo , posterior predictive distribution

Rights: Copyright © 2022 Institute of Mathematical Statistics

Vol.37 • No. 3 • August 2022
Back to Top