Open Access
Translator Disclaimer
September 2021 Fast Exact Bayesian Inference for Sparse Signals in the Normal Sequence Model
Tim van Erven, Botond Szabó
Author Affiliations +
Bayesian Anal. 16(3): 933-960 (September 2021). DOI: 10.1214/20-BA1227


We consider exact algorithms for Bayesian inference with model selection priors (including spike-and-slab priors) in the sparse normal sequence model. Because the best existing exact algorithm becomes numerically unstable for sample sizes over n=500, there has been much attention for alternative approaches like approximate algorithms (Gibbs sampling, variational Bayes, etc.), shrinkage priors (e.g. the Horseshoe prior and the Spike-and-Slab LASSO) or empirical Bayesian methods. However, by introducing algorithmic ideas from online sequential prediction, we show that exact calculations are feasible for much larger sample sizes: for general model selection priors we reach n=25000, and for certain spike-and-slab priors we can easily reach n=100000. We further prove a de Finetti-like result for finite sample sizes that characterizes exactly which model selection priors can be expressed as spike-and-slab priors. The computational speed and numerical accuracy of the proposed methods are demonstrated in experiments on simulated data, on a differential gene expression data set, and to compare the effect of multiple hyper-parameter settings in the beta-binomial prior. In our experimental evaluation we compute guaranteed bounds on the numerical accuracy of all new algorithms, which shows that the proposed methods are numerically reliable whereas an alternative based on long division is not.


Download Citation

Tim van Erven. Botond Szabó. "Fast Exact Bayesian Inference for Sparse Signals in the Normal Sequence Model." Bayesian Anal. 16 (3) 933 - 960, September 2021.


Published: September 2021
First available in Project Euclid: 6 August 2020

Digital Object Identifier: 10.1214/20-BA1227

Primary: 62G05
Secondary: 62F15

Keywords: High-dimensional statistics , Model selection , spike-and-slab prior


Vol.16 • No. 3 • September 2021
Back to Top