Electronic Journal of Statistics

Empirical Bayes analysis of spike and slab posterior distributions

Ismaël Castillo and Romain Mismer

Full-text: Open access

Abstract

In the sparse normal means model, convergence of the Bayesian posterior distribution associated to spike and slab prior distributions is considered. The key sparsity hyperparameter is calibrated via marginal maximum likelihood empirical Bayes. The plug-in posterior squared–$L^{2}$ norm is shown to converge at the minimax rate for the euclidean norm for appropriate choices of spike and slab distributions. Possible choices include standard spike and slab with heavy tailed slab, and the spike and slab LASSO of Ročková and George with heavy tailed slab. Surprisingly, the popular Laplace slab is shown to lead to a suboptimal rate for the empirical Bayes posterior itself. This provides a striking example where convergence of aspects of the empirical Bayes posterior such as the posterior mean or median does not entail convergence of the complete empirical Bayes posterior itself.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 3953-4001.

Dates
Received: December 2017
First available in Project Euclid: 8 December 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1544238109

Digital Object Identifier
doi:10.1214/18-EJS1494

Subjects
Primary: 62G20: Asymptotic properties

Keywords
Convergence rates of posterior distributions spike and slab spike and slab LASSO Empirical Bayes

Rights
Creative Commons Attribution 4.0 International License.

Citation

Castillo, Ismaël; Mismer, Romain. Empirical Bayes analysis of spike and slab posterior distributions. Electron. J. Statist. 12 (2018), no. 2, 3953--4001. doi:10.1214/18-EJS1494. https://projecteuclid.org/euclid.ejs/1544238109


Export citation

References

  • [1] A. Babenko and E. Belitser. Oracle convergence rate of posterior under projection prior and Bayesian model selection., Math. Methods Statist., 19(3):219–245, 2010.
  • [2] E. Belitser and N. Nurushev. Needles and straw in a haystack: robust empirical Bayes confidence for possibly sparse sequences., ArXiv e-print 1511.01803, Nov. 2015.
  • [3] I. Castillo. Lower bounds for posterior rates with Gaussian process priors., Electronic Journal of Statistics, 2 :1281–1299, 2008.
  • [4] I. Castillo. Discussion of: Uncertainty quantification with the horseshoe., Bayesian Analysis, 12(4) :1250–1253, 2017.
  • [5] I. Castillo and B. T. Szabó. Spike and Slab Empirical Bayes sparse credible sets., ArXiv e-print 1808.07721, aug 2018.
  • [6] I. Castillo and A. W. van der Vaart. Needles and straw in a haystack: posterior concentration for possibly sparse sequences., Ann. Statist., 40(4) :2069–2101, 2012.
  • [7] D. L. Donoho, I. M. Johnstone, J. C. Hoch, and A. S. Stern. Maximum entropy and the nearly black object., J. Roy. Statist. Soc. Ser. B, 54(1):41–81, 1992. With discussion and a reply by the authors.
  • [8] E. I. George. The variable selection problem., J. Amer. Statist. Assoc., 95(452) :1304–1308, 2000.
  • [9] E. I. George and D. P. Foster. Calibration and empirical Bayes variable selection., Biometrika, 87(4):731–747, 2000.
  • [10] S. Ghosal, J. K. Ghosh, and A. W. van der Vaart. Convergence rates of posterior distributions., Ann. Statist., 28(2):500–531, 2000.
  • [11] W. Jiang and C.-H. Zhang. General maximum likelihood empirical Bayes estimation of normal means., Ann. Statist., 37(4) :1647–1684, 2009.
  • [12] I. M. Johnstone and B. W. Silverman. Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences., Ann. Statist., 32(4) :1594–1649, 2004.
  • [13] I. M. Johnstone and B. W. Silverman. EbayesThresh: R Programs for Empirical Bayes Thresholding., Journal of Statistical Software, 12(8), 2005.
  • [14] R. Martin and S. G. Walker. Asymptotically minimax empirical bayes estimation of a sparse normal mean vector., Electron. J. Statist., 8(2) :2188–2206, 2014.
  • [15] T. J. Mitchell and J. J. Beauchamp. Bayesian variable selection in linear regression., J. Amer. Statist. Assoc., 83(404) :1023–1036, 1988. With comments by James Berger and C. L. Mallows and with a reply by the authors.
  • [16] V. Ročková. Bayesian estimation of sparse signals with a continuous spike-and-slab prior., Ann. Statist., 46(1):401–437, 2018.
  • [17] V. Ročková and E. I. George. The spike-and-slab LASSO., J. Amer. Statist. Assoc., 113(521):431–444, 2018.
  • [18] S. van der Pas, B. Szabó, and A. van der Vaart. Adaptive posterior contraction rates for the horseshoe., Electron. J. Stat., 11(2) :3196–3225, 2017.
  • [19] S. van der Pas, B. Szabó, and A. van der Vaart. Uncertainty quantification for the horseshoe (with discussion)., Bayesian Anal., 12(4) :1221–1274, 2017. With a rejoinder by the authors.
  • [20] S. L. van der Pas, J.-B. Salomond, and J. Schmidt-Hieber. Conditions for posterior contraction in the sparse normal means problem., Electron. J. Stat., 10(1):976 –1000, 2016.
  • [21] M. Yuan and Y. Lin. Efficient empirical Bayes variable selection and estimation in linear models., J. Amer. Statist. Assoc., 100(472) :1215–1225, 2005.