Statistical Science

Bayes, Oracle Bayes and Empirical Bayes

Bradley Efron

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


This article concerns the Bayes and frequentist aspects of empirical Bayes inference. Some of the ideas explored go back to Robbins in the 1950s, while others are current. Several examples are discussed, real and artificial, illustrating the two faces of empirical Bayes methodology: “oracle Bayes” shows empirical Bayes in its most frequentist mode, while “finite Bayes inference” is a fundamentally Bayesian application. In either case, modern theory and computation allow us to present a sharp finite-sample picture of what is at stake in an empirical Bayes analysis.

Article information

Statist. Sci., Volume 34, Number 2 (2019), 177-201.

First available in Project Euclid: 19 July 2019

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Finite Bayes inference $g$-modeling relevance empirical Bayes regret


Efron, Bradley. Bayes, Oracle Bayes and Empirical Bayes. Statist. Sci. 34 (2019), no. 2, 177--201. doi:10.1214/18-STS674.

Export citation


  • Brown, L. D. and Greenshtein, E. (2009). Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means. Ann. Statist. 37 1685–1704.
  • Carlin, B. P. and Gelfand, A. E. (1991). A sample reuse method for accurate parametric empirical Bayes confidence intervals. J. Roy. Statist. Soc. Ser. B 53 189–200.
  • Deely, J. J. and Lindley, D. V. (1981). Bayes empirical Bayes. J. Amer. Statist. Assoc. 76 833–841.
  • Efron, B. (1996). Empirical Bayes methods for combining likelihoods. J. Amer. Statist. Assoc. 91 538–565.
  • Efron, B. (2010). Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. Institute of Mathematical Statistics (IMS) Monographs 1. Cambridge Univ. Press, Cambridge.
  • Efron, B. (2011). Tweedie’s formula and selection bias. J. Amer. Statist. Assoc. 106 1602–1614.
  • Efron, B. (2014). Two modeling strategies for empirical Bayes estimation. Statist. Sci. 29 285–301.
  • Efron, B. (2016). Empirical Bayes deconvolution estimates. Biometrika 103 1–20.
  • Efron, B. and Hastie, T. (2016). Computer Age Statistical Inference: Algorithms, Evidence, and Data Science. Institute of Mathematical Statistics (IMS) Monographs 5. Cambridge Univ. Press, New York.
  • Efron, B. and Morris, C. (1972). Limiting the risk of Bayes and empirical Bayes estimators. II. The empirical Bayes case. J. Amer. Statist. Assoc. 67 130–139.
  • Fisher, R. A., Corbet, A. S. and Williams, C. B. (1943). The relation between the number of species and the number of individuals in a random sample of an animal population. J. Anim. Ecol. 12 42–58.
  • Good, I. J. and Toulmin, G. H. (1956). The number of new species, and the increase in population coverage, when a sample is increased. Biometrika 43 45–63.
  • Gu, J. and Koenker, R. (2016). On a problem of Robbins. Int. Stat. Rev. 84 224–244.
  • Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer Series in Statistics. Springer, New York.
  • Jiang, W. and Zhang, C.-H. (2009). General maximum likelihood empirical Bayes estimation of normal means. Ann. Statist. 37 1647–1684.
  • Kiefer, J. and Wolfowitz, J. (1956). Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann. Math. Stat. 27 887–906.
  • Koenker, R. and Mizera, I. (2014). Convex optimization, shape constraints, compound decisions, and empirical Bayes rules. J. Amer. Statist. Assoc. 109 674–685.
  • Laird, N. (1978). Nonparametric maximum likelihood estimation of a mixed distribution. J. Amer. Statist. Assoc. 73 805–811.
  • Laird, N. M. and Louis, T. A. (1987). Empirical Bayes confidence intervals based on bootstrap samples. J. Amer. Statist. Assoc. 82 739–757.
  • Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications. J. Amer. Statist. Assoc. 78 47–65.
  • Robbins, H. (1951). Asymptotically subminimax solutions of compound statistical decision problems. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950 131–148. Univ. California Press, Berkeley.
  • Robbins, H. (1956). An empirical Bayes approach to statistics. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 157–163. Univ. California Press, Berkeley.
  • Schwartzman, A., Dougherty, R. F. and Taylor, J. E. (2005). Cross-subject comparison of principal diffusion direction maps. Magn. Reson. Med. 53 1423–1431.
  • Zhang, C.-H. (2003). Compound decision theory and empirical Bayes methods. Ann. Statist. 31 379–390.