Annals of Statistics

Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means

Lawrence D. Brown and Eitan Greenshtein

Full-text: Open access


We consider the classical problem of estimating a vector μ=(μ1, …, μn) based on independent observations YiN(μi, 1), i=1, …, n.

Suppose μi, i=1, …, n are independent realizations from a completely unknown G. We suggest an easily computed estimator μ̂, such that the ratio of its risk E(μ̂μ)2 with that of the Bayes procedure approaches 1. A related compound decision result is also obtained.

Our asymptotics is of a triangular array; that is, we allow the distribution G to depend on n. Thus, our theoretical asymptotic results are also meaningful in situations where the vector μ is sparse and the proportion of zero coordinates approaches 1.

We demonstrate the performance of our estimator in simulations, emphasizing sparse setups. In “moderately-sparse” situations, our procedure performs very well compared to known procedures tailored for sparse setups. It also adapts well to nonsparse situations.

Article information

Ann. Statist., Volume 37, Number 4 (2009), 1685-1704.

First available in Project Euclid: 18 June 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C12: Empirical decision procedures; empirical Bayes procedures 62C25: Compound decision problems

Empirical Bayes compound decision


Brown, Lawrence D.; Greenshtein, Eitan. Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means. Ann. Statist. 37 (2009), no. 4, 1685--1704. doi:10.1214/08-AOS630.

Export citation


  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855–904.
  • Brown, L. D. (2008). In-season prediction of bating averages: A field test of simple empirical Bayes and Bayes methodologies. Ann. Appl. Statist. 2 113–152.
  • Copas, J. B. (1969). Compound decisions and empirical Bayes (with discussion). J. Roy. Statist. Soc. Ser. B 31 397–425.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
  • Efron, B. (2003). Robbins, empirical Bayes, and microarrays. Ann. Statist. 31 366–378.
  • Greenshtein, E. and Park, J. (2007). Application of nonparametric empirical Bayes to high-dimensional classification.
  • Greenshtein, E. and Ritov, Y. (2008). Asymptotic efficiency of simple decisions for the compound decision problem. In The 3rd Lehmann Symposium (J. Rojo, ed.). IMS Lecture Notes Monograph. To appear.
  • Johnstone, I. M. and Silverman, B. W. (2004). Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594–1649.
  • Robbins, H. (1951). Asymptotically subminimax solutions of compound decision problems. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950 131–148. Univ. California, Berkeley.
  • Robbins, H. (1956). An empirical Bayes approach to statistics. In Proc. Third Berkeley Symp. 157–164. Univ. California Press, Berkeley.
  • Robbins, H. (1964). The empirical Bayes approach to statistical decision problems. Ann. Math. Statist. 35 1–20.
  • Samuel, E. (1965). On simple rules for the compound decision problem. J. Roy. Statist. Soc. Ser. B 27 238–244.
  • van der Vaart, A. W. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • Wenhua, J. and Zhang, C.-H. (2007). General maximum likelihood empirical Bayes estimation of normal means. Manuscript.
  • Zhang, C.-H. (1997). Empirical Bayes and compound estimation of a normal mean. Statist. Sinica 7 181–193.
  • Zhang, C.-H. (2003). Compound decision theory and empirical Bayes methods (invited paper). Ann. Statist. 31 379–390.
  • Zhang C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54–100.