On general maximum likelihood empirical Bayes estimation of heteroscedastic IID normal means

Abstract

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the heteroscedastic normal means estimation with known variances. The idea is to plug the generalized maximum likelihood estimator in the oracle Bayes rule. From the point of view of restricted empirical Bayes, the general empirical Bayes aims at a benchmark risk smaller than the linear empirical Bayes methods when the unknown means are i.i.d. variables. We prove an oracle inequality which states that under mild conditions, the regret of the GMLEB is of smaller order than $(\log n)^{5}/n$. The proof is based on a large deviation inequality for the generalized maximum likelihood estimator. The oracle inequality leads to the property that the GMLEB is adaptive minimax in $L_{p}$ balls when the order of the norm of the ball is larger than $((\log n)^{5/2}/\sqrt{n})^{1/(p\wedge 2)}$. We demonstrate the superb risk performance of the GMLEB through simulation experiments.