The unbearable transparency of Stein estimation

Abstract

Charles Stein [10] discovered that, under quadratic loss, the usual unbiased estimator for the mean vector of a multivariate normal distribution is inadmissible if the dimension n of the mean vector exceeds two. On the way, he constructed shrinkage estimators that dominate the usual estimator asymptotically in n. It has since been claimed that Stein’s results and the subsequent James–Stein estimator are counter-intuitive, even paradoxical, and not very useful. In response to such doubts, various authors have presented alternative derivations of Stein shrinkage estimators. Surely Stein himself did not find his results paradoxical. This paper argues that assertions of “paradoxical" or “counter-intuitive" or “not practical" have overlooked essential arguments and remarks in Stein’s beautifully written paper [10]. Among these overlooked aspects are the asymptotic geometry of quadratic loss in high dimensions that makes Stein estimation transparent; the asymptotic optimality results that can be associate with Stein estimation; the explicit mention of practical multiple shrinkage estimators; and the foreshadowing of Stein confidence balls. These ideas are fundamental for studies of modern regularization estimators that rely on multiple shrinkage, whether implicitly or overtly.