Statistical Science

A Geometrical Explanation of Stein Shrinkage

Lawrence D. Brown and Linda H. Zhao

Full-text: Open access

Abstract

Shrinkage estimation has become a basic tool in the analysis of high-dimensional data. Historically and conceptually a key development toward this was the discovery of the inadmissibility of the usual estimator of a multivariate normal mean.

This article develops a geometrical explanation for this inadmissibility. By exploiting the spherical symmetry of the problem it is possible to effectively conceptualize the multidimensional setting in a two-dimensional framework that can be easily plotted and geometrically analyzed. We begin with the heuristic explanation for inadmissibility that was given by Stein [In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, Vol. I (1956) 197–206, Univ. California Press]. Some geometric figures are included to make this reasoning more tangible. It is also explained why Stein’s argument falls short of yielding a proof of inadmissibility, even when the dimension, p, is much larger than p = 3.

We then extend the geometric idea to yield increasingly persuasive arguments for inadmissibility when p ≥ 3, albeit at the cost of increased geometric and computational detail.

Article information

Source
Statist. Sci., Volume 27, Number 1 (2012), 24-30.

Dates
First available in Project Euclid: 14 March 2012

Permanent link to this document
https://projecteuclid.org/euclid.ss/1331729980

Digital Object Identifier
doi:10.1214/11-STS382

Mathematical Reviews number (MathSciNet)
MR2953493

Zentralblatt MATH identifier
1330.62282

Keywords
Stein estimation shrinkage minimax empirical Bayes high-dimensional geometry

Citation

Brown, Lawrence D.; Zhao, Linda H. A Geometrical Explanation of Stein Shrinkage. Statist. Sci. 27 (2012), no. 1, 24--30. doi:10.1214/11-STS382. https://projecteuclid.org/euclid.ss/1331729980


Export citation

References

  • Blyth, C. R. (1951). On minimax statistical decision procedures and their admissibility. Ann. Math. Statist. 22 22–42.
  • Brandwein, A. C. and Strawderman, W. E. (1990). Stein estimation: The spherically symmetric case. Statist. Sci. 5 356–369.
  • Brown, L. D. (1966). On the admissibility of invariant estimators of one or more location parameters. Ann. Math. Statist. 37 1087–1136.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855–903. [Correction. Ann. Statist. 1 (1973) 594–596. ]
  • Brown, L. D. and Fox, M. (1974). Admissibility in statistical problems involving a location or scale parameter. Ann. Statist. 2 807–814.
  • Efron, B. and Morris, C. (1971). Limiting the risk of Bayes and empirical Bayes estimators. I. The Bayes case. J. Amer. Statist. Assoc. 66 807–815.
  • Girshick, M. A. and Savage, L. J. (1951). Bayes and minimax estimates for quadratic loss functions. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950 53–73. Univ. California Press, Berkeley.
  • Hodges, J. L., Jr. and Lehmann, E. L. (1951). Some applications of the Cramér-Rao inequality. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950 13–22. Univ. California Press, Berkeley.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, CA.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • Lehmann, E. L. and Stein, C. M. (1953). The admissibility of certain invariant statistical tests involving a translation parameter. Ann. Math. Statist. 24 473–479.
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 197–206. Univ. California Press, Berkeley.
  • Stein, C. (1973). Estimation of the mean of a multivariate normal distribution. In Proceedings of the Prague Symposium on Asymptotic Statistics (Charles Univ., Prague, 1973), Vol. II 345–381. Charles Univ., Prague.
  • Stein, C. M. (1962). Confidence sets for the mean of a multivariate normal distribution. J. R. Statist. Soc. Ser. B Stat. Methodol. 24 265–296.
  • Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.