Statistical Science

From Minimax Shrinkage Estimation to Minimax Shrinkage Prediction

Edward I. George, Feng Liang, and Xinyi Xu

Full-text: Open access

Abstract

In a remarkable series of papers beginning in 1956, Charles Stein set the stage for the future development of minimax shrinkage estimators of a multivariate normal mean under quadratic loss. More recently, parallel developments have seen the emergence of minimax shrinkage estimators of multivariate normal predictive densities under Kullback–Leibler risk. We here describe these parallels emphasizing the focus on Bayes procedures and the derivation of the superharmonic conditions for minimaxity as well as further developments of new minimax shrinkage predictive density estimators including multiple shrinkage estimators, empirical Bayes estimators, normal linear model regression estimators and nonparametric regression estimators.

Article information

Source
Statist. Sci., Volume 27, Number 1 (2012), 82-94.

Dates
First available in Project Euclid: 14 March 2012

Permanent link to this document
https://projecteuclid.org/euclid.ss/1331729984

Digital Object Identifier
doi:10.1214/11-STS383

Mathematical Reviews number (MathSciNet)
MR2953497

Zentralblatt MATH identifier
1330.62288

Keywords
Asymptotic minimaxity Bayesian prediction empirical Bayes inadmissibility multiple shrinkage prior distributions superharmonic marginals unbiased estimates of risk

Citation

George, Edward I.; Liang, Feng; Xu, Xinyi. From Minimax Shrinkage Estimation to Minimax Shrinkage Prediction. Statist. Sci. 27 (2012), no. 1, 82--94. doi:10.1214/11-STS383. https://projecteuclid.org/euclid.ss/1331729984


Export citation

References

  • Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547–554.
  • Akaike, H. (1978). A new look at the Bayes procedure. Biometrika 65 53–59.
  • Belitser, E. N. and Levit, B. Y. (1995). On minimax filtering over ellipsoids. Math. Methods Statist. 4 259–273.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855–903.
  • Belitser, E. N. and Levit, B. Y. (1996). Asymptotically minimax nonparametric regression in L2. Statistics 28 105–122.
  • Brown, L. D., George, E. I. and Xu, X. (2008). Admissible predictive density estimation. Ann. Statist. 36 1156–1170.
  • Brown, L. D. and Hwang, J. T. (1982). A unified admissibility proof. In Statistical Decision Theory and Related Topics, III, Vol. 1 (West Lafayette, Ind., 1981) (S. S. Gupta and J. O. Berger, eds.) 205–230. Academic Press, New York.
  • Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory, and Applications. Springer, New York.
  • Faith, R. E. (1978). Minimax Bayes estimators of a multivariate normal mean. J. Multivariate Anal. 8 372–379.
  • Fourdrinier, D., Strawderman, W. E. and Wells, M. T. (1998). On the construction of Bayes minimax estimators. Ann. Statist. 26 660–671.
  • George, E. I. (1986a). Minimax multiple shrinkage estimation. Ann. Statist. 14 188–205.
  • George, E. I. (1986b). Combining minimax shrinkage estimators. J. Amer. Statist. Assoc. 81 437–445.
  • George, E. I. (1986c). A formal Bayes multiple shrinkage estimator. Comm. Statist. A—Theory Methods 15 2099–2114.
  • George, E. I., Liang, F. and Xu, X. (2006). Improved minimax predictive densities under Kullback–Leibler loss. Ann. Statist. 34 78–91.
  • George, E. I. and Xu, X. (2008). Predictive density estimation for multiple regression. Econometric Theory 24 528–544.
  • Ghosh, M., Mergel, V. and Datta, G. S. (2008). Estimation, prediction and the Stein phenomenon under divergence loss. J. Multivariate Anal. 99 1941–1961.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, CA.
  • Johnstone, I. M. (2003). Function estimation and Gaussian sequence models. Draft of a Monograph, Dept. Statistics, Stanford Univ.
  • Kato, K. (2009). Improved prediction for a multivariate normal distribution with unknown mean and variance. Ann. Inst. Statist. Math. 61 531–542.
  • Kobayashi, K. and Komaki, F. (2008). Bayesian shrinkage prediction for the regression problem. J. Multivariate Anal. 99 1888–1905.
  • Komaki, F. (2001). A shrinkage predictive distribution for multivariate normal observables. Biometrika 88 859–864.
  • Komaki, F. (2004). Simultaneous prediction of independent Poisson observables. Ann. Statist. 32 1744–1769.
  • Komaki, F. (2006). Shrinkage priors for Bayesian prediction. Ann. Statist. 34 808–819.
  • Komaki, F. (2009). Bayesian predictive densities based on superharmonic priors for the 2-dimensional Wishart model. J. Multivariate Anal. 100 2137–2154.
  • Liang, F. (2002). Exact minimax procedures for predictive density estimation and data compression. Ph.D. thesis, Dept. Statistics, Yale Univ.
  • Liang, F. and Barron, A. (2004). Exact minimax strategies for predictive density estimation, data compression, and model selection. IEEE Trans. Inform. Theory 50 2708–2726.
  • Lindley, D. V. (1962). Discussion of “Confidence sets for the mean of a multivariate normal distribution” by C. Stein. J. Roy. Statist. Soc. Ser. B 24 285–287.
  • Murray, G. D. (1977). A note on the estimation of probability density functions. Biometrika 64 150–152.
  • Ng, V. M. (1980). On the estimation of parametric density functions. Biometrika 67 505–506.
  • Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 2 120–133.
  • Steele, J. M. (2001). Stochastic Calculus and Financial Applications. Applications of Mathematics (New York) 45. Springer, New York.
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 197–206. Univ. California Press, Berkeley.
  • Stein, C. (1960). Multiple regression. In Contributions to Probability and Statistics (I. Olkin, ed.) 424–443. Stanford Univ. Press, Stanford, Calif.
  • Stein, C. M. (1962). Confidence sets for the mean of a multivariate normal distribution (with discussion). J. Roy. Statist. Soc. Ser. B 24 265–296.
  • Stein, C. (1974). Estimation of the mean of a multivariate normal distribution. In Proceedings of the Prague Symposium on Asymptotic Statistics (Charles Univ., Prague, 1973), Vol. II 345–381. Charles Univ., Prague.
  • Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.
  • Wasserman, L. (2006). All of Nonparametric Statistics. Springer, New York.
  • Xu, X. and Liang, F. (2010). Asymptotic minimax risk of predictive density estimation for non-parametric regression. Bernoulli 16 543–560.
  • Xu, X. and Zhou, D. (2011). Empirical Bayes predictive densities for high-dimensional normal models. J. Multivariate Anal. 102 1417–1428.