Bernoulli

  • Bernoulli
  • Volume 21, Number 1 (2015), 574-603.

Improved minimax estimation of a multivariate normal mean under heteroscedasticity

Zhiqiang Tan

Full-text: Open access

Abstract

Consider the problem of estimating a multivariate normal mean with a known variance matrix, which is not necessarily proportional to the identity matrix. The coordinates are shrunk directly in proportion to their variances in Efron and Morris’ (J. Amer. Statist. Assoc. 68 (1973) 117–130) empirical Bayes approach, whereas inversely in proportion to their variances in Berger’s (Ann. Statist. 4 (1976) 223–226) minimax estimators. We propose a new minimax estimator, by approximately minimizing the Bayes risk with a normal prior among a class of minimax estimators where the shrinkage direction is open to specification and the shrinkage magnitude is determined to achieve minimaxity. The proposed estimator has an interesting simple form such that one group of coordinates are shrunk in the direction of Berger’s estimator and the remaining coordinates are shrunk in the direction of the Bayes rule. Moreover, the proposed estimator is scale adaptive: it can achieve close to the minimum Bayes risk simultaneously over a scale class of normal priors (including the specified prior) and achieve close to the minimax linear risk over a corresponding scale class of hyper-rectangles. For various scenarios in our numerical study, the proposed estimators with extreme priors yield more substantial risk reduction than existing minimax estimators.

Article information

Source
Bernoulli, Volume 21, Number 1 (2015), 574-603.

Dates
First available in Project Euclid: 17 March 2015

Permanent link to this document
https://projecteuclid.org/euclid.bj/1426597082

Digital Object Identifier
doi:10.3150/13-BEJ580

Mathematical Reviews number (MathSciNet)
MR3322331

Zentralblatt MATH identifier
1311.62086

Keywords
Bayes risk empirical Bayes minimax estimation multivariate normal mean shrinkage estimation unequal variances

Citation

Tan, Zhiqiang. Improved minimax estimation of a multivariate normal mean under heteroscedasticity. Bernoulli 21 (2015), no. 1, 574--603. doi:10.3150/13-BEJ580. https://projecteuclid.org/euclid.bj/1426597082


Export citation

References

  • [1] Baranchik, A.J. (1964). Multiple regression and estimation of the mean of a multivariate normal distribution. Technical Report 51, Dept. Statistics, Stanford Univ.
  • [2] Baranchik, A.J. (1970). A family of minimax estimators of the mean of a multivariate normal distribution. Ann. Math. Statist. 41 642–645.
  • [3] Berger, J.O. (1976). Admissible minimax estimation of a multivariate normal mean with arbitrary quadratic loss. Ann. Statist. 4 223–226.
  • [4] Berger, J. (1980). A robust generalized Bayes estimator and confidence region for a multivariate normal mean. Ann. Statist. 8 716–761.
  • [5] Berger, J.O. (1982). Selecting a minimax estimator of a multivariate normal mean. Ann. Statist. 10 81–92.
  • [6] Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer Series in Statistics. New York: Springer.
  • [7] Berger, J.O. and Robert, C. (1990). Subjective hierarchical Bayes estimation of a multivariate normal mean: On the frequentist interface. Ann. Statist. 18 617–651.
  • [8] Berger, J.O. and Srinivasan, C. (1978). Generalized Bayes estimators in multivariate problems. Ann. Statist. 6 783–801.
  • [9] Bhattacharya, P.K. (1966). Estimating the mean of a multivariate normal population with general quadratic loss function. Ann. Math. Statist. 37 1819–1824.
  • [10] Bock, M.E. (1975). Minimax estimators of the mean of a multivariate normal distribution. Ann. Statist. 3 209–218.
  • [11] Brandwein, A.C. and Strawderman, W.E. (1990). Stein estimation: The spherically symmetric case. Statist. Sci. 5 356–369.
  • [12] Brown, L.D. (1975). Estimation with incompletely specified loss functions (the case of several location parameters). J. Amer. Statist. Assoc. 70 417–427.
  • [13] Brown, L.D. and Zhao, L.H. (2009). Estimators for Gaussian models having a block-wise structure. Statist. Sinica 19 885–903.
  • [14] Cai, T.T. (2012). Minimax and adaptive inference in nonparametric function estimation. Statist. Sci. 27 31–50.
  • [15] Casella, G. (1985). Condition numbers and minimax ridge regression estimators. J. Amer. Statist. Assoc. 80 753–758.
  • [16] Donoho, D.L., Liu, R.C. and MacGibbon, B. (1990). Minimax risk over hyperrectangles, and implications. Ann. Statist. 18 1416–1437.
  • [17] Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors—an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117–130.
  • [18] Harville, D.A. (2008). Matrix Algebra from a Statistician’s Perspective. New York: Springer.
  • [19] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Berkeley, CA: Univ. California Press.
  • [20] Jones, M.C. (1986). Expressions for inverse moments of positive quadratic forms in normal variables. Austral. J. Statist. 28 242–250.
  • [21] Lehmann, E.L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer Texts in Statistics. New York: Springer.
  • [22] Morris, C.N. (1983). Parametric empirical Bayes inference: Theory and applications. J. Amer. Statist. Assoc. 78 47–65. With discussion.
  • [23] Morris, C.N. and Lysy, M. (2012). Shrinkage estimation in multilevel normal models. Statist. Sci. 27 115–134.
  • [24] Satterthwaite, F.E. (1946). An approximate distribution of estimates of variance components. Biometrics 2 110–114.
  • [25] Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 197–206. Berkeley and Los Angeles: Univ. California Press.
  • [26] Stein, C.M. (1962). Confidence sets for the mean of a multivariate normal distribution. J. Roy. Statist. Soc. Ser. B 24 265–296.
  • [27] Stein, C.M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • [28] Strawderman, W.E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.
  • [29] Strawderman, W.E. (2010). Bayesian decision based estimation and predictive inference. In Frontiers of Statistical Decision Making and Bayesian Analysis: In Honor of James O. Berger (M.-H. Chen et al., eds.) 69–82. New York: Springer.
  • [30] Tan, Z. (2014). Supplement to “Improved minimax estimation of a multivariate normal mean under heteroscedasticity”. DOI:10.3150/13-BEJ580SUPP.
  • [31] Xie, X., Kou, S.C. and Brown, L.D. (2012). SURE estimates for a heteroscedastic hierarchical model. J. Amer. Statist. Assoc. 107 1465–1479.

Supplemental materials

  • Supplementary material: Supplementary Material for “Improved minimax estimation of a multivariate normal mean under heteroscedasticity”. We present additional results from the simulation study in Section 4.