The Annals of Statistics

Improved minimax predictive densities under Kullback–Leibler loss

Edward I. George, Feng Liang, and Xinyi Xu

Full-text: Open access

Abstract

Let X|μNp(μ,vxI) and Y|μNp(μ,vyI) be independent p-dimensional multivariate normal vectors with common unknown mean μ. Based on only observing X=x, we consider the problem of obtaining a predictive density (y|x) for Y that is close to p(y|μ) as measured by expected Kullback–Leibler loss. A natural procedure for this problem is the (formal) Bayes predictive density U(y|x) under the uniform prior πU(μ)1, which is best invariant and minimax. We show that any Bayes predictive density will be minimax if it is obtained by a prior yielding a marginal that is superharmonic or whose square root is superharmonic. This yields wide classes of minimax procedures that dominate U(y|x), including Bayes predictive densities under superharmonic priors. Fundamental similarities and differences with the parallel theory of estimating a multivariate normal mean under quadratic loss are described.

Article information

Source
Ann. Statist., Volume 34, Number 1 (2006), 78-91.

Dates
First available in Project Euclid: 2 May 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1146576256

Digital Object Identifier
doi:10.1214/009053606000000155

Mathematical Reviews number (MathSciNet)
MR2275235

Zentralblatt MATH identifier
1091.62003

Subjects
Primary: 62C20: Minimax procedures
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62F15: Bayesian inference

Keywords
Bayes rules heat equation inadmissibility multiple shrinkage multivariate normal prior distributions shrinkage estimation superharmonic marginals superharmonic priors unbiased estimate of risk

Citation

George, Edward I.; Liang, Feng; Xu, Xinyi. Improved minimax predictive densities under Kullback–Leibler loss. Ann. Statist. 34 (2006), no. 1, 78--91. doi:10.1214/009053606000000155. https://projecteuclid.org/euclid.aos/1146576256


Export citation

References

  • Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547--554.
  • Aslan, M. (2002). Asymptotically minimax Bayes predictive densities. Ph.D. dissertation, Dept. Statistics, Yale Univ.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855--903.
  • Brown, L. D., DasGupta, A., Haff, L. R. and Strawderman, W. E. (2006). The heat equation and Stern's identity: Connections, applications. J. Statist. Plann. Inference 136 2254--2278.
  • Fourdrinier, D., Strawderman, W. E. and Wells, M. T. (1998). On the construction of Bayes minimax estimators. Ann. Statist. 26 660-- 671.
  • George, E. I. (1986). Minimax multiple shrinkage estimation. Ann. Statist. 14 188--205.
  • George, E. I. (1986). Combining minimax shrinkage estimators. J. Amer. Statist. Assoc. 81 437--445.
  • George, E. I. (1986). A formal Bayes multiple shrinkage estimator. Comm. Statist. A---Theory Methods 15 2099--2114.
  • Harris, I. R. (1989). Predictive fit for natural exponential families. Biometrika 76 675--684.
  • Hartigan, J. A. (1998). The maximum likelihood prior. Ann. Statist. 26 2083--2103.
  • Komaki, F. (1996). On asymptotic properties of predictive distributions. Biometrika 83 299--313.
  • Komaki, F. (2001). A shrinkage predictive distribution for multivariate normal observables. Biometrika 88 859--864.
  • Komaki, F. (2004). Simultaneous prediction of independent Poisson observables. Ann. Statist. 32 1744--1769.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • Liang, F. (2002). Exact minimax procedures for predictive density estimation and data compression. Ph.D. dissertation, Dept. Statistics, Yale Univ.
  • Liang, F. and Barron, A. (2004). Exact minimax strategies for predictive density estimation, data compression and model selection. IEEE Trans. Inform. Theory 50 2708--2726.
  • Murray, G. D. (1977). A note on the estimation of probability density functions. Biometrika 64 150--152.
  • Ng, V. M. (1980). On the estimation of parametric density functions. Biometrika 67 505--506.
  • Steele, J. M. (2001). Stochastic Calculus and Financial Applications. Springer, New York.
  • Stein, C. (1974). Estimation of the mean of a multivariate normal distribution. In Proc. Prague Symposium on Asymptotic Statistics (J. Hájek, ed.) 2 345--381. Univ. Karlova, Prague.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135--1151.
  • Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385--388.
  • Sweeting, T. J., Datta, G. S. and Ghosh, M. (2006). Nonsubjective priors via predictive relative entropy regret. Ann. Statist. 34 441--468.