Annals of Statistics

Improved minimax predictive densities under Kullback–Leibler loss

Edward I. George, Feng Liang, and Xinyi Xu

Full-text: Open access


Let X|μNp(μ,vxI) and Y|μNp(μ,vyI) be independent p-dimensional multivariate normal vectors with common unknown mean μ. Based on only observing X=x, we consider the problem of obtaining a predictive density (y|x) for Y that is close to p(y|μ) as measured by expected Kullback–Leibler loss. A natural procedure for this problem is the (formal) Bayes predictive density U(y|x) under the uniform prior πU(μ)1, which is best invariant and minimax. We show that any Bayes predictive density will be minimax if it is obtained by a prior yielding a marginal that is superharmonic or whose square root is superharmonic. This yields wide classes of minimax procedures that dominate U(y|x), including Bayes predictive densities under superharmonic priors. Fundamental similarities and differences with the parallel theory of estimating a multivariate normal mean under quadratic loss are described.

Article information

Ann. Statist., Volume 34, Number 1 (2006), 78-91.

First available in Project Euclid: 2 May 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C20: Minimax procedures
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62F15: Bayesian inference

Bayes rules heat equation inadmissibility multiple shrinkage multivariate normal prior distributions shrinkage estimation superharmonic marginals superharmonic priors unbiased estimate of risk


George, Edward I.; Liang, Feng; Xu, Xinyi. Improved minimax predictive densities under Kullback–Leibler loss. Ann. Statist. 34 (2006), no. 1, 78--91. doi:10.1214/009053606000000155.

Export citation


  • Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547–554.
  • Aslan, M. (2002). Asymptotically minimax Bayes predictive densities. Ph.D. dissertation, Dept. Statistics, Yale Univ.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855–903.
  • Brown, L. D., DasGupta, A., Haff, L. R. and Strawderman, W. E. (2006). The heat equation and Stern's identity: Connections, applications. J. Statist. Plann. Inference 136 2254–2278.
  • Fourdrinier, D., Strawderman, W. E. and Wells, M. T. (1998). On the construction of Bayes minimax estimators. Ann. Statist. 26 660– 671.
  • George, E. I. (1986). Minimax multiple shrinkage estimation. Ann. Statist. 14 188–205.
  • George, E. I. (1986). Combining minimax shrinkage estimators. J. Amer. Statist. Assoc. 81 437–445.
  • George, E. I. (1986). A formal Bayes multiple shrinkage estimator. Comm. Statist. A–-Theory Methods 15 2099–2114.
  • Harris, I. R. (1989). Predictive fit for natural exponential families. Biometrika 76 675–684.
  • Hartigan, J. A. (1998). The maximum likelihood prior. Ann. Statist. 26 2083–2103.
  • Komaki, F. (1996). On asymptotic properties of predictive distributions. Biometrika 83 299–313.
  • Komaki, F. (2001). A shrinkage predictive distribution for multivariate normal observables. Biometrika 88 859–864.
  • Komaki, F. (2004). Simultaneous prediction of independent Poisson observables. Ann. Statist. 32 1744–1769.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • Liang, F. (2002). Exact minimax procedures for predictive density estimation and data compression. Ph.D. dissertation, Dept. Statistics, Yale Univ.
  • Liang, F. and Barron, A. (2004). Exact minimax strategies for predictive density estimation, data compression and model selection. IEEE Trans. Inform. Theory 50 2708–2726.
  • Murray, G. D. (1977). A note on the estimation of probability density functions. Biometrika 64 150–152.
  • Ng, V. M. (1980). On the estimation of parametric density functions. Biometrika 67 505–506.
  • Steele, J. M. (2001). Stochastic Calculus and Financial Applications. Springer, New York.
  • Stein, C. (1974). Estimation of the mean of a multivariate normal distribution. In Proc. Prague Symposium on Asymptotic Statistics (J. Hájek, ed.) 2 345–381. Univ. Karlova, Prague.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.
  • Sweeting, T. J., Datta, G. S. and Ghosh, M. (2006). Nonsubjective priors via predictive relative entropy regret. Ann. Statist. 34 441–468.