The Annals of Statistics

Nonsubjective priors via predictive relative entropy regret

Trevor J. Sweeting, Gauri S. Datta, and Malay Ghosh

Full-text: Open access


We explore the construction of nonsubjective prior distributions in Bayesian statistics via a posterior predictive relative entropy regret criterion. We carry out a minimax analysis based on a derived asymptotic predictive loss function and show that this approach to prior construction has a number of attractive features. The approach here differs from previous work that uses either prior or posterior relative entropy regret in that we consider predictive performance in relation to alternative nondegenerate prior distributions. The theory is illustrated with an analysis of some specific examples.

Article information

Ann. Statist. Volume 34, Number 1 (2006), 441-468.

First available in Project Euclid: 2 May 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F15: Bayesian inference
Secondary: 62B10: Information-theoretic topics [See also 94A17] 62C20: Minimax procedures

Nonsubjective Bayesian inference predictive inference relative entropy loss higher-order asymptotics


Sweeting, Trevor J.; Datta, Gauri S.; Ghosh, Malay. Nonsubjective priors via predictive relative entropy regret. Ann. Statist. 34 (2006), no. 1, 441--468. doi:10.1214/009053605000000804.

Export citation


  • Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547--554.
  • Akaike, H. (1978). A new look at the Bayes procedure. Biometrika 65 53--59.
  • Barron, A. R. (1999). Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems. In Bayesian Statistics 6 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 27--52. Oxford Univ. Press, New York.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • Bernardo, J. M. (1979). Reference posterior distributions for Bayesian inference (with discussion). J. Roy. Statist. Soc. Ser. B 41 113--147.
  • Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester.
  • Clarke, B. S. and Barron, A. R. (1990). Information-theoretic asymptotics of Bayes methods. IEEE Trans. Inform. Theory 36 453--471.
  • Clarke, B. S. and Barron, A. R. (1994). Jeffreys' prior is asymptotically least favorable under entropy risk. J. Statist. Plann. Inference 41 37--60.
  • Clarke, B. and Yuan, A. (2004). Partial information reference priors: Derivation and interpretations. J. Statist. Plann. Inference 123 313--345.
  • Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley, New York.
  • Datta, G. S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Lecture Notes in Statist. 178. Springer, New York.
  • Datta, G. S., Mukerjee, R., Ghosh, M. and Sweeting, T. J. (2000). Bayesian prediction with approximate frequentist validity. Ann. Statist. 28 1414--1426.
  • Ghosh, J. K. and Mukerjee, R. (1991). Characterization of priors under which Bayesian and frequentist Bartlett corrections are equivalent in the multiparameter case. J. Multivariate Anal. 38 385--393.
  • Hartigan, J. A. (1998). The maximum likelihood prior. Ann. Statist. 26 2083--2103.
  • Kass, R. E., Tierney, L. and Kadane, J. (1990). The validity of posterior expansions based on Laplace's method. In Bayesian and Likelihood Methods in Statistics and Econometrics (S. Geisser, J. S. Hodges, S. J. Press and A. Zellner, eds.) 473--488. North-Holland, Amsterdam.
  • Komaki, F. (1996). On asymptotic properties of predictive distributions. Biometrika 83 299--313.
  • Kuboki, H. (1998). Reference priors for prediction. J. Statist. Plann. Inference 69 295--317.
  • Laud, P. W. and Ibrahim, J. G. (1995). Predictive model selection. J. Roy. Statist. Soc. Ser. B 57 247--262.
  • Liang, F. and Barron, A. R. (2004). Exact minimax strategies for predictive density estimation, data compression, and model selection. IEEE Trans. Inform. Theory 50 2708--2726.
  • McCullagh, P. (1987). Tensor Methods in Statistics. Chapman and Hall, London.
  • Sun, D. and Berger, J. O. (1998). Reference priors with partial information. Biometrika 85 55--71.
  • Sweeting, T. J. (1996). Approximate Bayesian computation based on signed roots of log-density ratios (with discussion). In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 427--444. Oxford Univ. Press, New York.
  • Sweeting, T. J. (2001). Coverage probability bias, objective Bayes and the likelihood principle. Biometrika 88 657--675.
  • Sweeting, T. J. (2005). On the implementation of local probability matching priors for interest parameters. Biometrika 92 47--57.
  • Tibshirani, R. (1989). Noninformative priors for one parameter of many. Biometrika 76 604--608.
  • Xie, Q. and Barron, A. R. (1997). Minimax redundancy for the class of memoryless sources. IEEE Trans. Inform. Theory 43 646--657.