Nonsubjective priors via predictive relative entropy regret
Trevor J. Sweeting, Gauri S. Datta, and Malay Ghosh
Source: Ann. Statist. Volume 34, Number 1 (2006), 441-468.
Abstract
We explore the construction of nonsubjective prior distributions in Bayesian statistics via a posterior predictive relative entropy regret criterion. We carry out a minimax analysis based on a derived asymptotic predictive loss function and show that this approach to prior construction has a number of attractive features. The approach here differs from previous work that uses either prior or posterior relative entropy regret in that we consider predictive performance in relation to alternative nondegenerate prior distributions. The theory is illustrated with an analysis of some specific examples.
Primary Subjects: 62F15
Secondary Subjects: 62B10, 62C20
Keywords: Nonsubjective Bayesian inference; predictive inference; relative entropy loss; higher-order asymptotics
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aos/1146576270
Digital Object Identifier: doi:10.1214/009053605000000804
Mathematical Reviews number (MathSciNet):
MR2275249
Zentralblatt MATH identifier:
05034318
References
Aitchison, J. (1975). Goodness of prediction fit. Biometrika 62 547--554.
Mathematical Reviews (MathSciNet):
MR0391353
Zentralblatt MATH:
0339.62018
Digital Object Identifier: doi:10.2307/2335509
JSTOR: links.jstor.org
Akaike, H. (1978). A new look at the Bayes procedure. Biometrika 65 53--59.
Mathematical Reviews (MathSciNet):
MR0501450
Zentralblatt MATH:
0373.62008
Digital Object Identifier: doi:10.2307/2335276
JSTOR: links.jstor.org
Barron, A. R. (1999). Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems. In Bayesian Statistics 6 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 27--52. Oxford Univ. Press, New York.
Mathematical Reviews (MathSciNet):
MR1723492
Zentralblatt MATH:
0974.62020
Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
Mathematical Reviews (MathSciNet):
MR0804611
Zentralblatt MATH:
0572.62008
Bernardo, J. M. (1979). Reference posterior distributions for Bayesian inference (with discussion). J. Roy. Statist. Soc. Ser. B 41 113--147.
Mathematical Reviews (MathSciNet):
MR0547240
JSTOR: links.jstor.org
Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester.
Mathematical Reviews (MathSciNet):
MR1274699
Zentralblatt MATH:
0796.62002
Clarke, B. S. and Barron, A. R. (1990). Information-theoretic asymptotics of Bayes methods. IEEE Trans. Inform. Theory 36 453--471.
Mathematical Reviews (MathSciNet):
MR1053841
Digital Object Identifier: doi:10.1109/18.54897
Clarke, B. S. and Barron, A. R. (1994). Jeffreys' prior is asymptotically least favorable under entropy risk. J. Statist. Plann. Inference 41 37--60.
Mathematical Reviews (MathSciNet):
MR1292146
Digital Object Identifier: doi:10.1016/0378-3758(94)90153-8
Clarke, B. and Yuan, A. (2004). Partial information reference priors: Derivation and interpretations. J. Statist. Plann. Inference 123 313--345.
Mathematical Reviews (MathSciNet):
MR2062985
Digital Object Identifier: doi:10.1016/S0378-3758(03)00157-5
Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR1122806
Zentralblatt MATH:
0762.94001
Datta, G. S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Lecture Notes in Statist. 178. Springer, New York.
Mathematical Reviews (MathSciNet):
MR2053794
Zentralblatt MATH:
1044.62031
Datta, G. S., Mukerjee, R., Ghosh, M. and Sweeting, T. J. (2000). Bayesian prediction with approximate frequentist validity. Ann. Statist. 28 1414--1426.
Mathematical Reviews (MathSciNet):
MR1805790
Digital Object Identifier: doi:10.1214/aos/1015957400
Project Euclid: euclid.aos/1015957400
Ghosh, J. K. and Mukerjee, R. (1991). Characterization of priors under which Bayesian and frequentist Bartlett corrections are equivalent in the multiparameter case. J. Multivariate Anal. 38 385--393.
Mathematical Reviews (MathSciNet):
MR1131727
Digital Object Identifier: doi:10.1016/0047-259X(91)90052-4
Hartigan, J. A. (1998). The maximum likelihood prior. Ann. Statist. 26 2083--2103.
Mathematical Reviews (MathSciNet):
MR1700222
Digital Object Identifier: doi:10.1214/aos/1024691462
Project Euclid: euclid.aos/1024691462
Kass, R. E., Tierney, L. and Kadane, J. (1990). The validity of posterior expansions based on Laplace's method. In Bayesian and Likelihood Methods in Statistics and Econometrics (S. Geisser, J. S. Hodges, S. J. Press and A. Zellner, eds.) 473--488. North-Holland, Amsterdam.
Komaki, F. (1996). On asymptotic properties of predictive distributions. Biometrika 83 299--313.
Mathematical Reviews (MathSciNet):
MR1439785
Zentralblatt MATH:
0864.62007
Digital Object Identifier: doi:10.1093/biomet/83.2.299
JSTOR: links.jstor.org
Kuboki, H. (1998). Reference priors for prediction. J. Statist. Plann. Inference 69 295--317.
Mathematical Reviews (MathSciNet):
MR1631332
Digital Object Identifier: doi:10.1016/S0378-3758(97)00163-8
Laud, P. W. and Ibrahim, J. G. (1995). Predictive model selection. J. Roy. Statist. Soc. Ser. B 57 247--262.
Mathematical Reviews (MathSciNet):
MR1325389
JSTOR: links.jstor.org
Liang, F. and Barron, A. R. (2004). Exact minimax strategies for predictive density estimation, data compression, and model selection. IEEE Trans. Inform. Theory 50 2708--2726.
Mathematical Reviews (MathSciNet):
MR2096988
Digital Object Identifier: doi:10.1109/TIT.2004.836922
McCullagh, P. (1987). Tensor Methods in Statistics. Chapman and Hall, London.
Mathematical Reviews (MathSciNet):
MR0907286
Zentralblatt MATH:
0732.62003
Sun, D. and Berger, J. O. (1998). Reference priors with partial information. Biometrika 85 55--71.
Mathematical Reviews (MathSciNet):
MR1627242
Zentralblatt MATH:
1067.62521
Digital Object Identifier: doi:10.1093/biomet/85.1.55
JSTOR: links.jstor.org
Sweeting, T. J. (1996). Approximate Bayesian computation based on signed roots of log-density ratios (with discussion). In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 427--444. Oxford Univ. Press, New York.
Mathematical Reviews (MathSciNet):
MR1425418
Sweeting, T. J. (2001). Coverage probability bias, objective Bayes and the likelihood principle. Biometrika 88 657--675.
Mathematical Reviews (MathSciNet):
MR1859400
Zentralblatt MATH:
0985.62025
Digital Object Identifier: doi:10.1093/biomet/88.3.657
JSTOR: links.jstor.org
Sweeting, T. J. (2005). On the implementation of local probability matching priors for interest parameters. Biometrika 92 47--57.
Mathematical Reviews (MathSciNet):
MR2158609
Zentralblatt MATH:
1068.62036
Digital Object Identifier: doi:10.1093/biomet/92.1.47
Tibshirani, R. (1989). Noninformative priors for one parameter of many. Biometrika 76 604--608.
Mathematical Reviews (MathSciNet):
MR1040654
Zentralblatt MATH:
0678.62010
Digital Object Identifier: doi:10.2307/2336126
JSTOR: links.jstor.org
Xie, Q. and Barron, A. R. (1997). Minimax redundancy for the class of memoryless sources. IEEE Trans. Inform. Theory 43 646--657.
The Annals of Statistics
