The formal definition of reference priors
James O. Berger, José M. Bernardo, and Dongchu Sun
Source: Ann. Statist. Volume 37, Number 2
(2009), 905-938.
Abstract
Reference analysis produces objective Bayesian inference, in the sense that inferential statements depend only on the assumed model and the available data, and the prior distribution used to make an inference is least informative in a certain information-theoretic sense. Reference priors have been rigorously defined in specific contexts and heuristically defined in general, but a rigorous general definition has been lacking. We produce a rigorous general definition here and then show how an explicit expression for the reference prior can be obtained under very weak regularity conditions. The explicit expression can be used to derive new reference priors both analytically and numerically.
First Page:
Show
Hide
Keywords: Amount of information; Bayesian asymptotics; consensus priors; Fisher information; Jeffreys priors; noninformative priors; objective priors; reference priors
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aos/1236693154
Digital Object Identifier: doi:10.1214/07-AOS587
Zentralblatt MATH identifier: 1162.62013
Mathematical Reviews number (MathSciNet): MR2502655
References
[1] Ayyangar, A. S. K. (1941). The triangular distribution. Math. Students 9 85–87.
Mathematical Reviews (MathSciNet): MR5557
Zentralblatt MATH: 0063.03368
[2] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, Berlin.
Mathematical Reviews (MathSciNet): MR804611
Zentralblatt MATH: 0572.62008
[3] Berger, J. O. (2006). The case for objective Bayesian analysis (with discussion). Bayesian Anal. 1 385–402 and 457–464.
Mathematical Reviews (MathSciNet): MR2221271
Digital Object Identifier: doi:10.1214/06-BA115
[4] Berger, J. O. and Bernardo, J. M. (1989). Estimating a product of means: Bayesian analysis with reference priors. J. Amer. Statist. Assoc. 84 200–207.
Mathematical Reviews (MathSciNet): MR999679
Zentralblatt MATH: 0682.62018
Digital Object Identifier: doi:10.2307/2289864
JSTOR: links.jstor.org
[5] Berger, J. O. and Bernardo, J. M. (1992a). Ordered group reference priors with applications to a multinomial problem. Biometrica 79 25–37.
[6] Berger, J. O. and Bernardo, J. M. (1992b). Reference priors in a variance components problem. In Bayesian Analysis in Statistics and Econometrics (P. K. Goel and N. S. Iyengar, eds.) 323–340. Springer, Berlin.
Mathematical Reviews (MathSciNet): MR1194392
Zentralblatt MATH: 0770.62054
[7] Berger, J. O. and Bernardo, J. M. (1992c). On the development of reference priors (with discussion). In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 35–60. Oxford Univ. Press.
Mathematical Reviews (MathSciNet): MR1380269
[8] Berger, J. O., de Oliveira, V. and Sansó, B. (2001). Objective Bayesian analysis of spatially correlated data. J. Amer. Statist. Assoc. 96 1361–1374.
[9] Berger, J. O. and Yang, R. (1994). Noninformative priors and Bayesian testing for the AR(1) model. Econometric Theory 10 461–482.
Mathematical Reviews (MathSciNet): MR1309107
Digital Object Identifier: doi:10.1017/S026646660000863X
JSTOR: links.jstor.org
[10] Bernardo, J. M. (1979). Reference posterior distributions for Bayesian inference (with discussion). J. Roy. Statist. Soc. Ser. B 41 113–147. [Reprinted in Bayesian Inference (N. G. Polson and G. C. Tiao, eds.) 229–263. Edward Elgar, Brookfield, VT, 1995.]
Mathematical Reviews (MathSciNet): MR547240
JSTOR: links.jstor.org
[11] Bernardo, J. M. (2005). Reference analysis. In Handbook of Statistics 25 (D. K. Dey and C. R. Rao, eds.) 17–90. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet): MR2490522
Zentralblatt MATH: 0682.62018
[12] Bernardo, J. M. and Rueda, R. (2002). Bayesian hypothesis testing: A reference approach. Internat. Statist. Rev. 70 351–372.
[13] Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester.
Mathematical Reviews (MathSciNet): MR1274699
[14] Boros, G. and Moll, V. (2004). The Psi function. In Irresistible Integrals: Symbolics, Analysis and Experiments in the Evaluation of Integral 212–215. Cambridge Univ. Press.
Mathematical Reviews (MathSciNet): MR2070237
Zentralblatt MATH: 1090.11075
[15] Chernoff, H. (1956). Large-sample theory: Parametric case. Ann. Math. Statist. 27 1–22.
Mathematical Reviews (MathSciNet): MR76245
Zentralblatt MATH: 0072.35703
Digital Object Identifier: doi:10.1214/aoms/1177728347
Project Euclid: euclid.aoms/1177728347
[16] Clarke, B. (1999). Asymptotic normality of the posterior in relative entropy. IEEE Trans. Inform. Theory 45 165–176.
Mathematical Reviews (MathSciNet): MR1677856
Digital Object Identifier: doi:10.1109/18.746784
[17] Clarke, B. and Barron, A. R. (1994). Jeffreys’ prior is asymptotically least favorable under entropy risk. J. Statist. Plann. Inference 41 37–60.
Mathematical Reviews (MathSciNet): MR1292146
Zentralblatt MATH: 0820.62006
Digital Object Identifier: doi:10.1016/0378-3758(94)90153-8
[18] Csiszar, I. (1967). Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2 299–318.
Mathematical Reviews (MathSciNet): MR219345
[19] Csiszar, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146–158.
Mathematical Reviews (MathSciNet): MR365798
Digital Object Identifier: doi:10.1214/aop/1176996454
[20] Datta, G. S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Springer, New York.
Mathematical Reviews (MathSciNet): MR2053794
Zentralblatt MATH: 1044.62031
[21] Fraser, D. A. S., Monette, G. and Ng, K. W. (1985). Marginalization, likelihood and structural models. In Multivariate Analysis 6 (P. R. Krishnaiah, ed.) 209–217. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet): MR822296
Zentralblatt MATH: 0594.62001
[22] Gibbs, J. W. (1902). Elementary Principles in Statistical Mechanics. Constable, London. Reprinted by Dover, New York, 1960.
Mathematical Reviews (MathSciNet): MR116523
[23] Ghosh, J. K., Delampady, M. and Samanta, T. (2006). An Introduction to Bayesian Analysis: Theory and Methods. Springer, New York.
Mathematical Reviews (MathSciNet): MR2247439
Zentralblatt MATH: 1135.62002
[24] Good, I. J. (1950). Probability and the Weighing of Evidence. Hafner Press, New York.
Mathematical Reviews (MathSciNet): MR41366
Zentralblatt MATH: 0036.08402
[25] Good, I. J. (1969). What is the use of a distribution? In Multivariate Analysis 2 (P. R. Krishnaiah, ed.) 183–203. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet): MR260075
[26] Ghosal, S. (1997). Reference priors in multiparameter nonregular cases. Test 6 159–186.
Mathematical Reviews (MathSciNet): MR1466439
Zentralblatt MATH: 0905.62023
Digital Object Identifier: doi:10.1007/BF02564432
[27] Ghosal, S. and Samanta, T. (1997). Expansion of Bayes risk for entropy loss and reference prior in nonregular cases. Statist. Decisions 15 129–140.
Mathematical Reviews (MathSciNet): MR1475184
[28] Heath, D. L. and Sudderth, W. D. (1989). Coherent inference from improper priors and from finitely additive priors. Ann. Statist. 17 907–919.
Mathematical Reviews (MathSciNet): MR994275
Zentralblatt MATH: 0687.62003
Digital Object Identifier: doi:10.1214/aos/1176347150
Project Euclid: euclid.aos/1176347150
[29] Jaynes, E. T. (1957). Information theory and statistical mechanics. Phys. Rev. 106 620–630.
Mathematical Reviews (MathSciNet): MR87305
Digital Object Identifier: doi:10.1103/PhysRev.106.620
[30] Jaynes, E. T. (1968). Prior probabilities. IEEE Trans. Systems, Science and Cybernetics 4 227–291.
[31] Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. London Ser. A 186 453–461.
Mathematical Reviews (MathSciNet): MR17504
Digital Object Identifier: doi:10.1098/rspa.1946.0056
[32] Jeffreys, H. (1961). Theory of Probability, 3rd ed. Oxford Univ. Press.
Mathematical Reviews (MathSciNet): MR187257
[33] Johnson, N. J. and Kotz, S. (1999). Nonsmooth sailing or triangular distributions revisited after some 50 years. The Statistician 48 179–187.
[34] Kullback, S. (1959). Information Theory and Statistics, 2nd ed. Dover, New York.
Mathematical Reviews (MathSciNet): MR1461541
[35] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. Ann. Math. Statist. 22 79–86.
Mathematical Reviews (MathSciNet): MR39968
Zentralblatt MATH: 0042.38403
Digital Object Identifier: doi:10.1214/aoms/1177729694
Project Euclid: euclid.aoms/1177729694
[36] Lindley, D. V. (1956). On a measure of information provided by an experiment. Ann. Math. Statist. 27 986–1005.
Mathematical Reviews (MathSciNet): MR83936
Zentralblatt MATH: 0073.14103
Digital Object Identifier: doi:10.1214/aoms/1177728069
Project Euclid: euclid.aoms/1177728069
[37] Schmidt, R. (1934). Statistical analysis if one-dimentional distributions. Ann. Math. Statist. 5 30–43.
[38] Shannon, C. E. (1948). A mathematical theory of communication. Bell System Tech. J. 27 379–423, 623–656.
Mathematical Reviews (MathSciNet): MR26286
[39] Simpson, T. (1755). A letter to the right honourable George Earls of Maclesfield. President of the Royal Society, on the advantage of taking the mean of a number of observations in practical astronomy. Philos. Trans. 49 82–93.
[40] Stone, M. (1965). Right Haar measures for convergence in probability to invariant posterior distributions. Ann. Math. Statist. 36 440–453.
Mathematical Reviews (MathSciNet): MR175213
Zentralblatt MATH: 0156.38803
Digital Object Identifier: doi:10.1214/aoms/1177700154
Project Euclid: euclid.aoms/1177700154
[41] Stone, M. (1970). Necessary and sufficient condition for convergence in probability to invariant posterior distributions. Ann. Math. Statist. 41 1349–1353.
Mathematical Reviews (MathSciNet): MR266359
Zentralblatt MATH: 0225.62041
Digital Object Identifier: doi:10.1214/aoms/1177696911
Project Euclid: euclid.aoms/1177696911
[42] Sun, D. and Berger, J. O. (1998). Reference priors under partial information. Biometrika 85 55–71.
Mathematical Reviews (MathSciNet): MR1627242
Zentralblatt MATH: 1067.62521
Digital Object Identifier: doi:10.1093/biomet/85.1.55
JSTOR: links.jstor.org
[43] Wasserman, L. (2000). Asymptotic inference for mixture models using data-dependent priors. J. Roy. Statist. Soc. Ser. B 62 159–180.
Mathematical Reviews (MathSciNet): MR1747402
Zentralblatt MATH: 0976.62028
Digital Object Identifier: doi:10.1111/1467-9868.00226
JSTOR: links.jstor.org
