Is Bayes Posterior just Quick and Dirty Confidence?
D. A. S. Fraser
Source: Statist. Sci. Volume 26, Number 3
(2011), 299-316.
Abstract
Bayes [Philos. Trans. R. Soc. Lond. 53 (1763) 370–418; 54 296–325] introduced the observed likelihood function to statistical inference and provided a weight function to calibrate the parameter; he also introduced a confidence distribution on the parameter space but did not provide present justifications. Of course the names likelihood and confidence did not appear until much later: Fisher [Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 222 (1922) 309–368] for likelihood and Neyman [Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 237 (1937) 333–380] for confidence. Lindley [J. Roy. Statist. Soc. Ser. B 20 (1958) 102–107] showed that the Bayes and the confidence results were different when the model was not location. This paper examines the occurrence of true statements from the Bayes approach and from the confidence approach, and shows that the proportion of true statements in the Bayes case depends critically on the presence of linearity in the model; and with departure from this linearity the Bayes approach can be a poor approximation and be seriously misleading. Bayesian integration of weighted likelihood thus provides a first-order linear approximation to confidence, but without linearity can give substantially incorrect results.
First Page:
Show
Hide
Related Works:
Keywords: Bayes; Bayes error rate; confidence; default prior; evaluating a prior; nonlinear parameter; posterior; prior
Full-text: Access denied (no subscription
detected)
We're sorry, but we are unable to provide
you with the full text of this article because we are not able to identify
you as a subscriber.
If you have a personal subscription to
this journal, then please login. If you are already logged in, then you
may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ss/1320066918
Digital Object Identifier: doi:10.1214/11-STS352
Mathematical Reviews number (MathSciNet): MR2918001
Zentralblatt MATH identifier: 06075172
References
Abebe, F., Cakmak, S., Cheah, P. K., Fraser, D. A. S., Kuhn, J., McDunnough, P., Reid, N. and Tapia, A. (1995). Third order asymptotic model: Exponential and location type approximations. Parisankhyan Samikkha 2 25–33.
Mathematical Reviews (MathSciNet): MR1613190
Andrews, D. F., Fraser, D. A. S. and Wong, A. C. M. (2005). Computation of distribution functions from likelihood information near observed data. J. Statist. Plann. Inference 134 180–193.
Mathematical Reviews (MathSciNet): MR2146092
Zentralblatt MATH: 1066.62021
Digital Object Identifier: doi:10.1016/j.jspi.2003.12.021
Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53 370–418; 54 296–325. Reprinted in Biometrika 45 (1958) 293–315.
Bédard, M., Fraser, D. A. S. and Wong, A. (2007). Higher accuracy for Bayesian and frequentist inference: Large sample theory for small sample likelihood. Statist. Sci. 22 301–321.
Mathematical Reviews (MathSciNet): MR2416807
Digital Object Identifier: doi:10.1214/07-STS240
Project Euclid: euclid.ss/1199285030
Bernardo, J.-M. (1979). Reference posterior distributions for Bayesian inference (with discussion). J. Roy. Statist. Soc. Ser. B 41 113–147.
Mathematical Reviews (MathSciNet): MR547240
Bernardo, J.-M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester.
Mathematical Reviews (MathSciNet): MR1274699
Box, G. E. P. and Cox, D. R. (1964). An analysis of transformations (with discussion). J. Roy. Statist. Soc. Ser. B 26 211–252.
Mathematical Reviews (MathSciNet): MR192611
Cakmak, S., Fraser, D. A. S., McDunnough, P., Reid, N. and Yuan, X. (1998). Likelihood centered asymptotic model: Exponential and location model versions. J. Statist. Plann. Inference 66 211–222.
Mathematical Reviews (MathSciNet): MR1614476
Zentralblatt MATH: 0953.62017
Digital Object Identifier: doi:10.1016/S0378-3758(97)00085-2
Cox, D. R. (1958). Some problems connected with statistical inference. Ann. Math. Statist. 29 357–372.
Mathematical Reviews (MathSciNet): MR94890
Zentralblatt MATH: 0088.11702
Digital Object Identifier: doi:10.1214/aoms/1177706618
Project Euclid: euclid.aoms/1177706618
Dawid, A. P., Stone, M. and Zidek, J. V. (1973). Marginalization paradoxes in Bayesian and structural inference (with discussion). J. Roy. Statist. Soc. Ser. B 35 189–233.
Mathematical Reviews (MathSciNet): MR365805
Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 222 309–368.
Fisher, R. A. (1930). Inverse probability. Proc. Camb. Phil. Soc. 26 528–535.
Fisher, R. A. (1935). The fiducial argument in statistical inference. Ann. Eugenics B 391–398.
Fraser, A. M. Fraser, D. A. S. and Fraser, M. J. (2010a). Parameter curvature revisited and the Bayesian frequentist divergence. J. Statist. Res. 44 335–346.
Fraser, A. M., Fraser, D. A. S. and Staicu, A. M. (2010c). Second order ancillary: A differential view with continuity. Bernoulli 16 1208–1223.
Mathematical Reviews (MathSciNet): MR2759176
Digital Object Identifier: doi:10.3150/10-BEJ248
Project Euclid: euclid.bj/1290092903
Fraser, D. A. S. and McDunnough, P. (1980). Some remarks on conditional and unconditional inference for location-scale models. Statist. Hefte (N.F.) 21 224–231.
Mathematical Reviews (MathSciNet): MR639293
Zentralblatt MATH: 0444.62045
Digital Object Identifier: doi:10.1007/BF02932616
Fraser, D. A. S., Reid, N. and Wong, A. (2004). Setting confidence intervals for bounded parameters: A different perspective. Phys. Rev. D 69 033002.
Fraser, D. A. S., Reid, N., Marras, E. and Yi, G. Y. (2010b). Default prior for Bayesian and frequentist inference. J. Roy. Statist. Soc. Ser. B 75 631–654.
Mathematical Reviews (MathSciNet): MR2758239
Digital Object Identifier: doi:10.1111/j.1467-9868.2010.00750.x
Heinrich, J. (2006). The Bayesian approach to setting limits: What to avoid? In Statistical Problems in Particle Physics, Astrophysics and Cosmology (L. Lyons and Ü. M. Karagöz, eds.) 98–102. Imperial College Press, London.
Jeffreys, H. (1939). Theory of Probability, 3rd ed. Oxford Univ. Press, Oxford.
Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. London. Ser. A 186 453–461.
Mathematical Reviews (MathSciNet): MR17504
Digital Object Identifier: doi:10.1098/rspa.1946.0056
Lindley, D. V. (1958). Fiducial distributions and Bayes’ theorem. J. Roy. Statist. Soc. Ser. B 20 102–107.
Mathematical Reviews (MathSciNet): MR95550
Mandelkern, M. (2002). Setting confidence intervals for bounded parameters. Statist. Sci. 17 149–172.
Mathematical Reviews (MathSciNet): MR1939335
Digital Object Identifier: doi:10.1214/ss/1030550859
Project Euclid: euclid.ss/1030550859
Neyman, J. (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 237 333–380.
Reid, N. and Fraser, D. A. S. (2003). Likelihood inference in the presence of nuisance parameters. In Proceedings of PHYSTAT2003 (L. Lyons, R. Mount and R. Reitmeyer, eds.) 265–271. SLAC E-Conf C030908.
Stainforth, D. A., Allen, M. R., Tredger, E. R. and Smith, L. A. (2007). Confidence, uncertainty and decision-support relevance in climate predictions. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 365 2145–2162. See also: Gambling on tomorrow. Modelling the Earth’s climate mathematically is hard already. Now a new difficulty is emerging. Economist August 18 (2007) 69.
Wasserman, L. (2000). Asymptotic inference for mixture models using data-dependent priors. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 159–180.
Mathematical Reviews (MathSciNet): MR1747402
Zentralblatt MATH: 0976.62028
Digital Object Identifier: doi:10.1111/1467-9868.00226
Woodroofe, M. and Wang, H. (2000). The problem of low counts in a signal plus noise model. Ann. Statist. 28 1561–1569.
Mathematical Reviews (MathSciNet): MR1835031
Zentralblatt MATH: 1105.62300
Digital Object Identifier: doi:10.1214/aos/1015956708
Project Euclid: euclid.aos/1015957470
Zhang, T. and Woodroofe, M. (2003). Credible and confidence sets for restricted parameter spaces. J. Statist. Plann. Inference 115 479–490.
Mathematical Reviews (MathSciNet): MR1985880
Zentralblatt MATH: 1030.62019
Digital Object Identifier: doi:10.1016/S0378-3758(02)00170-2
