Statistical Science

Unified frequentist and Bayesian testing of a precise hypothesis

J. O. Berger, B. Boukai, and Y. Wang

Full-text: Open access

Abstract

In this paper, we show that the conditional frequentist method of testing a precise hypothesis can be made virtually equivalent to Bayesian testing. The conditioning strategy proposed by Berger, Brown and Wolpert in 1994, for the simple versus simple case, is generalized to testing a precise null hypothesis versus a composite alternative hypothesis. Using this strategy, both the conditional frequentist and the Bayesian will report the same error probabilities upon rejecting or accepting. This is of considerable interest because it is often perceived that Bayesian and frequentist testing are incompatible in this situation. That they are compatible, when conditional frequentist testing is allowed, is a strong indication that the "wrong" frequentist tests are currently being used for postexperimental assessment of accuracy. The new unified testing procedure is discussed and illustrated in several common testing situations.

Article information

Source
Statist. Sci., Volume 12, Number 3 (1997), 133-160.

Dates
First available in Project Euclid: 22 August 2002

Permanent link to this document
https://projecteuclid.org/euclid.ss/1030037904

Digital Object Identifier
doi:10.1214/ss/1030037904

Mathematical Reviews number (MathSciNet)
MR1617518

Zentralblatt MATH identifier
0955.62527

Keywords
Bayes factor likelihood ratio composite hypothesis conditional test error probabilities

Citation

Berger, J. O.; Boukai, B.; Wang, Y. Unified frequentist and Bayesian testing of a precise hypothesis. Statist. Sci. 12 (1997), no. 3, 133--160. doi:10.1214/ss/1030037904. https://projecteuclid.org/euclid.ss/1030037904


Export citation

References

  • Berger, J. O., Brown, L. D. and Wolpert, R. L. (1994). A unified conditional frequentist and Bayesian test for fixed and sequential simple hy pothesis testing. Ann. Statist. 22 1787- 1807.
  • Berger, J. O. and Delampady, M. (1987). Testing precise hy potheses. Statist. Sci. 3 317-352.
  • Berger, J. O. and Pericchi, L. R. (1996). The intrinsic Bay es factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109-122.
  • Berger, J. O. and Sellke, T. (1987). Testing a point null hy pothesis: the irreconcilability of P-values and evidence. J. Amer. Statist. Assoc. 82 112-122.
  • Berger, J. O. and Wolpert, R. L. (1988). The Likelihood Principle, 2nd ed. IMS, Hay ward, CA.
  • Bernardo, J. M. (1980). A Bayesian analysis of classical hy pothesis testing. In Bayesian Statistics (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 605-647. Valencia Univ. Press.
  • Brown, L. D. (1978). A contribution to Kiefer's theory of conditional confidence procedures. Ann. Statist. 6 59-71.
  • Brownie, C. and Keifer, J. (1977). The ideas of conditional confidence in the simplest setting. Comm. Statist. Theory Methods 6 691-751.
  • Casella, G. and Berger, R. (1987). Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J. Amer. Statist. Assoc. 82 106-111.
  • Chatterjee, S. K. and Chattopadhy ay, G. (1993). Detailed statistical inference-multiple decision problems. Calcutta Statist. Assoc. Bull. 43 155-180.
  • Chernoff, H. (1972). Sequential Analy sis and Optimal Design.
  • SIAM, Philadelphia.
  • Delampady, M. and Berger, J. O. (1990). Lower bounds on Bay es factors for the multinomial distribution, with application to chi-squared tests of fit. Ann. Statist. 18 1295-1316.
  • Edwards, W., Lindman, H. and Savage, L. J. (1963). Bayesian statistical inference for psy chological research. Psy chological Review 70 193-242.
  • Evans, M. (1994). Bayesian inference procedures derived via the concept of relative surprise. Technical report, Dept. Statistics, Univ. Toronto.
  • Fisher, L. D. and Van Belle, G. (1993). Biostatistics: A Methodology for the Health Sciences. Wiley, New York.
  • Good, I. J. (1992). The Bayesian/Non-Bayesian compromise: a brief review. J. Amer. Statist. Assoc. 87 597-606. Hwang, J. T., Casella, G., Robert, C., Wells, M. T. and Far
  • rell, R. (1992). Estimation of accuracy in testing. Ann. Statist. 20 490-509.
  • Jeffrey s, H. (1961). Theory of Probability. Oxford Univ. Press.
  • Kiefer, J. (1975). Conditional confidence approach in multi decision problems. In Multivariate Analy sis IV (P. R. Krishnaiah, ed.) 143-158. Academic Press, New York.
  • Kiefer, J. (1976). Admissibility of conditional confidence procedures. Ann. Math. Statist. 4 836-865.
  • Kiefer, J. (1977). Conditional confidence statements and confidence estimators (with discussion). J. Amer. Statist. Assoc. 72 789-827.
  • O'Hagan, A. (1995). Fractional Bay es factors for model comparisons. J. Roy. Statist. Soc. Ser. B 57 99-138.
  • Pappas, T. and Mitchell, C. A. (1985). Effects of seismic stress on the vegetative growth of Gly cine max (L.) Merr. cv. Wells II. Plant, Cell and Environment 8 143-148.
  • Robert, C. P. and Caron, N. (1995). Noninformative Bayesian testing and neutral Bay es factors. Technical report, CREST,
  • INSEE, Paris.
  • Schaafsma, W. and van der Meulen, A. E. (1993). Assessing weights of evidence for discussing classical statistical hy potheses. Statist. Decisions 11 201-220.
  • Wolpert, R. L. (1995). Testing simple hy potheses. In Studies in Classification, Data Analy sis, and Knowledge Organization (H. H. Bock and W. Polasek, eds.) 7 289-297. Springer, Berlin.
  • Berger, J. O., Brown, L. D. and Wolpert, R. L. (1994). A unified conditional frequentist and Bayesian test for fixed and sequential simple hy pothesis testing. Ann. Statist. 22 1787- 1807.
  • Lindley, D. V. (1972). Bayesian Statistics. A Review. SIAM, Philadelphia.
  • Rubin, H. (1987). A weak sy stem of axioms for `rational' behavior and the non-separability of utility from prior. Statist. Decisions 5 47-58.
  • Carlin, B. P. and Louis, T. A. (1996). Bay es and Empirical Bay es Methods for Data Analy sis. Chapman and Hall, London.
  • Fisher, L. D. (1996). Comments on Bayesian and frequentist analysis and interpretation of clinical trials. Controlled Clinical Trials 17 423-434.
  • Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12 1151-1172.
  • Atkinson, A. C. (1985). Plots, Transformations and Regression. Oxford Univ. Press.
  • Cohen, R. (1994). The Earth is round p < 05. American Psy chologist 49 997-1003.
  • Davison, A. C. and Hinkley, D. V. (1997). The Bootstrap and Its Application. Cambridge Univ. Press.
  • Nester, M. R. (1996). An applied statistician's creed. J. Roy. Statist. Soc. Ser. C 45 401-410.
  • Ripley, B. D. (1996). Pattern Recognition and Classification. Cambridge Univ. Press.

See also

  • Includes: Dennis V. Lindley. Comment by Dennis V. Lindley.
  • Includes: Thomas A. Louis. Comment by Thomas A. Louis.
  • Includes: David Hinkley. Comment by David Hinkley.
  • Includes: J. O. Berger, B. Boukai, Y. Wang. Rejoinder by J. O. Berger, B. Boukai and Y. Wang.