Electronic Journal of Statistics

Controlling the degree of caution in statistical inference with the Bayesian and frequentist approaches as opposite extremes

David R. Bickel

Full-text: Open access

Abstract

In statistical practice, whether a Bayesian or frequentist approach is used in inference depends not only on the availability of prior information but also on the attitude taken toward partial prior information, with frequentists tending to be more cautious than Bayesians. The proposed framework defines that attitude in terms of a specified amount of caution, thereby enabling data analysis at the level of caution desired and on the basis of prior information. The caution parameter represents the attitude toward partial prior information in much the same way as a loss function represents the attitude toward risk. When there is very little prior information and nonzero caution, the resulting inferences correspond to those of the candidate confidence intervals and p-values that are most similar to the credible intervals and hypothesis probabilities of the specified Bayesian posterior. On the other hand, in the presence of a known physical distribution of the parameter, inferences are based only on the corresponding physical posterior. In those extremes of either negligible prior information or complete prior information, inferences do not depend on the degree of caution. Partial prior information between those two extremes leads to intermediate inferences that are more frequentist to the extent that the caution is high and more Bayesian to the extent that the caution is low.

Article information

Source
Electron. J. Statist., Volume 6 (2012), 686-709.

Dates
First available in Project Euclid: 27 April 2012

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1335531219

Digital Object Identifier
doi:10.1214/12-EJS689

Mathematical Reviews number (MathSciNet)
MR2988425

Zentralblatt MATH identifier
1274.62052

Subjects
Primary: 62A01: Foundations and philosophical topics
Secondary: 62A99: None of the above, but in this section

Keywords
Ambiguity blended inference conditional Gamma-minimax confidence distribution confidence posterior Ellsberg paradox imprecise probability maximum entropy maxmin expected utility minimum cross entropy minimum divergence minimum information for discrimination minimum relative entropy observed confidence level robust Bayesian analysis

Citation

Bickel, David R. Controlling the degree of caution in statistical inference with the Bayesian and frequentist approaches as opposite extremes. Electron. J. Statist. 6 (2012), 686--709. doi:10.1214/12-EJS689. https://projecteuclid.org/euclid.ejs/1335531219


Export citation

References

  • Augustin, T. (2002). Expected utility within a generalized concept of probability - a comprehensive framework for decision making under ambiguity., Statistical Papers 43 5-22.
  • Bayati Eshkaftaki, A. and Parsian, A. (2011). Robust Bayes estimation., Communications in Statistics - Theory and Methods 40 929–941.
  • Berger, J. O., Insua, D. R. and Ruggeri, F. (2000)., Bayesian Robustness. Robust Bayesian Analysis 1 1-32. Springer, New York.
  • Betrò, B. and Ruggeri, F. (1992). Conditional, Γ-minimax actions under convex losses. Communications in Statistics - Theory and Methods 21 1051–1066.
  • Bickel, D. R. (2011a). Blending Bayesian and frequentist methods according to the precision of prior information with an application to hypothesis testing., Technical Report, Ottawa Institute of Systems Biology, arXiv:1107.2353.
  • Bickel, D. R. (2011b). Estimating the null distribution to adjust observed confidence levels for genome-scale screening., Biometrics 67 363-370.
  • Bickel, D. R. (2011c). Small-scale inference: Empirical Bayes and confidence methods for as few as a single comparison., Technical Report, Ottawa Institute of Systems Biology, arXiv:1104.0341.
  • Bickel, D. R. (2011d). The strength of statistical evidence for composite hypotheses: Inference to the best explanation., Statistica Sinica DOI: 10.5705/ss.2009.125 (online ahead of print).
  • Bickel, D. R. (2012a). Coherent frequentism: A decision theory based on confidence sets. To appear in, Communications in Statistics - Theory and Methods; 2009 version available from arXiv:0907.0139.
  • Bickel, D. R. (2012b). Game-theoretic probability combination with applications to resolving conflicts between statistical methods. To appear in, International Journal of Approximate Reasoning; 2011 version available from arXiv:1111.6174.
  • Carlin, B. P. and Louis, T. A. (2009)., Bayesian Methods for Data Analysis, Third Edition. Chapman & Hall/CRC, New York.
  • Cover, T. M. and Thomas, J. A. (2006)., Elements of Information Theory. John Wiley and Sons, New York.
  • Csiszár, I. (1985). An extended maximum entropy principle and a Bayesian justification. In, Bayesian Statistics 2 (J. M. Bernardo, M. B. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 83–98. Elsevier Inc., Amsterdam.
  • Efron, B. (1993). Bayes and likelihood calculations from confidence intervals., Biometrika 80 3-26.
  • Efron, B. (2005). Bayesians, Frequentists, and Scientists., Journal of the American Statistical Association 100 1–5.
  • Ellsberg, D. (1961). Risk, ambiguity, and the Savage axioms., The Quarterly Journal of Economics 75 pp. 643-669.
  • Fraser, D. A. S. (2011). Is Bayes posterior just quick and dirty confidence?, Statistical Science 26 299–316.
  • Gajdos, T., Hayashi, T., Tallon, J. M. and Vergnaud, J. C. (2008). Attitude toward imprecise information., Journal of Economic Theory 140 27-65.
  • Gajdos, T., Tallon, J. M. and Vergnaud, J. C. (2004). Decision making with imprecise probabilistic information., Journal of Mathematical Economics 40 647-681.
  • Gärdenfors, P. and Sahlin, N.-E. (1982). Unreliable probabilities, risk taking, and decision making., Synthese 53 361-386. 10.1007/BF00486156.
  • Gilboa, I. and Schmeidler, D. (1989). Maxmin expected utility with non-unique prior., Journal of Mathematical Economics 18 141-153.
  • Grünwald, P. D. and Dawid, A. P. (2004). Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory., Annals of Statistics 32 1367-1433.
  • Hannig, J. (2009). On generalized fiducial inference., Statistica Sinica 19 491-544.
  • Harremoës, P. (2007). Information topologies with applications. In, Entropy, Search, Complexity, (I. Csiszár, G. O. H. Katona, G. Tardos and G. Wiener, eds.). Bolyai Society Mathematical Studies 16 113–150. Springer Berlin Heidelberg, Berlin, Heidelberg.
  • Hurwicz, L. (1951a). Optimality criteria for decision making under ignorance., Cowles Commission Discussion Paper 370.
  • Hurwicz, L. (1951b). The generalized Bayes-minimax principle: a criterion for decision-making under uncertainty., Cowles Commission Discussion Paper 355.
  • Jaffray, J. Y. (1989a). Généralisation du critère de l’utilité espérée aux choix dans l’incertain régulier., RAIRO: Recherche opérationnelle 23 237–267.
  • Jaffray, J.-Y. (1989b). Linear utility theory for belief functions., Operations Research Letters 8 107-112.
  • Jaynes, E. T. (2003)., Probability Theory: The Logic of Science.
  • Kardaun, O. J. W. F., Salomi, D., Schaafsma, W., Steerneman, A. G. M., Willems, J. C. and Cox, D. R. (2003). Reflections on Fourteen Cryptic Issues concerning the Nature of Statistical Inference., International Statistical Review / Revue Internationale de Statistique 71 277-303.
  • Kullback, S. (1968)., Information Theory and Statistics. Dover, New York.
  • Paris, J. B. (1994)., The Uncertain Reasoner’s Companion: A Mathematical Perspective. Cambridge University Press, New York.
  • Pfaffelhuber, E. (1977). Minimax Information Gain and Minimum Discrimination Principle. In, Topics in Information Theory (I. Csiszár and P. Elias, eds.). Colloquia Mathematica Societatis János Bolyai 16 493–519. János Bolyai Mathematical Society and North-Holland.
  • Polansky, A. M. (2007)., Observed Confidence Levels: Theory and Application. Chapman and Hall, New York.
  • Samaniego, F. J. (2010)., A Comparison of the Bayesian and Frequentist Approaches to Estimation (Springer Series in Statistics). Springer, New York.
  • Savage, L. J. (1954)., The Foundations of Statistics. John Wiley and Sons, New York.
  • Schweder, T. and Hjort, N. L. (2002). Confidence and likelihood., Scandinavian Journal of Statistics 29 309-332.
  • Sellke, T., Bayarri, M. J. and Berger, J. O. (2001). Calibration of p values for testing precise null hypotheses., American Statistician 55 62-71.
  • Singh, K., Xie, M. and Strawderman, W. E. (2005). Combining information from independent sources through confidence distributions., Annals of Statistics 33 159-183.
  • Tapking, J. (2004). Axioms for preferences revealing subjective uncertainty and uncertainty aversion., Journal of Mathematical Economics 40 771–797.
  • Topsøe, F. (1979). Information theoretical optimization techniques., Kybernetika 15 8-27.
  • Topsøe, F. (2004). Information theory and complexity in probability and statistics. In, Soft Methodology and Random Information Systems (LopezDiaz, M. and Gil, M. A. and Grzegorzewski, P. and Hryniewicz, O. and Lawry, J., ed.). Advances in Soft Computing 363-370.
  • Topsøe, F. (2007). Information Theory at the Service of Science. In, Entropy, Search, Complexity, (I. Csiszár, G. O. H. Katona, G. Tardos and G. Wiener, eds.). Bolyai Society Mathematical Studies 179-207. Springer Berlin Heidelberg.
  • van Berkum, E. E. M., Linssen, H. N. and Overdijk, D. A. (1996). Inference rules and inferential distributions., Journal of Statistical Planning and Inference 49 305-317.
  • Vidakovic, B. (2000)., Gamma-minimax: A paradigm for conservative robust Bayesians. Robust Bayesian Analysis 241–260. Springer, New York.
  • von Neumann, J. and Morgenstern, O. (1953)., Theory of Games and Economic Behavior. Princeton University Press, Princeton.
  • Wald, A. (1961)., Statistical Decision Functions. John Wiley and Sons, New York.
  • Walley, P. (1991)., Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, London.
  • Weichselberger, K. (2001)., Elementare Grundbegriffe einer allgemeineren Wahrscheinlichkeitsrechnung I: Intervallwahrscheinlichkeit als umfassendes Konzept. Physica-Verlag, Heidelberg.
  • Williams, P. M. (1980). Bayesian Conditionalisation and the Principle of Minimum Information., The British Journal for the Philosophy of Science 31 131-144.