Calibration-Based Empirical Probability

Abstract

Probability forecasts for a sequence of uncertain events may be compared with the outcomes of those events by means of a natural criterion of empirical validity, calibration. It is shown that any two sequences of forecasts which both meet this criterion must be in asymptotic agreement. These agreed values can then be considered as correct objective probability forecasts for the particular sequence of outcome results obtained. However, the objective forecasts vary with the extent of the information taken into account when they are formulated. We thus obtain a general theory of empirical probability, relative to an information base. This theory does not require that such probabilities be interpreted in terms of repeated trials of the same event. Some implications of this theory are discussed.