Annals of Statistics

Proper local scoring rules on discrete sample spaces

Abstract

A scoring rule is a loss function measuring the quality of a quoted probability distribution Q for a random variable X, in the light of the realized outcome x of X; it is proper if the expected score, under any distribution P for X, is minimized by quoting Q = P. Using the fact that any differentiable proper scoring rule on a finite sample space $\mathcal{X}$ is the gradient of a concave homogeneous function, we consider when such a rule can be local in the sense of depending only on the probabilities quoted for points in a nominated neighborhood of x. Under mild conditions, we characterize such a proper local scoring rule in terms of a collection of homogeneous functions on the cliques of an undirected graph on the space $\mathcal{X}$. A useful property of such rules is that the quoted distribution Q need only be known up to a scale factor. Examples of the use of such scoring rules include Besag’s pseudo-likelihood and Hyvärinen’s method of ratio matching.

Article information

Source
Ann. Statist., Volume 40, Number 1 (2012), 593-608.

Dates
First available in Project Euclid: 7 May 2012

https://projecteuclid.org/euclid.aos/1336396184

Digital Object Identifier
doi:10.1214/12-AOS972

Mathematical Reviews number (MathSciNet)
MR3014318

Zentralblatt MATH identifier
1246.62010

Subjects
Primary: 62C99: None of the above, but in this section
Secondary: 62A99: None of the above, but in this section

Citation

Dawid, A. Philip; Lauritzen, Steffen; Parry, Matthew. Proper local scoring rules on discrete sample spaces. Ann. Statist. 40 (2012), no. 1, 593--608. doi:10.1214/12-AOS972. https://projecteuclid.org/euclid.aos/1336396184

References

• Bernardo, J.-M. (1979). Expected information as expected utility. Ann. Statist. 7 686–690.
• Besag, J. (1975). Statistical analysis of non-lattice data. J. Roy. Statist. Soc. Ser. D (The Statistician) 24 179–195.
• Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review 78 1–3.
• Czado, C., Gneiting, T. and Held, L. (2009). Predictive model assessment for count data. Biometrics 65 1254–1261.
• Dawid, A. P. (1986). Probability forecasting. In Encyclopedia of Statistical Sciences (S. Kotz, N. L. Johnson and C. B. Read, eds.) 7 210–218. Wiley, New York.
• Dawid, A. P. (2007). The geometry of proper scoring rules. Ann. Inst. Statist. Math. 59 77–93.
• Dawid, A. P. and Lauritzen, S. L. (2005). The geometry of decision theory. In Proceedings of the Second International Symposium on Information Geometry and Its Applications 22–28. Univ. Tokyo, Tokyo, Japan.
• Ehm, W. and Gneiting, T. (2012). Local proper scoring rules of order two. Ann. Statist. 40 609–637.
• Good, I. J. (1971). Comment on “Measuring information and uncertainty,” by Robert J. Buehler. In Foundations of Statistical Inference (V. P. Godambe and D. A. Sprott, eds.) 337–339. Holt, Rinehart and Winston, Toronto.
• Grimmett, G. R. (1973). A theorem about random fields. Bull. Lond. Math. Soc. 5 81–84.
• Hendrickson, A. D. and Buehler, R. J. (1971). Proper scores for probability forecasters. Ann. Math. Statist. 42 1916–1921.
• Hyvärinen, A. (2007). Some extensions of score matching. Comput. Statist. Data Anal. 51 2499–2512.
• Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series 17. Clarendon Press, Oxford, UK.
• McCarthy, J. (1956). Measures of the value of information. Proc. Nat. Acad. Sci. 42 654–655.
• Parry, M. F., Dawid, A. P. and Lauritzen, S. L. (2012). Proper local scoring rules. Ann. Statist. 40 561–592.
• Rockafellar, R. T. (1970). Convex Analysis. Princeton Mathematical Series 28. Princeton Univ. Press, Princeton, NJ.
• Zelterman, D. (1988). Robust estimation in truncated discrete distributions with application to capture–recapture experiments. J. Statist. Plann. Inference 18 225–237.