Translator Disclaimer
2016 A note on the use of empirical AUC for evaluating probabilistic forecasts
Simon Byrne
Electron. J. Statist. 10(1): 380-393 (2016). DOI: 10.1214/16-EJS1109


Scoring functions are used to evaluate and compare partially probabilistic forecasts. We investigate the use of rank-sum functions such as empirical Area Under the Curve (AUC), a widely used measure of classification performance, as a scoring function for the prediction of probabilities of a set of binary outcomes. It is shown that the AUC is not generally a proper scoring function, that is, under certain circumstances it is possible to improve on the expected AUC by modifying the quoted probabilities from their true values. However with some restrictions, or with certain modifications, it can be made proper.


Download Citation

Simon Byrne. "A note on the use of empirical AUC for evaluating probabilistic forecasts." Electron. J. Statist. 10 (1) 380 - 393, 2016.


Received: 1 August 2015; Published: 2016
First available in Project Euclid: 17 February 2016

zbMATH: 06549025
MathSciNet: MR3466187
Digital Object Identifier: 10.1214/16-EJS1109

Primary: 62C99

Rights: Copyright © 2016 The Institute of Mathematical Statistics and the Bernoulli Society


Vol.10 • No. 1 • 2016
Back to Top