Translator Disclaimer
2016 A note on the use of empirical AUC for evaluating probabilistic forecasts
Simon Byrne
Electron. J. Statist. 10(1): 380-393 (2016). DOI: 10.1214/16-EJS1109

Abstract

Scoring functions are used to evaluate and compare partially probabilistic forecasts. We investigate the use of rank-sum functions such as empirical Area Under the Curve (AUC), a widely used measure of classification performance, as a scoring function for the prediction of probabilities of a set of binary outcomes. It is shown that the AUC is not generally a proper scoring function, that is, under certain circumstances it is possible to improve on the expected AUC by modifying the quoted probabilities from their true values. However with some restrictions, or with certain modifications, it can be made proper.

Citation

Download Citation

Simon Byrne. "A note on the use of empirical AUC for evaluating probabilistic forecasts." Electron. J. Statist. 10 (1) 380 - 393, 2016. https://doi.org/10.1214/16-EJS1109

Information

Received: 1 August 2015; Published: 2016
First available in Project Euclid: 17 February 2016

zbMATH: 06549025
MathSciNet: MR3466187
Digital Object Identifier: 10.1214/16-EJS1109

Subjects:
Primary: 62C99

Rights: Copyright © 2016 The Institute of Mathematical Statistics and the Bernoulli Society

JOURNAL ARTICLE
14 PAGES


SHARE
Vol.10 • No. 1 • 2016
Back to Top