Open Access
Translator Disclaimer
December 2017 Focusing on regions of interest in forecast evaluation
Hajo Holzmann, Bernhard Klar
Ann. Appl. Stat. 11(4): 2404-2431 (December 2017). DOI: 10.1214/17-AOAS1088


Often, interest in forecast evaluation focuses on certain regions of the whole potential range of the outcome, and forecasts should mainly be ranked according to their performance within these regions. A prime example is risk management, which relies on forecasts of risk measures such as the value-at-risk or the expected shortfall, and hence requires appropriate loss distribution forecasts in the tails. Further examples include weather forecasts with a focus on extreme conditions, or forecasts of environmental variables such as ozone with a focus on concentration levels with adverse health effects.

In this paper, we show how weighted scoring rules can be used to this end, and in particular that they allow to rank several potentially misspecified forecasts objectively with the region of interest in mind. This is demonstrated in various simulation scenarios. We introduce desirable properties of weighted scoring rules and present general construction principles based on conditional densities or distributions and on scoring rules for probability forecasts. In our empirical application to log-return time series, all forecasts seem to be slightly misspecified, as is often unavoidable in practice, and no method performs best overall. However, using weighted scoring functions the best method for predicting losses can be identified, which is hence the method of choice for the purpose of risk management.


Download Citation

Hajo Holzmann. Bernhard Klar. "Focusing on regions of interest in forecast evaluation." Ann. Appl. Stat. 11 (4) 2404 - 2431, December 2017.


Received: 1 March 2017; Revised: 1 August 2017; Published: December 2017
First available in Project Euclid: 28 December 2017

zbMATH: 1383.62243
MathSciNet: MR3743302
Digital Object Identifier: 10.1214/17-AOAS1088

Keywords: financial time series , locally proper weighted scoring rule , misspecified forecast , predictive performance , probabilistic forecast , rare and extreme events , risk management

Rights: Copyright © 2017 Institute of Mathematical Statistics


Vol.11 • No. 4 • December 2017
Back to Top