The Annals of Statistics

Higher order elicitability and Osband’s principle

Tobias Fissler and Johanna F. Ziegel

Full-text: Open access

Enhanced PDF (314 KB)

Abstract

A statistical functional, such as the mean or the median, is called elicitable if there is a scoring function or loss function such that the correct forecast of the functional is the unique minimizer of the expected score. Such scoring functions are called strictly consistent for the functional. The elicitability of a functional opens the possibility to compare competing forecasts and to rank them in terms of their realized scores. In this paper, we explore the notion of elicitability for multi-dimensional functionals and give both necessary and sufficient conditions for strictly consistent scoring functions. We cover the case of functionals with elicitable components, but we also show that one-dimensional functionals that are not elicitable can be a component of a higher order elicitable functional. In the case of the variance, this is a known result. However, an important result of this paper is that spectral risk measures with a spectral measure with finite support are jointly elicitable if one adds the “correct” quantiles. A direct consequence of applied interest is that the pair (Value at Risk, Expected Shortfall) is jointly elicitable under mild conditions that are usually fulfilled in risk management applications.

Article information

Source
Ann. Statist., Volume 44, Number 4 (2016), 1680-1707.

Dates
Received: April 2015
Revised: January 2016
First available in Project Euclid: 7 July 2016

Permanent link to this document
https://projecteuclid.org/euclid.aos/1467894712

Digital Object Identifier
doi:10.1214/16-AOS1439

Mathematical Reviews number (MathSciNet)
MR3519937

Zentralblatt MATH identifier
1355.62006

Subjects
Primary: 62C99: None of the above, but in this section
Secondary: 91B06: Decision theory [See also 62Cxx, 90B50, 91A35]

Keywords
Consistency decision theory elicitability Expected Shortfall point forecasts propriety scoring functions scoring rules spectral risk measures Value at Risk

Citation

Fissler, Tobias; Ziegel, Johanna F. Higher order elicitability and Osband’s principle. Ann. Statist. 44 (2016), no. 4, 1680--1707. doi:10.1214/16-AOS1439. https://projecteuclid.org/euclid.aos/1467894712


Export citation

References

  • Abernethy, J. D. and Frongillo, R. M. (2012). A characterization of scoring rules for linear properties. In Proceedings of the 25th Annual Conference on Learning Theory. Journal of Machine Learning Research: Workshop and Conference Proceedings, Vol. 23 (S. Mannor, N. Srebro andR. C. Williamson, eds.) 27.1–27.13.
  • Acerbi, C. (2002). Spectral measures of risk: A coherent representation of subjective risk aversion. J. Bank. Financ. 26 1505–1518.
  • Acerbi, C. and Székely, B. (2014). Backtesting expected shortfall. Risk Mag. 27 76–81.
  • Artzner, P., Delbaen, F., Eber, J.-M. and Heath, D. (1999). Coherent measures of risk. Math. Finance 9 203–228.
    Mathematical Reviews (MathSciNet): MR1850791
    Digital Object Identifier: doi:10.1111/1467-9965.00068
  • Banerjee, A., Guo, X. and Wang, H. (2005). On the optimality of conditional expectation as a Bregman predictor. IEEE Trans. Inform. Theory 51 2664–2669.
    Mathematical Reviews (MathSciNet): MR2246384
    Digital Object Identifier: doi:10.1109/TIT.2005.850145
  • Bellini, F. and Bignozzi, V. (2015). On elicitable risk measures. Quant. Finance 15 725–733.
    Mathematical Reviews (MathSciNet): MR3334566
    Digital Object Identifier: doi:10.1080/14697688.2014.946955
  • Cont, R., Deguest, R. and Scandolo, G. (2010). Robustness and sensitivity analysis of risk measurement procedures. Quant. Finance 10 593–606.
    Mathematical Reviews (MathSciNet): MR2676786
    Digital Object Identifier: doi:10.1080/14697681003685597
  • Daníelsson, J., Embrechts, P., Goodhart, C., Keating, C., Muennich, F., Renault, O. and Shin, H. S. (2001). An academic response to Basel II. Special Paper No. 130.
  • Davis, M. H. A. (2016). Verification of internal risk measure estimates. Stat. Risk Model. To appear.
  • Dawid, A. P. and Sebastiani, P. (1999). Coherent dispersion criteria for optimal experimental design. Ann. Statist. 27 65–81.
    Mathematical Reviews (MathSciNet): MR1701101
    Digital Object Identifier: doi:10.1214/aos/1018031101
    Project Euclid: euclid.aos/1018031101
  • Delbaen, F. (2012). Monetary Utility Functions. Osaka Univ. Press, Osaka.
  • Delbaen, F., Bellini, F., Bignozzi, V. and Ziegel, J. F. (2016). Risk measures with the CxLS property. Finance Stoch. 20 433–453.
    Mathematical Reviews (MathSciNet): MR3479327
    Digital Object Identifier: doi:10.1007/s00780-015-0279-6
  • Embrechts, P. and Hofert, M. (2014). Statistics and quantitative risk management for banking and insurance. Ann. Rev. Stat. Appl. 1.
  • Emmer, S., Kratz, M. and Tasche, D. (2015). What is the best risk measure in practice? A comparison of standard measures. J. Risk 18 31–60.
  • Engelberg, J., Manski, C. F. and Williams, J. (2009). Comparing the point predictions and subjective probability distributions of professional forecasters. J. Bus. Econom. Statist. 27 30–41.
    Mathematical Reviews (MathSciNet): MR2484982
  • Fissler, T. and Ziegel, J. F. (2016). Supplement to “Higher order elicitability and Osband’s principle.” DOI:10.1214/16-AOS1439SUPP.
    Mathematical Reviews (MathSciNet): MR3519937
    Digital Object Identifier: doi:10.1214/16-AOS1439
    Project Euclid: euclid.aos/1467894712
  • Frongillo, R. and Kash, I. (2015). Vector-valued property elicitation. J. Mach. Learn. Res. Workshop Conf. Proc. 40 1–18.
  • Gneiting, T. (2011). Making and evaluating point forecasts. J. Amer. Statist. Assoc. 106 746–762.
    Mathematical Reviews (MathSciNet): MR2847988
    Digital Object Identifier: doi:10.1198/jasa.2011.r10138
  • Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359–378.
    Mathematical Reviews (MathSciNet): MR2345548
    Digital Object Identifier: doi:10.1198/016214506000001437
  • Grauert, H. and Fischer, W. (1978). Differential- und Integralrechnung. II: Differentialrechnung in Mehreren Veränderlichen Differentialgleichungen, corrected ed. Heidelberger Taschenbücher [Heidelberg Paperbacks] 36. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR514400
  • Heinrich, C. (2014). The mode functional is not elicitable. Biometrika 101 245–251.
    Mathematical Reviews (MathSciNet): MR3180670
    Digital Object Identifier: doi:10.1093/biomet/ast048
  • Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35 73–101.
    Mathematical Reviews (MathSciNet): MR161415
    Digital Object Identifier: doi:10.1214/aoms/1177703732
    Project Euclid: euclid.aoms/1177703732
  • Jouini, E., Schachermayer, W. and Touzi, N. (2006). Law invariant risk measures have the Fatou property. In Advances in Mathematical Economics. Vol. 9. Adv. Math. Econ. 9 49–71. Springer, Tokyo.
    Mathematical Reviews (MathSciNet): MR2277714
    Digital Object Identifier: doi:10.1007/4-431-34342-3_4
  • Koenker, R. (2005). Quantile Regression. Econometric Society Monographs 38. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR2268657
  • Kou, S. and Peng, X. (2014). On the measurement of economic tail risk. Preprint. Available at http://arxiv.org/pdf/1401.4787v2.pdf.
  • Kou, S., Peng, X. and Heyde, C. C. (2013). External risk measures and Basel accords. Math. Oper. Res. 38 393–417.
    Mathematical Reviews (MathSciNet): MR3092538
    Digital Object Identifier: doi:10.1287/moor.1120.0577
  • Krätschmer, V., Schied, A. and Zähle, H. (2012). Qualitative and infinitesimal robustness of tail-dependent statistical functionals. J. Multivariate Anal. 103 35–47.
    Mathematical Reviews (MathSciNet): MR2823706
    Digital Object Identifier: doi:10.1016/j.jmva.2011.06.005
  • Krätschmer, V., Schied, A. and Zähle, H. (2015). Quasi-Hadamard differentiability of general risk functionals and its applications. Stat. Risk Model. 32 25–47.
    Mathematical Reviews (MathSciNet): MR3331780
  • Krätschmer, V., Schied, A. and Zähle, H. (2014). Comparative and qualitative robustness for law-invariant risk measures. Finance Stoch. 18 271–295.
    Mathematical Reviews (MathSciNet): MR3177407
    Digital Object Identifier: doi:10.1007/s00780-013-0225-4
  • Kusuoka, S. (2001). On law invariant coherent risk measures. In Advances in Mathematical Economics, Vol. 3. Adv. Math. Econ. 3 83–95. Springer, Tokyo.
    Mathematical Reviews (MathSciNet): MR1886557
    Digital Object Identifier: doi:10.1007/978-4-431-67891-5_4
  • Lambert, N. (2013). Elicitation and evaluation of statistical functionals. Preprint. Available at http://web.stanford.edu/~nlambert/papers/elicitation.pdf.
  • Lambert, N., Pennock, D. M. and Shoham, Y. (2008). Eliciting properties of probability distributions. In Proceedings of the 9th ACM Conference on Electronic Commerce 129–138. ACM, Chicago, IL.
  • Murphy, A. H. and Daan, H. (1985). Forecast evaluation. In Probability, Statistics and Decision Making in the Atmospheric Sciences (A. H. Murphy and R. W. Katz, eds.) 379–437. Westview Press, Boulder, CO.
  • Newey, W. K. and Powell, J. L. (1987). Asymmetric least squares estimation and testing. Econometrica 55 819–847.
    Mathematical Reviews (MathSciNet): MR906565
    Digital Object Identifier: doi:10.2307/1911031
  • Osband, K. H. (1985). Providing incentives for better cost forecasting. Ph.D. thesis, Univ. California, Berkeley.
  • Osband, K. and Reichelstein, S. (1985). Information-eliciting compensation schemes. J. Public Econ. 27 107–115.
  • Savage, L. J. (1971). Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66 783–801.
    Mathematical Reviews (MathSciNet): MR331571
    Digital Object Identifier: doi:10.1080/01621459.1971.10482346
  • Steinwart, I., Pasin, C., Williamson, R. and Zhang, S. (2014). Elicitation and identification of properties. In Proceedings of the 27th Conference on Learning Theory. Journal of Machine Learning Research: Workshop and Conference Proceedings, Vol. 35 (M. F. Balcan andC. Szepesvari, eds.) 482–526.
  • Wang, R. and Ziegel, J. F. (2015). Elicitable distortion risk measures: A concise proof. Statist. Probab. Lett. 100 172–175.
    Mathematical Reviews (MathSciNet): MR3324090
  • Weber, S. (2006). Distribution-invariant risk measures, information, and dynamic consistency. Math. Finance 16 419–441.
    Mathematical Reviews (MathSciNet): MR2212272
    Digital Object Identifier: doi:10.1111/j.1467-9965.2006.00277.x
  • Ziegel, J. F. (2014). Coherence and elicitability. Math. Finance. To appear.

Supplemental materials

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more