Open Access
June 2014 Probability aggregation in time-series: Dynamic hierarchical modeling of sparse expert beliefs
Ville A. Satopää, Shane T. Jensen, Barbara A. Mellers, Philip E. Tetlock, Lyle H. Ungar
Ann. Appl. Stat. 8(2): 1256-1280 (June 2014). DOI: 10.1214/14-AOAS739


Most subjective probability aggregation procedures use a single probability judgment from each expert, even though it is common for experts studying real problems to update their probability estimates over time. This paper advances into unexplored areas of probability aggregation by considering a dynamic context in which experts can update their beliefs at random intervals. The updates occur very infrequently, resulting in a sparse data set that cannot be modeled by standard time-series procedures. In response to the lack of appropriate methodology, this paper presents a hierarchical model that takes into account the expert’s level of self-reported expertise and produces aggregate probabilities that are sharp and well calibrated both in- and out-of-sample. The model is demonstrated on a real-world data set that includes over 2300 experts making multiple probability forecasts over two years on different subsets of 166 international political events.


Download Citation

Ville A. Satopää. Shane T. Jensen. Barbara A. Mellers. Philip E. Tetlock. Lyle H. Ungar. "Probability aggregation in time-series: Dynamic hierarchical modeling of sparse expert beliefs." Ann. Appl. Stat. 8 (2) 1256 - 1280, June 2014.


Published: June 2014
First available in Project Euclid: 1 July 2014

zbMATH: 06333795
MathSciNet: MR3262553
Digital Object Identifier: 10.1214/14-AOAS739

Keywords: bias estimation , Calibration , Dynamic linear model , expert forecast , hierarchical modeling , Probability aggregation , subjective probability , time series

Rights: Copyright © 2014 Institute of Mathematical Statistics

Vol.8 • No. 2 • June 2014
Back to Top