The Annals of Statistics

Merging and testing opinions

Luciano Pomatto, Nabil Al-Najjar, and Alvaro Sandroni

Full-text: Open access


We study the merging and the testing of opinions in the context of a prediction model. In the absence of incentive problems, opinions can be tested and rejected, regardless of whether or not data produces consensus among Bayesian agents. In contrast, in the presence of incentive problems, opinions can only be tested and rejected when data produces consensus among Bayesian agents. These results show a strong connection between the testing and the merging of opinions. They also relate the literature on Bayesian learning and the literature on testing strategic experts.

Article information

Ann. Statist., Volume 42, Number 3 (2014), 1003-1028.

First available in Project Euclid: 20 May 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62A01: Foundations and philosophical topics
Secondary: 91A40: Game-theoretic models

Test manipulation Bayesian learning


Pomatto, Luciano; Al-Najjar, Nabil; Sandroni, Alvaro. Merging and testing opinions. Ann. Statist. 42 (2014), no. 3, 1003--1028. doi:10.1214/14-AOS1212.

Export citation


  • Al-Najjar, N., Pomatto, L. and Sandroni, A. (2013). An economic model of induction. Mimeo.
  • Al-Najjar, N. I. and Weinstein, J. (2008). Comparative testing of experts. Econometrica 76 541–559.
  • Al-Najjar, N. I., Sandroni, A., Smorodinsky, R. and Weinstein, J. (2010). Testing theories with learnable and predictive representations. J. Econom. Theory 145 2203–2217.
  • Aliprantis, C. D. and Border, K. C. (2006). Infinite Dimensional Analysis: A Hitchhiker’s Guide, 3rd ed. Springer, Berlin.
  • Babaioff, M., Blumrosen, L., Lambert, N. and Reingold, O. (2011). Only valuable experts can be valued. In Proceedings of the 12th ACM Conference on Electronic Commerce 221–222. ACM.
  • Berti, P., Regazzini, E. and Rigo, P. (1998). Well-calibrated, coherent forecasting systems. Theory Probab. Appl. 42 82–102.
  • Berti, P. and Rigo, P. (2002). On coherent conditional probabilities and disintegrations. Ann. Math. Artif. Intell. 35 71–82.
  • Berti, P. and Rigo, P. (2006). Finitely additive uniform limit theorems. Sankhyā 68 24–44.
  • Bhaskara Rao, K. P. S. and Bhaskara Rao, M. (1983). Theory of Charges: A Study of Finitely Additive Measures. Pure and Applied Mathematics 109. Academic Press, New York.
  • Blackwell, D. and Dubins, L. (1962). Merging of opinions with increasing information. Ann. Math. Statist. 33 882–886.
  • Ceder, J. (1966). On maximally Borel resolvable spaces. Rev. Roumaine Math. Pures Appl. 11 89–94.
  • Cesa-Bianchi, N. and Lugosi, G. (2006). Prediction, Learning, and Games. Cambridge Univ. Press, Cambridge.
  • D’Aristotile, A., Diaconis, P. and Freedman, D. (1988). On merging of probabilities. Sankhyā Ser. A 50 363–380.
  • Dawid, A. P. (1985). Calibration-based empirical probability. Ann. Statist. 13 1251–1285.
  • de Finetti, B. (1990). Theory of Probability. Wiley, Chicester.
  • Dekel, E. and Feinberg, Y. (2006). Non-Bayesian testing of a stochastic prediction. Rev. Econom. Stud. 73 893–906.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates. Ann. Statist. 14 1–26.
  • Fan, K. (1953). Minimax theorems. Proc. Natl. Acad. Sci. USA 39 42–47.
  • Feinberg, Y. and Lambert, N. (2011). Mostly calibrated. Mimeo.
  • Feinberg, Y. and Stewart, C. (2008). Testing multiple forecasters. Econometrica 76 561–582.
  • Fortnow, L. and Vohra, R. V. (2009). The complexity of forecast testing. Econometrica 77 93–105.
  • Foster, D. P. and Vohra, R. (2011). Calibration: Respice, adspice, prospice. Mimeo.
  • Foster, D. P. and Vohra, R. V. (1998). Asymptotic calibration. Biometrika 85 379–390.
  • Foster, D. and Young, P. (2001). On the impossibility of predicting the behavior of rational agents. Proc. Natl. Acad. Sci. USA 98 12848–12853.
  • Foster, D. P. and Young, H. P. (2003). Learning, hypothesis testing, and Nash equilibrium. Games Econom. Behav. 45 73–96.
  • Fudenberg, D. and Kreps, D. M. (1993). Learning mixed equilibria. Games Econom. Behav. 5 320–367.
  • Fudenberg, D. and Levine, D. K. (1998). The Theory of Learning in Games. MIT Press Series on Economic Learning and Social Evolution 2. MIT Press, Cambridge, MA.
  • Fudenberg, D. and Levine, D. K. (1999). An easier way to calibrate. Games Econom. Behav. 29 131–137.
  • Fudenberg, D. and Levine, D. K. (2009). Learning and equilibrium. Annu. Rev. Econ. 1 385–420.
  • Gradwohl, R. and Salant, Y. (2011). How to buy advice. Mimeo.
  • Gradwohl, R. and Shmaya, E. (2013). Tractable falsification. Mimeo.
  • Hart, S. and Mas-Colell, A. (2013). Simple Adaptive Strategies: From Regret-Matching to Uncoupled Dynamics. World Scientific, Singapore.
  • Hu, T. W. and Shmaya, E. (2013). Expressible inspections. Theor. Econ. 8 263–280.
  • Jackson, M. O., Kalai, E. and Smorodinsky, R. (1999). Bayesian representation of stochastic processes under learning: de Finetti revisited. Econometrica 67 875–893.
  • Kalai, E. and Lehrer, E. (1993a). Rational learning leads to Nash equilibrium. Econometrica 61 1019–1045.
  • Kalai, E. and Lehrer, E. (1993b). Subjective equilibrium in repeated games. Econometrica 61 1231–1240.
  • Lehrer, E. (2001). Any inspection is manipulable. Econometrica 69 1333–1347.
  • Lehrer, E. and Smorodinsky, R. (1996a). Compatible measures and merging. Math. Oper. Res. 21 697–706.
  • Lehrer, E. and Smorodinsky, R. (1996b). Merging and learning. In Statistics, Probability and Game Theory. Institute of Mathematical Statistics Lecture Notes—Monograph Series 30 147–168. IMS, Hayward, CA.
  • Lipecki, Z. (2001). Cardinality of the set of extreme extensions of a quasi-measure. Manuscripta Math. 104 333–341.
  • Lipecki, Z. (2007). On compactness and extreme points of some sets of quasi-measures and measures. IV. Manuscripta Math. 123 133–146.
  • Monderer, D., Samet, D. and Sela, A. (1997). Belief affirming in learning processes. J. Econom. Theory 73 438–452.
  • Nachbar, J. H. (1997). Prediction, optimization, and learning in repeated games. Econometrica 65 275–309.
  • Nachbar, J. H. (2001). Bayesian learning in repeated games of incomplete information. Soc. Choice Welf. 18 303–326.
  • Nachbar, J. H. (2005). Beliefs in repeated games. Econometrica 73 459–480.
  • Olszewski, W. (2011). Calibration and expert testing. Mimeo.
  • Olszewski, W. and Peski, M. (2011). The principal agent approach to testing-experts. American Economic Journal: Microeconomics 3 89–113.
  • Olszewski, W. and Sandroni, A. (2007). Contracts and uncertainty. Theor. Econ. 2 1–13.
  • Olszewski, W. and Sandroni, A. (2008). Manipulability of future-independent tests. Econometrica 76 1437–1466.
  • Olszewski, W. and Sandroni, A. (2009a). A nonmanipulable test. Ann. Statist. 37 1013–1039.
  • Olszewski, W. and Sandroni, A. (2009b). Strategic manipulation of empirical tests. Math. Oper. Res. 34 57–70.
  • Olszewski, W. and Sandroni, A. (2011). Falsifiability. American Economic Review 101 788–818.
  • Plachky, D. (1976). Extremal and monogenic additive set functions. Proc. Amer. Math. Soc. 54 193–196.
  • Regazzini, E. (1985). Finitely additive conditional probabilities. Rend. Sem. Mat. Fis. Milano 55 69–89.
  • Regazzini, E. (1987). de Finetti’s coherence and statistical inference. Ann. Statist. 15 845–864.
  • Sandroni, A. (1998). Necessary and sufficient conditions for convergence to Nash equilibrium: The almost absolute continuity hypothesis. Games Econom. Behav. 22 121–147.
  • Sandroni, A. (2003). The reproducible properties of correct forecasts. Internat. J. Game Theory 32 151–159.
  • Sandroni, A., Smorodinsky, R. and Vohra, R. V. (2003). Calibration with many checking rules. Math. Oper. Res. 28 141–153.
  • Savage, L. J. (1954). The Foundations of Statistics. Wiley, New York.
  • Schervish, M. and Seidenfeld, T. (1990). An approach to consensus and certainty with increasing evidence. J. Statist. Plann. Inference 25 401–414.
  • Shmaya, E. (2008). Many inspections are manipulable. Theor. Econ. 3 367–382.
  • Sobczyk, A. and Hammer, P. C. (1944). A decomposition of additive set functions. Duke Math. J. 11 839–846.
  • Stewart, C. (2011). Nonmanipulable Bayesian testing. J. Econom. Theory 146 2029–2041.
  • Vovk, V. and Shafer, G. (2005). Good randomized sequential probability forecasting is always possible. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 747–763.
  • Walker, S. G., Lijoi, A. and Prünster, I. (2005). Data tracking and the understanding of Bayesian consistency. Biometrika 92 765–778.
  • Young, H. P. (2002). On the limits to rational learning. European Economic Review 46 791–799.
  • Young, P. (2004). Strategic Learning and Its Limits. Oxford Univ. Press, Oxford, UK.