We review a substantial literature, spanning 50 years, concerning the resolution of conflicts using Bayesian heavy-tailed models. Conflicts arise when different sources of information about the model parameters (e.g., prior information, or the information in individual observations) suggest quite different plausible regions for those parameters. Traditional Bayesian models based on normal distributions or other conjugate structures typically resolve conflicts by centring the posterior at some compromise position, but this is not a realistic resolution when it means that the posterior is then in conflict with the different information sources. Bayesian modelling with heavy-tailed distributions has been shown to produce more reasonable conflict resolution, typically by favouring one source of information over the other. The less favoured source is ultimately wholly or partially rejected as the conflict becomes increasingly extreme.
The literature reviewed here provides formal proofs of conflict resolution by asymptotic rejection of some information sources. Results are given for a variety of models, from the simplest case of a single observation relating to a single location parameter up to models with many location parameters, location and scale parameters, or other kinds of parameters. However, these results do not begin to address models of the kind of complexity that are routinely used in practical Bayesian modelling. In addition to reviewing the available theory, we also identify clearly the gaps in the literature that need to be filled in order for modellers to be able to develop applications with appropriate “built-in robustness.”
Andrade, J. A. A. and O’Hagan, A. (2006). Bayesian robustness modeling using regularly varying distributions. Bayesian Analysis 1, 169–188.
Andrade, J. A. A. and O’Hagan, A. (2011). Bayesian robustness modelling of location and scale parameters. Scandinavian Journal of Statistics 38, 691–711.
Angers, J.-F. (2000). P-credence and outliers. Metron 58, 81–108.
Angers, J.-F. and Berger, J. O. (1991). Robust hierarchical Bayes estimation of exchangeable means. Canadian Journal of Statistics 19, 39–56.
Barnett, V. and Lewis, T. (1994). Outliers in Statistical Data, 3rd ed. Chichester: Wiley.
Carlin, B. P. and Polson, N. G. (1991). Inference for nonconjugate Bayesian models using the Gibbs sampler. Canadian Journal of Statistics 19, 399–405.
Choy, S. T. B. and Smith, A. F. M. (1997). On robust analysis of a normal location parameter. Journal of the Royal Statistical Society, Ser. B 59, 463–474.
Choy, S. T. B. and Walker, S. G. (2003). The extended exponential power distribution and Bayesian robustness. Statistics and Probability Letters 65, 227–232.
Dawid, A. P. (1973). Posterior expectations for large observations. Biometrika 60, 664–667.
de Finetti, B. (1961). The Bayesian approach to the rejection of outliers. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability I, 199–210. Berkeley, CA: California Univ. Press.
Desgagné, A. and Angers, J.-F. (2007). Conflicting information and location parameter inference. Metron 65, 67–97.
Fan, T. H. and Berger, J. O. (1992). Behaviour of the posterior distribution and inferences for a normal mean with $t$ prior distributions. Statistics & Decisions 10, 99–120.
Fernández, C., Osiewalski, J. and Steel, M. F. J. (1995). Modelling and inference with $v$-spherical distributions. Journal of the American Statistical Association 90, 1331–1340.
Fúquene, J. A., Cook. J. D. and Pericchi, L. R. (2009). A case for robust Bayesian priors with applications to clinical trials. Bayesian Analysis 4, 817–846.
Goldstein, M. (1983). Outlier resistant distributions: Where does the probability go? Journal of the Royal Statistical Society, Ser. B 45, 355–357.
Haro-López, R. A. and Smith, A. F. M. (1999). On robust Bayesian analysis for location and scale parameters. Journal of Multivariate Analysis 70, 30–56.
Hill, B. M. (1975). On coherence, inadmissibility and inference about many parameters in the theory of least squares. In Studies in Bayesian Econometrics and Statistics: In Honor of Leonard J. Savage (S. E. Fienberg and A. Zellner, eds.) 555–584. Amsterdam: North-Holland.
Le, H. and O’Hagan, A. (1998). A class of bivariate heavy-tailed distributions. Sankhya, Ser. B 60, 82–100.
Lindley, D. V. (1968). The choice of variables in multiple regression (with discussion). Journal of the Royal Statistical Society, Ser. B 30, 31–66.
Meeden, G. and Isaacson, D. (1977). Approximate behavior of the posterior distribution for a large observation. The Annals of Statistics 5, 899–908.
Meinhold, R. J. and Singpurwalla, N. D. (1989). Robustification of Kalman filter models. Journal of the American Statistical Association 84, 479–486.
Mitchell, A. F. S. (1994). A note on posterior moments for a normal mean with double-exponential prior. Journal of the Royal Statistical Society, Ser. B 56, 605–610.
O’Hagan, A. (1979). On outlier rejection phenomena in Bayes inference. Journal of the Royal Statistical Society, Ser. B 41, 358–367.
O’Hagan, A. (1981). A moment of indecision. Biometrika 68, 329–330.
O’Hagan, A. (1988). Modelling with heavy tails. In Bayesian Statistics 3 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.), 345–359. Oxford: Clarendon Press.
O’Hagan, A. (1990). Outliers and credence for location parameter inference. Journal of the American Statistical Association 85, 172–176.
O’Hagan, A. and Le, H. (1994). Conflicting information and a class of bivariate heavy-tailed distributions. In Aspects of Uncertainty: A Tribute to D. V. Lindley (P. R. Freeman and A. F. M. Smith, eds.) 311–327. New York: Wiley.
Pericchi, L. R. and Sansó, B. (1995). A note on bounded influence in Bayesian analysis. Biometrika 82, 223–225.
Pericchi, L. R., Sansó, B. and Smith, A. F. M. (1993). Posterior cumulant relationships in Bayesian inference involving the exponential family. Journal of the American Statistical Association 88, 1419–1426.
Pericchi, L. R. and Smith, A. F. M. (1992). Exact and approximate posterior moments for a normal location parameter. Journal of the Royal Statistical Society, Ser. B 54, 793–804.
Wakefield, J. C., Smith, A. F. M., Racine-Poon, A. and Gelfand, A. E. (1994). Bayesian analysis of linear and non-linear population models by using the Gibbs sampler. Applied Statistics 43, 201–221.
West, M. (1981). Robust sequential approximate Bayesian estimation. Journal of the Royal Statistical Society, Ser. B 43, 157–166.
West, M. (1984). Outlier models and prior distributions in Bayesian linear regression. Journal of the Royal Statistical Society, Ser. B 46, 431–439.