Institute of Mathematical Statistics Lecture Notes - Monograph Series

Bayesian Decision Theory for Multiple Comparisons

Charles Lewis and Dorothy T. Thayer

Full-text: Open access


Applying a decision theoretic approach to multiple comparisons very similar to that described by Lehmann [Ann. Math. Statist. 21 (1950) 1–26; Ann. Math. Statist. 28 (1975a) 1–25; Ann. Math. Statist. 28 (1975b) 547–572], we introduce a loss function based on the concept of the false discovery rate (FDR). We derive a Bayes rule for this loss function and show that it is very closely related to a Bayesian version of the original multiple comparisons procedure proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] to control the sampling theory FDR. We provide the results of a Monte Carlo simulation that illustrates the very similar sampling behavior of our Bayes rule and Benjamini and Hochberg’s procedure when applied to making all pair-wise comparisons in a one-way fixed effects analysis of variance setup with 10 and with 20 means.

Chapter information

Javier Rojo, ed., Optimality: The Third Erich L. Lehmann Symposium (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2009), 326-332

First available in Project Euclid: 3 August 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons 62C10: Bayesian problems; characterization of Bayes procedures 62F15: Bayesian inference

Bayesian decision theory loss function multiple comparisons false discovery rate

Copyright © 2009, Institute of Mathematical Statistics


Rojo, Javier. Bayesian Decision Theory for Multiple Comparisons. Optimality, 326--332, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2009. doi:10.1214/09-LNMS5719.

Export citation


  • [1] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
  • [2] Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Analysis. Addison Wesley, Reading, MA.
  • [3] Duncan, D. B. (1965). A Bayesian approach to multiple comparisons. Technometrics 7 171–222.
  • [4] Jones, L. V. and Tukey, J. W. (2000). A sensible formulation of the significance test. Psychological Methods 5 411–414.
  • [5] Lehmann, E. L. (1950). Some principles of the theory of testing hypotheses. Ann. Math. Statist. 21 1–26.
  • [6] Lehmann, E. L. (1957a). A theory of some multiple decision problems. I. Ann. Math. Statist. 28 1–25.
  • [7] Lehmann, E. L. (1957b). A theory of some multiple decision problems. II. Ann. Math. Statist. 28 547–572.
  • [8] Lewis, C. and Thayer, D. T. (2004). A loss function related to the FDR for random effects multiple comparisons. J. Statist. Plann. Inference 125 49–58.
  • [9] Sakar, S. K. and Zhou, T. (2008). Controlling Bayes directional false discovery rate in random effects model. J. Statist. Plann. Inference 138 682–693.
  • [10] Shaffer, J. P. (1999). A semi-Bayesian study of Duncan’s Bayesian multiple comparison procedure. J. Statist. Plann. Inference 82 197–213.
  • [11] Shaffer, J. P. (2002). Multiplicity, directional (Type III) errors and the null hypothesis. Psychological Methods 7 356–369.
  • [12] Waller, R. A. and Duncan, D. B. (1969). A Bayes rule for symmetric multiple comparisons problems. J. Amer. Statist. Assoc. 64 1484–1503.
  • [13] Williams, V. S. L., Jones, L. V. and Tukey, J. W. (1999). Controlling error in multiple comparisons, with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics 24 42–69.