Institute of Mathematical Statistics Lecture Notes - Monograph Series

Implementing Shaffer’s multiple comparison procedure for a large number of groups

John R. Donoghue

Full-text: Open access


Shaffer's Modified sequentially rejective multiple test procedures presented two more powerful modifications (commonly referred to as S1 and S2) of Holm’s (1979) A simple sequentially rejective multiple test procedure, sequentially rejective Bonferroni procedure. Unfortunately, use of the more-powerful S2 procedure has been severely limited by the complexity of its implementation. This paper presents a method of allowing Shaffer’s S2 procedure to be used in a much larger class of problems.

Theoretical results concerning an aspect of Shaffer’s S2 procedure are derived. Use of these results in implementing the procedure is described. Next, two heuristics are described which greatly enhance the efficiency of the method. Finally, an efficient algorithmic method of implementing Shaffer’s procedure is outlined.

The present work allows Shaffer’s procedure for multiple comparisons to be applied to a large number of groups. Unlike many other methods, no assumptions about the joint distributions of the test statistics need be made. Shaffer’s S2 method and other methods are illustrated by application to two data sets; comparisons of the order of 11 clustering methods, and comparison of the means of 44 states from the 1994 National Assessment of Educational Progress Trial State Assessment of Reading at Grade 4.

Chapter information

Y. Benjamini, F. Bretz and S. Sarkar, eds., Recent Developments in Multiple Comparison Procedures (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2004), 1-23

First available in Project Euclid: 28 November 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J15: Paired and multiple comparisons

Shaffer’s procedure multiple comparisons familywise error rate pairwise comparisons National Assessment of Educational Progress NAEP

Copyright © 2004, Institute of Mathematical Statistics


Donoghue, John R. Implementing Shaffer’s multiple comparison procedure for a large number of groups. Recent Developments in Multiple Comparison Procedures, 1--23, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2004. doi:10.1214/lnms/1196285622.

Export citation