Holm's (1979) step-down and Hochberg's (1988)
step-up procedures for tests of multiple hypotheses are simple to apply and
are widely used. Holm's procedure controls the familywise error rate
(FWE), while Hochberg's is more powerful. This paper investigates a
step-down procedure (labelled CS) of Seneta & Chen (1997) which is a
sharpening of Holm's, takes into account the degree of association
between test statistics, and also controls the FWE. Computation for
the CS procedure may be minimized by using the procedure as an
adjustment to Holm's. The computational steps are detailed, and the
adjustment is then illustrated by an application to a text-book example of
multiple comparisons, in which step-wise procedures are shown to
perform better than the usual Tukey T-comparison. Simulation
investigations in a standard comparison with a control setting show
that the CS step-down procedure is more powerful than Hochberg's
step-up procedure and the procedure of Simes (1986), especially in
regard to error rate, and not much less powerful than an optimal, but
very specific, step-up procedure of Dunnett & Tamhane (1992).
References
[1] Chen, J.T. & Seneta, E. (1999). A stepwise rejective test procedure with strong control of familywise error rate. Bulletin of the International Statistical Institute, ISI 99, 52nd Session. Contributed papers. Tome LVII, Three Books. Book 3. Helsinki, 1999, pp. 241-242.
[2] Devore, J.L. (2000). Probability and Statistics for Engineering and the Sciences. Pacific Grove, CA: Duxbury.
[3] Dunnett, C.W. & Tamhane, A.C. (1992). A step-up multiple test procedure. J. Amer. Statist. Assoc., 87, 162-170.
[4] Hattan, D.G. & Eacho, P.I. (1978). Relationship of ethanol blood level to REM and non-REM sleep time and distribution in the rat. Life Sciences, 22, 839-846.
[5] Hayter, A.J. (1984). A proof of the conjecture that the Tukey-Kramer multiple comparisons procedure is conservative. Annals of Statistics, 12, 61-75.
[6] Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800-802.
[7] Hochberg, Y. & Rom, D. (1995). Extensions of multiple testing procedures based on Simes' test. J. Statist. Plan. Inf., 48, 141-152.
[8] Hochberg, Y. & Tamhane, A.C. (1987). Multiple Comparison Procedures. New York: Wiley.
[9] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65-70.
[10] Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika, 75, 383-386.
[11] Hommel, G. (1989). A comparison of two modified Bonferroni procedures. Biometrika, 76, 624-625.
[12] Hunter, D. (1976). An upper bound for the probability of a union. Journal of Applied Probability, 13, 597-603.
[13] Kounias, E.G. (1968). Bounds for the probability of a union of events, with applications. Ann. Math. Statist., 39, 2154-2158.
[14] Margaritescu, E. (1986). A note on Bonferroni's inequalities. Biometrical Journal, 28, 937-943.
[15] Miller, R.G. (1981). Simultaneous Statistical Inference. 2nd Edition. New York: Springer.
[16] Miller, R.G. (1985). Multiple comparisons. In Encyclopedia of Statistical Sciences, Eds. S. Kotz and N.L. Johnson, 5, pp. 679-689. New York: Wiley.
[17] Miller, R.G. (1997). Beyond ANOVA, Basics of Applied Statistics. London: Chapman and Hall.
[18] Rom, D.M. (1990). A sequentially rejective test procedure based on a modified Bonferroni inequality. Biometrika, 77, 663-666.
[19] Samuel-Cahn, E. (1996). Is Simes improved Bonferroni procedure conservative? Biometrika, 83, 929-933.
[20] Sarkar, S.K. (1998). Some probability inequalities for ordered MTP_2 random variables: a proof of the Simes conjecture. Ann. Statist., 26, 494-504.
[21] Sarkar, S.K. & Chang, C.K. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc., 92, 1601-1608.
[22] Seneta, E. (1988). Degree, iteration and permutation in improving Bonferroni-type bounds. Australian J. Statist., 30A, 27-38.
[23] Seneta, E. (1993). Probability Inequalities and Dunnett's Test. In Multiple Comparisons, Selection, and Applications in Biometry, Ed. F.M. Hoppe, Chapter 3, pp 29-45. New York: Marcel Dekker.
[24] Seneta, E. & Chen, T. (1997). A sequentially rejective test procedure. Theory of Stochastic Processes, 3(19), No. 3-4, 393-402. Kyiv, Ukraine: TBIMC Scientific Publishers.
[25] Shaffer, J.P. (1986). Modified sequentially rejective multiple test procedure. J. Amer. Statist. Assoc., 81, 826-831.
[26] Simes, R.J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika, 73, 751-754.
[27] Stoline, M.R. (1983). The Hunter method of simultaneous inference and its recommended use for applications having large known correlation structures. J. Amer. Statist. Assoc., 78, 366-370.
[28] Wright, S.P. (1992). Adjusted P-Values for simultaneous inference. Biometrics, 48, 1005-1013.
