Decision theory results for one-sided multiple comparison procedures

Abstract

A resurgence of interest in multiple hypothesis testing has occurred in the last decade. Motivated by studies in genomics, microarrays, DNA sequencing, drug screening, clinical trials, bioassays, education and psychology, statisticians have been devoting considerable research energy in an effort to properly analyze multiple endpoint data. In response to new applications, new criteria and new methodology, many ad hoc procedures have emerged. The classical requirement has been to use procedures which control the strong familywise error rate (FWE) at some predetermined level α. That is, the probability of any false rejection of a true null hypothesis should be less than or equal to α. Finding desirable and powerful multiple test procedures is difficult under this requirement.

One of the more recent ideas is concerned with controlling the false discovery rate (FDR), that is, the expected proportion of rejected hypotheses which are, in fact, true. Many multiple test procedures do control the FDR.

A much earlier approach to multiple testing was formulated by Lehmann [Ann. Math. Statist. 23 (1952) 541–552 and 28 (1957) 1–25]. Lehmann’s approach is decision theoretic and he treats the multiple endpoints problem as a 2^k finite action problem when there are k endpoints. This approach is appealing since unlike the FWE and FDR criteria, the finite action approach pays attention to false acceptances as well as false rejections. In this paper we view the multiple endpoints problem as a 2^k finite action problem. We study the popular procedures single-step, step-down and step-up from the point of view of admissibility, Bayes and limit of Bayes properties. For our model, which is a prototypical one, and our loss function, we are able to demonstrate the following results under some fairly general conditions to be specified:

(i) The single-step procedure is admissible.

(ii) A sequence of prior distributions is given for which the step-down procedure is a limit of a sequence of Bayes procedures.

(iii) For a vector risk function, where each component is the risk for an individual testing problem, various admissibility and inadmissibility results are obtained.

In a companion paper [Cohen and Sackrowitz, Ann. Statist. 33 (2005) 145–158], we are able to give a characterization of Bayes procedures and their limits. The characterization yields a complete class and the additional useful result that the step-up procedure is inadmissible. The inadmissibility of step-up is demonstrated there for a more stringent loss function. Additional decision theoretic type results are also obtained in this paper.