Enhancing multiple testing: two applications of the probability of correct selection statistic

Abstract

The calculation of the probability of correct selection (PCS) shows how likely it is that the populations chosen as “best” truly are the top populations, according to a well-defined standard. PCS is useful for the researcher with limited resources or the statistician attempting to test the quality of two different statistics. This paper explores the theory behind two selection goals for PCS, $G$-best and $d$-best, and how they improve previous definitions of PCS for massive datasets. This paper also calculates PCS for two applications that have already been analyzed by multiple testing procedures in the literature. The two applications are in neuroimaging and econometrics. It is shown through these applications that PCS not only supports the multiple testing conclusions but also provides further information about the statistics used.