Multiple comparisons with the best ROC curve

Jason C. Hsu, Peihua Qiu, Lin Yee Hin, Donald O. Mutti, Karla Zadnik

Source: Y. Benjamini, F. Bretz and S. Sarkar, eds., Recent Developments in Multiple Comparison Procedures (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2004), 65-75.

Abstract

The accuracy of a medical diagnostic tool depends on its specificity, the probability that it classifies a normal person as normal, and its sensitivity, the probability that it classifies a diseased person as diseased. The receiver operating characteristic (ROC) curve of such a tool is its sensitivity plotted against (1$-$specificity) as the threshold defining "normal" versus "diseased" ranges over all possible values. A common, global measure of the accuracy of a diagnostic tool is the area under the curve (AUC), the curve being the ROC curve. Thus, one way to compare the accuracies of medical diagnostic tools is to compare their AUCs. By comparing each diagnostic tool with the truly most accurate diagnostic tool, one can eliminate diagnostic tools that are not the most accurate, and discover diagnostic tools which are either the most accurate or practically the most accurate. This article shows how the method of multiple comparison with the best (MCB) for normal error general linear models can be adapted to compare diagnostic tools in terms of AUCs of their ROC curves. MCB of AUCs of ROC curves is illustrated by comparing diagnostic variables for predicting the need for emergency Cesarean section, and for predicting the onset of juvenile myopia.