Electronic Journal of Statistics

Bias–corrected methods for estimating the receiver operating characteristic surface of continuous diagnostic tests

Khanh To Duc, Monica Chiogna, and Gianfranco Adimari

Full-text: Open access


Verification bias is a well-known problem that may occur in the evaluation of predictive ability of diagnostic tests. When a binary disease status is considered, various solutions can be found in the literature to correct inference based on usual measures of test accuracy, such as the receiver operating characteristic (ROC) curve or the area underneath. Evaluation of the predictive ability of continuous diagnostic tests in the presence of verification bias for an ordinal three-class disease status is here discussed. In particular, several verification bias-corrected estimators of the ROC surface and of the volume underneath are proposed. Consistency and asymptotic normality of the proposed estimators are established and their finite sample behavior is investigated by means of Monte Carlo simulation studies. Two illustrations are also given.

Article information

Electron. J. Statist., Volume 10, Number 2 (2016), 3063-3113.

Received: June 2016
First available in Project Euclid: 10 November 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C99: None of the above, but in this section
Secondary: 62P10: Applications to biology and medical sciences

Verification bias missing at random ROC surface analysis true class fractions


To Duc, Khanh; Chiogna, Monica; Adimari, Gianfranco. Bias–corrected methods for estimating the receiver operating characteristic surface of continuous diagnostic tests. Electron. J. Statist. 10 (2016), no. 2, 3063--3113. doi:10.1214/16-EJS1202. https://projecteuclid.org/euclid.ejs/1478747030

Export citation


  • [1] Adimari, G. and Chiogna, M. (2015). Nearest-neighbor estimation for ROC analysis under verification bias., The International Journal of Biostatistics 11 109–124.
  • [2] Alonzo, T. A., Pepe, M. S. and Lumley, T. (2003). Estimating disease prevalence in two-phase studies., Biostatistics 4 313–326.
  • [3] Alonzo, T. A. and Pepe, M. S. (2005). Assessing accuracy of a continuous screening test in the presence of verification bias., Journal of the Royal Statistical Society: Series C (Applied Statistics) 54 173–190.
  • [4] Chi, Y. Y. and Zhou, X. H. (2008). Receiver operating characteristic surfaces in the presence of verification bias., Journal of the Royal Statistical Society: Series C (Applied Statistics) 57 1–23.
  • [5] Daganzo, C. (1979)., Multinomial probit: the theory and its application to demand forecasting. Elsevier.
  • [6] Fluss, R., Reiser, B., Faraggi, D. and Rotnitzky, A. (2009). Estimation of the ROC curve under verification bias., Biometrical Journal 51 475–490.
  • [7] Gu, J., Ghosal, S. and Kleiner, D. E. (2014). Bayesian ROC curve estimation under verification bias., Statistics in medicine 33 5081–5096.
  • [8] Kang, L. and Tian, L. (2013). Estimation of the volume under the ROC surface with three ordinal diagnostic categories., Computational Statistics & Data Analysis 62 39–51.
  • [9] Li, J. and Zhou, X. H. (2009). Nonparametric and semiparametric estimation of the three way receiver operating characteristic surface., Journal of Statistical Planning and Inference 139 4133–4142.
  • [10] Li, J., Zhou, X. H. and Fine, J. P. (2012). A regression approach to ROC surface, with applications to Alzheimer’s disease., Science China Mathematics 55 1583–1595.
  • [11] Little, R. J. and Rubin, D. B. (1987)., Statistical analysis with missing data. John Wiley & Sons.
  • [12] Nakas, C. T. (2014). Developments in ROC surface analysis and assessment of diagnostic markers in three-class classification problems., REVSTAT–Statistical Journal 12 43–65.
  • [13] Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements., Statistics in Medicine 23 3437–3449.
  • [14] Pepe, M. S. (2003)., The statistical evaluation of medical tests for classification and prediction. Oxford University Press.
  • [15] Rotnitzky, A., Faraggi, D. and Schisterman, E. (2006). Doubly robust estimation of the area under the receiver-operating characteristic curve in the presence of verification bias., Journal of the American Statistical Association 101 1276–1288.
  • [16] Scurfield, B. K. (1996). Multiple-event forced-choice tasks in the theory of signal detectability., Journal of Mathematical Psychology 40 253–269.
  • [17] Xiong, C., van Belle, G., Miller, J. P. and Morris, J. C. (2006). Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups., Statistics in Medicine 25 1251–1273.
  • [18] Zhou, X. H., McClish, D. K. and Obuchowski, N. A. (2002)., Statistical methods in diagnostic medicine. John Wiley & Sons.