Diagnostic Accuracy

\(~\)

Supervisor: Tahani Coolen-Maturi

Project research area: Statistics

Background

Assessing the accuracy of diagnostic tests is a fundamental problem in statistics, with important applications in medicine and healthcare. Reliable diagnostic tools are essential for informing decision-making, for example by guiding treatment choices according to a patient’s condition or estimated risk.

Suppose a diagnostic test produces a continuous score \(X\), where larger values indicate stronger evidence for disease. A threshold \(c\) may be used to classify an individual as positive or negative, according to whether \(X>c\). The performance of the test can then be summarised through quantities such as Sensitivity and Specificity,

\[\mbox{Sensitivity}(c)=P(X>c∣\mbox{diseased}),\quad \mbox{Specificity}(c)=P(X≤c∣\mbox{non-diseased}).\]

As the threshold \(c\) varies, there is a trade-off between Sensitivity and Specificity. The receiver operating characteristic (ROC) curve summarises this relationship by plotting Sensitivity against 1−Specificity across all possible values of \(c\). A commonly used summary of diagnostic performance is the area under the ROC curve (AUC), which provides a single measure of how well the test discriminates between groups.

While classical ROC methods focus on two-group settings, many practical applications are more complex. For example, one may wish to combine multiple biomarkers into a single diagnostic measure, account for variability across readers, or compare competing diagnostic procedures. More advanced settings include free-response ROC analysis, convex hull ROC curves, and meta-analysis of diagnostic accuracy studies. In problems involving more than two groups, the ROC curve can be extended to a ROC surface, with an associated summary given by the volume under the surface.

The project will involve both theoretical understanding of these methods and their computational implementation in R.

Learning outcomes

By the end of this project, you will have an understanding of:

  • The statistical principles underlying diagnostic accuracy
  • ROC curves and associated summary measures such as AUC
  • Extensions of ROC methodology to more complex settings
  • The challenges involved in comparing and combining diagnostic procedures

By the end of this project, you will be able to:

  • Construct and interpret ROC curves and related summaries
  • Implement diagnostic accuracy methods using statistical software (e.g. R)
  • Analyse and compare diagnostic procedures
  • Critically assess different approaches to evaluating diagnostic performance

Mode of operation and evidence of learning

The project will involve learning through reading and programming in R. Students will demonstrate their understanding by implementing core methodology, analysing simulated or real data sets, and clearly communicating their findings in both written and oral formats.

Prerequisites

Statistical Inference; Data Science and Statistical Modelling

Further information

For more information, please contact me at: tahani.maturi@durham.ac.uk

Resources

Articles

  • Fawcett, T. An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874, 2006. PDF
  • Kamarudin, A.N., Cox, T. & Kolamunnage-Dona, R. Time-dependent ROC curve analysis in medical research: current methods and applications. BMC Med Res Methodol 17, 53, 2017. PDF
  • Hsu, M., Chang, Y.I. & Hsueh, H. Biomarker selection for medical diagnosis using the partial area under the ROC curve. BMC Res Notes 7, 25, 2014. PDF
  • Mandrekar, J.N. Receiver Operating Characteristic Curve in Diagnostic Test Assessment. Journal of Thoracic Oncology, 5(9), 1315–1316, 2010. PDF
  • Park, S.H., Goo, J.M. & Jo, C.H. Receiver operating characteristic (ROC) curve: practical review for radiologists. Korean Journal of Radiology, 5(1), 11–18, 2004. PDF
  • Gonçalves, L., Subtil, A., Oliveira, M. & Bermudez, P. ROC curve estimation: an overview. REVSTAT Statistical Journal, 12, 1–20, 2014. PDF
  • Pardo-Fernández, J.C., Rodríguez-Álvarez, M.X. & Van Keilegom, I. A review on ROC curves in the presence of covariates. REVSTAT Statistical Journal, 12, 21–41, 2014. PDF
  • Nakas, C. T. Developments in ROC surface analysis and assessment of diagnostic markers in three-class classification problems. REVSTAT Statistical Journal, 12, 43–65, 2014. PDF

Books

  • Nakas, Bantis and Gatsonis. ROC Analysis for Classification and Prediction in Practice. CRC Press, 2023.
  • Zhou, X.H., Obuchowski, N.A. and McClish, D.K. Statistical Methods in Diagnostic Medicine. Wiley, New York, 2002.
  • Pepe, M.S. The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press, Oxford, 2003.
  • Zou, Liu, Bandos, Ohno-Machado, Rockette. Statistical Evaluation of Diagnostic Performance: Topics in ROC Analysis. Chapman and Hall/CRC, 2001.
  • Krzanowski, W.J. and Hand, D.J. ROC Curves for Continuous Data. Chapman and Hall/CRC, 2009.