Subject-centered free-response ROC (FROC) analysis

A Review of Artificial Intelligence in Cerebrovascular Disease Imaging: Applications and Challenges

BACKGROUND

A variety of emerging medical imaging technologies based on artificial intelligence have been widely applied in many diseases, but they are still limitedly used in the cerebrovascular field even though the diseases can lead to catastrophic consequences.

OBJECTIVE

This work aims to discuss the current challenges and future directions of artificial intelligence technology in cerebrovascular diseases through reviewing the existing literature related to applications in terms of computer-aided detection, prediction and treatment of cerebrovascular diseases.

METHODS

Based on artificial intelligence applications in four representative cerebrovascular diseases including intracranial aneurysm, arteriovenous malformation, arteriosclerosis and moyamoya disease, this paper systematically reviews studies published between 2006 and 2021 in five databases: National Center for Biotechnology Information, Elsevier Science Direct, IEEE Xplore Digital Library, Web of Science and Springer Link. And three refinement steps were further conducted after identifying relevant literature from these databases.

RESULTS

For the popular research topic, most of the included publications involved computer-aided detection and prediction of aneurysms, while studies about arteriovenous malformation, arteriosclerosis and moyamoya disease showed an upward trend in recent years. Both conventional machine learning and deep learning algorithms were utilized in these publications, but machine learning techniques accounted for a larger proportion.

CONCLUSION

Algorithms related to artificial intelligence, especially deep learning, are promising tools for medical imaging analysis and will enhance the performance of computer-aided detection, prediction and treatment of cerebrovascular diseases.

Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools

Diagnostic systems designed to detect possibly multiple lesions per patient (e.g. multiple polyps during CT colonoscopy) are often evaluated in “free-response” studies that allow for diagnostic responses unconstrained in their number and locations. Analysis of free-response studies requires extensions of the traditional receiver operating characteristic (ROC) analysis, which are termed free-response ROC (FROC) methodology. Despite substantial developments in this area, FROC tools and approaches are much more cumbersome than traditional ROC methods. Alternative approaches that use well-known ROC tools (e.g. ROI-ROC) require defining and physically delineating regions of interest (ROI) and combine FROC data within ROIs. We propose an approach that allows analyzing FROC data using conventional ROC tools without delineating the actual ROIs or reducing data. The design parameters of FROC study are used to make FROC data analyzable using ROC tools and to calibrate the corresponding FROC and ROC curves on both conceptual and numerical levels. Differences in the performance indices of the nonparametric FROC and the new approach are shown to be asymptotically negligible and typically rather small in practice. Data from a large multi-reader study of colon cancer detection are used to illustrate the new approach.