Head CT deep learning model is highly accurate for early infarct estimation

Diagnostic test accuracy of externally validated convolutional neural network (CNN) artificial intelligence (AI) models for emergency head CT scans - A systematic review

BACKGROUND

The surge in emergency head CT imaging and artificial intelligence (AI) advancements, especially deep learning (DL) and convolutional neural networks (CNN), have accelerated the development of computer-aided diagnosis (CADx) for emergency imaging. External validation assesses model generalizability, providing preliminary evidence of clinical potential.

OBJECTIVES

This study systematically reviews externally validated CNN-CADx models for emergency head CT scans, critically appraises diagnostic test accuracy (DTA), and assesses adherence to reporting guidelines.

METHODS

Studies comparing CNN-CADx model performance to reference standard were eligible. The review was registered in PROSPERO (CRD42023411641) and conducted on Medline, Embase, EBM-Reviews and Web of Science following PRISMA-DTA guideline. DTA reporting were systematically extracted and appraised using standardised checklists (STARD, CHARMS, CLAIM, TRIPOD, PROBAST, QUADAS-2).

RESULTS

Six of 5636 identified studies were eligible. The common target condition was intracranial haemorrhage (ICH), and intended workflow roles auxiliary to experts. Due to methodological and clinical between-study variation, meta-analysis was inappropriate. The scan-level sensitivity exceeded 90 % in 5/6 studies, while specificities ranged from 58,0-97,7 %. The SROC 95 % predictive region was markedly broader than the confidence region, ranging above 50 % sensitivity and 20 % specificity. All studies had unclear or high risk of bias and concern for applicability (QUADAS-2, PROBAST), and reporting adherence was below 50 % in 20 of 32 TRIPOD items.

CONCLUSION

0.01 % of identified studies met the eligibility criteria. The evidence on the DTA of CNN-CADx models for emergency head CT scans remains limited in the scope of this review, as the reviewed studies were scarce, inapt for meta-analysis and undermined by inadequate methodological conduct and reporting. Properly conducted, external validation remains preliminary for evaluating the clinical potential of AI-CADx models, but prospective and pragmatic clinical validation in comparative trials remains most crucial. In conclusion, future AI-CADx research processes should be methodologically standardized and reported in a clinically meaningful way to avoid research waste.

Deep learning on pre-procedural computed tomography and clinical data predicts outcome following stroke thrombectomy

BACKGROUND

Deep learning using clinical and imaging data may improve pre-treatment prognostication in ischemic stroke patients undergoing endovascular thrombectomy (EVT).

METHODS

Deep learning models were trained and tested on baseline clinical and imaging (CT head and CT angiography) data to predict 3-month functional outcomes in stroke patients who underwent EVT. Classical machine learning models (logistic regression and random forest classifiers) were constructed to compare their performance with the deep learning models. An external validation dataset was used to validate the models. The MR PREDICTS prognostic tool was tested on the external validation set, and its performance was compared with the deep learning and classical machine learning models.

RESULTS

A total of 975 patients (550 men; mean±SD age 67.5±15.1 years) were studied with 778 patients in the model development cohort and 197 in the external validation cohort. The deep learning model trained on baseline CT and clinical data, and the logistic regression model (clinical data alone) demonstrated the strongest discriminative abilities for 3-month functional outcome and were comparable (AUC 0.811 vs 0.817, Q=0.82). Both models exhibited superior prognostic performance than the other deep learning (CT head alone, CT head, and CT angiography) and MR PREDICTS models (all Q<0.05).

CONCLUSIONS

The discriminative performance of deep learning for predicting functional independence was comparable to logistic regression. Future studies should focus on whether incorporating procedural and post-procedural data significantly improves model performance.

Random expert sampling for deep learning segmentation of acute ischemic stroke on non-contrast CT

BACKGROUND

Outlining acutely infarcted tissue on non-contrast CT is a challenging task for which human inter-reader agreement is limited. We explored two different methods for training a supervised deep learning algorithm: one that used a segmentation defined by majority vote among experts and another that trained randomly on separate individual expert segmentations.

METHODS

The data set consisted of 260 non-contrast CT studies in 233 patients with acute ischemic stroke recruited from the multicenter DEFUSE 3 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke 3) trial. Additional external validation was performed using 33 patients with matched stroke onset times from the University Hospital Lausanne. A benchmark U-Net was trained on the reference annotations of three experienced neuroradiologists to segment ischemic brain tissue using majority vote and random expert sampling training schemes. The median of volume, overlap, and distance segmentation metrics were determined for agreement in lesion segmentations between (1) three experts, (2) the majority model and each expert, and (3) the random model and each expert. The two sided Wilcoxon signed rank test was used to compare performances (1) to 2) and (1) to (3). We further compared volumes with the 24 hour follow-up diffusion weighted imaging (DWI, final infarct core) and correlations with clinical outcome (modified Rankin Scale (mRS) at 90 days) with the Spearman method.

RESULTS

The random model outperformed the inter-expert agreement ((1) to (2)) and the majority model ((1) to (3)) (dice 0.51±0.04 vs 0.36±0.05 (P<0.0001) vs 0.45±0.05 (P<0.0001)). The random model predicted volume correlated with clinical outcome (0.19, P<0.05), whereas the median expert volume and majority model volume did not. There was no significant difference when comparing the volume correlations between random model, median expert volume, and majority model to 24 hour follow-up DWI volume (P>0.05, n=51).

CONCLUSION

The random model for ischemic injury delineation on non-contrast CT surpassed the inter-expert agreement ((1) to (2)) and the performance of the majority model ((1) to (3)). We showed that the random model volumetric measures of the model were consistent with 24 hour follow-up DWI.

Priorities for Advancements in Neuroimaging in the Diagnostic Workup of Acute Stroke

STAIR XII (12th Stroke Treatment Academy Industry Roundtable) included a workshop to discuss the priorities for advancements in neuroimaging in the diagnostic workup of acute ischemic stroke. The workshop brought together representatives from academia, industry, and government. The participants identified 10 critical areas of priority for the advancement of acute stroke imaging. These include enhancing imaging capabilities at primary and comprehensive stroke centers, refining the analysis and characterization of clots, establishing imaging criteria that can predict the response to reperfusion, optimizing the Thrombolysis in Cerebral Infarction scale, predicting first-pass reperfusion outcomes, improving imaging techniques post-reperfusion therapy, detecting early ischemia on noncontrast computed tomography, enhancing cone beam computed tomography, advancing mobile stroke units, and leveraging high-resolution vessel wall imaging to gain deeper insights into pathology. Imaging in acute ischemic stroke treatment has advanced significantly, but important challenges remain that need to be addressed. A combined effort from academic investigators, industry, and regulators is needed to improve imaging technologies and, ultimately, patient outcomes.

Incorporating algorithmic uncertainty into a clinical machine deep learning algorithm for urgent head CTs

Machine learning (ML) algorithms to detect critical findings on head CTs may expedite patient management. Most ML algorithms for diagnostic imaging analysis utilize dichotomous classifications to determine whether a specific abnormality is present. However, imaging findings may be indeterminate, and algorithmic inferences may have substantial uncertainty. We incorporated awareness of uncertainty into an ML algorithm that detects intracranial hemorrhage or other urgent intracranial abnormalities and evaluated prospectively identified, 1000 consecutive noncontrast head CTs assigned to Emergency Department Neuroradiology for interpretation. The algorithm classified the scans into high (IC+) and low (IC-) probabilities for intracranial hemorrhage or other urgent abnormalities. All other cases were designated as No Prediction (NP) by the algorithm. The positive predictive value for IC+ cases (N = 103) was 0.91 (CI: 0.84-0.96), and the negative predictive value for IC- cases (N = 729) was 0.94 (0.91-0.96). Admission, neurosurgical intervention, and 30-day mortality rates for IC+ was 75% (63-84), 35% (24-47), and 10% (4-20), compared to 43% (40-47), 4% (3-6), and 3% (2-5) for IC-. There were 168 NP cases, of which 32% had intracranial hemorrhage or other urgent abnormalities, 31% had artifacts and postoperative changes, and 29% had no abnormalities. An ML algorithm incorporating uncertainty classified most head CTs into clinically relevant groups with high predictive values and may help accelerate the management of patients with intracranial hemorrhage or other urgent intracranial abnormalities.