Quality assessment of stroke radiomics studies: Promoting clinical application

Need for Transparency and Clinical Interpretability in Hemorrhagic Stroke Artificial Intelligence Research: Promoting Effective Clinical Application

PURPOSE

This study aimed to evaluate the quality of artificial intelligence (AI)/machine learning (ML) studies on hemorrhagic stroke using the Minimum Information for Medical AI Reporting (MINIMAR) and Minimum Information About Clinical Artificial Intelligence Modeling (MI-CLAIM) frameworks to promote clinical application.

MATERIALS AND METHODS

PubMed, MEDLINE, and Embase were searched for AI/ML studies on hemorrhagic stroke. Out of the 531 articles found, 29 relevant original research articles were included. MINIMAR and MI-CLAIM scores were assigned by two experienced radiologists to assess the quality of the studies.

RESULTS

We analyzed 29 investigations that utilized AI/ML in the field of hemorrhagic stroke, involving a median of 224.5 patients. The majority of studies focused on diagnostic outcomes using computed tomography scans (89.7%) and were published in computer science journals (48.3%). The overall adherence rates to reporting guidelines, as assessed through the MINIMAR and MI-CLAIM frameworks, were 47.6% and 46.0%, respectively. In MINIMAR, none of the studies reported the socioeconomic status of the patients or how missing values had been addressed. In MI-CLAIM, only two studies applied model-examination techniques to improve model interpretability. Transparency and reproducibility were limited, as only 10.3% of the studies had publicly shared their code. Cohen's kappa between the two radiologists was 0.811 and 0.779 for MINIMAR and MI-CLAIM, respectively.

CONCLUSION

The overall reporting quality of published AI/ML studies on hemorrhagic stroke is suboptimal. It is necessary to incorporate model examination techniques for interpretability and promote code openness to enhance transparency and increase the clinical applicability of AI/ML studies.

Radiomics using non-contrast CT to predict hemorrhagic transformation risk in stroke patients undergoing revascularization

OBJECTIVES

This study explores whether textural features from initial non-contrast CT scans of infarcted brain tissue are linked to hemorrhagic transformation susceptibility.

MATERIALS AND METHODS

Stroke patients undergoing thrombolysis or thrombectomy from Jan 2012 to Jan 2022 were analyzed retrospectively. Hemorrhagic transformation was defined using follow-up magnetic resonance imaging. A total of 94 radiomic features were extracted from the infarcted tissue on initial NCCT scans. Patients were divided into training and test sets (7:3 ratio). Two models were developed with fivefold cross-validation: one incorporating first-order and textural radiomic features, and another using only textural radiomic features. A clinical model was also constructed using logistic regression with clinical variables, and test set validation was performed.

RESULTS

Among 362 patients, 218 had hemorrhagic transformations. The LightGBM model with all radiomics features had the best performance, with an area under the receiver operating characteristic curve (AUROC) of 0.986 (95% confidence interval [CI], 0.971-1.000) on the test dataset. The ExtraTrees model performed best when textural features were employed, with an AUROC of 0.845 (95% CI, 0.774-0.916). Minimum, maximum, and ten percentile values were significant predictors of hemorrhagic transformation. The clinical model showed an AUROC of 0.544 (95% CI, 0.431-0.658). The performance of the radiomics models was significantly better than that of the clinical model on the test dataset (p < 0.001).

CONCLUSIONS

The radiomics model can predict hemorrhagic transformation using NCCT in stroke patients. Low Hounsfield unit was a strong predictor of hemorrhagic transformation, while textural features alone can predict hemorrhagic transformation.

CLINICAL RELEVANCE STATEMENT

Using radiomic features extracted from initial non-contrast computed tomography, early prediction of hemorrhagic transformation has the potential to improve patient care and outcomes by aiding in personalized treatment decision-making and early identification of at-risk patients.

KEY POINTS

• Predicting hemorrhagic transformation following thrombolysis in stroke is challenging since multiple factors are associated. • Radiomics features of infarcted tissue on initial non-contrast CT are associated with hemorrhagic transformation. • Textural features on non-contrast CT are associated with the frailty of the infarcted tissue.

Quality Assessment of Radiomics Studies on Functional Outcomes After Acute Ischemic Stroke-A Systematic Review

OBJECTIVE

Radiomics is a machine-learning method that extracts features from medical images. The objective of the present systematic review was to assess the quality of existing studies that use radiomics methods to predict functional outcomes in patients after acute ischemic stroke.

METHODS

Studies using radiomics-extracted features to predict functional outcomes among patients with acute ischemic stroke using the modified Rankin Scale were included. PubMed, Scopus, Web of Science, and Embase were screened using the terms "radiomics" and "texture" in combination with "stroke." Quality scores were calculated based on Radiomics Quality Score, the IBSI (Image Biomarkers Standardization Initiative), and the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2).

RESULTS

Fourteen studies were included. The median total Radiomics Quality Score was 14.5 (13-16) out of 36. Domains 1, 5, and 6 on protocol quality and stability of imaging and segmentation, level of evidence, and use of open science and data, respectively, were poor. Median IBSI score was 2.5 (1-5) out of 6. Few studies included bias-field correction algorithms, isovoxel resampling, skull stripping, or gray-level discretization. Of 14 studies, none received +6 points, 1 received +5 points, 5 received +4 points, 1 study received +3 points, 5 received +2 points, 2 received +1 points, and none received 0 points. As per the QUADAS-2, 6/14 (42.9%) studies had a risk of bias concern and 0/14 (0%) had applicability concern.

CONCLUSIONS

The quality of the included studies was low to moderate. With increasing use of radiomics, future studies should attempt to adhere to and report established radiomics quality guidelines.

Predicting long-term outcomes for acute ischemic stroke using multi-model MRI radiomics and clinical variables

Purpose

The objective of this study was to create and validate a novel prediction model that incorporated both multi-modal radiomics features and multi-clinical features, with the aim of accurately identifying acute ischemic stroke (AIS) patients who faced a higher risk of poor outcomes.

Methods

A cohort of 461 patients diagnosed with AIS from four centers was divided into a training cohort and a validation cohort. Radiomics features were extracted and selected from diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) images to create a radiomic signature. Prediction models were developed using multi-clinical and selected radiomics features from DWI and ADC.

Results

A total of 49 radiomics features were selected from DWI and ADC images by the least absolute shrinkage and selection operator (LASSO). Additionally, 20 variables were collected as multi-clinical features. In terms of predicting poor outcomes in validation set, the area under the curve (AUC) was 0.727 for the DWI radiomics model, 0.821 for the ADC radiomics model, 0.825 for the DWI + ADC radiomics model, and 0.808 for the multi-clinical model. Furthermore, a prediction model was built using all selected features, the AUC for predicting poor outcomes increased to 0.86.

Conclusion

Radiomics features extracted from DWI and ADC images can serve as valuable biomarkers for predicting poor clinical outcomes in patients with AIS. Furthermore, when these radiomics features were combined with multi-clinical features, the predictive performance was enhanced. The prediction model has the potential to provide guidance for tailoring rehabilitation therapies based on individual patient risks for poor outcomes.