An updated systematic review of radiomics in osteosarcoma: utilizing CLAIM to adapt the increasing trend of deep learning application in radiomics

Development of the interpretable typing prediction model for osteosarcoma and chondrosarcoma based on machine learning and radiomics: a multicenter retrospective study

Background

Osteosarcoma and chondrosarcoma are common malignant bone tumors, and accurate differentiation between these two tumors is crucial for treatment strategies and prognosis assessment. However, traditional radiological methods face diagnostic challenges due to the similarity in imaging between the two.

Methods

Clinical CT images and pathological data of 76 patients confirmed by pathology from January 2018 to January 2024 were retrospectively collected from Guizhou Medical University Affiliated Hospital and Guizhou Medical University Second Affiliated Hospital. A total of 788 radiomic features, including shape, texture, and first-order statistics, were extracted in this study. Six machine learning models, including Random Forest (RF), Extra Trees (ET), AdaBoost, Gradient Boosting Tree (GB), Linear Discriminant Analysis (LDA), and XGBoost (XGB), were trained and validated. Additionally, the importance of features and the interpretability of the models were evaluated through SHAP value analysis.

Results

The RF model performed best in distinguishing between these two tumor types, with an AUC value close to perfect at 1.00. The ET and AdaBoost models also demonstrated high performance, with AUC values of 0.98 and 0.93, respectively. SHAP value analysis revealed significant influences of wavelet-transformed GLCM and First Order features on model predictions, further enhancing diagnostic interpretability.

Conclusion

This study confirms the effectiveness of combining machine learning with radiomic features in improving the accuracy and interpretability of osteosarcoma and chondrosarcoma diagnosis. The excellent performance of the RF model is particularly suitable for complex imaging data processing, providing valuable insights for the future.

Magnetic Resonance Imaging Texture Analysis Based on Intraosseous and Extraosseous Lesions to Predict Prognosis in Patients with Osteosarcoma

Objectives: To construct an optimal magnetic resonance imaging (MRI) texture model to evaluate histological patterns and predict prognosis in patients with osteosarcoma (OS). Methods: Thirty-four patients underwent pretreatment MRI and were diagnosed as having OS by surgical resection or biopsy between September 2008 and June 2018. Histological patterns and 3-year survival were recorded. Manual segmentation was performed in intraosseous, extraosseous, and entire lesions on T1-weighted, T2-weighted, and contrast-enhanced T1-weighted images to extract texture features and perform principal component analysis. A support vector machine algorithm with 3-fold cross-validation was used to construct and validate the models. The area under the receiver operating characteristic curve (AUC) was calculated to evaluate diagnostic performance in evaluating histological patterns and 3-year survival. Results: Eight patients were chondroblastic and the remaining twenty-six patients were non-chondroblastic patterns. Twenty-seven patients were 3-year survivors, and the remaining seven patients were non-survivors. In discriminating chondroblastic from non-chondroblastic patterns, the model from extraosseous lesions on the T2-weighted images showed the highest diagnostic performance (AUCs of 0.94 and 0.89 in the training and validation sets). The model from intraosseous lesions on the T1-weighted images showed the highest diagnostic performance in discriminating 3-year non-survivors from survivors (AUCs of 0.99 and 0.88 in the training and validation sets) with a sensitivity, specificity, positive predictive value, and negative predictive value of 85.7%, 92.6%, 75.0%, and 96.2%, respectively. Conclusions: The texture models of extraosseous lesions on T2-weighted images can discriminate the chondroblastic pattern from non-chondroblastic patterns, while the texture models of intraosseous lesions on T1-weighted images can discriminate 3-year non-survivors from survivors.

Artificial intelligence in predicting recurrence after first-line treatment of liver cancer: a systematic review and meta-analysis

BACKGROUND

The aim of this study was to conduct a systematic review and meta-analysis to comprehensively evaluate the performance and methodological quality of artificial intelligence (AI) in predicting recurrence after single first-line treatment for liver cancer.

METHODS

A rigorous and systematic evaluation was conducted on the AI studies related to recurrence after single first-line treatment for liver cancer, retrieved from the PubMed, Embase, Web of Science, Cochrane Library, and CNKI databases. The area under the curve (AUC), sensitivity (SENC), and specificity (SPEC) of each study were extracted for meta-analysis.

RESULTS

Six percutaneous ablation (PA) studies, 16 surgical resection (SR) studies, and 5 transarterial chemoembolization (TACE) studies were included in the meta-analysis for predicting recurrence after hepatocellular carcinoma (HCC) treatment, respectively. Four SR studies and 2 PA studies were included in the meta-analysis for recurrence after intrahepatic cholangiocarcinoma (ICC) and colorectal cancer liver metastasis (CRLM) treatment. The pooled SENC, SEPC, and AUC of AI in predicting recurrence after primary HCC treatment via PA, SR, and TACE were 0.78, 0.90, and 0.92; 0.81, 0.77, and 0.86; and 0.73, 0.79, and 0.79, respectively. The values for ICC treated with SR and CRLM treated with PA were 0.85, 0.71, 0.86 and 0.69, 0.63,0.74, respectively.

CONCLUSION

This systematic review and meta-analysis demonstrates the comprehensive application value of AI in predicting recurrence after a single first-line treatment of liver cancer, with satisfactory results, indicating the clinical translation potential of AI in predicting recurrence after liver cancer treatment.

A systematic review of deep learning-based spinal bone lesion detection in medical images

Spinal bone lesions encompass a wide array of pathologies, spanning from benign abnormalities to aggressive malignancies, such as diffusely localized metastases. Early detection and accurate differentiation of the underlying diseases is crucial for every patient's clinical treatment and outcome, with radiological imaging being a core element in the diagnostic pathway. Across numerous pathologies and imaging techniques, deep learning (DL) models are progressively considered a valuable resource in the clinical setting. This review describes not only the diagnostic performance of these models and the differing approaches in the field of spinal bone malignancy recognition, but also the lack of standardized methodology and reporting that we believe is currently hampering this newly founded area of research. In line with their established and reliable role in lesion detection, this publication focuses on both computed tomography and magnetic resonance imaging, as well as various derivative modalities (i.e. SPECT). After conducting a systematic literature search and subsequent analysis for applicability and quality using a modified QUADAS-2 scoring system, we confirmed that most of the 14 identified studies were plagued by major limitations, such as insufficient reporting of model statistics and data acquisition, a lacking external validation dataset, and potentially biased annotation. Although we experienced these limitations, we nonetheless conclude that the potential of these methods shines through in the presented results. These findings underline the need for more stringent quality controls in DL studies, as well as model development to afford increased insight and progress in this promising novel field.

Preoperative prediction of high-grade osteosarcoma response to neoadjuvant therapy based on a plain CT radiomics model: A dual-center study

Objective

To develop a model combining clinical and radiomics features from CT scans for a preoperative noninvasive evaluation of Huvos grading of neoadjuvant chemotherapy in patients with HOS.

Methods

183 patients from center A and 42 from center B were categorized into training and validation sets. Features derived from radiomics were obtained from unenhanced CT scans.Following dimensionality reduction, the most optimal features were selected and utilized in creating a radiomics model through logistic regression analysis. Integrating clinical features, a composite clinical radiomics model was developed, and a nomogram was constructed. Predictive performance of the model was evaluated using ROC curves and calibration curves. Additionally, decision curve analysis was conducted to assess practical utility of nomogram in clinical settings.

Results

LASSO LR analysis was performed, and finally, three selected image omics features were obtained.Radiomics model yielded AUC values with a good diagnostic effect for both patient sets (AUCs: 0.69 and 0.68, respectively). Clinical models (including sex, age, pre-chemotherapy ALP and LDH levels, new lung metastases within 1 year after surgery, and incidence) performed well in terms of Huvos grade prediction, with an AUC of 0.74 for training set. The AUC for independent validation set stood at 0.70. Notably, the amalgamation of radiomics and clinical features exhibited commendable predictive prowess in training set, registering an AUC of 0.78. This robust performance was subsequently validated in the independent validation set, where the AUC remained high at 0.75. Calibration curves of nomogram showed that the predictions were in good agreement with actual observations.

Conclusion

Combined model can be used for Huvos grading in patients with HOS after preoperative chemotherapy, which is helpful for adjuvant treatment decisions.

Self-reporting with checklists in artificial intelligence research on medical imaging: a systematic review based on citations of CLAIM

OBJECTIVE

To evaluate the usage of a well-known and widely adopted checklist, Checklist for Artificial Intelligence in Medical imaging (CLAIM), for self-reporting through a systematic analysis of its citations.

METHODS

Google Scholar, Web of Science, and Scopus were used to search for citations (date, 29 April 2023). CLAIM's use for self-reporting with proof (i.e., filled-out checklist) and other potential use cases were systematically assessed in research papers. Eligible papers were evaluated independently by two readers, with the help of automatic annotation. Item-by-item confirmation analysis on papers with checklist proof was subsequently performed.

RESULTS

A total of 391 unique citations were identified from three databases. Of the 118 papers included in this study, 12 (10%) provided a proof of self-reported CLAIM checklist. More than half (70; 59%) only mentioned some sort of adherence to CLAIM without providing any proof in the form of a checklist. Approximately one-third (36; 31%) cited the CLAIM for reasons unrelated to their reporting or methodological adherence. Overall, the claims on 57 to 93% of the items per publication were confirmed in the item-by-item analysis, with a mean and standard deviation of 81% and 10%, respectively.

CONCLUSION

Only a small proportion of the publications used CLAIM as checklist and supplied filled-out documentation; however, the self-reported checklists may contain errors and should be approached cautiously. We hope that this systematic citation analysis would motivate artificial intelligence community about the importance of proper self-reporting, and encourage researchers, journals, editors, and reviewers to take action to ensure the proper usage of checklists.

CLINICAL RELEVANCE STATEMENT

Only a small percentage of the publications used CLAIM for self-reporting with proof (i.e., filled-out checklist). However, the filled-out checklist proofs may contain errors, e.g., false claims of adherence, and should be approached cautiously. These may indicate inappropriate usage of checklists and necessitate further action by authorities.

KEY POINTS

• Of 118 eligible papers, only 12 (10%) followed the CLAIM checklist for self-reporting with proof (i.e., filled-out checklist). More than half (70; 59%) only mentioned some kind of adherence without providing any proof. • Overall, claims on 57 to 93% of the items were valid in item-by-item confirmation analysis, with a mean and standard deviation of 81% and 10%, respectively. • Even with the checklist proof, the items declared may contain errors and should be approached cautiously.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

Reporting Quality of Research Studies on AI Applications in Medical Images According to the CLAIM Guidelines in a Radiology Journal With a Strong Prominence in Asia

OBJECTIVE

We aimed to evaluate the reporting quality of research articles that applied deep learning to medical imaging. Using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines and a journal with prominence in Asia as a sample, we intended to provide an insight into reporting quality in the Asian region and establish a journal-specific audit.

MATERIALS AND METHODS

A total of 38 articles published in the Korean Journal of Radiology between June 2018 and January 2023 were analyzed. The analysis included calculating the percentage of studies that adhered to each CLAIM item and identifying items that were met by ≤ 50% of the studies. The article review was initially conducted independently by two reviewers, and the consensus results were used for the final analysis. We also compared adherence rates to CLAIM before and after December 2020.

RESULTS

Of the 42 items in the CLAIM guidelines, 12 items (29%) were satisfied by ≤ 50% of the included articles. None of the studies reported handling missing data (item #13). Only one study respectively presented the use of de-identification methods (#12), intended sample size (#19), robustness or sensitivity analysis (#30), and full study protocol (#41). Of the studies, 35% reported the selection of data subsets (#10), 40% reported registration information (#40), and 50% measured inter and intrarater variability (#18). No significant changes were observed in the rates of adherence to these 12 items before and after December 2020.

CONCLUSION

The reporting quality of artificial intelligence studies according to CLAIM guidelines, in our study sample, showed room for improvement. We recommend that the authors and reviewers have a solid understanding of the relevant reporting guidelines and ensure that the essential elements are adequately reported when writing and reviewing the manuscripts for publication.

An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation

OBJECTIVE

To conduct an overview of meta-analyses of radiomics studies assessing their study quality and evidence level.

METHODS

A systematical search was updated via peer-reviewed electronic databases, preprint servers, and systematic review protocol registers until 15 November 2022. Systematic reviews with meta-analysis of primary radiomics studies were included. Their reporting transparency, methodological quality, and risk of bias were assessed by PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 checklist, AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews, version 2) tool, and ROBIS (Risk Of Bias In Systematic reviews) tool, respectively. The evidence level supporting the radiomics for clinical use was rated.

RESULTS

We identified 44 systematic reviews with meta-analyses on radiomics research. The mean ± standard deviation of PRISMA adherence rate was 65 ± 9%. The AMSTAR-2 tool rated 5 and 39 systematic reviews as low and critically low confidence, respectively. The ROBIS assessment resulted low, unclear and high risk in 5, 11, and 28 systematic reviews, respectively. We reperformed 53 meta-analyses in 38 included systematic reviews. There were 3, 7, and 43 meta-analyses rated as convincing, highly suggestive, and weak levels of evidence, respectively. The convincing level of evidence was rated in (1) T2-FLAIR radiomics for IDH-mutant vs IDH-wide type differentiation in low-grade glioma, (2) CT radiomics for COVID-19 vs other viral pneumonia differentiation, and (3) MRI radiomics for high-grade glioma vs brain metastasis differentiation.

CONCLUSIONS

The systematic reviews on radiomics were with suboptimal quality. A limited number of radiomics approaches were supported by convincing level of evidence.

CLINICAL RELEVANCE STATEMENT

The evidence supporting the clinical application of radiomics are insufficient, calling for researches translating radiomics from an academic tool to a practicable adjunct towards clinical deployment.

A systematic review of radiomics in giant cell tumor of bone (GCTB): the potential of analysis on individual radiomics feature for identifying genuine promising imaging biomarkers

PURPOSE

To systematically assess the quality of radiomics research in giant cell tumor of bone (GCTB) and to test the feasibility of analysis at the level of radiomics feature.

METHODS

We searched PubMed, Embase, Web of Science, China National Knowledge Infrastructure, and Wanfang Data to identify articles of GCTB radiomics until 31 July 2022. The studies were assessed by radiomics quality score (RQS), transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement, checklist for artificial intelligence in medical imaging (CLAIM), and modified quality assessment of diagnostic accuracy studies (QUADAS-2) tool. The radiomic features selected for model development were documented.

RESULTS

Nine articles were included. The average of the ideal percentage of RQS, the TRIPOD adherence rate and the CLAIM adherence rate were 26%, 56%, and 57%, respectively. The risk of bias and applicability concerns were mainly related to the index test. The shortness in external validation and open science were repeatedly emphasized. In GCTB radiomics models, the gray level co-occurrence matrix features (40%), first order features (28%), and gray-level run-length matrix features (18%) were most selected features out of all reported features. However, none of the individual feature has appeared repeatably in multiple studies. It is not possible to meta-analyze radiomics features at present.

CONCLUSION

The quality of GCTB radiomics studies is suboptimal. The reporting of individual radiomics feature data is encouraged. The analysis at the level of radiomics feature has potential to generate more practicable evidence for translating radiomics into clinical application.

Prediction of response to preoperative neoadjuvant chemotherapy in extremity high-grade osteosarcoma using X-ray and multiparametric MRI radiomics

PURPOSE

This study aims to evaluate the value of applying X-ray and magnetic resonance imaging (MRI) models based on radiomics feature to predict response of extremity high-grade osteosarcoma to neoadjuvant chemotherapy (NAC).

MATERIALS AND METHODS

A retrospective dataset was assembled involving 102 consecutive patients (training dataset, n = 72; validation dataset, n = 30) diagnosed with extremity high-grade osteosarcoma. The clinical features of age, gender, pathological type, lesion location, bone destruction type, size, alkaline phosphatase (ALP), and lactate dehydrogenase (LDH) were evaluated. Imaging features were extracted from X-ray and multi-parametric MRI (T1-weighted, T2-weighted, and contrast-enhanced T1-weighted) data. Features were selected using a two-stage process comprising minimal-redundancy-maximum-relevance (mRMR) and least absolute shrinkage and selection operator (LASSO) regression. Logistic regression (LR) modelling was then applied to establish models based on clinical, X-ray, and multi-parametric MRI data, as well as combinations of these datasets. Each model was evaluated using sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) with a 95% confidence interval (CI).

RESULTS

AUCs of 5 models using clinical, X-ray radiomics, MRI radiomics, X-ray plus MRI radiomics, and combination of all were 0.760 (95% CI: 0.583-0.937), 0.706 (95% CI: 0.506-0.905), 0.751 (95% CI: 0.572-0.930), 0.796 (95% CI: 0.629-0.963), 0.828 (95% CI: 0.676-0.980), respectively. The DeLong test showed no significant difference between any pair of models (p > 0.05). The combined model yielded higher performance than the clinical and radiomics models as demonstrated by net reclassification improvement (NRI) and integrated difference improvement (IDI) values, respectively. This combined model was also found to be clinically useful in the decision curve analysis (DCA).

CONCLUSION

Modelling based on combination of clinical and radiomics data improves the ability to predict pathological responses to NAC in extremity high-grade osteosarcoma compared to the models based on either clinical or radiomics data.