Ultrasound-based radiomics XGBoost model to assess the risk of central cervical lymph node metastasis in patients with papillary thyroid carcinoma: Individual application of SHAP

Predicting central cervical lymph node metastasis in papillary thyroid microcarcinoma using deep learning

Background

The aim of this study is to design a deep learning (DL) model to preoperatively predict the occurrence of central lymph node metastasis (CLNM) in patients with papillary thyroid microcarcinoma (PTMC).

Methods

This research collected preoperative ultrasound (US) images and clinical factors of 611 PTMC patients. The clinical factors were analyzed using multivariate regression. Then, a DL model based on US images and clinical factors was developed to preoperatively predict CLNM. The model's efficacy was evaluated using the receiver operating characteristic (ROC) curve, along with accuracy, sensitivity, specificity, and the F1 score.

Results

The multivariate analysis indicated an independent correlation factors including age ≥55 (OR = 0.309, p < 0.001), tumor diameter (OR = 2.551, p = 0.010), macrocalcifications (OR = 1.832, p = 0.002), and capsular invasion (OR = 1.977, p = 0.005). The suggested DL model utilized US images achieved an average area under the curve (AUC) of 0.65, slightly outperforming the model that employed traditional clinical factors (AUC = 0.64). Nevertheless, the model that incorporated both of them did not enhance prediction accuracy (AUC = 0.63).

Conclusions

The suggested approach offers a reference for the treatment and supervision of PTMC. Among three models used in this study, the deep model relied generally more on image modalities than the data modality of clinic records when making the predictions.

Interpretable machine learning model integrating clinical and elastosonographic features to detect renal fibrosis in Asian patients with chronic kidney disease

BACKGROUND

Non-invasive renal fibrosis assessment is critical for tailoring personalized decision-making and managing follow-up in patients with chronic kidney disease (CKD). We aimed to exploit machine learning algorithms using clinical and elastosonographic features to distinguish moderate-severe fibrosis from mild fibrosis among CKD patients.

METHODS

A total of 162 patients with CKD who underwent shear wave elastography examinations and renal biopsies at our institution were prospectively enrolled. Four classifiers using machine learning algorithms, including eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), Light Gradient Boosting Machine (LightGBM), and K-Nearest Neighbor (KNN), which integrated elastosonographic features and clinical characteristics, were established to differentiate moderate-severe renal fibrosis from mild forms. The area under the receiver operating characteristic curve (AUC) and average precision were employed to compare the performance of constructed models, and the SHapley Additive exPlanations (SHAP) strategy was used to visualize and interpret the model output.

RESULTS

The XGBoost model outperformed the other developed machine learning models, demonstrating optimal diagnostic performance in both the primary (AUC = 0.97, 95% confidence level (CI) 0.94-0.99; average precision = 0.97, 95% CI 0.97-0.98) and five-fold cross-validation (AUC = 0.85, 95% CI 0.73-0.98; average precision = 0.90, 95% CI 0.86-0.93) datasets. The SHAP approach provided visual interpretation for XGBoost, highlighting the features' impact on the diagnostic process, wherein the estimated glomerular filtration rate provided the largest contribution to the model output, followed by the elastic modulus, then renal length, renal resistive index, and hypertension.

CONCLUSION

This study proposed an XGBoost model for distinguishing moderate-severe renal fibrosis from mild forms in CKD patients, which could be used to assist clinicians in decision-making and follow-up strategies. Moreover, the SHAP algorithm makes it feasible to visualize and interpret the feature processing and diagnostic processes of the model output.

A Radiomics Approach to Identify Immunologically Active Tumor in Patients with Head and Neck Squamous Cell Carcinomas

BACKGROUND

We recently developed a gene-expression-based HOT score to identify the hot/cold phenotype of head and neck squamous cell carcinomas (HNSCCs), which is associated with the response to immunotherapy. Our goal was to determine whether radiomic profiling from computed tomography (CT) scans can distinguish hot and cold HNSCC.

METHOD

We included 113 patients from The Cancer Genome Atlas (TCGA) and 20 patients from the Groupe Hospitalier Pitié-Salpêtrière (GHPS) with HNSCC, all with available pre-treatment CT scans. The hot/cold phenotype was computed for all patients using the HOT score. The IBEX software (version 4.11.9, accessed on 30 march 2020) was used to extract radiomic features from the delineated tumor region in both datasets, and the intraclass correlation coefficient (ICC) was computed to select robust features. Machine learning classifier models were trained and tested in the TCGA dataset and validated using the area under the receiver operator characteristic curve (AUC) in the GHPS cohort.

RESULTS

A total of 144 radiomic features with an ICC >0.9 was selected. An XGBoost model including these selected features showed the best performance prediction of the hot/cold phenotype with AUC = 0.86 in the GHPS validation dataset.

CONCLUSIONS AND RELEVANCE

We identified a relevant radiomic model to capture the overall hot/cold phenotype of HNSCC. This non-invasive approach could help with the identification of patients with HNSCC who may benefit from immunotherapy.

Ultrasound radiomics models based on multimodal imaging feature fusion of papillary thyroid carcinoma for predicting central lymph node metastasis

Objective

This retrospective study aimed to establish ultrasound radiomics models to predict central lymph node metastasis (CLNM) based on preoperative multimodal ultrasound imaging features fusion of primary papillary thyroid carcinoma (PTC).

Methods

In total, 498 cases of unifocal PTC were randomly divided into two sets which comprised 348 cases (training set) and 150 cases (validition set). In addition, the testing set contained 120 cases of PTC at different times. Post-operative histopathology was the gold standard for CLNM. The following steps were used to build models: the regions of interest were segmented in PTC ultrasound images, multimodal ultrasound image features were then extracted by the deep learning residual neural network with 50-layer network, followed by feature selection and fusion; subsequently, classification was performed using three classical classifiers-adaptive boosting (AB), linear discriminant analysis (LDA), and support vector machine (SVM). The performances of the unimodal models (Unimodal-AB, Unimodal-LDA, and Unimodal-SVM) and the multimodal models (Multimodal-AB, Multimodal-LDA, and Multimodal-SVM) were evaluated and compared.

Results

The Multimodal-SVM model achieved the best predictive performance than the other models (P < 0.05). For the Multimodal-SVM model validation and testing sets, the areas under the receiver operating characteristic curves (AUCs) were 0.910 (95% CI, 0.894-0.926) and 0.851 (95% CI, 0.833-0.869), respectively. The AUCs of the Multimodal-SVM model were 0.920 (95% CI, 0.881-0.959) in the cN0 subgroup-1 cases and 0.828 (95% CI, 0.769-0.887) in the cN0 subgroup-2 cases.

Conclusion

The ultrasound radiomics model only based on the PTC multimodal ultrasound image have high clinical value in predicting CLNM and can provide a reference for treatment decisions.

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

Prediction of Surgical Approach in Mitral Valve Disease by XGBoost Algorithm Based on Echocardiographic Features

In this study, we aimed to develop a prediction model to assist surgeons in choosing an appropriate surgical approach for mitral valve disease patients. We retrospectively analyzed a total of 143 patients who underwent surgery for mitral valve disease. The XGBoost algorithm was used to establish a predictive model to decide a surgical approach (mitral valve repair or replacement) based on the echocardiographic features of the mitral valve apparatus, such as leaflets, the annulus, and sub-valvular structures. The results showed that the accuracy of the predictive model was 81.09% in predicting the appropriate surgical approach based on the patient's preoperative echocardiography. The result of the predictive model was superior to the traditional complexity score (81.09% vs. 75%). Additionally, the predictive model showed that the three main factors affecting the choice of surgical approach were leaflet restriction, calcification of the leaflet, and perforation or cleft of the leaflet. We developed a novel predictive model using the XGBoost algorithm based on echocardiographic features to assist surgeons in choosing an appropriate surgical approach for patients with mitral valve disease.

Development and validation of an interpretable radiomic nomogram for severe radiation proctitis prediction in postoperative cervical cancer patients

Background

Radiation proctitis is a common complication after radiotherapy for cervical cancer. Unlike simple radiation damage to other organs, radiation proctitis is a complex disease closely related to the microbiota. However, analysis of the gut microbiota is time-consuming and expensive. This study aims to mine rectal information using radiomics and incorporate it into a nomogram model for cheap and fast prediction of severe radiation proctitis prediction in postoperative cervical cancer patients.

Methods

The severity of the patient's radiation proctitis was graded according to the RTOG/EORTC criteria. The toxicity grade of radiation proctitis over or equal to grade 2 was set as the model's target. A total of 178 patients with cervical cancer were divided into a training set (n = 124) and a validation set (n = 54). Multivariate logistic regression was used to build the radiomic and non-raidomic models.

Results

The radiomics model [AUC=0.6855(0.5174-0.8535)] showed better performance and more net benefit in the validation set than the non-radiomic model [AUC=0.6641(0.4904-0.8378)]. In particular, we applied SHapley Additive exPlanation (SHAP) method for the first time to a radiomics-based logistic regression model to further interpret the radiomic features from case-based and feature-based perspectives. The integrated radiomic model enables the first accurate quantitative assessment of the probability of radiation proctitis in postoperative cervical cancer patients, addressing the limitations of the current qualitative assessment of the plan through dose-volume parameters only.

Conclusion

We successfully developed and validated an integrated radiomic model containing rectal information. SHAP analysis of the model suggests that radiomic features have a supporting role in the quantitative assessment of the probability of radiation proctitis in postoperative cervical cancer patients.