Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up

Current status of artificial intelligence technologies in pituitary adenoma surgery: a scoping review

PURPOSE

Pituitary adenoma surgery is a complex procedure due to critical adjacent neurovascular structures, variations in size and extensions of the lesions, and potential hormonal imbalances. The integration of artificial intelligence (AI) and machine learning (ML) has demonstrated considerable potential in assisting neurosurgeons in decision-making, optimizing surgical outcomes, and providing real-time feedback. This scoping review comprehensively summarizes the current status of AI/ML technologies in pituitary adenoma surgery, highlighting their strengths and limitations.

METHODS

PubMed, Embase, Web of Science, and Scopus were searched following the PRISMA-ScR guidelines. Studies discussing the use of AI/ML in pituitary adenoma surgery were included. Eligible studies were grouped to analyze the different outcomes of interest of current AI/ML technologies.

RESULTS

Among the 2438 identified articles, 44 studies met the inclusion criteria, with a total of seventeen different algorithms utilized across all studies. Studies were divided into two groups based on their input type: clinicopathological and imaging input. The four main outcome variables evaluated in the studies included: outcome (remission, recurrence or progression, gross-total resection, vision improvement, and hormonal recovery), complications (CSF leak, readmission, hyponatremia, and hypopituitarism), cost, and adenoma-related factors (aggressiveness, consistency, and Ki-67 labeling) prediction. Three studies focusing on workflow analysis and real-time navigation were discussed separately.

CONCLUSION

AI/ML modeling holds promise for improving pituitary adenoma surgery by enhancing preoperative planning and optimizing surgical strategies. However, addressing challenges such as algorithm selection, performance evaluation, data heterogeneity, and ethics is essential to establish robust and reliable ML models that can revolutionize neurosurgical practice and benefit patients.

The State of Machine Learning in Outcomes Prediction of Transsphenoidal Surgery: A Systematic Review

The purpose of this analysis is to assess the use of machine learning (ML) algorithms in the prediction of postoperative outcomes, including complications, recurrence, and death in transsphenoidal surgery. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically reviewed all papers that used at least one ML algorithm to predict outcomes after transsphenoidal surgery. We searched Scopus, PubMed, and Web of Science databases for studies published prior to May 12, 2021. We identified 13 studies enrolling 5,048 patients. We extracted the general characteristics of each study; the sensitivity, specificity, area under the curve (AUC) of the ML models developed as well as the features identified as important by the ML models. We identified 12 studies with 5,048 patients that included ML algorithms for adenomas, three with 1807 patients specifically for acromegaly, and five with 2105 patients specifically for Cushing's disease. Nearly all were single-institution studies. The studies used a heterogeneous mix of ML algorithms and features to build predictive models. All papers reported an AUC greater than 0.7, which indicates clinical utility. ML algorithms have the potential to predict postoperative outcomes of transsphenoidal surgery and can improve patient care. Ensemble algorithms and neural networks were often top performers when compared with other ML algorithms. Biochemical and preoperative features were most likely to be selected as important by ML models. Inexplicability remains a challenge, but algorithms such as local interpretable model-agnostic explanation or Shapley value can increase explainability of ML algorithms. Our analysis shows that ML algorithms have the potential to greatly assist surgeons in clinical decision making.

Supervised machine learning to validate a novel scoring system for the prediction of disease remission of functional pituitary adenomas following transsphenoidal surgery

Functional pituitary adenomas (FPAs) are associated with hormonal hypersecretion resulting in systemic endocrinopathies and increased mortality. The heterogenous composition of the FPA population has made modeling predictive factors of postoperative disease remission a challenge. Here, we aim to define a novel scoring system predictive of disease remission following transsphenoidal surgery (TSS) for FPAs and validate our process using supervised machine learning (SML). 392 patients with FPAs treated at one of the three Mayo Clinic campuses were retrospectively reviewed. Variables found significant on multivariate analysis were incorporated into our novel Pit-SCHEME score. The Pit-SCHEME score with a cut-off value ≥ 6 achieved a sensitivity of 86% and positive likelihood ratio of 2.88. In SML models, without the Pit-SCHEME score, the k-nearest neighbor (KNN) model achieved the highest accuracy at 75.6%. An increase in model sensitivity was achieved with inclusion of the Pit-SCHEME score with the linear discriminant analysis (LDA) model achieving an accuracy of 86.9%, which suggests the Pit-SCHEME score is the variable of most importance for prediction of postoperative disease remission. Ultimately, these results support the potential clinical utility of the Pit-SCHEME score and its prospective future for aiding in the perioperative decision making in patients with FPAs.

Integration of IDPC Clustering Analysis and Interpretable Machine Learning for Survival Risk Prediction of Patients with ESCC

Precise forecasting of survival risk plays a pivotal role in comprehending and predicting the prognosis of patients afflicted with esophageal squamous cell carcinoma (ESCC). The existing methods have the problems of insufficient fitting ability and poor interpretability. To address this issue, this work proposes a novel interpretable survival risk prediction method for ESCC patients based on extreme gradient boosting improved by whale optimization algorithm (WOA-XGBoost) and shapley additive explanations (SHAP). Given the imbalanced nature of the data set, the adaptive synthetic sampling (ADASYN) is first used to generate the samples with high survival risk. Then, an improved clustering by fast search and find of density peaks (IDPC) algorithm based on cosine distance and K nearest neighbors is used to cluster the patients. Next, the prediction model for each cluster is obtained by WOA-XGBoost and the constructed model is visualized with SHAP to uncover the factors hidden in the structured model and improve the interpretability of the black-box model. Finally, the effectiveness of the proposed scheme is demonstrated by analyzing the data collected from the First Affiliated Hospital of Zhengzhou University. The results of the analysis reveal that the proposed methodology exhibits superior performance, as indicated by the area under the receiver operating characteristic curve (AUROC) of 0.918 and accuracy of 0.881.

Predictive models for endoscopic disease activity in patients with ulcerative colitis: Practical machine learning-based modeling and interpretation

Background

Endoscopic disease activity monitoring is important for the long-term management of patients with ulcerative colitis (UC), there is currently no widely accepted non-invasive method that can effectively predict endoscopic disease activity. We aimed to develop and validate machine learning (ML) models for predicting it, which are desired to reduce the frequency of endoscopic examinations and related costs.

Methods

The patients with a diagnosis of UC in two hospitals from January 2016 to January 2021 were enrolled in this study. Thirty nine clinical and laboratory variables were collected. All patients were divided into four groups based on MES or UCEIS scores. Logistic regression (LR) and four ML algorithms were applied to construct the prediction models. The performance of models was evaluated in terms of accuracy, sensitivity, precision, F1 score, and area under the receiver-operating characteristic curve (AUC). Then Shapley additive explanations (SHAP) was applied to determine the importance of the selected variables and interpret the ML models.

Results

A total of 420 patients were entered into the study. Twenty four variables showed statistical differences among the groups. After synthetic minority oversampling technique (SMOTE) oversampling and RFE variables selection, the random forests (RF) model with 23 variables in MES and the extreme gradient boosting (XGBoost) model with 21 variables in USEIS, had the greatest discriminatory ability (AUC = 0.8192 in MES and 0.8006 in UCEIS in the test set). The results obtained from SHAP showed that albumin, rectal bleeding, and CRP/ALB contributed the most to the overall model. In addition, the above three variables had a more balanced contribution to each classification under the MES than the UCEIS according to the SHAP values.

Conclusion

This proof-of-concept study demonstrated that the ML model could serve as an effective non-invasive approach to predicting endoscopic disease activity for patients with UC. RF and XGBoost, which were first introduced into data-based endoscopic disease activity prediction, are suitable for the present prediction modeling.

Forecasting determinants of recurrence in lung cancer patients exploiting various machine learning models

Lung cancer recurrence seems to be the most leading cause of death as well as deterioration of lifespan. Proper assessment of the probability of recurrence in early-stage lung cancer is necessary to push up the treatment progress. We therefore employed machine-learning technologies to forecast post-operative recurrence risks using 174 lung cancer patient records. Six classification algorithms logistic regression, SVM, decision tree classification, random forest classification, XGBoost and lightGBM were used to predict the cancer recurrence. The patient samples were divided into training and test group with the split ratio of 3:1 for model generation and the accuracy were validated using k-fold cross-validation method. It is worth noting that the logistic regression model outperformed all the models in both training (Accuracy = 0.82) and test set (Accuracy = 0.79) on k-fold validation. Further, the optimal features (n = 7) identified using the RFE method is certainly helpful to improve the model in a high precision. The imperative risk factors associated with recurrence were identified using three feature selection methods. Importantly, our research showed that age is an important prognostic factor to be considered during the recurrence prediction. Indeed, severe concern on the identified risk factors combined with predictive models assists the physician to reduce the cancer recurrence rate in patients with lung cancer.

Opening the black box: interpretable machine learning for predictor finding of metabolic syndrome

OBJECTIVE

The internal workings ofmachine learning algorithms are complex and considered as low-interpretation "black box" models, making it difficult for domain experts to understand and trust these complex models. The study uses metabolic syndrome (MetS) as the entry point to analyze and evaluate the application value of model interpretability methods in dealing with difficult interpretation of predictive models.

METHODS

The study collects data from a chain of health examination institution in Urumqi from 2017 ~ 2019, and performs 39,134 remaining data after preprocessing such as deletion and filling. RFE is used for feature selection to reduce redundancy; MetS risk prediction models (logistic, random forest, XGBoost) are built based on a feature subset, and accuracy, sensitivity, specificity, Youden index, and AUROC value are used to evaluate the model classification performance; post-hoc model-agnostic interpretation methods (variable importance, LIME) are used to interpret the results of the predictive model.

RESULTS

Eighteen physical examination indicators are screened out by RFE, which can effectively solve the problem of physical examination data redundancy. Random forest and XGBoost models have higher accuracy, sensitivity, specificity, Youden index, and AUROC values compared with logistic regression. XGBoost models have higher sensitivity, Youden index, and AUROC values compared with random forest. The study uses variable importance, LIME and PDP for global and local interpretation of the optimal MetS risk prediction model (XGBoost), and different interpretation methods have different insights into the interpretation of model results, which are more flexible in model selection and can visualize the process and reasons for the model to make decisions. The interpretable risk prediction model in this study can help to identify risk factors associated with MetS, and the results showed that in addition to the traditional risk factors such as overweight and obesity, hyperglycemia, hypertension, and dyslipidemia, MetS was also associated with other factors, including age, creatinine, uric acid, and alkaline phosphatase.

CONCLUSION

The model interpretability methods are applied to the black box model, which can not only realize the flexibility of model application, but also make up for the uninterpretable defects of the model. Model interpretability methods can be used as a novel means of identifying variables that are more likely to be good predictors.

A machine learning model based on ultrasound image features to assess the risk of sentinel lymph node metastasis in breast cancer patients: Applications of scikit-learn and SHAP

Background

This study aimed to determine an optimal machine learning (ML) model for evaluating the preoperative diagnostic value of ultrasound signs of breast cancer lesions for sentinel lymph node (SLN) status.

Method

This study retrospectively analyzed the ultrasound images and postoperative pathological findings of lesions in 952 breast cancer patients. Firstly, the univariate analysis of the relationship between the ultrasonographic features of breast cancer morphological features and SLN metastasis. Then, based on the ultrasound signs of breast cancer lesions, we screened ten ML models: support vector machine (SVM), extreme gradient boosting (XGBoost), random forest (RF), linear discriminant analysis (LDA), logistic regression (LR), naive bayesian model (NB), k-nearest neighbors (KNN), multilayer perceptron (MLP), long short-term memory (LSTM), and convolutional neural network (CNN). The diagnostic performance of the model was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC), Kappa value, accuracy, F1-score, sensitivity, and specificity. Then we constructed a clinical prediction model which was based on the ML algorithm with the best diagnostic performance. Finally, we used SHapley Additive exPlanation (SHAP) to visualize and analyze the diagnostic process of the ML model.

Results

Of 952 patients with breast cancer, 394 (41.4%) had SLN metastasis, and 558 (58.6%) had no metastasis. Univariate analysis found that the shape, orientation, margin, posterior features, calculations, architectural distortion, duct changes and suspicious lymph node of breast cancer lesions in ultrasound signs were associated with SLN metastasis. Among the 10 ML algorithms, XGBoost had the best comprehensive diagnostic performance for SLN metastasis, with Average-AUC of 0.952, Average-Kappa of 0.763, and Average-Accuracy of 0.891. The AUC of the XGBoost model in the validation cohort was 0.916, the accuracy was 0.846, the sensitivity was 0.870, the specificity was 0.862, and the F1-score was 0.826. The diagnostic performance of the XGBoost model was significantly higher than that of experienced radiologists in some cases (P<0.001). Using SHAP to visualize the interpretation of the ML model screen, it was found that the ultrasonic detection of suspicious lymph nodes, microcalcifications in the primary tumor, burrs on the edge of the primary tumor, and distortion of the tissue structure around the lesion contributed greatly to the diagnostic performance of the XGBoost model.

Conclusions

The XGBoost model based on the ultrasound signs of the primary breast tumor and its surrounding tissues and lymph nodes has a high diagnostic performance for predicting SLN metastasis. Visual explanation using SHAP made it an effective tool for guiding clinical courses preoperatively.

Machine learning as a clinical decision support tool for patients with acromegaly

OBJECTIVE

To develop machine learning (ML) models that predict postoperative remission, remission at last visit, and resistance to somatostatin receptor ligands (SRL) in patients with acromegaly and to determine the clinical features associated with the prognosis.

METHODS

We studied outcomes using the area under the receiver operating characteristics (AUROC) values, which were reported as the performance metric. To determine the importance of each feature and easy interpretation, Shapley Additive explanations (SHAP) values, which help explain the outputs of ML models, are used.

RESULTS

One-hundred fifty-two patients with acromegaly were included in the final analysis. The mean AUROC values resulting from 100 independent replications were 0.728 for postoperative 3 months remission status classification, 0.879 for remission at last visit classification, and 0.753 for SRL resistance status classification. Extreme gradient boosting model demonstrated that preoperative growth hormone (GH) level, age at operation, and preoperative tumor size were the most important predictors for early remission; resistance to SRL and preoperative tumor size represented the most important predictors of remission at last visit, and postoperative 3-month insulin-like growth factor 1 (IGF1) and GH levels (random and nadir) together with the sparsely granulated somatotroph adenoma subtype served as the most important predictors of SRL resistance.

CONCLUSIONS

ML models may serve as valuable tools in the prediction of remission and SRL resistance.

The Application of Artificial Intelligence and Machine Learning in Pituitary Adenomas

Pituitary adenomas (PAs) are a group of tumors with complex and heterogeneous clinical manifestations. Early accurate diagnosis, individualized management, and precise prediction of the treatment response and prognosis of patients with PA are urgently needed. Artificial intelligence (AI) and machine learning (ML) have garnered increasing attention to quantitatively analyze complex medical data to improve individualized care for patients with PAs. Therefore, we critically examined the current use of AI and ML in the management of patients with PAs, and we propose improvements for future uses of AI and ML in patients with PAs. AI and ML can automatically extract many quantitative features based on massive medical data; moreover, related diagnosis and prediction models can be developed through quantitative analysis. Previous studies have suggested that AI and ML have wide applications in early accurate diagnosis; individualized treatment; predicting the response to treatments, including surgery, medications, and radiotherapy; and predicting the outcomes of patients with PAs. In addition, facial imaging-based AI and ML, pathological picture-based AI and ML, and surgical microscopic video-based AI and ML have also been reported to be useful in assisting the management of patients with PAs. In conclusion, the current use of AI and ML models has the potential to assist doctors and patients in making crucial surgical decisions by providing an accurate diagnosis, response to treatment, and prognosis of PAs. These AI and ML models can improve the quality and safety of medical services for patients with PAs and reduce the complication rates of neurosurgery. Further work is needed to obtain more reliable algorithms with high accuracy, sensitivity, and specificity for the management of PA patients.

Machine Learning Models for Predicting In-Hospital Mortality in Acute Aortic Dissection Patients

Background: Acute aortic dissection is a potentially fatal cardiovascular disorder associated with high mortality. However, current predictive models show a limited ability to efficiently and flexibly detect this mortality risk, and have been unable to discover a relationship between the mortality rate and certain variables. Thus, this study takes an artificial intelligence approach, whereby clinical data-driven machine learning was utilized to predict the in-hospital mortality of acute aortic dissection. Methods: Patients diagnosed with acute aortic dissection between January 2015 to December 2018 were voluntarily enrolled from the Second Xiangya Hospital of Central South University in the study. The diagnosis was defined by magnetic resonance angiography or computed tomography angiography, with an onset time of the symptoms being within 14 days. The analytical variables included demographic characteristics, physical examination, symptoms, clinical condition, laboratory results, and treatment strategies. The machine learning algorithms included logistic regression, decision tree, K nearest neighbor, Gaussian naive bayes, and extreme gradient boost (XGBoost). Evaluation of the predictive performance of the models was mainly achieved using the area under the receiver operating characteristic curve. SHapley Additive exPlanation was also implemented to interpret the final prediction model. Results: A total of 1,344 acute aortic dissection patients were recruited, including 1,071 (79.7%) patients in the survivor group and 273 (20.3%) patients in non-survivor group. The extreme gradient boost model was found to be the most effective model with the greatest area under the receiver operating characteristic curve (0.927, 95% CI: 0.860-0.968). The three most significant aspects of the extreme gradient boost importance matrix plot were treatment, type of acute aortic dissection, and ischemia-modified albumin levels. In the SHapley Additive exPlanation summary plot, medical treatment, type A acute aortic dissection, and higher ischemia-modified albumin level were shown to increase the risk of hospital-based mortality.