Explainable machine learning model for predicting the occurrence of postoperative malnutrition in children with congenital heart disease

A review of evaluation approaches for explainable AI with applications in cardiology

Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10462-024-10852-w.

Risk factors of malnutrition in children with congenital heart disease: a meta-analysis

Background

The associated factors of malnutrition in children with congenital heart disease (CHD) must be evaluated to provide evidence for the treatment and care of such children.

Methods

Two investigators searched the PubMed database until 25 June 2023 for literature about the associated factors of malnutrition in children with CHD. A meta-analysis of associated factors of malnutrition was performed by RevMan 5.3 software.

Results

Thirteen studies involving 8,031 children with CHD were included. Pulmonary hypertension (OR = 3.81, 95% CI: 2.46-4.12), low birth weight (OR = 2.69, 95% CI: 1.25-5.77) and parents' height (OR = 2.15, 95% CI: 1.89-2.92) were the associated factors of growth retardation (all P < 0.05). Pulmonary hypertension (OR = 3.77, 95% CI: 3.13-4.24), low birth weight (OR = 3.04, 95% CI: 2.61-4.18) and pneumonia (OR = 2.35, 95% CI: 2.08-2.83) were the associated factors of low body weight of children with CHD (all P < 0.05).

Conclusions

Medical staff should fully understand the risk factors, strengthen nutritional support and enhance nursing care for children with CHD to reduce malnutrition.

Development and evaluation of a deep learning framework for the diagnosis of malnutrition using a 3D facial points cloud: A cross-sectional study

BACKGROUND

The feasibility of diagnosing malnutrition using facial features has been validated. A tool to integrate all facial features associated with malnutrition for disease screening is still demanded. This work aims to develop and evaluate a deep learning (DL) framework to accurately determine malnutrition based on a 3D facial points cloud.

METHODS

A group of 482 patients were studied in this perspective work. The 3D facial data were obtained using a 3D camera and represented as a 3D facial points cloud. A DL model, PointNet++, was trained and evaluated using the points cloud as inputs and classified the malnutrition states. The performance was evaluated with the area under the receiver operating characteristic curve, accuracy, specificity, sensitivity, and F1 score.

RESULTS

Among the 482 patients, 150 patients (31.1%) were diagnosed as having moderate malnutrition and 54 patients (11.2%) as having severe malnutrition. The DL model achieved the performance with an area under the receiver operating characteristic curve of 0.7240 ± 0.0416.

CONCLUSION

The DL model achieved encouraging performance in accurately classifying nutrition states based on a points cloud of 3D facial information of patients with malnutrition.

eXplainable Artificial Intelligence (XAI) for improving organisational regility

Since the pandemic started, organisations have been actively seeking ways to improve their organisational agility and resilience (regility) and turn to Artificial Intelligence (AI) to gain a deeper understanding and further enhance their agility and regility. Organisations are turning to AI as a critical enabler to achieve these goals. AI empowers organisations by analysing large data sets quickly and accurately, enabling faster decision-making and building agility and resilience. This strategic use of AI gives businesses a competitive advantage and allows them to adapt to rapidly changing environments. Failure to prioritise agility and responsiveness can result in increased costs, missed opportunities, competition and reputational damage, and ultimately, loss of customers, revenue, profitability, and market share. Prioritising can be achieved by utilising eXplainable Artificial Intelligence (XAI) techniques, illuminating how AI models make decisions and making them transparent, interpretable, and understandable. Based on previous research on using AI to predict organisational agility, this study focuses on integrating XAI techniques, such as Shapley Additive Explanations (SHAP), in organisational agility and resilience. By identifying the importance of different features that affect organisational agility prediction, this study aims to demystify the decision-making processes of the prediction model using XAI. This is essential for the ethical deployment of AI, fostering trust and transparency in these systems. Recognising key features in organisational agility prediction can guide companies in determining which areas to concentrate on in order to improve their agility and resilience.

A risk prediction model based on machine learning for postoperative cognitive dysfunction in elderly patients with non-cardiac surgery

BACKGROUND

Early identification of elderly patients undergoing non-cardiac surgery who may be at high risk for postoperative cognitive dysfunction (POCD) can increase the chances of prevention for them, as extra attention and limited resources can be allocated more to these patients.

AIM

We performed this analysis with the aim of developing a simple, clinically useful machine learning (ML) model to predict the probability of POCD at 3 months in elderly patients after non-cardiac surgery.

METHODS

We collected information on patients who received surgical treatment at Nanjing First Hospital from May 2020 to May 2021. We used LASSO regression to select key features and built 5 ML models to assess the risk of POCD at 3 months in elderly patients after non-cardiac surgery. The Shapley Additive exPlanations (SHAP) and methods were introduced to interpret the best model.

RESULTS

A total of 415 patients with non-cardiac surgery were included. The support vector machine (SVM) was the best-performing model of the five ML models. The model showed excellent performance compared to the other four models. The SHAP results showed that VAS score, age, intraoperative hypotension, and preoperative hemoglobin were the four most important features, indicating that the SVM model had good interpretability and reliability. The website of the web-based calculator was https://modricreagan-non-3-pocd-9w2q78.streamlit.app/ .

CONCLUSION

Based on six important perioperative variables, we successfully established a series of ML models for predicting POCD occurrence at 3 months after surgery in elderly non-cardiac patients, with SVM model being the best-performing model. Our models are expected to serve as decision aids for clinicians to monitor screened high-risk patients more closely or to consider further interventions.

Explainable machine learning model for predicting skeletal muscle loss during surgery and adjuvant chemotherapy in ovarian cancer

BACKGROUND

Skeletal muscle loss during treatment is associated with poor survival outcomes in patients with ovarian cancer. Although changes in muscle mass can be assessed on computed tomography (CT) scans, this labour-intensive process can impair its utility in clinical practice. This study aimed to develop a machine learning (ML) model to predict muscle loss based on clinical data and to interpret the ML model by applying SHapley Additive exPlanations (SHAP) method.

METHODS

This study included the data of 617 patients with ovarian cancer who underwent primary debulking surgery and platinum-based chemotherapy at a tertiary centre between 2010 and 2019. The cohort data were split into training and test sets based on the treatment time. External validation was performed using 140 patients from a different tertiary centre. The skeletal muscle index (SMI) was measured from pre- and post-treatment CT scans, and a decrease in SMI ≥ 5% was defined as muscle loss. We evaluated five ML models to predict muscle loss, and their performance was determined using the area under the receiver operating characteristic curve (AUC) and F1 score. The features for analysis included demographic and disease-specific characteristics and relative changes in body mass index (BMI), albumin, neutrophil-to-lymphocyte ratio (NLR), and platelet-to-lymphocyte ratio (PLR). The SHAP method was applied to determine the importance of the features and interpret the ML models.

RESULTS

The median (inter-quartile range) age of the cohort was 52 (46-59) years. After treatment, 204 patients (33.1%) experienced muscle loss in the training and test datasets, while 44 (31.4%) patients experienced muscle loss in the external validation dataset. Among the five evaluated ML models, the random forest model achieved the highest AUC (0.856, 95% confidence interval: 0.854-0.859) and F1 score (0.726, 95% confidence interval: 0.722-0.730). In the external validation, the random forest model outperformed all ML models with an AUC of 0.874 and an F1 score of 0.741. The results of the SHAP method showed that the albumin change, BMI change, malignant ascites, NLR change, and PLR change were the most important factors in muscle loss. At the patient level, SHAP force plots demonstrated insightful interpretation of our random forest model to predict muscle loss.

CONCLUSIONS

Explainable ML model was developed using clinical data to identify patients experiencing muscle loss after treatment and provide information of feature contribution. Using the SHAP method, clinicians may better understand the contributors to muscle loss and target interventions to counteract muscle loss.

Rapid assessment of malnutrition based on GLIM diagnosis in Crohn's disease

Background and aims

Malnutrition is strongly linked to adverse outcomes in patients with Crohn's disease (CD). In this study, our objective was to validate the Global Leadership Initiative on Malnutrition (GLIM) criteria and develop a fast and accurate diagnostic approach for identifying malnutrition in CD patients.

Methods

This study assessed 177 CD patients from four general hospitals. The efficacy of the GLIM criteria for the diagnosis of CD malnutrition was compared. By analyzing the independent factors, a nomogram model was derived and internally validated to predict the diagnosis of malnutrition in patients with CD. Model performance was assessed using discrimination and calibration, decision curves, and net benefit analyses.

Results

Compared with the SGA criteria, the GLIM criteria was consistent in sensitivity (88.89%) and specificity (78.43%) [AUC = 0.84; 95% Confidence Interval (CI): 0.77-0.89]. The Harvey-Bradshaw index(HBI) score (OR: 1.58; 95% CI: 1.15-2.18), body mass index (OR: 0.41; 95% CI: 0.27-0.64), and mid-upper arm circumference (OR: 0.68; 95% CI: 0.47-0.9) were independent factors associated with malnutrition. The nomogram was developed based on these indicators showing good discrimination in malnutrition diagnosis (AUC = 0.953; 95% CI: 0.922-0.984), with agreement after calibration curve and decision curve analysis.

Conclusion

The GLIM criteria are appropriate for diagnosing malnutrition in CD patients. The HBI score may be used to diagnose malnutrition in patients with CD and become a possible selection for the GLIM etiologic criteria of inflammation. The HBM nomogram could be a simple, rapid, and efficient method for diagnosing malnutrition in CD patients.

Development and Assessment of Assisted Diagnosis Models Using Machine Learning for Identifying Elderly Patients With Malnutrition: Cohort Study

BACKGROUND

Older patients are at an increased risk of malnutrition due to many factors related to poor clinical outcomes.

OBJECTIVE

This study aims to develop an assisted diagnosis model using machine learning (ML) for identifying older patients with malnutrition and providing the focus of individualized treatment.

METHODS

We reanalyzed a multicenter, observational cohort study including 2660 older patients. Baseline malnutrition was defined using the global leadership initiative on malnutrition (GLIM) criteria, and the study population was randomly divided into a derivation group (2128/2660, 80%) and a validation group (532/2660, 20%). We applied 5 ML algorithms and further explored the relationship between features and the risk of malnutrition by using the Shapley additive explanations visualization method.

RESULTS

The proposed ML models were capable to identify older patients with malnutrition. In the external validation cohort, the top 3 models by the area under the receiver operating characteristic curve were light gradient boosting machine (92.1%), extreme gradient boosting (91.9%), and the random forest model (91.5%). Additionally, the analysis of the importance of features revealed that BMI, weight loss, and calf circumference were the strongest predictors to affect GLIM. A BMI of below 21 kg/m2 was associated with a higher risk of GLIM in older people.

CONCLUSIONS

We developed ML models for assisting diagnosis of malnutrition based on the GLIM criteria. The cutoff values of laboratory tests generated by Shapley additive explanations could provide references for the identification of malnutrition.

TRIAL REGISTRATION

Chinese Clinical Trial Registry ChiCTR-EPC-14005253; https://www.chictr.org.cn/showproj.aspx?proj=9542.

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

Artificial Intelligence in Pediatric Cardiology: A Scoping Review

The evolution of AI and data science has aided in mechanizing several aspects of medical care requiring critical thinking: diagnosis, risk stratification, and management, thus mitigating the burden of physicians and reducing the likelihood of human error. AI modalities have expanded feet to the specialty of pediatric cardiology as well. We conducted a scoping review searching the Scopus, Embase, and PubMed databases covering the recent literature between 2002-2022. We found that the use of neural networks and machine learning has significantly improved the diagnostic value of cardiac magnetic resonance imaging, echocardiograms, computer tomography scans, and electrocardiographs, thus augmenting the clinicians' diagnostic accuracy of pediatric heart diseases. The use of AI-based prediction algorithms in pediatric cardiac surgeries improves postoperative outcomes and prognosis to a great extent. Risk stratification and the prediction of treatment outcomes are feasible using the key clinical findings of each CHD with appropriate computational algorithms. Notably, AI can revolutionize prenatal prediction as well as the diagnosis of CHD using the EMR (electronic medical records) data on maternal risk factors. The use of AI in the diagnostics, risk stratification, and management of CHD in the near future is a promising possibility with current advancements in machine learning and neural networks. However, the challenges posed by the dearth of appropriate algorithms and their nascent nature, limited physician training, fear of over-mechanization, and apprehension of missing the 'human touch' limit the acceptability. Still, AI proposes to aid the clinician tomorrow with precision cardiology, paving a way for extremely efficient human-error-free health care.

Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011-2022)

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) has branched out to various applications in healthcare, such as health services management, predictive medicine, clinical decision-making, and patient data and diagnostics. Although AI models have achieved human-like performance, their use is still limited because they are seen as a black box. This lack of trust remains the main reason for their low use in practice, especially in healthcare. Hence, explainable artificial intelligence (XAI) has been introduced as a technique that can provide confidence in the model's prediction by explaining how the prediction is derived, thereby encouraging the use of AI systems in healthcare. The primary goal of this review is to provide areas of healthcare that require more attention from the XAI research community.

METHODS

Multiple journal databases were thoroughly searched using PRISMA guidelines 2020. Studies that do not appear in Q1 journals, which are highly credible, were excluded.

RESULTS

In this review, we surveyed 99 Q1 articles covering the following XAI techniques: SHAP, LIME, GradCAM, LRP, Fuzzy classifier, EBM, CBR, rule-based systems, and others.

CONCLUSION

We discovered that detecting abnormalities in 1D biosignals and identifying key text in clinical notes are areas that require more attention from the XAI research community. We hope this is review will encourage the development of a holistic cloud system for a smart city.