A Hybrid Explainable Ensemble Transformer Encoder for Pneumonia Identification from Chest X-ray Images

INTRODUCTION

Pneumonia is a microorganism infection that causes chronic inflammation of the human lung cells. Chest X-ray imaging is the most well-known screening approach used for detecting pneumonia in the early stages. While chest-Xray images are mostly blurry with low illumination, a strong feature extraction approach is required for promising identification performance.

OBJECTIVES

A new hybrid explainable deep learning framework is proposed for accurate pneumonia disease identification using chest X-ray images.

METHODS

The proposed hybrid workflow is developed by fusing the capabilities of both ensemble convolutional networks and the Transformer Encoder mechanism. The ensemble learning backbone is used to extract strong features from the raw input X-ray images in two different scenarios: ensemble A (i.e., DenseNet201, VGG16, and GoogleNet) and ensemble B (i.e., DenseNet201, InceptionResNetV2, and Xception). Whereas, the Transformer Encoder is built based on the self-attention mechanism with multilayer perceptron (MLP) for accurate disease identification. The visual explainable saliency maps are derived to emphasize the crucial predicted regions on the input X-ray images. The end-to-end training process of the proposed deep learning models over all scenarios is performed for binary and multi-class classification tasks.

RESULTS

The proposed hybrid deep learning model recorded 99.21% classification performance in terms of overall accuracy and F1-score for the binary classification task, while it achieved 98.19% accuracy and 97.29% F1-score for multi-classification task. For the ensemble binary identification scenario, ensemble A recorded 97.22% accuracy and 97.14% F1-score, while ensemble B achieved 96.44% for both accuracy and F1-score. For the ensemble, multiclass identification scenario, ensemble A recorded 97.2% accuracy and 95.8% F1-score, while ensemble B recorded 96.4% accuracy and 94.9% F1-score.

CONCLUSION

The proposed hybrid deep learning framework could provide promising and encouraging explainable identification performance comparing with individual, ensemble models, or even the latest models in the literature. The code is available here: https://github.com/chiagoziemchima/Pneumonia_Identificaton.

Early Prediction of Mortality for Septic Patients Visiting Emergency Room Based on Explainable Machine Learning: A Real-World Multicenter Study

BACKGROUND

Worldwide, sepsis is the leading cause of death in hospitals. If mortality rates in patients with sepsis can be predicted early, medical resources can be allocated efficiently. We constructed machine learning (ML) models to predict the mortality of patients with sepsis in a hospital emergency department.

METHODS

This study prospectively collected nationwide data from an ongoing multicenter cohort of patients with sepsis identified in the emergency department. Patients were enrolled from 19 hospitals between September 2019 and December 2020. For acquired data from 3,657 survivors and 1,455 deaths, six ML models (logistic regression, support vector machine, random forest, extreme gradient boosting [XGBoost], light gradient boosting machine, and categorical boosting [CatBoost]) were constructed using fivefold cross-validation to predict mortality. Through these models, 44 clinical variables measured on the day of admission were compared with six sequential organ failure assessment (SOFA) components (PaO2/FIO2 [PF], platelets (PLT), bilirubin, cardiovascular, Glasgow Coma Scale score, and creatinine). The confidence interval (CI) was obtained by performing 10,000 repeated measurements via random sampling of the test dataset. All results were explained and interpreted using Shapley's additive explanations (SHAP).

RESULTS

Of the 5,112 participants, CatBoost exhibited the highest area under the curve (AUC) of 0.800 (95% CI, 0.756-0.840) using clinical variables. Using the SOFA components for the same patient, XGBoost exhibited the highest AUC of 0.678 (95% CI, 0.626-0.730). As interpreted by SHAP, albumin, lactate, blood urea nitrogen, and international normalization ratio were determined to significantly affect the results. Additionally, PF and PLTs in the SOFA component significantly influenced the prediction results.

CONCLUSION

Newly established ML-based models achieved good prediction of mortality in patients with sepsis. Using several clinical variables acquired at the baseline can provide more accurate results for early predictions than using SOFA components. Additionally, the impact of each variable was identified.

Towards an Explainable AI-Based Tool to Predict Preterm Birth

Preterm birth (PTB) is defined as delivery occurring before 37 weeks of gestation. In this paper, Artificial Intelligence (AI)-based predictive models are adapted to accurately estimate the probability of PTB. In doing so, pregnant women' objective results and variables extracted from the screening procedure in combination with demographics, medical history, social history, and other medical data are used. A dataset consisting of 375 pregnant women is used and a number of alternative Machine Learning (ML) algorithms are applied to predict PTB. The ensemble voting model produced the best results across all performance metrics with an area under the curve (ROC-AUC) of approximately 0.84 and a precision-recall curve (PR-AUC) of approximately 0.73. An attempt to provide clinicians with an explanation of the prediction is performed to increase trustworthiness.

Coalitional Strategies for Efficient Individual Prediction Explanation

As Machine Learning (ML) is now widely applied in many domains, in both research and industry, an understanding of what is happening inside the black box is becoming a growing demand, especially by non-experts of these models. Several approaches had thus been developed to provide clear insights of a model prediction for a particular observation but at the cost of long computation time or restrictive hypothesis that does not fully take into account interaction between attributes. This paper provides methods based on the detection of relevant groups of attributes -named coalitions- influencing a prediction and compares them with the literature. Our results show that these coalitional methods are more efficient than existing ones such as SHapley Additive exPlanation (SHAP). Computation time is shortened while preserving an acceptable accuracy of individual prediction explanations. Therefore, this enables wider practical use of explanation methods to increase trust between developed ML models, end-users, and whoever impacted by any decision where these models played a role.

Exploring Hospital Overcrowding with an Explainable Time-to-Event Machine Learning Approach

Emergency department (ED) overcrowding is a complex problem that is intricately linked with the operations of other hospital departments. Leveraging ED real-world production data provides a unique opportunity to comprehend this multifaceted problem holistically. This paper introduces a novel approach to analyse healthcare production data, treating the length of stay of patients, and the follow up decision regarding discharge or admission to the hospital as a time-to-event analysis problem. Our methodology employs traditional survival estimators and machine learning models, and Shapley additive explanations values to interpret the model outcomes. The most relevant features influencing length of stay were whether the patient received a scan at the ED, emergency room urgent visit, age, triage level, and the medical alarm unit category. The clinical insights derived from the explanation of the models holds promise for increase understanding of the overcrowding from the data. Our work demonstrates that a time-to-event approach to the over- crowding serves as a valuable initial to uncover crucial insights for further investigation and policy design.

Current methods in explainable artificial intelligence and future prospects for integrative physiology

Explainable artificial intelligence (XAI) is gaining importance in physiological research, where artificial intelligence is now used as an analytical and predictive tool for many medical research questions. The primary goal of XAI is to make AI models understandable for human decision-makers. This can be achieved in particular through providing inherently interpretable AI methods or by making opaque models and their outputs transparent using post hoc explanations. This review introduces XAI core topics and provides a selective overview of current XAI methods in physiology. It further illustrates solved and discusses open challenges in XAI research using existing practical examples from the medical field. The article gives an outlook on two possible future prospects: (1) using XAI methods to provide trustworthy AI for integrative physiological research and (2) integrating physiological expertise about human explanation into XAI method development for useful and beneficial human-AI partnerships.

Deep learning framework for rapid and accurate respiratory COVID-19 prediction using chest X-ray images

COVID-19 is a contagious disease that affects the human respiratory system. Infected individuals may develop serious illnesses, and complications may result in death. Using medical images to detect COVID-19 from essentially identical thoracic anomalies is challenging because it is time-consuming, laborious, and prone to human error. This study proposes an end-to-end deep-learning framework based on deep feature concatenation and a Multi-head Self-attention network. Feature concatenation involves fine-tuning the pre-trained backbone models of DenseNet, VGG-16, and InceptionV3, which are trained on a large-scale ImageNet, whereas a Multi-head Self-attention network is adopted for performance gain. End-to-end training and evaluation procedures are conducted using the COVID-19_Radiography_Dataset for binary and multi-classification scenarios. The proposed model achieved overall accuracies (96.33% and 98.67%) and F1_scores (92.68% and 98.67%) for multi and binary classification scenarios, respectively. In addition, this study highlights the difference in accuracy (98.0% vs. 96.33%) and F_1 score (97.34% vs. 95.10%) when compared with feature concatenation against the highest individual model performance. Furthermore, a virtual representation of the saliency maps of the employed attention mechanism focusing on the abnormal regions is presented using explainable artificial intelligence (XAI) technology. The proposed framework provided better COVID-19 prediction results outperforming other recent deep learning models using the same dataset.

Leveraging explainable AI to predict soil respiration sensitivity and its drivers for climate change mitigation

Global warming is one of the most pressing and critical problems facing the world today. It is mainly caused by the increase in greenhouse gases in the atmosphere, such as carbon dioxide (CO2). Understanding how soils respond to rising temperatures is critical for predicting carbon release and informing climate mitigation strategies. Q10, a measure of soil microbial respiration, quantifies the increase in CO2 release caused by a [Formula: see text] Celsius rise in temperature, serving as a key indicator of this sensitivity. However, predicting Q10 across diverse soil types remains a challenge, especially when considering the complex interactions between biochemical, microbiome, and environmental factors. In this study, we applied explainable artificial intelligence (XAI) to machine learning models to predict soil respiration sensitivity (Q10) and uncover the key factors driving this process. Using SHAP (SHapley Additive exPlanations) values, we identified glucose-induced soil respiration and the proportion of bacteria positively associated with Q10 as the most influential predictors. Our machine learning models achieved an accuracy of [Formula: see text], precision of [Formula: see text], an AUC-ROC of [Formula: see text], and an AUC-PRC of [Formula: see text], ensuring robust and reliable predictions. By leveraging t-SNE (t-distributed Stochastic Neighbor Embedding) and clustering techniques, we further segmented low Q10 soils into distinct subgroups, identifying soils with a higher probability of transitioning to high Q10 states. Our findings not only highlight the potential of XAI in making model predictions transparent and interpretable, but also provide actionable insights into managing soil carbon release in response to climate change. This research bridges the gap between AI-driven environmental modeling and practical applications in agriculture, offering new directions for targeted soil management and climate resilience strategies.

Exploring the anticancer activities of Sulfur and magnesium oxide through integration of deep learning and fuzzy rough set analyses based on the features of Vidarabine alkaloid

Drug discovery and development is a challenging and time-consuming process. Laboratory experiments conducted on Vidarabine showed IC50 6.97 µg∕mL, 25.78 µg∕mL, and ˃ 100 µg∕mL against non-small Lung cancer (A-549), Human Melanoma (A-375), and Human epidermoid Skin carcinoma (skin/epidermis) (A-431) respectively. To address these challenges, this paper presents an Artificial Intelligence (AI) model that combines the capabilities of Deep Learning (DL) to identify potential new drug candidates, Fuzzy Rough Set (FRS) theory to determine the most important chemical compound features, Explainable Artificial Intelligence (XAI) to explain the features' importance in the last layer, and medicinal chemistry to rediscover anticancer drugs based on natural products like Vidarabine. The proposed model aims to identify potential new drug candidates. By analyzing the results from laboratory experiments on Vidarabine, the model identifies Sulfur and magnesium oxide (MgO) as new potential anticancer agents. The proposed model selected Sulfur and MgO based on Interpreting their promising features, and further laboratory experiments were conducted to validate the model's predictions. The results demonstrated that, while Vidarabine was inactive against the A-431 cell line (IC50 ˃ 100 µg∕mL), Sulfur and MgO exhibited significant anticancer activity (IC50 4.55 and 17.29 µg/ml respectively). Sulfur displayed strong activity against A-549 and A-375 cell lines (IC50 3.06 and 1.86 µg/ml respectively) better than Vidarabine (IC50 6.97 and 25.78 µg/ml respectively). However, MgO showed weaker activity against these two cell lines. This paper emphasizes the importance of uncovering hidden chemical features that may not be discernible without the assistance of AI. This highlights the ability of AI to discover novel compounds with therapeutic potential, which can significantly impact the field of drug discovery. The promising anticancer activity exhibited by Sulfur and MgO warrants further preclinical studies.

ECG-based data-driven solutions for diagnosis and prognosis of cardiovascular diseases: A systematic review

Cardiovascular diseases (CVD) are a leading cause of death globally, and result in significant morbidity and reduced quality of life. The electrocardiogram (ECG) plays a crucial role in CVD diagnosis, prognosis, and prevention; however, different challenges still remain, such as an increasing unmet demand for skilled cardiologists capable of accurately interpreting ECG. This leads to higher workload and potential diagnostic inaccuracies. Data-driven approaches, such as machine learning (ML) and deep learning (DL) have emerged to improve existing computer-assisted solutions and enhance physicians' ECG interpretation of the complex mechanisms underlying CVD. However, many ML and DL models used to detect ECG-based CVD suffer from a lack of explainability, bias, as well as ethical, legal, and societal implications (ELSI). Despite the critical importance of these Trustworthy Artificial Intelligence (AI) aspects, there is a lack of comprehensive literature reviews that examine the current trends in ECG-based solutions for CVD diagnosis or prognosis that use ML and DL models and address the Trustworthy AI requirements. This review aims to bridge this knowledge gap by providing a systematic review to undertake a holistic analysis across multiple dimensions of these data-driven models such as type of CVD addressed, dataset characteristics, data input modalities, ML and DL algorithms (with a focus on DL), and aspects of Trustworthy AI like explainability, bias and ethical considerations. Additionally, within the analyzed dimensions, various challenges are identified. To these, we provide concrete recommendations, equipping other researchers with valuable insights to understand the current state of the field comprehensively.

Interpretable Artificial Intelligence through Locality Guided Neural Networks

In current deep learning architectures, each of the deeper layers in networks tends to contain hundreds of unorganized neurons which makes it hard for humans to understand how they interact with each other. By organizing the neurons using correlation as the criteria, humans can observe how clusters of neighbouring neurons interact with each other. Research in Explainable Artificial Intelligence (XAI) aims to all alleviate the black-box nature of current AI methods and make them understandable by humans. In this paper, we extend our previous algorithm for XAI in deep learning, called Locality Guided Neural Network (LGNN). LGNN preserves locality between neighbouring neurons within each layer of a deep network during training. Motivated by Self-Organizing Maps (SOMs), the goal is to enforce a local topology on each layer of a deep network such that neighbouring neurons are highly correlated with each other. Our algorithm can be easily plugged into current state of the art Convolutional Neural Network (CNN) models to make the neighbouring neurons more correlated. A cluster of neighbouring neurons activating for a class makes the network both quantitatively and qualitatively more interpretable when visualized, as we show through our experiments. This paper focuses on image processing with CNNs, but can theoretically be applied to any type of deep learning architecture. In our experiments, we train VGG and WRN networks for image classification on CIFAR100 and Imagenette. Our experiments analyse different perceptible clusters of activations in response to different input classes.

An explainable AI-assisted web application in cancer drug value prediction

In recent years, there has been an increase in the interest in adopting Explainable Artificial Intelligence (XAI) for healthcare. The proposed system includes•An XAI model for cancer drug value prediction. The model provides data that is easy to understand and explain, which is critical for medical decision-making. It also produces accurate projections.•A model outperformed existing models due to extensive training and evaluation on a large cancer medication chemical compounds dataset.•Insights into the causation and correlation between the dependent and independent actors in the chemical composition of the cancer cell. While the model is evaluated on Lung Cancer data, the architecture offered in the proposed solution is cancer agnostic. It may be scaled out to other cancer cell data if the properties are similar. The work presents a viable route for customizing treatments and improving patient outcomes in oncology by combining XAI with a large dataset. This research attempts to create a framework where a user can upload a test case and receive forecasts with explanations, all in a portable PDF report.

Advanced interpretable diagnosis of Alzheimer's disease using SECNN-RF framework with explainable AI

Early detection of Alzheimer's disease (AD) is vital for effective treatment, as interventions are most successful in the disease's early stages. Combining Magnetic Resonance Imaging (MRI) with artificial intelligence (AI) offers significant potential for enhancing AD diagnosis. However, traditional AI models often lack transparency in their decision-making processes. Explainable Artificial Intelligence (XAI) is an evolving field that aims to make AI decisions understandable to humans, providing transparency and insight into AI systems. This research introduces the Squeeze-and-Excitation Convolutional Neural Network with Random Forest (SECNN-RF) framework for early AD detection using MRI scans. The SECNN-RF integrates Squeeze-and-Excitation (SE) blocks into a Convolutional Neural Network (CNN) to focus on crucial features and uses Dropout layers to prevent overfitting. It then employs a Random Forest classifier to accurately categorize the extracted features. The SECNN-RF demonstrates high accuracy (99.89%) and offers an explainable analysis, enhancing the model's interpretability. Further exploration of the SECNN framework involved substituting the Random Forest classifier with other machine learning algorithms like Decision Tree, XGBoost, Support Vector Machine, and Gradient Boosting. While all these classifiers improved model performance, Random Forest achieved the highest accuracy, followed closely by XGBoost, Gradient Boosting, Support Vector Machine, and Decision Tree which achieved lower accuracy.

Decoding the black box: Explainable AI (XAI) for cancer diagnosis, prognosis, and treatment planning-A state-of-the art systematic review

OBJECTIVE

Explainable Artificial Intelligence (XAI) is increasingly recognized as a crucial tool in cancer care, with significant potential to enhance diagnosis, prognosis, and treatment planning. However, the holistic integration of XAI across all stages of cancer care remains underexplored. This review addresses this gap by systematically evaluating the role of XAI in these critical areas, identifying key challenges and emerging trends.

MATERIALS AND METHODS

Following the PRISMA guidelines, a comprehensive literature search was conducted across Scopus and Web of Science, focusing on publications from January 2020 to May 2024. After rigorous screening and quality assessment, 69 studies were selected for in-depth analysis.

RESULTS

The review identified critical gaps in the application of XAI within cancer care, notably the exclusion of clinicians in 83% of studies, which raises concerns about real-world applicability and may lead to explanations that are technically sound but clinically irrelevant. Additionally, 87% of studies lacked rigorous evaluation of XAI explanations, compromising their reliability in clinical practice. The dominance of post-hoc visual methods like SHAP, LIME and Grad-CAM reflects a trend toward explanations that may be inherently flawed due to specific input perturbations and simplifying assumptions. The lack of formal evaluation metrics and standardization constrains broader XAI adoption in clinical settings, creating a disconnect between AI development and clinical integration. Moreover, translating XAI insights into actionable clinical decisions remains challenging due to the absence of clear guidelines for integrating these tools into clinical workflows.

CONCLUSION

This review highlights the need for greater clinician involvement, standardized XAI evaluation metrics, clinician-centric interfaces, context-aware XAI systems, and frameworks for integrating XAI into clinical workflows for informed clinical decision-making and improved outcomes in cancer care.

Identification of patients' smoking status using an explainable AI approach: a Danish electronic health records case study

BACKGROUND

Smoking is a critical risk factor responsible for over eight million annual deaths worldwide. It is essential to obtain information on smoking habits to advance research and implement preventive measures such as screening of high-risk individuals. In most countries, including Denmark, smoking habits are not systematically recorded and at best documented within unstructured free-text segments of electronic health records (EHRs). This would require researchers and clinicians to manually navigate through extensive amounts of unstructured data, which is one of the main reasons that smoking habits are rarely integrated into larger studies. Our aim is to develop machine learning models to classify patients' smoking status from their EHRs.

METHODS

This study proposes an efficient natural language processing (NLP) pipeline capable of classifying patients' smoking status and providing explanations for the decisions. The proposed NLP pipeline comprises four distinct components, which are; (1) considering preprocessing techniques to address abbreviations, punctuation, and other textual irregularities, (2) four cutting-edge feature extraction techniques, i.e. Embedding, BERT, Word2Vec, and Count Vectorizer, employed to extract the optimal features, (3) utilization of a Stacking-based Ensemble (SE) model and a Convolutional Long Short-Term Memory Neural Network (CNN-LSTM) for the identification of smoking status, and (4) application of a local interpretable model-agnostic explanation to explain the decisions rendered by the detection models. The EHRs of 23,132 patients with suspected lung cancer were collected from the Region of Southern Denmark during the period 1/1/2009-31/12/2018. A medical professional annotated the data into 'Smoker' and 'Non-Smoker' with further classifications as 'Active-Smoker', 'Former-Smoker', and 'Never-Smoker'. Subsequently, the annotated dataset was used for the development of binary and multiclass classification models. An extensive comparison was conducted of the detection performance across various model architectures.

RESULTS

The results of experimental validation confirm the consistency among the models. However, for binary classification, BERT method with CNN-LSTM architecture outperformed other models by achieving precision, recall, and F1-scores between 97% and 99% for both Never-Smokers and Active-Smokers. In multiclass classification, the Embedding technique with CNN-LSTM architecture yielded the most favorable results in class-specific evaluations, with equal performance measures of 97% for Never-Smoker and measures in the range of 86 to 89% for Active-Smoker and 91-92% for Never-Smoker.

CONCLUSION

Our proposed NLP pipeline achieved a high level of classification performance. In addition, we presented the explanation of the decision made by the best performing detection model. Future work will expand the model's capabilities to analyze longer notes and a broader range of categories to maximize its utility in further research and screening applications.