Explainable, trustworthy, and ethical machine learning for healthcare: A survey

With the advent of machine learning (ML) and deep learning (DL) empowered applications for critical applications like healthcare, the questions about liability, trust, and interpretability of their outputs are raising. The black-box nature of various DL models is a roadblock to clinical utilization. Therefore, to gain the trust of clinicians and patients, we need to provide explanations about the decisions of models. With the promise of enhancing the trust and transparency of black-box models, researchers are in the phase of maturing the field of eXplainable ML (XML). In this paper, we provided a comprehensive review of explainable and interpretable ML techniques for various healthcare applications. Along with highlighting security, safety, and robustness challenges that hinder the trustworthiness of ML, we also discussed the ethical issues arising because of the use of ML/DL for healthcare. We also describe how explainable and trustworthy ML can resolve all these ethical problems. Finally, we elaborate on the limitations of existing approaches and highlight various open research problems that require further development.

Explainable classifier for improving the accountability in decision-making for colorectal cancer diagnosis from histopathological images

Pathologists are responsible for cancer type diagnoses from histopathological cancer tissues. However, it is known that microscopic examination is tedious and time-consuming. In recent years, a long list of machine learning approaches to image classification and whole-slide segmentation has been developed to support pathologists. Although many showed exceptional performances, the majority of them are not able to rationalize their decisions. In this study, we developed an explainable classifier to support decision making for medical diagnoses. The proposed model does not provide an explanation about the causality between the input and the decisions, but offers a human-friendly explanation about the plausibility of the decision. Cumulative Fuzzy Class Membership Criterion (CFCMC) explains its decisions in three ways: through a semantical explanation about the possibilities of misclassification, showing the training sample responsible for a certain prediction and showing training samples from conflicting classes. In this paper, we explain about the mathematical structure of the classifier, which is not designed to be used as a fully automated diagnosis tool but as a support system for medical experts. We also report on the accuracy of the classifier against real world histopathological data for colorectal cancer. We also tested the acceptability of the system through clinical trials by 14 pathologists. We show that the proposed classifier is comparable to state of the art neural networks in accuracy, but more importantly it is more acceptable to be used by human experts as a diagnosis tool in the medical domain.

Importance of socioeconomic factors in predicting tooth loss among older adults in Japan: Evidence from a machine learning analysis

Prevalence of tooth loss has increased due to population aging. Tooth loss negatively affects the overall physical and social well-being of older adults. Understanding the role of socio-demographic and other predictors associated with tooth loss that are measured in non-clinical settings can be useful in community-level prevention. We used high-dimensional epidemiological data to investigate important factors in predicting tooth loss among older adults over a 6-year period of follow-up. Data was from participants of 2010 and 2016 waves of the Japan Gerontological Evaluation Study (JAGES). A total of 19,407 community-dwelling functionally independent older adults aged 65 and older were included in the analysis. Tooth loss was measured as moving from a higher number of teeth category at the baseline to a lower number of teeth category at the follow-up. Out of 119 potential predictors, age, sex, number of teeth, denture use, chewing difficulty, household income, employment, education, smoking, fruit and vegetable consumption, community participation, time since last health check-up, having a hobby, and feeling worthless were selected using Boruta algorithm. Within the 6-year follow-up, 3013 individuals (15.5%) reported incidence of tooth loss. People who experienced tooth loss were older (72.9 ± 5.2 vs 71.8 ± 4.7), and predominantly men (18.3% vs 13.1%). Extreme gradient boosting (XGBoost) machine learning prediction model had a mean accuracy of 90.5% (±0.9%). A visual analysis of machine learning predictions revealed that the prediction of tooth loss was mainly driven by demographic (older age), baseline oral health (having 10-19 teeth, wearing dentures), and socioeconomic (lower household income, manual occupations) variables. Predictors related to wide a range of determinants contribute towards tooth loss among older adults. In addition to oral health related and demographic factors, socioeconomic factors were important in predicting future tooth loss. Understanding the behaviour of these predictors can thus be useful in developing prevention strategies for tooth loss among older adults.

An explainable and interpretable model for attention deficit hyperactivity disorder in children using EEG signals

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder that affects a person's sleep, mood, anxiety, and learning. Early diagnosis and timely medication can help individuals with ADHD perform daily tasks without difficulty. Electroencephalogram (EEG) signals can help neurologists to detect ADHD by examining the changes occurring in it. The EEG signals are complex, non-linear, and non-stationary. It is difficult to find the subtle differences between ADHD and healthy control EEG signals visually. Also, making decisions from existing machine learning (ML) models do not guarantee similar performance (unreliable).

METHOD

The paper explores a combination of variational mode decomposition (VMD), and Hilbert transform (HT) called VMD-HT to extract hidden information from EEG signals. Forty-one statistical parameters extracted from the absolute value of analytical mode functions (AMF) have been classified using the explainable boosted machine (EBM) model. The interpretability of the model is tested using statistical analysis and performance measurement. The importance of the features, channels and brain regions has been identified using the glass-box and black-box approach. The model's local and global explainability has been visualized using Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), Partial Dependence Plot (PDP), and Morris sensitivity. To the best of our knowledge, this is the first work that explores the explainability of the model prediction in ADHD detection, particularly for children.

RESULTS

Our results show that the explainable model has provided an accuracy of 99.81%, a sensitivity of 99.78%, 99.84% specificity, an F-1 measure of 99.83%, the precision of 99.87%, a false detection rate of 0.13%, and Mathew's correlation coefficient, negative predicted value, and critical success index of 99.61%, 99.73%, and 99.66%, respectively in detecting the ADHD automatically with ten-fold cross-validation. The model has provided an area under the curve of 100% while the detection rate of 99.87% and 99.73% has been obtained for ADHD and HC, respectively.

CONCLUSIONS

The model show that the interpretability and explainability of frontal region is highest compared to pre-frontal, central, parietal, occipital, and temporal regions. Our findings has provided important insight into the developed model which is highly reliable, robust, interpretable, and explainable for the clinicians to detect ADHD in children. Early and rapid ADHD diagnosis using robust explainable technologies may reduce the cost of treatment and lessen the number of patients undergoing lengthy diagnosis procedures.

Analysis of mobile phone use engagement during naturalistic driving through explainable imbalanced machine learning

While driver distraction remains an issue in modernized societies, technological advancements in data collection, storage and analysis provide the means for deeper insights of this complex phenomenon. In this research, factors influencing when driver distraction through mobile phone use occurs during naturalistic driving are investigated. Naturalistic data from a 6-stage, 230-driver experiment are exploited, in which drivers installed a non-intrusive driving recording application in their devices and conducted their trips normally across a 21-month timespan, coupled with corresponding questionnaire data. The various experiment stages involved providing progressively more behavioral feedback to drivers while continuing to record them. Subsequently, supervised Machine Learning XGBoost algorithms were employed to model the contributions of naturalistic driving and questionnaire features to the decision to engage mobile phone use. Mobile phone use percentages were heavily skewed towards zero, therefore imbalanced ML with a minority-oversampling approach in a binary format was employed. To increase the explainability offered by the algorithm, SHAP values were calculated for the informative features. Results indicate that the decision of drivers to use a mobile while driving is governed by a number of complex, non-linear relationships. Total trip distance is the most significant predictor variable by a wide margin, with mean SHAP values of 0.79 towards affecting the model decisions for the probability of mobile phone use of each driver. However, other variables influence the final predictions as well, such as the number of tickets in the last three years (m.SHAP = 0.30), declared mobile phone use (m.SHAP = 0.26), the amount and variety of provided feedback (m.SHAP = 0.17) (i.e. experiment phase number) and family member numbers (m.SHAP = 0.09) decrease the probability of using a mobile phone while driving. Conversely, increases in driver experience (m.SHAP = 0.22), driver age (m.SHAP = 0.11), engine capacity (m.SHAP = 0.11) and total kilometers driven annually (m.SHAP = 0.08) increase the probability of using a mobile phone in naturalistic driving conditions. SHAP dependency plots reveal non-linear effects present in almost all variables. Fuel consumption had a particularly strong non-linear effect, as higher values of this variable lead to both higher and lower probability of drivers using a mobile phone, deviating from the safer average. Legislation, campaigns and enforcement measures can be restructured to take advantage of gains margins in terms of understanding and predicting driver distraction behavior, as explored in the present study.

An explainable knowledge distillation method with XGBoost for ICU mortality prediction

BACKGROUND AND OBJECTIVE

Mortality prediction is an important task in intensive care unit (ICU) for quantifying the severity of patients' physiological condition. Currently, scoring systems are widely applied for mortality prediction, while the performance is unsatisfactory in many clinical conditions due to the non-specificity and linearity characteristics of the used model. As the availability of the large volume of data recorded in electronic health records (EHRs), deep learning models have achieved state-of-art predictive performance. However, deep learning models are hard to meet the requirement of explainability in clinical conditions. Hence, an explainable Knowledge Distillation method with XGBoost (XGB-KD) is proposed to improve the predictive performance of XGBoost while supporting better explainability.

METHODS

In this method, we first use outperformed deep learning teacher models to learn the complex patterns hidden in high-dimensional multivariate time series data. Then, we distill knowledge from soft labels generated by the ensemble of teacher models to guide the training of XGBoost student model, whose inputs are meaningful features obtained from feature engineering. Finally, we conduct model calibration to obtain predicted probabilities reflecting the true posterior probabilities and use SHapley Additive exPlanations (SHAP) to obtain insights about the trained model.

RESULTS

We conduct comprehensive experiments on MIMIC-III dataset to evaluate our method. The results demonstrate that our method achieves better predictive performance than vanilla XGBoost, deep learning models and several state-of-art baselines from related works. Our method can also provide intuitive explanations.

CONCLUSIONS

Our method is useful for improving the predictive performance of XGBoost by distilling knowledge from deep learning models and can provide meaningful explanations for predictions.

Estimating daily air temperature and pollution in Catalonia: A comprehensive spatiotemporal modelling of multiple exposures

Environmental epidemiology studies require models of multiple exposures to adjust for co-exposure and explore interactions. We estimated spatiotemporal exposure to surface air temperature and pollution (PM2.5, PM10, NO2, O3) at high spatiotemporal resolution (daily, 250 m) for 2018-2020 in Catalonia. Innovations include the use of TROPOMI products, a data split for remote sensing gap-filling evaluation, estimation of prediction uncertainty, and use of explainable machine learning. We compiled meteorological and air quality station measurements, climate and atmospheric composition reanalyses, remote sensing products, and other spatiotemporal data. We performed gap-filling of remotely-sensed products using Random Forest (RF) models and validated them using Out-Of-Bag (OOB) samples and a structured data split. The exposure modelling workflow consisted of: 1) PM2.5 station imputation with PM10 data; 2) quantile RF (QRF) model fitting; and 3) geostatistical residual spatial interpolation. Prediction uncertainty was estimated using QRF. SHAP values were used to examine variable importance and the fitted relationships. Model performance was assessed via nested CV at the station level. Evaluation of the gap-filling models using the structured split showed error underestimation when using OOB. Temperature models had the best performance (R2 =0.98) followed by the gaseous air pollutants (R2 =0.81 for NO2 and 0.86 for O3), while the performance of the PM2.5 and PM10 models was lower (R2 =0.57 and 0.63 respectively). Predicted exposure patterns captured urban heat island effects, dust advection events, and NO2 hotspots. SHAP values estimated a high importance of TROPOMI tropospheric NO2 columns in PM and NO2 models, and confirmed that the fitted associations conformed to prior knowledge. Our work highlights the importance of correctly validating gap-filling models and the potential of TROPOMI measurements. Moderate performance in PM models can be partly explained by the poor station coverage. Our exposure estimates can be used in epidemiological studies potentially accounting for exposure uncertainty.

Diagnosing autism severity associated with physical fitness and gray matter volume in children with autism spectrum disorder: Explainable machine learning method

PURPOSE

This study aimed to investigate the relationship between physical fitness, gray matter volume (GMV), and autism severity in children with autism spectrum disorder (ASD). Besides, we sought to diagnose autism severity associated with physical fitness and GMV using machine learning methods.

METHODS

Ninety children diagnosed with ASD underwent physical fitness tests, magnetic resonance imaging scans, and autism severity assessments. Diagnosis models were established using extreme gradient boosting (XGB), random forest (RF), support vector machine (SVM), and decision tree (DT) algorithms. Hyperparameters were optimized through the grid search cross-validation method. The shapley additive explanation (SHAP) method was employed to explain the diagnosis results.

RESULTS

Our study revealed associations between muscular strength in physical fitness and GMV in specific brain regions (left paracentral lobule, bilateral thalamus, left inferior temporal gyrus, and cerebellar vermis I-II) with autism severity in children with ASD. The accuracy (95 % confidence interval) of the XGB, RF, SVM, and DT models were 77.9 % (77.3, 78.6 %), 72.4 % (71.7, 73.2 %), 71.9 % (71.1, 72.6 %), and 66.9 % (66.2, 67.7 %), respectively. SHAP analysis revealed that muscular strength and thalamic GMV significantly influenced the decision-making process of the XGB model.

CONCLUSION

Machine learning methods can effectively diagnose autism severity associated with physical fitness and GMV in children with ASD. In this respect, the XGB model demonstrated excellent performance across various indicators, suggesting its potential for diagnosing autism severity.

Deep Learning Based Prediction of Pulmonary Hypertension in Newborns Using Echocardiograms

Pulmonary hypertension (PH) in newborns and infants is a complex condition associated with several pulmonary, cardiac, and systemic diseases contributing to morbidity and mortality. Thus, accurate and early detection of PH and the classification of its severity is crucial for appropriate and successful management. Using echocardiography, the primary diagnostic tool in pediatrics, human assessment is both time-consuming and expertise-demanding, raising the need for an automated approach. Little effort has been directed towards automatic assessment of PH using echocardiography, and the few proposed methods only focus on binary PH classification on the adult population. In this work, we present an explainable multi-view video-based deep learning approach to predict and classify the severity of PH for a cohort of 270 newborns using echocardiograms. We use spatio-temporal convolutional architectures for the prediction of PH from each view, and aggregate the predictions of the different views using majority voting. Our results show a mean F1-score of 0.84 for severity prediction and 0.92 for binary detection using 10-fold cross-validation and 0.63 for severity prediction and 0.78 for binary detection on the held-out test set. We complement our predictions with saliency maps and show that the learned model focuses on clinically relevant cardiac structures, motivating its usage in clinical practice. To the best of our knowledge, this is the first work for an automated assessment of PH in newborns using echocardiograms.

Characterization of attentional event-related potential from REM sleep behavior disorder patients based on explainable machine learning

BACKGROUND AND OBJECTIVE

Idiopathic rapid eye movement sleep behavior disorder (iRBD) is a prodromal stage of neurodegeneration and is associated with cortical dysfunction. The purpose of this study was to investigate the spatiotemporal characteristics of cortical activities underlying impaired visuospatial attention in iRBD patients using an explainable machine-learning approach.

METHODS

An algorithm based on a convolutional neural network (CNN) was devised to discriminate cortical current source activities of iRBD patients due to single-trial event-related potentials (ERPs), from those of normal controls. The ERPs from 16 iRBD patients and 19 age- and sex-matched normal controls were recorded while the subjects were performing visuospatial attentional task, and converted to two-dimensional images representing current source densities on flattened cortical surface. The CNN classifier was trained based on overall data, and then, a transfer learning approach was applied for the fine-tuning to each patient.

RESULTS

The trained classifier yielded high classification accuracy. The critical features for the classification were determined by layer-wise relevance propagation, so that the spatiotemporal characteristics of cortical activities that were most relevant to cognitive impairment in iRBD were revealed.

CONCLUSIONS

These results suggest that the recognized dysfunction in visuospatial attention of iRBD patients originates from neural activity impairment in relevant cortical regions and may contribute to the development of useful iRBD biomarkers based on neural activity.

Real-time streamflow forecasting in a reservoir-regulated river basin using explainable machine learning and conceptual reservoir module

Real-time streamflow forecasting is essential to manage water resources effectively in a reservoir-regulated basin. However, forecasting becomes challenging without weather and upstream reservoir outflows forecasts in real-time. In this context, a novel hybrid approach is proposed in this study to forecast the streamflows and reservoir outflows in real-time. In this approach, the Explainable Machine Learning model is embedded with a conceptual reservoir module for forecasting streamflows using short-term weather forecasts. Long Short Term Memory (LSTM), a Machine Learning model, is used in this study to predict the streamflow, and the model's explainability is examined by Shapley additive explanations method (SHAP). Panchet reservoir catchment, which contains Tenughat and Konar reservoirs, is selected as a study area. The LSTM model performance is excellent in predicting the streamflows of Tenughat, Konar and Panchet catchments with NSE values of 0.93, 0.87, and 0.96, respectively. The SHAP method identified the high-impact variables as streamflows and precipitation of 1-day lag. In forecasting, bias-corrected Global Forecast System data is used with the LSTM model to forecast the streamflows in three catchments. The inflows are forecasted well up to a 3-day lead in Tenughat and Konar reservoirs with NSE values above 0.88 and 0.87, respectively. The reservoir module performance in forecasting Tenughat and Konar reservoirs' outflows with the inflow forecasts is also promising up to a 3-day lead with NSE values above 0.88 for both reservoirs. The inflows forecasting to Panchet reservoir with reservoirs' outflows as additional inputs is excellent up to 5-day lead (NSE = 0.96-0.88). However, the forecasting error increased from 77 m³/s to 134 m³/s with the lead time. This approach could provide an efficient way to reduce flood risks in the reservoir-regulated basin.

Characterising paediatric mortality during and after acute illness in Sub-Saharan Africa and South Asia: a secondary analysis of the CHAIN cohort using a machine learning approach

BACKGROUND

A better understanding of which children are likely to die during acute illness will help clinicians and policy makers target resources at the most vulnerable children. We used machine learning to characterise mortality in the 30-days following admission and the 180-days after discharge from nine hospitals in low and middle-income countries (LMIC).

METHODS

A cohort of 3101 children aged 2-24 months were recruited at admission to hospital for any acute illness in Bangladesh (Dhaka and Matlab Hospitals), Pakistan (Civil Hospital Karachi), Kenya (Kilifi, Mbagathi, and Migori Hospitals), Uganda (Mulago Hospital), Malawi (Queen Elizabeth Central Hospital), and Burkina Faso (Banfora Hospital) from November 2016 to January 2019. To record mortality, children were observed during their hospitalisation and for 180 days post-discharge. Extreme gradient boosted models of death within 30 days of admission and mortality in the 180 days following discharge were built. Clusters of mortality sharing similar characteristics were identified from the models using Shapley additive values with spectral clustering.

FINDINGS

Anthropometric and laboratory parameters were the most influential predictors of both 30-day and post-discharge mortality. No WHO/IMCI syndromes were among the 25 most influential mortality predictors of mortality. For 30-day mortality, two lower-risk clusters (N = 1915, 61%) included children with higher-than-average anthropometry (1% died, 95% CI: 0-2), and children without signs of severe illness (3% died, 95% CI: 2-4%). The two highest risk 30-day mortality clusters (N = 118, 4%) were characterised by high urea and creatinine (70% died, 95% CI: 62-82%); and nutritional oedema with low platelets and reduced consciousness (97% died, 95% CI: 92-100%). For post-discharge mortality risk, two low-risk clusters (N = 1753, 61%) were defined by higher-than-average anthropometry (0% died, 95% CI: 0-1%), and gastroenteritis with lower-than-average anthropometry and without major laboratory abnormalities (0% died, 95% CI: 0-1%). Two highest risk post-discharge clusters (N = 267, 9%) included children leaving against medical advice (30% died, 95% CI: 25-37%), and severely-low anthropometry with signs of illness at discharge (46% died, 95% CI: 34-62%).

INTERPRETATION

WHO clinical syndromes are not sufficient at predicting risk. Integrating basic laboratory features such as urea, creatinine, red blood cell, lymphocyte and platelet counts into guidelines may strengthen efforts to identify high-risk children during paediatric hospitalisations.

FUNDING

Bill & Melinda Gates FoundationOPP1131320.

Datasets of narrow thermal hysteresis behaviour Ti-Ni-based HT-SMAs and the predicted accumulated local effects

This article refers to data derived from a research article entitled "Prediction of narrow HT-SMA thermal hysteresis behaviour using explainable machine learning" [1]. It is based on the knowledge that alloying Ti-Ni-based shape memory alloys (SMAs) with additional ternary or multicomponent elements can alter the SMAs' characteristic transformation temperatures, including the thermal hystereses. Two datasets are reported. The first and primary dataset documents experimental Ti-Ni-based shape memory alloys' high-transformation temperature characteristics reported in the literature. The second auxiliary dataset presented in this article was obtained following the explainable prediction of the narrow high-temperature thermal hysteresis behaviour in Ti-Ni-based high-transformation temperature SMAs (HT-SMAs). The second dataset is intended to generalise and summarise the ML prediction and visualisation of the thermal hysteresis behaviour as also observed experimentally in multiple reports elsewhere. The datasets are provided as supplementary files and the second dataset is also visualised as an intuitive marginal effects plot. We believe that these data will find applications in advancing experimental and theoretical HT-SMA research.

Estimating visibility and understanding factors influencing its variations at Bangkok airport using machine learning and a game theory-based approach

In this study, six individual machine learning (ML) models and a stacked ensemble model (SEM) were used for daytime visibility estimation at Bangkok airport during the dry season (November-April) for 2017-2022. The individual ML models are random forest, adaptive boosting, gradient boosting, extreme gradient boosting, light gradient boosting machine, and cat boosting. The SEM was developed by the combination of outputs from the individual models. Furthermore, the impact of factors affecting visibility was examined using the Shapley Additive exPlanation (SHAP) method, an interpretable ML technique inspired by the game theory-based approach. The predictor variables include different air pollutants, meteorological variables, and time-related variables. The light gradient boosting machine model is identified as the most effective individual ML model. On an hourly time scale, it showed the best performance across three out of four metrics with the ρ = 0.86, MB = 0, ME = 0.48 km (second lowest), and RMSE = 0.8 km. On a daily time scale, the model performed the best for all evaluation metrics with ρ = 0.92, MB = 0.0 km, ME = 0.3 km, and RMSE = 0.43 km. The SEM outperformed all the individual models across three out of four metrics on an hourly time scale with ρ = 0.88, MB = 0.0 km, (second lowest), and RMSE = 0.75 km. On the daily scale, it performed the best with ρ = 0.93, MB = 0.02 km, ME = 0.27 km, and RMSE = 0.4 km. The seasonal average original (VISorig) and meteorologically normalized visibility (VISnorm) decrease from 2017 to 2021 but increase in 2022. The rate of decrease in VISorig is double than rate of decrease in VISnorm which suggests the effect of meteorology visibility degradation. The SHAP analysis identified relative humidity (RH), PM2.5, PM10, day of the season year (i.e., Julian day) (JD), and O3 as the most important variables affecting visibility. At low RH, visibility is not sensitive to changes in RH. However, beyond a threshold, a negative correlation between RH and visibility is found potentially due to the hygroscopic growth of aerosols. The dependence of the Shapley values of PM2.5 and PM10 on RH and the change in average visibilities under different RH intervals also suggest the effect of hygroscopic growth of aerosol on visibility. A negative relationship has been identified between visibility and both PM2.5 and PM10. Visibility is positively correlated with O3 at lower to moderate concentrations, with diminishing impact at very high concentrations. The JD is strongly negatively related to visibility during winter while weakly associated positively later in summer. Findings from this research suggest the feasibility of employing machine learning techniques for predicting visibility and understanding the factors influencing its fluctuations.

Explainable deep learning model to predict invasive bacterial infection in febrile young infants: A retrospective study

BACKGROUND

Machine learning models have demonstrated superior performance in predicting invasive bacterial infection (IBI) in febrile infants compared to commonly used risk stratification criteria in recent studies. However, the black-box nature of these models can make them difficult to apply in clinical practice. In this study, we developed and validated an explainable deep learning model that can predict IBI in febrile infants ≤ 60 days of age visiting the emergency department.

METHODS

We conducted a retrospective study of febrile infants aged ≤ 60 days who presented to the pediatric emergency department of a medical center in Taiwan between January 1, 2011 and December 31, 2019. Patients with uncertain test results and complex chronic health conditions were excluded. IBI was defined as the growth of a pathogen in the blood or cerebrospinal fluid. We used a deep neural network to develop a predictive model for IBI and compared its performance to the IBI score and step-by-step approach. The SHapley Additive Explanations (SHAP) technique was used to explain the model's predictions at different levels.

RESULTS

Our study included 1847 patients, 53 (2.7%) of whom had IBI. The deep learning model performed similarly to the IBI score and step-by-step approach in terms of sensitivity and negative predictive value, but provided better specificity (54%), positive predictive value (5%), and area under the receiver-operating characteristic curve (0.87). SHapley Additive exPlanations identified five influential predictive variables (absolute neutrophil count, body temperature, heart rate, age, and C-reactive protein).

CONCLUSION

We have developed an explainable deep learning model that can predict IBI in febrile infants aged 0-60 days. The model not only performs better than previous scoring systems, but also provides insight into how it arrives at its predictions through individual features and cases.

Reliable prediction of difficult airway for tracheal intubation from patient preoperative photographs by machine learning methods

BACKGROUND

Estimating the risk of a difficult tracheal intubation should help clinicians in better anaesthesia planning, to maximize patient safety. Routine bedside screenings suffer from low sensitivity.

OBJECTIVE

To develop and evaluate machine learning (ML) and deep learning (DL) algorithms for the reliable prediction of intubation risk, using information about airway morphology.

METHODS

Observational, prospective cohort study enrolling n=623 patients who underwent tracheal intubation: 53/623 difficult cases (prevalence 8.51%). First, we used our previously validated deep convolutional neural network (DCNN) to extract 2D image coordinates for 27 + 13 relevant anatomical landmarks in two preoperative photos (frontal and lateral views). Here we propose a method to determine the 3D pose of the camera with respect to the patient and to obtain the 3D world coordinates of these landmarks. Then we compute a novel set of dM=59 morphological features (distances, areas, angles and ratios), engineered with our anaesthesiologists to characterize each individual's airway anatomy towards prediction. Subsequently, here we propose four ad hoc ML pipelines for difficult intubation prognosis, each with four stages: feature scaling, imputation, resampling for imbalanced learning, and binary classification (Logistic Regression, Support Vector Machines, Random Forests and eXtreme Gradient Boosting). These compound ML pipelines were fed with the dM=59 morphological features, alongside dD=7 demographic variables. Here we trained them with automatic hyperparameter tuning (Bayesian search) and probability calibration (Platt scaling). In addition, we developed an ad hoc multi-input DCNN to estimate the intubation risk directly from each pair of photographs, i.e. without any intermediate morphological description. Performance was evaluated using optimal Bayesian decision theory. It was compared against experts' judgement and against state-of-the-art methods (three clinical formulae, four ML, four DL models).

RESULTS

Our four ad hoc ML pipelines with engineered morphological features achieved similar discrimination capabilities: median AUCs between 0.746 and 0.766. They significantly outperformed both expert judgement and all state-of-the-art methods (highest AUC at 0.716). Conversely, our multi-input DCNN yielded low performance due to overfitting. This same behaviour occurred for the state-of-the-art DL algorithms. Overall, the best method was our XGB pipeline, with the fewest false negatives at the optimal Bayesian decision threshold.

CONCLUSIONS

We proposed and validated ML models to assist clinicians in anaesthesia planning, providing a reliable calibrated estimate of airway intubation risk, which outperformed expert assessments and state-of-the-art methods. Our novel set of engineered features succeeded in providing informative descriptions for prognosis.

Exploring the impact of socioeconomic and natural factors on pulmonary tuberculosis incidence in China (2013-2019) using explainable machine learning: A nationwide study

Pulmonary tuberculosis (PTB) stands as a significant and prevalent infectious disease in China. Integrating 13 natural and socioeconomic factors, we conduct nine machine learning (ML) models alongside the Tree-Structured Parzen Estimator to predict the monthly PTB incidence rate from 2013 to 2019 in mainland China. With explainable ML techniques, our research highlights that population size, per capita GDP, and PM10 concentration emerge as the primary determinants influencing the PTB incidence rate. We delineate both the independent and interactive impacts of these factors on the PTB incidence rate. Furthermore, crucial thresholds associated with factors influencing the PTB incidence rate are identified. Taking factors that have a positive effect on reducing the incidence rate of PTB as an example, the thresholds at which the effects of factors PM2.5, PM10, O3, and RH on the incidence rate change from increase to decrease are 105.5 µg/m3, 75.5 µg/m3, 90.8 µg/m3, and 72.3 % respectively. Our work will contribute valuable insights for public health interventions.

Application of explainable machine learning in the production of pullulan by Aureobasidium pullulans CGMCCNO.7055

The application of machine learning in pullulan biofermentation has demonstrated significant potential. Explainable machine learning enhances model transparency and interpretability by revealing the relationships between variables. In this study, we compared the predictive performance of six machine learning models. The Categorical Boosting (CatBoost) model demonstrated the best fit for biomass and pullulan molecular weight, while eXtreme Gradient Boosting (XGBoost) excelled in predicting pullulan production. Additionally, feature importance and SHapley Additive exPlanations (SHAP) analyses visualized the complex relationships between medium conditions and objectives. Yeast extract emerged as the most influential factor for all three targets. Meanwhile, NaCl and initial pH showed potential in regulating pullulan production and molecular weight, respectively. Finally, optimal medium conditions for maximizing biomass, pullulan production, and molecular weight were determined using the Non-dominated Sorting Genetic Algorithm III (NSGA-III) algorithm, achieving a maximum integrated optimization rate of 275.08 % (calculated as the average of improvements across the three objectives). This study effectively expands the application of the NSGA-III algorithm in multi-objective optimization for pullulan production. These findings contribute to advancing the application of explainable machine learning and advanced intelligent algorithms in the field of pullulan production.

Socioexposomics of COVID-19 across New Jersey: a comparison of geostatistical and machine learning approaches

BACKGROUND

Disparities in adverse COVID-19 health outcomes have been associated with multiple social and environmental stressors. However, research is needed to evaluate the consistency and efficiency of methods for studying these associations at local scales.

OBJECTIVE

To assess socioexposomic associations with COVID-19 outcomes across New Jersey and evaluate consistency of findings from multiple modeling approaches.

METHODS

We retrieved data for COVID-19 cases and deaths for the 565 municipalities of New Jersey up to the end of the first phase of the pandemic, and calculated mortality rates with and without long-term-care (LTC) facility deaths. We considered 84 spatially heterogeneous environmental, demographic and socioeconomic factors from publicly available databases, including air pollution, proximity to industrial sites/facilities, transportation-related noise, occupation and commuting, neighborhood and housing characteristics, age structure, racial/ethnic composition, poverty, etc. Six geostatistical models (Poisson/Negative-Binomial regression, Poison/Negative-Binomial mixed effect model, Poisson/Negative-Binomial Bersag-York-Mollie spatial model) and two Machine Learning (ML) methods (Random Forest, Extreme Gradient Boosting) were implemented to assess association patterns. The Shapley effects plot was established for explainable ML and change of support validation was introduced to compare performances of different approaches.

RESULTS

We found robust positive associations of COVID-19 mortality with historic exposures to NO2, population density, percentage of minority and below high school education, and other social and environmental factors. Exclusion of LTC deaths does not significantly affect correlations for most factors but findings can be substantially influenced by model structures and assumptions. The best performing geostatistical models involved flexible structures representing data variations. ML methods captured association patterns consistent with the best performing geostatistical models, and furthermore detected consistent nonlinear associations not captured by geostatistical models.

SIGNIFICANCE

The findings of this work improve the understanding of how social and environmental disparities impacted COVID-19 outcomes across New Jersey.

Explainable 18F-FDG PET/CT radiomics model for predicting EGFR mutation status in lung adenocarcinoma: a two-center study

PURPOSE

To establish an explainable 18F-FDG PET/CT-derived prediction model to identify EGFR mutation status and subtypes (EGFR wild, EGFR-E19, and EGFR-E21) in lung adenocarcinoma (LUAD).

METHODS

Baseline 18F-FDG PET/CT images of 478 patients with LUAD from 2 hospitals were collected. Data from hospital A (n = 390) was randomly split into a training group (n = 312) and an internal test group (n = 78), with data from hospital B (n = 88) utilized for external test. Further, a total of 4,760 handcrafted radiomics features (HRFs) were extracted from PET/CT scans. Candidates for the prediction model were constructed by cross-combinations of 11 feature selection methods and 7 classifiers. The optimal model was determined by combining the results of cross-center data validation and model visualization (Yellowbrick). The predictive performance was assessed via receiver operating characteristic curve, confusion matrix and classification report. Four explainable artificial intelligence technologies were used for optimal model interpretation.

RESULTS

Sex and SUVmax were selected as clinical risk factors, which were then combined with 8 robust PET/CT HRFs to establish the models. The optimal performance was obtained by combining a light gradient boosting machine classifier with random forest feature selection method achieving an optimal performance with a macro-average AUC of 0.75 in the internal test group and 0.81 in the external test group.

CONCLUSION

The explainable EGFR mutation status prediction model have certain clinical practicability and good generalization performance, which may help in the timely selection of treatment options and prognosis prediction in patients with LUAD.

Assessing chemical exposure risk in breastfeeding infants: An explainable machine learning model for human milk transfer prediction

Breast milk is essential for infant health, but the transfer of xenobiotic chemicals poses significant risks. Ethical challenges in clinical trials necessitate the use of in vitro predictive models to assess chemical exposure risks in breastfeeding infants. This study introduces an explainable machine learning model to predict the risk of chemical transfer through human milk. Our novel framework integrates ensemble resampling methods with advanced feature selection techniques, addressing data imbalance and enhancing predictive accuracy. The balanced random forest classifier, optimized using the genetic algorithm for feature selection, achieved an area under the receiver operating characteristic curve (AUC) of 0.8708 and an accuracy of 82.67 % on the internal test set, with an accuracy of 86.36 % on the external validation set. The integration of the SHapley Additive exPlanations approach provided deeper insights by revealing how specific chemical properties influence the transfer of high-risk compounds into breast milk. This enhanced interpretability offers a clearer understanding of the associated risks and informs strategies for their mitigation. Structural alert analysis further identified molecular fragments linked to high-risk chemicals, enabling targeted risk assessments. Additionally, the model was applied to evaluate the transfer risks of FDA-approved drugs from 2019 to 2024, identifying several with high transfer probabilities. To broaden its application, we developed an online prediction tool that offers real-time risk assessments, providing an accessible resource for healthcare professionals and researchers. These contributions present a robust, ethically sound tool for assessing chemical exposure risks in breastfeeding infants, supporting informed decisions on drug use and environmental contaminant exposure.

From physical activity patterns to cognitive status: development and validation of novel digital biomarkers for cognitive assessment in older adults

BACKGROUND

This study aims to investigate the associations between signal-level physical activity (PA) features derived from wrist accelerometry data and cognitive status in older adults, and to evaluate their potential predictive value when combined with demographics.

METHODS

We analyzed PA data from 3,363 older adults (NHATS: n = 747; NHANES: n = 2,616), with each participant contributing a complete 3-day continuous activity sequence. We extracted the most relevant PA features associated with cognitive function using feature engineering and recursive feature elimination. Demographic characteristics were also incorporated, and multimodal data fusion was achieved through canonical correlation analysis. We then developed explainable machine learning models, primarily random forest, optimized with hyperparameters, to predict individual cognitive function status.

RESULTS

Using recursive feature elimination, we identified the top 20 PA features from each dataset and combined them with demographic features for modeling. The models achieved AUCs of 0.84 and 0.80 for NHATS and NHANES. Change quantiles and FFT coefficients emerged as the consistently top-ranked PA features across datasets, ranking 1st and 2nd respectively in their predictive importance for cognitive function. Models based on the top 10 PA features common to both datasets, along with demographic features, achieved AUCs above 0.8.

CONCLUSIONS

This study identifies novel time-frequency domain features of physical activity that show robust associations with cognitive status across two independent cohorts. These features, particularly those capturing activity variability and rhythmicity, provide complementary information beyond traditional cumulative PA measures. Based on these findings, we developed a proof-of-concept application that demonstrates the feasibility of translating these PA analytics into practical monitoring tools in real-world settings.

Application of explainable machine learning for estimating direct and diffuse components of solar irradiance

The inclusion of diffuse horizontal irradiance (DHI) and direct normal irradiance (DNI) is crucial in the context of solar energy applications. However, most solar irradiance instruments primarily prioritize the measurement of global horizontal irradiance (GHI) due to the high cost associated with devices used to measure DNI and DHI. Hence, numerous prior works have investigated various solar decomposition models aimed at computing direct and diffuse irradiance from GHI. The present study introduces a novel separation approach for direct and diffuse irradiance, employing machine learning algorithms and utilizing data with a temporal resolution of 1 min. Three machine learning models utilizing the gradient boost technique are suggested and trained using data collected from 10 stations across the world with different climate conditions. The machine learning model called CatBoost outperforms all the solar decomposition models at every station. It achieves the lowest root mean squared error (RMSE) of 8.73% when calculating DNI. The concept of explainable machine learning is further explored through the utilization of shapley additive explanations (SHAP), which allows for the assessment of the significance and interaction of the input parameters. In summary, the results of this study reveal that humidity is an important parameter for the estimation of DNI and DHI.

Neonatal brain inflammation enhances methamphetamine-induced reinstated behavioral sensitization in adult rats analyzed with explainable machine learning

Neonatal brain inflammation produced by intraperitoneal (i.p.) injection of lipopolysaccharide (LPS) results in long-lasting brain dopaminergic injury and motor disturbances in adult rats. The goal of the present work is to investigate the effect of neonatal systemic LPS exposure (1 or 2 mg/kg, i.p. injection in postnatal day 5, P5, male rats)-induced dopaminergic injury to examine methamphetamine (METH)-induced behavioral sensitization as an indicator of drug addiction. On P70, subjects underwent a treatment schedule of 5 once daily subcutaneous (s.c.) administrations of METH (0.5 mg/kg) (P70-P74) to induce behavioral sensitization. Ninety-six hours following the 5th treatment of METH (P78), the rats received one dose of 0.5 mg/kg METH (s.c.) to reintroduce behavioral sensitization. Hyperlocomotion is a critical index caused by drug abuse, and METH administration has been shown to produce remarkable locomotor-enhancing effects. Therefore, a random forest model was used as the detector to extract the feature interaction patterns among the collected high-dimensional locomotor data. Our approaches identified neonatal systemic LPS exposure dose and METH-treated dates as features significantly associated with METH-induced behavioral sensitization, reinstated behavioral sensitization, and perinatal inflammation in this experimental model of drug addiction. Overall, the analysis suggests that the implementation of machine learning strategies is sensitive enough to detect interaction patterns in locomotor activity. Neonatal LPS exposure also enhanced METH-induced reduction of dopamine transporter expression and [3H]dopamine uptake, reduced mitochondrial complex I activity, and elevated interleukin-1β and cyclooxygenase-2 concentrations in the P78 rat striatum. These results indicate that neonatal systemic LPS exposure produces a persistent dopaminergic lesion leading to a long-lasting change in the brain reward system as indicated by the enhanced METH-induced behavioral sensitization and reinstated behavioral sensitization later in life. These findings indicate that early-life brain inflammation may enhance susceptibility to drug addiction development later in life, which provides new insights for developing potential therapeutic treatments for drug addiction.

Physiological variables in machine learning QSARs allow for both cross-chemical and cross-species predictions

A major challenge in ecological risk assessment is estimating chemical-induced effects across taxa without species-specific testing. Where ecotoxicological data may be more challenging to gather, information on species physiology is more available for a broad range of taxa. Physiology is known to drive species sensitivity but understanding about the relative contribution of specific underlying processes is still elusive. Consequently, there remains a need to understand which physiological processes lead to differences in species sensitivity. The objective of our study was to utilize existing knowledge about organismal physiology to both understand and predict differences in species sensitivity. Machine learning models were trained to predict chemical- and species-specific endpoints as a function of both chemical fingerprints/descriptors and physiological properties represented by dynamic energy budget (DEB) parameters. We found that random forest models were able to predict chemical- and species-specific endpoints, and that DEB parameters were relatively important in the models, particularly for invertebrates. Our approach illuminates how physiological properties may drive species sensitivity, which will allow more realistic predictions of effects across species without the need for additional animal testing.