An interpretable deep-learning model for early prediction of sepsis in the emergency department

A machine learning framework for interpretable predictions in patient pathways: The case of predicting ICU admission for patients with symptoms of sepsis

Proactive analysis of patient pathways helps healthcare providers anticipate treatment-related risks, identify outcomes, and allocate resources. Machine learning (ML) can leverage a patient's complete health history to make informed decisions about future events. However, previous work has mostly relied on so-called black-box models, which are unintelligible to humans, making it difficult for clinicians to apply such models. Our work introduces PatWay-Net, an ML framework designed for interpretable predictions of admission to the intensive care unit (ICU) for patients with symptoms of sepsis. We propose a novel type of recurrent neural network and combine it with multi-layer perceptrons to process the patient pathways and produce predictive yet interpretable results. We demonstrate its utility through a comprehensive dashboard that visualizes patient health trajectories, predictive outcomes, and associated risks. Our evaluation includes both predictive performance - where PatWay-Net outperforms standard models such as decision trees, random forests, and gradient-boosted decision trees - and clinical utility, validated through structured interviews with clinicians. By providing improved predictive accuracy along with interpretable and actionable insights, PatWay-Net serves as a valuable tool for healthcare decision support in the critical case of patients with symptoms of sepsis.

Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis

Today's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that SepsisLab enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.

Prediction of Cancer Symptom Trajectory Using Longitudinal Electronic Health Record Data and Long Short-Term Memory Neural Network

PURPOSE

Ability to predict symptom severity and progression across treatment trajectories would allow clinicians to provide timely intervention and treatment planning. However, such predictions are difficult because of sparse and inconsistent assessment, and simplistic measures such as the last observed symptom severity are often used. The purpose of this study is to develop a model for predicting future cancer symptom experiences on the basis of past symptom experiences.

PATIENTS AND METHODS

We performed a retrospective, longitudinal analysis using records of patients with cancer (n = 208) hospitalized between 2008 and 2014. A long short-term memory (LSTM)-based recurrent neural network, a linear regression, and random forest models were trained on previous symptoms experienced and used to predict future symptom trajectories.

RESULTS

We found that at least one of three tested models (LSTM, linear regression, and random forest) outperform predictions based solely on the previous clinical observation. LSTM models significantly outperformed linear regression and random forest models in predicting nausea (P < .1) and psychosocial status (P < .01). Linear regression outperformed all models when predicting oral health (P < .01), while random forest outperformed all models when predicting mobility (P < .01) and nutrition (P < .01).

CONCLUSION

We can successfully predict patients' symptom trajectories with a prediction model, built with sparse assessment data, using routinely collected nursing documentation. The results of this project can be applied to better individualize symptom management to support cancer patients' quality of life.

Artificial intelligence in emergency medicine. A systematic literature review

Motivation and objective: Emergency medicine is becoming a popular application area for artificial intelligence methods but remains less investigated than other healthcare branches. The need for time-sensitive decision-making on the basis of high data volumes makes the use of quantitative technologies inevitable. However, the specifics of healthcare regulations impose strict requirements for such applications. Published contributions cover separate parts of emergency medicine and use disparate data and algorithms. This study aims to systematize the relevant contributions, investigate the main obstacles to artificial intelligence applications in emergency medicine, and propose directions for further studies.

METHODS

The contributions selection process was conducted with systematic electronic databases querying and filtering with respect to established exclusion criteria. Among the 380 papers gathered from IEEE Xplore, ACM Digital Library, Springer Library, ScienceDirect, and Nature databases 116 were considered to be a part of the survey. The main features of the selected papers are the focus on emergency medicine and the use of machine learning or deep learning algorithms.

FINDINGS AND DISCUSSION

The selected papers were classified into two branches: diagnostics-specific and triage-specific. The former ones are focused on either diagnosis prediction or decision support. The latter covers such applications as mortality, outcome, admission prediction, condition severity estimation, and urgent care prediction. The observed contributions are highly specialized within a single disease or medical operation and often use privately collected retrospective data, making them incomparable. These and other issues can be addressed by creating an end-to-end solution based on human-machine interaction.

CONCLUSION

Artificial intelligence applications are finding their place in emergency medicine, while most of the corresponding studies remain isolated and lack higher generalization and more sophisticated methodology, which can be a matter of forthcoming improvements.

An interpretable deep learning model for time-series electronic health records: Case study of delirium prediction in critical care

Deep Learning (DL) models have received increasing attention in the clinical setting, particularly in intensive care units (ICU). In this context, the interpretability of the outcomes estimated by the DL models is an essential step towards increasing adoption of DL models in clinical practice. To address this challenge, we propose an ante-hoc, interpretable neural network model. Our proposed model, named double self-attention architecture (DSA), uses two attention-based mechanisms, including self-attention and effective attention. It can capture the importance of input variables in general, as well as changes in importance along the time dimension for the outcome of interest. We evaluated our model using two real-world clinical datasets covering 22840 patients in predicting onset of delirium 12 h and 48 h in advance. Additionally, we compare the descriptive performance of our model with three post-hoc interpretable algorithms as well as with the opinion of clinicians based on the published literature and clinical experience. We find that our model covers the majority of the top-10 variables ranked by the other three post-hoc interpretable algorithms as well as the clinical opinion, with the advantage of taking into account both, the dependencies among variables as well as dependencies between varying time-steps. Finally, our results show that our model can improve descriptive performance without sacrificing predictive performance.

Personalized event prediction for Electronic Health Records

Clinical event sequences consist of hundreds of clinical events that represent records of patient care in time. Developing accurate predictive models of such sequences is of a great importance for supporting a variety of models for interpreting/classifying the current patient condition, or predicting adverse clinical events and outcomes, all aimed to improve patient care. One important challenge of learning predictive models of clinical sequences is their patient-specific variability. Based on underlying clinical conditions, each patient's sequence may consist of different sets of clinical events (observations, lab results, medications, procedures). Hence, simple population-wide models learned from event sequences for many different patients may not accurately predict patient-specific dynamics of event sequences and their differences. To address the problem, we propose and investigate multiple new event sequence prediction models and methods that let us better adjust the prediction for individual patients and their specific conditions. The methods developed in this work pursue refinement of population-wide models to subpopulations, self-adaptation, and a meta-level model switching that is able to adaptively select the model with the best chance to support the immediate prediction. We analyze and test the performance of these models on clinical event sequences of patients in MIMIC-III database.

Assessment of Prediction Tasks and Time Window Selection in Temporal Modeling of Electronic Health Record Data: a Systematic Review

Temporal electronic health record (EHR) data are often preferred for clinical prediction tasks because they offer more complete representations of a patient's pathophysiology than static data. A challenge when working with temporal EHR data is problem formulation, which includes defining the time windows of interest and the prediction task. Our objective was to conduct a systematic review that assessed the definition and reporting of concepts relevant to temporal clinical prediction tasks. We searched PubMed® and IEEE Xplore® databases for studies from January 1, 2010 applying machine learning models to EHR data for patient outcome prediction. Publications applying time-series methods were selected for further review. We identified 92 studies and summarized them by clinical context and definition and reporting of the prediction problem. For the time windows of interest, 12 studies did not discuss window lengths, 57 used a single set of window lengths, and 23 evaluated the relationship between window length and model performance. We also found that 72 studies had appropriate reporting of the prediction task. However, evaluation of prediction problem formulation for temporal EHR data was complicated by heterogeneity in assessing and reporting of these concepts. Even among studies modeling similar clinical outcomes, there were variations in terminology used to describe the prediction problem, rationale for window lengths, and determination of the outcome of interest. As temporal modeling using EHR data expands, minimal reporting standards should include time-series specific concerns to promote rigor and reproducibility in future studies and facilitate model implementation in clinical settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00143-4.

Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review

BACKGROUND

Sepsis, a life-threatening infection-induced inflammatory condition, has significant global health impacts. Timely detection is crucial for improving patient outcomes as sepsis can rapidly progress to severe forms. The application of machine learning (ML) and deep learning (DL) to predict sepsis using electronic health records (EHRs) has gained considerable attention for timely intervention.

METHODS

PubMed, IEEE Xplore, Google Scholar, and Scopus were searched for relevant studies. All studies that used ML/DL to detect or early-predict the onset of sepsis in the adult population using EHRs were considered. Data were extracted and analyzed from all studies that met the criteria and were also evaluated for their quality.

RESULTS

This systematic review examined 1942 articles, selecting 42 studies while adhering to strict criteria. The chosen studies were predominantly retrospective (n = 38) and spanned diverse geographic settings, with a focus on the United States. Different datasets, sepsis definitions, and prevalence rates were employed, necessitating data augmentation. Heterogeneous parameter utilization, diverse model distribution, and varying quality assessments were observed. Longitudinal data enabled early sepsis prediction, and quality criteria fulfillment varied, with inconsistent funding-article quality correlation.

CONCLUSIONS

This systematic review underscores the significance of ML/DL methods for sepsis detection and early prediction through EHR data.

Sepsis Prediction by Using a Hybrid Metaheuristic Algorithm: A Novel Approach for Optimizing Deep Neural Networks

The early diagnosis of sepsis reduces the risk of the patient's death. Gradient-based algorithms are applied to the neural network models used in the estimation of sepsis in the literature. However, these algorithms become stuck at the local minimum in solution space. In recent years, swarm intelligence and an evolutionary approach have shown proper results. In this study, a novel hybrid metaheuristic algorithm was proposed for optimization with regard to the weights of the deep neural network and applied for the early diagnosis of sepsis. The proposed algorithm aims to reach the global minimum with a local search strategy capable of exploring and exploiting particles in Particle Swarm Optimization (PSO) and using the mental search operator of the Human Mental Search algorithm (HMS). The benchmark functions utilized to compare the performance of HMS, PSO, and HMS-PSO revealed that the proposed approach is more reliable, durable, and adjustable than other applied algorithms. HMS-PSO is integrated with a deep neural network (HMS-PSO-DNN). The study focused on predicting sepsis with HMS-PSO-DNN, utilizing a dataset of 640 patients aged 18 to 60. The HMS-PSO-DNN model gave a better mean squared error (MSE) result than other algorithms in terms of accuracy, robustness, and performance. We obtained the MSE value of 0.22 with 30 independent runs.

Prediction of Acid-Base and Potassium Imbalances in Intensive Care Patients Using Machine Learning Techniques

Acid–base disorders occur when the body’s normal pH is out of balance. They can be caused by problems with kidney or respiratory function or by an excess of acids or bases that the body cannot properly eliminate. Acid–base and potassium imbalances are mechanistically linked because acid–base imbalances can alter the transport of potassium. Both acid–base and potassium imbalances are common in critically ill patients. This study investigated machine learning models for predicting the occurrence of acid–base and potassium imbalances in intensive care patients. We used an institutional dataset of 1089 patients with 87 variables, including vital signs, general appearance, and laboratory results. Gradient boosting (GB) was able to predict nine clinical conditions related to acid–base and potassium imbalances: mortality (AUROC = 0.9822), hypocapnia (AUROC = 0.7524), hypercapnia (AUROC = 0.8228), hypokalemia (AUROC = 0.9191), hyperkalemia (AUROC = 0.9565), respiratory acidosis (AUROC = 0.8125), respiratory alkalosis (AUROC = 0.7685), metabolic acidosis (AUROC = 0.8682), and metabolic alkalosis (AUROC = 0.8284). Some predictions remained relatively robust even when the prediction window was increased. Additionally, the decision-making process was made more interpretable and transparent through the use of SHAP analysis. Overall, the results suggest that machine learning could be a useful tool to gain insight into the condition of intensive care patients and assist in the management of acid–base and potassium imbalances.

End-to-end learning with interpretation on electrohysterography data to predict preterm birth

Prediction of preterm birth is a difficult task for clinicians. By examining an electrohysterogram, electrical activity of the uterus that can lead to preterm birth can be detected. Since signals associated with uterine activity are difficult to interpret for clinicians without a background in signal processing, machine learning may be a viable solution. We are the first to employ Deep Learning models, a long-short term memory and temporal convolutional network model, on electrohysterography data using the Term-Preterm Electrohysterogram database. We show that end-to-end learning achieves an AUC score of 0.58, which is comparable to machine learning models that use handcrafted features. Moreover, we evaluate the effect of adding clinical data to the model and conclude that adding the available clinical data to electrohysterography data does not result in a gain in performance. Also, we propose an interpretability framework for time series classification that is well-suited to use in case of limited data, as opposed to existing methods that require large amounts of data. Clinicians with extensive work experience as gynaecologist used our framework to provide insights on how to link our results to clinical practice and stress that in order to decrease the number of false positives, a dataset with patients at high risk of preterm birth should be collected. All code is made publicly available.

Early prediction of sepsis using double fusion of deep features and handcrafted features

Sepsis is a life-threatening medical condition that is characterized by the dysregulated immune system response to infections, having both high morbidity and mortality rates. Early prediction of sepsis is critical to the decrease of mortality. This paper presents a novel early warning model called Double Fusion Sepsis Predictor (DFSP) for sepsis onset. DFSP is a double fusion framework that combines the benefits of early and late fusion strategies. First, a hybrid deep learning model that combines both the convolutional and recurrent neural networks to extract deep features is proposed. Second, deep features and handcrafted features, such as clinical scores, are concatenated to build the joint feature representation (early fusion). Third, several tree-based models based on joint feature representation are developed to generate the risk scores of sepsis onset that are combined with an End-to-End neural network for final sepsis detection (late fusion). To evaluate DFSP, a retrospective study was conducted, which included patients admitted to the ICUs of a hospital in Shanghai China. The results demonstrate that the DFSP outperforms state-of-the-art approaches in early sepsis prediction.

Transferability and interpretability of the sepsis prediction models in the intensive care unit

BACKGROUND

We aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples.

METHODS

Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital.

RESULTS

The datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 - 0.98 and 0.95 - 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 - 0.86 and 0.84 - 0.87 respectively on HDRJH, from 1 to  5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 - 0.94 on HDRJH and to 0.86 - 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable.

CONCLUSIONS

Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.

TRIAL REGISTRATION

NCT05088850 (retrospectively registered).

A Machine Learning Model for Early Prediction and Detection of Sepsis in Intensive Care Unit Patients

In today's scenario, sepsis is impacting millions of patients in the intensive care unit due to the fact that the mortality rate is increased exponentially and has become a major challenge in the field of healthcare. Such peoples require determinant care which increases the cost of the treatment by using a large number of resources because of the nonavailability of the resources. The treatment of sepsis is available in the early state, but treatment is not started at the right time, and then it converts to the advanced level of sepsis and increases the fatalities. Thus, an intensive analysis is required to detect and identify sepsis at the early stage. There are some models available that work based on the manual score and based on only the biomark features, but these are not fully automated. Some machine learning-based models are also available, which can reduce the mortality rate, but accuracy is not up to date. This paper proposes a machine learning model for early detecting and predicting sepsis in intensive care unit patients. Various models, random forest (RF), linear regression (LR), support vector machine (SVM), naive Bayes (NB), ensemble (of SVM, RF, NB, and LR), XGBoost, and proposed ensemble (of SVM, RF, NB, LR, and XGBoost), are simulated by using the collected data from intensive care unit patient's database that is based on the clinical laboratory values and vital signs. The performance of the models is evaluated by considering the same datasets. The balanced accuracy of RF, LR, SVM, NB, ensemble (of SVM, RF, NB, and LR), XGBoost, and proposed ensemble (of SVM, RF, NB, LR, and XGBoost) is 0.90, 0.73, 0.93, 0.74, 0.94, 0.95, and 0.96, respectively. It is also evident from the experimental results that the proposed ensemble model performs well as compared to the other models.

Analyzing Patient Trajectories With Artificial Intelligence

In digital medicine, patient data typically record health events over time (eg, through electronic health records, wearables, or other sensing technologies) and thus form unique patient trajectories. Patient trajectories are highly predictive of the future course of diseases and therefore facilitate effective care. However, digital medicine often uses only limited patient data, consisting of health events from only a single or small number of time points while ignoring additional information encoded in patient trajectories. To analyze such rich longitudinal data, new artificial intelligence (AI) solutions are needed. In this paper, we provide an overview of the recent efforts to develop trajectory-aware AI solutions and provide suggestions for future directions. Specifically, we examine the implications for developing disease models from patient trajectories along the typical workflow in AI: problem definition, data processing, modeling, evaluation, and interpretation. We conclude with a discussion of how such AI solutions will allow the field to build robust models for personalized risk scoring, subtyping, and disease pathway discovery.

Predicting Hemodynamic Failure Development in PICU Using Machine Learning Techniques

The present work aims to identify the predictors of hemodynamic failure (HF) developed during pediatric intensive care unit (PICU) stay testing a set of machine learning techniques (MLTs), comparing their ability to predict the outcome of interest. The study involved patients admitted to PICUs between 2010 and 2020. Data were extracted from the Italian Network of Pediatric Intensive Care Units (TIPNet) registry. The algorithms considered were generalized linear model (GLM), recursive partition tree (RPART), random forest (RF), neural networks models, and extreme gradient boosting (XGB). Since the outcome is rare, upsampling and downsampling algorithms have been applied for imbalance control. For each approach, the main performance measures were reported. Among an overall sample of 29,494 subjects, only 399 developed HF during the PICU stay. The median age was about two years, and the male gender was the most prevalent. The XGB algorithm outperformed other MLTs in predicting HF development, with a median ROC measure of 0.780 (IQR 0.770-0.793). PIM 3, age, and base excess were found to be the strongest predictors of outcome. The present work provides insights for the prediction of HF development during PICU stay using machine-learning algorithms.