Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

A scoping review of machine learning for sepsis prediction- feature engineering strategies and model performance: a step towards explainability

BACKGROUND

Sepsis, an acute and potentially fatal systemic response to infection, significantly impacts global health by affecting millions annually. Prompt identification of sepsis is vital, as treatment delays lead to increased fatalities through progressive organ dysfunction. While recent studies have delved into leveraging Machine Learning (ML) for predicting sepsis, focusing on aspects such as prognosis, diagnosis, and clinical application, there remains a notable deficiency in the discourse regarding feature engineering. Specifically, the role of feature selection and extraction in enhancing model accuracy has been underexplored.

OBJECTIVES

This scoping review aims to fulfill two primary objectives: To identify pivotal features for predicting sepsis across a variety of ML models, providing valuable insights for future model development, and To assess model efficacy through performance metrics including AUROC, sensitivity, and specificity.

RESULTS

The analysis included 29 studies across diverse clinical settings such as Intensive Care Units (ICU), Emergency Departments, and others, encompassing 1,147,202 patients. The review highlighted the diversity in prediction strategies and timeframes. It was found that feature extraction techniques notably outperformed others in terms of sensitivity and AUROC values, thus indicating their critical role in improving sepsis prediction models.

CONCLUSION

Key dynamic indicators, including vital signs and critical laboratory values, are instrumental in the early detection of sepsis. Applying feature selection methods significantly boosts model precision, with models like Random Forest and XG Boost showing promising results. Furthermore, Deep Learning models (DL) reveal unique insights, spotlighting the pivotal role of feature engineering in sepsis prediction, which could greatly benefit clinical practice.

Use of Artificial Intelligence in Triage in Hospital Emergency Departments: A Scoping Review

The integration of artificial intelligence (AI) and machine learning (ML) in healthcare has become a major point of interest and raises the question of its impact on the emergency department (ED) triaging process. AI's capacity to emulate human cognitive processes coupled with advancements in computing has shown positive outcomes in various aspects of healthcare but little is known about the use of AI in triaging patients in ED. AI algorithms may allow for earlier diagnosis and intervention; however, overconfident answers may present dangers to patients. The purpose of this review was to explore comprehensively recently published literature regarding the effect of AI and ML in ED triage and identify research gaps. A systemized search was conducted in September 2023 using the electronic databases EMBASE, Ovid MEDLINE, and Web of Science. To meet inclusion criteria, articles had to be peer-reviewed, written in English, and based on primary data research studies published in US journals 2013-2023. Other criteria included 1) studies with patients needing to be admitted to hospital EDs, 2) AI must have been used when triaging a patient, and 3) patient outcomes must be represented. The search was conducted using controlled descriptors from the Medical Subject Headings (MeSH) that included the terms "artificial intelligence" OR "machine learning" AND "emergency ward" OR "emergency care" OR "emergency department" OR "emergency room" AND "patient triage" OR "triage" OR "triaging." The search initially identified 1,142 citations. After a rigorous, systemized screening process and critical appraisal of the evidence, 29 studies were selected for the final review. The findings indicated that 1) ML models consistently demonstrated superior discrimination abilities compared to conventional triage systems, 2) the integration of AI into the triage process yielded significant enhancements in predictive accuracy, disease identification, and risk assessment, 3) ML accurately determined the necessity of hospitalization for patients requiring urgent attention, and 4) ML improved resource allocation and quality of patient care, including predicting length of stay. The suggested superiority of ML models in prioritizing patients in the ED holds the potential to redefine triage precision.

Preserving multi-dimensional information: A hypersphere method for parameter space analysis

Background

Physiological modelling often involves models described by large numbers of variables and significant volumes of clinical data. Mathematical interpretation of such models frequently necessitates analysing data points in high-dimensional spaces. Existing algorithms for analysing high-dimensional points either lose important dimensionality or do not describe the full position of points. Hence, there is a need for an algorithm which preserves this information.

Methods

The most-distant uncovered point (MDUP) hypersphere method is a binary classification approach which defines a collection of equidistant N-dimensional points as the union of hyperspheres. The method iteratively generates hyperspheres at the most distant point in the interest region not yet contained within any hypersphere, until the entire region of interest is defined by the union of all generated hyperspheres. This method is tested on a 7-dimensional space with up to 35.8 million points representing feasible and infeasible spaces of model parameters for a clinically validated cardiovascular system model.

Results

For different numbers of input points, the MDUP hypersphere method tends to generate large spheres away from the boundary of feasible and infeasible points, but generates the greatest number of relatively much smaller spheres around the boundary of the region of interest to fill this space. Runtime scales quadratically, in part because the current MDUP implementation is not parallelised.

Conclusions

The MDUP hypersphere method can define points in a space of any dimension using only a collection of centre points and associated radii, making the results easily interpretable. It can identify large continuous regions, and in many cases capture the general structure of a region in only a relative few hyperspheres. The MDUP method also shows promise for initialising optimisation algorithm starting conditions within pre-defined feasible regions of model parameter spaces, which could improve model identifiability and the quality of optimisation results.

Advancements in Artificial Intelligence in Emergency Medicine in Taiwan: A Narrative Review

The rapid progression of artificial intelligence (AI) in healthcare has greatly influenced emergency medicine, particularly in Taiwan-a nation celebrated for its technological innovation and advanced public healthcare. This narrative review examines the current status of AI applications in Taiwan's emergency medicine and highlights notable achievements and potential areas for growth. AI has wide capabilities encompass a broad range, including disease prediction, diagnostic imaging interpretation, and workflow enhancement. While the integration of AI presents promising advancements, it is not devoid of challenges. Concerns about the interpretability of AI models, the importance of dataset accuracy, the necessity for external validation, and ethical quandaries emphasize the need for a balanced approach. Regulatory oversight also plays a crucial role in ensuring the safe and effective deployment of AI tools in clinical settings. As its footprint continues to expand in medical education and other areas, addressing these challenges is imperative to harness the full potential of AI for transforming emergency medicine in Taiwan.

ADVANCING THE UNDERSTANDING OF CLINICAL SEPSIS USING GENE EXPRESSION-DRIVEN MACHINE LEARNING TO IMPROVE PATIENT OUTCOMES

ABSTRACT

Sepsis remains a major challenge that necessitates improved approaches to enhance patient outcomes. This study explored the potential of machine learning (ML) techniques to bridge the gap between clinical data and gene expression information to better predict and understand sepsis. We discuss the application of ML algorithms, including neural networks, deep learning, and ensemble methods, to address key evidence gaps and overcome the challenges in sepsis research. The lack of a clear definition of sepsis is highlighted as a major hurdle, but ML models offer a workaround by focusing on endpoint prediction. We emphasize the significance of gene transcript information and its use in ML models to provide insights into sepsis pathophysiology and biomarker identification. Temporal analysis and integration of gene expression data further enhance the accuracy and predictive capabilities of ML models for sepsis. Although challenges such as interpretability and bias exist, ML research offers exciting prospects for addressing critical clinical problems, improving sepsis management, and advancing precision medicine approaches. Collaborative efforts between clinicians and data scientists are essential for the successful implementation and translation of ML models into clinical practice. Machine learning has the potential to revolutionize our understanding of sepsis and significantly improve patient outcomes. Further research and collaboration between clinicians and data scientists are needed to fully understand the potential of ML in sepsis management.

Machine learning model identification and prediction of patients' need for ICU admission: A systematic review

BACKGROUND

The emergency department (ED) triage process serves as a crucial first step for patients seeking acute care, This initial assessment holds crucial implications for patient survival and prognosis. In this study, a systematic review of the existing literature was performed to investigate the performance of machine learning (ML) models in recognizing and predicting the need for intensive care among ED patients.

METHODS

Four prominent databases (PubMed, Embase, Cochrane Library and Web of Science) were searched for relevant literature published up to April 28, 2023. The Prediction model study Risk of Bias Assessment Tool (PROBAST) was employed to evaluate the risk of bias and feasibility of prediction models.

RESULTS

In ten studies, the main algorithms used were Gradient Boostin, Logistic Regressio, Neural Network, Support Vector Machines, Random Forest. The performance of each model was as follows: Gradient Boosting had a sensitivity range of 0.3 to 0.96, specificity range of 0.6 to 0.99, accuracy range of 0.37 to 0.99, precision range of 0.3 to 0.96, and AUC value range of 0.68 to 0.93; Logistic Regression had a sensitivity range of 0.46 to 0.97, specificity range of 0.28 to 0.99, accuracy range of 0.66 to 0.97, precision range of 0.27 to 0.63, and AUC value range of 0.72 to 0.97; Neural Networks had a sensitivity range of 0.45 to 0.96, specificity range of 0.58 to 0.99, accuracy range of 0.36 to 0.97, precision range of 0.27 to 0.96, and AUC value range of 0.67 to 0.91; Support Vector Machines had a sensitivity range of 0.49 to 0.83, specificity range of 0.94 to 0.98, accuracy range of 0.33 to 0.97, precision range of 0.53 to 0.94, and AUC values were not reported; Random Forests had a sensitivity range of 0.75 to 0.91, specificity range of 0.77 to 0.94, accuracy range of 0.35 to 0.77, precision range of 0.36 to 0.94, and AUC value of 0.83.

CONCLUSION

ML models have demonstrated good performance in identifying and predicting critically ill patients in ED triage. However, because of the limited number of studies on each model, further high-quality prospective research is needed to validate these findings.

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis

BACKGROUND

Sepsis is a life-threatening condition caused by an abnormal response of the body to infection and imposes a significant health and economic burden worldwide due to its high mortality rate. Early recognition of sepsis is crucial for effective treatment. This study aimed to systematically evaluate the performance of various machine learning models in predicting the onset of sepsis.

METHODS

We conducted a comprehensive search of the Cochrane Library, PubMed, Embase, and Web of Science databases, covering studies from database inception to November 14, 2022. We used the PROBAST tool to assess the risk of bias. We calculated the predictive performance for sepsis onset using the C-index and accuracy. We followed the PRISMA guidelines for this study.

RESULTS

We included 23 eligible studies with a total of 4,314,145 patients and 26 different machine learning models. The most frequently used models in the studies were random forest (n = 9), extreme gradient boost (n = 7), and logistic regression (n = 6) models. The random forest (test set n = 9, acc = 0.911) and extreme gradient boost (test set n = 7, acc = 0.957) models were the most accurate based on our analysis of the predictive performance. In terms of the C-index outcome, the random forest (n = 6, acc = 0.79) and extreme gradient boost (n = 7, acc = 0.83) models showed the highest performance.

CONCLUSION

Machine learning has proven to be an effective tool for predicting sepsis at an early stage. However, to obtain more accurate results, additional machine learning methods are needed. In our research, we discovered that the XGBoost and random forest models exhibited the best predictive performance and were most frequently utilized for predicting the onset of sepsis.

TRIAL REGISTRATION

CRD42022384015.

Machine Learning for Early Prediction of Sepsis in Intensive Care Unit (ICU) Patients

Background and Objectives: Early detection of sepsis is crucial and can save lives. However, identifying sepsis early and accurately remains a difficult task in the medical field. This study aims to investigate a new machine-learning approach. By analyzing the clinical laboratory results and vital signs of adult patients in the ICU, this approach can predict and detect the initial signs of sepsis. Materials and Methods: To examine survival rates and predict outcomes, the study utilized several models, including the proportional hazards model and data mining algorithms. We analyzed data from the BESTCare database at KAMC, with a focus on patients aged 14 and older who were admitted to the ICU between April and October 2018. We conducted a thorough analysis of the medical records of a total of 1182 patients who were diagnosed with sepsis. Results: We studied two approaches to predict sepsis in ICU patients. The regression model utilizing survival analysis showed moderate predictive ability, emphasizing the importance of only three factors-time (from sepsis to an outcome; discharge or death), lactic acid, and temperature-had a significant p-value (p = 0.000568, p = 0.01, p = 0.02, respectively). Other data mining algorithms may have limitations due to their assumptions of variable independence and linear classification nature. Conclusions: To achieve progress and accuracy in the field of sepsis prediction, it is important to continuously strive for improvement. By meticulously cleaning and selecting data attributes, we can create a strong foundation for future advancements in this area.

SIRS, SOFA, qSOFA, and NEWS in the diagnosis of sepsis and prediction of adverse outcomes: a systematic review and meta-analysis

BACKGROUND

We compared Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), Quick Sepsis-related Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS) for sepsis diagnosis and adverse outcomes prediction.

METHODS

Clinical studies that used SIRS, SOFA, qSOFA, and NEWS for sepsis diagnosis and prognosis assessment were included. Data were extracted, and meta-analysis was performed for outcome measures, including sepsis diagnosis, in-hospital mortality, 7/10/14-day mortality, 28/30-day mortality, and ICU admission.

RESULTS

Fifty-seven included studies showed good overall quality. Regarding sepsis prediction, SIRS demonstrated high sensitivity (0.85) but low specificity (0.41), qSOFA showed low sensitivity (0.42) but high specificity (0.98), and NEWS exhibited high sensitivity (0.71) and specificity (0.85). For predicting in-hospital mortality, SOFA demonstrated the highest sensitivity (0.89) and specificity (0.69). In terms of predicting 7/10/14-day mortality, SIRS exhibited high sensitivity (0.87), while qSOFA had high specificity (0.75). For predicting 28/30-day mortality, SOFA showed high sensitivity (0.97) but low specificity (0.14), whereas qSOFA displayed low sensitivity (0.41) but high specificity (0.88).

CONCLUSIONS

NEWS independently demonstrates good diagnostic capability for sepsis, especially in high-income countries. SOFA emerges as the optimal choice for predicting in-hospital mortality and can be employed as a screening tool for 28/30-day mortality in low-income countries.

Impact of an electronic alert system for pediatric sepsis screening a tertiary hospital experience

This study aimed to assess the potential impact of implementing an electronic alert system (EAS) for systemic inflammatory syndrome (SIRS) and sepsis in pediatric patients mortality. This retrospective study had a pre and post design. We enrolled patients aged ≤ 14 years who were diagnosed with sepsis/severe sepsis upon admission to the pediatric intensive care unit (PICU) of our tertiary hospital from January 2014 to December 2018. We implemented an EAS for the patients with SIRS/sepsis. The patients who met the inclusion criteria pre-EAS implementation comprised the control group, and the group post-EAS implementation was the experimental group. Mortality was the primary outcome, while length of stay (LOS) and mechanical ventilation in the first hour were the secondary outcomes. Of the 308 enrolled patients, 147 were in the pre-EAS group and 161 in the post-EAS group. In terms of mortality, 44 patients in the pre-EAS group and 28 in the post-EAS group died (p 0.011). The average LOS in the PICU was 7.9 days for the pre-EAS group and 6.8 days for the post-EAS group (p 0.442). Considering the EAS initiation time as the “zero time”, early recognition of SIRS and sepsis via the EAS led to faster treatment interventions in post-EAS group, which included fluid boluses with median (25th, 75th percentile) time of 107 (37, 218) min vs. 30 (11,112) min, p < 0.001) and time to initiate antimicrobial therapy median (25th, 75th percentile) of 170.5 (66,320) min vs. 131 (53,279) min, p 0.042). The difference in mechanical ventilation in the first hour of admission was not significant between the groups (25.17% vs. 24.22%, p 0.895). The implementation of the EAS resulted in a statistically significant reduction in the mortality rate among the patients admitted to the PICU in our study. An EAS can play an important role in saving lives and subsequent reduction in healthcare costs. Further enhancement of systematic screening is therefore highly recommended to improve the prognosis of pediatric SIRS and sepsis. The implementation of the EAS, warrants further validation in multicenter or national studies.