Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index

Accelerated Chest Pain Treatment With Artificial Intelligence-Informed, Risk-Driven Triage

This quality improvement study evaluates the use of artificial intelligence to accelerate triage of patients presenting to the emergency department with chest pain.

Emergency Patient Triage Improvement through a Retrieval-Augmented Generation Enhanced Large-Scale Language Model

OBJECTIVES

Emergency medical triage is crucial for prioritizing patient care in emergency situations, yet its effectiveness can vary significantly based on the experience and training of the personnel involved. This study aims to evaluate the efficacy of integrating Retrieval Augmented Generation (RAG) with Large Language Models (LLMs), specifically OpenAI's GPT models, to standardize triage procedures and reduce variability in emergency care.

METHODS

We created 100 simulated triage scenarios based on modified cases from the Japanese National Examination for Emergency Medical Technicians. These scenarios were processed by the RAG-enhanced LLMs, and the models were given patient vital signs, symptoms, and observations from emergency medical services (EMS) teams as inputs. The primary outcome was the accuracy of triage classifications, which was used to compare the performance of the RAG-enhanced LLMs with that of emergency medical technicians and emergency physicians. Secondary outcomes included the rates of under-triage and over-triage.

RESULTS

The Generative Pre-trained Transformer 3.5 (GPT-3.5) with RAG model achieved a correct triage rate of 70%, significantly outperforming Emergency Medical Technicians (EMTs) with 35% and 38% correct rates, and emergency physicians with 50% and 47% correct rates (p < 0.05). Additionally, this model demonstrated a substantial reduction in under-triage rates to 8%, compared with 33% for GPT-3.5 without RAG, and 39% for GPT-4 without RAG.

CONCLUSIONS

The integration of RAG with LLMs shows promise in improving the accuracy and consistency of medical assessments in emergency settings. Further validation in diverse medical settings with broader datasets is necessary to confirm the effectiveness and adaptability of these technologies in live environments.

Admission prioritization of heart failure patients with multiple comorbidities

The primary objective of this study was to enhance the operational efficiency of the current healthcare system by proposing a quicker and more effective approach for healthcare providers to deliver services to individuals facing acute heart failure (HF) and concurrent medical conditions. The aim was to support healthcare staff in providing urgent services more efficiently by developing an automated decision-support Patient Prioritization (PP) Tool that utilizes a tailored machine learning (ML) model to prioritize HF patients with chronic heart conditions and concurrent comorbidities during Urgent Care Unit admission. The study applies key ML models to the PhysioNet dataset, encompassing hospital admissions and mortality records of heart failure patients at Zigong Fourth People's Hospital in Sichuan, China, between 2016 and 2019. In addition, the model outcomes for the PhysioNet dataset are compared with the Healthcare Cost and Utilization Project (HCUP) Maryland (MD) State Inpatient Data (SID) for 2014, a secondary dataset containing heart failure patients, to assess the generalizability of results across diverse healthcare settings and patient demographics. The ML models in this project demonstrate efficiencies surpassing 97.8% and specificities exceeding 95% in identifying HF patients at a higher risk and ranking them based on their mortality risk level. Utilizing this machine learning for the PP approach underscores risk assessment, supporting healthcare professionals in managing HF patients more effectively and allocating resources to those in immediate need, whether in hospital or telehealth settings.

Using a clinical narrative-aware pre-trained language model for predicting emergency department patient disposition and unscheduled return visits

The increasing prevalence of overcrowding in Emergency Departments (EDs) threatens the effective delivery of urgent healthcare. Mitigation strategies include the deployment of monitoring systems capable of tracking and managing patient disposition to facilitate appropriate and timely care, which subsequently reduces patient revisits, optimizes resource allocation, and enhances patient outcomes. This study used ∼ 250,000 emergency department visit records from Taipei Medical University-Shuang Ho Hospital to develop a natural language processing model using BlueBERT, a biomedical domain-specific pre-trained language model, to predict patient disposition status and unplanned readmissions. Data preprocessing and the integration of both structured and unstructured data were central to our approach. Compared to other models, BlueBERT outperformed due to its pre-training on a diverse range of medical literature, enabling it to better comprehend the specialized terminology, relationships, and context present in ED data. We found that translating Chinese-English clinical narratives into English and textualizing numerical data into categorical representations significantly improved the prediction of patient disposition (AUROC = 0.9014) and 72-hour unscheduled return visits (AUROC = 0.6475). The study concludes that the BlueBERT-based model demonstrated superior prediction capabilities, surpassing the performance of prior patient disposition predictive models, thus offering promising applications in the realm of ED clinical practice.

How artificial intelligence could transform emergency care

Artificial intelligence (AI) in healthcare is the ability of a computer to perform tasks typically associated with clinical care (e.g. medical decision-making and documentation). AI will soon be integrated into an increasing number of healthcare applications, including elements of emergency department (ED) care. Here, we describe the basics of AI, various categories of its functions (including machine learning and natural language processing) and review emerging and potential future use-cases for emergency care. For example, AI-assisted symptom checkers could help direct patients to the appropriate setting, models could assist in assigning triage levels, and ambient AI systems could document clinical encounters. AI could also help provide focused summaries of charts, summarize encounters for hand-offs, and create discharge instructions with an appropriate language and reading level. Additional use cases include medical decision making for decision rules, real-time models that predict clinical deterioration or sepsis, and efficient extraction of unstructured data for coding, billing, research, and quality initiatives. We discuss the potential transformative benefits of AI, as well as the concerns regarding its use (e.g. privacy, data accuracy, and the potential for changing the doctor-patient relationship).

Precision emergency medicine

BACKGROUND

Precision health is a burgeoning scientific discipline that aims to incorporate individual variability in biological, behavioral, and social factors to develop personalized health solutions. To date, emergency medicine has not deeply engaged in the precision health movement. However, rapid advances in health technology, data science, and medical informatics offer new opportunities for emergency medicine to realize the promises of precision health.

METHODS

In this article, we conceptualize precision emergency medicine as an emerging paradigm and identify key drivers of its implementation into current and future clinical practice. We acknowledge important obstacles to the specialty-wide adoption of precision emergency medicine and offer solutions that conceive a successful path forward.

RESULTS

Precision emergency medicine is defined as the use of information and technology to deliver acute care effectively, efficiently, and authentically to individual patients and their communities. Key drivers and opportunities include leveraging human data, capitalizing on technology and digital tools, providing deliberate access to care, advancing population health, and reimagining provider education and roles. Overcoming challenges in equity, privacy, and cost is essential for success. We close with a call to action to proactively incorporate precision health into the clinical practice of emergency medicine, the training of future emergency physicians, and the research agenda of the specialty.

CONCLUSIONS

Precision emergency medicine leverages new technology and data-driven artificial intelligence to advance diagnostic testing, individualize patient care plans and therapeutics, and strategically refine the convergence of the health system and the community.

Can artificial intelligence help ED nurses more accurately triage patients?

ABSTRACT

The Emergency Severity Index (ESI) is the most popular tool used to triage patients in the US and abroad. Evidence has shown that ESI has its limitations in correctly assigning acuity. To address this, AI can be incorporated into the triage process, decreasing the likelihood of assigning an incorrect ESI level.

Leveraging graph neural networks for supporting automatic triage of patients

Patient triage is crucial in emergency departments, ensuring timely and appropriate care based on correctly evaluating the emergency grade of patient conditions. Triage methods are generally performed by human operator based on her own experience and information that are gathered from the patient management process. Thus, it is a process that can generate errors in emergency-level associations. Recently, Traditional triage methods heavily rely on human decisions, which can be subjective and prone to errors. A growing interest has recently been focused on leveraging artificial intelligence (AI) to develop algorithms to maximize information gathering and minimize errors in patient triage processing. We define and implement an AI-based module to manage patients' emergency code assignments in emergency departments. It uses historical data from the emergency department to train the medical decision-making process. Data containing relevant patient information, such as vital signs, symptoms, and medical history, accurately classify patients into triage categories. Experimental results demonstrate that the proposed algorithm achieved high accuracy outperforming traditional triage methods. By using the proposed method, we claim that healthcare professionals can predict severity index to guide patient management processing and resource allocation.

Drinking from the Holy Grail-Does a Perfect Triage System Exist? And Where to Look for It?

The Emergency Department (ED) is a facility meant to treat patients in need of medical assistance. The choice of triage system hugely impactsed the organization of any given ED and it is important to analyze them for their effectiveness. The goal of this review is to briefly describe selected triage systems in an attempt to find the perfect one. Papers published in PubMed from 1990 to 2022 were reviewed. The following terms were used for comparison: "ED" and "triage system". The papers contained data on the design and function of the triage system, its validation, and its performance. After studies comparing the distinct means of patient selection were reviewed, they were meant to be classified as either flawed or non-ideal. The validity of all the comparable segregation systems was similar. A possible solution would be to search for a new, measurable parameter for a more accurate risk estimation, which could be a game changer in terms of triage assessment. The dynamic development of artificial intelligence (AI) technologies has recently been observed. The authors of this study believe that the future segregation system should be a combination of the experience and intuition of trained healthcare professionals and modern technology (artificial intelligence).

Artificial Intelligence in Pediatric Emergency Medicine: Applications, Challenges, and Future Perspectives

The dawn of Artificial intelligence (AI) in healthcare stands as a milestone in medical innovation. Different medical fields are heavily involved, and pediatric emergency medicine is no exception. We conducted a narrative review structured in two parts. The first part explores the theoretical principles of AI, providing all the necessary background to feel confident with these new state-of-the-art tools. The second part presents an informative analysis of AI models in pediatric emergencies. We examined PubMed and Cochrane Library from inception up to April 2024. Key applications include triage optimization, predictive models for traumatic brain injury assessment, and computerized sepsis prediction systems. In each of these domains, AI models outperformed standard methods. The main barriers to a widespread adoption include technological challenges, but also ethical issues, age-related differences in data interpretation, and the paucity of comprehensive datasets in the pediatric context. Future feasible research directions should address the validation of models through prospective datasets with more numerous sample sizes of patients. Furthermore, our analysis shows that it is essential to tailor AI algorithms to specific medical needs. This requires a close partnership between clinicians and developers. Building a shared knowledge platform is therefore a key step.

Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage?

INTRODUCTION

Chat-GPT is rapidly emerging as a promising and potentially revolutionary tool in medicine. One of its possible applications is the stratification of patients according to the severity of clinical conditions and prognosis during the triage evaluation in the emergency department (ED).

METHODS

Using a randomly selected sample of 30 vignettes recreated from real clinical cases, we compared the concordance in risk stratification of ED patients between healthcare personnel and Chat-GPT. The concordance was assessed with Cohen's kappa, and the performance was evaluated with the area under the receiver operating characteristic curve (AUROC) curves. Among the outcomes, we considered mortality within 72 h, the need for hospitalization, and the presence of a severe or time-dependent condition.

RESULTS

The concordance in triage code assignment between triage nurses and Chat-GPT was 0.278 (unweighted Cohen's kappa; 95% confidence intervals: 0.231-0.388). For all outcomes, the ROC values were higher for the triage nurses. The most relevant difference was found in 72-h mortality, where triage nurses showed an AUROC of 0.910 (0.757-1.000) compared to only 0.669 (0.153-1.000) for Chat-GPT.

CONCLUSIONS

The current level of Chat-GPT reliability is insufficient to make it a valid substitute for the expertise of triage nurses in prioritizing ED patients. Further developments are required to enhance the safety and effectiveness of AI for risk stratification of ED patients.

Usability evaluation of a glove-type wearable device for efficient biometric collection during triage

To efficiently allocate medical resources at disaster sites, medical workers perform triage to prioritize medical treatments based on the severity of the wounded or sick. In such instances, evaluators often assess the severity status of the wounded or sick quickly, but their measurements are qualitative and rely on experience. Therefore, we developed a wearable device called Medic Hand in this study to extend the functionality of a medical worker's hand so as to measure multiple biometric indicators simultaneously without increasing the number of medical devices to be carried. Medic Hand was developed to quantitatively and efficiently evaluate "perfusion" during triage. Speed is essential during triage at disaster sites, where time and effort are often spared to attach medical devices to patients, so the use of Medic Hand as a biometric measurement device is more efficient for collecting biometric information. For Medic Hand to be handy during disasters, it is essential to understand and improve upon factors that facilitate its public acceptance. To this end, this paper reports on the usability evaluation of Medic Hand through a questionnaire survey of nonmedical workers.

Use of Reverse Shock Index Multiplied by Simplified Motor Score in a Five-Level Triage System: Identifying Trauma in Adult Patients at a High Risk of Mortality

Background and Objectives: The Taiwan Triage and Acuity Scale (TTAS) is reliable for triaging patients in emergency departments in Taiwan; however, most triage decisions are still based on chief complaints. The reverse-shock index (SI) multiplied by the simplified motor score (rSI-sMS) is a more comprehensive approach to triage that combines the SI and a modified consciousness assessment. We investigated the combination of the TTAS and rSI-sMS for triage compared with either parameter alone as well as the SI and modified SI. Materials and Methods: We analyzed 13,144 patients with trauma from the Taipei Tzu Chi Trauma Database. We investigated the prioritization performance of the TTAS, rSI-sMS, and their combination. A subgroup analysis was performed to evaluate the trends in all clinical outcomes for different rSI-sMS values. The sensitivity and specificity of rSI-sMS were investigated at a cutoff value of 4 (based on previous study and the highest score of the Youden Index) in predicting injury severity clinical outcomes under the TTAS system were also investigated. Results: Compared with patients in triage level III, those in triage levels I and II had higher odds ratios for major injury (as indicated by revised trauma score < 7 and injury severity score [ISS] ≥ 16), intensive care unit (ICU) admission, prolonged ICU stay (≥14 days), prolonged hospital stay (≥30 days), and mortality. In all three triage levels, the rSI-sMS < 4 group had severe injury and worse outcomes than the rSI-sMS ≥ 4 group. The TTAS and rSI-sMS had higher area under the receiver operating characteristic curves (AUROCs) for mortality, ICU admission, prolonged ICU stay, and prolonged hospital stay than the SI and modified SI. The combination of the TTAS and rSI-sMS had the highest AUROC for all clinical outcomes. The prediction performance of rSI-sMS < 4 for major injury (ISS ≥ 16) exhibited 81.49% specificity in triage levels I and II and 87.6% specificity in triage level III. The specificity for mortality was 79.2% in triage levels I and II and 87.4% in triage level III. Conclusions: The combination of rSI-sMS and the TTAS yielded superior prioritization performance to TTAS alone. The integration of rSI-sMS and TTAS effectively enhances the efficiency and accuracy of identifying trauma patients at a high risk of mortality.

Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

Multisite development and validation of machine learning models to predict severe outcomes and guide decision-making for emergency department patients with influenza

Objective

Millions of Americans are infected by influenza annually. A minority seek care in the emergency department (ED) and, of those, only a limited number experience severe disease or death. ED clinicians must distinguish those at risk for deterioration from those who can be safely discharged.

Methods

We developed random forest machine learning (ML) models to estimate needs for critical care within 24 h and inpatient care within 72 h in ED patients with influenza. Predictor data were limited to those recorded prior to ED disposition decision: demographics, ED complaint, medical problems, vital signs, supplemental oxygen use, and laboratory results. Our study population was comprised of adults diagnosed with influenza at one of five EDs in our university health system between January 1, 2017 and May 18, 2022; visits were divided into two cohorts to facilitate model development and validation. Prediction performance was assessed by the area under the receiver operating characteristic curve (AUC) and the Brier score.

Results

Among 8032 patients with laboratory-confirmed influenza, incidence of critical care needs was 6.3% and incidence of inpatient care needs was 19.6%. The most common reasons for ED visit were symptoms of respiratory tract infection, fever, and shortness of breath. Model AUCs were 0.89 (95% CI 0.86-0.93) for prediction of critical care and 0.90 (95% CI 0.88-0.93) for inpatient care needs; Brier scores were 0.026 and 0.042, respectively. Importantpredictors included shortness of breath, increasing respiratory rate, and a high number of comorbid diseases.

Conclusions

ML methods can be used to accurately predict clinical deterioration in ED patients with influenza and have potential to support ED disposition decision-making.

Explainable Machine Learning Models for Rapid Risk Stratification in the Emergency Department: A Multicenter Study

BACKGROUND

Risk stratification of patients presenting to the emergency department (ED) is important for appropriate triage. Diagnostic laboratory tests are an essential part of the workup and risk stratification of these patients. Using machine learning, the prognostic power and clinical value of these tests can be amplified greatly. In this study, we applied machine learning to develop an accurate and explainable clinical decision support tool model that predicts the likelihood of 31-day mortality in ED patients (the RISKINDEX). This tool was developed and evaluated in four Dutch hospitals.

METHODS

Machine learning models included patient characteristics and available laboratory data collected within the first 2 h after ED presentation, and were trained using 5 years of data from consecutive ED patients from the Maastricht University Medical Center (Maastricht), Meander Medical Center (Amersfoort), and Zuyderland Medical Center (Sittard and Heerlen). A sixth year of data was used to evaluate the models using area under the receiver-operating-characteristic curve (AUROC) and calibration curves. The Shapley additive explanations (SHAP) algorithm was used to obtain explainable machine learning models.

RESULTS

The present study included 266 327 patients with 7.1 million laboratory results available. Models show high diagnostic performance with AUROCs of 0.94, 0.98, 0.88, and 0.90 for Maastricht, Amersfoort, Sittard and Heerlen, respectively. The SHAP algorithm was utilized to visualize patient characteristics and laboratory data patterns that underlie individual RISKINDEX predictions.

CONCLUSIONS

Our clinical decision support tool has excellent diagnostic performance in predicting 31-day mortality in ED patients. Follow-up studies will assess whether implementation of these algorithms can improve clinically relevant end points.

Triage Accuracy in Pediatrics Using the Emergency Severity Index

INTRODUCTION

Although the Emergency Severity Index is the most widely used tool in the United States to prioritize care for patients who seek emergency care, including children, there are significant deficiencies in the tool's performance. Inaccurate triage has been associated with delayed treatment, unnecessary diagnostic testing, and bias in clinical care. We evaluated the accuracy of the Emergency Severity Index to stratify patient priority based on predicted resource utilization in pediatric emergency department patients and identified covariates influencing performance.

METHODS

This cross-sectional, retrospective study used a data platform that links clinical and research data sets from a single freestanding pediatric hospital in the United States. Chi-square analysis was used to describes rates of over- and undertriage. Mixed effects ordinal logistic regression identified associations between Emergency Severity Index categories assigned at triage and key emergency department resources using discrete data elements and natural language processing of text notes.

RESULTS

We analyzed 304,422 emergency department visits by 153,984 unique individuals in the final analysis; 80% of visits were triaged as lower acuity Emergency Severity Index levels 3 to 5, with the most common level being Emergency Severity Index 4 (43%). Emergency department visits scored Emergency Severity Index levels 3 and 4 were triaged accurately 46% and 38%, respectively. We noted racial differences in overall triage accuracy.

DISCUSSION

Although the plurality of patients was scored as Emergency Severity Index 4, 50% were mistriaged, and there were disparities based on race indicating Emergency Severity Index mistriages pediatric patients. Further study is needed to elucidate the application of the Emergency Severity Indices in pediatrics using a multicenter emergency department population with diverse clinical and demographic characteristics.

Who should decide how limited healthcare resources are prioritized? Autonomous technology as a compelling alternative to humans

Who should decide how limited resources are prioritized? We ask this question in a healthcare context where patients must be prioritized according to their need and where advances in autonomous artificial intelligence-based technology offer a compelling alternative to decisions by humans. Qualitative (Study 1a; N = 50) and quantitative (Study 1b; N = 800) analysis identified agency, emotional experience, bias-free, and error-free as four main qualities describing people's perceptions of autonomous computer programs (ACPs) and human staff members (HSMs). Yet, the qualities were not perceived to be possessed equally by HSMs and ACPs. HSMs were endorsed with human qualities of agency and emotional experience, whereas ACPs were perceived as more capable than HSMs of bias- and error-free decision-making. Consequently, better than average (Study 2; N = 371), or relatively better (Studies 3, N = 181; & 4, N = 378), ACP performance, especially on qualities characteristic of ACPs, was sufficient to reverse preferences to favor ACPs over HSMs as the decision makers for how limited healthcare resources should be prioritized. Our findings serve a practical purpose regarding potential barriers to public acceptance of technology, and have theoretical value for our understanding of perceptions of autonomous technologies.

Prediction of major adverse cardiac events in the emergency department using an artificial neural network with a systematic grid search

BACKGROUND

The aim of our research was to design and evaluate an Artificial Neural Network (ANN) model using a systemic grid search for the early prediction of major adverse cardiac events (MACE) among patients presenting to the triage of an emergency department.

METHODS

This is a single-center, cross-sectional study using electronic health records from January 2017 to December 2020. The research population consists of adults coming to our emergency department triage at Aga Khan University Hospital. The MACE during hospitalization was the main outcome. To enhance the architecture of an ANN using triage data, we used a systematic grid search strategy. Four hidden ANN layers were used, followed by an output layer. Following each hidden layer was back normalization and a dropout layer. MACE was predicted using three binary classifiers: ANN, Random Forests (RF), and logistic regression (LR). The overall accuracy, sensitivity, specificity, precision, and recall of these models were examined. Each model was evaluated using the receiver operating characteristic curve (ROC) and an F1-score with a 95% confidence interval.

RESULTS

A total of 97,333 emergency department visits were recorded during the study period, with 33% of patients having cardiovascular symptoms. The mean age was 54.08 (19.18) years old. The MACE was observed in 23,052 (23.7%) of the patients, in-hospital (up to 30 days) mortality in 10,888 (11.2%) patients, and cardiac arrest in 5483 (5.6%) patients. The data used for training and validation were 77,866 and 19,467 in an 80:20 ratio, respectively. The AUC score for MACE with ANN was 0.97, which was greater than RF (0.96) and LR (0.96). Similarly, the precision-recall curve for MACE utilizing ANN was greater (0.94 vs. 0.93 for RF and 0.93 for LR). The sensitivity for MACE prediction using ANN, RF, and LR classifiers was 99.3%, 99.4%, and 99.2%, respectively, with the specificities being 94.5%, 94.2%, and 94.2%, respectively.

CONCLUSION

When triage data is used to predict MACE, death, and cardiac arrest, ANN with systemic grid search gives precise and valid outcomes and will benefit in predicting MACE in emergency rooms with limited resources that have to deal with a substantial number of patients.

Application of Machine Learning Techniques to Development of Emergency Medical Rapid Triage Prediction Models in Acute Care

Given the critical and complex features of medical emergencies, it is essential to develop models that enable prompt and suitable clinical decision-making based on considerable information. Emergency nurses are responsible for categorizing and prioritizing injuries and illnesses on the frontlines of the emergency room. This study aims to create an Emergency Medical Rapid Triage and Prediction Assistance model using electronic medical records and machine learning techniques. Patient information was retrieved from the emergency department of a large regional teaching hospital in Taiwan, and five supervised learning techniques were used to construct classification models for predicting critical outcomes. Of these models, the model using logistic regression had superior prediction performance, with an F1 score of 0.861 and an area under the receiver operating characteristic curve of 0.855. The Emergency Medical Rapid Triage and Prediction Assistance model demonstrated superior performance in predicting intensive care and hospitalization outcomes compared with the Taiwan Triage and Acuity Scale and three clinical early warning tools. The proposed model has the potential to assist emergency nurses in executing challenging triage assessments and emergency teams in treating critically ill patients promptly, leading to improved clinical care and efficient utilization of medical resources.

Applications of natural language processing at emergency department triage: A narrative review

INTRODUCTION

Natural language processing (NLP) uses various computational methods to analyse and understand human language, and has been applied to data acquired at Emergency Department (ED) triage to predict various outcomes. The objective of this scoping review is to evaluate how NLP has been applied to data acquired at ED triage, assess if NLP based models outperform humans or current risk stratification techniques when predicting outcomes, and assess if incorporating free-text improve predictive performance of models when compared to predictive models that use only structured data.

METHODS

All English language peer-reviewed research that applied an NLP technique to free-text obtained at ED triage was eligible for inclusion. We excluded studies focusing solely on disease surveillance, and studies that used information obtained after triage. We searched the electronic databases MEDLINE, Embase, Cochrane Database of Systematic Reviews, Web of Science, and Scopus for medical subject headings and text keywords related to NLP and triage. Databases were last searched on 01/01/2022. Risk of bias in studies was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Due to the high level of heterogeneity between studies and high risk of bias, a metanalysis was not conducted. Instead, a narrative synthesis is provided.

RESULTS

In total, 3730 studies were screened, and 20 studies were included. The population size varied greatly between studies ranging from 1.8 million patients to 598 triage notes. The most common outcomes assessed were prediction of triage score, prediction of admission, and prediction of critical illness. NLP models achieved high accuracy in predicting need for admission, triage score, critical illness, and mapping free-text chief complaints to structured fields. Incorporating both structured data and free-text data improved results when compared to models that used only structured data. However, the majority of studies (80%) were assessed to have a high risk of bias, and only one study reported the deployment of an NLP model into clinical practice.

CONCLUSION

Unstructured free-text triage notes have been used by NLP models to predict clinically relevant outcomes. However, the majority of studies have a high risk of bias, most research is retrospective, and there are few examples of implementation into clinical practice. Future work is needed to prospectively assess if applying NLP to data acquired at ED triage improves ED outcomes when compared to usual clinical practice.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BACKGROUND

Acute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients.

OBJECTIVE

To devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables.

METHODS

This study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities.

RESULTS

The findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance.

CONCLUSIONS

The suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage

Objective

To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage.

Materials and Methods

Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity.

Results

There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients.

Discussion

Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race.

Conclusion

Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Delays in treatment and disposition attributable to undertriage of pediatric emergency medicine patients

BACKGROUND

Triage, the initial assessment and sorting of patients in the Emergency Department (ED), determines priority of evaluation and treatment. Little is known about the impact of undertriage, the underestimation of disease severity at triage, on clinical care in pediatric ED patients. We evaluate the impact of undertriage on time to disposition and treatment decisions in pediatric ED patients.

METHODS

This was a case control study of ED visits for patients <22 years of age, with an assigned Emergency Severity Index (ESI) score of 4 or 5, and associated hospital admission, nebulized treatment, supplemental oxygen, and/or intravenous (IV) line placement, between January 1, 2018, to June 30, 2022. Controls were sampled from a pool of patient visits with an ESI score of 3, matched by intervention, disposition, and date and hour of arrival. Primary outcome measures were time to order of intervention (nebulized treatment, oxygen administration, or IV placement) and time to disposition decision. A secondary outcome measure was return visits requiring admission or emergency intervention within 14 days of the index visit. Continuous variables (time to orders) were analyzed using Wilcoxon rank sum test and dichotomous outcomes (return visits) were compared using odds ratios with 95% confidence intervals. Analysis was performed with Python v3.10.

RESULTS

The final analysis included 7245 undertriaged patients. Undertriaged patients had longer times to orders for nebulized treatments, (p < 0.001) IV placement, (p < 0.001) and admission (p < 0.001) when compared to controls. There were no significant differences in time to supplemental oxygen delivery and time to discharge compared to controls. Undertriaged patients were more likely to experience a return visit requiring admission or emergency intervention (OR 3.74, 95% CI 3.32,4.22).

CONCLUSIONS

Undertriage in the pediatric ED is associated with delays in care and disposition decisions and increases likelihood of return visits.

ChatGPT's performance before and after teaching in mass casualty incident triage

Since its initial launching, ChatGPT has gained significant attention from the media, with many claiming that ChatGPT's arrival is a transformative milestone in the advancement of the AI revolution. Our aim was to assess the performance of ChatGPT before and after teaching the triage of mass casualty incidents by utilizing a validated questionnaire specifically designed for such scenarios. In addition, we compared the triage performance between ChatGPT and medical students. Our cross-sectional study employed a mixed-methods analysis to assess the performance of ChatGPT in mass casualty incident triage, pre- and post-teaching of Simple Triage And Rapid Treatment (START) triage. After teaching the START triage algorithm, ChatGPT scored an overall triage accuracy of 80%, with only 20% of cases being over-triaged. The mean accuracy of medical students on the same questionnaire yielded 64.3%. Qualitative analysis on pre-determined themes on 'walking-wounded', 'respiration', 'perfusion', and 'mental status' on ChatGPT showed similar performance in pre- and post-teaching of START triage. Additional themes on 'disclaimer', 'prediction', 'management plan', and 'assumption' were identified during the thematic analysis. ChatGPT exhibited promising results in effectively responding to mass casualty incident questionnaires. Nevertheless, additional research is necessary to ensure its safety and efficacy before clinical implementation.

Development of a machine learning-based acuity score prediction model for virtual care settings

OBJECTIVE

Healthcare is increasingly digitized, yet remote and automated machine learning (ML) triage prediction systems for virtual urgent care use remain limited. The Canadian Triage and Acuity Scale (CTAS) is the gold standard triage tool for in-person care in Canada. The current work describes the development of a ML-based acuity score modelled after the CTAS system.

METHODS

The ML-based acuity score model was developed using 2,460,109 de-identified patient-level encounter records from three large healthcare organizations (Ontario, Canada). Data included presenting complaint, clinical modifiers, age, sex, and self-reported pain. 2,041,987 records were high acuity (CTAS 1-3) and 416,870 records were low acuity (CTAS 4-5). Five models were trained: decision tree, k-nearest neighbors, random forest, gradient boosting regressor, and neural net. The outcome variable of interest was the acuity score predicted by the ML system compared to the CTAS score assigned by the triage nurse.

RESULTS

Gradient boosting regressor demonstrated the greatest prediction accuracy. This final model was tuned toward up triaging to minimize patient risk if adopted into the clinical context. The algorithm predicted the same score in 47.4% of cases, and the same or more acute score in 95.0% of cases.

CONCLUSIONS

The ML algorithm shows reasonable predictive accuracy and high predictive safety and was developed using the largest dataset of its kind to date. Future work will involve conducting a pilot study to validate and prospectively assess reliability of the ML algorithm to assign acuity scores remotely.

Race and Ethnicity and Primary Language in Emergency Department Triage

Importance

Emergency department (ED) triage substantially affects how long patients wait for care but triage scoring relies on few objective criteria. Prior studies suggest that Black and Hispanic patients receive unequal triage scores, paralleled by disparities in the depth of physician evaluations.

Objectives

To examine whether racial disparities in triage scores and physician evaluations are present across a multicenter network of academic and community hospitals and evaluate whether patients who do not speak English face similar disparities.

Design, Setting, and Participants

This was a cross-sectional, multicenter study examining adults presenting between February 28, 2019, and January 1, 2023, across the Mass General Brigham Integrated Health Care System, encompassing 7 EDs: 2 urban academic hospitals and 5 community hospitals. Analysis included all patients presenting with 1 of 5 common chief symptoms.

Exposures

Emergency department nurse-led triage and physician evaluation.

Main Outcomes and Measures

Average Triage Emergency Severity Index [ESI] score and average visit work relative value units [wRVUs] were compared across symptoms and between individual minority racial and ethnic groups and White patients.

Results

There were 249 829 visits (149 861 female [60%], American Indian or Alaska Native 0.2%, Asian 3.3%, Black 11.8%, Hispanic 18.8%, Native Hawaiian or Other Pacific Islander <0.1%, White 60.8%, and patients identifying as Other race or ethnicity 5.1%). Median age was 48 (IQR, 29-66) years. White patients had more acute ESI scores than Hispanic or Other patients across all symptoms (eg, chest pain: Hispanic, 2.68 [95% CI, 2.67-2.69]; White, 2.55 [95% CI, 2.55-2.56]; Other, 2.66 [95% CI, 2.64-2.68]; P < .001) and Black patients across most symptoms (nausea/vomiting: Black, 2.97 [95% CI, 2.96-2.99]; White: 2.90 [95% CI, 2.89-2.91]; P < .001). These differences were reversed for wRVUs (chest pain: Black, 4.32 [95% CI, 4.25-4.39]; Hispanic, 4.13 [95% CI, 4.08-4.18]; White 3.55 [95% CI, 3.52-3.58]; Other 3.96 [95% CI, 3.84-4.08]; P < .001). Similar patterns were seen for patients whose primary language was not English.

Conclusions and Relevance

In this cross-sectional study, patients who identified as Black, Hispanic, and Other race and ethnicity were assigned less acute ESI scores than their White peers despite having received more involved physician workups, suggesting some degree of mistriage. Clinical decision support systems might reduce these disparities but would require careful calibration to avoid replicating bias.

Criticality and clinical department prediction of ED patients using machine learning based on heterogeneous medical data

PROBLEM

Emergency triage faces multiple challenges, including limited medical resources and inadequate manual triage nurses, which cause incorrect triage, overcrowding in the emergency department (ED), and long patient waiting time.

OBJECTIVE

This paper aims to propose and validate an accurate and efficient artificial intelligence-based method for effectively ED triage and alleviating the pressure on medical resources.

METHODS

We propose two novel machine learning models, TransNet and TextRNN, for predicting patient severity levels and clinical departments using heterogeneous medical data in ED triage. Our models employ a parallel structure for feature extraction and incorporate an attention mechanism to extract essential information from the fused features, enabling accurate predictions. The models analyze the triage data (2020-2022) from the ED of Beijing University People's Hospital, incorporating variables (demographics, triage vital signs, and chief complaints) to identify patient severity levels and clinical departments. We performed data cleaning, categorization, and encoding first. Then, we divided the available data into a training set (56%), a validation set (24%), and a test set (20%) by random sampling. Finally, our models underwent 5-fold cross-validation and were compared with other state-of-the-art models.

RESULTS

We comprehensively evaluated the proposed models against various Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), Traditional Machine Learning (TML), and Transformer-based (TF) models, achieving excellent performance in predicting triage outcomes. Specifically, TextRNN achieved a prediction success rate of 86.23% [85.86-86.70] for severity levels and 94.30% [94.00-94.46] for clinical departments among 161,198 ED visits. Moreover, TransNet demonstrated higher sensitivities of 84.08% and 90.05% for severity levels and clinical departments, respectively, with specificities of 76.48% and 95.16%. The accuracy of our model is 0.87%, 0.18%, 4.29%, and 1.96%, higher than that of the above four family models on average. Furthermore, our method significantly reduced under-triage by 12.06% and over-triage by 17.92% compared to manual triage.

CONCLUSIONS

Experimental results demonstrated that the proposed models fuse heterogeneous medical data in the triage process, successfully predicting patients' triage outcomes. Our models can improve triage efficiency, reduce the under/over-triage rate, and provide physicians with valuable decision-making support.

Machine learning for the diagnosis of acute coronary syndrome using a 12-lead ECG: a systematic review

OBJECTIVES

Prompt diagnosis of acute coronary syndrome (ACS) using a 12-lead electrocardiogram (ECG) is a critical task for emergency physicians. While computerized algorithms for ECG interpretation are limited in their accuracy, machine learning (ML) models have shown promise in several areas of clinical medicine. We performed a systematic review to compare the performance of ML-based ECG analysis to clinician or non-ML computerized ECG interpretation in the diagnosis of ACS for emergency department (ED) or prehospital patients.

METHODS

We searched Medline, Embase, Cochrane Central, and CINAHL databases from inception to May 18, 2022. We included studies that compared ML algorithms to either clinicians or non-ML based software in their ability to diagnose ACS using only a 12-lead ECG, in adult patients experiencing chest pain or symptoms concerning for ACS in the ED or prehospital setting. We used QUADAS-2 for risk of bias assessment. Prospero registration CRD42021264765.

RESULTS

Our search yielded 1062 abstracts. 10 studies met inclusion criteria. Five model types were tested, including neural networks, random forest, and gradient boosting. In five studies with complete performance data, ML models were more sensitive but less specific (sensitivity range 0.59-0.98, specificity range 0.44-0.95) than clinicians (sensitivity range 0.22-0.93, specificity range 0.63-0.98) in diagnosing ACS. In four studies that reported it, ML models had better discrimination (area under ROC curve range 0.79-0.98) than clinicians (area under ROC curve 0.67-0.78). Heterogeneity in both methodology and reporting methods precluded a meta-analysis. Several studies had high risk of bias due to patient selection, lack of external validation, and unreliable reference standards for ACS diagnosis.

CONCLUSIONS

ML models have overall higher discrimination and sensitivity but lower specificity than clinicians and non-ML software in ECG interpretation for the diagnosis of ACS. ML-based ECG interpretation could potentially serve a role as a "safety net", alerting emergency care providers to a missed acute MI when it has not been diagnosed. More rigorous primary research is needed to definitively demonstrate the ability of ML to outperform clinicians at ECG interpretation.

Do prehospital sepsis alerts decrease time to complete CMS sepsis measures?

INTRODUCTION

In an effort to improve sepsis outcomes the Centers for Medicare and Medicaid Services (CMS) established a time sensitive sepsis management bundle as a core quality measure that includes blood culture collection, serum lactate collection, initiation of intravenous fluid administration, and initiation of broad-spectrum antibiotics. Few studies examine the effects of a prehospital sepsis alert protocol on decreasing time to complete CMS sepsis core measures.

METHODS

This study was a retrospective cohort study of patients transported via EMS from December 1, 2018 to December 1, 2019 who met the criteria of the Maryland Statewide EMS sepsis protocol and compared outcomes between patients who activated a prehospital sepsis alert and patients who did not activate a prehospital sepsis alert. The Maryland Institute for Emergency Medical Services Systems developed a sepsis protocol that instructs EMS providers to notify the nearest appropriate facility with a sepsis alert if a patient 18 years of age and older is suspected of having an infection and also presents with at least two of the following: temperature >38 °C or <35.5 °C, a heart rate >100 beats per minute, a respiratory rate >25 breaths per minute or end-tidal carbon dioxide less than or equal to 32 mmHg, a systolic blood pressure <90 mmHg, or a point of care lactate reading greater than or equal to 4 mmol/L.

RESULTS

Median time to achieve all four studied CMS sepsis core measures was 103 min [IQR 61-153] for patients who received a prehospital sepsis alert and 106.5 min [IQR 75-189] for patients who did not receive a prehospital sepsis alert (p-value 0.105). Median time to completion was shorter for serum lactate collection (28 min. vs 35 min., p-value 0.019), blood culture collection (28 min. vs 38 min., p-value <0.01), and intravenous fluid administration (54 min. vs 61 min., p-value 0.025) but was not significantly different for antibiotic administration (94 min. vs 103 min., p-value 0.12) among patients who triggered a sepsis alert.

CONCLUSION

This study questions the effectiveness of prehospital sepsis alert protocols on decreasing time to complete CMS sepsis core measures. Future studies should address if these times can be impacted by having EMS providers independently administer antibiotics.

The effect of computer-based decision support system on emergency department triage: Non-randomised controlled trial

BACKGROUND

Deciding on triage in emergency departments is difficult and requires comprehensive knowledge and experience.

PURPOSE

This study was conducted to evaluate the effect of a "computer-based emergency department triage decision support system (DSS)," which was designed and integrated into the hospital information management system, on triage decision accuracy and triage duration by using real patient data.

METHODS

Single-group, pretest-posttest non-randomised clinical trial. The study was conducted with the real data of patients who had been triaged in the adult emergency department of a university hospital. The pretest was applied between July 16 and September 16, 2019, and the post-test on September 1 and October 31, 2020. In the pre-test and post-test phases of the study, triage decision accuracy rates, and triage duration were evaluated. In the post-test phase, Emergency Triage Decision Support System (ETDSS) was prepared with a rule-based decision trees method using the Emergency Severity Index Version 4 and The Australasian Triage Scale and was integrated into the hospital information management system. The effect of the developed ETDSS was evaluated. The mean, standard deviation, frequency and percentage values were calculated for the descriptive characteristics. Independent samples t-test, analysis of variance, Sidak paired comparison, and Bonferroni tests were applied.

RESULTS

The effect of the computer-based emergency triage DSS on triage management was tested based on the data of 16,409 patients in the pretest phase and 7,765 patients in the posttest phase. While the accuracy rate of nurses' triage decisions was 57.8% in the pretest, it was found to increase to 64.9% in the posttest. The mean duration of triage was 1.47 ± 0.72 in the pretest and 1.79 ± 0.85 min in the posttest.

CONCLUSIONS

The DSS increased triage decision accuracy independently of professional and triage experience and brought the triage duration closer to the time recommended in the literature. Clinically, this is associated with patient safety, quality improvement processes, and professional accountability.

Predicting acute clinical deterioration with interpretable machine learning to support emergency care decision making

The emergency department (ED) is a fast-paced environment responsible for large volumes of patients with varied disease acuity. Operational pressures on EDs are increasing, which creates the imperative to efficiently identify patients at imminent risk of acute deterioration. The aim of this study is to systematically compare the performance of machine learning algorithms based on logistic regression, gradient boosted decision trees, and support vector machines for predicting imminent clinical deterioration for patients based on cross-sectional patient data extracted from electronic patient records (EPR) at the point of entry to the hospital. We apply state-of-the-art machine learning methods to predict early patient deterioration, based on their first recorded vital signs, observations, laboratory results, and other predictors documented in the EPR. Clinical deterioration in this study is measured by in-hospital mortality and/or admission to critical care. We build on prior work by incorporating interpretable machine learning and fairness-aware modelling, and use a dataset comprising 118, 886 unplanned admissions to Salford Royal Hospital, UK, to systematically compare model variations for predicting mortality and critical care utilisation within 24 hours of admission. We compare model performance to the National Early Warning Score 2 (NEWS2) and yield up to a 0.366 increase in average precision, up to a [Formula: see text] reduction in daily alert rate, and a median 0.599 reduction in differential bias amplification across the protected demographics of age and sex. We use Shapely Additive exPlanations to justify the models' outputs, verify that the captured data associations align with domain knowledge, and pair predictions with the causal context of each patient's most influential characteristics. Introducing our modelling to clinical practice has the potential to reduce alert fatigue and identify high-risk patients with a lower NEWS2 that might be missed currently, but further work is needed to trial the models in clinical practice. We encourage future research to follow a systematised approach to data-driven risk modelling to obtain clinically applicable support tools.

Clinical support system for triage based on federated learning for the Korea triage and acuity scale

Background and aims

This study developed a clinical support system based on federated learning to predict the need for a revised Korea Triage Acuity Scale (KTAS) to facilitate triage.

Methods

This was a retrospective study that used data from 11,952,887 patients in the Korean National Emergency Department Information System (NEDIS) from 2016 to 2018 for model development. Separate cohorts were created based on the emergency medical center level in the NEDIS: regional emergency medical center (REMC), local emergency medical center (LEMC), and local emergency medical institution (LEMI). External and temporal validation used data from emergency department (ED) of the study site from 2019 to 2021. Patient features obtained during the triage process and the initial KTAS scores were used to develop the prediction model. Federated learning was used to rectify the disparity in data quality between EDs. The patient's demographic information, vital signs in triage, mental status, arrival information, and initial KTAS were included in the input feature.

Results

3,626,154 patients' visits were included in the regional emergency medical center cohort; 8,278,081 patients' visits were included in the local emergency medical center cohort; and 48,652 patients' visits were included in the local emergency medical institution cohort. The study site cohort, which is used for external and temporal validation, included 135,780 patients visits. Among the patients in the REMC and study site cohorts, KTAS level 3 patients accounted for the highest proportion at 42.4% and 45.1%, respectively, whereas in the LEMC and LEMI cohorts, KTAS level 4 patients accounted for the highest proportion. The area under the receiver operating characteristic curve for the prediction model was 0.786, 0.750, and 0.770 in the external and temporal validation. Patients with revised KTAS scores had a higher admission rate and ED mortality rate than those with unaltered KTAS scores.

Conclusions

This novel system might accurately predict the likelihood of KTAS acuity revision and support clinician-based triage.

Behavioral Health Decision Support Systems and User Interface Design in the Emergency Department

OBJECTIVE

The objective of this qualitative study is to gauge physician sentiment about an emergency department (ED) clinical decision support (CDS) system implemented in multiple adult EDs within a university hospital system. This CDS system focuses on predicting patients' likelihood of ED recidivism and/or adverse opioid-related events.

METHODS

The study was conducted among adult emergency physicians working in three EDs of a single academic health system in Rhode Island. Qualitative, semistructured interviews were conducted with ED physicians. Interviews assessed physicians' prior experience with predictive analytics, thoughts on the alert's placement, design, and content, the alert's overall impact, and potential areas for improvement. Responses were aggregated and common themes identified.

RESULTS

Twenty-three interviews were conducted (11 preimplementation and 12 postimplementation). Themes were identified regarding each physician familiarity with predictive analytics, alert rollout, alert appearance and content, and on alert sentiments. Most physicians viewed these alerts as a neutral or positive EHR addition, with responses ranging from neutral to positive. The alert placement was noted to be largely intuitive and nonintrusive. The design of the alert was generally viewed positively. The alert's content was believed to be accurate, although the decision to respond to the alert's call-to-action was physician dependent. Those who tended to ignore the alert did so for a few reasons, including already knowing the information the alert contains, the alert offering information that is not relevant to this particular patient, and the alert not containing enough information to be useful.

CONCLUSION

Ultimately, this alert appears to have a marginally positive effect on ED physician workflow. At its most beneficial, the alert reminded physicians to deeply consider the care provided to high-risk populations and to potentially adjust their care and referrals. At its least beneficial, the alert did not affect physician decision-making but was not intrusive to the point of negatively impacting workflow.

Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

Objectives

Efficient and accurate emergency department (ED) triage is critical to prioritize the sickest patients and manage department flow. We explored the use of electronic health record data and advanced predictive analytics to improve triage performance.

Methods

Using a data set of over 5 million ED encounters of patients 18 years and older across 21 EDs from 2016 to 2020, we derived triage models using deep learning to predict 2 outcomes: hospitalization (primary outcome) and fast-track eligibility (exploratory outcome), defined as ED discharge with <2 resource types used (eg, laboratory or imaging studies) and no critical events (eg, resuscitative medications use or intensive care unit [ICU] admission). We report area under the receiver operator characteristic curve (AUC) and 95% confidence intervals (CI) for models using (1) triage variables alone (demographics and vital signs), (2) triage nurse clinical assessment alone (unstructured notes), and (3) triage variables plus clinical assessment for each prediction target.

Results

We found 12.7% of patients were hospitalized (n = 673,659) and 37.0% were fast-track eligible (n = 1,966,615). The AUC was lowest for models using triage variables alone: AUC 0.77 (95% CI 0.77-0.78) and 0.70 (95% CI 0.70-0.71) for hospitalization and fast-track eligibility, respectively, and highest for models incorporating clinical assessment with triage variables for both hospitalization and fast-track eligibility: AUC 0.87 (95% CI 0.87-0.87) for both prediction targets.

Conclusion

Our findings highlight the potential to use advanced predictive analytics to accurately predict key ED triage outcomes. Predictive accuracy was optimized when clinical assessments were added to models using simple structured variables alone.

Medical emergency department triage data processing using a machine-learning solution

Over the years, artificial intelligence has demonstrated its ability to overcome many challenges in our day-to-day life. The evolution of it inquired more studies about Machine Learning possible solutions for different domains, including health care. The increasing demand for artificial intelligence solutions has brought accessibility to loads of data, including clinical data. The availability of medical records facilitates new opportunities to explore Machine Learning models and their abilities to process a significant amount of data and to identify patterns with the purpose of solving a medical problem. Understanding the applicability of artificial intelligence on this type of data has to be a compelling aim for emergency medicine clinicians. This paper focuses on the general clinical problem of the complex correlation between medical records and later diagnosis and, especially, on the process of emergency department triage which uses the Emergency Severity Index (ESI) as triage protocol. This study presents a comparison between three different Machine Learning models, such as Logistic Regression, Random Forest Tree and NN-Sequentail, with the purpose of classifying patients with an emergency code. We conducted four experiments because of imbalanced data. A web-based application was developed to improve the triage process after our theoretical and exploratory results. Overall, in all experiments, the NN-Sequential model had better results, having, in the first experiment, a ROC-AUC score for each ESI emergency code of: 0.59%, 0.76%, 0.71%, 0.78% 0.64%. After applying methods to balance the data, the model yielded a ROC-AUC score for each emergency code of 0.72%, 0.75%, 0.69%, 0.74%, 0.78%. In the last experiment consisting of a three-class classification problem, the NN-Sequential and Random Forest Tree models had similar metric outcomes, and the NN-Sequential algorithm had a ROC-AUC score for each emergency code of: 0.76%, 0.72%, 0.84%. Without any doubt, our research results presented in this paper endorse this tremendous curiosity in Machine Learning applications to enrich aspects of emergency medical care by applying specific methods for processing both medical data and medical records.

Artificial Intelligence in Pharmaceutical Technology and Drug Delivery Design

Artificial intelligence (AI) has emerged as a powerful tool that harnesses anthropomorphic knowledge and provides expedited solutions to complex challenges. Remarkable advancements in AI technology and machine learning present a transformative opportunity in the drug discovery, formulation, and testing of pharmaceutical dosage forms. By utilizing AI algorithms that analyze extensive biological data, including genomics and proteomics, researchers can identify disease-associated targets and predict their interactions with potential drug candidates. This enables a more efficient and targeted approach to drug discovery, thereby increasing the likelihood of successful drug approvals. Furthermore, AI can contribute to reducing development costs by optimizing research and development processes. Machine learning algorithms assist in experimental design and can predict the pharmacokinetics and toxicity of drug candidates. This capability enables the prioritization and optimization of lead compounds, reducing the need for extensive and costly animal testing. Personalized medicine approaches can be facilitated through AI algorithms that analyze real-world patient data, leading to more effective treatment outcomes and improved patient adherence. This comprehensive review explores the wide-ranging applications of AI in drug discovery, drug delivery dosage form designs, process optimization, testing, and pharmacokinetics/pharmacodynamics (PK/PD) studies. This review provides an overview of various AI-based approaches utilized in pharmaceutical technology, highlighting their benefits and drawbacks. Nevertheless, the continued investment in and exploration of AI in the pharmaceutical industry offer exciting prospects for enhancing drug development processes and patient care.

Sociodemographic Disparities in Queue Jumping for Emergency Department Care

Importance

Emergency department (ED) triage models are intended to queue patients for treatment. In the absence of higher acuity, patients of the same acuity should room in order of arrival.

Objective

To characterize disparities in ED care access as unexplained queue jumps (UQJ), or instances in which acuity and first come, first served principles are violated.

Design, Setting, and Participants

Retrospective, cross-sectional study between July 2017 and February 2020. Participants were all ED patient arrivals at 2 EDs within a large Northeast health system. Data were analyzed from July to September 2022.

Exposure

UQJ was defined as a patient being placed in a treatment space ahead of a patient of higher acuity or of a same acuity patient who arrived earlier.

Main Outcomes and Measures

Primary outcomes were odds of a UQJ and association with ED outcomes of hallway placement, leaving before treatment complete, escalation to higher level of care while awaiting inpatient bed placement, and 72-hour ED revisitation. Secondary analysis examined UQJs among high acuity ED arrivals. Regression models (zero-inflated Poisson and logistic regression) adjusted for patient demographics and ED operational variables at time of triage.

Results

Of 314 763 included study visits, 170 391 (54.1%) were female, the mean (SD) age was 50.46 (20.5) years, 132 813 (42.2%) patients were non-Hispanic White, 106 401 (33.8%) were non-Hispanic Black, and 66 465 (21.1%) were Hispanic or Latino. Overall, 90 698 (28.8%) patients experienced a queue jump, and 78 127 (24.8%) and 44 551 (14.2%) patients were passed over by a patient of the same acuity or lower acuity, respectively. A total of 52 959 (16.8%) and 23 897 (7.6%) patients received care ahead of a patient of the same acuity or higher acuity, respectively. Patient demographics including Medicaid insurance (incident rate ratio [IRR], 1.11; 95% CI, 1.07-1.14), Black non-Hispanic race (IRR, 1.05; 95% CI, 1.03-1.07), Hispanic or Latino ethnicity (IRR, 1.05; 95% CI, 1.02-1.08), and Spanish as primary language (IRR, 1.06; 95% CI, 1.02-1.10) were independent social factors associated with being passed over. The odds of a patient receiving care ahead of others were lower for ED visits by Medicare insured (odds ratio [OR], 0.92; 95% CI, 0.88-0.96), Medicaid insured (OR, 0.81; 95% CI, 0.77-0.85), Black non-Hispanic (OR, 0.94; 95% CI, 0.91-0.97), and Hispanic or Latino ethnicity (OR, 0.87; 95% CI, 0.83-0.91). Patients who were passed over by someone of the same triage severity level had higher odds of hallway bed placement (OR, 1.01; 95% CI, 1.00-1.02) and leaving before disposition (OR, 1.02; 95% CI, 1.01-1.04).

Conclusions and Relevance

In this cross-sectional study of ED patients in triage, there were consistent disparities among marginalized populations being more likely to experience a UQJ, hallway placement, and leaving without receiving treatment despite being assigned the same triage acuity as others. EDs should seek to standardize triage processes to mitigate conscious and unconscious biases that may be associated with timely access to emergency care.

Implementation of Prediction Models in the Emergency Department from an Implementation Science Perspective-Determinants, Outcomes, and Real-World Impact: A Scoping Review

STUDY OBJECTIVE

Prediction models offer a promising form of clinical decision support in the complex and fast-paced environment of the emergency department (ED). Despite significant advancements in model development and validation, implementation of such models in routine clinical practice remains elusive. This scoping review aims to survey the current state of prediction model implementation in the ED and to provide insights on contributing factors and outcomes from an implementation science perspective.

METHODS

We searched 4 databases from their inception to May 20, 2022: MEDLINE (through PubMed), Embase, Scopus, and CINAHL. Articles that reported implementation outcomes and/or contextual determinants under the Reach, Effectiveness, Adoption, Implementation Maintenance (RE-AIM)/Practical, Robust, Implementation, and Sustainability Model (PRISM) framework were included. Characteristics of studies, models, and results of the RE-AIM/PRISM domains were summarized narratively.

RESULTS

Thirty-six reports on 31 implementations were included. The most common prediction models implemented were early warning scores. The most common implementation strategies used were training stakeholders, infrastructural changes, and using evaluative or iterative strategies. Only one report examined ED patients' perspectives, whereas the rest were focused on the experience of health care workers or organizational stakeholders. Key determinants of successful implementation include strong stakeholder engagement, codevelopment of workflows and implementation strategies, education, and usability.

CONCLUSION

Examining ED prediction models from an implementation science perspective can provide valuable insights and help guide future implementations.

Comparison of Six Scoring Systems for Predicting In-hospital Mortality among Patients with SARS-COV2 Presenting to the Emergency Department

Background

The study aimed to compare the prognostic accuracy of six different severity-of-illness scoring systems for predicting in-hospital mortality among patients with confirmed SARS-COV2 who presented to the emergency department (ED). The scoring systems assessed were worthing physiological score (WPS), early warning score (EWS), rapid acute physiology score (RAPS), rapid emergency medicine score (REMS), national early warning score (NEWS), and quick sequential organ failure assessment (qSOFA).

Materials and methods

A cohort study was conducted using data obtained from electronic medical records of 6,429 confirmed SARS-COV2 patients presenting to the ED. Logistic regression models were fitted on the original severity-of-illness scores to assess the models' performance using the Area Under the Curve for ROC (AUC-ROC) and Precision-Recall curves (AUC-PR), Brier Score (BS), and calibration plots were used to assess the models' performance. Bootstrap samples with multiple imputations were used for internal validation.

Results

The mean age of the patients was 64 years (IQR:50-76) and 57.5% were male. The WPS, REMS, and NEWS models had AUROC of 0.714, 0.705, and 0.701, respectively. The poorest performance was observed in the RAPS model, with an AUROC of 0.601. The BS for the NEWS, qSOFA, EWS, WPS, RAPS, and REMS was 0.18, 0.09, 0.03, 0.14, 0.15, and 0.11 respectively. Excellent calibration was obtained for the NEWS, while the other models had proper calibration.

Conclusion

The WPS, REMS, and NEWS have a fair discriminatory performance and may assist in risk stratification for SARS-COV2 patients presenting to the ED. Generally, underlying diseases and most vital signs are positively associated with mortality and were different between the survivors and non-survivors.

How to cite this article

Rahmatinejad Z, Hoseini B, Reihani H, Hanna AA, Pourmand A, Tabatabaei SM, et al. Comparison of Six Scoring Systems for Predicting In-hospital Mortality among Patients with SARS-COV2 Presenting to the Emergency Department. Indian J Crit Care Med 2023;27(6):416-425.

Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department

This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.

Risk-factor analysis for extended-spectrum beta-lactamase-producing Enterobacterales colonization or infection: Evaluation of a novel approach to assess local prevalence as a risk factor

OBJECTIVE

To explore an approach to identify the risk of local prevalence of extended-spectrum β-lactamase-producing Enterobacterales (ESBL-E) on ESBL-E colonization or infection and to reassess known risk factors.

DESIGN

Case-control study.

SETTING

Johns Hopkins Health System emergency departments (EDs) in the Baltimore-Washington, DC, region.

PATIENTS

Patients aged ≥18 years with a culture growing Enterobacterales between April 2019 and December 2021. Cases had a culture growing an ESBL-E.

METHODS

Addresses were linked to Census Block Groups and placed into communities using a clustering algorithm. Prevalence in each community was estimated using the proportion of ESBL-E among Enterobacterales isolates. Logistic regression was used to determine risk factors for ESBL-E colonization or infection.

RESULTS

ESBL-E were detected in 1,167 of 11,224 patients (10.4%). Risk factors included a history of ESBL-E in the prior 6 months (aOR, 20.67; 95% CI, 13.71-31.18), exposure to a skilled nursing or long-term care facility (aOR, 1.64; 95% CI, 1.37-1.96), exposure to a third-generation cephalosporin (aOR, 1.79; 95% CI, 1.46-2.19), exposure to a carbapenem (aOR, 2.31; 95% CI, 1.68-3.18), or exposure to a trimethoprim-sulfamethoxazole (aOR, 1.54; 95% CI, 1.06-2.25) within the prior 6 months. Patients were at lower risk if their community had a prevalence <25th percentile in the prior 3 months (aOR, 0.83; 95% CI, 0.71-0.98), 6 months (aOR, 0.83; 95% CI, 0.71-0.98), or 12 months (aOR, 0.81; 95% CI, 0.68-0.95). There was no association between being in a community in the >75th percentile and the outcome.

CONCLUSIONS

This method of defining the local prevalence of ESBL-E may partially capture differences in the likelihood of a patient having an ESBL-E.

Modern Paediatric Emergency Department: Potential Improvements in Light of New Evidence

The increasing attendance of paediatric emergency departments has become a serious health issue. To reduce an elevated burden of medical errors, inevitably caused by a high level of stress exerted on emergency physicians, we propose potential areas for improvement in regular paediatric emergency departments. In an effort to guarantee the demanded quality of care to all incoming patients, the workflow in paediatric emergency departments should be sufficiently optimised. The key component remains to implement one of the validated paediatric triage systems upon the patient's arrival at the emergency department and fast-tracking patients with a low level of risk according to the triage system. To ensure the patient's safety, emergency physicians should follow issued guidelines. Cognitive aids, such as well-designed checklists, posters or flow charts, generally improve physicians' adherence to guidelines and should be available in every paediatric emergency department. To sharpen diagnostic accuracy, the use of ultrasound in a paediatric emergency department, according to ultrasound protocols, should be targeted to answer specific clinical questions. Combining all mentioned improvements might reduce the number of errors linked to overcrowding. The review serves not only as a blueprint for modernising paediatric emergency departments but also as a bin of useful literature which can be suitable in the paediatric emergency field.

Validity of the Korean triage and acuity scale in older patients compared to the adult group

INTRODUCTION

While many patients visit the emergency department (ED) for various reasons, medical resources are limited. Therefore, various triage scale systems have been used to predict patient urgency and severity. South Korea has developed and used the Korean Triage and Accuracy Scale (KTAS) based on the Canadian classification tool. As the elderly population increases, the number of elderly patients visiting the ED also increases. However, in KTAS, there is no consideration for the elderly, and the same classification system as adults. The aim of this study is to verify the ability of KTAS to predict severity levels in the elderly group, compared to the adult group.

METHODS

This is a retrospective study for patients who visited the ED at two centers between February 1, 2018 and January 31, 2021. The initial KTAS level, changed level at ED discharge, general patient character, ED treatment results, in-hospital mortality, and lengths of hospital and ED stays were acquired. Area under the receiver operating characteristics (AUROC) was used to verify the severity prediction ability of the elderly group to KTAS, and logistic regression analysis was used for the prediction up-triage of KTAS.

RESULTS

The enrolled patients in the study were 87,220 in the adult group and 37,627 in the elderly group. The proportion of KTAS up-triage was higher in the elderly group (1.9 % vs. 1.2 %, p < 0.001). The AUROC for the overall admission rate was 0.686, 0.667 in the adult and elderly group, the AUROC for ICU admission was 0.842, 0.767, and the AUROC for in-hospital mortality prediction was 0.809, 0.711, indicating a decrease in the AUROC value in the elderly group. The independent factors of the up-triage predictors were old age, male gender, pulse, and ED length of stay, and old age was the most influential variable.

CONCLUSION

KTAS was poorly associated with severity in the elderly than in adults, and it was found that up-triaging was more likely to occur in the elderly. The severity and urgency of patients over 65 years of age should not be underestimated when initially determining the triage scale.

Evaluation of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage

Importance

Accurate emergency department (ED) triage is essential to prioritize the most critically ill patients and distribute resources appropriately. The most used triage system in the US is the Emergency Severity Index (ESI).

Objectives

To derive and validate an algorithm to assess the rate of mistriage and to identify characteristics associated with mistriage.

Design, Setting, and Participants

This retrospective cohort study created operational definitions for each ESI level that use ED visit electronic health record data to classify encounters as undertriaged, overtriaged, or correctly triaged. These definitions were applied to a retrospective cohort to assess variation in triage accuracy by facility and patient characteristics in 21 EDs within the Kaiser Permanente Northern California (KPNC) health care system. All ED encounters by patients 18 years and older between January 1, 2016, and December 31, 2020, were assessed for eligibility. Encounters with missing ESI or incomplete ED time variables and patients who left against medical advice or without being seen were excluded. Data were analyzed between January 1, 2021, and November 30, 2022.

Exposures

Assigned ESI level.

Main Outcomes and Measures

Rate of undertriage and overtriage by assigned ESI level based on a mistriage algorithm and patient and visit characteristics associated with undertriage and overtriage.

Results

A total of 5 315 176 ED encounters were included. The mean (SD) patient age was 52 (21) years; 44.3% of patients were men and 55.7% were women. In terms of race and ethnicity, 11.1% of participants were Asian, 15.1% were Black, 21.4% were Hispanic, 44.0% were non-Hispanic White, and 8.5% were of other (includes American Indian or Alaska Native, Native Hawaiian or other Pacific Islander, and multiple races or ethnicities), unknown, or missing race or ethnicity. Mistriage occurred in 1 713 260 encounters (32.2%), of which 176 131 (3.3%) were undertriaged and 1 537 129 (28.9%) were overtriaged. The sensitivity of ESI to identify a patient with high-acuity illness (correctly assigning ESI I or II among patients who had a life-stabilizing intervention) was 65.9%. In adjusted analyses, Black patients had a 4.6% (95% CI, 4.3%-4.9%) greater relative risk of overtriage and an 18.5% (95% CI, 16.9%-20.0%) greater relative risk of undertriage compared with White patients, while Black male patients had a 9.9% (95% CI, 9.8%-10.0%) greater relative risk of overtriage and a 41.0% (95% CI, 40.0%-41.9%) greater relative risk of undertriage compared with White female patients. High relative risk of undertriage was found among patients taking high-risk medications (30.3% [95% CI, 28.3%-32.4%]) and those with a greater comorbidity burden (22.4% [95% CI, 20.1%-24.4%]) and recent intensive care unit utilization (36.7% [95% CI, 30.5%-41.4%]).

Conclusions and Relevance

In this retrospective cohort study of over 5 million ED encounters, mistriage with ESI was common. Quality improvement should focus on limiting critical undertriage, optimizing resource allocation by patient need, and promoting equity.

Development of outcomes for evaluating emergency care triage: a Delphi approach

BACKGROUND

Triage is used as standard of care for prioritization and identification of time-critical patients in the emergency department (ED) globally, but it is unclear what outcomes should be used to evaluate triage. Currently used outcomes do not include important time-critical diagnoses and conditions.

METHOD

We used 18 Swedish triage experts to collect and assess outcomes for the evaluation of 5-level triage systems. The experts suggested 68 outcomes which were then tested through a modified Delphi approach in three rounds. The outcomes aimed to identify correctly prioritized red patients (in need of a resuscitation team), and orange patients (other time critical conditions). Consensus was pre-defined as 70% dichotomized (positive/negative) concordance.

RESULTS

Diagnoses, interventions, mortality, level of care and lab results were included in the outcomes. Positive consensus was reached for 49 outcomes and negative consensus for 7 outcomes, with an 83% response rate. The five most approved outcomes were the interventions Percutaneous coronary intervention, Surgical airway and Massive transfusion together with the diagnoses Tension pneumothorax and Intracerebral hemorrhage that received specific interventions. The outcomes with the clearest disapproval included Admittance to a ward, Treatment with antihistamines and The ordering of a head computed tomography scan. The outcomes were considered valid only if occurring in or from the ED.

CONCLUSION

This study proposes a standard of 49 outcomes divided into two sets tied to red and orange priority respectively, to be used when evaluating 5-level priority triage systems; Lund Outcome Set for Evaluation of Triage (LOSET). The proposed outcomes include diagnoses, interventions and laboratory results. Before widespread implementation of LOSET, prospective testing is needed, preferably at multiple sites.

Clinical narrative-aware deep neural network for emergency department critical outcome prediction

Since early identification of potential critical patients in the Emergency Department (ED) can lower mortality and morbidity, this study seeks to develop a machine learning model capable of predicting possible critical outcomes based on the history and vital signs routinely collected at triage. We compare emergency physicians and the predictive performance of the machine learning model. Predictors including patients' chief complaints, present illness, past medical history, vital signs, and demographic data of adult patients (aged ≥ 18 years) visiting the ED at Shuang-Ho Hospital in New Taipei City, Taiwan, are extracted from the hospital's electronic health records. Critical outcomes are defined as in-hospital cardiac arrest (IHCA) or intensive care unit (ICU) admission. A clinical narrative-aware deep neural network was developed to handle the text-intensive data and standardized numerical data, which is compared against other machine learning models. After this, emergency physicians were asked to predict possible clinical outcomes of thirty visits that were extracted randomly from our dataset, and their results were further compared to our machine learning model. A total of 4,308 (2.5 %) out of the 171,275 adult visits to the ED included in this study resulted in critical outcomes. The area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) of our proposed prediction model is 0.874 and 0.207, respectively, which not only outperforms the other machine learning models, but even has better sensitivity (0.95 vs 0.41) and accuracy (0.90 vs 0.67) as compared to the emergency physicians. This model is sensitive and accurate in predicting critical outcomes and highlights the potential to use predictive analytics to support post-triage decision-making.

Support Systems of Clinical Decisions in the Triage of the Emergency Department Using Artificial Intelligence: The Efficiency to Support Triage

Purpose

In the Emergency Departments (ED) the current triage systems that are been implemented are based completely on medical education and the perception of each health professional who is in charge. On the other hand, cutting-edge technology, Artificial Intelligence (AI) can be incorporated into healthcare systems, supporting the healthcare professionals' decisions, and augmenting the performance of triage systems. The aim of the study is to investigate the efficiency of AI to support triage in ED.

Patients–Methods

The study included 332 patients from whom 23 different variables related to their condition were collected. From the processing of patient data for input variables, it emerged that the average age was 56.4 ± 21.1 years and 50.6% were male. The waiting time had an average of 59.7 ± 56.3 minutes while 3.9% ± 0.1% entered the Intensive Care Unit (ICU). In addition, qualitative variables related to the patient's history and admission clinics were used. As target variables were taken the days of stay in the hospital, which were on average 1.8 ± 5.9, and the Emergency Severity Index (ESI) for which the following distribution applies: ESI: 1, patients: 2; ESI: 2, patients: 18; ESI: 3, patients: 197; ESI: 4, patients: 73; ESI: 5, patients: 42.

Results

To create an automatic patient screening classifier, a neural network was developed, which was trained based on the data, so that it could predict each patient's ESI based on input variables.The classifier achieved an overall accuracy (F1 score) of 72.2% even though there was an imbalance in the classes.

Conclusions

The creation and implementation of an AI model for the automatic prediction of ESI, highlighted the possibility of systems capable of supporting healthcare professionals in the decision-making process. The accuracy of the classifier has not reached satisfactory levels of certainty, however, the performance of similar models can increase sharply with the collection of more data.