Improving ED Emergency Severity Index Acuity Assignment Using Machine Learning and Clinical Natural Language Processing

Triage at shift changes and distortions in the perception and treatment of emergency patients

Employing more than 2 million emergency department (ED) records, we combine machine learning and regression discontinuity to document novel distortions in triage nurses' assessments of patients' conditions and investigate the short- and medium-term consequences for patients. We show that triage nurses progressively become more lenient during their shifts, and identical ED patients arriving just after a shift change are thus assigned a lower priority. We show that these patients receive lower levels of care and require additional emergency care afterward. We conclude that distortions in nurses' initial assessments of urgency bias' medical staff's perceptions.

Determining Emergency Severity Index Acuity: Key Triage Elements Identified by Emergency Nurses

INTRODUCTION

The conflation of mandated screening question data collection with patient assessment at the initial triage encounter challenges the ability of the emergency nurse to identify patients at risk for deterioration rapidly and accurately. Further, inexperienced triage nurses are generally challenged in differentiating between questions that establish stability and questions that meet other requirements. The aims of the study included exploration of how triage nurses identified critical data elements to facilitate more rapid and accurate patient triage and Emergency Severity Index acuity assignment, perceptions of appropriate location of assessment elements, and identifying common triage processes.

METHODS

A quantitative descriptive exploratory study using survey data was used to address study aims.

RESULTS

Participants identified the following elements appropriate to triage as chief complaint, vital signs, allergies (and latex allergy), pain/pain description, weight, history of present illness, suicide risk, preferred language, Glasgow Coma Scale, pregnancy status/last menstrual period, travel history, infectious diseases, arrival method, height, and use of blood thinners. All other screenings were identified as "belonging" during provision of care, at discharge, or never.

DISCUSSION

Emergency nurses identified critical triage data necessary to assign an Emergency Severity Index level. We recommend that future research focus on evaluation of a triage process that removes screening not directly related to the triage decision in terms of nursing accuracy in assigning an Emergency Severity Index level and patient outcomes.

Risk factors for prediabetes in community-dwelling adults: A generalized estimating equation logistic regression approach with natural language processing insights

The global prevalence of prediabetes is expected to reach 8.3% (587 million people) by 2045, with 70% of people with prediabetes developing diabetes during their lifetimes. We aimed to classify community-dwelling adults with a high risk for prediabetes based on prediabetes-related symptoms and to identify their characteristics, which might be factors associated with prediabetes. We analyzed homecare nursing records (n = 26,840) of 1628 patients aged over 20 years. Using a natural language processing algorithm, we classified each nursing episode as either low-risk or high-risk for prediabetes based on the detected number and category of prediabetes-symptom words. To identify differences between the risk groups, we employed t-tests, chi-square tests, and data visualization. Risk factors for prediabetes were identified using multiple logistic regression models with generalized estimating equations. A total of 3270 episodes (12.18%) were classified as potentially high-risk for prediabetes. There were significant differences in the personal, social, and clinical factors between groups. Results revealed that female sex, age, cancer coverage as part of homecare insurance coverage, and family caregivers were significantly associated with an increased risk of prediabetes. Although prediabetes is not a life-threatening disease, uncontrolled blood glucose can cause unfavorable outcomes for other major diseases. Thus, medical professionals should consider the associated symptoms and risk factors of prediabetes. Moreover, the proposed algorithm may support the detection of individuals at a high risk for prediabetes. Implementing this approach could facilitate proactive monitoring and early intervention, leading to reduced healthcare expenses and better health outcomes for community-dwelling adults.

Advancing Emergency Department Triage Prediction With Machine Learning to Optimize Triage for Abdominal Pain Surgery Patients

BACKGROUND

The development of emergency department (ED) triage systems remains challenging in accurately differentiating patients with acute abdominal pain (AAP) who are critical and urgent for surgery due to subjectivity and limitations. We use machine learning models to predict emergency surgical abdominal pain patients in triage, and then compare their performance with conventional Logistic regression models.

METHODS

Using 38 214 patients presenting with acute abdominal pain at Zhongnan Hospital of Wuhan University between March 1, 2014, and March 1, 2022, we identified all adult patients (aged ≥18 years). We utilized routinely available triage data in electronic medical records as predictors, including structured data (eg, triage vital signs, gender, and age) and unstructured data (chief complaints and physical examinations in free-text format). The primary outcome measure was whether emergency surgery was performed. The dataset was randomly sampled, with 80% assigned to the training set and 20% to the test set. We developed 5 machine learning models: Light Gradient Boosting Machine (Light GBM), eXtreme Gradient Boosting (XGBoost), Deep Neural Network (DNN), and Random Forest (RF). Logistic regression (LR) served as the reference model. Model performance was calculated for each model, including the area under the receiver-work characteristic curve (AUC) and net benefit (decision curve), as well as the confusion matrix.

RESULTS

Of all the 38 214 acute abdominal pain patients, 4208 underwent emergency abdominal surgery while 34 006 received non-surgical treatment. In the surgery outcome prediction, all 4 machine learning models outperformed the reference model (eg, AUC, 0.899 [95%CI 0.891-0.903] in the Light GBM vs. 0.885 [95%CI 0.876-0.891] in the reference model), Similarly, most machine learning models exhibited significant improvements in net reclassification compared to the reference model (eg, NRIs of 0.0812[95%CI, 0.055-0.1105] in the XGBoost), with the exception of the RF model. Decision curve analysis shows that across the entire range of thresholds, the net benefits of the XGBoost and the Light GBM models were higher than the reference model. In particular, the Light GBM model performed well in predicting the need for emergency abdominal surgery with higher sensitivity, specificity, and accuracy.

CONCLUSIONS

Machine learning models have demonstrated superior performance in predicting emergency abdominal pain surgery compared to traditional models. Modern machine learning improves clinical triage decisions and ensures that critically needy patients receive priority for emergency resources and timely, effective treatment.

Improving triage performance in emergency departments using machine learning and natural language processing: a systematic review

BACKGROUND

In Emergency Departments (EDs), triage is crucial for determining patient severity and prioritizing care, typically using the Manchester Triage Scale (MTS). Traditional triage systems, reliant on human judgment, are prone to under-triage and over-triage, resulting in variability, bias, and incorrect patient classification. Studies suggest that Machine Learning (ML) and Natural Language Processing (NLP) could enhance triage accuracy and consistency. This review analyzes studies on ML and/or NLP algorithms for ED patient triage.

METHODS

Following Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines, we conducted a systematic review across five databases: Web of Science, PubMed, Scopus, IEEE Xplore, and ACM Digital Library, from their inception of each database to October 2023. The risk of bias was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Only articles employing at least one ML and/or NLP method for patient triage classification were included.

RESULTS

Sixty studies covering 57 ML algorithms were included. Logistic Regression (LR) was the most used model, while eXtreme Gradient Boosting (XGBoost), decision tree-based algorithms with Gradient Boosting (GB), and Deep Neural Networks (DNNs) showed superior performance. Frequent predictive variables included demographics and vital signs, with oxygen saturation, chief complaints, systolic blood pressure, age, and mode of arrival being the most retained. The ML algorithms showed significant bias risk due to critical bias assessment in classification models.

CONCLUSION

NLP methods improved ML algorithms' classification capability using triage nursing and medical notes and structured clinical data compared to algorithms using only structured data. Feature engineering (FE) and class imbalance correction methods enhanced ML workflows' performance, but FE and eXplainable Artificial Intelligence (XAI) were underexplored in this field. Registration and funding. This systematic review has been registered (registration number: CRD42024604529) in the International Prospective Register of Systematic Reviews (PROSPERO) and can be accessed online at the following URL: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=604529 . Funding for this work was provided by the National Council for Scientific and Technological Development (CNPq), Brazil.

The accuracy of triage classification using Emergency Severity Index

INTRODUCTION

Accurate emergency triage is essential for timely and appropriate care based on patient acuity. We sought to evaluate triage accuracy among emergency department (ED) nurses and examine potential influencing factors.

METHODS

We conducted an observational study using an electronic medical record chart review of 100 patients admitted at one of three EDs in a large academic medical system in the southern United States from December 1 to 7, 2021. Descriptive statistics were used to summarize the data. We compared the nurses' initial assigned triage acuity level documented in the medical chart with triage acuity assigned using the Emergency Severity Index Version 4 handbook and assessed inter-rater agreement using Cohen's kappa coefficient.

RESULTS

Overall triage accuracy was 67%, with most patients (62%) triaged as ESI level 3; under- and over-triage occurred in 25% and 8% of cases, respectively. Cohen's kappa coefficient was 0.437, indicating moderate interrater reliability between the triage nurses and the ESI handbook. Triage accuracy varied across ED locations and patient characteristics of sex (male: 55.6%, female: 72.3%), type of complaint (trauma: 57.1%, non-trauma: 69.4%), shift (day: 63.5%, night: 73.0%), and mode of arrival (ambulance: 80.8%, ambulatory: 65.2%, and private vehicle: 37.5%).

CONCLUSION

Triage inaccuracy may be attributed to a combination of nursing- and patient-specific factors. Further study of those factors associated with triage accuracy is warranted.

The Potential of Artificial Intelligence Tools for Reducing Uncertainty in Medicine and Directions for Medical Education

Unlabelled

In the field of medicine, uncertainty is inherent. Physicians are asked to make decisions on a daily basis without complete certainty, whether it is in understanding the patient's problem, performing the physical examination, interpreting the findings of diagnostic tests, or proposing a management plan. The reasons for this uncertainty are widespread, including the lack of knowledge about the patient, individual physician limitations, and the limited predictive power of objective diagnostic tools. This uncertainty poses significant problems in providing competent patient care. Research efforts and teaching are attempts to reduce uncertainty that have now become inherent to medicine. Despite this, uncertainty is rampant. Artificial intelligence (AI) tools, which are being rapidly developed and integrated into practice, may change the way we navigate uncertainty. In their strongest forms, AI tools may have the ability to improve data collection on diseases, patient beliefs, values, and preferences, thereby allowing more time for physician-patient communication. By using methods not previously considered, these tools hold the potential to reduce the uncertainty in medicine, such as those arising due to the lack of clinical information and provider skill and bias. Despite this possibility, there has been considerable resistance to the implementation of AI tools in medical practice. In this viewpoint article, we discuss the impact of AI on medical uncertainty and discuss practical approaches to teaching the use of AI tools in medical schools and residency training programs, including AI ethics, real-world skills, and technological aptitude.

Precision emergency medicine

BACKGROUND

Precision health is a burgeoning scientific discipline that aims to incorporate individual variability in biological, behavioral, and social factors to develop personalized health solutions. To date, emergency medicine has not deeply engaged in the precision health movement. However, rapid advances in health technology, data science, and medical informatics offer new opportunities for emergency medicine to realize the promises of precision health.

METHODS

In this article, we conceptualize precision emergency medicine as an emerging paradigm and identify key drivers of its implementation into current and future clinical practice. We acknowledge important obstacles to the specialty-wide adoption of precision emergency medicine and offer solutions that conceive a successful path forward.

RESULTS

Precision emergency medicine is defined as the use of information and technology to deliver acute care effectively, efficiently, and authentically to individual patients and their communities. Key drivers and opportunities include leveraging human data, capitalizing on technology and digital tools, providing deliberate access to care, advancing population health, and reimagining provider education and roles. Overcoming challenges in equity, privacy, and cost is essential for success. We close with a call to action to proactively incorporate precision health into the clinical practice of emergency medicine, the training of future emergency physicians, and the research agenda of the specialty.

CONCLUSIONS

Precision emergency medicine leverages new technology and data-driven artificial intelligence to advance diagnostic testing, individualize patient care plans and therapeutics, and strategically refine the convergence of the health system and the community.

Emergency department triaging using ChatGPT based on emergency severity index principles: a cross-sectional study

Erroneous and delayed triage in an increasingly crowded emergency department (ED). ChatGPT is an artificial intelligence model developed by OpenAI® and is being trained for use in natural language processing tasks. Our study aims to determine the accuracy of patient triage using ChatGPT according to the emergency severity index (ESI) for triage in EDs. In our cross-sectional study, 18 years and over patients who consecutively presented to our ED within 24 h were included. Age, gender, admission method, chief complaint, state of consciousness, and comorbidities were recorded on the case form, and the vital signs were detected at the triage desk. A five-member expert committee (EC) was formed from the fourth-year resident physicians. The investigators converted real-time patient information into a standardized case format. The urgency status of the patients was evaluated simultaneously by EC and ChatGPT according to ESI criteria. The median value of the EC decision was accepted as the gold standard. There was a statistically significant moderate agreement between EC and ChatGPT assessments regarding urgency status (Cohen's Kappa = 0.659; P < 0.001). The accuracy between these two assessments was calculated as 76.6%. There was a high degree of agreement between EC and ChatGPT for the prediction of ESI-1 and 2, indicating high acuity (Cohen's Kappa = 0.828). The diagnostic specificity, NPV, and accuracy of ChatGPT were determined as 95.63, 98.17 and 94.90%, respectively, for ESI high acuity categories. Our study shows that ChatGPT can successfully differentiate patients with high urgency. The findings are promising for integrating artificial intelligence-based applications such as ChatGPT into triage processes in EDs.

The AI Future of Emergency Medicine

In the coming years, artificial intelligence (AI) and machine learning will likely give rise to profound changes in the field of emergency medicine, and medicine more broadly. This article discusses these anticipated changes in terms of 3 overlapping yet distinct stages of AI development. It reviews some fundamental concepts in AI and explores their relation to clinical practice, with a focus on emergency medicine. In addition, it describes some of the applications of AI in disease diagnosis, prognosis, and treatment, as well as some of the practical issues that they raise, the barriers to their implementation, and some of the legal and regulatory challenges they create.

How artificial intelligence could transform emergency care

Artificial intelligence (AI) in healthcare is the ability of a computer to perform tasks typically associated with clinical care (e.g. medical decision-making and documentation). AI will soon be integrated into an increasing number of healthcare applications, including elements of emergency department (ED) care. Here, we describe the basics of AI, various categories of its functions (including machine learning and natural language processing) and review emerging and potential future use-cases for emergency care. For example, AI-assisted symptom checkers could help direct patients to the appropriate setting, models could assist in assigning triage levels, and ambient AI systems could document clinical encounters. AI could also help provide focused summaries of charts, summarize encounters for hand-offs, and create discharge instructions with an appropriate language and reading level. Additional use cases include medical decision making for decision rules, real-time models that predict clinical deterioration or sepsis, and efficient extraction of unstructured data for coding, billing, research, and quality initiatives. We discuss the potential transformative benefits of AI, as well as the concerns regarding its use (e.g. privacy, data accuracy, and the potential for changing the doctor-patient relationship).

Can artificial intelligence help ED nurses more accurately triage patients?

ABSTRACT

The Emergency Severity Index (ESI) is the most popular tool used to triage patients in the US and abroad. Evidence has shown that ESI has its limitations in correctly assigning acuity. To address this, AI can be incorporated into the triage process, decreasing the likelihood of assigning an incorrect ESI level.

Artificial intelligence to advance acute and intensive care medicine

PURPOSE OF REVIEW

This review explores recent key advancements in artificial intelligence for acute and intensive care medicine. As artificial intelligence rapidly evolves, this review aims to elucidate its current applications, future possibilities, and the vital challenges that are associated with its integration into emergency medical dispatch, triage, medical consultation and ICUs.

RECENT FINDINGS

The integration of artificial intelligence in emergency medical dispatch (EMD) facilitates swift and accurate assessment. In the emergency department (ED), artificial intelligence driven triage models leverage diverse patient data for improved outcome predictions, surpassing human performance in retrospective studies. Artificial intelligence can streamline medical documentation in the ED and enhances medical imaging interpretation. The introduction of large multimodal generative models showcases the future potential to process varied biomedical data for comprehensive decision support. In the ICU, artificial intelligence applications range from early warning systems to treatment suggestions.

SUMMARY

Despite promising academic strides, widespread artificial intelligence adoption in acute and critical care is hindered by ethical, legal, technical, organizational, and validation challenges. Despite these obstacles, artificial intelligence's potential to streamline clinical workflows is evident. When these barriers are overcome, future advancements in artificial intelligence have the potential to transform the landscape of patient care for acute and intensive care medicine.

Use of Artificial Intelligence in Triage in Hospital Emergency Departments: A Scoping Review

The integration of artificial intelligence (AI) and machine learning (ML) in healthcare has become a major point of interest and raises the question of its impact on the emergency department (ED) triaging process. AI's capacity to emulate human cognitive processes coupled with advancements in computing has shown positive outcomes in various aspects of healthcare but little is known about the use of AI in triaging patients in ED. AI algorithms may allow for earlier diagnosis and intervention; however, overconfident answers may present dangers to patients. The purpose of this review was to explore comprehensively recently published literature regarding the effect of AI and ML in ED triage and identify research gaps. A systemized search was conducted in September 2023 using the electronic databases EMBASE, Ovid MEDLINE, and Web of Science. To meet inclusion criteria, articles had to be peer-reviewed, written in English, and based on primary data research studies published in US journals 2013-2023. Other criteria included 1) studies with patients needing to be admitted to hospital EDs, 2) AI must have been used when triaging a patient, and 3) patient outcomes must be represented. The search was conducted using controlled descriptors from the Medical Subject Headings (MeSH) that included the terms "artificial intelligence" OR "machine learning" AND "emergency ward" OR "emergency care" OR "emergency department" OR "emergency room" AND "patient triage" OR "triage" OR "triaging." The search initially identified 1,142 citations. After a rigorous, systemized screening process and critical appraisal of the evidence, 29 studies were selected for the final review. The findings indicated that 1) ML models consistently demonstrated superior discrimination abilities compared to conventional triage systems, 2) the integration of AI into the triage process yielded significant enhancements in predictive accuracy, disease identification, and risk assessment, 3) ML accurately determined the necessity of hospitalization for patients requiring urgent attention, and 4) ML improved resource allocation and quality of patient care, including predicting length of stay. The suggested superiority of ML models in prioritizing patients in the ED holds the potential to redefine triage precision.

The Relationship Between Accurate Triage and Core Measures Compliance for Acute Myocardial Infarction and Heart Failure in Older Adults Presenting to the Emergency Department

BACKGROUND

Accurate emergency department (ED) triage in the geriatric population is an important nurse-sensitive quality indicator; however, few quality indicators are verified for impact.

PURPOSE

To examine the relationship between triage accuracy in adults older than 65 years and Core Measures for acute myocardial infarction (AMI) and heart failure (HF).

METHODS

A correlational approach was used to determine strength and direction of the relationship between variables.

RESULTS

Strong positive correlations were found between triage accuracy and mortality for AMI and HF, as well as with 30-day hospital readmissions for AMI. A weak negative correlation was found between triage accuracy and 30-day hospital readmissions for HF.

CONCLUSIONS

Accurate triage can lead to a more effective care trajectory for patients, better adherence to Core Measures, and better outcomes. Accuracy in triage for AMI and HF is a valid indicator of ED quality care.

Interpretable Deep Learning System for Identifying Critical Patients Through the Prediction of Triage Level, Hospitalization, and Length of Stay: Prospective Study

BACKGROUND

Triage is the process of accurately assessing patients' symptoms and providing them with proper clinical treatment in the emergency department (ED). While many countries have developed their triage process to stratify patients' clinical severity and thus distribute medical resources, there are still some limitations of the current triage process. Since the triage level is mainly identified by experienced nurses based on a mix of subjective and objective criteria, mis-triage often occurs in the ED. It can not only cause adverse effects on patients, but also impose an undue burden on the health care delivery system.

OBJECTIVE

Our study aimed to design a prediction system based on triage information, including demographics, vital signs, and chief complaints. The proposed system can not only handle heterogeneous data, including tabular data and free-text data, but also provide interpretability for better acceptance by the ED staff in the hospital.

METHODS

In this study, we proposed a system comprising 3 subsystems, with each of them handling a single task, including triage level prediction, hospitalization prediction, and length of stay prediction. We used a large amount of retrospective data to pretrain the model, and then, we fine-tuned the model on a prospective data set with a golden label. The proposed deep learning framework was built with TabNet and MacBERT (Chinese version of bidirectional encoder representations from transformers [BERT]).

RESULTS

The performance of our proposed model was evaluated on data collected from the National Taiwan University Hospital (901 patients were included). The model achieved promising results on the collected data set, with accuracy values of 63%, 82%, and 71% for triage level prediction, hospitalization prediction, and length of stay prediction, respectively.

CONCLUSIONS

Our system improved the prediction of 3 different medical outcomes when compared with other machine learning methods. With the pretrained vital sign encoder and repretrained mask language modeling MacBERT encoder, our multimodality model can provide a deeper insight into the characteristics of electronic health records. Additionally, by providing interpretability, we believe that the proposed system can assist nursing staff and physicians in taking appropriate medical decisions.

Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation

OBJECTIVE

Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementation. This is the first of such studies to investigate real-time AI performance in the emergency setting.

METHODS

The novel AI algorithm that predicts admission using a triage note was translated into clinical practice and integrated within St Vincent's Hospital Melbourne's electronic emergency patient management system. The data were collected from 1 January 2021 to 17 August 2022 to evaluate the diagnostic accuracy of the AI system after implementation.

RESULTS

A total of 77 125 ED presentations were included. The live AI algorithm has a sensitivity of 73.1% (95% confidence interval 72.5-73.8), specificity of 74.3% (73.9-74.7), positive predictive value of 50% (49.6-50.4) and negative predictive value of 88.7% (88.5-89) with a total accuracy of 74% (73.7-74.3). The accuracy of the system was at the lowest for admission to psychiatric units (34%) and at the highest for gastroenterology and medical admission (84% and 80%, respectively).

CONCLUSION

Our study showed the diagnostic evaluation of a real-time AI clinical decision-support tool became less accurate than the original. Although real-time sensitivity and specificity of the AI tool was still acceptable as a decision-support tool in the ED, we propose that continuous training and evaluation of AI-enabled clinical support tools in healthcare are conducted to ensure consistent accuracy and performance to prevent inadvertent consequences.

Applications of natural language processing at emergency department triage: A narrative review

INTRODUCTION

Natural language processing (NLP) uses various computational methods to analyse and understand human language, and has been applied to data acquired at Emergency Department (ED) triage to predict various outcomes. The objective of this scoping review is to evaluate how NLP has been applied to data acquired at ED triage, assess if NLP based models outperform humans or current risk stratification techniques when predicting outcomes, and assess if incorporating free-text improve predictive performance of models when compared to predictive models that use only structured data.

METHODS

All English language peer-reviewed research that applied an NLP technique to free-text obtained at ED triage was eligible for inclusion. We excluded studies focusing solely on disease surveillance, and studies that used information obtained after triage. We searched the electronic databases MEDLINE, Embase, Cochrane Database of Systematic Reviews, Web of Science, and Scopus for medical subject headings and text keywords related to NLP and triage. Databases were last searched on 01/01/2022. Risk of bias in studies was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Due to the high level of heterogeneity between studies and high risk of bias, a metanalysis was not conducted. Instead, a narrative synthesis is provided.

RESULTS

In total, 3730 studies were screened, and 20 studies were included. The population size varied greatly between studies ranging from 1.8 million patients to 598 triage notes. The most common outcomes assessed were prediction of triage score, prediction of admission, and prediction of critical illness. NLP models achieved high accuracy in predicting need for admission, triage score, critical illness, and mapping free-text chief complaints to structured fields. Incorporating both structured data and free-text data improved results when compared to models that used only structured data. However, the majority of studies (80%) were assessed to have a high risk of bias, and only one study reported the deployment of an NLP model into clinical practice.

CONCLUSION

Unstructured free-text triage notes have been used by NLP models to predict clinically relevant outcomes. However, the majority of studies have a high risk of bias, most research is retrospective, and there are few examples of implementation into clinical practice. Future work is needed to prospectively assess if applying NLP to data acquired at ED triage improves ED outcomes when compared to usual clinical practice.

AI and machine learning in resuscitation: Ongoing research, new concepts, and key challenges

Aim

Artificial intelligence (AI) and machine learning (ML) are important areas of computer science that have recently attracted attention for their application to medicine. However, as techniques continue to advance and become more complex, it is increasingly challenging for clinicians to stay abreast of the latest research. This overview aims to translate research concepts and potential concerns to healthcare professionals interested in applying AI and ML to resuscitation research but who are not experts in the field.

Main text

We present various research including prediction models using structured and unstructured data, exploring treatment heterogeneity, reinforcement learning, language processing, and large-scale language models. These studies potentially offer valuable insights for optimizing treatment strategies and clinical workflows. However, implementing AI and ML in clinical settings presents its own set of challenges. The availability of high-quality and reliable data is crucial for developing accurate ML models. A rigorous validation process and the integration of ML into clinical practice is essential for practical implementation. We furthermore highlight the potential risks associated with self-fulfilling prophecies and feedback loops, emphasizing the importance of transparency, interpretability, and trustworthiness in AI and ML models. These issues need to be addressed in order to establish reliable and trustworthy AI and ML models.

Conclusion

In this article, we overview concepts and examples of AI and ML research in the resuscitation field. Moving forward, appropriate understanding of ML and collaboration with relevant experts will be essential for researchers and clinicians to overcome the challenges and harness the full potential of AI and ML in resuscitation.

Medical emergency department triage data processing using a machine-learning solution

Over the years, artificial intelligence has demonstrated its ability to overcome many challenges in our day-to-day life. The evolution of it inquired more studies about Machine Learning possible solutions for different domains, including health care. The increasing demand for artificial intelligence solutions has brought accessibility to loads of data, including clinical data. The availability of medical records facilitates new opportunities to explore Machine Learning models and their abilities to process a significant amount of data and to identify patterns with the purpose of solving a medical problem. Understanding the applicability of artificial intelligence on this type of data has to be a compelling aim for emergency medicine clinicians. This paper focuses on the general clinical problem of the complex correlation between medical records and later diagnosis and, especially, on the process of emergency department triage which uses the Emergency Severity Index (ESI) as triage protocol. This study presents a comparison between three different Machine Learning models, such as Logistic Regression, Random Forest Tree and NN-Sequentail, with the purpose of classifying patients with an emergency code. We conducted four experiments because of imbalanced data. A web-based application was developed to improve the triage process after our theoretical and exploratory results. Overall, in all experiments, the NN-Sequential model had better results, having, in the first experiment, a ROC-AUC score for each ESI emergency code of: 0.59%, 0.76%, 0.71%, 0.78% 0.64%. After applying methods to balance the data, the model yielded a ROC-AUC score for each emergency code of 0.72%, 0.75%, 0.69%, 0.74%, 0.78%. In the last experiment consisting of a three-class classification problem, the NN-Sequential and Random Forest Tree models had similar metric outcomes, and the NN-Sequential algorithm had a ROC-AUC score for each emergency code of: 0.76%, 0.72%, 0.84%. Without any doubt, our research results presented in this paper endorse this tremendous curiosity in Machine Learning applications to enrich aspects of emergency medical care by applying specific methods for processing both medical data and medical records.

Performance of emergency triage prediction of an open access natural language processing based chatbot application (ChatGPT): A preliminary, scenario-based cross-sectional study

OBJECTIVES

Artificial intelligence companies have been increasing their initiatives recently to improve the results of chatbots, which are software programs that can converse with a human in natural language. The role of chatbots in health care is deemed worthy of research. OpenAI's ChatGPT is a supervised and empowered machine learning-based chatbot. The aim of this study was to determine the performance of ChatGPT in emergency medicine (EM) triage prediction.

METHODS

This was a preliminary, cross-sectional study conducted with case scenarios generated by the researchers based on the emergency severity index (ESI) handbook v4 cases. Two independent EM specialists who were experts in the ESI triage scale determined the triage categories for each case. A third independent EM specialist was consulted as arbiter, if necessary. Consensus results for each case scenario were assumed as the reference triage category. Subsequently, each case scenario was queried with ChatGPT and the answer was recorded as the index triage category. Inconsistent classifications between the ChatGPT and reference category were defined as over-triage (false positive) or under-triage (false negative).

RESULTS

Fifty case scenarios were assessed in the study. Reliability analysis showed a fair agreement between EM specialists and ChatGPT (Cohen's Kappa: 0.341). Eleven cases (22%) were over triaged and 9 (18%) cases were under triaged by ChatGPT. In 9 cases (18%), ChatGPT reported two consecutive triage categories, one of which matched the expert consensus. It had an overall sensitivity of 57.1% (95% confidence interval [CI]: 34-78.2), specificity of 34.5% (95% CI: 17.9-54.3), positive predictive value (PPV) of 38.7% (95% CI: 21.8-57.8), negative predictive value (NPV) of 52.6 (95% CI: 28.9-75.6), and an F1 score of 0.461. In high acuity cases (ESI-1 and ESI-2), ChatGPT showed a sensitivity of 76.2% (95% CI: 52.8-91.8), specificity of 93.1% (95% CI: 77.2-99.2), PPV of 88.9% (95% CI: 65.3-98.6), NPV of 84.4 (95% CI: 67.2-94.7), and an F1 score of 0.821. The receiver operating characteristic curve showed an area under the curve of 0.846 (95% CI: 0.724-0.969, P < 0.001) for high acuity cases.

CONCLUSION

The performance of ChatGPT was best when predicting high acuity cases (ESI-1 and ESI-2). It may be useful when determining the cases requiring critical care. When trained with more medical knowledge, ChatGPT may be more accurate for other triage category predictions.

Effect of triage training on nurses' practice and triage outcomes of patients with acute coronary syndrome

BACKGROUND

Accurate assessment and prompt management of patients with acute coronary syndrome (ACS) is a complex process for emergency department (ED) nurses and has variable clinical outcomes. The aim of the present study was to determine the effectiveness of an educational intervention on nurses' practice during the triage of patients with ACS and the triage outcomes in this group of patients.

METHODS

In this quasi-experimental study, a pretest-posttest group of 24 nurses were included by convenience sampling method and 960 patients with ACS were selected by sequential sampling during the pre-intervention (n = 480) and post-intervention (n = 480) phases. A case-based learning (CBL) intervention was performed for nurses for one month considering the role of the triage nurse according to the American College of Cardiology (ACC) and the American Heart Association (AHA) recommendations as well as the factors affecting the proper identification and management of patients with ACS. During patient triage in the pre- and post-intervention phases, the "Triage Nurse Practice Checklist" and the "Medical Electronic Records" were used to assess nurses' practice and the triage outcomes in patients, respectively.

RESULTS

The overall mean score of the triage nurses' practice and its subscales, including Primary monitoring and assessment, cardiovascular risk factors assessment, evaluation of coronary heart disease (CHD) symptoms, chest pain management, and adherence to the ACC/AHA practice guidelines were significantly improved in the post-intervention phase compared with the pre-intervention phase (p < 0.001). There was no significant difference between the triage outcomes, including in-hospital mortality within 24 hours, death in ED, hospitalization in other wards, and discharge from ED in the pre and post-intervention phases (P = 0.723).

CONCLUSION

The development of a cardiac triage-specific educational program could improve the performance of nurses in the evaluation and management of patients with ACS, but had no effect on the triage outcomes in this group of patients. We recommend a quality improvement project or a critical outcomes-based triage system to assess ACS patients' care needs in the ED.

Demystifying the Role of Natural Language Processing (NLP) in Smart City Applications: Background, Motivation, Recent Advances, and Future Research Directions

Smart cities provide an efficient infrastructure for the enhancement of the quality of life of the people by aiding in fast urbanization and resource management through sustainable and scalable innovative solutions. The penetration of Information and Communication Technology (ICT) in smart cities has been a major contributor to keeping up with the agility and pace of their development. In this paper, we have explored Natural Language Processing (NLP) which is one such technical discipline that has great potential in optimizing ICT processes and has so far been kept away from the limelight. Through this study, we have established the various roles that NLP plays in building smart cities after thoroughly analyzing its architecture, background, and scope. Subsequently, we present a detailed description of NLP's recent applications in the domain of smart healthcare, smart business, and industry, smart community, smart media, smart research, and development as well as smart education accompanied by NLP's open challenges at the very end. This work aims to throw light on the potential of NLP as one of the pillars in assisting the technical advancement and realization of smart cities.

The Use and Structure of Emergency Nurses' Triage Narrative Data: Scoping Review

BACKGROUND

Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner.

OBJECTIVE

The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted.

METHODS

We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research.

RESULTS

We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses' narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%).

CONCLUSIONS

The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055132.

The Impact of Cultural Embeddedness on the Implementation of an Artificial Intelligence Program at Triage: A Qualitative Study

INTRODUCTION

Triage requires rapid determination of acuity and resources. Current modalities allow for individual judgment, with varied application of algorithmic rules. Although artificial intelligence can improve triage accuracy, gaps remain in understanding implementation facilitators and barriers, especially those related to the cultural understandings by nurses of emergency department presentations. The purpose of this study was to explore the cultural and technological elements of the implementation of an artificial intelligence clinical decision support aid (i.e., KATE) in an emergency nursing triage process in an urban community hospital on the West Coast of the United States.

METHOD

An exploratory qualitative study using semi-structured small group and individual interviews and constant comparison analysis strategies. The sample comprised 13 emergency department triage nurses at one site. Campinha-Bacote's theory of cultural competence framed the study.

RESULTS

Responses yielded the overall theme of We know these people and we know these things. Supporting categories included the problem of aire; just another checkbox; gut trumps data; higher acuity with no resources; and technology as a safety net. Participants reported reliance on clinical experience and cultural knowledge to assign acuity.

DISCUSSION

The implementation of an artificial intelligence program was initially received skeptically due to the acontextual nature of AI, but grew to be perceived as a safety net for triage decision making among emergency nurses.

Using natural language processing in facilitating pre-hospital telephone triage of emergency calls

Introduction

Natural language processing (NLP) is an area of computer science that involves the use of computers to understand human language and semantics (meaning) and to offer consistent and reliable responses. There is good evidence of significant advancement in the use of NLP technology in dealing with acutely ill patients in hospital (such as differential diagnosis assistance, clinical decision-making and treatment options). Further technical development and research into the use of NLP could enable further improvements in the quality of pre-hospital emergency care. The aim of this literature review was to explore the opportunities and potential obstacles in implementing NLP during this phase of emergency care and to question if NLP could contribute towards improving the process of nature of call screening (NoCS) to enable earlier recognition of life-threatening situations during telephone triage of emergency calls.

Methods

A systematic search strategy using two electronic databases (CINAHL and MEDLINE) was conducted in December 2021. The PRISMA systematic approach was used to conduct a review of the literature, and selected studies were identified and used to support a critical review of the actual and potential use of NLP for the call-taking phase of emergency care.

Results

An initial search offered 204 records: 23 remained after eliminating duplicates and a consideration of title and abstracts. A further 16 full-text articles were deemed ineligible (not related to the subject under investigation), leaving seven included studies. Following a thematic review of these studies two themes emerged, that are considered individually and together: (i) use of NLP for dealing with out-of-hospital cardiac arrest and (ii) responding to increased accuracy of NLP.

Conclusions

NLP has the potential to reduce or eliminate human bias during the emergency triage assessment process and contribute towards improving triage accuracy in pre-hospital decision-making and an early identification and categorisation of life-threatening conditions. Evidence to date is mostly linked to cardiac arrest identification; this review proposes that during the call-taking phase NLP should be extended to include further medical emergencies (including fracture/trauma, stroke and ketoacidosis). Further research is indicated to test the reliability of these findings and a proportionate introduction of NLP simultaneous with increased quality and reliability.

Emergency nurses' triage narrative data, their uses and structure: a scoping review protocol

INTRODUCTION

The first clinical interaction most patients have in the emergency department occurs during triage. An unstructured narrative is generated during triage and is the first source of in-hospital documentation. These narratives capture the patient's reported reason for the visit and the initial assessment and offer significantly more nuanced descriptions of the patient's complaints than fixed field data. Previous research demonstrated these data are useful for predicting important clinical outcomes. Previous reviews examined these narratives in combination or isolation with other free-text sources, but used restricted searches and are becoming outdated. Furthermore, there are no reviews focused solely on nurses' (the primary collectors of these data) narratives.

METHODS AND ANALYSIS

Using the Arksey and O'Malley scoping review framework and PRISMA-ScR reporting guidelines, we will perform structured searches of CINAHL, Ovid MEDLINE, ProQuest Central, Ovid Embase and Cochrane Library (via Wiley). Additionally, we will forward citation searches of all included studies. No geographical or study design exclusion criteria will be used. Studies examining disaster triage, published before 1990, and non-English language literature will be excluded. Data will be managed using online management tools; extracted data will be independently confirmed by a separate reviewer using prepiloted extraction forms. Cohen's kappa will be used to examine inter-rater agreement on pilot and final screening. Quantitative data will be expressed using measures of range and central tendency, counts, proportions and percentages, as appropriate. Qualitative data will be narrative summaries of the authors' primary findings.

PATIENT AND PUBLIC INVOLVEMENT

No patients involved.

ETHICS AND DISSEMINATION

No ethics approval is required. Findings will be submitted to peer-reviewed conferences and journals. Results will be disseminated using individual and institutional social media platforms.

ESI Triage Distribution in U.S. Emergency Departments

The accurate triage of arriving emergency department (ED) patients is a key component of emergency nursing practice. Overtriage assignment of patients misallocates scarce resources in a time of department overcrowding, whereas patient undertriage can create risks for negative patient outcomes secondary to care delays. Limited evidence is available regarding ED triage accuracy. It is estimated that appropriate adherence to the Emergency Severity Index (ESI) triage tool and assigning triage categories could be as low as 60% (McFarlane, 2019a, 2019b). The purpose of this retrospective observational study was to examine the 2019 triage distribution of 954,847 ED encounters at 25 hospitals. Comparisons were then made with the spreads identified in the ESI Implementation Handbook (Gilboy, Tanabe, Travers, & Rosenau, 2020). Study results reflect the presence of wide variations in distribution when compared with the expected spread published by Gilboy et al. (2020). These variations illustrate the need for further facility-level evaluation. ESI Level 2 percentages varied from as little as 2.6% to as high as 69% of each facility's ED visit population. Examining an individual facility's annualized triage distribution may serve as a swift method in determining whether additional investigation into triage accuracy is warranted. EDs must implement and sustain an ongoing quality control program to achieve and maintain triage inter- and intrarater reliability. Further research is needed on the value of triage inaccuracy with real-time feedback on nurses' clinical decision-making and patient outcomes. It is also imperative that the expected and observed ESI triage distribution in U.S. EDs is updated when established accuracy quality control programs are present.

Artificial intelligence and machine learning in emergency medicine: a narrative review

AIM

The emergence and evolution of artificial intelligence (AI) has generated increasing interest in machine learning applications for health care. Specifically, researchers are grasping the potential of machine learning solutions to enhance the quality of care in emergency medicine.

METHODS

We undertook a narrative review of published works on machine learning applications in emergency medicine and provide a synopsis of recent developments.

RESULTS

This review describes fundamental concepts of machine learning and presents clinical applications for triage, risk stratification specific to disease, medical imaging, and emergency department operations. Additionally, we consider how machine learning models could contribute to the improvement of causal inference in medicine, and to conclude, we discuss barriers to safe implementation of AI.

CONCLUSION

We intend that this review serves as an introduction to AI and machine learning in emergency medicine.

Machine learning methods applied to triage in emergency services: A systematic review

BACKGROUND

In emergency services is important to accurately assess and classify symptoms, which may be improved with the help of technology. One mechanism that could help and improve predictions from health records or patient flow is machine learning (ML).

AIM

To analyse the effectiveness of ML systems in triage for making predictions at the emergency department in comparison with other triage scales/scores.

METHODS

Following the PRISMA recommendations, a systematic review was conducted using CINAHL, Cochrane, Cuiden, Medline and Scopus databases with the search equation "Machine learning AND triage AND emergency".

RESULTS

Eleven studies were identified. The studies show that the use of ML methods consistently predict important outcomes like mortality, critical care outcomes and admission, and the need for hospitalization in comparison with scales like Emergency Severity Index or others. Among the ML models considered, XGBoost and Deep Neural Networks obtained the highest levels of prediction accuracy, while Logistic Regression performed obtained the worst values.

CONCLUSIONS

Machine learning methods can be a good instrument for helping triage process with the prediction of important emergency variables like mortality or the need for critical care or hospitalization.

Prediction of hospitalization using artificial intelligence for urgent patients in the emergency department

Timely assessment to accurately prioritize patients is crucial for emergency department (ED) management. Urgent (i.e., level-3, on a 5-level emergency severity index system) patients have become a challenge since under-triage and over-triage often occur. This study was aimed to develop a computational model by artificial intelligence (AI) methodologies to accurately predict urgent patient outcomes using data that are readily available in most ED triage systems. We retrospectively collected data from the ED of a tertiary teaching hospital between January 1, 2015 and December 31, 2019. Eleven variables were used for data analysis and prediction model building, including 1 response, 2 demographic, and 8 clinical variables. A model to predict hospital admission was developed using neural networks and machine learning methodologies. A total of 282,971 samples of urgent (level-3) visits were included in the analysis. Our model achieved a validation area under the curve (AUC) of 0.8004 (95% CI 0.7963-0.8045). The optimal cutoff value identified by Youden's index for determining hospital admission was 0.5517. Using this cutoff value, the sensitivity was 0.6721 (95% CI 0.6624-0.6818), and the specificity was 0.7814 (95% CI 0.7777-0.7851), with a positive predictive value of 0.3660 (95% CI 0.3586-0.3733) and a negative predictive value of 0.9270 (95% CI 0.9244-0.9295). Subgroup analysis revealed that this model performed better in the nontraumatic adult subgroup and achieved a validation AUC of 0.8166 (95% CI 0.8199-0.8212). Our AI model accurately assessed the need for hospitalization for urgent patients, which constituted nearly 70% of ED visits. This model demonstrates the potential for streamlining ED operations using a very limited number of variables that are readily available in most ED triage systems. Subgroup analysis is an important topic for future investigation.