Comparison of the Emergency Severity Index versus the Patient Acuity Category Scale in an emergency setting

FAIM: Fairness-aware interpretable modeling for trustworthy machine learning in healthcare

The escalating integration of machine learning in high-stakes fields such as healthcare raises substantial concerns about model fairness. We propose an interpretable framework, fairness-aware interpretable modeling (FAIM), to improve model fairness without compromising performance, featuring an interactive interface to identify a "fairer" model from a set of high-performing models and promoting the integration of data-driven evidence and clinical expertise to enhance contextualized fairness. We demonstrate FAIM's value in reducing intersectional biases arising from race and sex by predicting hospital admission with two real-world databases, the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) and the database collected from Singapore General Hospital Emergency Department (SGH-ED). For both datasets, FAIM models not only exhibit satisfactory discriminatory performance but also significantly mitigate biases as measured by well-established fairness metrics, outperforming commonly used bias mitigation methods. Our approach demonstrates the feasibility of improving fairness without sacrificing performance and provides a modeling mode that invites domain experts to engage, fostering a multidisciplinary effort toward tailored AI fairness.

FedScore: A privacy-preserving framework for federated scoring system development

OBJECTIVE

We propose FedScore, a privacy-preserving federated learning framework for scoring system generation across multiple sites to facilitate cross-institutional collaborations.

MATERIALS AND METHODS

The FedScore framework includes five modules: federated variable ranking, federated variable transformation, federated score derivation, federated model selection and federated model evaluation. To illustrate usage and assess FedScore's performance, we built a hypothetical global scoring system for mortality prediction within 30 days after a visit to an emergency department using 10 simulated sites divided from a tertiary hospital in Singapore. We employed a pre-existing score generator to construct 10 local scoring systems independently at each site and we also developed a scoring system using centralized data for comparison.

RESULTS

We compared the acquired FedScore model's performance with that of other scoring models using the receiver operating characteristic (ROC) analysis. The FedScore model achieved an average area under the curve (AUC) value of 0.763 across all sites, with a standard deviation (SD) of 0.020. We also calculated the average AUC values and SDs for each local model, and the FedScore model showed promising accuracy and stability with a high average AUC value which was closest to the one of the pooled model and SD which was lower than that of most local models.

CONCLUSION

This study demonstrates that FedScore is a privacy-preserving scoring system generator with potentially good generalizability.

Mental health-related presentations to a tertiary emergency department during the COVID-19 pandemic

Introduction

The coronavirus disease 2019 (COVID-19) pandemic has caused significant mental distress in populations globally. At the frontline of the pandemic, emergency departments (EDs) are the prime setting to observe the effects of the pandemic on the mental health of the population. We aimed to describe the trend of mental health-related ED attendances at an acute hospital in Singapore before and during the various stages of the COVID-19 pandemic.

Methods

This is a retrospective, descriptive study of patients who presented to the ED between 1 January 2019 and 31 December 2020. Patients diagnosed with mental health-related systematised nomenclature of medicine who visited the ED during this period were identified and were placed into mental health diagnosis categories for analysis. A comparison was made between patients who presented before the pandemic (2019) and during the pandemic (2020).

Results

During the study periods, we identified 1,421 patients, of whom 27 were excluded due to non-mental health-related diagnoses, leaving 1,394 patients for analysis. There was a 36.7% increase in mental health-related ED presentations from 2019 to 2020. The proportion of higher-acuity mental health-related ED attendances and number of suicide attempts also increased.

Conclusion

Our study described an increase in the proportion of high-acuity mental health-related ED attendances during the COVID-19 pandemic. Emergency physicians must be cognisant of the effects of the pandemic on mental health. Further research should be conducted to better equip the healthcare system for handling all aspects of the pandemic.

Building a Machine Learning-based Ambulance Dispatch Triage Model for Emergency Medical Services

Background

In charge of dispatching the ambulances, Emergency Medical Services (EMS) call center specialists often have difficulty deciding the acuity of a case given the information they can gather within a limited time. Although there are protocols to guide their decision-making, observed performance can still lack sensitivity and specificity. Machine learning models have been known to capture complex relationships that are subtle, and well-trained data models can yield accurate predictions in a split of a second.

Methods

In this study, we proposed a proof-of-concept approach to construct a machine learning model to better predict the acuity of emergency cases. We used more than 360,000 structured emergency call center records of cases received by the national emergency call center in Singapore from 2018 to 2020. Features were created using call records, and multiple machine learning models were trained.

Results

A Random Forest model achieved the best performance, reducing the over-triage rate by an absolute margin of 15% compared to the call center specialists while maintaining a similar level of under-triage rate.

Conclusions

The model has the potential to be deployed as a decision support tool for dispatchers alongside current protocols to optimize ambulance dispatch triage and the utilization of emergency ambulance resources.

Protocol for a multi-site randomized controlled trial of a stepped-care intervention for emergency department patients with panic-related anxiety

BACKGROUND

Approximately 40% of Emergency Department (ED) patients with chest pain meet diagnostic criteria for panic-related anxiety, but only 1-2% are correctly diagnosed and appropriately managed in the ED. A stepped-care model, which focuses on providing evidence-based interventions in a resource-efficient manner, is the state-of-the art for treating panic disorder patients in medical settings such as primary care. Stepped-care has yet to be tested in the ED setting, which is the first point of contact with the healthcare system for most patients with panic symptoms.

METHODS

This multi-site randomized controlled trial (RCT) aims to evaluate the clinical, patient-centred, and economic effectiveness of a stepped-care intervention in a sample of 212 patients with panic-related anxiety presenting to the ED of Singapore's largest public healthcare group. Participants will be randomly assigned to either: 1) an enhanced care arm consisting of a stepped-care intervention for panic-related anxiety; or 2) a control arm consisting of screening for panic attacks and panic disorder. Screening will be followed by baseline assessments and blocked randomization in a 1:1 ratio. Masked follow-up assessments will be conducted at 1, 3, 6, and 12 months. Clinical outcomes will be panic symptom severity and rates of panic disorder. Patient-centred outcomes will be health-related quality of life, daily functioning, psychiatric comorbidity, and health services utilization. Economic effectiveness outcomes will be the incremental cost-effectiveness ratio of the stepped-care intervention relative to screening alone.

DISCUSSION

This trial will examine the impact of early intervention for patients with panic-related anxiety in the ED setting. The results will be used to propose a clinically-meaningful and cost-effective model of care for ED patients with panic-related anxiety.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03632356. Retrospectively registered 15 August 2018.

A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study

Risk scores are widely used for clinical decision making and commonly generated from logistic regression models. Machine-learning-based methods may work well for identifying important predictors to create parsimonious scores, but such 'black box' variable selection limits interpretability, and variable importance evaluated from a single model can be biased. We propose a robust and interpretable variable selection approach using the recently developed Shapley variable importance cloud (ShapleyVIC) that accounts for variability in variable importance across models. Our approach evaluates and visualizes overall variable contributions for in-depth inference and transparent variable selection, and filters out non-significant contributors to simplify model building steps. We derive an ensemble variable ranking from variable contributions across models, which is easily integrated with an automated and modularized risk score generator, AutoScore, for convenient implementation. In a study of early death or unplanned readmission after hospital discharge, ShapleyVIC selected 6 variables from 41 candidates to create a well-performing risk score, which had similar performance to a 16-variable model from machine-learning-based ranking. Our work contributes to the recent emphasis on interpretability of prediction models for high-stakes decision making, providing a disciplined solution to detailed assessment of variable importance and transparent development of parsimonious clinical risk scores.

Patient disposition using the Emergency Severity Index: a retrospective observational study at an interdisciplinary emergency department

OBJECTIVES

Early patient disposition is crucial to prevent crowding in emergency departments (EDs). Our study aimed to characterise the need of in-house resources for patients treated in the ED according to the Emergency Severity Index (ESI) and the presenting complaint at the timepoint of triage.

DESIGN

A retrospective single-centre study was conducted.

SETTING

Data of all patients who presented to the interdisciplinary ED of a tertiary care hospital in Munich, Germany, from 2014 to 2017 were analysed.

PARTICIPANTS

n=113 694 patients were included.

MEASURES

ESI Score, medical speciality according to the chief complaint, mode of arrival, admission rates and discharge destination from the ED were evaluated.

RESULTS

Patient disposition varied according to ESI scores in combination with the chief complaint. Patients with low ESI scores were more likely to be admitted after treatment in the ED than patients with high ESI scores. Highly prioritised patients (ESI 1) mainly required admission to an intensive care unit (ICU, 27%), intermediate care unit (IMC, 37%) or immediate intervention (11%). In this critical patient group, 30% of patients with neurological or medical symptoms required immediate intensive care, whereas only 17% of patients with surgical problems were admitted to an ICU. A significant number of patients (particularly with neurological or medical problems) required hospital (and in some cases even ICU or IMC) admission despite high ESI scores.

CONCLUSIONS

Overall, ESI seems to be a useful tool to anticipate the need for specialised in-hospital resources on arrival. Patients with symptoms pointing at neurological or medical problems need particular attention as ESI may fail to sufficiently predict the care facility level for this patient group.

Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation

Background

There is a growing demand globally for emergency department (ED) services. An increase in ED visits has resulted in overcrowding and longer waiting times. The triage process plays a crucial role in assessing and stratifying patients’ risks and ensuring that the critically ill promptly receive appropriate priority and emergency treatment. A substantial amount of research has been conducted on the use of machine learning tools to construct triage and risk prediction models; however, the black box nature of these models has limited their clinical application and interpretation.

Objective

In this study, we plan to develop an innovative, dynamic, and interpretable System for Emergency Risk Triage (SERT) for risk stratification in the ED by leveraging large-scale electronic health records (EHRs) and machine learning.

Methods

To achieve this objective, we will conduct a retrospective, single-center study based on a large, longitudinal data set obtained from the EHRs of the largest tertiary hospital in Singapore. Study outcomes include adverse events experienced by patients, such as the need for an intensive care unit and inpatient death. With preidentified candidate variables drawn from expert opinions and relevant literature, we will apply an interpretable machine learning–based AutoScore to develop 3 SERT scores. These 3 scores can be used at different times in the ED, that is, on arrival, during ED stay, and at admission. Furthermore, we will compare our novel SERT scores with established clinical scores and previously described black box machine learning models as baselines. Receiver operating characteristic analysis will be conducted on the testing cohorts for performance evaluation.

Results

The study is currently being conducted. The extracted data indicate approximately 1.8 million ED visits by over 810,000 unique patients. Modelling results are expected to be published in 2022.

Conclusions

The SERT scoring system proposed in this study will be unique and innovative because of its dynamic nature and modelling transparency. If successfully validated, our proposed solution will establish a standard for data processing and modelling by taking advantage of large-scale EHRs and interpretable machine learning tools.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34201

Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions

IMPORTANCE

Triage in the emergency department (ED) is a complex clinical judgment based on the tacit understanding of the patient's likelihood of survival, availability of medical resources, and local practices. Although a scoring tool could be valuable in risk stratification, currently available scores have demonstrated limitations.

OBJECTIVES

To develop an interpretable machine learning tool based on a parsimonious list of variables available at ED triage; provide a simple, early, and accurate estimate of patients' risk of death; and evaluate the tool's predictive accuracy compared with several established clinical scores.

DESIGN, SETTING, AND PARTICIPANTS

This single-site, retrospective cohort study assessed all ED patients between January 1, 2009, and December 31, 2016, who were subsequently admitted to a tertiary hospital in Singapore. The Score for Emergency Risk Prediction (SERP) tool was derived using a machine learning framework. To estimate mortality outcomes after emergency admissions, SERP was compared with several triage systems, including Patient Acuity Category Scale, Modified Early Warning Score, National Early Warning Score, Cardiac Arrest Risk Triage, Rapid Acute Physiology Score, and Rapid Emergency Medicine Score. The initial analyses were completed in October 2020, and additional analyses were conducted in May 2021.

MAIN OUTCOMES AND MEASURES

Three SERP scores, namely SERP-2d, SERP-7d, and SERP-30d, were developed using the primary outcomes of interest of 2-, 7-, and 30-day mortality, respectively. Secondary outcomes included 3-day mortality and inpatient mortality. The SERP's predictive power was measured using the area under the curve in the receiver operating characteristic analysis.

RESULTS

The study included 224 666 ED episodes in the model training cohort (mean [SD] patient age, 63.60 [16.90] years; 113 426 [50.5%] female), 56 167 episodes in the validation cohort (mean [SD] patient age, 63.58 [16.87] years; 28 427 [50.6%] female), and 42 676 episodes in the testing cohort (mean [SD] patient age, 64.85 [16.80] years; 21 556 [50.5%] female). The mortality rates in the training cohort were 0.8% at 2 days, 2.2% at 7 days, and 5.9% at 30 days. In the testing cohort, the areas under the curve of SERP-30d were 0.821 (95% CI, 0.796-0.847) for 2-day mortality, 0.826 (95% CI, 0.811-0.841) for 7-day mortality, and 0.823 (95% CI, 0.814-0.832) for 30-day mortality and outperformed several benchmark scores.

CONCLUSIONS AND RELEVANCE

In this retrospective cohort study, SERP had better prediction performance than existing triage scores while maintaining easy implementation and ease of ascertainment in the ED. It has the potential to be widely applied and validated in different circumstances and health care settings.

The new emergency department "Tuscan triage System". Validation study

INTRODUCTION

A new organizational framework was recently implemented in Tuscan Emergency Departments (EDs), including specific low-priority streaming. A new ED triage system, named "Tuscan Triage System" (TTS), was devised with the purpose of applying this reorganization.

METHODS

A validation study was designed with the primary aims of assessing the content, face, and criterion validities, and the inter-rater reliability of the TTS. The secondary aim was to estimate the differences in triage level assignation between the previous "Regional Triage System" (RTS) and the TTS. Twenty-four nurses trained for the TTS were enrolled to assign TTS priority levels to 100 triage clinical case vignettes drawn up by the developers of the TTS (reference standard).

RESULTS

The Content Validity Index - Scale/Average (S-CVI/Ave) of TTS was 0.98. Concerning to face validity, the S-CVI/Ave was 1. The highest adherence of triage level assignation to the reference standard was for levels 1 and 2. The Krippendorff α value was 0.808. Undertriage and overtriage were 10.45% and 14.29%, respectively. Overall, the comparation between RTS and TTS showed a marked shift of level assignation towards TTS low priority levels.

CONCLUSIONS

The TTS seems to be safe. These results should be confirmed through studies in the real clinical settings.

Important factors for planning nurse staffing in the emergency department: A consensus study

INTRODUCTION

Planning adequate nurse staffing in the emergency department (ED) is challenging. Although there are models to determine nurse staffing in EDs, these models do not consider all the factors. Inadequate nurse staffing causes overcrowding, poor quality of patient care, increased hospital costs, poor patient outcomes and high levels of burnout amongst nurses. In this paper, we report stakeholders' perceptions of important factors to be considered when planning ED nursing ratios.

METHODS

We applied a consensus research design. The data was generated from modified nominal group techniques followed by an e-Delphi with two rounds. The factors were generated during two nominal groups by 19 stakeholders which included management and healthcare professionals working in EDs. The generated factors were then put on a survey format for use in an e-Delphi. Using purposive and snowball sampling the survey was distributed to 74 national and international experts for consensus.

RESULTS

Ultimately, 43 experts agreed (a validity index of ≥ 80%) on four categories namely: hospital, staff, patient and additional categories which included 17 related factors.

CONCLUSION

Ideal nurse staffing ratios are influenced by the complexity of the environment and interactions between multiple factors. The categories and factors identified emphasised the need for extensive further research to ensure a financially viable model that will be accepted by both staff and patient, and thus promote optimal outcomes.

Prognostic performance of peripheral perfusion index and shock index combined with ESI to predict hospital outcome

INTRODUCTION

Peripheral perfusion index (PPI) and shock index (SI) are considered valuable predictors of hospital outcome and mortality in various operative and intensive care settings. In the present study, we evaluated the prognostic capabilities of these parameters for performing emergency department (ED) triage, as represented by the emergency severity index (ESI).

METHODS

This prospective cross-sectional study included 367 patients aged older than 18 years who visited the ED of a tertiary referral hospital. The ESI triage levels with PPI, SI, and other basic vital sign parameters were recorded for each patient. The hospital outcome of the patients at the end of the ED period, such as discharge, admission to the hospital and death were recorded.

RESULTS

A total of 367 patients (M/F: 178/189) admitted to the ED were categorized according to ESI and included in the study. A decrease in diastolic BP, SpO2 and PPI increased the likelihood of hospitalization and 30-day mortality. Based on univariate analysis, a significant improvement in performance was found by using age, diastolic BP, mean arterial pressure, SpO2, SI and PPI in terms of predicting high acuity level patients (ESI < 3). In the multivariable analysis only SpO2 and PPI were found to predict ESI < 3 patients.

CONCLUSION

Peripheral perfusion index and SI as novel triage instruments might provide useful information for predicting hospital admission and mortality in ED patients. The addition of these parameters to existing triage instruments such as ESI could enhance the triage specificity in unselected patients admitted to ED.

Severe sepsis and septic shock in patients transported by prehospital services versus walk in patients to the emergency department

BACKGROUND

Sepsis is a leading cause of death in the hospital for which aggressive treatment is recommended to improve patient outcomes. It is possible that sepsis patients brought in by emergency medical services (EMS) have a unique advantage in the emergency department (ED) which could improve sepsis bundle compliance.

OBJECTIVE

To evaluate patient care processes and outcome differences between severe sepsis and septic shock patients in the emergency department who were brought in by EMS compared to non-EMS patients.

METHODS

We performed a retrospective chart review of all severe sepsis and septic shock patients who declared in the ED during January 2012 thru December 2014. We compared differences in patient characteristics, patient care processes, sepsis bundle compliance metrics, and outcomes between both groups.

RESULTS

Of the 1066 patients included in the study, 387 (36.6%) were brought in by EMS and 679 (63.7%) patients arrived via non-EMS transport. In the multivariate regression model, time of triage to sepsis declaration (coeff = -0.406; 95% CI = -0.809, -0.003; p = 0.048) and time of triage to physician (coeff = -0.543; 95% CI = -0.864, -0.221; p = 0.001) was significantly shorter for EMS patients. We found no statistical difference in adjusted individual sepsis compliance metrics, overall bundle compliance, or mortality between both groups.

CONCLUSION

EMS transported patients have quicker sepsis declaration times and are seen sooner by ED providers. However, we found no statistical difference in bundle compliance or patient outcomes between walk in patients and EMS transported patients.

Management of oncology-related emergencies at the emergency department: A long-term undertaking

Background:

The emergency department at the Singapore General Hospital is an emergency department with an annual census of 140,000 and oncology-related attendances of about 4000 (2.8%). These patients are often admitted for further care. Palliative care in the emergency department for these patients is often minimal. The aim of this study was to determine the state of current management of oncology-related emergencies at the Singapore General Hospital’s emergency department, hence identifying specific areas for intervention.

Methods:

We carried out a retrospective data review of all Singapore General Hospital’s emergency department patients who had either cancer-related diagnoses or were admitted to the Medical Oncology Department in October 2018. Simple statistical analysis was then performed using IBM SPSS version 21.

Results:

Of 308 identified patients, there was approximately equal distribution by sex. The women were generally younger than the men (61.33 ± 13.63 years vs 67.36 ± 12.02 years, p = 0.063, confidence interval −8.94 to −3.13). Seventy-two (23.4%) of the patients arrived at emergency department by ambulance. The mean emergency department length of stay was 4.25 h. About half of the patients had either lung, colorectal, or breast as their primary site of cancer. There was no correlation between clinical severity according to the National Early Warning Scores and triage complaint-type or emergency clinical diagnosis. More than 90% were admitted, with about 32.6% dying during their inpatient stay. High National Early Warning Scores were significantly associated with mortality.

Conclusion:

There is large potential for interventions to improve patient well-being in the pre-hospital setting and emergency department. Given the sizable number of patients with poor outcomes, palliative care is also of paramount importance.

Modifications of emergency dental clinic protocols to combat COVID-19 transmission

During the COVID-19 pandemic, incidence rates for dental diseases will continue unabated. However, the intent to prevent the spread of this lethal respiratory disease will likely lead to reduced treatment access due to restrictions on population movements. These changes have the potential to increase dental-related emergency department visits and subsequently contribute to greater viral transmission. Moreover, dentists experience unique challenges with preventing transmission due to frequent aerosol-producing procedures. This paper presents reviews and protocols implemented by directors and residents at the Dental College of Georgia to manage a dental emergency clinic during the COVID-19 pandemic. The methods presented include committee-based prioritization of dental patients, a multilayered screening process, team rotations with social and temporal spacing, and modified treatment room protocols. These efforts aid in the reduction of viral transmission, conservation of personal protective equipment, and expand provider availability. These protocols transcend a university and hospital-based models and are applicable to private and corporate models.