A simple tool to predict admission at the time of triage

Strategies to identify medical patients suitable for management through Same Day Emergency Care Services: A Systematic Review

Same Day Emergency Care (SDEC) in unplanned and emergency care is an NHSE priority. Optimal use of these services requires rapid identification of suitable patients. NHSE suggests the use of one tool for this purpose. This systematic review compares studies which evaluate the performance of selection tools for SDEC pathways. Nine studies met the inclusion criteria. Three scores were evaluated: the Amb score (7 studies), Glasgow Admission Prediction Score (GAPS)(6 studies) and Sydney Triage to Admission Risk Tool (START)(2 studies). There was heterogeneity in the populations assessed, exclusion criteria used, and definitions used for SDEC suitability, with proportions of patients deemed 'suitable' for SDEC ranging from 20-80%. Reported score sensitivity and specificity ranged between 18-99% and 10-89%. Score performance could not be compared due to heterogeneity between studies. No studies assessed clinical implementation. The current evidence to support the use of a specific tool for SDEC is limited and requires further evaluation.

Drinking from the Holy Grail-Does a Perfect Triage System Exist? And Where to Look for It?

The Emergency Department (ED) is a facility meant to treat patients in need of medical assistance. The choice of triage system hugely impactsed the organization of any given ED and it is important to analyze them for their effectiveness. The goal of this review is to briefly describe selected triage systems in an attempt to find the perfect one. Papers published in PubMed from 1990 to 2022 were reviewed. The following terms were used for comparison: "ED" and "triage system". The papers contained data on the design and function of the triage system, its validation, and its performance. After studies comparing the distinct means of patient selection were reviewed, they were meant to be classified as either flawed or non-ideal. The validity of all the comparable segregation systems was similar. A possible solution would be to search for a new, measurable parameter for a more accurate risk estimation, which could be a game changer in terms of triage assessment. The dynamic development of artificial intelligence (AI) technologies has recently been observed. The authors of this study believe that the future segregation system should be a combination of the experience and intuition of trained healthcare professionals and modern technology (artificial intelligence).

Applying Sydney Triage to Admission Risk Tool (START) to improve patient flow in emergency departments: a multicentre randomised, implementation study

BACKGROUND

To determine the effectiveness of applying the Sydney Triage to Admission Risk Tool (START) in conjunction with senior early assessment in different Emergency Departments (EDs).

METHODS

This multicentre implementation study, conducted in two metropolitan EDs, used a convenience sample of ED patients. Patients who were admitted, after presenting to both EDs, and were assessed using the existing senior ED clinician assessment, were included in the study. Patients in the intervention group were assessed with the assistance of START, while patients in the control group were assessed without the assistance of START. Outcomes measured were ED length of stay and proportion of patients correctly identified as an in-patient admission by START.

RESULTS

A total of 773 patients were evaluated using the START tool at triage across both sites (Intervention group n = 355 and control group n = 418 patients). The proportion of patients meeting the 4-hour length of stay thresholds was similar between the intervention and control groups (30.1% vs. 28.2%; p = 0.62). The intervention group was associated with a reduced ED length of stay when compared to the control group (351 min, interquartile range (IQR) 221.0-565.0 min versus 383 min, IQR 229.25-580.0 min; p = 0.85). When stratified into admitted and discharged patients, similar results were seen.

CONCLUSION

In this extension of the START model of care implementation study in two metropolitan EDs, START, when used in conjunction with senior early assessment was associated with some reduced ED length of stay.

Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation

OBJECTIVE

Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementation. This is the first of such studies to investigate real-time AI performance in the emergency setting.

METHODS

The novel AI algorithm that predicts admission using a triage note was translated into clinical practice and integrated within St Vincent's Hospital Melbourne's electronic emergency patient management system. The data were collected from 1 January 2021 to 17 August 2022 to evaluate the diagnostic accuracy of the AI system after implementation.

RESULTS

A total of 77 125 ED presentations were included. The live AI algorithm has a sensitivity of 73.1% (95% confidence interval 72.5-73.8), specificity of 74.3% (73.9-74.7), positive predictive value of 50% (49.6-50.4) and negative predictive value of 88.7% (88.5-89) with a total accuracy of 74% (73.7-74.3). The accuracy of the system was at the lowest for admission to psychiatric units (34%) and at the highest for gastroenterology and medical admission (84% and 80%, respectively).

CONCLUSION

Our study showed the diagnostic evaluation of a real-time AI clinical decision-support tool became less accurate than the original. Although real-time sensitivity and specificity of the AI tool was still acceptable as a decision-support tool in the ED, we propose that continuous training and evaluation of AI-enabled clinical support tools in healthcare are conducted to ensure consistent accuracy and performance to prevent inadvertent consequences.

Medical Admission Prediction Score (MAPS); a simple tool to predict medical admissions in the emergency department

INTRODUCTION

Overcrowding in the emergency departments (ED) is linked to adverse clinical outcomes, a negative impact on patient safety, patient satisfaction, and physician efficiency. We aimed to design a medical admission prediction scoring system based on readily available clinical data during ED presentation.

METHODS

In this retrospective cross-sectional study, data on ED presentations and medical admissions were extracted from the Emergency and Internal Medicine departments of a tertiary care facility in Qatar. Primary outcome was medical admission.

RESULTS

Of 320299 ED presentations, 218772 were males (68.3%). A total of 11847 (3.7%) medical admissions occurred. Most patients were Asians (53.7%), followed by Arabs (38.7%). Patients who got admitted were older than those who did not (p <0.001). Admitted patients were predominantly males (56.8%), had a higher number of comorbid conditions and a higher frequency of recent discharge (within the last 30 days) (p <0.001). Age > 60 years, female gender, discharge within the last 30 days, and worse vital signs at presentations were independently associated with higher odds of admission (p<0.001). These factors generated the scoring system with a cut-off of >17, area under the curve (AUC) 0.831 (95% CI 0.827-0.836), and a predictive accuracy of 83.3% (95% CI 83.2-83.4). The model had a sensitivity of 69.1% (95% CI 68.2-69.9), specificity was 83.9% (95% CI 83.7-84.0), positive predictive value (PPV) 14.2% (95% CI 13.8-14.4), negative predictive value (NPV) 98.6% (95% CI 98.5-98.7) and positive likelihood ratio (LR+) 4.28% (95% CI 4.27-4.28).

CONCLUSION

Medical admission prediction scoring system can be reliably applied to the regional population to predict medical admissions and may have better generalizability to other parts of the world owing to the diverse patient population in Qatar.

The association of demographics, frailty and multiple health conditions with outcomes from acute medical admissions to hospitals in England: exploratory analysis of an administrative dataset

Emergency and acute hospital services in England are under increasing pressure. The aim of this study was to investigate the association between key case-mix indicators and outcomes for adults admitted to hospital with an acute medical condition in England. All patients aged ≥16 years admitted to hospital in England as an acute unselected medical admission and who survived to discharge during the financial year 2021-2022 were included. Length of hospital stay was the primary outcome of interest. Data were available for 1,586,168 unique patients. A case-mix index was developed with a score that ranged from 0 to 12. Frailty was the most important variable in the index, followed by multiple health conditions and patient age. The mean case-mix score across hospital trusts in England ranged from 5.3 to 7.8. The case-mix index will support initiatives to better understand factors contributing to outcomes from acute medical admissions to hospital.

Using the Field Artificial Intelligence Triage (FAIT) tool to predict hospital critical care resource utilization in patients with truncal gunshot wounds

BACKGROUND

Tiered trauma triage systems have resulted in a significant mortality reduction, but models have remained unchanged. The aim of this study was to develop and test an artificial intelligence algorithm to predict critical care resource utilization.

METHODS

We queried the ACS-TQIP 2017-18 database for truncal gunshot wounds(GSW). An information-aware deep neural network (DNN-IAD) model was trained to predict ICU admission and need for mechanical ventilation (MV). Input variables included demographics, comorbidities, vital signs, and external injuries. The model's performance was assessed using the area under receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC).

RESULTS

For the ICU admission analysis, we included 39,916 patients. For the MV need analysis, 39,591 patients were included. Median (IQR) age was 27 (22,36). AUROC and AUPRC for predicting ICU need were 84.8 ± 0.5 and 75.4 ± 0.5, and the AUROC and AUPRC for MV need were 86.8 ± 0.5 and 72.5 ± 0.6.

CONCLUSIONS

Our model predicts hospital utilization outcomes in patients with truncal GSW with high accuracy, allowing early resource mobilization and rapid triage decisions in hospitals with capacity issues and austere environments.

Predicting Hospital Ward Admission from the Emergency Department: A Systematic Review

BACKGROUND

The emergency department (ED) is often overburdened, due to the high influx of patients and limited availability of attending physicians. This situation highlights the need for improvement in the management of, and assistance provided in the ED. A key point for this purpose is the identification of patients with the highest risk, which can be achieved using machine learning predictive models. The objective of this study is to conduct a systematic review of predictive models used to detect ward admissions from the ED. The main targets of this review are the best predictive algorithms, their predictive capacity, the studies' quality, and the predictor variables.

METHODS

This review is based on PRISMA methodology. The information has been searched in PubMed, Scopus and Google Scholar databases. Quality assessment has been performed using the QUIPS tool.

RESULTS

Through the advanced search, a total of 367 articles were found, of which 14 were of interest that met the inclusion criteria. Logistic regression is the most used predictive model, achieving AUC values between 0.75-0.92. The two most used variables are the age and ED triage category.

CONCLUSIONS

artificial intelligence models can contribute to improving the quality of care in the ED and reducing the burden on healthcare systems.

Prospective External Validation of a Commercial Model Predicting the Likelihood of Inpatient Admission From the Emergency Department

STUDY OBJECTIVE

Early notification of admissions from the emergency department (ED) may allow hospitals to plan for inpatient bed demand. This study aimed to assess Epic's ED Likelihood to Occupy an Inpatient Bed predictive model and its application in improving hospital bed planning workflows.

METHODS

All ED adult (18 years and older) visits from September 2021 to August 2022 at a large regional health care system were included. The primary outcome was inpatient admission. The predictive model is a random forest algorithm that uses demographic and clinical features. The model was implemented prospectively, with scores generated every 15 minutes. The area under the receiver operator curves (AUROC) and precision-recall curves (AUPRC) were calculated using the maximum score prior to the outcome and for each prediction independently. Test characteristics and lead time were calculated over a range of model score thresholds.

RESULTS

Over 11 months, 329,194 encounters were evaluated, with an incidence of inpatient admission of 25.4%. The encounter-level AUROC was 0.849 (95% confidence interval [CI], 0.848 to 0.851), and the AUPRC was 0.643 (95% CI, 0.640 to 0.647). With a prediction horizon of 6 hours, the AUROC was 0.758 (95% CI, 0.758 to 0.759,) and the AUPRC was 0.470 (95% CI, 0.469 to 0.471). At a predictive model threshold of 40, the sensitivity was 0.49, the positive predictive value was 0.65, and the median lead-time warning was 127 minutes before the inpatient bed request.

CONCLUSION

The Epic ED Likelihood to Occupy an Inpatient Bed model may improve hospital bed planning workflows. Further study is needed to determine its operational effect.

Predicting hospital admission from emergency department triage data for patients presenting with fall-related fractures

PURPOSE

Predict in advance the need for hospitalization of adult patients for fall-related fractures based on information available at the time of triage to help decision-making at the emergency department (ED).

METHODS

We developed machine learning models using routinely collected triage data at a regional hospital chain in Pennsylvania to predict admission to an inpatient unit. We considered all patients presenting to the ED for fall-related fractures. Patients who were 18 years or younger, who left the ED against medical advice, left the ED waiting room without being seen by a provider, and left the ED after initial diagnostics were excluded from the analysis. We compared models obtained using triage data (pre-model) with models developed using additional data obtained after physicians' diagnoses (post-model).

RESULTS

Our results show good discriminatory power on predicting hospital admissions. Neural network models performed the best (AUC: pre-model = 0.938 [CI 0.920-0.956], post-model = 0.983 [0.974-0.992]). The logistic regression analysis provides additional insights into the data and the relationships between the variables.

CONCLUSIONS

Using limited data available at the time of triage, we developed four machine learning models aimed at predicting hospitalization for patients presenting to the ED for fall-related fractures. All the four models were robust and performed well. Neural network method, however, performed the best for both pre- and post-models. Simple, parsimonious machine learning models can provide high accuracy for predicting hospital admission.

Performance of scoring systems in selecting short stay medical admissions suitable for assessment in same day emergency care: an analysis of diagnostic accuracy in a UK hospital setting

OBJECTIVES

To assess the performance of the Amb score and Glasgow Admission Prediction Score (GAPS) in identifying acute medical admissions suitable for same day emergency care (SDEC) in a large urban secondary centre.

DESIGN

Retrospective assessment of routinely collected data from electronic healthcare records.

SETTING

Single large urban tertiary care centre.

PARTICIPANTS

All unplanned admissions to general medicine on Monday-Friday, episodes starting 08:00-16:59 hours and lasting up to 48 hours, between 1 April 2019 and 9 March 2020.

MAIN OUTCOME MEASURES

Sensitivity, specificity, positive and negative predictive value of the Amb score and GAPS in identifying patients discharged within 12 hours of arrival.

RESULTS

7365 episodes were assessed. 94.6% of episodes had an Amb score suggesting suitability for SDEC. The positive predictive value of the Amb score in identifying those discharged within 12 hours was 54.5% (95% CI 53.3% to 55.8%). The area under the receiver operating characteristic curve (AUROC) for the Amb score was 0.612 (95% CI 0.599 to 0.625).42.4% of episodes had a GAPS suggesting suitability for SDEC. The positive predictive value of the GAPS in identifying those discharged within 12 hours was 50.5% (95% CI 48.4% to 52.7%). The AUROC for the GAPS was 0.606 (95% CI 0.590 to 0.622).41.4% of the population had both an Amb and GAPS score suggestive of suitability for SDEC and 5.7% of the population had both and Amb and GAPS score suggestive of a lack of suitability for SDEC.

CONCLUSIONS

The Amb score and GAPS had poor discriminatory ability to identify acute medical admissions suitable for discharge within 12 hours, limiting their utility in selecting patients for assessment within SDEC services within this diverse patient population.

Benchmarking emergency department prediction models with machine learning and public electronic health records

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop prediction models and decision support systems to address these challenges. To date, there is no widely accepted clinical prediction benchmark related to the ED based on large-scale public EHRs. An open-source benchmark data platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. Based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we created a benchmark dataset and proposed three clinical prediction benchmarks. This study provides future researchers with insights, suggestions, and protocols for managing data and developing predictive tools for emergency care.

Reducing Crowding in Emergency Departments With Early Prediction of Hospital Admission of Adult Patients Using Biomarkers Collected at Triage: Retrospective Cohort Study

BACKGROUND

Emergency department crowding continues to threaten patient safety and cause poor patient outcomes. Prior models designed to predict hospital admission have had biases. Predictive models that successfully estimate the probability of patient hospital admission would be useful in reducing or preventing emergency department "boarding" and hospital "exit block" and would reduce emergency department crowding by initiating earlier hospital admission and avoiding protracted bed procurement processes.

OBJECTIVE

To develop a model to predict imminent adult patient hospital admission from the emergency department early in the patient visit by utilizing existing clinical descriptors (ie, patient biomarkers) that are routinely collected at triage and captured in the hospital's electronic medical records. Biomarkers are advantageous for modeling due to their early and routine collection at triage; instantaneous availability; standardized definition, measurement, and interpretation; and their freedom from the confines of patient histories (ie, they are not affected by inaccurate patient reports on medical history, unavailable reports, or delayed report retrieval).

METHODS

This retrospective cohort study evaluated 1 year of consecutive data events among adult patients admitted to the emergency department and developed an algorithm that predicted which patients would require imminent hospital admission. Eight predictor variables were evaluated for their roles in the outcome of the patient emergency department visit. Logistic regression was used to model the study data.

RESULTS

The 8-predictor model included the following biomarkers: age, systolic blood pressure, diastolic blood pressure, heart rate, respiration rate, temperature, gender, and acuity level. The model used these biomarkers to identify emergency department patients who required hospital admission. Our model performed well, with good agreement between observed and predicted admissions, indicating a well-fitting and well-calibrated model that showed good ability to discriminate between patients who would and would not be admitted.

CONCLUSIONS

This prediction model based on primary data identified emergency department patients with an increased risk of hospital admission. This actionable information can be used to improve patient care and hospital operations, especially by reducing emergency department crowding by looking ahead to predict which patients are likely to be admitted following triage, thereby providing needed information to initiate the complex admission and bed assignment processes much earlier in the care continuum.

Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation

Background

There is a growing demand globally for emergency department (ED) services. An increase in ED visits has resulted in overcrowding and longer waiting times. The triage process plays a crucial role in assessing and stratifying patients’ risks and ensuring that the critically ill promptly receive appropriate priority and emergency treatment. A substantial amount of research has been conducted on the use of machine learning tools to construct triage and risk prediction models; however, the black box nature of these models has limited their clinical application and interpretation.

Objective

In this study, we plan to develop an innovative, dynamic, and interpretable System for Emergency Risk Triage (SERT) for risk stratification in the ED by leveraging large-scale electronic health records (EHRs) and machine learning.

Methods

To achieve this objective, we will conduct a retrospective, single-center study based on a large, longitudinal data set obtained from the EHRs of the largest tertiary hospital in Singapore. Study outcomes include adverse events experienced by patients, such as the need for an intensive care unit and inpatient death. With preidentified candidate variables drawn from expert opinions and relevant literature, we will apply an interpretable machine learning–based AutoScore to develop 3 SERT scores. These 3 scores can be used at different times in the ED, that is, on arrival, during ED stay, and at admission. Furthermore, we will compare our novel SERT scores with established clinical scores and previously described black box machine learning models as baselines. Receiver operating characteristic analysis will be conducted on the testing cohorts for performance evaluation.

Results

The study is currently being conducted. The extracted data indicate approximately 1.8 million ED visits by over 810,000 unique patients. Modelling results are expected to be published in 2022.

Conclusions

The SERT scoring system proposed in this study will be unique and innovative because of its dynamic nature and modelling transparency. If successfully validated, our proposed solution will establish a standard for data processing and modelling by taking advantage of large-scale EHRs and interpretable machine learning tools.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34201

How do we identify acute medical admissions that are suitable for same day emergency care?

Medical emergencies causing unplanned hospital admission place considerable demands on acute healthcare services. Some patients can be assessed and treated through ambulatory pathways without inpatient admission, via same day emergency care (SDEC), potentially benefiting patients and reducing demands on inpatient services. There is currently considerable variation within acute medicine in aspects of SDEC delivery ranging from overall service design to patient selection methods. Scoring systems identifying patients likely to be successfully managed through SDEC services have been suggested, but evidence of utility in diverse populations is lacking. Specific scoring systems exist for some common medical problems, including cardiac chest pain and pulmonary embolism, but further research is needed to demonstrate how these are most effectively incorporated into SDEC services. This review defines SDEC and describes the variation in services nationally. It reviews the evidence for their clinical impact, tools to screen patients for SDEC and current gaps in our knowledge regarding service deployment.

Prospective comparison of AMB, GAP AND START scores and triage nurse clinical judgement for predicting admission from an ED: a single-centre prospective study

BACKGROUND

It is postulated that early determination of the need for admission can improve flow through EDs. There are several scoring systems which have been developed for predicting patient admission at triage, although they have not been directly compared. In addition, it is not known if these scoring systems perform better than clinical judgement. Therefore, the aim of this study was to validate existing tools in predicting hospital admission during triage and then compare them with the clinical judgement of triage nurses.

METHODS

To conduct this prospective, single-centre observational study, we enrolled consecutive adult patients who presented between 30 September 2019 and 25 October 2019 at the ED of a large teaching hospital in Milan, Italy. For each patient, triage nurses recorded all of the variables needed to perform Ambulatory (AMB), Glasgow Admission Prediction (GAP) and Sydney Triage to Admission Risk Tool (START) scoring. The probability of admission was estimated by the triage nurses using clinical judgement and expressed as a percentage from 0 to 100 with intervals of 5. Nurse estimates were dichotomised for analysis, with ≥50% likelihood being a prediction of admission. Receiver operating characteristic curves were generated for accuracy of the predictions. Area under the curve (AUC) with 95% CI for each of the scores and for the nursing judgements was also calculated.

RESULTS

A total of 1710 patients (844 men; median age, 54 years (IQR: 34-75)) and 35 nurses (15 men; median age, 37 years (IQR: 33-48)) were included in this study. Among these patients, 310 (18%) were admitted to hospital from the ED. AUC values for AMB, GAP and START scores were 0.77 (95% CI: 0.74 to 0.79), 0.72 (95% CI: 0.69 to 0.75) and 0.61 (95% CI: 0.58 to 0.64), respectively. The AUC for nurse clinical judgement was 0.86 (95% CI: 0.84 to 0.89).

CONCLUSION

AMB, GAP and START scores provided moderate accuracy in predicting patient admission. However, all of the scores were significantly worse than the clinical judgement of the triage nurses.

Prediction of admission to a low-resource sub-Saharan hospital by mental status, mobility and oxygen saturation recorded on arrival: a prospective observational study

BACKGROUND

The decision to admit patients to hospital in low-resource settings have been poorly investigated.

AIM

We aimed to determine the association of a disposition score determined on arrival with the decision subsequently made to admit or discharge the patient. The score awarded one point for altered mental status, one point for impaired mobility and one point for low oxygen saturation.

METHODS

The mental status, mobility and oxygen saturation on arrival of 5,334 consecutive patients attending a combined emergency and outpatient department in a low-resource Ugandan hospital were recorded. Admission decisions were subsequently made independently by clinicians unaware to the score.

RESULTS

Most patients (n=3,876; 73%) had a disposition score of zero and only 25 of these patients (0.6%) were subsequently admitted. A total of 646 (12.1%) patients were admitted. Only 301 (5.6%) patients had a score of 3 points and 263 (87.4%) of these were admitted. The C statistic for the discrimination of the score for admission was 0.953 (95% confidence interval 0.941-0.964).

CONCLUSION

In a low-resource setting, a simple score based on mental status, mobility and oxygen saturation identified outpatient and emergency department patients most and least likely to be subsequently admitted to hospital with a high degree of discrimination.

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19

A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model's performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature's SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.

Predicting inhospital admission at the emergency department: a systematic review

Background

ED crowding has potential detrimental consequences for both patient care and staff. Advancing disposition can reduce crowding. This may be achieved by using prediction models for admission. This systematic review aims to present an overview of prediction models for admission at the ED. Furthermore, we aimed to identify the best prediction tool based on its performance, validation, calibration and clinical usability.

Methods

We included observational studies published in Embase.com, Medline Ovid, Cochrane CENTRAL, Web of Science Core Collection or Google scholar, in which admission models were developed or validated in a general medical population in European EDs including the UK. We used the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist to assess quality of model development. Model performance was presented as discrimination and calibration. The search was performed on 11 October 2020.

Results

In total, 18 539 articles were identified. We included 11 studies, describing 16 different models, comprising the development of 9 models and 12 external validations of 11 models. The risk of bias of the development studies was considered low to medium. Discrimination, as represented by the area under the curve ranged from 0.630 to 0.878. Calibration was assessed in seven models and was strong. The best performing models are the models of Lucke et al and Cameron et al. These models combine clinical applicability, by inclusion of readily available parameters, and appropriate discrimination, calibration and validation.

Conclusion

None of the models are yet implemented in EDs. Further research is needed to assess the applicability and implementation of the best performing models in the ED.

Systematic review registration number

PROSPERO CRD42017057975.

Developing machine learning models to personalize care levels among emergency room patients for hospital admission

Objective

To develop prediction models for intensive care unit (ICU) vs non-ICU level-of-care need within 24 hours of inpatient admission for emergency department (ED) patients using electronic health record data.

Materials and Methods

Using records of 41 654 ED visits to a tertiary academic center from 2015 to 2019, we tested 4 algorithms—feed-forward neural networks, regularized regression, random forests, and gradient-boosted trees—to predict ICU vs non-ICU level-of-care within 24 hours and at the 24th hour following admission. Simple-feature models included patient demographics, Emergency Severity Index (ESI), and vital sign summary. Complex-feature models added all vital signs, lab results, and counts of diagnosis, imaging, procedures, medications, and lab orders.

Results

The best-performing model, a gradient-boosted tree using a full feature set, achieved an AUROC of 0.88 (95%CI: 0.87–0.89) and AUPRC of 0.65 (95%CI: 0.63–0.68) for predicting ICU care need within 24 hours of admission. The logistic regression model using ESI achieved an AUROC of 0.67 (95%CI: 0.65–0.70) and AUPRC of 0.37 (95%CI: 0.35–0.40). Using a discrimination threshold, such as 0.6, the positive predictive value, negative predictive value, sensitivity, and specificity were 85%, 89%, 30%, and 99%, respectively. Vital signs were the most important predictors.

Discussion and Conclusions

Undertriaging admitted ED patients who subsequently require ICU care is common and associated with poorer outcomes. Machine learning models using readily available electronic health record data predict subsequent need for ICU admission with good discrimination, substantially better than the benchmarking ESI system. The results could be used in a multitiered clinical decision-support system to improve ED triage.

Models Predicting Hospital Admission of Adult Patients Utilizing Prehospital Data: Systematic Review Using PROBAST and CHARMS

Background

Emergency department boarding and hospital exit block are primary causes of emergency department crowding and have been conclusively associated with poor patient outcomes and major threats to patient safety. Boarding occurs when a patient is delayed or blocked from transitioning out of the emergency department because of dysfunctional transition or bed assignment processes. Predictive models for estimating the probability of an occurrence of this type could be useful in reducing or preventing emergency department boarding and hospital exit block, to reduce emergency department crowding.

Objective

The aim of this study was to identify and appraise the predictive performance, predictor utility, model application, and model utility of hospital admission prediction models that utilized prehospital, adult patient data and aimed to address emergency department crowding.

Methods

We searched multiple databases for studies, from inception to September 30, 2019, that evaluated models predicting adult patients’ imminent hospital admission, with prehospital patient data and regression analysis. We used PROBAST (Prediction Model Risk of Bias Assessment Tool) and CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) to critically assess studies.

Results

Potential biases were found in most studies, which suggested that each model’s predictive performance required further investigation. We found that select prehospital patient data contribute to the identification of patients requiring hospital admission. Biomarker predictors may add superior value and advantages to models. It is, however, important to note that no models had been integrated with an information system or workflow, operated independently as electronic devices, or operated in real time within the care environment. Several models could be used at the site-of-care in real time without digital devices, which would make them suitable for low-technology or no-electricity environments.

Conclusions

There is incredible potential for prehospital admission prediction models to improve patient care and hospital operations. Patient data can be utilized to act as predictors and as data-driven, actionable tools to identify patients likely to require imminent hospital admission and reduce patient boarding and crowding in emergency departments. Prediction models can be used to justify earlier patient admission and care, to lower morbidity and mortality, and models that utilize biomarker predictors offer additional advantages.

Impact of medication characteristics and adverse drug events on hospital admission after an emergency department visit: Prospective cohort study

OBJECTIVES

Emergency department (ED) overcrowding is a problem for the delivery of adequate and timely emergency care. To improve patient flow and the admission process, the quick prediction of a patient's need for admission is crucial. We aimed to investigate the variables associated with hospitalisation after an ED visit, with a particular focus on the variables related to medication.

METHODS

This prospective study was conducted from 2011 to 2018 in subacute medical ED of a French University Hospital. Specialised EDs (paediatric, gynaecologic, head and neck and psychiatric) and the outpatient unit of the ED were not included. Participation in this study was proposed to all adult patients who underwent a medication history interview with a pharmacist. Pharmacists conducted structured interviews for the completion of the medication history and the detection of adverse drug events (ADE). Relations between patient characteristics and hospitalisation were analysed using logistic regression.

RESULTS

Among the 14 511 included patients, 5972 (41.2%) were hospitalised including 69 deaths. In total, 7458 patients (51.4%) took more than 5 medications and 2846 patients (19.6%) had an ADE detected during the ED visit. In hospitalised patients, bleeding (32.2%) and metabolic disorders (16.8%) were the most observed ADE symptoms. Variables associated with increased hospital admission included 2 demographic variables (age, male gender), 4 clinical variables (renal and hepatic failures, alcohol addiction, ED visit for respiratory reason) and 6 medication-related variables (medications >5, use of blood, systemic anti-infective, metabolism and antineoplastic/immunomodulating medications and ADE).

CONCLUSION

We identified variables associated with hospitalisation including drug-related variables. These results point out the importance and the relevance of collecting medication data in a subacute medical ED (study registered on ClinicalTrials.gov, NCT03442010).

Prediction of patient disposition: comparison of computer and human approaches and a proposed synthesis

OBJECTIVE

To compare the accuracy of computer versus physician predictions of hospitalization and to explore the potential synergies of hybrid physician-computer models.

MATERIALS AND METHODS

A single-center prospective observational study in a tertiary pediatric hospital in Boston, Massachusetts, United States. Nine emergency department (ED) attending physicians participated in the study. Physicians predicted the likelihood of admission for patients in the ED whose hospitalization disposition had not yet been decided. In parallel, a random-forest computer model was developed to predict hospitalizations from the ED, based on data available within the first hour of the ED encounter. The model was tested on the same cohort of patients evaluated by the participating physicians.

RESULTS

198 pediatric patients were considered for inclusion. Six patients were excluded due to incomplete or erroneous physician forms. Of the 192 included patients, 54 (28%) were admitted and 138 (72%) were discharged. The positive predictive value for the prediction of admission was 66% for the clinicians, 73% for the computer model, and 86% for a hybrid model combining the two. To predict admission, physicians relied more heavily on the clinical appearance of the patient, while the computer model relied more heavily on technical data-driven features, such as the rate of prior admissions or distance traveled to hospital.

DISCUSSION

Computer-generated predictions of patient disposition were more accurate than clinician-generated predictions. A hybrid prediction model improved accuracy over both individual predictions, highlighting the complementary and synergistic effects of both approaches.

CONCLUSION

The integration of computer and clinician predictions can yield improved predictive performance.

Machine learning for developing a prediction model of hospital admission of emergency department patients: Hype or hope?

OBJECTIVE

Early identification of emergency department (ED) patients who need hospitalization is essential for quality of care and patient safety. We aimed to compare machine learning (ML) models predicting the hospitalization of ED patients and conventional regression techniques at three points in time after ED registration.

METHODS

We analyzed consecutive ED patients of three hospitals using the Netherlands Emergency Department Evaluation Database (NEED). We developed prediction models for hospitalization using an increasing number of data available at triage, ∼30 min (including vital signs) and ∼2 h (including laboratory tests) after ED registration, using ML (random forest, gradient boosted decision trees, deep neural networks) and multivariable logistic regression analysis (including spline transformations for continuous predictors). Demographics, urgency, presenting complaints, disease severity and proxies for comorbidity, and complexity were used as covariates. We compared the performance using the area under the ROC curve in independent validation sets from each hospital.

RESULTS

We included 172,104 ED patients of whom 66,782 (39 %) were hospitalized. The AUC of the multivariable logistic regression model was 0.82 (0.78-0.86) at triage, 0.84 (0.81-0.86) at ∼30 min and 0.83 (0.75-0.92) after ∼2 h. The best performing ML model over time was the gradient boosted decision trees model with an AUC of 0.84 (0.77-0.88) at triage, 0.86 (0.82-0.89) at ∼30 min and 0.86 (0.74-0.93) after ∼2 h.

CONCLUSIONS

Our study showed that machine learning models had an excellent but similar predictive performance as the logistic regression model for predicting hospital admission. In comparison to the 30-min model, the 2-h model did not show a performance improvement. After further validation, these prediction models could support management decisions by real-time feedback to medical personal.

Development and Validation of Machine Learning Models to Predict Admission From Emergency Department to Inpatient and Intensive Care Units

STUDY OBJECTIVE

This study aimed to develop and validate 2 machine learning models that use historical and current-visit patient data from electronic health records to predict the probability of patient admission to either an inpatient unit or ICU at each hour (up to 24 hours) of an emergency department (ED) encounter. The secondary goal was to provide a framework for the operational implementation of these machine learning models.

METHODS

Data were curated from 468,167 adult patient encounters in 3 EDs (1 academic and 2 community-based EDs) of a large academic health system from August 1, 2015, to October 31, 2018. The models were validated using encounter data from January 1, 2019, to December 31, 2019. An operational user dashboard was developed, and the models were run on real-time encounter data.

RESULTS

For the intermediate admission model, the area under the receiver operating characteristic curve was 0.873 and the area under the precision-recall curve was 0.636. For the ICU admission model, the area under the receiver operating characteristic curve was 0.951 and the area under the precision-recall curve was 0.461. The models had similar performance in both the academic- and community-based settings as well as across the 2019 and real-time encounter data.

CONCLUSION

Machine learning models were developed to accurately make predictions regarding the probability of inpatient or ICU admission throughout the entire duration of a patient's encounter in ED and not just at the time of triage. These models remained accurate for a patient cohort beyond the time period of the initial training data and were integrated to run on live electronic health record data, with similar performance.

Predicting Admissions From a Paediatric Emergency Department - Protocol for Developing and Validating a Low-Dimensional Machine Learning Prediction Model

Introduction: Patients boarding in the Emergency Department can contribute to overcrowding, leading to longer waiting times and patients leaving without being seen or completing their treatment. The early identification of potential admissions could act as an additional decision support tool to alert clinicians that a patient needs to be reviewed for admission and would also be of benefit to bed managers in advance bed planning for the patient. We aim to create a low-dimensional model predicting admissions early from the paediatric Emergency Department.

Methods and Analysis: The methodology Cross Industry Standard Process for Data Mining (CRISP-DM) will be followed. The dataset will comprise of 2 years of data, ~76,000 records. Potential predictors were identified from previous research, comprising of demographics, registration details, triage assessment, hospital usage and past medical history. Fifteen models will be developed comprised of 3 machine learning algorithms (Logistic regression, naïve Bayes and gradient boosting machine) and 5 sampling methods, 4 of which are aimed at addressing class imbalance (undersampling, oversampling, and synthetic oversampling techniques). The variables of importance will then be identified from the optimal model (selected based on the highest Area under the curve) and used to develop an additional low-dimensional model for deployment.

Discussion: A low-dimensional model comprised of routinely collected data, captured up to post triage assessment would benefit many hospitals without data rich platforms for the development of models with a high number of predictors. Novel to the planned study is the use of data from the Republic of Ireland and the application of sampling techniques aimed at improving model performance impacted by an imbalance between admissions and discharges in the outcome variable.

Social Determinant of Health Documentation Trends and Their Association with Emergency Department Admissions

Research has shown that health outcomes are significantly driven by patient's social and economic needs and environment, commonly referred to as the social determinants of health (SDoH). Standardized documentation of social and economic needs in healthcare are underutilized. This study examines the prevalence of documented social and economic needs (Z-codes) in a nationwide inpatient database and the association with emergency department (ED) admissions. Multivariate logistic regression was used to assess the effect of social and economic Z-codes on hospital admission through the ED. Payer source, gender, age at admission, comorbidity count, and median ZIP code income quartile covariates were included in the logistic regression analyses. Patients with documented social and economic Z-codes were significantly more likely to be admitted through the ED than those without documented social and economic needs, after adjusting for covariates. Standardized and widespread collection of these valuable Z-codes within EHR systems or administrative claims databases can help with targeted resource allocation to alleviate possible barriers to care and mitigate ED utilization.

Association Between the Simplest Clinical Factors and Emergency Department Dispositions: A Retrospective Observational Study

The emergency department (ED) is a complex and busy environment that requires rapid decision making. We assessed the relationship between disposition from the ED and information that can be obtained at a glance in the ED. The presentation of the implications of commonplace information could assist healthcare providers in ensuring smooth and safe ED care. Thus, we aimed to quantitatively assess how readily obtainable findings, such as age, sex, and vital signs, are involved in the disposition of adult patients transferred to the ED. This retrospective observational study was conducted in the ED of a regional university hospital containing approximately 600 beds. Of the 685 patients included in the analysis, 351 patients were admitted to the hospital (including 12 deaths in the ED) and 334 patients were discharged from the ED. A multiple logistic regression model that included age, sex, systolic blood pressure, heart rate, respiration rate, temperature, and SpO₂ as variables identified independent associations between age (p=0.003), sex (p<0.001), systolic blood pressure (p=0.023), heart rate (p<0.001), and respiratory rate (p=0.028) and admission from the ED. The receiver operating characteristic curves drawn from the multiple logistic regression model comprising these five variables had an area under the curve (AUC) of 0.701 (95% confidence interval: 0.657-0.744, p<0.001). Examination of sensitivity, specificity, and likelihood ratios (LRs) for these five variables for clinical utility showed a slightly higher sensitivity for age ≥50 years (0.849) and respiratory rate ≥18 bpm (0.769); higher specificity for systolic blood pressure ≤100 mmHg (0.938), pulse rate ≥100 bpm (0.834), and respiratory rate ≥26 bpm (0.887); higher positive LR for systolic blood pressure ≤100 mmHg (2.039) and pulse rate ≥110 bpm (2.729); and slightly lower negative LR for age ≥50 years (0.656), male sex (0.647), respiratory rate ≥20 bpm (0.669). These results are meaningful as they quantify the intuition of a skilled clinician, which can help in clinical decision making, reduce errors, and promote clinical education. Our study provides a basis for explaining to novice healthcare providers that the careful observation of ED patients, even in the absence of special laboratory tests, can help them to make judgments regarding the disposition of the patients from the ED. In conclusion, age, sex, systolic blood pressure, heart rate, and respiratory rate were independently associated with a disposition from the ED. A multivariate model including these five variables showed the moderate-quality potential to predict admission from the ED. The sensitivity, specificity, and LR of systolic blood pressure, heart rate, and respiratory rate showed the characteristics of each vital sign. These provide healthcare providers in the ED an immediate clue regarding the patient’s illness.

Factors associated with admission to bed-based care: observational prospective cohort study in a multidisciplinary same day emergency care unit (SDEC)

BACKGROUND

The development of ambulatory emergency care services, now called 'Same Day Emergency Care' (SDEC) has been advocated to provide sustainable high quality healthcare in an ageing population. However, there are few data on SDEC and the factors associated with successful ambulatory care in frail older people. We therefore undertook a prospective observational study to determine i) the clinical characteristics and frailty burden of a cohort in an SDEC designed around the needs of older patients and ii) the factors associated with hospital admission within 30-days after initial assessment.

METHODS

The study setting was the multidisciplinary Abingdon Emergency Medical Unit (EMU) located in a community hospital and led by a senior interface physician (geriatrician or general practitioner). Consecutive patients from August-December 2015 were assessed using a structured paper proforma including cognitive/delirium screen, comorbidities, functional, social, and nutritional status. Physiologic parameters were recorded. Illness severity was quantified using the Systemic Inflammatory Response Syndrome (SIRS> 1). Factors associated with hospitalization within 30-days were determined using multivariable logistic regression.

RESULTS

Among 533 patients (median (IQR) age = 81 (68-87), 315 (59%) female), 453 (86%) were living at home but 283 (54%) required some form of care and 299 (56%) had Barthel< 20. Falls, urinary incontinence and dementia affected 81/189 (43%), 50 (26%) and 40 (21%) of those aged > 85 years." Severe illness was present in 148 (28%) with broadly similar rates across age groups. Overall, 210 (39%) patients had a hospital admission within 30-days with higher rates in older patients: 96 (87%) of < 65 years remained on an ambulatory pathway versus only 91 (48%) of ≥ 85 years (p < 0.0001). Factors independently associated with hospital admission were severe illness (SIRS/point, OR = 1.46,95% CI = 1.15-1.87, p = 0.002) and markers of frailty: delirium (OR = 11.28,3.07-41.44, p < 0.0001), increased care needs (OR = 3.08,1.55-6.12, p = 0.001), transport requirement (OR = 1.92,1.13-3.27), and poor nutrition (OR = 1.13-3.79, p = 0.02).

CONCLUSIONS

Even in an SDEC with a multidisciplinary approach, rates of hospital admission in those with severe illness and frailty were high. Further studies are required to understand the key components of hospital bed-based care that need to be replicated by models delivering acute frailty care closer to home, and the feasibility, cost-effectiveness and patient/carer acceptability of such models.

Provider-in-triage prediction of hospital admission after brief patient interaction

BACKGROUND AND OBJECTIVES

We sought to determine if emergency physician providers working in the triage area (PIT) of the ED could accurately predict the likelihood of admission for patients at the time of triage. Such predictions, if accurate, could decrease the time spent in the ED for patients who are admitted to the hospital by hastening downstream workflow.

METHODS

This is a prospective cohort study of PIT providers at a large urban hospital. Physicians were asked to predict the likelihood of admission and confidence of prediction for patients after evaluating them in triage. Measures of predictive accuracy were calculated, including sensitivity, specificity, and area under the receiver operator characteristic (AUROC).

RESULTS

36 physicians (20 attendings, 16 residents) evaluated 340 patients and made predictions. The average patient age was 48 (range 18-94) and 52% were female. Seventy-three patients (21%) were admitted (5% observation, 85% general care/telemetry, 7% progressive care, 3% ICU). The sensitivity of determining admission for the entire cohort was 74%, the specificity was 84%, and the AUROC was 0.81. When physicians were at least 80% confident in their predictions, the predictions improved to sensitivity of 93%, specificity of 96%, and AUROC 0.95 (Graph 1).

CONCLUSION

The accuracy of physician providers-in-triage of predicting hospital admission was very good when those predictions were made with higher degrees of confidence. These results indicate that while general predictions of admission are likely inadequate to guide downstream workflow, predictions in which the physician is confident could provide utility.

Preventable emergency admissions of older adults: an observational mixed-method study of rates, associative factors and underlying causes in two Dutch hospitals

OBJECTIVE

Older adults are hospitalised from the emergency department (ED) without potentially needing hospital care. Knowledge about rates, associative factors and causes of these preventable emergency admissions (PEAs) is limited. This study aimed to determine the rates, associative factors and causes for PEAs of older adults.

DESIGN

A mixed-method observational study.

SETTING

The EDs of two Dutch hospitals.

PARTICIPANTS

492 patients aged >70 years and hospitalised from the ED.

MEASUREMENTS

Quantitative data were retrospectively extracted from the electronical medical record over a 1-month period. Admissions were classified (non)preventable based on a standardised approach. Univariate and multivariate multilevel logistic regression analyses were performed to identify possible associations between PEAs and demographic, clinical and care process factors. Qualitative data were prospectively collected by email and telephone interviews and analysed thematically to explore hospital physician's perceived causes for the identified PEAs.

RESULTS

Of the 492 included cases, 86 (17.5%) were classified as PEA. Patients with a higher age (adjusted OR 1.04, 95% CI 1.01 to 1.08; p=0.04), a low urgency classification (adjusted OR 1.89, 95% CI 1.14 to 3.15; p=0.01), and attending the ED in the weekend (adjusted OR 2.02, 95% CI 1.22 to 3.37; p<0.01) were associated with an increased likelihood of a PEA. 49 physicians were interviewed by telephone and email. Perceived causes for PEAs were related to patient's attitudes (eg, postponement of medical care at home), provider's attitudes (eg, deciding for admission after family pressure), health system deficiencies (eg, limited access to community services during out-of-hours and delayed access to inpatient diagnostic resources) and poor communication between primary care and hospital professionals about patient treatment preferences.

CONCLUSIONS

Our findings contribute to existing evidence that many emergency admissions of older adults are preventable, thereby indicating a possible source of unnecessary expensive, and potentially harmful, hospital care.

Institution-Specific Machine Learning Models for Prehospital Assessment to Predict Hospital Admission: Prediction Model Development Study

Background

Although multiple prediction models have been developed to predict hospital admission to emergency departments (EDs) to address overcrowding and patient safety, only a few studies have examined prediction models for prehospital use. Development of institution-specific prediction models is feasible in this age of data science, provided that predictor-related information is readily collectable.

Objective

We aimed to develop a hospital admission prediction model based on patient information that is commonly available during ambulance transport before hospitalization.

Methods

Patients transported by ambulance to our ED from April 2018 through March 2019 were enrolled. Candidate predictors were age, sex, chief complaint, vital signs, and patient medical history, all of which were recorded by emergency medical teams during ambulance transport. Patients were divided into two cohorts for derivation (3601/5145, 70.0%) and validation (1544/5145, 30.0%). For statistical models, logistic regression, logistic lasso, random forest, and gradient boosting machine were used. Prediction models were developed in the derivation cohort. Model performance was assessed by area under the receiver operating characteristic curve (AUROC) and association measures in the validation cohort.

Results

Of 5145 patients transported by ambulance, including deaths in the ED and hospital transfers, 2699 (52.5%) required hospital admission. Prediction performance was higher with the addition of predictive factors, attaining the best performance with an AUROC of 0.818 (95% CI 0.792-0.839) with a machine learning model and predictive factors of age, sex, chief complaint, and vital signs. Sensitivity and specificity of this model were 0.744 (95% CI 0.716-0.773) and 0.745 (95% CI 0.709-0.776), respectively.

Conclusions

For patients transferred to EDs, we developed a well-performing hospital admission prediction model based on routinely collected prehospital information including chief complaints.

Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing

Emergency department triage is the first point in time when a patient's acuity level is determined. The time to assign a priority at triage is short and it is vital to accurately stratify patients at this stage, since under-triage can lead to increased morbidity, mortality and costs. Our aim was to present a model that can assist healthcare professionals in triage decision making, namely in the stratification of patients through the risk prediction of a composite critical outcome-mortality and cardiopulmonary arrest. Our study cohort consisted of 235826 adult patients triaged at a Portuguese Emergency Department from 2012 to 2016. Patients were assigned to emergent, very urgent or urgent priorities of the Manchester Triage System (MTS). Demographics, clinical variables routinely collected at triage and the patients' chief complaint were used. Logistic regression, random forests and extreme gradient boosting were developed using all available variables. The term frequency-inverse document frequency (TF-IDF) natural language processing weighting factor was applied to vectorize the chief complaint. Stratified random sampling was used to split the data into train (70%) and test (30%) data sets. Ten-fold cross validation was performed in train to optimize model hyper-parameters. The performance obtained with the best model was compared against the reference model-a regularized logistic regression trained using only triage priorities. Extreme gradient boosting exhibited good calibration properties and yielded areas under the receiver operating characteristic and precision-recall curves of 0.96 (95% CI 0.95-0.97) and 0.31 (95% CI 0.26-0.36), respectively. The predictors ranked with higher importance by this model were the Glasgow coma score, the patients' age, pulse oximetry and arrival mode. Compared to the reference, the extreme gradient boosting model using clinical variables and the chief complaint presented higher recall for patients assigned MTS-3 and can identify those who are at risk of the composite outcome.

Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing

The risk stratification of patients in the emergency department begins at triage. It is vital to stratify patients early based on their severity, since undertriage can lead to increased morbidity, mortality and costs. Our aim was to present a new approach to assist healthcare professionals at triage in the stratification of patients and in identifying those with higher risk of ICU admission. Adult patients assigned Manchester Triage System (MTS) or Emergency Severity Index (ESI) 1 to 3 from a Portuguese and a United States Emergency Departments were analyzed. Variables routinely collected at triage were used and natural language processing was applied to the patient chief complaint. Stratified random sampling was applied to split the data in train (70%) and test (30%) sets and 10-fold cross validation was performed for model training. Logistic regression, random forests, and a random undersampling boosting algorithm were used. We compared the performance obtained with the reference model—using only triage priorities—with the models using additional variables. For both hospitals, a logistic regression model achieved higher overall performance, yielding areas under the receiver operating characteristic and precision-recall curves of 0.91 (95% CI 0.90-0.92) and 0.30 (95% CI 0.27-0.33) for the United States hospital and of 0.85 (95% CI 0.83-0.86) and 0.06 (95% CI 0.05-0.07) for the Portuguese hospital. Heart rate, pulse oximetry, respiratory rate and systolic blood pressure were the most important predictors of ICU admission. Compared to the reference models, the models using clinical variables and the chief complaint presented higher recall for patients assigned MTS/ESI 3 and can identify patients assigned MTS/ESI 3 who are at risk for ICU admission.

Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: a systematic review

BACKGROUND

The primary objective of this review is to assess the accuracy of machine learning methods in their application of triaging the acuity of patients presenting in the Emergency Care System (ECS). The population are patients that have contacted the ambulance service or turned up at the Emergency Department. The index test is a machine-learning algorithm that aims to stratify the acuity of incoming patients at initial triage. This is in comparison to either an existing decision support tool, clinical opinion or in the absence of these, no comparator. The outcome of this review is the calibration, discrimination and classification statistics.

METHODS

Only derivation studies (with or without internal validation) were included. MEDLINE, CINAHL, PubMed and the grey literature were searched on the 14th December 2019. Risk of bias was assessed using the PROBAST tool and data was extracted using the CHARMS checklist. Discrimination (C-statistic) was a commonly reported model performance measure and therefore these statistics were represented as a range within each machine learning method. The majority of studies had poorly reported outcomes and thus a narrative synthesis of results was performed.

RESULTS

There was a total of 92 models (from 25 studies) included in the review. There were two main triage outcomes: hospitalisation (56 models), and critical care need (25 models). For hospitalisation, neural networks and tree-based methods both had a median C-statistic of 0.81 (IQR 0.80-0.84, 0.79-0.82). Logistic regression had a median C-statistic of 0.80 (0.74-0.83). For critical care need, neural networks had a median C-statistic of 0.89 (0.86-0.91), tree based 0.85 (0.84-0.88), and logistic regression 0.83 (0.79-0.84).

CONCLUSIONS

Machine-learning methods appear accurate in triaging undifferentiated patients entering the Emergency Care System. There was no clear benefit of using one technique over another; however, models derived by logistic regression were more transparent in reporting model performance. Future studies should adhere to reporting guidelines and use these at the protocol design stage.

REGISTRATION AND FUNDING

This systematic review is registered on the International prospective register of systematic reviews (PROSPERO) and can be accessed online at the following URL: https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42020168696This study was funded by the NIHR as part of a Clinical Doctoral Research Fellowship.

Failure of vital sign normalization is more strongly associated than single measures with mortality and outcomes

BACKGROUND

Modified Early Warning Systems (MEWS) scores offer proxies for morbidity and mortality that are easily acquired, but there are limited data on what changing MEWS scores within the ED indicate. We examined the correlation of changing MEWS scores during resuscitation in the ED and in-hospital morbidity and mortality.

METHODS

We conducted a retrospective analysis on medical ED patients with simplified MEWS scores (without urine output or mental status) admitted to a single academic tertiary care center over one year. Triage-to-Last delta MEWS score and Triage-to-Max delta MEWS scores were calculated and correlated to in-hospital mortality, ICU admission, length of stay (LOS) and diagnosis of sepsis.

RESULTS

Our analysis included 8322 ED patients with an ICU admission rate of 17% and a mortality rate of 2%. Every point of worsened MEWS after triage was more strongly associated with all-cause mortality (OR 2.41, 95% CI 1.96-2.97) than triage MEWS alone (OR 1.33, 95% CI 1.23-1.44; p < 0.001). Likewise, each point of worsened MEWS was associated with increased odds of ICU admission (Triage-to-Last: OR 2.12, 95% CI 1.92-2.33 and Triage-to-Max: OR 1.52, 95% CI 1.45-1.60, respectively). Among patients with suspected infection, similar associations are found.

CONCLUSIONS

Dynamic vital signs in the emergency department, as categorized by delta MEWS, and failure to normalize abnormalities, were associated with increased mortality, ICU admission, LOS, and the diagnosis of sepsis. Our results suggest that MEWS scores that do not normalize, from triage onward, are more strongly associated with outcome than any single score.

The Sydney triage to admission risk tool (START) to improve patient flow in an emergency department: a model of care implementation pilot study

Background

The Sydney Triage to Admission Risk Tool (START) is a validated clinical analytics tool designed to estimate the probability of in-patient admission based on Emergency Department triage characteristics.

Methods

This was a single centre pilot implementation study using a matched case control sample of patients assessed at ED triage. Patients in the intervention group were identified at triage by the START tool as likely requiring in-patient admission and briefly assessed by an ED Consultant. Bed management were notified of these patients and their likely admitting team based on senior early assessment. Matched controls were identified on the same day of presentation if they were admitted to the same in-patient teams as patients in the intervention group and same START score category. Outcomes were ED length of stay and proportion of patients correctly classified as an in-patient admission by the START tool.

Results

One hundred and thirteen patients were assessed using the START-based model of care. When compared with matched control patients, this intervention model of care was associated with a significant reduction in ED length of stay [301 min (IQR 225–397) versus 423 min (IQR 297–587) p < 0.001] and proportion of patients meeting 4 h length of stay thresholds increased from 24 to 45% (p < 0.001).

Conclusion

In this small pilot implementation study, the START tool, when used in conjunction with senior early assessment was associated with a reduction in ED length of stay. Further controlled studies are now underway to further examine its utility across other ED settings.

Prediction of emergency department patient disposition decision for proactive resource allocation for admission

We investigate the capability of information from electronic health records of an emergency department (ED) to predict patient disposition decisions for reducing "boarding" delays through the proactive initiation of admission processes (e.g., inpatient bed requests, transport, etc.). We model the process of ED disposition decision prediction as a hierarchical multiclass classification while dealing with the progressive accrual of clinical information throughout the ED caregiving process. Multinomial logistic regression as well as machine learning models are built for carrying out the predictions. Utilizing results from just the first set of ED laboratory tests along with other prior information gathered for each patient (2.5 h ahead of the actual disposition decision on average), our model predicts disposition decisions with positive predictive values of 55.4%, 45.1%, 56.9%, and 47.5%, while controlling false positive rates (1.4%, 1.0%, 4.3%, and 1.4%), with AUC values of 0.97, 0.95, 0.89, and 0.84 for the four admission (minor) classes, i.e., intensive care unit (3.6% of the testing samples), telemetry unit (2.2%), general practice unit (11.9%), and observation unit (6.6%) classes, respectively. Moreover, patients destined to intensive care unit present a more drastic increment in prediction quality at triage than others. Disposition decision classification models can provide more actionable information than a binary admission vs. discharge prediction model for the proactive initiation of admission processes for ED patients. Observing the distinct trajectories of information accrual and prediction quality evolvement for ED patients destined to different types of units, proactive coordination strategies should be tailored accordingly for each destination unit.

Multicentre, prospective observational study of the correlation between the Glasgow Admission Prediction Score and adverse outcomes

OBJECTIVES

To assess whether the Glasgow Admission Prediction Score (GAPS) is correlated with hospital length of stay, 6-month hospital readmission and 6-month all-cause mortality. This study represents a 6-month follow-up of patients who were included in an external validation of the GAPS' ability to predict admission at the point of triage.

SETTING

Sampling was conducted between February and May 2016 at two separate emergency departments (EDs) in Sheffield and Glasgow.

PARTICIPANTS

Data were collected prospectively at triage for consecutive adult patients who presented to the ED within sampling times. Any patients who avoided formal triage were excluded from the study. In total, 1420 patients were recruited.

PRIMARY OUTCOMES

GAPS was calculated following triage and did not influence patient management. Length of hospital stay, hospital readmission and mortality against GAPS were modelled using survival analysis at 6 months.

RESULTS

Of the 1420 patients recruited, 39.6% of these patients were initially admitted to hospital. At 6 months, 30.6% of patients had been readmitted and 5.6% of patients had died. For those admitted at first presentation, the chance of being discharged fell by 4.3% (95% CI 3.2% to 5.3%) per GAPS point increase. Cox regression indicated a 9.2% (95% CI 7.3% to 11.1%) increase in the chance of 6-month hospital readmission per point increase in GAPS. An association between GAPS and 6-month mortality was demonstrated, with a hazard increase of 9.0% (95% CI 6.9% to 11.2%) for every point increase in GAPS.

CONCLUSION

A higher GAPS is associated with increased hospital length of stay, 6-month hospital readmission and 6-month all-cause mortality. While GAPS's primary application may be to predict admission and support clinical decision making, GAPS may provide valuable insight into inpatient resource allocation and bed planning.

Real-time estimation of inpatient beds required in emergency departments

BACKGROUND

Long boarding time in emergency department (ED) leads to increased morbidity and mortality. Prediction of admissions upon triage could improve ED care efficiency and decrease boarding time.

OBJECTIVE

To develop a real-time automated model (MA) to predict admissions upon triage and compare this model with triage nurse prediction (TNP).

PATIENTS AND METHODS

A cross-sectional study was conducted in four EDs during 1 month. MA used only variables available upon triage and included in the national French Electronic Emergency Department Abstract. For each patient, the triage nurse assessed the hospitalization risk on a 10-point Likert scale. Performances of MA and TNP were compared using the area under the receiver operating characteristic curves, the accuracy, and the daily and hourly mean difference between predicted and observed number of admission.

RESULTS

A total of 11 653 patients visited the EDs, and 19.5-24.7% were admitted according to the emergency. The area under the curves (AUCs) of TNP [0.815 (0.805-0.826)] and MA [0.815 (0.805-0.825)] were similar. Across EDs, the AUCs of TNP were significantly different (P < 0.001) in all EDs, whereas AUCs of MA were all similar (P>0.2). Originally, using daily and hourly aggregated data, the percentage of errors concerning the number of predicted admission were 8.7 and 34.4%, respectively, for MA and 9.9 and 35.4%, respectively, for TNP.

CONCLUSION

A simple model using variables available in all EDs in France performed well to predict admission upon triage. However, when analyzed at an hourly level, it overestimated the number of inpatient beds needed by a third. More research is needed to define adequate use of these models.

Predicting hospital admission at the emergency department triage: A novel prediction model

BACKGROUND

Emergency department (ED) overcrowding is a growing international patient safety issue. A major contributor to overcrowding is long wait times for inpatient hospital admission. The objective of this study is to create a model that can predict a patient's need for hospital admission at the time of triage.

METHODS

Retrospective observational study of electronic clinical records of all ED visits over ten years to a large urban hospital in Singapore. The data was randomly divided into a derivation set and a validation set. We used the derivation set to develop a logistic regression model that predicts probability of hospital admission for patients presenting to the ED. We tested the model on the validation set and evaluated the performance with receiver operating characteristic (ROC) curve analysis.

RESULTS

A total of 1,232,016 visits were included for final analysis, of which 38.7% were admitted. Eight variables were included in the final model: age group, race, postal code, day of week, time of day, triage category, mode of arrival, and fever status. The model performed well on the validation set with an area under the curve of 0.825 (95% CI 0.824-0.827). Increasing age, increasing triage acuity, and mode of arrival via private patient transport were most predictive of the need for admission.

CONCLUSIONS

We developed a model that accurately predicts admission for patients presenting to the ED using demographic, administrative, and clinical data routinely collected at triage. Implementation of the model into the electronic health record could help reduce the burden of overcrowding.

Refining and testing the diagnostic accuracy of an assessment tool (PAT-POPS) to predict admission and discharge of children and young people who attend an emergency department: protocol for an observational study

BACKGROUND

Increasing attendances by children (aged 0-16 years) to United Kingdom Emergency Departments (EDs) challenges patient safety within the National Health Service (NHS) with health professionals required to make complex judgements on whether children attending urgent and emergency care services can be sent home safely or require admission. Health regulation bodies have recommended that an early identification systems should be developed to recognise children developing critical illnesses. The Pennine Acute Hospitals NHS Trust Paediatric Observation Priority Score (PAT-POPS) was developed as an ED-specific tool for this purpose. This study aims to revise and improve the existing tool and determine its utility in determining safe admission and discharge decision making.

METHODS/DESIGN

An observational study to improve diagnostic accuracy using data from children and young people attending the ED and Urgent Care Centre (UCC) at three hospitals over a 12 month period. The data being collected is part of routine practice; therefore opt-out methods of consent will be used. The reference standard is admission or discharge. A revised PAT-POPs scoring tool will be developed using clinically guided logistic regression models to explore which components best predict hospital admission and safe discharge. Suitable cut-points for safe admission and discharge will be established using sensitivity and specificity as judged by an expert consensus meeting. The diagnostic accuracy of the revised tool will be assessed, and it will be compared to the former version of PAT-POPS using ROC analysis.

DISCUSSION

This new predictive tool will aid discharge and admission decision-making in relation to children and young people in hospital urgent and emergency care facilities.

TRIAL REGISTRATION

NIHR RfPB Grant: PB-PG-0815-20034. ClinicalTrials.gov: 213469. Retrospectively registered on 11 April 2018.