The emergency department prediction of disposition (EPOD) study

The accuracy of predicting hospital admission by emergency medical service and emergency department personnel compared to the prehospital MEWS: a prospective multicenter study

INTRODUCTION

Overcrowding in the emergency department (ED) is a global problem. Early and accurate recognition of a patient's disposition could limit time spend at the ED and thus improve throughput and quality of care provided. This study aims to compare the accuracy among healthcare providers and the prehospital Modified Early Warning Score (MEWS) in predicting the requirement for hospital admission.

METHODS

A prospective, observational, multi-centre study was performed including adult patients brought to the ED by ambulance. Involved Emergency Medical Service (EMS) personnel, ED nurses and physicians were asked to predict the need for hospital admission using a structured questionnaire. Primary endpoint was the comparison between the accuracy of healthcare providers and prehospital MEWS in predicting patients' need for hospital admission.

RESULTS

In total 798 patients were included of whom 393 (49.2%) were admitted to the hospital. Sensitivity of predicting hospital admission varied from 80.0 to 91.9%, with physicians predicting hospital admission significantly more accurately than EMS and ED nurses (p < 0.001). Specificity ranged from 56.4 to 67.0%. All healthcare providers outperformed MEWS ≥ 3 score on predicting hospital admission (sensitivity 80.0-91.9% versus 44.0%; all p < 0.001). Predictions for ward admissions specifically were significantly more accurate than MEWS (specificity 94.7-95.9% versus 60.6%, all p < 0.001).

CONCLUSIONS

Healthcare providers can accurately predict the need for hospital admission, and all providers outperformed the MEWS score.

Determination of emergency nurse practitioner and plastic surgery trainee disposition decision agreement for plastic surgery emergency department presentations: A prospective study

OBJECTIVE

To examine if there was a high degree of agreement for disposition decisions of emergency nurse practitioners (ENP) compared to plastic surgery trainees (PST) for plastic surgery presentations.

METHODS

A prospective study of disposition decision agreement from February 2020 to January 2021 for patients who required plastic surgery consultation and managed exclusively by an ENP. Absolute percentages were used to determine the exact disposition decision accuracy of ENP and the PST, while Cohen's kappa compared disposition decision agreement. Sub-analyses of age, gender, ENP experience and presenting condition agreement were also completed. To mitigate confounding factors, operative management (OM) and non-OM groups were analysed.

RESULTS

The study recruited 342 patients who presented mostly with finger or hand-related conditions (82%, n = 279) and managed by an ENP with less than 10 years of experience (65%, n = 224). Disposition decisions by ENP compared to PST were the same in 80% (n = 274) of cases. Disposition agreement for all patients was 0.72 (95% confidence interval 0.66-0.78). For the OM and non-OM groups, disposition decisions were the same in 94% (n = 320), with a Cohen's kappa 0.85 (95% confidence interval 0.79-0.91). Seven patients (2%) were discharged to GP care by the ENP when determined to need further plastic surgery involvement by the PST.

CONCLUSIONS

Disposition decisions by ENP and PST were the same in most cases and had a high overall level of agreement. This may lead to greater autonomy of ENP care and reduced ED length of stay and occupancy.

Predicting Admissions From a Paediatric Emergency Department - Protocol for Developing and Validating a Low-Dimensional Machine Learning Prediction Model

Introduction: Patients boarding in the Emergency Department can contribute to overcrowding, leading to longer waiting times and patients leaving without being seen or completing their treatment. The early identification of potential admissions could act as an additional decision support tool to alert clinicians that a patient needs to be reviewed for admission and would also be of benefit to bed managers in advance bed planning for the patient. We aim to create a low-dimensional model predicting admissions early from the paediatric Emergency Department.

Methods and Analysis: The methodology Cross Industry Standard Process for Data Mining (CRISP-DM) will be followed. The dataset will comprise of 2 years of data, ~76,000 records. Potential predictors were identified from previous research, comprising of demographics, registration details, triage assessment, hospital usage and past medical history. Fifteen models will be developed comprised of 3 machine learning algorithms (Logistic regression, naïve Bayes and gradient boosting machine) and 5 sampling methods, 4 of which are aimed at addressing class imbalance (undersampling, oversampling, and synthetic oversampling techniques). The variables of importance will then be identified from the optimal model (selected based on the highest Area under the curve) and used to develop an additional low-dimensional model for deployment.

Discussion: A low-dimensional model comprised of routinely collected data, captured up to post triage assessment would benefit many hospitals without data rich platforms for the development of models with a high number of predictors. Novel to the planned study is the use of data from the Republic of Ireland and the application of sampling techniques aimed at improving model performance impacted by an imbalance between admissions and discharges in the outcome variable.

Physician Perceptions of Disposition Decision-making for Older Adults in the Emergency Department: A Preliminary Analysis

Disposition decision-making in the emergency department (ED) is critical to patient safety and quality of care. Disposition decision-making has particularly important implications for older adults who comprise a significant portion of ED visits annually and are vulnerable to suboptimal outcomes throughout ED care transitions. We conducted a secondary inductive content analysis of interviews with ED physicians (N= 11) to explore their perceptions of who they involve in disposition decision-making and what information they use to make disposition decisions for older adults. ED physicians cited 7 roles (5 types of clinicians, patients and families) and 11 information types, both clinical (e.g. test/lab results) and non-clinical (e.g. family's preference). Our preliminary findings represent a key first step toward the development of interventions that promote patient safety and quality of care for older adults in the ED by supporting the cognitive and communicative aspects of disposition decision-making.

Statistical learning methodologies and admission prediction in an emergency department

BACKGROUND

The quality of an emergency department (ED) is highly dependent on its ability to supply efficient, as well as high-quality treatment for all patients. Key performance indicators are important when measuring the performance of an emergency department. This study aimed to perform an exploratory data analysis and to develop an admission prediction model based on a dataset that was constructed from key performance indicators selected by a panel of expert physicians, nurses and hospital administrators.

METHODS

A dataset of 172,695 records was retrospectively collected from an Emergency Department. The relationships within the dataset were analyzed and three machine learning algorithms were compared for an admission predictive model based on the initial patient information.

RESULTS

The dataset showed that mean length of stay was similar in the different weekdays, there was a positive linear relationship between the length of stay and patient age and the admission predictive model yielded an AUC of 0.79.

CONCLUSIONS

The selected indicators can be used to study whether emergency department allocates its resources properly to cope with overcrowding and the predictive model may be employed by Hospital and ED administrates to fill information gaps and support decision making for the improvement of the key performance indicators.

Provider-in-triage prediction of hospital admission after brief patient interaction

BACKGROUND AND OBJECTIVES

We sought to determine if emergency physician providers working in the triage area (PIT) of the ED could accurately predict the likelihood of admission for patients at the time of triage. Such predictions, if accurate, could decrease the time spent in the ED for patients who are admitted to the hospital by hastening downstream workflow.

METHODS

This is a prospective cohort study of PIT providers at a large urban hospital. Physicians were asked to predict the likelihood of admission and confidence of prediction for patients after evaluating them in triage. Measures of predictive accuracy were calculated, including sensitivity, specificity, and area under the receiver operator characteristic (AUROC).

RESULTS

36 physicians (20 attendings, 16 residents) evaluated 340 patients and made predictions. The average patient age was 48 (range 18-94) and 52% were female. Seventy-three patients (21%) were admitted (5% observation, 85% general care/telemetry, 7% progressive care, 3% ICU). The sensitivity of determining admission for the entire cohort was 74%, the specificity was 84%, and the AUROC was 0.81. When physicians were at least 80% confident in their predictions, the predictions improved to sensitivity of 93%, specificity of 96%, and AUROC 0.95 (Graph 1).

CONCLUSION

The accuracy of physician providers-in-triage of predicting hospital admission was very good when those predictions were made with higher degrees of confidence. These results indicate that while general predictions of admission are likely inadequate to guide downstream workflow, predictions in which the physician is confident could provide utility.

Ability of triage nurses to predict, at the time of triage, the eventual disposition of patients attending the emergency department (ED): a systematic literature review and meta-analysis

Introduction

Exit block is the most significant cause of poor patient flow and crowding in the emergency department (ED). One proposed strategy to reduce exit block is early admission predictions by triage nurses to allow proactive bed management. We report a systematic review and meta-analysis of the accuracy of nurse prediction of admission at triage.

Methodology

We searched MEDLINE, Cochrane, Embase, CINAHL and grey literature, up to and including February 2019. Our criteria were as follows: prospective studies analysing the accuracy of triage nurse intuition—after gathering standard triage information—for predicting disposition for adult ED patients. We analysed the results of this test—nurse prediction of disposition—in a diagnostic test analysis review style, assessing methodology with the Quality Assessment of Diagnostic Accuracy Studies 2 checklist. We generated sensitivity, specificity and likelihood ratios (LRs). We used LRs and pretest probability of admission (baseline admission rate) to find positive and negative post-test probabilities.

Results

We reviewed 10 articles. Of these, seven had meta-analysable data (12 282 participants). The studies varied in participant selection and admission rate, but the majority were of moderate quality and exclusion of each in sensitivity analyses made little difference. Sensitivity was 72% and specificity was 83%. Pretest probability of admission was 29%. Positive and negative post-test probabilities of admission were 63% and 12%, respectively.

Conclusion

Triage nurse prediction of disposition is not accurate enough to expedite admission for ED patients on a one-to-one basis. Future research should explore the benefit, and best method, of predicting total demand.

Using emergency physicians’ abilities to predict patient admission to decrease admission delay time

Background

In many EDs, emergency physicians (EPs) do not have admitting privileges and must wait for consultants to further assess and admit patients. This delays bed requests and increases ED crowding. We measured EPs’ abilities to predict patient admission prior to consultation and estimated the potential ED stretcher time saved if EPs requested a bed with consultation.

Methods

We conducted a prospective cohort study in an academic centre in Canada between October 2017 and February 2018 using a convenience sample of ED patient encounters requiring consultation. We excluded patients under 18 years or those clearly likely to be admitted (traumas, strokes, S-T elevation myocardial infarctions and Canadian Triage and Acuity Scale of 1). EPs predicted patient admission just before consultation. Potential ED stretcher time saved was estimated for correctly predicted admissions assuming bed requests were initiated with consultation and a constant time to inpatient bed.

Results

Characteristics of 454 patients were: mean age 60.1 years, 48.5% male, 46.9% evening presentation, 69.4% admitted and median time to bed request of 3.5 hours (IQR 2.0–5.3 hours). Overall, EPs prediction sensitivity, specificity, positive predictive value and negative predictive value were 90.5% (95% CI 86.7% to 93.5%), 84.2% (95% CI 77.0% to 89.8%), 92.8% (95% CI 89.8% to 95.0%) and 79.6% (95% CI 73.4% to 84.7%). Approximately 922.1 hours of ED stretcher time could have been saved during the 5-month study period if EPs initiated a bed request with consultation.

Conclusion

Crowding is a reality for EDs worldwide, and many systems could benefit from EP-initiated hospital admissions to decrease the amount of time admitted patients wait in the ED.

Development and evaluation of a code frame to identify potential primary care presentations in the hospital emergency department

OBJECTIVE

A major challenge in evaluating the appropriateness of ED presentations is the lack of a universal and workable definition of patients who could have received primary care instead. Our objective was to develop a standardised code frame to identify potential primary care patients in the ED.

METHODS

A standardised code frame to identify which patients could potentially be treated in a primary care setting was developed and tested on all patient episodes of care who presented to the ED of the St George Hospital, Sydney, between December 2016 and February 2017. Sensitivity and specificity of the code frame were performed. The code frame was then tested on all presentations from 2011 to 2016 in the St George Hospital and The Sutherland Hospital in Sydney.

RESULTS

Of 19 916 ED presentations, 5810 (29%) were potential primary care presentations. The code frame had a sensitivity of 99.9% and a specificity of 49.0%. Results were consistent (28%) when applied to 5 years of presentations (601 168 presentations).

CONCLUSION

This standardised code frame enables accurate retrospective local and national data estimations. The code frame could be used prospectively to evaluate interventions such as diverting patients to primary care settings, and to identify populations for specifically targeted interventions. The conservative nature of the code frame ensures that only those that can safely receive care in a primary care setting are identified as potential primary care.

CCMapper: An adaptive NLP-based free-text chief complaint mapping algorithm

OBJECTIVE

Chief complaint (CC) is among the earliest health information recorded at the beginning of a patient's visit to an emergency department (ED). We propose a heuristic methodology for automatically mapping the free-text data into a structured list of CCs.

METHODS

A comprehensive structured list categorizing CCs was developed by experienced Emergency Medicine (EM) physicians. Using this list, we developed a natural language processing-based algorithm, referred to as Chief Complaint Mapper (CCMapper), for automatically mapping a CC into the most appropriate category (ies). We trained and validated CCMapper using free-text CC data from the Mayo Clinic ED in Rochester, MN. We developed a consensus-based validation approach to handle both indifferences and disagreements between the two EM physicians who manually mapped a random sample of free-text CCs into categories within the structured list.

RESULTS

The kappa statistic demonstrated a high level of agreement (κ = 0.958) between the two physicians with less than 2% human error. CCMapper achieved a total sensitivity of 94.2% with a specificity of 99.8% and F-score of 94.7% on the validation set. The sensitivity of CCMapper when mapping free-text data with multiple CCs was 82.3% with a specificity of 99.1% and total F-score of 82.3%.

CONCLUSION

Due to its simplicity, high performance, and capability of incorporating new free-text CC data, CCMapper can be readily adopted by other EDs to support clinical decision making. CCMapper can facilitate the development of predictive models for the type and timing of important events in ED (e.g., ICU admission).

Characterizing Potentially Preventable Admissions: A Mixed Methods Study of Rates, Associated Factors, Outcomes, and Physician Decision-Making

BACKGROUND

Potentially preventable admissions are a target for healthcare cost containment.

OBJECTIVE

To identify rates of, characterize associations with, and explore physician decision-making around potentially preventable admissions.

DESIGN

A comparative cohort study was used to determine rates of potentially preventable admissions and to identify associated factors and patient outcomes. A qualitative case study was used to explore physicians' clinical decision-making.

PARTICIPANTS

Patients admitted from the emergency department (ED) to the general medicine (GM) service over a total of 4 weeks were included as cases (N = 401). Physicians from both emergency medicine (EM) and GM that were involved in the cases were included (N = 82).

APPROACH

Physicians categorized admissions as potentially preventable. We examined differences in patient characteristics, admission characteristics, and patient outcomes between potentially preventable and control admissions. Interviews with participating physicians were conducted and transcribed. Transcriptions were systematically analyzed for key concepts regarding potentially preventable admissions.

KEY RESULTS

EM and GM physicians categorized 22.2% (90/401) of admissions as potentially preventable. There were no significant differences between potentially preventable and control admissions in patient or admission characteristics. Potentially preventable admissions had shorter length of stay (2.1 vs. 3.6 days, p < 0.001). There was no difference in other patient outcomes. Physicians discussed several provider, system, and patient factors that affected clinical decision-making around potentially preventable admissions, particularly in the "gray zone," including risk of deterioration at home, the risk of hospitalization, the cost to the patient, and the presence of outpatient resources. Differences in provider training, risk assessment, and provider understanding of outpatient access accounted for differences in decisions between EM and GM physicians.

CONCLUSIONS

Collaboration between EM and GM physicians around patients in the gray zone, focusing on patient risk, cost, and outpatient resources, may provide an avenues for reducing potentially preventable admissions and lowering healthcare spending.

The Sydney Triage to Admission Risk Tool (START) to predict Emergency Department Disposition: A derivation and internal validation study using retrospective state-wide data from New South Wales, Australia

Background

Disposition decisions are critical to the functioning of Emergency Departments. The objectives of the present study were to derive and internally validate a prediction model for inpatient admission from the Emergency Department to assist with triage, patient flow and clinical decision making.

Methods

This was a retrospective analysis of State-wide Emergency Department data in New South Wales, Australia. Adult patients (age ≥ 16 years) were included if they presented to a Level five or six (tertiary level) Emergency Department in New South Wales, Australia between 2013 and 2014. The outcome of interest was in-patient admission from the Emergency Department. This included all admissions to short stay and medical assessment units and being transferred out to another hospital. Analyses were performed using logistic regression. Discrimination was assessed using area under curve and derived risk scores were plotted to assess calibration.

Results

1,721,294 presentations from twenty three Level five or six hospitals were analysed. Of these 49.38% were male and the mean (sd) age was 49.85 years (22.13). Level 6 hospitals accounted for 47.70% of cases and 40.74% of cases were classified as an in-patient admission based on their mode of separation. The final multivariable model including age, arrival by ambulance, triage category, previous admission and presenting problem had an AUC of 0.82 (95% CI 0.81, 0.82).

Conclusion

By deriving and internally validating a risk score model to predict the need for in-patient admission based on basic demographic and triage characteristics, patient flow in ED, clinical decision making and overall quality of care may be improved. Further studies are now required to establish clinical effectiveness of this risk score model.