Chief complaint-based performance measures: a new focus for acute care quality measurement

Concordance in Medical Urgency Classification of Discharge Diagnoses and Reasons for Visit

Importance

Current policies to divert emergency department (ED) visits for less medically urgent conditions to more cost-effective settings rely on retrospective adjudication of discharge diagnoses. However, patients present to the ED with concerns, making it challenging for clinicians.

Objective

To characterize ED visits based on the medical urgency of the presenting reasons for visit and to explore the concordance between discharge diagnoses and reasons for visit.

Design, Setting, and Participants

In this retrospective, cross-sectional study, a nationwide sample of ED visits by adults (aged ≥18 years) in the US from the 2018 and 2019 calendar years' ED data of the National Hospital Ambulatory Medical Care Survey was used. An algorithm to probabilistically assign ED visits into medical urgency categories based on the presenting reason for visit was developed. A 3-step, look-back method was applied using an updated version of the New York University ED algorithm, and a map of all possible discharge diagnoses to the same reasons for visit was developed. Analyses were conducted in July and August 2023.

Main Outcomes and Measures

The main outcome was probabilistic medical urgency classification of reasons for visits and discharge diagnoses and their concordance.

Results

We analyzed 27 068 ED visits (mean age, 48.2% years [95% CI, 47.5%-48.9% years]) representing 190.7 million visits nationwide. Women (mean, 57.0% [95% CI, 55.9%-58.1%]) and patients with public health insurance coverage, including Medicare (mean, 24.9% [95% CI, 21.9%-28.0%]) and Medicaid (mean, 25.1% [95% CI, 21.0%-29.2%]), accounted for the largest share of ED visits, and a mean of 13.2% (95% CI, 11.4%-15.0%) of all visits resulted in a hospital admission. Overall, about 38.5% and 53.9% of all ED visits were classified with 100% and 75% probabilities, respectively, as injury related, emergency care needed, emergent but primary care treatable, nonemergent, or mental health or substance use disorders related based on discharge diagnosis compared with 0.4% and 12.4%, respectively, of all encounters based on patients' reason for visit. Among discharge diagnoses assigned with high certainty to only 1 urgency category using the New York University ED algorithm, between 38.0% (95% CI, 36.3%-39.6%) and 57.4% (95% CI, 56.0%-58.8%) aligned with the probabilistic categorical assignments of their corresponding reasons for visit.

Conclusions and Relevance

In this cross-sectional study of 190.7 million ED visits among adults aged 18 years or older, a smaller percentage of reasons for visit could be prospectively categorized with high accuracy to a specific medical urgency category compared with all visits based on discharge diagnoses, and a limited concordance between reasons for visit and discharge diagnoses was found. Alternative methods are needed to identify the medical necessity of ED encounters more accurately.

Strategies to Measure and Improve Emergency Department Performance: A Review

Emergency Departments (EDs) globally face escalating challenges such as overcrowding, resource limitations, and increased patient demand. This study aims to identify and analyze strategies to enhance the structural performance of EDs, with a focus on reducing overcrowding, optimizing resource allocation, and improving patient outcomes. Through a comprehensive review of the literature and observational studies, the research highlights the effectiveness of various approaches, including triage optimization, dynamic staffing, technological integration, and strategic resource management. Key findings indicate that tailored strategies, such as implementing advanced triage protocols and leveraging telemedicine, can significantly reduce wait times and enhance patient throughput. Furthermore, evidence suggests that dynamic staffing models and the integration of cutting-edge diagnostic tools contribute to operational efficiency and improved quality of care. These strategies, when combined, offer a multifaceted solution to the complex challenges faced by EDs, promising better patient care and satisfaction. The study underscores the need for a comprehensive approach, incorporating both organizational and technological innovations, to address the evolving needs of emergency healthcare.

Choice Architecture to Assist Clinicians with Appropriate COVID-19 Test Ordering

BACKGROUND

Despite improving supplies, SARS-CoV-2 nucleic acid amplification tests remain limited during surges and more so given concerns around COVID-19/influenza co-occurrence. Matching clinical guidelines to available supplies ensures resources remain available to meet clinical needs. We report a change in clinician practice after an electronic health record (EHR) order redesign to impact emergency department (ED) testing patterns.

METHODS

We included all ED visits between December 1, 2021 and January 18, 2022 across a hospital system to assess the impact of EHR order changes on provider behavior 3 weeks before and after the change. The EHR order redesign included embedded symptom-based order guidance. Primary outcomes were the proportion of COVID-19 + flu/respiratory syncytial virus (RSV) testing performed on symptomatic, admitted, and discharged patients, and the proportion of COVID-19 + flu testing on symptomatic, discharged patients.

RESULTS

A total of 52 215 ED visits were included. For symptomatic, discharged patients, COVID-19 + flu/RSV testing decreased from 11.4 to 5.8 tests per 100 symptomatic visits, and the rate of COVID-19 + flu testing increased from 7.4 to 19.1 before and after the intervention, respectively. The rate of COVID-19 + flu/RSV testing increased from 5.7 to 13.1 tests per 100 symptomatic visits for symptomatic patients admitted to the hospital. All changes were significant (P < 0.0001).

CONCLUSIONS

A simple EHR order redesign was associated with increased adherence to institutional guidelines for SARS-CoV-2 and influenza testing amidst supply chain limitations necessitating optimal allocation of scarce testing resources. With continually shifting resource availability, clinician education is not sufficient. Rather, system-based interventions embedded within exiting workflows can better align resources and serve testing needs of the community.

The Antibiotic Likelihood Index: A New Measure of Antimicrobial Stewardship for Pediatric Primary Care Based on Reason for Visit

BACKGROUND

Quality metrics for antibiotic prescribing by pediatricians are limited. We sought to define a novel measure that assesses clinicians' overall antibiotic prescribing.

METHODS

Using electronic health record (EHR) data from 2018 to 2019 for children 3 months to 17 years of age from 53 practices within a large pediatric network, we grouped encounters into Reason for Visit categories using the classification system of the National Ambulatory Medical Care Survey and analyzed the proportion of encounters with an antibiotic prescription. Categories were sorted according to the attributable proportion of encounters with an antibiotic prescribed. The proposed metric-the Antibiotic Likelihood Index (ALI)-was defined as the proportion of encounters with an antibiotic prescribed among categories that accounted for >80% of all encounters with an antibiotic prescribed. The ALI was calculated for the entire network and for individual prescribers, and the distribution among prescribers was described.

RESULTS

Six Reason for Visit categories-cough, ear complaints, fever, sore throat, rash, and congestion/upper respiratory infection-accounted for 82.4% of all antibiotics prescribed. Among the 222 682 encounters for the top 6 categories combined, 67 368 (30.3%) had an antibiotic prescribed, defined as the ALI for the entire sample. The index among individual prescribers ranged from 7.5% to 57.2% (interquartile range 24.3% to 34.9%). The correlation for individual prescribers between 2018 and 2019 was high (R2 = 0.80).

CONCLUSIONS

The ALI, a proposed new metric of pediatric antibiotic prescribing, can be readily calculated from EHR data and captures the range of antibiotic prescribing among pediatricians for common clinical scenarios.

Informing the Alfred Registry for Emergency Care Project: An analysis of presenting complaint documentation in an emergency department

Objective

To assess the feasibility of an ED presenting complaint (PC) tool that categorised all ED PCs into 10 categories.

Methods

A retrospective analysis of 1445 consecutive patient encounters was conducted. The primary outcome was the frequency of use of the 10 PC categories.

Results

Of the 1203 patient encounters meeting inclusion criteria, the PC tool was completed by clinicians in 574 (47.7%). When completed, the tool's 10 options were selected for most presentations (72.3%).

Conclusion

The PC tool captured the majority of presenting complaints in 10 categories. External validation is recommended.

Calibrating a chief complaint list for low resource settings: a methodologic case study

BACKGROUND

The chief or presenting complaint is the reason for seeking health care, often in the patient's own words. In limited resource settings, a diagnosis-based approach to quantifying burden of disease is not possible, partly due to limited availability of an established lexicon or coding system. Our group worked with colleagues from the African Federation of Emergency Medicine building on the existing literature to create a pilot symptom list representing an attempt to standardize undifferentiated chief complaints in emergency and acute care settings. An ideal list for any setting is one that strikes a balance between ease of use and length, while covering the vast majority of diseases with enough detail to permit epidemiologic surveillance and make informed decisions about resource needs.

METHODS

This study was incorporated as a part of a larger prospective observational study on human immunodeficiency virus testing in Emergency Departments in South Africa. The pilot symptom list was used for chief complaint coding in three Emergency Departments. Data was collected on 3357 patients using paper case report forms. Chief complaint terms were reviewed by two study team members to determine the frequency of concordance between the coded chief complaint term and the selected symptom(s) from the pilot symptom list.

RESULTS

Overall, 3537 patients' chief complaints were reviewed, of which 640 were identified as 'potential mismatches.' When considering the 191 confirmed mismatches (29.8%), the Delphi process identified 6 (3.1%) false mismatches and 185 (96.9%) true mismatches. Significant chief-complaint clustering was identified with 9 sets of complaints frequently selected together for the same patient. "Pain" was used 2076 times for 58.7% of all patients. A combination of user feedback and expert-panel modified Delphi analysis of mismatched complaints and clustered complaints resulted in several substantial changes to the pilot symptom list.

CONCLUSIONS

This study presented a systematic methodology for calibrating a chief complaint list for the local context. Our revised list removed/reworded symptoms that frequently clustered together or were misinterpreted by health professionals. Recommendations for additions, modifications, and/or deletions from the pilot chief complaint list we believe will improve the functionality of the list in low resource environments.

[AKTIN - The German Emergency Department Data Registry - real-time data from emergency medicine : Implementation and first results from 15 emergency departments with focus on Federal Joint Committee's guidelines on acuity assessment]

Hintergrund

Die Notfallversorgung befindet sich im Umbruch. In Notaufnahmen werden Patienten ausgehend von Symptomen und Dringlichkeit versorgt; dies bildet sich jedoch in den etablierten Routinedaten der gesetzlichen Sozialversicherung nicht ab. Ziel des AKTIN-Projekts war der Aufbau einer datenschutzkonformen Registerinfrastruktur zur Nutzung von klinischen Routinedaten aus Notaufnahmen.

Methoden

Über eine standardisierte Schnittstelle werden aus den verschiedenen Dokumentationssystemen kontinuierlich Daten der Notaufnahmebehandlung in ein lokales Data Warehouse exportiert. Dort stehen sie sowohl für lokale Nutzungen, wie interne Berichte und Qualitätsmanagement, als auch gleichzeitig datenschutzkonform für multizentrische Auswertungen zur Verfügung.

Anhand der Registerpopulation wird die Ersteinschätzung sowie die Erhebung von Vitalparametern in Abhängigkeit von Vorstellungsgründen für einen 12-Monats-Zeitraum analysiert.

Ergebnisse

Für den Zeitraum 04/2018 bis 03/2019 wurden 436.149 gültige Fälle aus 15 Notaufnahmen übermittelt. In 86,0 % der Fälle ist eine Ersteinschätzung dokumentiert.

Diese fand in 70,5 % innerhalb von 10 min nach Ankunft des Patienten statt. In 10 Kliniken wird regelhaft (82,3 %) ein Vorstellungsgrund erfasst. Die Erfassung von Vitalparametern variiert plausibel zwischen den Vorstellungsgründen.

Schlussfolgerung

Das AKTIN-Notaufnahmeregister bietet einen zeitnahen Einblick in das Versorgungsgeschehen der Notaufnahmen ohne zusätzlichen Dokumentationsaufwand und unabhängig vom primären IT-System, Kostenträger, Fallart und Abrechnungsmodus. Die Vorgaben des Gemeinsamen Bundesausschusses zur Ersteinschätzung werden weitgehend umgesetzt. Durch die Etablierung von standardisierten Vorstellungsgründen werden symptombasierte Analysen und Gesundheitssurveillance ermöglicht.

Zusatzmaterial online

Die Onlineversion dieses Beitrags (10.1007/s00063-020-00764-2) enthält die Abb. S1 und S2. Beitrag und Zusatzmaterial stehen Ihnen auf www.springermedizin.de zur Verfügung. Bitte geben Sie dort den Beitragstitel in die Suche ein, das Zusatzmaterial finden Sie beim Beitrag unter „Ergänzende Inhalte“.

Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study

INTRODUCTION

Quality of emergency department (ED) care affects patient outcomes substantially. Quality indicators (QIs) for ED care are a major challenge due to the heterogeneity of patient populations, health care structures and processes in Germany. Although a number of quality measures are already in use, there is a paucity of data on the importance of these QIs on medium-term and long-term outcomes. The evaluation of outcome relevance of quality indicators in the emergency department study (ENQuIRE) aims to identify and investigate the relevance of QIs in the ED on patient outcomes in a 12-month follow-up.

METHODS AND ANALYSIS

The study is a prospective non-interventional multicentre cohort study conducted in 15 EDs throughout Germany. Included are all patients in 2019, who were ≥18 years of age, insured at the Techniker Krankenkasse (statutory health insurance (SHI)) and gave their written informed consent to the study.The primary objective of the study is to assess the effect of selected quality measures on patient outcome. The data collected for this purpose comprise medical records from the ED treatment, discharge (claims) data from hospitalised patients, a patient questionnaire to be answered 6-8 weeks after emergency admission, and outcome measures in a 12-month follow-up obtained as claims data from the SHI.Descriptive and analytical statistics will be applied to provide summaries about the characteristics of QIs and associations between quality measures and patient outcomes.

ETHICS AND DISSEMINATION

Approval of the leading ethics committee at the Medical Faculty of the University of Magdeburg (reference number 163/18 from 19 November 2018) has been obtained and adapted by responsible local ethics committees.The findings of this work will be disseminated by publication of peer-reviewed manuscripts and presentations as conference contributions (abstracts, poster or oral presentations).Moreover, results will be discussed with clinical experts and medical associations before being proposed for implementation into the quality management of EDs.

TRIAL REGISTRATION NUMBER

German Clinical Trials Registry (DRKS00015203); Pre-results.

Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact

Introduction

For early detection of sepsis, automated systems within the electronic health record have evolved to alert emergency department (ED) personnel to the possibility of sepsis, and in some cases link them to suggested care pathways. We conducted a systematic review of automated sepsis-alert detection systems in the ED.

Methods

We searched multiple health literature databases from the earliest available dates to August 2018. Articles were screened based on abstract, again via manuscript, and further narrowed with set inclusion criteria: 1) adult patients in the ED diagnosed with sepsis, severe sepsis, or septic shock; 2) an electronic system that alerts a healthcare provider of sepsis in real or near-real time; and 3) measures of diagnostic accuracy or quality of sepsis alerts. The final, detailed review was guided by QUADAS-2 and GRADE criteria. We tracked all articles using an online tool (Covidence), and the review was registered with PROSPERO registry of reviews. A two-author consensus was reached at the article choice stage and final review stage. Due to the variation in alert criteria and methods of sepsis diagnosis confirmation, the data were not combined for meta-analysis.

Results

We screened 693 articles by title and abstract and 20 by full text; we then selected 10 for the study. The articles were published between 2009–2018. Two studies had algorithm-based alert systems, while eight had rule-based alert systems. All systems used different criteria based on systemic inflammatory response syndrome (SIRS) to define sepsis. Sensitivities ranged from 10–100%, specificities from 78–99%, and positive predictive value from 5.8–54%. Negative predictive value was consistently high at 99–100%. Studies showed some evidence for improved process-of-care markers, including improved time to antibiotics. Length of stay improved in two studies. One low quality study showed improved mortality.

Conclusion

The limited evidence available suggests that sepsis alerts in the ED setting can be set to high sensitivity. No high-quality studies showed a difference in mortality, but evidence exists for improvements in process of care. Significant further work is needed to understand the consequences of alert fatigue and sensitivity set points.

Generating contextual embeddings for emergency department chief complaints

Objective

We learn contextual embeddings for emergency department (ED) chief complaints using Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art language model, to derive a compact and computationally useful representation for free-text chief complaints.

Materials and methods

Retrospective data on 2.1 million adult and pediatric ED visits was obtained from a large healthcare system covering the period of March 2013 to July 2019. A total of 355 497 (16.4%) visits from 65 737 (8.9%) patients were removed for absence of either a structured or unstructured chief complaint. To ensure adequate training set size, chief complaint labels that comprised less than 0.01%, or 1 in 10 000, of all visits were excluded. The cutoff threshold was incremented on a log scale to create seven datasets of decreasing sparsity. The classification task was to predict the provider-assigned label from the free-text chief complaint using BERT, with Long Short-Term Memory (LSTM) and Embeddings from Language Models (ELMo) as baselines. Performance was measured as the Top-k accuracy from k = 1:5 on a hold-out test set comprising 5% of the samples. The embedding for each free-text chief complaint was extracted as the final 768-dimensional layer of the BERT model and visualized using t-distributed stochastic neighbor embedding (t-SNE).

Results

The models achieved increasing performance with datasets of decreasing sparsity, with BERT outperforming both LSTM and ELMo. The BERT model yielded Top-1 accuracies of 0.65 and 0.69, Top-3 accuracies of 0.87 and 0.90, and Top-5 accuracies of 0.92 and 0.94 on datasets comprised of 434 and 188 labels, respectively. Visualization using t-SNE mapped the learned embeddings in a clinically meaningful way, with related concepts embedded close to each other and broader types of chief complaints clustered together.

Discussion

Despite the inherent noise in the chief complaint label space, the model was able to learn a rich representation of chief complaints and generate reasonable predictions of their labels. The learned embeddings accurately predict provider-assigned chief complaint labels and map semantically similar chief complaints to nearby points in vector space.

Conclusion

Such a model may be used to automatically map free-text chief complaints to structured fields and to assist the development of a standardized, data-driven ontology of chief complaints for healthcare institutions.

Quality Assurance in the Emergency Department

Quality assurance (QA) of care in the emergency department encompasses activities ensuring that the care provided meets applicable standards. Health care delivery is complex and many factors affect quality of care. Thus, quantification of health care quality is challenging, especially with regard to attribution of outcomes to various factors contributing to such care. A critical component of the process of QA is determination of quality health care and the concept of (unjustified) deviation from the reference applicable standard of care.

Identifying patient-centered outcomes for children with cancer and their caregivers when they seek care in the emergency department

BACKGROUND

Children with cancer have high utilization of the emergency department (ED), but little is known about which outcomes are most important to them and their caregivers when they seek care in the ED.

PROCEDURE

A qualitative evaluation of ED experience for children with cancer and their caregivers was performed using self-reported interactive toolkits. Eligible participants included children with cancer (ages 11-19) and caregivers of children with cancer whose child received cancer therapy within the last year and had an ED visit within the last 2 years. Eligible participants received toolkits by mail and received incentives if they completed the toolkit. Toolkits were transcribed, thematically coded, and iteratively analyzed using Nvivo 11.0 software.

RESULTS

There were 26 toolkits received-seven by children aged 11-17 years and 19 by caregivers (11 with children aged 2-7 years, eight with children aged 11-17 years). About half were from within 1 h of their treating institution. The most important outcomes to this population included system-level issues (eg, cleanliness of space, timeliness of evaluation) and oncology-provider- and ED-provider-level issues (eg, ability to access port-a-caths, quality of communication). Participants also identified outcomes that were within the control of the patient/caregiver, such as improving their sense of preparedness.

CONCLUSION

The important outcomes to children with cancer and their caregivers when they seek care in the ED are distinct from current quality metrics. Future research should focus on the development and validation of a patient-centered outcomes tool.

Development and validation of a pragmatic natural language processing approach to identifying falls in older adults in the emergency department

BACKGROUND

Falls among older adults are both a common reason for presentation to the emergency department, and a major source of morbidity and mortality. It is critical to identify fall patients quickly and reliably during, and immediately after, emergency department encounters in order to deliver appropriate care and referrals. Unfortunately, falls are difficult to identify without manual chart review, a time intensive process infeasible for many applications including surveillance and quality reporting. Here we describe a pragmatic NLP approach to automating fall identification.

METHODS

In this single center retrospective review, 500 emergency department provider notes from older adult patients (age 65 and older) were randomly selected for analysis. A simple, rules-based NLP algorithm for fall identification was developed and evaluated on a development set of 1084 notes, then compared with identification by consensus of trained abstractors blinded to NLP results.

RESULTS

The NLP pipeline demonstrated a recall (sensitivity) of 95.8%, specificity of 97.4%, precision of 92.0%, and F1 score of 0.939 for identifying fall events within emergency physician visit notes, as compared to gold standard manual abstraction by human coders.

CONCLUSIONS

Our pragmatic NLP algorithm was able to identify falls in ED notes with excellent precision and recall, comparable to that of more labor-intensive manual abstraction. This finding offers promise not just for improving research methods, but as a potential for identifying patients for targeted interventions, quality measure development and epidemiologic surveillance.

Consensus Development of a Modern Ontology of Emergency Department Presenting Problems-The Hierarchical Presenting Problem Ontology (HaPPy)

OBJECTIVE

Numerous attempts have been made to create a standardized "presenting problem" or "chief complaint" list to characterize the nature of an emergency department visit. Previous attempts have failed to gain widespread adoption as they were not freely shareable or did not contain the right level of specificity, structure, and clinical relevance to gain acceptance by the larger emergency medicine community. Using real-world data, we constructed a presenting problem list that addresses these challenges.

MATERIALS AND METHODS

We prospectively captured the presenting problems for 180,424 consecutive emergency department patient visits at an urban, academic, Level I trauma center in the Boston metro area. No patients were excluded. We used a consensus process to iteratively derive our system using real-world data. We used the first 70% of consecutive visits to derive our ontology, followed by a 6-month washout period, and the remaining 30% for validation. All concepts were mapped to Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT).

RESULTS

Our system consists of a polyhierarchical ontology containing 692 unique concepts, 2,118 synonyms, and 30,613 nonvisible descriptions to correct misspellings and nonstandard terminology. Our ontology successfully captured structured data for 95.9% of visits in our validation data set.

DISCUSSION AND CONCLUSION

We present the HierArchical Presenting Problem ontologY (HaPPy). This ontology was empirically derived and then iteratively validated by an expert consensus panel. HaPPy contains 692 presenting problem concepts, each concept being mapped to SNOMED CT. This freely sharable ontology can help to facilitate presenting problem-based quality metrics, research, and patient care.

Potential Effect of the Protecting Access to Medicare Act on Use of Advanced Diagnostic Imaging in the Emergency Department: An Analysis of the National Hospital Ambulatory Care Survey

Background Clinical decision support is increasingly used to enhance clinicians' exposure to established evidence and patient information during an episode of patient care. Pending legislation specifies clinical decision support before performing advanced imaging at emergency department (ED) visits. Purpose To estimate the volume of advanced imaging tests (CT and MRI) that would require use of clinical decision support to achieve Protecting Access to Medicare Act (PAMA) compliance in the ED. Materials and Methods A retrospective, cross-sectional analysis of ED visits was conducted by using data from the 2012-2015 National Hospital Ambulatory Care Survey. PAMA-related visits were identified by selecting the patient reasons for visit (RFVs) related to the eight clinical conditions. Results Among the adult ED visits, 26.7% (20 506 of 77 299, representing 113 000 000 visits across 4 years, or 28 000 000 visits annually) patients presented with a RFV consistent with a PAMA priority clinical area (PCA). Among visits in which a patient described an RFV code consistent with a PAMA PCA, up to 22.9% (4681 of 20 506; 95% confidence interval: 21.8%, 24.1%) patients underwent advanced imaging, translating to approximately 6 000 000 visits annually. Conclusion Protecting Access to Medicare Act legislation targets eight priority clinical areas, estimated to be prevalent among one in four adult emergency department visits. CT and/or MRI studies are performed during up to 23% of these visits. Depending on the particular clinical decision support systems selected within a health system, and how they are implemented, the potential volume of studies in which clinicians must interact with clinical decision support system may either exceed or fall short of these estimates. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Forman in this issue.

Priorities to Overcome Barriers Impacting Data Science Application in Emergency Care Research

For a variety of reasons including cheap computing, widespread adoption of electronic medical records, digitalization of imaging and biosignals, and rapid development of novel technologies, the amount of health care data being collected, recorded, and stored is increasing at an exponential rate. Yet despite these advances, methods for the valid, efficient, and ethical utilization of these data remain underdeveloped. Emergency care research, in particular, poses several unique challenges in this rapidly evolving field. A group of content experts was recently convened to identify research priorities related to barriers to the application of data science to emergency care research. These recommendations included: 1) developing methods for cross-platform identification and linkage of patients; 2) creating central, deidentified, open-access databases; 3) improving methodologies for visualization and analysis of intensively sampled data; 4) developing methods to identify and standardize electronic medical record data quality; 5) improving and utilizing natural language processing; 6) developing and utilizing syndrome or complaint-based based taxonomies of disease; 7) developing practical and ethical framework to leverage electronic systems for controlled trials; 8) exploring technologies to help enable clinical trials in the emergency setting; and 9) training emergency care clinicians in data science and data scientists in emergency care medicine. The background, rationale, and conclusions of these recommendations are included in the present article.

Lessons learned in applying the International Society for Pharmacoeconomics and Outcomes Research methodology to translating Canadian Emergency Department Information System Presenting Complaints List into German

OBJECTIVES

The patient's presenting complaint guides diagnosis and treatment in the emergency department, but there is no classification system available in German. The Canadian Emergency Department Information System (CEDIS) Presenting Complaint List (PCL) is available only in English and French. As translation risks the altering of meaning, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has set guidelines to ensure translational accuracy. The aim of this paper is to describe our experiences of using the ISPOR guidelines to translate the CEDIS PCL into German.

MATERIALS AND METHODS

The CEDIS PCL (version 3.0) was forward-translated and back-translated in accordance with the ISPOR guidelines using bilingual clinicians/translators and an occupationally mixed evaluation group that completed a self-developed questionnaire.

RESULTS

The CEDIS PCL was forward-translated (four emergency physicians) and back-translated (three mixed translators). Back-translation uncovered eight PCL items requiring amendment. In total, 156 comments were received from 32 evaluators, six of which resulted in amendments.

CONCLUSION

The ISPOR guidelines facilitated adaptation of a PCL into German, but the process required time, language skills and clinical knowledge. The current methodology may be applicable to translating the CEDIS PCL into other languages, with the aim of developing a harmonized, multilingual PCL.

Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: The importance of data quality assessment

BACKGROUND

Big data analytics promise insights into healthcare processes and management, improving outcomes while reducing costs. However, data quality is a major challenge for reliable results. Business process discovery techniques and an associated data model were used to develop data management tool, ICU-DaMa, for extracting variables essential for overseeing the quality of care in the intensive care unit (ICU).

OBJECTIVE

To determine the feasibility of using ICU-DaMa to automatically extract variables for the minimum dataset and ICU quality indicators from the clinical information system (CIS).

METHODS

The Wilcoxon signed-rank test and Fisher's exact test were used to compare the values extracted from the CIS with ICU-DaMa for 25 variables from all patients attended in a polyvalent ICU during a two-month period against the gold standard of values manually extracted by two trained physicians. Discrepancies with the gold standard were classified into plausibility, conformance, and completeness errors.

RESULTS

Data from 149 patients were included. Although there were no significant differences between the automatic method and the manual method, we detected differences in values for five variables, including one plausibility error and two conformance and completeness errors. Plausibility: 1) Sex, ICU-DaMa incorrectly classified one male patient as female (error generated by the Hospital's Admissions Department). Conformance: 2) Reason for isolation, ICU-DaMa failed to detect a human error in which a professional misclassified a patient's isolation. 3) Brain death, ICU-DaMa failed to detect another human error in which a professional likely entered two mutually exclusive values related to the death of the patient (brain death and controlled donation after circulatory death). Completeness: 4) Destination at ICU discharge, ICU-DaMa incorrectly classified two patients due to a professional failing to fill out the patient discharge form when thepatients died. 5) Length of continuous renal replacement therapy, data were missing for one patient because the CRRT device was not connected to the CIS.

CONCLUSIONS

Automatic generation of minimum dataset and ICU quality indicators using ICU-DaMa is feasible. The discrepancies were identified and can be corrected by improving CIS ergonomics, training healthcare professionals in the culture of the quality of information, and using tools for detecting and correcting data errors.

Using Chief Complaint in Addition to Diagnosis Codes to Identify Falls in the Emergency Department

OBJECTIVES

To compare incidence of falls in an emergency department (ED) cohort using a traditional International Classification of Diseases, Ninth Revision (ICD-9) code-based scheme and an expanded definition that included chief complaint information and to examine the clinical characteristics of visits "missed" in the ICD-9-based scheme.

DESIGN

Retrospective electronic record review.

SETTING

Academic medical center ED.

PARTICIPANTS

Individuals aged 65 and older seen in the ED between January 1, 2013, and September 30, 2015.

MEASUREMENTS

Two fall definitions were applied (individually and together) to the cohort: an ICD-9-based definition and a chief complaint definition. Admission rates and 30-day mortality (per encounter) were measured for each definition.

RESULTS

Twenty-three thousand eight hundred eighty older adult visits occurred during the study period. Using the most-inclusive definition (ICD-9 code or chief complaint indicating a fall), 4,363 visits (18%) were fall related. Of these visits, 3,506 (80%) met the ICD-9 definition for a fall-related visit, and 2,664 (61%) met the chief complaint definition. Of visits meeting the chief complaint definition, 857 (19.6%) were missed when applying the ICD-9 definition alone. Encounters missed using the ICD-9 definition were less likely to lead to an admission (42.9%, 95% confidence interval (CI) = 39.7-46.3%) than those identified (54.4%, 95% CI = 52.7-56.0%).

CONCLUSION

Identifying individuals in the ED who have fallen based on diagnosis codes underestimates the true burden of falls. Individuals missed according to the code-based definition were less likely to have been admitted than those who were captured. These findings call attention to the value of using chief complaint information to identify individuals who have fallen in the ED-for research, clinical care, or policy reasons.

Patients' Perspectives on Outcomes of Care After Discharge From the Emergency Department: A Qualitative Study

STUDY OBJECTIVE

Much effort has been expended to understand what care experiences patients value in the emergency department (ED), yet little is known about which outcomes patients value after ED care. Our goal is to define outcomes of ED care that are valued by patients discharged from the ED, with the goal of informing the development of a patient-reported outcome measure for ED care.

METHODS

We conducted qualitative semistructured interviews with patients recruited during their care at 1 of 2 EDs and interviewed in either English or French 1 to 9 days after their visit. Patients who were hospitalized were excluded. Interviews focused on perceived outcomes of care since the ED visit and expectations of care before the ED visit. We identified themes with standard descriptive content analysis techniques and a modified version of the constant comparative method, drawing on grounded theory methods.

RESULTS

We interviewed 46 patients in English (n=38) or French (n=8). Participants with diverse reasons for seeking care appeared to value common outcomes from ED care that centered around 4 themes: understanding the cause and expected trajectory of their symptoms; reassurance; symptom relief; and having a plan to manage their symptoms, resolve their issue, or pursue further medical care. These themes were also reflected in the expectations participants recalled having when they decided to seek care in the ED.

CONCLUSION

The 4 outcomes defined constitute areas for improvement and will inform the development of an ED patient-reported outcome questionnaire. Consideration should be given to measuring patient-reported outcomes separately from patient experience.

Development of a Patient-centered Outcome Measure for Emergency Department Asthma Patients

BACKGROUND

Measuring outcomes of emergency care is of key importance, but current metrics, such as 72-hour return visit rates, are subject to ascertainment bias, incentivize overtesting and overtreatment at initial visit, and do not reflect the full burden of disease and morbidity experienced at home following ED care. There is increasing emphasis on including patient-reported outcomes, but the existing patient-reported measures have limited applicability to emergency care.

OBJECTIVE

The objective was to identify concepts for inclusion in a patient-reported outcome measure for ED care and assess differences in potential concepts by health literacy.

METHODS

A three-phase qualitative study was completed using freelisting and semistructured interviewing for concept identification, member checking for concept ranking, and cognitive interviewing for question development. Participants were drawn from three tertiary care EDs. Parents of patients (pediatric) or patients (adult) with asthma completed a demographic survey and an assessment of health literacy. Phase 1 participants also completed a freelisting exercise and qualitative interview regarding the definition of success following ED discharge. Phase 2 participants completed a member checking survey based on concepts identified in Phase 1. Phase 3 was a pilot of trial questions based on the highest-ranked concepts from Phase 2.

RESULTS

Phase 1 enrolled 22 adult patients and 37 parents of pediatric patients. Phase 2 enrolled 41 adult patients and 200 parents. Phase 3 involved 15 parents. Across all demographic/literacy groups, Phase 1 participants reported return to usual activity and lack of asthma symptoms as the most important markers of success. In Phase 2, symptom improvement, medication use and access, and asthma knowledge were identified as the most important components of the definition of post-ED discharge success. Phase 3 resulted in five questions for the proposed measure.

CONCLUSIONS

A stepwise qualitative process can identify, rank, and formulate questions based on patient-identified concepts for inclusion in a patient-reported outcome measure for ED discharge. The four key concepts identified for inclusion: symptom improvement, medication access, correct medication use, and asthma knowledge are not measured by existing quality metrics.

[Standardized collection of presenting complaints in the emergency room : Integration of coded presenting complaints into the electronic medical record system of an emergency department and their value for health care research]

BACKGROUND

The point of entry of a patient in emergency care is a symptom or a complaint. To evaluate subsequent processes in an emergency department until a diagnosis is made, this information has to be taken into account.

OBJECTIVES

We report the introduction of coded presenting complaints into the electronic medical record system of an emergency department and describe the patients based on these data.

METHODS

The CEDIS presenting complaint list was integrated into the emergency department information system of an emergency department (38,000 patients/year). After 8 months, we performed an exploratory analysis of the most common presenting complaints. Furthermore, we identified the most frequent diagnoses for presenting complaint "shortness of breath" and the most frequent presenting complaints for the diagnosis of sepsis.

RESULTS

After implementing the presenting complaint list, a presenting complaint code was assigned to each patient. In our sample (26,330 cases), "extremity pain and injury" comprised the largest group of patients (29.5%). "Chest pain-cardiac features" (3.7%) and "extremity weakness/symptoms of cerebrovascular accident" (2.4%) were the main cardiac and neurologic complaints, respectively. They were mostly triaged as urgent (>80%) and hospitalized in critical care units (>50%). The main diagnosis for presenting complaint "shortness of breath" was heart failure (25.1%), while the main presenting complaint for the diagnosis sepsis was "shortness of breath" (18.1%).

CONCLUSIONS

Containing 171 presenting complaints, this classification was implemented successfully without providing extensive staff training. The documentation of coded presenting complaints enables symptom-based analysis of the health care provided in emergency departments.

Value-Based Approaches for Emergency Care in a New Era

Although emergency departments (EDs) play an integral role in the delivery of acute unscheduled care, they have not been fully integrated into broader health care reform efforts. Communication and coordination with the ambulatory environment remain limited, leaving ED care disconnected from patients' longitudinal care. In a value-based environment focused on improving quality, decreasing costs, enhancing population health, and improving the patient experience, this oversight represents a missed opportunity for emergency care. When integrated with primary and subspecialty care, emergency care might meet the needs of patients, providers, and payers more efficiently than yet realized. This article uses the Merit-Based Incentive Payment System from the Medicare Access and CHIP Reauthorization Act as a framework to outline a strategy for improving the value of emergency care, including integrating quality and resource use measures across health care delivery settings and populations, encouraging care coordination from the ED, and implementing robust health information exchange systems.

Automatic quality improvement reports in the intensive care unit: One step closer toward meaningful use

AIM: To examine the feasibility and validity of electronic generation of quality metrics in the intensive care unit (ICU).

METHODS: This minimal risk observational study was performed at an academic tertiary hospital. The Critical Care Independent Multidisciplinary Program at Mayo Clinic identified and defined 11 key quality metrics. These metrics were automatically calculated using ICU DataMart, a near-real time copy of all ICU electronic medical record (EMR) data. The automatic report was compared with data from a comprehensive EMR review by a trained investigator. Data was collected for 93 randomly selected patients admitted to the ICU during April 2012 (10% of admitted adult population). This study was approved by the Mayo Clinic Institution Review Board.

RESULTS: All types of variables needed for metric calculations were found to be available for manual and electronic abstraction, except information for availability of free beds for patient-specific time-frames. There was 100% agreement between electronic and manual data abstraction for ICU admission source, admission service, and discharge disposition. The agreement between electronic and manual data abstraction of the time of ICU admission and discharge were 99% and 89%. The time of hospital admission and discharge were similar for both the electronically and manually abstracted datasets. The specificity of the electronically-generated report was 93% and 94% for invasive and non-invasive ventilation use in the ICU. One false-positive result for each type of ventilation was present. The specificity for ICU and in-hospital mortality was 100%. Sensitivity was 100% for all metrics.

CONCLUSION: Our study demonstrates excellent accuracy of electronically-generated key ICU quality metrics. This validates the feasibility of automatic metric generation.

Policy Measures and Reimbursement for Emergency Medical Imaging in the Era of Payment Reform: Proceedings From a Panel Discussion of the 2015 Academic Emergency Medicine Consensus Conference

The Affordable Care Act (ACA) of 2010 is expanding the use of quality measurement and promulgating new payment models that place downward pressure on health care utilization and costs. As emergency department (ED) computed tomography utilization has tripled in the past decade, stakeholders have identified advanced imaging as an area where quality and efficiency measures should expand. On May 12, 2015, Academic Emergency Medicine convened a consensus conference titled "Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization." As part of the conference, a panel of health care policy leaders and emergency physicians discussed the effect of the ACA and other quality programs on ED diagnostic imaging, specifically the way that quality metrics may affect ED care and how ED diagnostic imaging fits in the broader strategy of the U.S. government. This article discusses the content of the panel's presentations.

Emergency Department Diagnostic Imaging: The Journey to Quality

Priorities in health care delivery are shifting, with a greater focus on enhancing value, incentivizing quality, and advancing population health. While measurement of quality in emergency department (ED) care is still in its infancy, performance measures are increasingly being linked to reimbursement to encourage the delivery of high-value care. With such changes, there will be growing oversight of diagnostic imaging in all clinical settings, including the ED. Here, the authors examine the current state of quality measurement as it pertains to ED imaging. The authors review relevant policies and discuss both the associated challenges and the facilitators of using quality measures to help optimize ED imaging. Understanding such factors will help ensure the delivery of diagnostic imaging that is appropriate, high-quality, and patient-centered.

A National Evaluation of the Scholarly Activity Requirement in Residency Programs: A Survey of Emergency Medicine Program Directors

OBJECTIVES

The Review Committee for Emergency Medicine (RC-EM) requirement for scholarly activity, which programs may define as an original research project or some other form of scholarly activity, applies to all EM residents. The objectives of this study were to: 1) describe the percentage of residency programs that require an original research project to meet the RC-EM requirement for scholarly activity, 2) describe specific challenges and resources for residents completing the RC-EM scholarly activity requirement, and 3) identify associations between the interpretation of the requirement and early career outcomes.

METHODS

This was a cross-sectional online survey of program or research directors from all U.S. allopathic EM residency programs. Respondents were queried about key demographics and domains relating to research curriculum, resources, expectations, outcomes, challenges, and future opportunities. Data were analyzed using descriptive statistics.

RESULTS

The overall response rate was 113 of 156 (72%) EM residency programs. Respondents were more likely to represent university-based programs, but otherwise did not differ from nonrespondents across key demographic criteria. An original research project was required by 39% of responding programs, with a minimum deliverable in 93% of these programs. Program directors listed data collection and study design as the principle challenges residents face while completing their scholarly activities. Faculty mentorship, biostatistical support, and travel support were common resources reportedly available to residents. Comparison of programs with an original research requirement to those without revealed many differences in outcomes. Programs with a research requirement were more likely to have residents with oral or poster presentations (46% vs. 25%, mean difference = 21%, 95% confidence interval [CI] = 16% to 28%), published manuscripts (25% vs. 18%, mean difference = 7%, 95% CI = 2% to 10%), entering fellowship training after residency (27% vs. 20%, mean difference = 7%, 95% CI = 4% to 10%), and using a biostatistician (64% vs. 28%, median difference = 26%, 95% CI = 24% to 28%). There were no statistically significant differences in other evaluations of resources or outcome measures, including resident choice of academic career after leaving residency.

CONCLUSIONS

There is no consistent interpretation and implementation of the RC-EM requirement for scholarly activity among EM residency programs. Residency programs requiring an original research project were more likely to have residents with accepted oral or poster presentations, published manuscripts, and entering fellowships after residency training.

[Quality management in emergency departments: Lack of uniform standards for fact-based controlling]

BACKGROUND

The general high occupancy of emergency departments during the winter months of 2014/2015 outlined deficits in health politics. Whether on the regional, province, or federal level, verifiable and accepted figures to enable in depth analysis and fact-based controlling of emergency care systems are lacking.

OBJECTS

As the first step, reasons for the current situation are outlined in order to developed concrete recommendations for individual hospitals.

METHODS

This work is based on a selective literature search with focus on quality management, ratio driven management, and process management within emergency departments as well as personal experience with implementation of a key ratio system in a German maximum care hospital.

RESULTS AND CONCLUSION

The insufficient integration of emergencies into the DRG systematic, the role as gatekeeper between inpatient and outpatient care sector, the decentralized organization of emergency departments in many hospitals, and the inconsistent representation within the medical societies can be mentioned as reasons for the lack of key ratio systems. In addition to the important role within treatment procedures, emergency departments also have an immense economic importance. Consequently, the management of individual hospitals should promote implementation of key ratio systems to enable controlling of emergency care processes. Thereby the perspectives finance, employees, processes as well as partners and patients should be equally considered. Within the process perspective, milestones could be used to enable detailed controlling of treatment procedures. An implementation of key ratio systems without IT support is not feasible; thus, existing digital data should be used and future data analysis should already be considered during implementation of new IT systems.

Abdominal Computed Tomography Utilization and 30-day Revisitation in Emergency Department Patients Presenting With Abdominal Pain

OBJECTIVES

The objective was to explore which patient characteristics are associated with repeat emergency department (ED) visitation within 30 days of ED discharge for patients presenting with abdominal pain.

METHODS

A retrospective, observational study was conducted at a single, academic, urban ED with over 85,000 annual visits. A consecutive sample of adult patients with a chief complaint of abdominal pain from January 2010 through December 2010 who were discharged following ED evaluation were included in the analysis. A logistic regression model was used to determine which patient-level factors, including computed tomography (CT) utilization, were associated with the primary outcome of ED revisit within 30 days.

RESULTS

Of 80,619 total ED patient visits during the study period, 3,928 ED discharges with a chief complaint of abdominal pain were included. A total of 487 (12.4%) patients revisited the ED within 30 days. No deaths were recorded. CT imaging was associated with a lower 30-day revisit rate (odds ratio [OR] = 0.69, 95% confidence interval [CI] = 0.55 to 0.87) after controlling for multiple other patient-level factors associated with revisits. Increasing age (OR = 1.01, 95% CI = 1.00 to 1.02), increasing triage pain scores (OR = 1.13, 95% CI = 1.08 to 1.18), elevated triage heart rate (OR = 1.42, 95% CI = 1.07 to 1.89), low sodium levels (OR = 1.56, 95% CI = 1.07 to 2.23), and anemia (OR = 1.42, 95% CI = 1.04 to 1.95) were all associated with increased rate of return.

CONCLUSIONS

Performance of an abdominal CT was associated with fewer 30-day revisits, suggesting that future measures of "imaging appropriateness" and "ED overuse" consider downstream utilization of health care resources in addition to the index visit.