Can a 5-to-90-day Mortality Predictor Perform Consistently Across Time and Equitably Across Populations?

Advance care planning (ACP) facilitates end-of-life care, yet many die without it. Timely and accurate mortality prediction may encourage ACP. However, performance of predictors typically differs among sub-populations (e.g., rural vs. urban) and worsens over time ("concept drift"). Therefore, we assessed performance equity and consistency for a novel 5-to-90-day mortality predictor across various demographies, geographies, and timeframes (n = 76,812 total encounters). Predictions were made for the first day of included adult inpatient admissions on a retrospective dataset. AUC-PR remained at 29% both pre-COVID (throughout 2018) and during COVID (8 months in 2021). Pre-COVID-19 recall and precision were 58% and 25% respectively at the 12.5% certainty cutoff, and 12% and 44% at the 37.5% cutoff. During COVID-19, recall and precision were 59% and 26% at the 12.5% cutoff, and 11% and 43% at the 37.5% cutoff. Pre-COVID, compared to the overall population, recall was lower at the 12.5% cutoff in the White, non-Hispanic subgroup and at both cutoffs in the rural subgroup. During COVID-19, precision at the 12.5% cutoff was lower than that of the overall population for the non-White and non-White female subgroups. No other significant differences were seen between subgroups and the corresponding overall population. Overall performance during COVID was unchanged from pre-pandemic performance. Although some comparisons (especially precision at the 37.5% cutoff) were underpowered, precision at the 12.5% cutoff was equitable across most demographies, regardless of the pandemic. Mortality prediction to prioritize ACP conversations can be provided consistently and equitably across many studied timeframes and sub-populations.

Communicating Diagnostic Uncertainty at Emergency Department Discharge: A Simulation-Based Mastery Learning Randomized Trial

PURPOSE

There are no standardized approaches for communicating with patients discharged from the emergency department with diagnostic uncertainty. This trial tested efficacy of the Uncertainty Communication Education Module, a simulation-based mastery learning curriculum designed to establish competency in communicating diagnostic uncertainty.

METHOD

Resident physicians at 2 sites participated in a 2-arm waitlist randomized controlled trial from September 2019 to June 2020. After baseline (T1) assessment of all participants via a standardized patient encounter using the Uncertainty Communication Checklist (UCC), immediate access physicians received training in the Uncertainty Communication Education Module, which included immediate feedback, online educational modules, a smartphone-based application, and telehealth deliberate practice with standardized patients. All physicians were retested 16-19 weeks later (T2) via in-person standardized patient encounters; delayed access physicians then received the intervention. A final test of all physicians occurred 11-15 weeks after T2 (T3). The primary outcome measured the percentage of physicians in the immediate versus delayed access groups meeting or exceeding the UCC minimum passing standard at T2.

RESULTS

Overall, 109 physicians were randomized, with mean age 29 years (range 25-46). The majority were male (n = 69, 63%), non-Hispanic/Latino (n = 99, 91%), and White (n = 78, 72%). At T2, when only immediate access participants had received the curriculum, immediate access physicians demonstrated increased mastery (n = 29, 52.7%) compared with delayed access physicians (n = 2, 3.7%, P < .001; estimated adjusted odds ratio of mastery for the immediate access participants, 31.1 [95% CI, 6.8-143.1]). There were no significant differences when adjusting for training site or stage of training.

CONCLUSIONS

The Uncertainty Communication Education Module significantly increased mastery in communicating diagnostic uncertainty at the first postintervention test among emergency physicians in standardized patient encounters. Further work should assess the impact of clinical implementation of these communication skills.

Using second life virtual simulation environment for mock oral emergency medicine examination

OBJECTIVES

Oral examination is a method used to evaluate emergency medicine (EM) residents and is a requirement for board certification of emergency physicians. Second Life (SL) is a virtual three-dimensional (3-D) immersive learning environment that has been used for medical education. In this study we explore the use of SL virtual simulation technology to administer mock oral examinations to EM residents.

METHODS

This was a prospective observational study of EM residents who had previously completed mock oral examinations, participating in a similar mock oral examination case scenario conducted via SL. EM residents in this training program completed mock oral examinations in a traditional format, conducted face to face with a faculty examiner. All current residents were invited to participate in a similar case scenario conducted via SL for this study. The examinee managed the case while acting as the physician avatar and communicated via headset and microphone from a remote computer with a faculty examiner who acted as the patient avatar. Participants were surveyed regarding their experience with the traditional and virtual formats using a Likert scale.

RESULTS

Twenty-seven EM residents participated in the virtual oral examination. None of the examinees had used SL previously. SL proved easy for examinees to log into (92.6%) and navigate (96.3%). All felt comfortable communicating with the examiner via remote computer. Most examinees thought the SL encounter was realistic (92.6%), and many found it more realistic than the traditional format (70.3%). All examinees felt that the virtual examination was fair, objective, and conducted efficiently. A majority preferred to take oral examinations via SL over the traditional format and expressed interest in using SL for other educational experiences (66.6 and 92.6%, respectively).

CONCLUSIONS

Application of SL virtual simulation technology is a potential alternative to traditional mock oral examinations for EM residents.

Defining systems expertise: effective simulation at the organizational level--implications for patient safety, disaster surge capacity, and facilitating the systems interface

The Institute of Medicine's report "To Err is Human" identified simulation as a means to enhance safety in the medical field, just as flight simulation is used to improve the aviation industry. Yet, while there is evidence that simulation may improve task performance, there is little evidence that simulation actually improves patient outcome. Similarly, simulation is currently used to model teamwork-communication skills for disaster management and critical events, but little research or evidence exists to show that simulation improves disaster response or facilitates intersystem or interagency communication. Simulation ranges from the use of standardized patient encounters to robot-mannequins to computerized virtual environments. As such, the field of simulation covers a broad range of interactions, from patient-physician encounters to that of the interfaces between larger systems and agencies. As part of the 2008 Academic Emergency Medicine Consensus Conference on the Science of Simulation, our group sought to identify key research questions that would inform our understanding of simulation's impact at the organizational level. We combined an online discussion group of emergency physicians, an extensive review of the literature, and a "public hearing" of the questions at the Consensus Conference to establish recommendations. The authors identified the following six research questions: 1) what objective methods and measures may be used to demonstrate that simulator training actually improves patient safety? 2) How can we effectively feedback information from error reporting systems into simulation training and thereby improve patient safety? 3) How can simulator training be used to identify disaster risk and improve disaster response? 4) How can simulation be used to assess and enhance hospital surge capacity? 5) What methods and outcome measures should be used to demonstrate that teamwork simulation training improves disaster response? and 6) How can the interface of systems be simulated? We believe that exploring these key research questions will improve our understanding of how simulation affects patient safety, disaster surge capacity, and intersystem and interagency communication.

National growth in simulation training within emergency medicine residency programs, 2003-2008

OBJECTIVES

The use of medical simulation has grown dramatically over the past decade, yet national data on the prevalence and growth of use among individual specialty training programs are lacking. The objectives of this study were to describe the current role of simulation training in emergency medicine (EM) residency programs and to quantify growth in use of the technology over the past 5 years.

METHODS

In follow-up of a 2006 study (2003 data), the authors distributed an updated survey to program directors (PDs) of all 179 EM residency programs operating in early 2008 (140 Accreditation Council on Graduate Medical Education [ACGME]-approved allopathic programs and 39 American Osteopathic Association [AOA]-accredited osteopathic programs). The brief survey borrowed from the prior instrument, was edited and revised, and then distributed at a national PDs meeting. Subsequent follow-up was conducted by e-mail and telephone. The survey concentrated on technology-enhanced simulation modalities beyond routine static trainers or standardized patient-actors (high-fidelity mannequin simulation, part-task/procedural simulation, and dynamic screen-based simulation).

RESULTS

A total of 134 EM residency programs completed the updated survey, yielding an overall response rate of 75%. A total of 122 (91%) use some form of simulation in their residency training. One-hundred fourteen (85%) specifically use mannequin-simulators, compared to 33 (29%) in 2003 (p < 0.001). Mannequin-simulators are now owned by 58 (43%) of the programs, whereas only 9 (8%) had primary responsibility for such equipment in 2003 (p < 0.001). Fifty-eight (43%) of the programs reported that annual resident simulation use now averages more than 10 hours per year.

CONCLUSIONS

Use of medical simulation has grown significantly in EM residency programs in the past 5 years and is now widespread among training programs across the country.

Simulation in graduate medical education 2008: a review for emergency medicine

Health care simulation includes a variety of educational techniques used to complement actual patient experiences with realistic yet artificial exercises. This field is rapidly growing and is widely used in emergency medicine (EM) graduate medical education (GME) programs. We describe the state of simulation in EM resident education, including its role in learning and assessment. The use of medical simulation in GME is increasing for a number of reasons, including the limitations of the 80-hour resident work week, patient dissatisfaction with being "practiced on," a greater emphasis on patient safety, and the importance of early acquisition of complex clinical skills. Simulation-based assessment (SBA) is advancing to the point where it can revolutionize the way clinical competence is assessed in residency training programs. This article also discusses the design of simulation centers and the resources available for developing simulation programs in graduate EM education. The level of interest in these resources is evident by the numerous national EM organizations with internal working groups focusing on simulation. In the future, the health care system will likely follow the example of the airline industry, nuclear power plants, and the military, making rigorous simulation-based training and evaluation a routine part of education and practice.

Testing the use of symptom-based terrorism triage algorithms with hospital-based providers

INTRODUCTION

A set of symptom-based, all-hazards, decision-making algorithms was designed to aid the first-contact provider during early patient presentations after a terrorist incident.

OBJECTIVE

The primary objective was to assess the usability of these algorithms. A secondary objective was to assess the psychometric properties of the testing scenarios.

METHODS

This was a written, usability assessment of the algorithms employing a convenience sample of hospital-based, healthcare providers who had not taken any specific training in the use of the algorithms. A series of 26 paragraph-length, moderately difficult scenarios was created to reflect possible agents, means of attack, and types of patients. Each of the 26 scenarios requires that one make a triage choice on the "attack" algorithm (the trunk algorithm), then proceed to one of four other branch algorithms (dirty resuscitation, chemical agents, biological agents, bomb/blast/radiation dispersal device) to make a final triage choice. Conditional scores based on getting both the attack and final card correct were calculated for each algorithm.

RESULTS

Nineteen attending physicians, 50 emergency medicine residents, and 41 nurses took the assessment. The total score was 45% correct for all participants. The score on the attack algorithm was 66% correct. Dirty resuscitation, biological, chemical, and bomb/blast scores were 46%, 54%, 46%, and 51% respectively. The probability of guessing the correct answer on the attack algorithm was 1/7 or 14%. The conditional probability of guessing both the attack algorithm and the final card correct ranged from 4.7% for the biological, chemical, and bomb/blast algorithms to 2.4% for the dirty resuscitation algorithm. Item discrimination, item difficulty, and Cronbach's alpha were acceptable for the overall test. Certain individual items had item difficulty levels suggesting they were too difficult and should be replaced in future versions of the test.

CONCLUSIONS

Performance on the test suggests that participants did substantially better than would have been expected by chance alone. Future efforts will revise the algorithms with the goal of simplification. Revision of the testing instrument and testing algorithm use after instruction also are needed.

Simulation-based morbidity and mortality conference: new technologies augmenting traditional case-based presentations

The authors describe the use of a high-fidelity simulation laboratory to re-create a patient encounter for the purposes of enhancing a morbidity and mortality conference. The use of two separate technologies were enlisted: a METI high-fidelity patient simulator to re-create the case in a more lifelike fashion, and an audience response system to collect clinical impressions throughout the case presentation and survey data at the end of the presentation. The re-creation of the patient encounter with all relevant physical findings displayed in high fidelity, with relevant laboratory data, nursing notes, and imaging as it occurred in the actual case, provides a more engaging format for the resident-learner. This technological enhancement was deployed at a morbidity and mortality conference, and the authors report the impressions collected via the audience response system. Guidelines for those who wish to re-create this type of educational experience are presented in the discussion.

See one, do one, teach one: advanced technology in medical education

The concept of "learning by doing" has become less acceptable, particularly when invasive procedures and high-risk care are required. Restrictions on medical educators have prompted them to seek alternative methods to teach medical knowledge and gain procedural experience. Fortunately, the last decade has seen an explosion of the number of tools available to enhance medical education: web-based education, virtual reality, and high fidelity patient simulation. This paper presents some of the consensus statements in regard to these tools agreed upon by members of the Educational Technology Section of the 2004 AEM Consensus Conference for Informatics and Technology in Emergency Department Health Care, held in Orlando, Florida.

FINDINGS

Web-based teaching: 1) Every ED should have access to medical educational materials via the Internet, computer-based training, and other effective education methods for point-of-service information, continuing medical education, and training. 2) Real-time automated tools should be integrated into Emergency Department Information Systems [EDIS] for contemporaneous education. Virtual reality [VR]: 1) Emergency physicians and emergency medicine societies should become more involved in VR development and assessment. 2) Nationally accepted protocols for the proper assessment of VR applications should be adopted and large multi-center groups should be formed to perform these studies. High-fidelity simulation: Emergency medicine residency programs should consider the use of high-fidelity patient simulators to enhance the teaching and evaluation of core competencies among trainees.

CONCLUSIONS

Across specialties, patient simulation, virtual reality, and the Web will soon enable medical students and residents to... see one, simulate many, do one competently, and teach everyone.

Computerized physician order entry and online decision support

Computerized physician order entry (CPOE) and decision support systems (DSS) can reduce certain types of error but often slow clinicians and may increase other types of error. The net effect of these systems on an emergency department (ED) is unknown. The consensus participants combined published evidence with expert opinion to outline recommendations for success. These include seamless integration of CPOE and DSS into systems and workflow; ensuring access to Internet-based and other online support material in the clinical arena; designing systems specifically for the ED and measuring their impact to ensure an overall benefit; ensuring that CPOE systems provide error and interaction checking and facilitate weight- and physiology-based dosing; using interruptive alerts only for the highest-severity events; providing a simple, vendor-independent interface for institutional customization of CPOE alert thresholds; maximizing the use of automated systems and passive data capture; and ensuring the widespread availability of CPOE and DSS using secure wireless and portable technologies where appropriate. Decisions regarding CPOE and DSS in the ED should be guided by the ED chair or designee. Much of what is believed to be true regarding CPOE and DSS has not been adequately studied. Additional CPOE and DSS research is needed quickly, and this research should receive funding priority. DSS and CPOE hold great promise to improve patient care, but not all systems are equal. Evidence must guide these efforts, and the measured outcomes must consider the many factors of quality care.

Laparoscopic cholecystectomy in cirrhotic patients

BACKGROUND

Reported mortality for open cholecystectomy in patients with cirrhosis ranges from 10% to 80%. Laparoscopic cholecystectomy has gained acceptance in the general population and has become the procedure of choice for symptomatic cholelithiasis. We reviewed our experience with the use of laparoscopic cholecystectomy in this group.

STUDY DESIGN

We did a retrospective review of the records of 25 consecutive laparoscopic choleoystectomy procedures performed on cirrhotic patients from May 1992 to July 1996.

RESULTS

There were no mortalities in our group. All procedures were completed laparoscopically. Mean length of stay was 1.7 days (range, 1 to 8 days). Morbidity consisted of wound hematomas, pneumonia, and ascites for a rate of 32%. Only patients with Child's Class A and Class B cirrhosis were operated on.

CONCLUSIONS

Laparoscopic cholecystectomy can be performed safely in cirrhotic patients with well compensated liver function.