Evaluating educational interventions in emergency medicine

A Literature Study of Medical Simulations for Non-Technical Skills Training in Emergency Medicine: Twenty Years of Progress, an Integrated Research Framework, and Future Research Avenues

Medical simulations have led to extensive developments in emergency medicine. Apart from the growing number of applications and research efforts in patient safety, few studies have focused on modalities, research methods, and professions via a synthesis of simulation studies with a focus on non-technical skills training. Intersections between medical simulation, non-technical skills training, and emergency medicine merit a synthesis of progress over the first two decades of the 21st century. Drawing on research from the Web of Science Core Collection's Science Citation Index Expanded and Social Science Citation Index editions, results showed that medical simulations were found to be effective, practical, and highly motivating. More importantly, simulation-based education should be a teaching approach, and many simulations are utilised to substitute high-risk, rare, and complex circumstances in technical or situational simulations. (1) Publications were grouped by specific categories of non-technical skills, teamwork, communication, diagnosis, resuscitation, airway management, anaesthesia, simulation, and medical education. (2) Although mixed-method and quantitative approaches were prominent during the time period, further exploration of qualitative data would greatly contribute to the interpretation of experience. (3) High-fidelity dummy was the most suitable instrument, but the tendency of simulators without explicitly stating the vendor selection calls for a standardised training process. The literature study concludes with a ring model as the integrated framework of presently known best practices and a broad range of underexplored research areas to be investigated in detail.

Development of a lecture evaluation tool rooted in cognitive load theory: A modified Delphi study

Background

Didactics play a key role in medical education. There is no standardized didactic evaluation tool to assess quality and provide feedback to instructors. Cognitive load theory provides a framework for lecture evaluations. We sought to develop an evaluation tool, rooted in cognitive load theory, to assess quality of didactic lectures.

Methods

We used a modified Delphi method to achieve expert consensus for items in a lecture evaluation tool. Nine emergency medicine educators with expertise in cognitive load participated in three modified Delphi rounds. In the first two rounds, experts rated the importance of including each item in the evaluation rubric on a 1 to 9 Likert scale with 1 labeled as "not at all important" and 9 labeled as "extremely important." In the third round, experts were asked to make a binary choice of whether the item should be included in the final evaluation tool. In each round, the experts were invited to provide written comments, edits, and suggested additional items. Modifications were made between rounds based on item scores and expert feedback. We calculated descriptive statistics for item scores.

Results

We completed three Delphi rounds, each with 100% response rate. After Round 1, we removed one item, made major changes to two items, made minor wording changes to nine items, and modified the scale of one item. Following Round 2, we eliminated three items, made major wording changes to one item, and made minor wording changes to one item. After the third round, we made minor wording changes to two items. We also reordered and categorized items for ease of use. The final evaluation tool consisted of nine items.

Conclusions

We developed a lecture assessment tool rooted in cognitive load theory specific to medical education. This tool can be applied to assess quality of instruction and provide important feedback to speakers.

Development, Implementation, and Evaluation of an Open Access, Level-Specific, Core Content Curriculum for Emergency Medicine Residents

BACKGROUND

Leaders in graduate medical education must provide robust clinical and didactic experiences to prepare residents for independent practice. Programs traditionally create didactic experiences individually, requiring tremendous resources with variable content exposure and quality.

OBJECTIVE

We sought to create and implement a free, open access, learner-centric, level-specific, emergency medicine (EM) residency curriculum.

METHODS

We developed Foundations of Emergency Medicine (FoEM) Foundations I and II courses using Kern's model of curriculum development. Fundamental topics were identified through content guidelines from the American Board of Emergency Medicine. We incorporated learner-centric strategies into 2 flipped classroom, case-based courses targeting postgraduate year (PGY) 1 and PGY-2 residents. The curriculum was made freely available online in 2016. Faculty and resident users were surveyed annually for feedback, which informed iterative refinement of the curriculum.

RESULTS

Between 2016 and 2020, registration for FoEM expanded from 2 sites with 36 learners to 154 sites and 4453 learners. In 2019, 98 of 102 (96%) site leaders and 1618 of 2996 (54%) learners completed the evaluative survey. One hundred percent of responding leaders and 93% of learners were "satisfied" or "very satisfied" with FoEM content. Faculty and residents valued FoEM's usability, large volume of content, quality, adaptability, organization, resident-faculty interaction, and resident-as-teacher opportunities. Challenges to implementation included resident attendance, conference structure, technology limitations, and faculty engagement.

CONCLUSIONS

We developed and implemented a learner-centric, level-specific, national EM curriculum that has been widely adopted in the United States.

The Digital EMS California Academy of Learning: One State's Innovative Approach to EMS Fellow Education

INTRODUCTION

Emergency medical services (EMS) fellowships are growing in significance within the United States prehospital health care system. While fellowships represent a cornerstone of EMS subspecialty education, an individual learner's experiences are limited by local resources and practices. California EMS fellowships have developed an innovative method for expanding fellows' educational experiences outside their immediate programs.

THE INNOVATIVE EDUCATION METHOD

Each month, fellows, fellowship directors, and local EMS medical directors from throughout the state participate in a video conference. This meeting is divided into four distinct components: book chapter presentation, board-style question review, call review, and an EMS literature review.

CHAPTER REVIEW

The two-volume text Emergency Medical Services: Clinical Practice and Systems Oversight has been categorized into 12 modules, one for each month of the fellowship. Every meeting, one fellow prepares a didactic presentation summarizing the highlights from that month's chapters.

QUESTION REVIEW

Fellows each create five multiple-choice questions and answers, based on the section reading. Questions are assessed by the group, both for informational content and for appropriate formatting. After completion, these questions are submitted for future review for the EMS fellowship in-service examination.

CALL REVIEW

Based on that month's module topics, a call is chosen and reviewed. Regional protocol and practice differences from different systems are discussed. The online medical oversight provided and the prehospital provider performance are evaluated by the group.

LITERATURE REVIEW

Fellows not assigned to present a call or didactic segment each choose one paper focusing on a subject relevant to the module or call. Strengths of the study design, analysis, outcomes, and relevance to EMS practice are discussed.

OUTCOMES

Fellows and experienced EMS attendings are exposed to different protocol and system approaches in an interactive and accessible format. This partnership expands educational opportunities for fellows and promotes collaboration across EMS systems.

Educational interventions promoting evidence-based practice among emergency nurses: A systematic review

INTRODUCTION

Emergency nurses are expected to adopt evidence-based practice (EBP). The aim of this systematic review was to describe educational interventions promoting EBP and their outcomes among emergency nurses, compared with no education, to inform clinicians and researchers about effective educational interventions suitable for use in emergency departments (EDs).

METHODS

CINAHL, Cochrane, PubMed and Scopus were systematically searched to identify studies published between January 1, 2006 and October 20, 2016 describing educational interventions designed to promote EBP among emergency nurses. 711 studies were identified and screened; 10 were selected for inclusion and quality assessment. The studies were analyzed using deductive content analysis, and the review's results are presented in accordance with the PRISMA guidelines.

RESULTS

Ten relevant studies on nine different self-developed educational interventions were identified. Eight studies had highly significant or significant results. Interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses' knowledge, skills, and behavior. Interventions using written self-directed learning material led to significant improvements in nurses' knowledge of EBP. All the descriptions of the interventions were incomplete, and the reported details varied considerably between the studies.

CONCLUSIONS

There have been few studies on educational interventions to promote EBP among emergency nurses but the available results are promising.

Growing Pains: a Simulation-Based Curriculum for Improving the Transition to Hematology/Oncology Fellowship

Trainee exposure to clinical oncology during residency training is heterogeneous and often modest. The steep learning curve upon entry into fellowship can result in undue stress for fellows and their patients. Simulation-based training has been shown to be superior to classical didactic approaches. We have introduced several innovative simulation-based workshops into the curriculum for the Johns Hopkins Hematology/Oncology Fellowship Training Program in order to address this unmet need. During the first months of training, fellows were engaged in activities emphasizing essential clinical and procedural skills. Specific workshops included the following: (1) chemotherapy writing, (2) cadaveric and simulation-based bone marrow biopsy and intrathecal chemotherapy administration, and (3) simulation-based communication skills training. All first-year fellows in our program participated in these exercises. Pre- and post-workshop surveys were administered to assess knowledge, attitudes, and behaviors; additional distant post-workshop evaluations were disseminated to assess the durability/impact of the curricula and for program evaluation. Overall, participating fellows indicated that the workshops improved patient care and comfort with procedures and patient-centered communication. Continued implementation of these workshops was recommended for program improvement. To the best of our knowledge, ours is amongst the first oncology fellowship training programs to systematically implement simulation-based curricula into our schema for fellowship training. We hypothesize that proactively introducing fellows to these high-yield activities will translate into improved patient care and reduced stress for trainees. Additional investigation into the long-term impact of such curricula remains an area of ongoing need.

Using Twitter to increase content dissemination and control educational content with Presenter Initiated and Generated Live Educational Tweets (PIGLETs)

Live-tweeting during educational presentations is typically learner-generated and can lead to misquoted information. Presenter curated tweets have not been well described. We created Presenter Initiated and Generated Live Educational Tweets (PIGLETs) with the goal to broaden the reach of educational conferences. We hypothesized that using PIGLETs would increase the reach and exposure of our material. We developed a prospective single-arm intervention study performed during the "Not Another Boring Lecture" workshops presented at two national conferences in 2015. Presenters tweeted PIGLETs linked to unique hashtags #NotAnotherBoringLecture and #InnovateMedEd. Analytic software was used to measure the following outcomes: (1) number of tweets published by presenters versus learners, (2) reach (users exposed to content containing the hashtag), and (3) exposure (total number of times content was delivered). One hundred and twenty-six participants attended the workshops. A total of 636 tweets (including retweets) were sent by presenters containing the study hashtags, compared with 162 sent by learners. #NotAnotherBoringLecture reached 47,200 users and generated 136,400 impressions; #InnovateMedEd reached 36,400 users and generated 79,100 impressions. PIGLETs allowed presenters to reach a significant number of learners, as well as control the content delivered through Twitter. PIGLETs can be used to augment educational sessions beyond the physical confines of the classroom.

Low Back Pain in the Emergency Medicine Department: A Flipped Classroom Module

INTRODUCTION

Current residency didactic schedules that are built upon hour-long, lecture-based presentations are incongruous with adult learning theory and the needs of millennial generation residents. An alternative to the traditional lecture, the flipped classroom involves viewing a short video lecture at home, followed by an active discussion during class time. This module was developed for emergency medicine residents and rotating medical students without previous training on the subject.

METHODS

The at-home portion of the module was designed to be delivered at home, while the in-class discussion was designed to be carried out over 30-45 minutes during a regularly scheduled didactic time. Small-group size may be determined by faculty availability, though groups of five are optimal. There is no requirement for faculty preparation prior to the in-class session. Associated materials include objectives, the at-home video, a discussion guide for faculty facilitators, a case-based handout for students and residents, and assessment questions. We assessed our module with a pretest, immediate posttest, and the posttest again after 90 days.

RESULTS

The mean pretest score was 66%, mean posttest score 76%, and mean retention test score 66%. There was an immediate increase of 10%, which did not remain at 90 days.

DISCUSSION

We developed a flipped classroom module that can be implemented in any emergency medicine residency or clerkship. It addresses the theoretical challenges posed to traditional conference didactics by increasing the focus on problem solving and self-directed learning.

Obstetric training in Emergency Medicine: a needs assessment

BACKGROUND

Identification and management of obstetric emergencies is essential in emergency medicine (EM), but exposure to pregnant patients during EM residency training is frequently limited. To date, there is little data describing effective ways to teach residents this material. Current guidelines require completion of 2 weeks of obstetrics or 10 vaginal deliveries, but it is unclear whether this instills competency.

METHODS

We created a 15-item survey evaluating resident confidence and knowledge related to obstetric emergencies. To assess confidence, we asked residents about their exposure and comfort level regarding obstetric emergencies and eight common presentations and procedures. We assessed knowledge via multiple-choice questions addressing common obstetric presentations, pelvic ultrasound image, and cardiotocography interpretation. The survey was distributed to residency programs utilizing the Council of Emergency Medicine Residency Directors (CORD) listserv.

RESULTS

The survey was completed by 212 residents, representing 55 of 204 (27%) programs belonging to CORD and 11.2% of 1,896 eligible residents. Fifty-six percent felt they had adequate exposure to obstetric emergencies. The overall comfort level was 2.99 (1-5 scale) and comfort levels of specific presentations and procedures ranged from 2.58 to 3.97; all increased moderately with postgraduate year (PGY) level. Mean overall percentage of items answered correctly on the multiple-choice questions was 58% with no statistical difference by PGY level. Performance on individual questions did not differ by PGY level.

CONCLUSIONS

The identification and management of obstetric emergencies is the cornerstone of EM. We found preliminary evidence of a concerning lack of resident comfort regarding obstetric conditions and knowledge deficits on core obstetrics topics. EM residents may benefit from educational interventions to increase exposure to these topics.

Ten Tips for Engaging the Millennial Learner and Moving an Emergency Medicine Residency Curriculum into the 21st Century

Introduction

Millennial learners are changing the face of residency education because they place emphasis on technology with new styles and means of learning. While research on the most effective way to teach the millennial learner is lacking, programs should consider incorporating educational theories and multimedia design principles to update the curriculum for these new learners. The purpose of the study is to discuss strategies for updating an emergency medicine (EM) residency program’s curriculum to accommodate the modern learner.

Discussion

These 10 tips provide detailed examples and approaches to incorporate technology and learning theories into an EM curriculum to potentially enhance learning and engagement by residents.

Conclusion

While it is unclear whether technologies actually promote or enhance learning, millennials use these technologies. Identifying best practice, grounded by theory and active learning principles, may help learners receive quality, high-yield education. Future studies will need to evaluate the efficacy of these techniques to fully delineate best practices.

A Model Longitudinal Observation Medicine Curriculum for an Emergency Medicine Residency

The role of observation services for emergency department patients has increased in recent years. Driven by changing health care practices and evolving payer policies, many hospitals in the United States currently have or are developing an observation unit (OU) and emergency physicians are most often expected to manage patients in this setting. Yet, few residency programs dedicate a portion of their clinical curriculum to observation medicine. This knowledge set should be integrated into the core training curriculum of emergency physicians. Presented here is a model observation medicine longitudinal training curriculum, which can be integrated into an emergency medicine (EM) residency. It was developed by a consensus of content experts representing the observation medicine interest group and observation medicine section, respectively, from EM's two major specialty societies: the Society for Academic Emergency Medicine (SAEM) and the American College of Emergency Physicians (ACEP). The curriculum consists of didactic, clinical, and self-directed elements. It is longitudinal, with learning objectives for each year of training, focusing initially on the basic principles of observation medicine and appropriate observation patient selection; moving to the management of various observation appropriate conditions; and then incorporating further concepts of OU management, billing, and administration. This curriculum is flexible and designed to be used in both academic and community EM training programs within the United States. Additionally, scholarly opportunities, such as elective rotations and fellowship training, are explored.

How we used two social media tools to enhance aspects of active learning during lectures

BACKGROUND

Medical education is evolving to include active learning approaches, yet some courses will remain lecture-based. Social media tools used by students may foster collaborative learning during lectures.

AIM

We present preliminary results from a pilot study that integrated two 'social' technologies, Google Docs and SurveyMonkey, into 22 hour-long lectures for a course called "Social Influences on Health" attended by 154 students.

METHODS

At the conclusion of the semester, we reviewed student usage patterns with both technologies and collected data from students via course evaluations that included a standard Likert Scale. We used thematic analysis to identify emergent themes from evaluations.

RESULTS

On average, students contributed 6 comments/questions to the Google Doc in each lecture, and 35 students participated in SurveyMonkey. Engagement with both technologies increased throughout the semester and no unprofessional incidents were observed. The mean student rating for integration of Google Docs and SurveyMonkey was 3.4 or "above average" (SD = 1.17). Thematic analysis identified perceived strengths of this approach as well as areas for improvement.

CONCLUSIONS

Social media such as Google Docs and SurveyMonkey can facilitate interaction and provide students with control over content and flow of lecture-based courses, but educators must be mindful of practical and conceptual limitations.

Medical education research in the context of translational science

Health care struggles to transfer recent discoveries into high-quality medical care. Therefore, translational science seeks to improve the health of patients and communities by studying and promoting the translation of findings from bench research into clinical care. Similarly, medical education practice may be slow to adopt proven evidence of better learning and assessment. The Academic Emergency Medicine (AEM) consensus conference was designed to promote the dissemination of evidence-based education research and practice. We will pull from the work developed by the consensus conference as a means to create a roadmap for future medical education research using the framework of translational science.