Consensus development of a pediatric emergency medicine clerkship curriculum

Development and evaluation of a novel knowledge assessment tool for pediatric emergency medicine clerkships

Objectives

This study seeks to determine validity evidence for a newly developed multiple-choice examination (MCE) tool to assess retention and application of medical knowledge of students enrolled in a pediatric emergency medicine (PEM) clerkship.

Methods

A team of PEM physicians created a 110-item MCE covering the range of clinical topics in PEM relevant for medical students. The researchers determined examination content using the report of Clerkship Directors in Emergency Medicine and PEM Interest Group of the Society for Academic Emergency Medicine (SAEM). The authors administered the MCE to fourth-year medical students at the end of their PEM rotation from May 2020 to April 2023 at four institutions and then analyzed the examination using four of Messick's five sources of validity evidence: content, response process, internal structure, and relation to other variables.

Results

A total of 158 students took the test. In academic year (AY)20-21, 47 students took the test and scored, on average, 81%. After revision of poor and indeterminate questions, the 111 medical students who took the revised version of the test in AY21-AY23 scored on average 77.3% with a standard deviation of 5.7% with a normal distribution in scores. The revised questions were rated as excellent (10.0%), good (26.4%), fair (34.5%), poor (24.5%), or indeterminate (4.5%) based on test item discrimination. There was a positive correlation between MCE scores and students' clinical evaluations but no correlation between MCE scores and scores that students received on their clinical notes or patient presentations during case conference.

Conclusions

This novel PEM clerkship examination is a reliable test of medical knowledge. Future directions involve evaluating consequences of the MCE and offering the test to medical students in a dedicated PEM rotation at the national level.

Reliability of the Behaviorally Anchored Rating Scale (BARS) for assessing non-technical skills of medical students in simulated scenarios

PURPOSE

Caring for critically ill patients requires non-technical skills such as teamwork, communication, and task management. The Behaviorally Anchored Rating Scale (BARS) is a brief tool used to assess non-technical skills. The investigators determined inter- and intra-rater reliability of the BARS when used to assess medical students in simulated scenarios.

METHOD

The investigators created simulation scenarios for medical students during their pediatric clerkship. Content experts reviewed video recordings of the simulations and assigned BARS scores for four performance components (Situational Awareness, Decision-Making, Communication, and Teamwork) for the leader and for the team as a whole. Krippendorff's alpha with ordinal difference was calculated to measure inter- and intra-rater reliability.

RESULTS

Thirty medical students had recordings available for review. Inter- and intra-rater reliability for performance components were, respectively, Individual Situational Awareness (0.488, 0.638), Individual Decision-Making (0.529, 0.691), Individual Communication (0.347, 0.473), Individual Teamwork (0.414, 0.466), Team Situational Awareness (0.450, 0.593), Team Decision Making (0.423, 0.703), Team Communication (0.256, 0.517), and Team Teamwork (0.415, 0.490).

CONCLUSIONS

The BARS demonstrated limited reliability when assessing medical students during their pediatric clerkship. Given the unique needs of this population, a modified or new objective scoring system for assessing non-technical skills may be needed for medical students.

Consensus Development of a Core Content for a Standardized Medical Toxicology Curriculum for Medical Students

Background

Currently, no standardized core content in medical toxicology exists for medical students. The goals of this study were to (1) assess the current state and needs of medical toxicology clerkships and (2) develop a consensus-derived list of core topics that should be covered during a medical toxicology clerkship.

Methods

We assembled a task force established by the American College of Medical Toxicology (ACMT) of nine experts in medical toxicology or emergency medicine. We developed a needs assessment survey that was sent to all medical student clerkship directors in medical toxicology. Based on their responses, we used a modified Delphi process to develop a consensus of core topics that should be covered during a medical student clerkship.

Results

Nineteen out of 42 (45%) clerkship directors completed the survey; 18 met inclusion criteria. The majority of clerkships were 4 weeks in duration with an average of 15 students/year. The three most common teaching methods used were bedside teaching (n = 17/18), classroom teaching (n = 17/18), and journal club (n = 14/18). All the clerkship directors (n = 18/18) reported they would use a standardized curriculum as well as educational content developed by ACMT. There was overwhelming consensus on the core topics which included, but were not limited to, pharmacology/toxicology; drugs; drugs of abuse; natural products; pharmacological basis of antidote use; toxicologic syndromes; vital sign abnormalities; initial management; supportive and other care; withdrawal syndrome management; industrial, household, and environmental toxins; differential diagnosis by clinical findings; and ABCs-resuscitation.

Conclusion

The ACMT task force developed a medical toxicology clerkship core content. The task force also identified a need for shared resources among clerkships.

A pirate ship sailed into the yacht club: How we built a novel pediatric emergency medicine curriculum for an emergency medicine training program

BACKGROUND

Pediatric emergency medicine (PEM) has seen little progression toward a standardized PEM educational framework. The 2018 Academic Emergency Medicine Consensus Conference on Advancing PEM Education addressed this gap in core EM education. Absent elements include a "broad needs assessment to identify and evaluate existing curricula and systems gaps in EM training" and a "clearly defined core PEM curriculum that unifies and drives the learning process." PEM education innovators were called to construct a "unified foundation in PEM education for all levels of emergency care" and to "promote innovation in teaching and learning strategies in curricula." We endeavored to meet this challenge at our institution.

METHODS

The PEM curriculum design is based on the Kern model of curriculum development and included a needs assessment, development of goals and objectives, educational strategies, implementation, evaluation, and programmatic feedback. We committed to using effective learning strategies and active learning methods in developing our curriculum and conducted a 1-year pilot within our EM residency's didactic conference. We used exit surveys to collect feedback for each session as well as midyear focus groups to gauge the program's effectiveness. At the start and end of the pilot year residents completed the PEM survey regarding the effect of the PEM curriculum on their self-assessed knowledge, training, and comfort in managing PEM topics.

RESULTS

Feedback regarding the PEM curriculum was positive. Following 1 year of the pilot curriculum, learners in the PGY-1 and PGY-3 classes demonstrated statistically significant improvement in their self-assessed knowledge, training, and comfort with PEM topics. The PGY-2 class had a similar statistically significant improvement in self-assessed knowledge in PEM topics.

CONCLUSIONS

Our novel PEM curriculum was well received and has shown early evidence of improving self-assessed knowledge and comfort among EM residents.

A Modified Delphi Study for Development of a Pediatric Curriculum for Emergency Medicine Residents

OBJECTIVES

Emergency medicine (EM) trainees are expected to learn to provide acute care for patients of all ages. The American Council for Graduate Medical Education provides some guidance on topics related to caring for pediatric patients; however, education about pediatric topics varies across residency programs. The goal of this project was to develop a consensus curriculum for teaching pediatric emergency care.

METHODS

We recruited 13 physicians from six academic health centers to participate in a three-round electronic modified Delphi project. Participants were selected on the basis of expertise with both EM resident education and pediatric emergency care. The first modified Delphi survey asked participants to generate the core knowledge, skills, and experiences needed to prepare EM residents to effectively treat children in an acute care setting. The qualitative data from the first round was reformulated into a second-round questionnaire. During the second round, participants used rating scales to prioritize the curriculum content proposed during the first round. In round 3, participants were asked to make a determination about each curriculum topic using a three-point scale labeled required, optional, or not needed.

RESULTS

The first modified Delphi round yielded 400 knowledge topics, 206 clinical skills, and 44 specific types of experience residents need to prepare for acute pediatric patient care. These were narrowed to 153 topics, 84 skills, and 28 experiences through elimination of redundancy and two rounds of prioritization. The final lists contain topics classified by highly recommended, partially recommended, and not recommended. The partially recommended category is intended to help programs tailor their curriculum to the unique needs of their learners as well as account for variability between 3- and 4-year programs and the amount of time programs allocate to pediatric education.

CONCLUSION

The modified Delphi process yielded the broad outline of a consensus core pediatric emergency care curriculum.

Implementing a third-year emergency medicine medical student curriculum

BACKGROUND

Emergency medicine (EM) is commonly introduced in the fourth year of medical school because of a perceived need to have more experienced students in the complex and dynamic environment of the emergency department. However, there is no evidence supporting the optimal time or duration for an EM rotation, and a number of institutions offer third-year rotations.

OBJECTIVE

A recently published syllabus provides areas of knowledge, skills, and attitudes that third-year EM rotation directors can use to develop curricula. This article expands on that syllabus by providing a comprehensive curricular guide for the third-year medical student rotation with a focus on implementation.

DISCUSSION

Included are consensus-derived learning objectives, discussion of educational methods, considerations for implementation, and information on feedback and evaluation as proposed by the Clerkship Directors in Emergency Medicine Third-Year Curriculum Work Group. External validation results, derived from a survey of third-year rotation directors, are provided in the form of a content validity index for each content area.

CONCLUSIONS

This consensus-derived curricular guide can be used by faculty who are developing or revising a third-year EM medical student rotation and provide guidance for implementing this curriculum at their institution.