Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

Objectives

The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment. Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management.

Methods

This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists. Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests.

Results

Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p < 0.001, respectively. The kappa coefficients were 0.909 (n = 2698, p < 0.001) and 0.931 (n = 1872, p < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case (p < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case (p < 0.001).

Conclusion

A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10 competencies and entrustment scores. This study provides a curricular framework to support formative individualized assessments for EPA 10.

Learning rate of students detecting and annotating pediatric wrist fractures in supervised artificial intelligence dataset preparations

The use of artificial intelligence (AI) in image analysis is an intensively debated topic in the radiology community these days. AI computer vision algorithms typically rely on large-scale image databases, annotated by specialists. Developing and maintaining them is time-consuming, thus, the involvement of non-experts into the workflow of annotation should be considered. We assessed the learning rate of inexperienced evaluators regarding correct labeling of pediatric wrist fractures on digital radiographs. Students with and without a medical background labeled wrist fractures with bounding boxes in 7,000 radiographs over ten days. Pediatric radiologists regularly discussed their mistakes. We found F1 scores-as a measure for detection rate-to increase substantially under specialist feedback (mean 0.61±0.19 at day 1 to 0.97±0.02 at day 10, p<0.001), but not the Intersection over Union as a parameter for labeling precision (mean 0.27±0.29 at day 1 to 0.53±0.25 at day 10, p<0.001). The times needed to correct the students decreased significantly (mean 22.7±6.3 seconds per image at day 1 to 8.9±1.2 seconds at day 10, p<0.001) and were substantially lower as annotated by the radiologists alone. In conclusion our data showed, that the involvement of undergraduated students into annotation of pediatric wrist radiographs enables a substantial time saving for specialists, therefore, it should be considered.

Rethinking Radiology: An Active Learning Curriculum for Head Computed Tomography Interpretation

Introduction: Head computed tomography (CT) interpretation is a vital skill for emergency physicians. Existing literature shows poor concordance between emergency physicians and radiologists in head CT interpretation. Prior studies have used passive learning methods to address this knowledge gap. We created an active learning curriculum for teaching head CT interpretation to emergency medicine (EM) residents and compared its effectiveness to a passive learning strategy. Methods: We conducted a prospective, randomized controlled study of EM residents at a single institution. Three educational sessions were delivered over a three-month period via video conference. The active learning cohort (ALC) scrolled through head CT teaching cases we designed on Pascbin, a web-based radiology picture archiving and communication system. The passive learning cohort (PLC) watched instructional videos that scrolled through the same cases. Both cohorts were given equal time to review the cases and ask an instructor questions. Residents took pre-intervention and post-intervention tests on head CT interpretation. We analyzed scores using paired and unpaired t-tests. Results: Forty-two residents took the pre-intervention test. Mean pre- and post-test scores for the ALC were 43.8% and 59.0% (P <0.001), and for the PLC were 41.7% and 45.3% (P = 0.29). The difference in ALC and PLC post-test scores was statistically significant (P = 0.009) with a large effect size (Cohen’s d = 1.34). Conclusion: Our active learning head CT curriculum using Pacsbin showed superior learning outcomes when compared to a passive learning strategy and required no additional time or resources. This intervention offers a more effective and learner-centric method for implementing radiology curricula in EM residency programs.

UME-to-GME PandEMonium in COVID-19: Large-Scale Implementation of a Virtual ACGME Milestone-Based Curriculum for Senior Medical Students Matched Into Emergency Medicine

BACKGROUND

The COVID-19 pandemic displaced newly matched emergency medicine "pre-interns" from in-person educational experiences at the end of medical school. This called for novel remote teaching modalities.

OBJECTIVE

This study assesses effectiveness of a multisite Accreditation Council for Graduate Medical Education (ACGME) sub-competency-based curricular implementation on Slack during the first wave of the COVID-19 pandemic in the United States.

METHODS

Emergency medicine residency programs were recruited via national organization listservs. Programs designated instructors to manage communications and teaching for the senior medical students who had matched to their programs (pre-interns) in spring/summer 2020. Pre- and post-surveys of trainees and instructors assessed perceived preparedness for residency, perceived effectiveness of common virtual educational modalities, and concern for the pandemic's effects on medical education utilizing a Likert scale of 1 (very unconcerned) to 5 (very concerned). Data were analyzed using descriptive statistics and the t test.

RESULTS

Of 276 possible residency programs, 28 enrolled. Of 324 possible pre-interns, 297 (91.7%) completed pre-surveys in April/May and 249 (76.9%) completed post-surveys in June/July. The median weeks since performing a physical examination was 8 (IQR 7-12), since attending in-person didactics was 10 (IQR 8-15) and of rotation displacement was 4 (IQR 2-6). Perceived preparedness increased both overall and for 14 of 21 ACGME Milestone topics taught. Instructors reported higher mean concern (4.32, 95% CI 4.23-4.41) than pre-interns (2.88, 95% CI 2.74-3.02) regarding the pandemic's negative effects on medical education.

CONCLUSIONS

Pre-interns reported improvements in residency preparedness after participating in this ACGME sub-competency-based curriculum on Slack.

Radiology Education Among Emergency Medicine Residencies: A National Needs Assessment

Introduction

Radiology training is an important component of emergency medicine (EM) education, but its delivery has been variable. Program directors have reported a lack of radiology skills in incoming interns. A needs assessment is a crucial first step toward improving radiology education among EM residencies. Our objective was to explore the current state of radiology education in EM residency programs.

Methods

This was a cross-sectional survey study of all Accreditation Council for Graduate Medical Education-accredited EM programs in the United States. Program leadership completed an online survey consisting of multiple choice, Likert scale, and free-response items. We calculated and reported descriptive statistics.

Results

Of eligible EM programs, 142/252 (56%) completed the survey including 105 postgraduate year (PGY) 1–3 and 36 PGY 1–4 programs. One respondent opted out of answering demographic questions. 23/141 (16%) were from the Western region, 29/141 (21%) were from the North Central region, 14/141 (10%) were from the South-Central region, 28/141 (20%) were from the Southeast region, and 47/141 (33%) were from the Northeast region. A total of 88/142 (62%) of responding programs did not have formal radiology instruction. Of the education that is provided, 127/142 (89%) provide it via didactics/lectures and 115/142 (81%) rely on instruction during clinical shifts. Only 51/142 (36%) provide asynchronous opportunities, and 23/142 (16%) have a dedicated radiology rotation. The majority of respondents reported spending 0–2 hours per month on radiology instruction (108/142; 76%); 95/141 (67%) reported that EM faculty “often” or “always” provide radiology instruction; 134/142 (95%), felt that it was “extremely” or “very important” for ED providers to be able to independently interpret radiograph results; and 129/142 (90.84%) either “sometimes” or “always” rely on their independent radiograph interpretations to make clinical decisions. The radiology studies identified as most important to be able to independently interpret were radiographs obtained for lines/tubes, chest radiographs, and radiographs obtained for musculoskeletal-related complaints.

Conclusion

A minority of EM residency programs have formal instruction in radiology despite the majority of responding program leadership believing that these are important skills. The most important curricular areas were identified. These results may inform the development of formal radiology curricula in EM graduate medical education.

Speech-language therapists' perceptions of an eLearning program to support training in videofluoroscopic swallow studies

BACKGROUND

Speech-language therapists (SLTs) seek a range of educational opportunities for training in adult videofluoroscopic swallow studies (VFSS). However, variable training methods and/or unequal access to training can influence VFSS practice.

AIMS

To document current SLT needs and barriers to VFSS training and to determine if a new beginner-level VFSS eLearning program would assist to meet their training needs. The program incorporated multimedia modules on preparing, conducting, interpreting and reporting VFSS.

METHODS & PROCEDURES

SLTs with limited experience in adult VFSS completed surveys relating to VFSS training experience and barriers, and perceived changes in knowledge, skills and confidence on core VFSS module topics pre- (n = 36) and post- (n = 32) eLearning training.

OUTCOMES & RESULTS

Inconsistent access to VFSS training opportunities and time-related work pressures were reported as the greatest training barriers. SLTs viewed the eLearning program as a suitable option for VFSS training. Post-training, participants perceived they gained confidence, as well as improved knowledge and skills in all VFSS aspects along with generalised benefits for dysphagia management. SLTs indicated that key benefits of the eLearning program were its comprehensive content and self-directed learning with multimedia tools, which afforded theoretical and practical learning opportunities.

CONCLUSIONS & IMPLICATIONS

The eLearning program offered SLTs free access to beginner-level adult VFSS training, meeting many identified training needs and providing a foundation from which to develop further practical knowledge and skills within a VFSS clinic setting. What this paper adds What is already known on the subject SLTs demonstrate variable knowledge and skill in conducting and interpreting VFSS, which can impact dysphagia diagnosis and management. While access to VFSS training can be challenging, the barriers to training for SLTs have not been clearly documented. Research has confirmed that eLearning can be used effectively in healthcare education, and in some aspects of VFSS training; however, it is yet to be applied to address the broad range of VFSS training needs. What this paper adds to existing knowledge This study describes the SLT reported barriers to VFSS training which include limited access to formal and practical training, workload-related time pressures and the complexity of learning the VFSS skill set. The findings highlight that an eLearning program, was an accepted mode of learning for VFSS training. SLTs reported the online program met their learning needs by improving access to training, the multimedia program features supported their understanding of complex anatomical and physiological concepts, and training frameworks assisted their clinical reasoning and VFSS interpretation. What are the potential or actual clinical implications of this work? eLearning can assist in overcoming many VFSS training barriers identified by SLTs and the multimedia aspects of eLearning can effectively support VFSS beginner-level education to complement and expedite in-clinic practical training. Given that VFSS results inform decisions regarding commencement and progression of oral intake and swallow rehabilitation, enhanced VFSS training has the potential to positively influence dysphagia outcomes and quality of life.

Learning curve for endoscopic evaluation of vocal folds lesions with narrow band imaging

Introduction

The endoscopic methods are progressing and becoming more common in routine clinical diagnosis in the field of otorhinolaryngology. Relatively large amount of researches have proved high accuracy of narrow band imaging endoscopy in differentiating benign and malignant lesions within vocal folds. However, little is known about learning curve in narrow band imaging evaluation of laryngeal lesions.

Objective

The aim of this study was to determine the learning curve for the narrow band imaging evaluation of vocal folds pathologies depending on the duration of the procedure.

Methods

Records of 134 narrow band imaging that were analyzed in terms of the duration of the procedure and the accuracy of diagnosis confirmed by histopathological diagnosis were enrolled in the study. The narrow band imaging examinations were performed sequentially by one investigator over a period of 18 months.

Results

The average duration of narrow band imaging recordings was 127.82 s. All 134 studies were divided into subsequent series of several elements. An evident decrease in time of investigation was noticed between 13th and 14th series, when the examinations were divided into 5 elements series, which corresponds to the difference between 65th and 70th subsequent narrow band imaging examination. Parallel groups of 67 examinations were created. Group 1 included 1st to 67th subsequent narrow band imaging examination; Group 2 – 68th to 134th narrow band imaging examinations. The non-parametric U Mann–Whitney test confirmed statistically significant difference between the mean duration of narrow band imaging examination in both groups 160.5 s and 95.1 s, respectively (p < 10⁻⁷). Sensitivity and specificity of narrow band imaging examination in the first group were respectively: 83.7% and 76.7%. In the second group, these indicators amounted 98.1% and 80% respectively.

Conclusions

A minimum of 65th–70th narrow band imaging examinations are required to reach a plateau phase of the learning process in assessment of glottis lesions. Analysis of learning curves is useful for the development of training programs and determination of a mastery level.