1
|
Miller DT, Gibb W, Caretta‐Weyer H, Ng K, Sebok‐Syer SS, Gisondi MA. Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students. AEM EDUCATION AND TRAINING 2022; 6:e10787. [PMID: 36389650 PMCID: PMC9646936 DOI: 10.1002/aet2.10787] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 06/28/2022] [Accepted: 06/29/2022] [Indexed: 06/16/2023]
Abstract
Objectives The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment. Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management. Methods This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists. Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests. Results Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p < 0.001, respectively. The kappa coefficients were 0.909 (n = 2698, p < 0.001) and 0.931 (n = 1872, p < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case (p < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case (p < 0.001). Conclusion A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10 competencies and entrustment scores. This study provides a curricular framework to support formative individualized assessments for EPA 10.
Collapse
Affiliation(s)
- Danielle T. Miller
- Department of Emergency MedicineUniversity of Colorado School of MedicineDenverColoradoUSA
| | - William Gibb
- Department of Emergency MedicineUniversity of Colorado School of MedicineDenverColoradoUSA
| | - Holly Caretta‐Weyer
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Kristen Ng
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Stefanie S. Sebok‐Syer
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Michael A. Gisondi
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| |
Collapse
|
2
|
Nagy E, Marterer R, Hržić F, Sorantin E, Tschauner S. Learning rate of students detecting and annotating pediatric wrist fractures in supervised artificial intelligence dataset preparations. PLoS One 2022; 17:e0276503. [PMID: 36264961 PMCID: PMC9584407 DOI: 10.1371/journal.pone.0276503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 09/13/2022] [Indexed: 11/06/2022] Open
Abstract
The use of artificial intelligence (AI) in image analysis is an intensively debated topic in the radiology community these days. AI computer vision algorithms typically rely on large-scale image databases, annotated by specialists. Developing and maintaining them is time-consuming, thus, the involvement of non-experts into the workflow of annotation should be considered. We assessed the learning rate of inexperienced evaluators regarding correct labeling of pediatric wrist fractures on digital radiographs. Students with and without a medical background labeled wrist fractures with bounding boxes in 7,000 radiographs over ten days. Pediatric radiologists regularly discussed their mistakes. We found F1 scores-as a measure for detection rate-to increase substantially under specialist feedback (mean 0.61±0.19 at day 1 to 0.97±0.02 at day 10, p<0.001), but not the Intersection over Union as a parameter for labeling precision (mean 0.27±0.29 at day 1 to 0.53±0.25 at day 10, p<0.001). The times needed to correct the students decreased significantly (mean 22.7±6.3 seconds per image at day 1 to 8.9±1.2 seconds at day 10, p<0.001) and were substantially lower as annotated by the radiologists alone. In conclusion our data showed, that the involvement of undergraduated students into annotation of pediatric wrist radiographs enables a substantial time saving for specialists, therefore, it should be considered.
Collapse
Affiliation(s)
- Eszter Nagy
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
- * E-mail:
| | - Robert Marterer
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| | - Franko Hržić
- Faculty of Engineering, University of Rijeka, Rijeka, Croatia
| | - Erich Sorantin
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| | - Sebastian Tschauner
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| |
Collapse
|
3
|
Aliaga L, Clarke S. Rethinking Radiology: An Active Learning Curriculum for Head Computed Tomography Interpretation. West J Emerg Med 2022; 23:47-51. [PMID: 35060860 PMCID: PMC8782134 DOI: 10.5811/westjem.2021.10.53665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Accepted: 10/12/2021] [Indexed: 11/11/2022] Open
Abstract
Introduction: Head computed tomography (CT) interpretation is a vital skill for emergency physicians. Existing literature shows poor concordance between emergency physicians and radiologists in head CT interpretation. Prior studies have used passive learning methods to address this knowledge gap. We created an active learning curriculum for teaching head CT interpretation to emergency medicine (EM) residents and compared its effectiveness to a passive learning strategy.
Methods: We conducted a prospective, randomized controlled study of EM residents at a single institution. Three educational sessions were delivered over a three-month period via video conference. The active learning cohort (ALC) scrolled through head CT teaching cases we designed on Pascbin, a web-based radiology picture archiving and communication system. The passive learning cohort (PLC) watched instructional videos that scrolled through the same cases. Both cohorts were given equal time to review the cases and ask an instructor questions. Residents took pre-intervention and post-intervention tests on head CT interpretation. We analyzed scores using paired and unpaired t-tests.
Results: Forty-two residents took the pre-intervention test. Mean pre- and post-test scores for the ALC were 43.8% and 59.0% (P <0.001), and for the PLC were 41.7% and 45.3% (P = 0.29). The difference in ALC and PLC post-test scores was statistically significant (P = 0.009) with a large effect size (Cohen’s d = 1.34).
Conclusion: Our active learning head CT curriculum using Pacsbin showed superior learning outcomes when compared to a passive learning strategy and required no additional time or resources. This intervention offers a more effective and learner-centric method for implementing radiology curricula in EM residency programs.
Collapse
Affiliation(s)
- Leonardo Aliaga
- University of California – Davis, Department of Emergency Medicine, Sacramento, California
| | - Samuel Clarke
- University of California – Davis, Department of Emergency Medicine, Sacramento, California
| |
Collapse
|
4
|
McLean ME, Cotarelo AA, Huls TA, Husain A, Hillman EA, Cygan LD, Archer LO, Beck-Esmay J, Burke SM, Carrick AI, Chen AS, Hyde RJ, Karalius VP, Lee E, Park JC, Pugliese AM, Wilbanks MD, Young A, Kulkarni ML. UME-to-GME PandEMonium in COVID-19: Large-Scale Implementation of a Virtual ACGME Milestone-Based Curriculum for Senior Medical Students Matched Into Emergency Medicine. J Grad Med Educ 2021; 13:848-857. [PMID: 35070098 PMCID: PMC8672831 DOI: 10.4300/jgme-d-21-00620.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Revised: 08/17/2021] [Accepted: 10/04/2021] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND The COVID-19 pandemic displaced newly matched emergency medicine "pre-interns" from in-person educational experiences at the end of medical school. This called for novel remote teaching modalities. OBJECTIVE This study assesses effectiveness of a multisite Accreditation Council for Graduate Medical Education (ACGME) sub-competency-based curricular implementation on Slack during the first wave of the COVID-19 pandemic in the United States. METHODS Emergency medicine residency programs were recruited via national organization listservs. Programs designated instructors to manage communications and teaching for the senior medical students who had matched to their programs (pre-interns) in spring/summer 2020. Pre- and post-surveys of trainees and instructors assessed perceived preparedness for residency, perceived effectiveness of common virtual educational modalities, and concern for the pandemic's effects on medical education utilizing a Likert scale of 1 (very unconcerned) to 5 (very concerned). Data were analyzed using descriptive statistics and the t test. RESULTS Of 276 possible residency programs, 28 enrolled. Of 324 possible pre-interns, 297 (91.7%) completed pre-surveys in April/May and 249 (76.9%) completed post-surveys in June/July. The median weeks since performing a physical examination was 8 (IQR 7-12), since attending in-person didactics was 10 (IQR 8-15) and of rotation displacement was 4 (IQR 2-6). Perceived preparedness increased both overall and for 14 of 21 ACGME Milestone topics taught. Instructors reported higher mean concern (4.32, 95% CI 4.23-4.41) than pre-interns (2.88, 95% CI 2.74-3.02) regarding the pandemic's negative effects on medical education. CONCLUSIONS Pre-interns reported improvements in residency preparedness after participating in this ACGME sub-competency-based curriculum on Slack.
Collapse
Affiliation(s)
- Mary E. McLean
- Mary E. McLean, MD, is Assistant Residency Director, Department of Emergency Medicine, St. John's Riverside Hospital
| | - Adrian A. Cotarelo
- Adrian A. Cotarelo, MD, MHS, is a Resident Physician, Department of Emergency Medicine, St. John's Riverside Hospital
| | - Thomas A. Huls
- Thomas A. Huls, MD, is Associate Physician, Department of Emergency Medicine, Kaiser Permanente Modesto Medical Center
| | - Abbas Husain
- Abbas Husain, MD, is Associate Program Director, Department of Emergency Medicine, Staten Island University Hospital, Northwell Health
| | - Emily A. Hillman
- Emily A. Hillman, MD, MHPE, is Associate Program Director, Department of Emergency Medicine, University of Missouri
| | - Lukasz D. Cygan
- Lukasz D. Cygan, DO, is Assistant Residency Director, Department of Emergency Medicine, NewYork-Presbyterian Brooklyn Methodist Hospital
| | - Linette O. Archer
- Linette O. Archer, MD, is Program Director, Department of Emergency Medicine, Memorial Health System
| | - Jennifer Beck-Esmay
- Jennifer Beck-Esmay, MD, is Assistant Residency Director, Department of Emergency Medicine, Mount Sinai Morningside-Mount Sinai West
| | - Shannon M. Burke
- Shannon M. Burke, MD, is a Resident Physician, Department of Emergency Medicine, University of Wisconsin-BerbeeWalsh
| | - Angela I. Carrick
- Angela I. Carrick, DO, is Associate Program Director, Department of Emergency Medicine, Norman Regional Health System
| | - Angela S. Chen
- Angela S. Chen, MD, is Assistant Program Director, Department of Emergency Medicine, Mount Sinai Hospital
| | - Robert J. Hyde
- Robert J. Hyde, MD, MA, is Clerkship Director, Department of Emergency Medicine, Mayo Clinic
| | - Vytas P. Karalius
- Vytas P. Karalius, MD, MPH, MA, is a Resident Physician, Department of Emergency Medicine, McGaw Medical Center of Northwestern University
| | - Eric Lee
- Eric Lee, MD, is Assistant Clerkship Director, Department of Emergency Medicine, Maimonides Medical Center
| | - Joel C. Park
- Joel C. Park, MD, MS, is Attending Physician, Department of Emergency Medicine, St. John's Riverside Hospital, Yonkers
| | - Angela M. Pugliese
- Angela M. Pugliese, MD, is Associate Residency Director, Department of Emergency Medicine, Wayne State University School of Medicine-Henry Ford Hospital
| | - Morgan D. Wilbanks
- Morgan D. Wilbanks, MD, is Interim Director of UME and M3 Elective Director, Department of Emergency Medicine, Medical College of Wisconsin
| | - Amanda Young
- Amanda Young, MD, is Assistant Residency Program Director, Department of Emergency Medicine, University of Arkansas for Medical Sciences
| | - Miriam L. Kulkarni
- Miriam L. Kulkarni, MD, is Residency Program Director, Department of Emergency Medicine, St. John's Riverside Hospital
| |
Collapse
|
5
|
Villa SE, Wheaton N, Lai S, Jordan J. Radiology Education Among Emergency Medicine Residencies: A National Needs Assessment. West J Emerg Med 2021; 22:1110-1116. [PMID: 34546887 PMCID: PMC8463068 DOI: 10.5811/westjem.2021.6.52470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 06/03/2021] [Indexed: 11/17/2022] Open
Abstract
Introduction Radiology training is an important component of emergency medicine (EM) education, but its delivery has been variable. Program directors have reported a lack of radiology skills in incoming interns. A needs assessment is a crucial first step toward improving radiology education among EM residencies. Our objective was to explore the current state of radiology education in EM residency programs. Methods This was a cross-sectional survey study of all Accreditation Council for Graduate Medical Education-accredited EM programs in the United States. Program leadership completed an online survey consisting of multiple choice, Likert scale, and free-response items. We calculated and reported descriptive statistics. Results Of eligible EM programs, 142/252 (56%) completed the survey including 105 postgraduate year (PGY) 1–3 and 36 PGY 1–4 programs. One respondent opted out of answering demographic questions. 23/141 (16%) were from the Western region, 29/141 (21%) were from the North Central region, 14/141 (10%) were from the South-Central region, 28/141 (20%) were from the Southeast region, and 47/141 (33%) were from the Northeast region. A total of 88/142 (62%) of responding programs did not have formal radiology instruction. Of the education that is provided, 127/142 (89%) provide it via didactics/lectures and 115/142 (81%) rely on instruction during clinical shifts. Only 51/142 (36%) provide asynchronous opportunities, and 23/142 (16%) have a dedicated radiology rotation. The majority of respondents reported spending 0–2 hours per month on radiology instruction (108/142; 76%); 95/141 (67%) reported that EM faculty “often” or “always” provide radiology instruction; 134/142 (95%), felt that it was “extremely” or “very important” for ED providers to be able to independently interpret radiograph results; and 129/142 (90.84%) either “sometimes” or “always” rely on their independent radiograph interpretations to make clinical decisions. The radiology studies identified as most important to be able to independently interpret were radiographs obtained for lines/tubes, chest radiographs, and radiographs obtained for musculoskeletal-related complaints. Conclusion A minority of EM residency programs have formal instruction in radiology despite the majority of responding program leadership believing that these are important skills. The most important curricular areas were identified. These results may inform the development of formal radiology curricula in EM graduate medical education.
Collapse
Affiliation(s)
- Stephen E Villa
- University of California, Los Angeles, David Geffen School of Medicine, Department of Emergency Medicine, Los Angeles, California
| | - Natasha Wheaton
- University of California, Los Angeles, David Geffen School of Medicine, Department of Emergency Medicine, Los Angeles, California
| | - Steven Lai
- University of California, Los Angeles, David Geffen School of Medicine, Department of Emergency Medicine, Los Angeles, California
| | - Jaime Jordan
- University of California, Los Angeles, David Geffen School of Medicine, Department of Emergency Medicine, Los Angeles, California
| |
Collapse
|
6
|
Burns CL, Taubert ST, Ward EC, McCarthy KA, Graham N. Speech-language therapists' perceptions of an eLearning program to support training in videofluoroscopic swallow studies. INTERNATIONAL JOURNAL OF LANGUAGE & COMMUNICATION DISORDERS 2021; 56:257-270. [PMID: 33459451 DOI: 10.1111/1460-6984.12594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2020] [Revised: 11/12/2020] [Accepted: 12/09/2020] [Indexed: 06/12/2023]
Abstract
BACKGROUND Speech-language therapists (SLTs) seek a range of educational opportunities for training in adult videofluoroscopic swallow studies (VFSS). However, variable training methods and/or unequal access to training can influence VFSS practice. AIMS To document current SLT needs and barriers to VFSS training and to determine if a new beginner-level VFSS eLearning program would assist to meet their training needs. The program incorporated multimedia modules on preparing, conducting, interpreting and reporting VFSS. METHODS & PROCEDURES SLTs with limited experience in adult VFSS completed surveys relating to VFSS training experience and barriers, and perceived changes in knowledge, skills and confidence on core VFSS module topics pre- (n = 36) and post- (n = 32) eLearning training. OUTCOMES & RESULTS Inconsistent access to VFSS training opportunities and time-related work pressures were reported as the greatest training barriers. SLTs viewed the eLearning program as a suitable option for VFSS training. Post-training, participants perceived they gained confidence, as well as improved knowledge and skills in all VFSS aspects along with generalised benefits for dysphagia management. SLTs indicated that key benefits of the eLearning program were its comprehensive content and self-directed learning with multimedia tools, which afforded theoretical and practical learning opportunities. CONCLUSIONS & IMPLICATIONS The eLearning program offered SLTs free access to beginner-level adult VFSS training, meeting many identified training needs and providing a foundation from which to develop further practical knowledge and skills within a VFSS clinic setting. What this paper adds What is already known on the subject SLTs demonstrate variable knowledge and skill in conducting and interpreting VFSS, which can impact dysphagia diagnosis and management. While access to VFSS training can be challenging, the barriers to training for SLTs have not been clearly documented. Research has confirmed that eLearning can be used effectively in healthcare education, and in some aspects of VFSS training; however, it is yet to be applied to address the broad range of VFSS training needs. What this paper adds to existing knowledge This study describes the SLT reported barriers to VFSS training which include limited access to formal and practical training, workload-related time pressures and the complexity of learning the VFSS skill set. The findings highlight that an eLearning program, was an accepted mode of learning for VFSS training. SLTs reported the online program met their learning needs by improving access to training, the multimedia program features supported their understanding of complex anatomical and physiological concepts, and training frameworks assisted their clinical reasoning and VFSS interpretation. What are the potential or actual clinical implications of this work? eLearning can assist in overcoming many VFSS training barriers identified by SLTs and the multimedia aspects of eLearning can effectively support VFSS beginner-level education to complement and expedite in-clinic practical training. Given that VFSS results inform decisions regarding commencement and progression of oral intake and swallow rehabilitation, enhanced VFSS training has the potential to positively influence dysphagia outcomes and quality of life.
Collapse
Affiliation(s)
- Clare L Burns
- Royal Brisbane & Women's Hospital, Metro North Hospital and Health Service, Brisbane, QLD, Australia
- School of Health & Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, Australia
- Centre for Research in Telerehabilitation, The University of Queensland, Brisbane, QLD, Australia
| | - Shana T Taubert
- Royal Brisbane & Women's Hospital, Metro North Hospital and Health Service, Brisbane, QLD, Australia
- School of Health & Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, Australia
| | - Elizabeth C Ward
- School of Health & Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, Australia
- Centre for Research in Telerehabilitation, The University of Queensland, Brisbane, QLD, Australia
- Centre for Functioning and Health Research, Metro South Hospital and Health Service, Brisbane, QLD, Australia
| | - Kellie A McCarthy
- Princess Alexandra Hospital, Metro South Hospital and Health Service, Brisbane, QLD, Australia
| | - Nicola Graham
- Children's Health Queensland Hospital and Health Service, Brisbane, QLD, Australia
| |
Collapse
|
7
|
Żurek M, Rzepakowska A, Osuch-Wójcikiewicz E, Niemczyk K. Learning curve for endoscopic evaluation of vocal folds lesions with narrow band imaging. Braz J Otorhinolaryngol 2019; 85:753-759. [PMID: 30126768 PMCID: PMC9443050 DOI: 10.1016/j.bjorl.2018.07.003] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2018] [Revised: 06/16/2018] [Accepted: 07/10/2018] [Indexed: 11/29/2022] Open
Abstract
Introduction The endoscopic methods are progressing and becoming more common in routine clinical diagnosis in the field of otorhinolaryngology. Relatively large amount of researches have proved high accuracy of narrow band imaging endoscopy in differentiating benign and malignant lesions within vocal folds. However, little is known about learning curve in narrow band imaging evaluation of laryngeal lesions. Objective The aim of this study was to determine the learning curve for the narrow band imaging evaluation of vocal folds pathologies depending on the duration of the procedure. Methods Records of 134 narrow band imaging that were analyzed in terms of the duration of the procedure and the accuracy of diagnosis confirmed by histopathological diagnosis were enrolled in the study. The narrow band imaging examinations were performed sequentially by one investigator over a period of 18 months. Results The average duration of narrow band imaging recordings was 127.82 s. All 134 studies were divided into subsequent series of several elements. An evident decrease in time of investigation was noticed between 13th and 14th series, when the examinations were divided into 5 elements series, which corresponds to the difference between 65th and 70th subsequent narrow band imaging examination. Parallel groups of 67 examinations were created. Group 1 included 1st to 67th subsequent narrow band imaging examination; Group 2 – 68th to 134th narrow band imaging examinations. The non-parametric U Mann–Whitney test confirmed statistically significant difference between the mean duration of narrow band imaging examination in both groups 160.5 s and 95.1 s, respectively (p < 10−7). Sensitivity and specificity of narrow band imaging examination in the first group were respectively: 83.7% and 76.7%. In the second group, these indicators amounted 98.1% and 80% respectively. Conclusions A minimum of 65th–70th narrow band imaging examinations are required to reach a plateau phase of the learning process in assessment of glottis lesions. Analysis of learning curves is useful for the development of training programs and determination of a mastery level.
Collapse
Affiliation(s)
- Michał Żurek
- Medical University of Warsaw, Students Scientific Research Group by Otolaryngology Department, Warszawa, Poland
| | - Anna Rzepakowska
- Medical University of Warsaw, Otolaryngology Department, Warszawa, Poland.
| | | | - Kazimierz Niemczyk
- Medical University of Warsaw, Otolaryngology Department, Warszawa, Poland
| |
Collapse
|