1
|
Brevik C, Miller D, Kendall J, Michael S. Nontechnically speaking: A review of tools and methods in the teaching and assessment of nontechnical skills in emergency medicine training. AEM EDUCATION AND TRAINING 2023; 7:e10911. [PMID: 37974662 PMCID: PMC10641174 DOI: 10.1002/aet2.10911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Revised: 08/14/2023] [Accepted: 08/29/2023] [Indexed: 11/19/2023]
Affiliation(s)
- Cody Brevik
- Department of Emergency MedicineUniversity of Colorado School of Medicine, Anschutz Medical CampusAuroraColoradoUSA
| | - Danielle Miller
- Department of Emergency MedicineUniversity of Colorado School of Medicine, Anschutz Medical CampusAuroraColoradoUSA
| | - John Kendall
- Department of Emergency MedicineUniversity of Colorado School of Medicine, Anschutz Medical CampusAuroraColoradoUSA
| | - Sarah Michael
- Department of Emergency MedicineUniversity of Colorado School of Medicine, Anschutz Medical CampusAuroraColoradoUSA
| |
Collapse
|
2
|
Gore KM, Schiebout J, Peksa GD, Hock S, Patwari R, Gottlieb M. The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents. Clin Exp Emerg Med 2023; 10:306-314. [PMID: 36796780 PMCID: PMC10579731 DOI: 10.15441/ceem.22.395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Revised: 01/19/2023] [Accepted: 02/07/2023] [Indexed: 02/18/2023] Open
Abstract
OBJECTIVE Feedback is critical to the growth of learners. However, feedback quality can be variable in practice. Most feedback tools are generic, with few targeting emergency medicine. We created a feedback tool designed for emergency medicine residents, and this study aimed to evaluate the effectiveness of this tool. METHODS This was a single-center, prospective cohort study comparing feedback quality before and after introducing a novel feedback tool. Residents and faculty completed a survey after each shift assessing feedback quality, feedback time, and the number of feedback episodes. Feedback quality was assessed using a composite score from seven questions, which were each scored 1 to 5 points (minimum total score, 7 points; maximum, 35 points). Preintervention and postintervention data were analyzed using a mixed-effects model that took into account the correlation of random effects between study participants. RESULTS Residents completed 182 surveys and faculty members completed 158 surveys. The use of the tool was associated with improved consistency in the summative score of effective feedback attributes as assessed by residents (P=0.040) but not by faculty (P=0.259). However, most of the individual scores for attributes of good feedback did not reach statistical significance. With the tool, residents perceived that faculty spent more time providing feedback (P=0.040) and that the delivery of feedback was more ongoing throughout the shift (P=0.020). Faculty felt that the tool allowed for more ongoing feedback (P=0.002), with no perceived increase in the time spent delivering feedback (P=0.833). CONCLUSION The use of a dedicated tool may help educators provide more meaningful and frequent feedback without impacting the perceived required time needed to provide feedback.
Collapse
Affiliation(s)
- Katarzyna M. Gore
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Jessen Schiebout
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Gary D. Peksa
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Sara Hock
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Rahul Patwari
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Michael Gottlieb
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| |
Collapse
|
3
|
Gottlieb M, Jordan J, Krzyzaniak S, Mannix A, King A, Cooney R, Fix M, Shappell E. Trends in emergency medicine resident procedural reporting over a 10-year period. AEM EDUCATION AND TRAINING 2023; 7:e10841. [PMID: 36777101 PMCID: PMC9899625 DOI: 10.1002/aet2.10841] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 12/02/2022] [Accepted: 12/09/2022] [Indexed: 06/18/2023]
Abstract
Background Procedural competency is expected of all emergency medicine (EM) residents upon graduation. The ACGME requires a minimum number of essential procedures to successfully complete training. However, data are limited on the actual number of procedures residents perform and prior studies are limited to single institutions over short time periods. This study sought to assess the number of Key Index Procedures completed during EM residency training and evaluate trends over time. Methods We conducted a retrospective review of graduating EM resident procedure logs across eight ACGME accredited residency programs over the last 10 years (2013-2022). Sites were selected to ensure diversity of program length, program type, and geography. All data from EM residents graduating in 2013-2022 were eligible for inclusion. Data from residents from combined training programs, those who did not complete their full training at that institution (i.e., transferred in/out), or those who did not have data available were excluded. We determined the list of procedures based upon the ACGME Key Index Procedures list. Sites obtained totals for each of the identified procedures for each resident upon graduation. We calculated the mean and 95% CI for each procedure. Results We collected data from a total of 914 residents, with 881 (96.4%) meeting inclusion criteria. The most common procedures were point-of-care ultrasound, adult medical resuscitation, adult trauma resuscitation, and intubation. The least frequent procedures included pericardiocentesis, cricothyroidotomy, cardiac pacing, vaginal delivery, and chest tubes. Most procedures were stable over time with the exception of lumbar punctures (decreased) and point-of-care ultrasound (increased). Conclusions In a national sample of EM programs, procedural numbers remained stable except for lumbar puncture and ultrasound. This information can inform residency training curricula and accreditation requirements.
Collapse
Affiliation(s)
- Michael Gottlieb
- Department of Emergency MedicineRush University Medical CenterChicagoIllinoisUSA
| | - Jaime Jordan
- Department of Emergency MedicineDavid Geffen School of Medicine at UCLALos AngelesCaliforniaUSA
| | - Sara Krzyzaniak
- Department of Emergency MedicineStanford UniversityPalo AltoCaliforniaUSA
| | - Alexandra Mannix
- Department of Emergency MedicineUniversity of Florida College of Medicine – JacksonvilleJacksonvilleFloridaUSA
| | - Andrew King
- Department of Emergency MedicineThe Ohio State University Wexner Medical CenterColumbusOhioUSA
| | - Robert Cooney
- Department of Emergency MedicineGeisinger Medical CenterDanvillePennsylvaniaUSA
| | - Megan Fix
- Department of Emergency MedicineUniversity of Utah HospitalSalt Lake CityUtahUSA
| | - Eric Shappell
- Department of Emergency MedicineMassachusetts General Hospital/Harvard Medical SchoolBostonMassachusettsUSA
| |
Collapse
|
4
|
Robinson TJG, Wagner N, Szulewski A, Dudek N, Cheung WJ, Hall AK. Exploring the use of rating scales with entrustment anchors in workplace-based assessment. MEDICAL EDUCATION 2021; 55:1047-1055. [PMID: 34060651 DOI: 10.1111/medu.14573] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 04/07/2021] [Accepted: 05/26/2021] [Indexed: 06/12/2023]
Abstract
PURPOSE Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. METHOD A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes. RESULTS Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not. CONCLUSIONS By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.
Collapse
Affiliation(s)
| | - Natalie Wagner
- Department of Biomedical & Molecular Sciences, Queen's University, Kingston, ON, Canada
- Office of Professional Development & Educational Scholarship, Queen's University, Kingston, ON, Canada
| | - Adam Szulewski
- Department of Emergency Medicine, Queen's University, Kingston, ON, Canada
- Department of Psychology, Queen's University, Kingston, ON, Canada
| | - Nancy Dudek
- Department of Medicine and The Ottawa Hospital, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Warren J Cheung
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, Queen's University, Kingston, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| |
Collapse
|
5
|
Miller DT, Krzyzaniak S, Mannix A, Alvarez A, Chan T, Davenport D, Eraso D, Foote CJ, Gore K, Parsons M, Gottlieb M. The standardized letter of evaluation in emergency medicine: Are the qualifications useful? AEM EDUCATION AND TRAINING 2021; 5:e10607. [PMID: 34222747 PMCID: PMC8194319 DOI: 10.1002/aet2.10607] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 04/26/2021] [Accepted: 04/29/2021] [Indexed: 06/03/2023]
Abstract
OBJECTIVES The standardized letter of evaluation (SLOE) in emergency medicine (EM) is a widely used metric for determining interview invitations and ranking of candidates. Previous research has questioned the validity of certain sections of the SLOE. However, there remains a paucity of literature on the qualifications for EM section, which evaluates seven attributes of applicants. The aim of this study was to determine the correlation between the qualifications questions and grades, global assessment, and anticipated rank list position for EM applicants. METHODS A multi-institutional cross-sectional study was performed using SLOEs from applicants to three geographically distinct U.S. EM residency programs during the 2019-2020 application cycle. We abstracted EM rotation grade, qualifications scores, global assessment, and anticipated rank list position from the SLOEs. A Spearman correlation was calculated between each of the qualifications scores and the applicant's grades, global assessment, and anticipated rank list position in a pairwise fashion. RESULTS In total, 2,106 unique applicants (4,939 SLOEs) were included. Of the seven qualifications for EM questions, three were moderately to strongly correlated with global assessment and anticipated rank list position: "ability to develop and justify an appropriate differential and a cohesive treatment plan" (ρ = 0.65 and ρ = 0.63, respectively; p < 0.001), "how much guidance do you predict this applicant will need during residency?" (ρ = 0.68 and ρ = 0.68, respectively; p < 0.001), and "what is your prediction of success for the applicant?" (ρ = 0.69 and ρ = 0.69, respectively; p < 0.001). There was no strong correlation between the seven qualifications and grades. CONCLUSIONS There was a moderate to strong correlation between three of seven qualifications for EM questions (ability to develop and justify a differential and develop a cohesive plan, anticipated need for the amount of guidance, and prediction of success) with both global assessment and anticipated rank list position, suggesting that these qualifications may provide the most useful data to residency selection while some of the other factors may not be needed.
Collapse
Affiliation(s)
- Danielle T. Miller
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Sara Krzyzaniak
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Alexandra Mannix
- Department of Emergency MedicineUniversity of Florida College of Medicine JacksonvilleJacksonvilleFloridaUSA
| | - Al’ai Alvarez
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Teresa Chan
- Department of Emergency MedicineMcMaster UniversityHamiltonOntarioCanada
| | - Dayle Davenport
- Department of Emergency MedicineRush University Medical CenterChicagoIllinoisUSA
| | - Daniel Eraso
- Department of Emergency MedicineUniversity of Florida College of Medicine JacksonvilleJacksonvilleFloridaUSA
| | | | - Katarzyna Gore
- Department of Emergency MedicineRush University Medical CenterChicagoIllinoisUSA
| | - Melissa Parsons
- Department of Emergency MedicineUniversity of Florida College of Medicine JacksonvilleJacksonvilleFloridaUSA
| | - Michael Gottlieb
- Department of Emergency MedicineRush University Medical CenterChicagoIllinoisUSA
| |
Collapse
|
6
|
Chan TM, Sebok‐Syer SS, Cheung WJ, Pusic M, Stehman C, Gottlieb M. Workplace-based Assessment Data in Emergency Medicine: A Scoping Review of the Literature. AEM EDUCATION AND TRAINING 2021; 5:e10544. [PMID: 34099992 PMCID: PMC8166307 DOI: 10.1002/aet2.10544] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/04/2020] [Revised: 10/02/2020] [Accepted: 10/05/2020] [Indexed: 06/01/2023]
Abstract
OBJECTIVE In the era of competency-based medical education (CBME), the collection of more and more trainee data is being mandated by accrediting bodies such as the Accreditation Council for Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada. However, few efforts have been made to synthesize the literature around the current issues surrounding workplace-based assessment (WBA) data. This scoping review seeks to synthesize the landscape of literature on the topic of data collection and utilization for trainees' WBAs in emergency medicine (EM). METHODS The authors conducted a scoping review in the style of Arksey and O'Malley, seeking to synthesize and map literature on collecting, aggregating, and reporting WBA data. The authors extracted, mapped, and synthesized literature that describes, supports, and substantiates effective data collection and utilization in the context of the CBME movement within EM. RESULTS Our literature search retrieved 189 potentially relevant references (after removing duplicates) that were screened to 29 abstracts and papers relevant to collecting, aggregating, and reporting WBAs. Our analysis shows that there is an increasing temporal trend toward contributions in these topics, with the majority of the papers (16/29) being published in the past 3 years alone. CONCLUSION There is increasing interest in the areas around data collection and utilization in the age of CBME. The field, however, is only beginning to emerge, leaving more work that can and should be done in this area.
Collapse
Affiliation(s)
- Teresa M. Chan
- From theDepartment of MedicineDivision of Emergency Medicine and the Division of Education & InnovationMcMaster UniversityHamiltonOntarioCanada
- theProgram for Faculty DevelopmentFaculty of Health SciencesMcMaster UniversityHamiltonOntarioCanada
- and the McMaster Program for Education Research, Innovation, and TheoryMcMaster UniversityHamiltonOntarioCanada
| | | | - Warren J. Cheung
- theDepartment of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| | - Martin Pusic
- theDepartment of PediatricsHarvard Medical SchoolBostonMAUSA
| | | | - Michael Gottlieb
- and theDepartment of Emergency MedicineRush University Medical CenterChicagoILUSA
| |
Collapse
|
7
|
Patel KA, Sexton A, Shah M, Hexom B, Gottlieb M. More Than Meets the Eye: Addressing the Role of Telemedicine in Resident Education. Ann Emerg Med 2021; 78:429-433. [PMID: 33781605 DOI: 10.1016/j.annemergmed.2021.01.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Indexed: 01/08/2023]
Affiliation(s)
- Keya A Patel
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL.
| | - Andrew Sexton
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL
| | - Meeta Shah
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL
| | - Braden Hexom
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL
| | - Michael Gottlieb
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL
| |
Collapse
|