Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy

Use of Low-Cost Task Trainer for Emergency Department Thoracotomy Training in General Surgery Residency Program

OBJECTIVE

Emergency department thoracotomy (EDT) is an uncommon but potentially lifesaving procedure that warrants familiarity with anatomy, instruments, and indications necessary for completion. To address this need, we developed a low-cost EDT trainer. The primary objective of this study was to compare the effectiveness of a low-cost EDT trainer to teach emergency department thoracotomy with a discussion-based teaching session. Secondary objective was to study the face validity of the low-cost EDT trainer.

DESIGN

A prospective 2-phase randomized control study was conducted. Participants were randomly divided into two groups. In phase one, baseline medical knowledge for both groups was assessed using a multiple-choice question pretest. In Group 1, each participant was taught EDT using a one-on-one discussion with a trauma surgeon, whereas Group 2 used the EDT trainer and debriefing for training. In phase 2 (1 month later), all participants completed a knowledge retention test and performed a videoed EDT using our EDT trainer, the video recordings were later reviewed by content experts blinded to the study participants using a checklist with a maximum score of 22. The participants also completed a reaction survey at the end of phase 2 of the study.

SETTING

OhioHealth Riverside Methodist Hospital, an urban tertiary care academic hospital in Columbus, Ohio.

PARTICIPANTS

Nine senior surgery residents from training years 3 to 5.

RESULTS

The mean score for the performance of the procedure for the simulation-based (Group 2) was significantly higher than that of the discussion-based (Group 1) (Rater 1: 21.2 ± 0.8 vs. 19.0 ± 2.0, p = 0.05, Rater 2: 20.4 ± 1.5 vs. 18.3±1.0, p = 0.04). Group 2 also was quicker than Group 1 in deciding to start the procedure by approximately 56 seconds. When comparing the mean pretest knowledge score to the mean knowledge retention score 30 days after training, the discussion-based group improved from 58.33% to 81.25% (p = 0.01); the simulation-trained group's scores remained at 68.33%. All the participants agreed or strongly agreed that the simulator provided a realistic opportunity to perform EDT and improved their confidence.

CONCLUSIONS

The results of this pilot study support our hypothesis that using a low-cost EDT trainer effectively improves general surgery residents' confidence and procedural skills scores in a simulated environment. Further training with low-cost simulators may provide surgical residents with deliberate practice opportunities and improve performance when learning low-frequency procedures.

Needs Assessment for a Resuscitative Thoracotomy Curriculum for General Surgery Residents in the Northeast Region

INTRODUCTION

Resuscitative thoracotomy (RT) is a high-acuity low occurrence (HALO) procedure with which general surgical resident (GSR) experience and confidence are unknown. We sought to identify and describe this educational gap by conducting a targeted needs assessment for an RT curriculum for GSRs.

METHODS

An online regional needs assessment survey was conducted for an RT curriculum for GSRs. The survey was developed by a group of trauma stakeholders and revised after being piloted on a small, representative group of GSRs. We surveyed GSRs in the Northeast region regarding their experience and confidence with RT; interest in an RT curriculum; and content, format, and scope for an RT curriculum.

RESULTS

The survey response rate was 43%, reflecting the viewpoints of GSRs at 8 major training centers across the Northeast. Only 13% of respondents were interested in pursuing a career in Trauma and Critical Care despite 97% of them training at a Level I Trauma Center. Twenty-nine percent and 33% of GSRs had ever assisted with or performed RT, respectively. Twenty-one percent of GSRs reported feeling confident performing RT. Most respondents (98%) agreed or strongly agreed that an RT curriculum would add value to their general surgery education. The most positively rated content topics were resuscitative maneuvers (100% positive responses [PR]), when to cease resuscitative efforts (100% PR), and morbidity and mortality associated with RT (98% PR). The most highly rated learning methods were individual RT simulation time (97% PR) and a tour of the trauma bay equipment (97% PR).

CONCLUSIONS

This needs assessment demonstrates a lack of experience and confidence with RT, a strong learner interest in an RT curriculum, and a desire for experiential learning methods. Learning objectives are defined herein, and the next steps involve developing educational materials for an RT curriculum for GSRs.

Simulation-based Mastery Learning Improves Emergency Medicine Residents' Ability to Perform Temporary Transvenous Cardiac Pacing

INTRODUCTION

Temporary transvenous cardiac pacing (TVP) is a critical intervention that emergency physicians perform infrequently in clinical practice. Prior simulation studies revealed that emergency medicine (EM) residents and board-certified emergency physicians perform TVP poorly during checklist-based assessments. Our objective in this report was to describe the design and implementation of a simulation-based mastery learning (SBML) curriculum and evaluate its impact on EM residents' ability to perform TVP.

METHODS

An expert panel of emergency physicians and cardiologists set a minimum passing standard (MPS) for a previously developed 30-item TVP checklist using the Mastery Angoff approach. Emergency medicine residents were assessed using this checklist and a high-fidelity TVP task trainer. Residents who did not meet the MPS during baseline testing viewed a procedure video and completed a 30-minute individual deliberate practice session before retesting. Residents who did not meet the MPS during initial post-testing completed additional deliberate practice and assessment until meeting or exceeding the MPS.

RESULTS

The expert panel set an MPS of correctly performing 28 (93.3%) checklist items. Fifty-seven EM residents participated. Mean checklist scores improved from 13.4 (95% CI 11.8-15.0) during baseline testing to 27.5 (95% CI 26.9-28.1) during initial post-testing (P < 0.01). No residents met the MPS at baseline testing. The 21 (36.8%) residents who did not meet the MPS during initial post-testing all met or exceeded the MPS after completing one additional 30-minute deliberate practice session.

CONCLUSION

Emergency medicine residents demonstrated significantly improved TVP performance with reduced variability in checklist scores after completing a simulation-based mastery learning curriculum.

Development of an expert consensus checklist for emergency ultrasound

BACKGROUND

Within today's competency-based medical education, traditional set number proficiency benchmarks have been called into question. Checklists may help guide individualized training and standardized outcomes. However, multicenter expert consensus checklists based on established guidelines with supporting validity evidence have not been published for specific emergency ultrasound (EUS) applications. We describe a robust national EUS expert consensus methodology for developing a checklist for the extended focused assessment with sonography in trauma (eFAST examination).

METHODS

Utilizing the ACEP imaging compendium as a primary reference, 10 national EUS experts iteratively refined and agreed upon a final checklist. To obtain initial reliability and validity evidence, two different EUS experts blinded to resident experience then assessed 24 residents performing an eFAST in a simulated environment. Inter-rater reliability of the checklist was assessed using Cohen's kappa coefficient. Validity was assessed by comparing mean performance with the Student's t-test and discriminant ability of individual checklist items using item-total correlation.

RESULTS

The 10 EUS experts agreed on the final checklist items after two rounds of iterations. When evaluating 24 emergency medicine (EM) PGY-1 to -4 residents, the kappa correlation between two blinded EUS faculty raters was moderate at 0.670. Kappa and agreement were near-perfect or perfect in right and left chest image optimization, right upper quadrant (RUQ) probe placement, RUQ anatomy identification, and pelvic first-view anatomy identification. The difference in checklist performance between junior and senior EM residents was significant at -8.1% (p = 0.004). Identification of pelvic structures and placement of the probe for pelvic views were found to have an excellent item-total correlation with values of >0.4.

CONCLUSIONS

We have described a robust national EUS expert consensus methodology for developing an eFAST checklist based on the ACEP imaging guidelines. Based on this encouraging initial reliability and validity evidence, further research and checklist development is warranted for additional EUS applications.

Standard Setting in Simulation-Based Training of Surgical Procedures: A Systematic Review

OBJECTIVE

This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures.

SUMMARY OF BACKGROUND

Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods.

METHODS

Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE.

RESULTS

Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence.

CONCLUSION

Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.