Development of a Standardized Assessment of Simulation-based Extracorporeal Membrane Oxygenation Educational Courses

Developing a generalizable pediatric ECMO emergency checklist for clinical specialist: Progress and challenges

INTRODUCTION

Extracorporeal membrane oxygenation (ECMO) emergencies require skilled clinical specialist (CS) who manage ECMO circuits. While tools for assessing CS skills have been published, there is significant variation in protocols and circuit design. This study aims to further develop these checklists to produce a generalizable ECMO skill assessment with adequate validity evidence to support its use as a summative evaluation tool.

METHODS

An initial survey determined variation in ECMO circuit components and configurations, and the original checklists and simulations were altered through a modified Delphi process. The finalized checklist and simulation were then assessed for validity and reliability. Three trained raters assessed ten simulations from five subjects at two different institutions using two circuit designs. Data analysis was conducted using a fully crossed subject x rater x circuit generalizability (G) and decision (D) study.

RESULTS

The G-study coefficient was 0 with 0% variance across subject and circuit. The greatest variance was among raters (28.7%). Significant variance was also associated with the subject and pump type relationship (27%).

CONCLUSION

Despite the rigorous process used to modify the assessment, generalizability was poor. Lack of familiarity with center-specific circuit design played a key role. Future endeavors in ECMO skill assessment should focus either on developing and validating site-specific tools or standardizing circuit designs.

A quality improvement initiative to increase adult ECMO decision-making abilities in a perfusion education program: The use of 3D ECMO simulation

INTRODUCTION

Adult and pediatric ECMO procedures have been increasingly established as conventional life-saving modalities in critical care services across the world. Since 2017, a multidisciplinary team of program advisors for our perfusion education program have aimed to increase cardiovascular perfusion (CVP) student ECMO exposure and improve clinical decision-making. In this QI intervention, the use of 3D computer-based simulation was assessed in establishing a standardized process to improve the diagnosis and treatment of adult ECMO complications among first year CVP students.

METHODS

The Califia 3D Patient Simulator was incorporated into the curriculum for first year CVP students (n = 26) along with traditional lecture for the adult ECMO complication laboratory session. Pre-class knowledge assessments using de-identified polling software were compared to post-class assessments following the first assigned learning activity. Assessments from students that received simulation before lecture (SIM, n = 15) were compared to students receiving lecture before simulation (LEC, n = 11). User experience questionnaires (UEQ) consisting of 26 questions for six scales of simulation instruction were administered to measure the comprehensive impression of the student experience.

RESULTS

Overall median [IQR] pre- and -post knowledge assessment scores were 74% [11] and 84% [11], respectively (p = 0.01). There were no significant differences in pre-class assessment scores between the SIM and LEC groups (74.0% and 74.0%, respectively, p = 0.959). The LEC group achieved higher median post-assessment scores than the SIM group (84% vs 79%, p = 0.032). Among the 26 UEQ survey scales, 23 were positively evaluated (>0.8), and three were a neutral evaluation (-0.8 to 0.8). Cronbach Alpha-Coefficients of >0.78 were measured for attractiveness, perspicuity, efficacy, and stimulation. The coefficient for dependability was 0.37. 25 (96.2%) students indicated that 3D simulation was beneficial to improving ECMO clinical decision-making.

CONCLUSIONS

In this QI intervention, the implementation of computer-based 3D simulation following lecture was perceived by learners to help improve the diagnosis and treatment of ECMO-related complications.