Simulation Fellowships: Survey of Current Summative Assessment Practices

Assessing Healthcare Simulation Facilitation: A Scoping Review of Available Tools, Validity Evidence, and Context Suitability for Faculty Development in Low-Resource Settings

SUMMARY STATEMENT

Assessment tools support simulation facilitation skill development by guiding practice, structuring feedback, and promoting reflective learning among educators. This scoping review followed a systematic process to identify facilitation assessment tools used in postlicensure healthcare simulation. Secondary objectives included mapping of the validity evidence to support their use and a critical appraisal of their suitability for simulation faculty development in low-resource settings. Database searching, gray literature searching, and stakeholder engagement identified 11,568 sources for screening, of which 72 met criteria for full text review. Thirty sources met inclusion; 16 unique tools were identified. Tools exclusively originated from simulation practice in high-resource settings and predominantly focused on debriefing. Many tools have limited validity evidence supporting their use. In particular, the validity evidence supporting the extrapolation and implications of assessment is lacking. No current tool has high context suitability for use in low-resource settings.

Impact of the PEARLS Healthcare Debriefing cognitive aid on facilitator cognitive load, workload, and debriefing quality: a pilot study

BACKGROUND

The Promoting Excellence and Reflective Learning in Simulation (PEARLS) Healthcare Debriefing Tool is a cognitive aid designed to deploy debriefing in a structured way. The tool has the potential to increase the facilitator's ability to acquire debriefing skills, by breaking down the complexity of debriefing and thereby improving the quality of a novice facilitator's debrief. In this pilot study, we aimed to evaluate the impact of the tool on facilitators' cognitive load, workload, and debriefing quality.

METHODS

Fourteen fellows from the New York City Health + Hospitals Simulation Fellowship, novice to the PEARLS Healthcare Debriefing Tool, were randomized to two groups of 7. The intervention group was equipped with the cognitive aid while the control group did not use the tool. Both groups had undergone an 8-h debriefing course. The two groups performed debriefings of 3 videoed simulated events and rated the cognitive load and workload of their experience using the Paas-Merriënboer scale and the raw National Aeronautics and Space Administration task load index (NASA-TLX), respectively. The debriefing performances were then rated using the Debriefing Assessment for Simulation in Healthcare (DASH) for debriefing quality. Measures of cognitive load were measured as Paas-Merriënboer scale and compared using Wilcoxon rank-sum tests. Measures of workload and debriefing quality were analyzed using mixed-effect linear regression models.

RESULTS

Those who used the tool had significantly lower median scores in cognitive load in 2 out of the 3 debriefings (median score with tool vs no tool: scenario A 6 vs 6, p=0.1331; scenario B: 5 vs 6, p=0.043; and scenario C: 5 vs 7, p=0.031). No difference was detected in the tool effectiveness in decreasing composite score of workload demands (mean difference in average NASA-TLX -4.5, 95%CI -16.5 to 7.0, p=0.456) or improving composite scores of debriefing qualities (mean difference in DASH 2.4, 95%CI -3.4 to 8.1, p=0.436).

CONCLUSIONS

The PEARLS Healthcare Debriefing Tool may serve as an educational adjunct for debriefing skill acquisition. The use of a debriefing cognitive aid may decrease the cognitive load of debriefing but did not suggest an impact on the workload or quality of debriefing in novice debriefers. Further research is recommended to study the efficacy of the cognitive aid beyond this pilot; however, the design of this research may serve as a model for future exploration of the quality of debriefing.

A Decade Later-Progress and Next Steps for Pediatric Simulation Research

SUMMARY STATEMENT

A decade ago, at the time of formation of the International Network for Pediatric Simulation-based Innovation, Research, and Education, the group embarked on a consensus building exercise. The goal was to forecast the facilitators and barriers to growth and maturity of science in the field of pediatric simulation-based research. This exercise produced 6 domains critical to progress in the field: (1) prioritization, (2) research methodology and outcomes, (3) academic collaboration, (4) integration/implementation/sustainability, (5) technology, and (6) resources/support/advocacy. This article reflects on and summarizes a decade of progress in the field of pediatric simulation research and suggests next steps in each domain as we look forward, including lessons learned by our collaborative grass roots network that can be used to accelerate research efforts in other domains within healthcare simulation science.

Development of a simulation technical competence curriculum for medical simulation fellows

Background and needs

Medical educators with simulation fellowship training have a unique skill set. Simulation fellowship graduates have the ability to handle basic and common troubleshooting issues with simulation software, hardware, and equipment setup. Outside of formal training programs such as this, simulation skills are inconsistently taught and organically learned. This is important to address because there are high expectations of medical educators who complete simulation fellowships. To fill the gap, we offer one way of teaching and assessing simulation technical skills within a fellowship curriculum and reflect on lessons learned throughout the process. This report describes the instructional designs, implementation, and program evaluation of an educational intervention: a simulation technology curriculum for simulation fellows.

Curriculum design

The current iteration of the simulation technical skill curriculum was introduced in 2018 and took approximately 8 months to develop under the guidance of expert simulation technology specialists, simulation fellowship-trained faculty, and simulation center administrators. Kern’s six steps to curriculum development was used as the guiding conceptual framework. The curriculum was categorized into four domains, which emerged from the outcome of a qualitative needs assessment. Instructional sessions occurred on 5 days spanning a 2-week block. The final session concluded with summative testing.

Program evaluation

Fellows were administered summative objective structured exams at three stations. The performance was rated by instructors using station-specific checklists. Scores approached 100% accuracy/completion for all stations.

Conclusions

The development of an evidence-based educational intervention, a simulation technical skill curriculum, was highly regarded by participants and demonstrated effective training of the simulation fellows. This curriculum serves as a template for other simulationists to implement formal training in simulation technical skills.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41077-022-00221-4.

A Scoping Literature Review of Simulation Training Program Curriculum Standards

SUMMARY STATEMENT

Simulation educator training is well supported in the literature and endorsed by the Society of Simulation in Healthcare as well as the International Nursing Association for Clinical Simulation and Learning. Despite growth of domestic and international training programs, there is a lack of consensus regarding curriculum standards. Our aim was to identify core curricular components of comprehensive simulation training programs. A scoping literature review of all relevant publications from 2000 to 2020 was conducted using a 6-step design. A team of 10 multidisciplinary, international simulation educators independently reviewed all citations with discrepancies resolved by third-person review. Of the initial 320 identified unique publications, a total of 15 articles were included, all published within the last 6 years. Four themes were identified: domains (n = 6), competencies (n = 3), objectives (n = 8), and other characteristics (n = 3). The findings support a greater understanding of the core curricular content across simulation training programs to support standardization.

Accreditation of Simulation Fellowships and Training Programs: More Checkboxes or Elevating the Field?

SUMMARY STATEMENT

As the field of healthcare simulation matures, formal accreditation for simulation fellowships and training programs has become increasingly available and touted as a solution to standardize the education of those specializing in healthcare simulation. Some simulation experts hold opposing views regarding the potential value of simulation fellowship program accreditation. We report on the proceedings of a spirited debate at the 20th International Meeting on Simulation in Healthcare in January 2020. Pro arguments view accreditation as the logical evolution of a maturing profession: improving training quality through standard setting, providing external validation for individual programs, and enhancing the program's return on investment. Con arguments view accreditation as an incompletely formulated construct; burdensome to the "financially strapped" fellowship director, misaligned with simulation fellows' needs and expectations, and confusing to administrators mistakenly equating accreditation with credentialing. In addition, opponents of accreditation postulate that incorporating curricular standards, practice guidelines, and strategies derived and implemented without rigor, supporting evidence and universal consensus is premature. This narrative review of our debate compares and contrasts contemporary perspectives on simulation fellowship program accreditation, concluding with formal recommendations for learners, administrators, sponsors, and accrediting bodies.

Fundamental underpinnings of simulation education: describing a four-component instructional design approach to healthcare simulation fellowships

Although in 2020, there are more than 120 healthcare simulation fellowships established globally, there is a paucity of literature on how to design fellowship programs most effectively, to equip graduates with the knowledge, skills, and attitudes of a competent simulation educator. Offering a systematic structure to approach simulation fellowship programmatic design may aid in better achieving program goals. In this manuscript, we present the application of the 4-component instructional design model as a blueprint to the development of Simulation Education Fellowships. We offer examples used at the NYC Health + Hospitals simulation fellowship to illustrate how the 4-component model informs fellowship program design which promotes the development of a simulation educator. This manuscript will provide a roadmap to designing curricula and assessment practices including self-reflective logbooks to focus the path toward achieving desired skills and shape future conversations around programmatic development.

Development and assessment of a simulation-based curriculum in pediatric surgical education: Conventional wisdom and lessons learned from the national training program in France

Traditional training in pediatric surgery has focused on knowledge acquisition and technical skills. Although recognized as important, the skills relating to communication, collaboration, leadership, management, health advocacy and professionalism have received less formal attention. Recognition of the importance of these softer skills in combination with the greater use of simulation in education prompted us to develop a national simulation-based curriculum in pediatric surgery. This paper describes the development, implementation and three-year follow-up of the mandatory, simulation-based curriculum for pediatric surgery residents in France. We review the available literature on the topic and present our results.