Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis

Procedural simulation in venipuncture for medical undergraduates and its transfer to the bedside: a cluster randomized study

Simulation is accepted as an effective method of learning procedural skills. However, the translational outcomes of skills acquired through simulation still warrants investigation. We designed this study to assess if skills laboratory training in addition to bedside learning (intervention group [IG]) would provide better learning results than bedside learning alone (control group [CG]) in the context of venipuncture training. This prospective, cluster-randomized, single-blind study took place at the Faculty of Medicine, University of Kelaniya, Sri Lanka. Seventeen clusters of second-year medical students were randomly assigned to either IG or CG. The IG trained on venipuncture in the skills laboratory, receiving instruction after modified Payton's Four Step Method. Following the training, students of both IG and CG underwent bedside learning for one month. Afterward, students of both groups performed venipuncture on actual patients in a clinical setting. An independent, blinded assessor scored students' performance using the Integrated Procedural Protocol Instrument (IPPI) and a checklist. Patients assessed students' performance with the Communication Assessment Tool (CAT). Eight and nine clusters were randomized to the intervention and control groups, respectively. IG completed significantly more single steps of the procedure correctly (IG: 19.36 ± 3.87 for checklist items; CG: 15.57 ± 4.95; p < 0.001). IG also scored significantly better on IPPI ratings (median: IG: 27 (12) vs. CG: 21 (8); p < 0.001). Rated by patients, students' communication skills did not significantly differ between the two groups. Simulation-based venipuncture training enabled students to perform the procedure on actual patients with a higher technical accuracy than students who learned venipuncture at the bedside. Students were able to transfer the skills acquired through venipuncture training at the skill laboratory to the bedside.

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Correlating Simulation Training and Assessment With Clinical Performance: A Feasibility Study

OBJECTIVE

Simulation education and assessment are increasingly used in prehospital curriculums. The objective of this study was to assess the challenges and feasibility of correlating evaluation data from an airway management simulation assessment with clinical performance.

METHODS

This study was undertaken in Ontario, the most populous province in Canada, where 13 bases are distributed in geographically diverse areas, from urban to rural and remote locations. This is a retrospective cohort study of paramedics who had completed simulation education and assessment in rapid sequence intubation. Logistic regression was used to assess for correlation between assessment scores (ie, the global score and the overall score and the definitive airway sans hypoxia/hypotension on the first attempt [DASH-1A] success in the field).

RESULTS

DASH-1A success when grouped by base varied from 25% to 100%. The odds of DASH-1A success increased for paramedics who had a higher overall score (odds ratio [OR]: 1.03; 95% confidence interval [CI], 0.96-1.11) and for paramedics who had a higher global rating (OR: 1.27; CI, 0.73-2.21) when accounting for base intubation frequency. The odds of DASH-1A success increased for paramedics who had a higher overall score (OR: 1.01; CI, 0.93-1.09) and decreased for paramedics who had a higher global rating (OR: 0.96; CI, 0.47-1.96) when accounting for base geography.

CONCLUSION

Although this study lacked a sample size large enough to draw conclusions, it provides a foundation and areas to improve in future work exploring the relationship between simulation assessments and clinical performance.

Innovative Research Methods: Using Simulation to Evaluate Health Care Policy

SUMMARY STATEMENT

Health care policies have the potential to improve patient outcomes, access to care, and reduce health disparities. However, new policy is often tested in the field, where unintended consequences are paid for by patients. In this perspective, we argue that health care simulations, which can elucidate the potential for policy to hinder clinicians' ability to provide high-quality care, are a complement to large-scale policy evaluations in the field.

Executive Summary of the Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR) Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) Guideline for difficult airway management

The Airway section of the Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR), Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) present the Guidelines for the integral management of difficult airway in adult patients. This document provides recommendations based on current scientific evidence, theoretical-educational tools and implementation tools, mainly cognitive aids, applicable to the treatment of the airway in the field of anesthesiology, critical care, emergencies and prehospital medicine. Its principles are focused on the human factors, cognitive processes for decision-making in critical situations and optimization in the progression of the application of strategies to preserve adequate alveolar oxygenation in order to improve safety and quality of care.

Assessing Healthcare Simulation Facilitation: A Scoping Review of Available Tools, Validity Evidence, and Context Suitability for Faculty Development in Low-Resource Settings

SUMMARY STATEMENT

Assessment tools support simulation facilitation skill development by guiding practice, structuring feedback, and promoting reflective learning among educators. This scoping review followed a systematic process to identify facilitation assessment tools used in postlicensure healthcare simulation. Secondary objectives included mapping of the validity evidence to support their use and a critical appraisal of their suitability for simulation faculty development in low-resource settings. Database searching, gray literature searching, and stakeholder engagement identified 11,568 sources for screening, of which 72 met criteria for full text review. Thirty sources met inclusion; 16 unique tools were identified. Tools exclusively originated from simulation practice in high-resource settings and predominantly focused on debriefing. Many tools have limited validity evidence supporting their use. In particular, the validity evidence supporting the extrapolation and implications of assessment is lacking. No current tool has high context suitability for use in low-resource settings.

Are serious games seriously good at preparing students for clinical practice?: A randomized controlled trial

PURPOSE

Serious games (SGs) have great potential for pediatric medical education. This study evaluated the efficacy of a SG in improving learner satisfaction, knowledge, and behavior.

MATERIALS AND METHODS

This was an investigator-blinded randomized controlled trial (RCT) comparing a SG against two controls: (i) adaptive tutorial (AT), and (ii) low-stimulus control (LSC). SG is a highly immersive role-playing game in a virtual hospital. AT delivers interactive web-based lessons. LSC is paper-based clinical practice guidelines. Metropolitan senior medical students at UNSW were eligible. A total of 154 enrolled and were block randomized to one intervention. Participants had access to one intervention for 8 weeks which taught pediatric acute asthma and seizure assessment and management. Satisfaction was assessed with Likert-scale responses to 5 statements and 2 free-text comments. Knowledge was assessed with 10 multiple-choice questions (MCQs). Clinical behavior was assessed during a 30-point simulated clinical management scenario (CMS). Primary analysis was performed on a modified intention-to-treat basis and compared: (1) SG vs. AT; and (2) SG vs. LSC.

RESULTS

A total of 118 participants were included in the primary analysis (modified intention-to-treat model). No significant differences in MCQ results between the SG and control groups. SG group outperformed the LSC group in the CMS, with a moderate effect (score out of 30: 20.8 (3.2) vs. 18.7 (3.2), respectively, d = 0.65 (0.2-1.1), p = 0.005). No statistically significant difference between SG and AT groups in the CMS (score: 20.8 (3.2) vs. 19.8 (3.1), respectively, d = 0.31 (-0.1 to 0.8), p = 0.18). A sensitivity analysis (per-protocol model) was performed with similar outcomes.

CONCLUSIONS

This is the first investigator-blinded RCT assessing the efficacy of a highly immersive SG on learner attitudes, knowledge acquisition, and performance in simulated pediatric clinical scenarios. The SG demonstrated improved translation of knowledge to a simulated clinical environment, particularly compared to LSC. SGs show promise in pediatric medical education.

Simulation-based medical education in Canadian anesthesiology academic institutions: a national survey

PURPOSE

Simulation-based medical education (SBME) is provided by all anesthesiology residency programs in Canada. The purpose of this study was to characterize SBME in Canadian anesthesiology residency training programs.

METHODS

We administered a 21-question survey to the simulation director/coordinator for all 17 Canadian academic departments of anesthesiology from October 2019 to January 2020. The survey consisted of questions pertaining to the characteristics of the simulation centres, their faculty, learners, curriculum, and assessment processes.

RESULTS

All 17 residency training programs participated in the survey and reported large variability in the number and formal training of simulation faculty and in content delivery. Five programs (29%) did not provide faculty recognition for curriculum design and running simulation sessions. Most programs offered one to four simulation sessions per academic year for each year of residency. All programs offered mannequin-based and part-task trainers for teaching technical and nontechnical skills. Fourteen programs (82%) offered interprofessional and interdisciplinary simulation sessions, and ten programs (59%) did not include in situ simulation training. Commonly reported barriers to faculty involvement were lack of protected time (12 programs, 71%), lack of financial compensation (ten programs, 59%), and lack of appreciation for SBME (seven programs, 41%).

CONCLUSION

Large variability exists in the delivery of SBME in Canadian anesthesiology residency simulation programs, in part because of differences in financial/human resources and educational content. Future studies should explore whether training and patient outcomes differ between SBME programs and, if so, whether additional standardization is warranted.

Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR), Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) Guideline for difficult airway management. Part II

The Airway Management section of the Spanish Society of Anesthesiology, Resuscitation, and Pain Therapy (SEDAR), the Spanish Society of Emergency Medicine (SEMES), and the Spanish Society of Otorhinolaryngology and Head and Neck Surgery (SEORL-CCC) present the Guide for the comprehensive management of difficult airway in adult patients. Its principles are focused on the human factors, cognitive processes for decision-making in critical situations, and optimization in the progression of strategies application to preserve adequate alveolar oxygenation in order to enhance safety and the quality of care. The document provides evidence-based recommendations, theoretical-educational tools, and implementation tools, mainly cognitive aids, applicable to airway management in the fields of anesthesiology, critical care, emergencies, and prehospital medicine. For this purpose, an extensive literature search was conducted following PRISMA-R guidelines and was analyzed using the GRADE methodology. Recommendations were formulated according to the GRADE methodology. Recommendations for sections with low-quality evidence were based on expert opinion through consensus reached via a Delphi questionnaire.

Future directions for simulation in healthcare: A critical review

There was a widespread discontinuation of simulation programs during and after the COVID-19 pandemic. The objective is to explore how to facilitate greater integration of simulation in healthcare organizations. A literature review was conducted in PubMed, MEDES, IBECS and DOCUMED databases. Twenty-three articles published after the pandemic were selected, categorized in seven themes and critically reviewed. In order to consistently and fully integrate simulation into the organizational culture it is recommended to prioritize the development of new strategies that enhance the efficiency and safety of healthcare delivery. And also strategies that enhance the satisfaction and well-being of all stakeholders.

Artificial Intelligence Improves Novices' Bronchoscopy Performance: A Randomized Controlled Trial in a Simulated Setting

BACKGROUND

Navigating through the bronchial tree and visualizing all bronchial segments is the initial step toward learning flexible bronchoscopy. A novel bronchial segment identification system based on artificial intelligence (AI) has been developed to help guide trainees toward more effective training.

RESEARCH QUESTION

Does feedback from an AI-based automatic bronchial segment identification system improve novice bronchoscopists' end-of-training performance?

STUDY DESIGN AND METHODS

The study was conducted as a randomized controlled trial in a standardized simulated setting. Novices without former bronchoscopy experience practiced on a mannequin. The feedback group (n = 10) received feedback from the AI, and the control group (n = 10) trained according to written instructions. Each participant decided when to end training and proceed to performing a full bronchoscopy without any aids.

RESULTS

The feedback group performed significantly better on all three outcome measures (median difference, P value): diagnostic completeness (3.5 segments, P < .001), structured progress (13.5 correct progressions, P < .001), and procedure time (-214 seconds, P = .002).

INTERPRETATION

Training guided by this novel AI makes novices perform more complete, more systematic, and faster bronchoscopies. Future studies should examine its use in a clinical setting and its effects on more advanced learners.

Building Global Partnerships: A Qualitative Exploration of In-Person Training for HealthCare Simulation Educators Working in Low-Resource Settings

INTRODUCTION

Effective educational initiatives can elevate the quality of patient care globally. Simulation-based education is widely used in high-resource settings, but barriers exist to its widespread use in low-resource settings. Vital Anesthesia Simulation Training (VAST) overcomes these barriers by offering immersive, low-cost portable simulation along with simulation facilitator training. During the COVID-19 pandemic, in-person courses were stopped for more than 2 years. Postpandemic, a 3-day VAST SIMposium was hosted in Rwanda to unite 42 VAST facilitators from 12 widespread countries to introduce new and revised course materials and to rejuvenate dormant skills. The purpose of this study was to explore how the VAST SIMposium influenced perceived development of skills, confidence, and engagement in a community of practice for simulation educators working in low-resource settings.

METHODS

This qualitative study involved in-depth interviews with a purposive sample of 16 VAST SIMposium attendees. Transcripts of audio recordings were analyzed using inductive thematic analysis.

RESULTS

Six themes were identified during data analysis: 1) Diversity with shared passion for medical education; 2) Supportive in-person learning environment; 3) Simulation-based education relevant to low-resource settings; 4) Camaraderie; 5) Building knowledge, skills, and confidence; and 6) Being part of a community of practice.

CONCLUSION

The SIMposium rejuvenated passion, knowledge, skills, and, most importantly, strengthened global connections and partnerships. These collaborations will benefit areas that are underrepresented in simulation and will ultimately improve patient outcomes. A SIMposium of this format is an efficient and effective way to foster sustainable global dissemination of simulation-based global health education.

Powerful medical education improves health care quality and return on investment

INTRODUCTION

Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness.

METHODS

This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations.

RESULTS

PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment.

CONCLUSIONS

The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

Perceptions of paramedic educators on assessments used in the first year of a paramedic programme: a qualitative exploration

BACKGROUND

In Ireland, there are currently three educational institutions (recognised institutions- RIs) providing paramedic programmes, accredited by the regulator, the Pre-Hospital Emergency Care Council (PHECC). Each RI assesses their students in-house, and in order to acquire a licence to practice, students must also pass summative assessments provided by PHECC. These assessments comprise multiple choice questions, short answer questions and skills assessments. The objective of this study was to explore the perceptions and experiences of paramedic educators of assessments used within their institution and by the regulator to provide insights that could inform the future design of paramedic assessments.

METHODS

A qualitative study with an interpretivist approach and purposive sampling strategy was performed. Semi-structured interviews were conducted with educators from one RI, across their three sites. Data were analysed using an inductive approach to thematic analysis.

RESULTS

Four major themes were identified in the data: improving assessments by enhancing authenticity, modifying the current process of assessment, aligning the PHECC and RI examinations and opportunities to use assessment as learning.

CONCLUSIONS

This study identifies perceived deficits and opportunities in the assessments currently used for paramedic students and ways in which these assessments could be improved. While participants were relatively content with their own RI assessments, they identified ways to improve both the RI and PHECC assessments. Modifying some of the current methods could be a useful first step. In particular, assessments used by PHECC could be improved by reflecting 'real-world' practice. The inclusion of additional assessment methods by PHECC, a continuous assessment process or devolvement of the entire assessment suite, to the RI/University has the potential to enhance assessments, particularly summative assessments, for paramedic students.

Distance simulation in the health professions: a scoping review

BACKGROUND

Distance simulation is defined as simulation experiences in which participants and/or facilitators are separated from each other by geographic distance and/or time. The use of distance simulation as an education technique expanded rapidly with the recent COVID-19 pandemic, with a concomitant increase in scholarly work.

METHODS

A scoping review was performed to review and characterize the distance simulation literature. With the assistance of an informationist, the literature was systematically searched. Each abstract was reviewed by two researchers and disagreements were addressed by consensus. Risk of bias of the included studies was evaluated using the Risk of Bias 2 (RoB 2) and Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) tools.

RESULTS

Six thousand nine hundred sixty-nine abstracts were screened, ultimately leading to 124 papers in the final dataset for extraction. A variety of simulation modalities, contexts, and distance simulation technologies were identified, with activities covering a range of content areas. Only 72 papers presented outcomes and sufficient detail to be analyzed for risk of bias. Most studies had moderate to high risk of bias, most commonly related to confounding factors, intervention classification, or measurement of outcomes.

CONCLUSIONS

Most of the papers reviewed during the more than 20-year time period captured in this study presented early work or low-level outcomes. More standardization around reporting is needed to facilitate a clear and shared understanding of future distance simulation research. As the broader simulation community gains more experience with distance simulation, more studies are needed to inform when and how it should be used.

Comparison of practice vs theory model training for vacuum-assisted delivery: A randomized controlled trial

OBJECTIVE

To assess the effect of two modalities of simulation training on acquisition/retention of skills for performing operative vaginal delivery.

DESIGN

Randomized, controlled, single-centre study.

SETTING

A tertiary referral hospital in Italy.

PARTICIPANTS

Twenty residents from a single university programme and two young specialists.

INTERVENTION

Group 1 had an individual training session with a single senior specialist using a fixed simulator model. After the session, trainees watched a pre-recorded 1-h lecture on vacuum-assisted operative vaginal delivery. Group 1 repeated the simulator session using the same test after 8-12 weeks and 12 months. Group 2 watched the pre-recorded lecture then undertook the same sessions as Group 1. Video recordings of all test performances were evaluated by five specialists in a blinded manner.

MAIN OUTCOME MEASURE

Each procedure was evaluated using a Global Rating Scale (GRS), scored with 0-5 points for each item on an Objective Structured Assessment of Technical Skills (OSATS) dataset with seven items (total 35 points). OSATS were evaluated over time, compared for the whole population, and weighted for route and year of residency. The primary outcome was comparison of the effectiveness of training between the two groups based on year of residency by assessing videos of the baseline test and GRS for OSATS scores. The secondary outcome was overall retention of skills at 8-12 weeks and 12 months.

RESULTS

Twenty-two participants were recruited and randomized to either Group 1 (n = 11) or Group 2 (n = 11). Five participants did not complete follow-up. The primary outcome of GRS for OSATS scores at time 0 differed significantly between groups for total GRS score, and 'Call for help' and 'Explanation to woman and relatives' item scores (p = 0.002 and p = 0.007, respectively). In a multiple linear regression analysis, OSATS scores were not independently influenced by year of residency. At 8-12-week follow-up, the 'Call for help' item score showed a significant improvement in Group 1 (p = 0.018), although this was not confirmed when year of residency was included as an independent variable. At 12-month follow-up, none of the item scores demonstrated a significant change (p = 0.033). Year of residency did not influence the difference between groups. One-way analysis of variance found significant differences between the groups for 'Localization of the flexion point' (p = 0.005), 'Traction of vacuum cup' (p = 0.039) and 'Use of second hand of the operator' (p = 0.009) item scores and total GRS score (p = 0.007). The values weighted by year of residency did not demonstrate any significant difference. The secondary outcomes evaluated all the candidates of both groups for retention of technical skills over time. A significant effect of time was found for total GRS score (p < 0.001) and OSATS item scores.

CONCLUSIONS

Independent of the sequence of theoretical teaching and simulation training, trainees demonstrated high retention - and, actually, improvement - of technical skills for operative vaginal delivery at 12-month follow-up.

Effectiveness of Flexible Bronchoscopy Simulation-Based Training: A Systematic Review

BACKGROUND

The implementation of simulation-based training (SBT) to teach flexible bronchoscopy (FB) skills to novice trainees has increased during the last decade. However, it is unknown whether SBT is effective to teach FB to novices and which instructional features contribute to training effectiveness.

RESEARCH QUESTION

How effective is FB SBT and which instructional features contribute to training effectiveness?

STUDY DESIGN AND METHODS

We searched Embase, PubMed, Scopus, and Web of Science for articles on FB SBT for novice trainees, considering all available literature until November 10, 2022. We assessed methodological quality of included studies using a modified version of the Medical Education Research Study Quality Instrument, evaluated risk of bias with relevant tools depending on study design, assessed instructional features, and intended to correlate instructional features to outcome measures.

RESULTS

We identified 14 studies from an initial pool of 544 studies. Eleven studies reported positive effects of FB SBT on most of their outcome measures. However, risk of bias was moderate or high in eight studies, and only six studies were of high quality (modified Medical Education Research Study Quality Instrument score ≥ 12.5). Moreover, instructional features and outcome measures varied highly across studies, and only four studies evaluated intervention effects on behavioral outcome measures in the patient setting. All of the simulation training programs in studies with the highest methodological quality and most relevant outcome measures included curriculum integration and a range in task difficulty.

INTERPRETATION

Although most studies reported positive effects of simulation training programs on their outcome measures, definitive conclusions regarding training effectiveness on actual bronchoscopy performance in patients could not be made because of heterogeneity of training features and the sparse evidence of training effectiveness on validated behavioral outcome measures in a patient setting.

TRIAL REGISTRATION

PROSPERO; No.: CRD42021262853; URL: https://www.crd.york.ac.uk/prospero/.

Academic half days, noon conferences and classroom-based education in postgraduate medical education: a scoping review

BACKGROUND

Classroom-based education (CBE) is ubiquitous in postgraduate medical education (PGME), but to date no studies have synthesized the literature on the topic. We conducted a scoping review focusing on academic half days and noon conferences.

METHODS

We searched 4 databases (MEDLINE [OVID], Embase [OVID], ERIC [EBSCO] and Web of Science) from inception to December 2021, performed reference and citation harvesting, and applied predetermined inclusion and exclusion criteria to our screening. We used 2 frameworks for the analysis: "experiences, trajectories and reifications" and "description, justification and clarification."

RESULTS

We included 90 studies, of which 55 focused on resident experiences, 29 on trajectories and 6 on reification. We classified 44 studies as "description," 38 as "justification" and 8 as "clarification." In the description studies, 12 compared academic half days with noon conferences, 23 described specific teaching topics, and 9 focused on resources needed for CBE. Justification studies examined the effects of CBE on outcomes, such as examination scores (17) and use of teaching strategies in team-based learning, principles of adult learning and e-learning (15). Of the 8 clarification studies, topics included the role of CBE in PGME, stakeholder perspectives and transfer of knowledge between classroom and workplace.

INTERPRETATION

Much of the existing literature is either a description of various aspects of CBE or justification of particular teaching strategies. Few studies exist on how and why CBE works; future studies should aim to clarify how CBE facilitates resident learning within the sociocultural framework of PGME.

Adaptation of a Simulation Model and Checklist to Assess Pediatric Emergency Care Performance by Prehospital Teams

INTRODUCTION

Simulation tools to assess prehospital team performance and identify patient safety events are lacking. We adapted a simulation model and checklist tool of individual paramedic performance to assess prehospital team performance and tested interrater reliability.

METHODS

We used a modified Delphi process to adapt 3 simulation cases (cardiopulmonary arrest, seizure, asthma) and checklist to add remote physician direction, target infants, and evaluate teams of 2 paramedics and 1 physician. Team performance was assessed with a checklist of steps scored as complete/incomplete by raters using direct observation or video review. The composite performance score was the percentage of completed steps. Interrater percent agreement was compared with the original tool. The tool was modified, and raters trained in iterative rounds until composite performance scoring agreement was 0.80 or greater (scale <0.20 = poor; 0.21-0.39 = fair, 0.40-0.59 = moderate; 0.60-0.79 = good; 0.80-1.00 = very good).

RESULTS

We achieved very good interrater agreement for scoring composite performance in 2 rounds using 6 prehospital teams and 4 raters. The original 175 step tool was modified to 171 steps. Interrater percent agreement for the final modified tool approximated the original tool for the composite checklist (0.80 vs. 0.85), cardiopulmonary arrest (0.82 vs. 0.86), and asthma cases (0.80 vs. 0.77) but was lower for the seizure case (0.76 vs. 0.91). Most checklist items (137/171, 80%) had good-very good agreement. Among 34 items with fair-moderate agreement, 15 (44%) related to patient assessment, 9 (26%) equipment use, 6 (18%) medication delivery, and 4 (12%) cardiopulmonary resuscitation quality.

CONCLUSIONS

The modified checklist has very good agreement for assessing composite prehospital team performance and can be used to test effects of patient safety interventions.

Technology-enhanced simulation in emergency medicine: Updated systematic review and meta-analysis 1991-2021

Background

Over the past decade, the use of technology-enhanced simulation in emergency medicine (EM) education has grown, yet we still lack a clear understanding of its effectiveness. This systematic review aims to identify and synthesize studies evaluating the comparative effectiveness of technology-enhanced simulation in EM.

Methods

We searched MEDLINE, EMBASE, PsycINFO, CINAHL, ERIC, Web of Science, and Scopus to identify EM simulation research that compares technology-enhanced simulation with other instructional modalities. Two reviewers screened articles for inclusion and abstracted information on learners, clinical topics, instructional design features, outcomes, cost, and study quality. Standardized mean difference (SMD) effect sizes were pooled using random effects.

Results

We identified 60 studies, enrolling at least 5279 learners. Of these, 23 compared technology-enhanced simulation with another instructional modality (e.g., living humans, lecture, small group), and 37 compared two forms of technology-enhanced simulation. Compared to lecture or small groups, we found simulation to have nonsignificant differences for time skills (SMD 0.33, 95% confidence interval [CI] -0.23 to 0.89, n = 3), but a large, significant effect for non-time skills (SMD 0.82, 95% CI 0.18 to 1.46, n = 8). Comparison of alternative types of technology-enhanced simulation found favorable associations with skills acquisition, of moderate magnitude, for computer-assisted guidance (compared to no computer-assisted guidance), for time skills (SMD 0.50, 95% CI -1.66 to 2.65, n = 2) and non-time skills (SMD 0.57, 95% CI 0.33 to 0.80, n = 6), and for more task repetitions (time skills SMD 1.01, 95% CI 0.16 to 1.86, n = 2) and active participation (compared to observation) for time skills (SMD 0.85, 95% CI 0.25 to 1.45, n = 2) and non-time skills (SMD 0.33 95% CI 0.08 to 0.58, n = 3).

Conclusions

Technology-enhanced simulation is effective for EM learners for skills acquisition. Features such as computer-assisted guidance, repetition, and active learning are associated with greater effectiveness.

Introducing the "SIMline"-A Simulation Course in the Management of Severe Burns as a Tool in Undergraduate Medical Education

Background: Management of burn injuries presents a complex and challenging situation for medical staff, especially for inexperienced young doctors. However, training on how to treat burn victims in the clinical setting is rarely taught in undergraduate medical classes. We have created the "SIMline", a simulation training program explicitly designed for coaching medical students in burn management. Methods: A total of 43 students participated in the "SIMline" course, which took place at the training facility at the Medical University of Graz, between 2018 and 2019. The course provided theoretical classes, practical exercises, and a full-scale care process simulation training. The learning progress of the students was monitored via a formative integrated test. Results: Students showed great progress throughout the course of the "SIMline" program, as their test scores improved by an average of 88%. The passing rate was 0% at the first exam (prior to course) as compared to 87% at the final exam, taken after the training. Conclusions: Comprehensive practical training programs in burn care are underrepresented in medical education. The "SIMline" course presents a novel and effective approach in training medical students in burn management. However, follow-up evaluation is necessary to confirm long-term educational benefits.

Computer-assisted simulated workplace-based assessment in surgery: application of the universal framework of intraoperative performance within a mixed-reality simulation

Objectives

Workplace-based assessment (WBA) is a key requirement of competency-based medical education in postgraduate surgical education. Although simulated workplace-based assessment (SWBA) has been proposed to complement WBA, it is insufficiently adopted in surgical education. In particular, approaches to criterion-referenced and automated assessment of intraoperative surgical competency in contextualized SWBA settings are missing.Main objectives were (1) application of the universal framework of intraoperative performance and exemplary adaptation to spine surgery (vertebroplasty); (2) development of computer-assisted assessment based on criterion-referenced metrics; and (3) implementation in contextualized, team-based operating room (OR) simulation, and evaluation of validity.

Design

Multistage development and assessment study: (1) expert-based definition of performance indicators based on framework's performance domains; (2) development of respective assessment metrics based on preoperative planning and intraoperative performance data; (3) implementation in mixed-reality OR simulation and assessment of surgeons operating in a confederate team. Statistical analyses included internal consistency and interdomain associations, correlations with experience, and technical and non-technical performances.

Setting

Surgical simulation center. Full surgical team set-up within mixed-reality OR simulation.

Participants

Eleven surgeons were recruited from two teaching hospitals. Eligibility criteria included surgical specialists in orthopedic, trauma, or neurosurgery with prior VP or kyphoplasty experience.

Main outcome measures

Computer-assisted assessment of surgeons' intraoperative performance.

Results

Performance scores were associated with surgeons' experience, observational assessment (Objective Structured Assessment of Technical Skill) scores and overall pass/fail ratings. Results provide strong evidence for validity of our computer-assisted SWBA approach. Diverse indicators of surgeons' technical and non-technical performances could be quantified and captured.

Conclusions

This study is the first to investigate computer-assisted assessment based on a competency framework in authentic, contextualized team-based OR simulation. Our approach discriminates surgical competency across the domains of intraoperative performance. It advances previous automated assessment based on the use of current surgical simulators in decontextualized settings. Our findings inform future use of computer-assisted multidomain competency assessments of surgeons using SWBA approaches.

Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

A Decade Later-Progress and Next Steps for Pediatric Simulation Research

SUMMARY STATEMENT

A decade ago, at the time of formation of the International Network for Pediatric Simulation-based Innovation, Research, and Education, the group embarked on a consensus building exercise. The goal was to forecast the facilitators and barriers to growth and maturity of science in the field of pediatric simulation-based research. This exercise produced 6 domains critical to progress in the field: (1) prioritization, (2) research methodology and outcomes, (3) academic collaboration, (4) integration/implementation/sustainability, (5) technology, and (6) resources/support/advocacy. This article reflects on and summarizes a decade of progress in the field of pediatric simulation research and suggests next steps in each domain as we look forward, including lessons learned by our collaborative grass roots network that can be used to accelerate research efforts in other domains within healthcare simulation science.

Taking Aim at the Injury Prevention Curriculum: Educating Residents on Talking to Patients About Firearm Injury

The medical community recognizes that firearm injury is a public health problem. Yet we lack both the tools for and the implementation of evidence-based firearm injury screening and counseling techniques. One reason for these deficits is the lack of clinical training related to engaging patients in firearm injury risk reduction. In this issue, Rickert et al describe a pre-post evaluation of a 2-part firearm injury prevention training curriculum for first-year medical residents at a single academic medical center. Their manuscript serves an important, but still preliminary, step forward for the field of postgraduate medical education on firearm injury and its prevention. Important elements of this project and paper consist of the inclusion of multiple medical disciplines and the use of standardized patients to evaluate participants' learning. This project also points to the need for further growth. We must commit to consistently and conscientiously framing injuries from guns not as "gun violence" but rather as "firearm injuries" to ensure that they are considered squarely in both the public health space and the clinical space. An ideal curriculum would also highlight the importance of trauma-informed care, cultural competency, and antiracist medical practice while countering implicit biases (e.g., toward gun owners, victims of firearm violence, perpetrators of violence). It should address barriers, as well as facilitators, to change. And most importantly, future educational work must evaluate the effect of these trainings on actual clinical practice-and, even better, the efficacy of education in changing behavior and patient-level outcomes.

Can virtual-reality simulation ensure transthoracic echocardiography skills before trainees examine patients?

OBJECTIVES

This study aimed to develop and gather the validity evidence for a standardised simulation-based skills test in transthoracic echocardiography and to establish a credible pass/fail score.

METHODS

Experts developed a virtual-reality simulator test in cardiology, medical education and simulation-based education. Thirty-six physicians with different experiences in transthoracic echocardiography completed the test at Odense University Hospital, Denmark. The performances of novice, intermediate and experienced participants were compared using the Bonferroni post hoc test. Cronbach's alpha was used to determine the internal consistency reliability of the test. The consistency of performance was analysed using the intraclass correlation coefficient. A pass/fail score was established using the contrasting groups' standard-setting method.

RESULTS

We developed a test with high consistent reliability (Alpha = .81), 95% CI [.69, .89]. In both cases, the performers' level was consistent, fitting others at the same level of experience (intraclass correlation r₍₃₅₎=.81, p<.001). A pass/fail score of 48/50 points was established based on the mean test score of novice and experienced physicians.

CONCLUSIONS

We developed a standardised virtual-reality simulation-based test of echocardiography skills with the ability to distinguish between participants with different levels of transthoracic echocardiography experience. This test could direct a mastery learning training program where trainees practise until they reach the pre-defined level and secure a higher level of competency to ensure quality and safety for patients.

Developing an integrated national simulation-based educational programme for Scottish junior doctors through structured, multistep action research cycles

OBJECTIVES

Simulation is widely employed to teach a range of skills, across healthcare professions and is most effective when embedded within a standarised curriculum. Although recommended by many governing bodies, establishing a national programme of simulation presents many challenges. Successful implementation requires a clear understanding of the priorities and needs of those it seeks to serve yet there are limited examples of how best to do this. This study aimed to develop an integrated national simulation-based educational programme for junior doctors in Scotland through a structed, multistep prioritisation process.

DESIGN

A series of action research cycles were undertaken to develop and evaluate a national simulation programme. This paper describes cycle 1, which employed a six-step structured approach to understand and prioritise learner needs.

SETTING

The study considered the educational needs of Scottish junior doctors in the UK Foundation Programme (UKFP).

PARTICIPANTS

Multiple stakeholder groups were involved in each stage of the process including recent Scottish UKFP graduates, clinical educators, UKFP programme directors and postgraduate deans.

RESULTS

Key stakeholders reviewed the 370 competencies in the UKFP curriculum and identified 18 initial competency areas. These 18 areas were subsequently prioritised through the analytical hierarchy process, resulting in a carefully ordered list of 12 competencies from which a targeted simulation-based educational programme could be developed.

CONCLUSIONS

To our knowledge, this is the first study to outline the methods of competency prioritisation to create a simulation curriculum that is integrated within a national curriculum in the medical education context. As well as demonstrating the practical steps of such a process, key implications for practice are identified. This robust approach to educational design also resulted in unexpected benefits, including educator and clinician acceptance and programme funding sustainability.

Comparison of multimodal active learning and single-modality procedural simulation for central venous catheter insertion for incoming residents in anesthesiology: a prospective and randomized study

BACKGROUND

Active learning methods, including low-fidelity simulation, are useful but the incremental learning effect of each method is however limited. We designed this study to assess if combining flipped classroom and the modified Peyton's « 4-steps» method during procedural simulation (intervention group [IG]) would provide better learning results than simulation alone (control group [CG]) in the context of central venous catheter insertion training.

METHODS

This prospective, single-center, and randomized study took place in 2017 in a single simulation center. All first year Anesthesiology residents of Ile de France area at the start of their residency were randomly included either in the IG or CG during a seminar aimed at providing initial procedural skills with low-fidelity simulation. A composite learning score which included knowledge MCQ and a questionnaire assessing satisfaction and value of the training session was recorded after training (primary outcome, /100). A randomized sub-group of learners of each group were video-recorded during central venous catheter insertion at the end of training and their skills were evaluated with validated tools, including a specific checklist and a global rating scale (GRS).

RESULTS

Among 89 anesthesiology residents, 48 and 41 learners were randomized in the intervention and control groups respectively. Of the IG residents, 40/48 (83%) had read the learning material prior to the session. There was no significant difference regarding the composite outcome ([IG]= 81.1 vs [CG] = 80.5 /100 (p = 0.68)). Results of the post-session MCQ knowledge questionnaire were also non-significantly different. Residents were similarly satisfied and described a significant improvement of their knowledge and skills after training. Learners highly valued the training session as a mean to improve their future practice. No significant differences regarding checklist and GRS scores were observed.

CONCLUSIONS

A multimodal active learning strategy of procedural learning did not provide better learning outcomes when compared to a traditional simulation method. In both groups, satisfaction was high and perception of the acquired theoretical and practical knowledge was improved after training.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Background

Objective

Design, setting, and participants

Outcome measurements and statistical analyses

Results and limitations

Conclusions

Patient summary

Expertise in colonoscopy intubation does not predict diagnostic accuracy: a simulation-based study

Background and study aims  Studies have linked cecal intubation rate with adenoma detection rate; however, the direct association between technical performance during colonoscopy intubation and withdrawal has never been explored. Thus, it remains unclear whether gentle and efficient intubation predicts superior mucosal inspection. The aim of this study was to investigate the correlation between performance during intubation and withdrawal in a simulation-based setup. Methods  Twenty-four physicians with various experience in colonoscopy performed twice on the Endoscopy Training System (ETS). Intubation skills were evaluated by assessing tasks on the ETS related to intubation (scope manipulation and loop management) and use of a computerized assessment tool called the 3D-Colonoscopy Progression Score (3D-CoPS). Diagnostic accuracy was defined by the number of polyps found during the ETS task of mucosal inspection. Pearson's correlations were calculated to explore associations between intubation skill and diagnostic accuracy. Results  The correlation analysis between 3D-CoPS and number of polyps found during mucosal inspection revealed a weak and insignificant correlation (0.157, P  = 0.3). Likewise, an insignificant correlation was seen between ETS intubation and number of polyps found (0.149, P  = 0.32). Conclusions  We found no evidence to support that technical performance during intubation is correlated with mucosal inspection performance in a simulation-based setting.

Improving Pediatric Readiness and Clinical Care in General Emergency Departments: A Multicenter Retrospective Cohort Study

OBJECTIVE

To evaluate the impact of a collaborative initiative between general emergency departments (EDs) and the pediatric academic medical center on the process of clinical care in a group of general EDs.

STUDY DESIGN

This retrospective cohort study assessed the process of clinical care delivered to critically ill children presenting to 3 general EDs. Our previous multifaceted intervention included the following components: postsimulation debriefing, designation of a pediatric champion, customized performance reports, pediatric resources toolkit, and ongoing interactions. Five pediatric emergency care physicians conducted chart reviews and scored encounters using the Pediatric Emergency Care Research Network's Quality of Care Implicit Review Instrument, which assigns scores between 5 and 35 across 5 domains. In addition, safety metrics were collected for medication, imaging, and laboratory orders.

RESULTS

A total of 179 ED encounters were reviewed, including 103 preintervention and 76 postintervention encounters, with an improvement in mean total quality score from 23.30 (SD 5.1) to 24.80 (4.0). In the domain of physician initial treatment plan and initial orders, scores increased from a mean of 4.18 (0.13) to 4.61 (0.15). In the category of safety, administration of wrong medications decreased from 28.2% to 11.8% after the intervention.

CONCLUSION

A multifaceted collaborative initiative involving simulation and enhanced pediatric readiness was associated with improvement in the processes of care in general EDs. This work provides evidence that innovative collaborations between academic medical centers and general EDs may serve as an effective strategy to improve pediatric care.

Quality of Resuscitative Care Provided to an Infant With Abusive Head Trauma in Community Emergency Departments: An In Situ, Prospective, Simulation-Based Study

OBJECTIVES

Abusive head trauma (AHT) is a very common and serious form of physical abuse, and a major cause of mortality and morbidity for young children. Early Recognition and supportive care of children with AHT is a common challenge in community emergency department (CEDs). We hypothesized that standardized, in situ simulation can be used to measure and compare the quality of resuscitative measures provided to children with AHT in a diverse set of CEDs.

METHODS

This prospective, simulation-based study measured teams' performance across CEDs. The primary outcome was overall adherence to AHT using a 15-item performance assessment checklist based on the number of tasks performed correctly on the checklist.

RESULTS

Fifty-three multiprofessional teams from 18 CEDs participated in the study. Of 270 participants, 20.7% were physicians, 65.2% registered nurses, and 14.1% were other providers. Out of all tasks, assessment of airway/breathing was the most successfully conducted task by 53/53 teams (100%). Although 43/53 teams (81%) verbalized the suspicion for AHT, only 21 (39.6%) of 53 teams used hyperosmolar agent, 4 (7.5%) of 53 teams applied cervical spine collar stabilization, and 6 (11.3%) of 53 teams raised the head of the bed. No significant difference in adherence to the checklist was found in the CEDs with an inpatient pediatric service or these with designated adult trauma centers compared with CEDs without. Community emergency departments closer to the main academic center outperformed CEDs these that are further away.

CONCLUSIONS

This study used in situ simulation to describe quality of resuscitative care provided to an infant presenting with AHT across a diverse set of CEDs, revealing variability in the initial recognition and stabilizing efforts and provided and targets for improvement. Future interventions focusing on reducing these gaps could improve the performance of CED providers and lead to improved patient outcomes.

Competency-based medical education in radiotherapy treatment planning

PURPOSE

To develop a technology-enhanced education methodology with competency-based evaluation for radiotherapy treatment planning. The education program is designed for integration in the existing framework of Commission on Accreditation of Medical Physics Education Programs (CAMPEP) accredited medical physics residency programs.

MATERIALS AND METHODS

This education program pairs an accessible, multi-institutional infrastructure with established medical education evaluation tools to modernize treatment planning education. This program includes three evaluation components: (i) competency-based evaluation, (ii) inter- and intra-modality comparison, and (iii) learner feedback. For this study, synchronous bilateral breast cancer was selected to demonstrate a complex treatment site and non-standardized technique. Additionally, an online study was made available to a public cohort of worldwide participants of certified Medical Dosimetrists and Medical Physicists to benchmark performance. Prior to evaluation, learners were given a disease site-specific education session on potential clinical treatment strategies. During the assessment, learners generated treatment plans in their institutional planning system under the direct observation of an expert evaluator. Qualitative proficiency was evaluated for all learners on a five-point scale of graduated task independence. Quantitative dosimetry was compared between the learner cohort and public cohort. A feedback session provided learners context of multi-institutional experience through multimodality and technique comparison. After study completion, learners were provided a survey that was used to gauge their perception of the education program.

RESULTS

In the public study, 34 participants submitted treatment plans. Across three CAMPEP-accredited residency programs, six learners participated in the education and evaluation program. All learners successfully completed treatment plans that met the dosimetric constraints described in the case study. All learners favourably reviewed the study either comprehensively or in specified domains.

CONCLUSION

The competency-based education and evaluation program developed in this work has been incorporated in CAMPEP-accredited residency programs and is adaptable to other residency programs with minimal resource commitment.

Simulation and Active Learning Decreases Training Time of an Emergency Triage Assessment and Treatment Course in Pilot Study in Malawi: Implications for Increasing Efficiency and Workforce Capacity in Low-Resource Settings

OBJECTIVES

The aim of the Emergency Triage Assessment and Treatment (ETAT) plus trauma course is to improve the quality of care provided to infants and children younger than 5 years. The curriculum was revised and shortened from 5 to 2.5 days by enhancing simulation and active learning opportunities. The aim of this study was to examine the feasibility and value of the new short-form ETAT course by assessing postcourse knowledge and satisfaction.

METHODS

We delivered the short-form ETAT course to a group of interdisciplinary health workers in Malawi. Precourse and postcourse knowledge was assessed using a standardized 20 questions short answer test used previously in the 5-day courses. A 13-statement survey with 2 open-ended questions was used to examine participant satisfaction.

RESULTS

Participants' postcourse knowledge improved significantly (P < 0.001) after the shorter ETAT course. Participants reported high levels of satisfaction with the short-form ETAT.

CONCLUSIONS

Simulation and other active learning strategies reduced training time by 50% in the short-form ETAT course. Participants with and without previous ETAT training improved their knowledge after participating in the short-form ETAT course. Reduced training time is beneficial in settings already burdened by scarce human resources, may facilitate better access to in-service training, and build capacity while conserving resources in low-resource settings.

Simulation-based assessment of care for infant cardiogenic shock in the emergency department

BACKGROUND

Congenital heart disease (CHD) is the leading cause of infant deaths associated with birth defects. Neonates with undiagnosed CHD often present to general emergency departments (GEDs) for initial resuscitation that are less prepared than paediatric centres, resulting in disparities in the quality of care. Neonates with undiagnosed CHD represent a challenge; thus, it is necessary for GEDs to be prepared for this population.

AIM

To evaluate the process of resuscitative care provided to a neonate in cardiogenic shock due to CHD in the GEDs in a simulated setting and to describe the impact of teams and GED variables on the process of care.

METHODS

This is a prospective simulation-based assessment of the process of care provided to a neonate with coarctation of the aorta in cardiogenic shock. Simulation sessions were conducted at participating GEDs utilizing each GED's interdisciplinary team and resources. The primary outcome was adherence to best practice, as measured by a 15-item overall composite adherence score (CAS). In addition, we stratified the overall CAS into CHD-critical items and the general resuscitation items CAS. The secondary outcome was the impact of the team's and GED's characteristics on the scores.

FINDINGS

This study enrolled 32 teams from 12 GEDs. Among 161 participants, 103 (63.97%) were registered nurses, 33 (20.50%) were physicians, 17 (10.56%) were respiratory therapists, and 8 (4.97%) were other medical professionals. The overall median CAS was 84, with the CHD-critical items having a median CAS of 34.5. The most underperformed tasks are checking pulses on the upper and lower extremities (44%), obtaining blood pressure in the upper and lower extremities (25%), and administering prostaglandin E1 (22%).

CONCLUSIONS

Using in situ simulation in a set of GEDs, we revealed gaps in the resuscitation care of neonates with CHD in cardiogenic shock.

RELEVANCE TO CLINICAL PRACTICE

These findings highlight the importance of targeted improvement programs for high-stakes illnesses in GED.

Harnessing the power of simulation for assessment: Consensus recommendations for the use of simulation-based assessment in emergency medicine

OBJECTIVES

To address the increasing demand for the use of simulation for assessment, our objective was to review the literature pertaining to simulation-based assessment and develop a set of consensus-based expert-informed recommendations on the use of simulation-based assessment as presented at the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium on Education.

METHODS

A panel of Emergency Medicine (EM) physicians from across Canada, with leadership roles in simulation and/or assessment, was formed to develop the recommendations. An initial scoping literature review was conducted to extract principles of simulation-based assessment. These principles were refined via thematic analysis, and then used to derive a set of recommendations for the use of simulation-based assessment, organized by the Consensus Framework for Good Assessment. This was reviewed and revised via a national stakeholder survey, and then the recommendations were presented and revised at the consensus conference to generate a final set of recommendations on the use of simulation-based assessment in EM.

CONCLUSION

We developed a set of recommendations for simulation-based assessment, using consensus-based expert-informed methods, across the domains of validity, reproducibility, feasibility, educational and catalytic effects, acceptability, and programmatic assessment. While the precise role of simulation-based assessment will be a subject of continued debate, we propose that these recommendations be used to assist educators and program leaders as they incorporate simulation-based assessment into their programs of assessment.

The role of weight bias and role-modeling in medical students' patient-centered communication with higher weight standardized patients

OBJECTIVE

Patients with obesity may experience less patient-centered care. We assessed whether medical students' implicit/explicit weight-related attitudes and perceptions of normative attitudes are associated with patient-centered care for patients with obesity.

METHODS

Third and fourth year medical students (N = 111) at one medical school completed a survey and participated in a patient care scenario with a standardized patient with obesity. Encounters were coded for patient-centered behavior. Predictors of patient-centered behaviors were assessed.

RESULTS

Student perceptions that negative attitudes about patients with obesity are normative in medical school were significantly associated with poorer patient-centered behaviors, including lower attentiveness (b=-0.19, p = 0.01), friendliness (b=-0.28, p < 0.001), responsiveness (b=-0.21, p = 0.002), respectfulness (b=-0.17, p = 0.003), interactivity (b=-0.22, p = 0.003), likelihood of being recommended by observers (b=-0.34, p < 0.001), and patient-centeredness index scores (b=-0.16, p = 0.002). Student reported faculty role-modeling of discrimination against patients with obesity predicted lower friendliness (b=-0.16, p = 0.03), recommendation likelihood (b=-0.22, p = 0.04), and patient-centeredness index score (b=-0.12, p = 0.03).

CONCLUSIONS

Negative normative attitudes and behaviors regarding obesity in the medical school environment may adversely influence the quality of patient-centered behaviors provided to patients with obesity.

PRACTICE IMPLICATIONS

Efforts to improve patient-centered communication quality among medical trainees may benefit from intervention to improve group normative attitudes about patients with obesity.

Qualities of Effective Vital Anaesthesia Simulation Training Facilitators Delivering Simulation-Based Education in Resource-Limited Settings

BACKGROUND

Lack of access to safe and affordable anesthesia and surgical care is a major contributor to avoidable death and disability across the globe. Effective education initiatives are a viable mechanism to address critical skill and process gaps in perioperative teams. Vital Anaesthesia Simulation Training (VAST) aims to overcome barriers limiting widespread application of simulation-based education (SBE) in resource-limited environments, providing immersive, low-cost, multidisciplinary SBE and simulation facilitator training. There is a dearth of knowledge regarding the factors supporting effective simulation facilitation in resource-limited environments. Frameworks evaluating simulation facilitation in high-income countries (HICs) are unlikely to fully assess the range of skills required by simulation facilitators working in resource-limited environments. This study explores the qualities of effective VAST facilitators; knowledge gained will inform the design of a framework for assessing simulation facilitators working in resource-limited contexts and promote more effective simulation faculty development.

METHODS

This qualitative study used in-depth interviews to explore VAST facilitators' perspectives on attributes and practices of effective simulation in resource-limited settings. Twenty VAST facilitators were purposively sampled and consented to be interviewed. They represented 6 low- and middle-income countries (LMICs) and 3 HICs. Interviews were conducted using a semistructured interview guide. Data analysis involved open coding to inductively identify themes using labels taken from the words of study participants and those from the relevant literature.

RESULTS

Emergent themes centered on 4 categories: Persona, Principles, Performance and Progression. Effective VAST facilitators embody a set of traits, style, and personal attributes (Persona) and adhere to certain Principles to optimize the simulation environment, maximize learning, and enable effective VAST Course delivery. Performance describes specific practices that well-trained facilitators demonstrate while delivering VAST courses. Finally, to advance toward competency, facilitators must seek opportunities for skill Progression.Interwoven across categories was the finding that effective VAST facilitators must be cognizant of how context, culture, and language may impact delivery of SBE. The complexity of VAST Course delivery requires that facilitators have a sensitive approach and be flexible, adaptable, and open-minded. To progress toward competency, facilitators must be open to self-reflection, be mentored, and have opportunities for practice.

CONCLUSIONS

The results from this study will help to develop a simulation facilitator evaluation tool that incorporates cultural sensitivity, flexibility, and a participant-focused educational model, with broad relevance across varied resource-limited environments.

Maintaining technical proficiency in senior surgical fellows during the COVID-19 pandemic through virtual teaching

Objectives

The novel coronavirus (COVID-19) pandemic resulted in a severe reduction in operative opportunities for trainees. We hypothesized that augmenting independent practice with a bench model of vascular anastomoses using regular videoconferences and individual feedback would provide meaningful benefit in maintenance of technical skills in senior lung transplant surgical fellows.

Methods

A lung transplantation virtual technical skills course was developed. Surgical fellows were provided with a bench model and surgical instruments. Using a virtual communication platform, teaching sessions were held twice weekly, and fellows performed an anastomosis on camera. Video recordings were reviewed and critiqued by attending staff. At the end of the 3-month course, participants were surveyed about their experience. Warm ischaemic time was compared between fellows' five most recent cases before and after the pandemic.

Results

Seven senior surgical fellows participated and provided feedback. Fellows had graduated medical school an average of 14 years prior to fellowship, and spent an average of 5 hours (range 1.3 - 15 hours) of home practice. Five of seven (71%) participants reported improvement in their technical skills and increased confidence in performing a lung transplant. No significant difference in warm ischaemic time in procedures performed by fellows was identified (70.3 minutes pre-pandemic vs. 68.3 minutes post pandemic, p = 0.68).

Conclusions

A program of virtual technical skills teaching, individual video coaching, and independent practice provided benefit in maintaining technical skills in lung transplant surgical fellows during the COVID-19 pandemic, when equivalent operative experience was unavailable. Lessons learned from this exceptional time can be used to create simulation curricula for senior trainees.

The assessment of researchers' competence in experimental procedures with laboratory animals: A three-step methodology to develop a global rating scale

To conduct animal experiments, researchers must be competent to handle and perform interventions on living animals in compliance with regulations. Laboratory animal science training programmes and licensing bodies therefore need to be able to reliably ensure and certify the professional competence of researchers and technicians. This requires access to assessment strategies which can verify knowledge as well as capturing performative and behavioural dimensions of assessment. In this paper, we describe the process of developing different global rating scales measuring candidates' competence in a performative assessment. We set out the following sequence, with three crucial phases, in the process of scale development: (a) Item Development, (b) Scale Development and (c) Piloting of the Scale. We note each phase's different sub-steps. Despite the emergent need to ensure the competence of researchers using animals in scientific procedures, to our best knowledge there are very few species and procedure/skill specific assessment tools for this purpose, and the assessment methodology literature in the field is very limited. This paper provides guidance for those who need to develop and assess proficiency in laboratory animal procedures by setting out a method that can be used to create the required tools and illustrating how competence assessment strategies can be implemented.

Exploration of simulation-based medical education for undergraduate students

Over the past decades, extensive studies have underscored the growing importance of simulation-based medical education (SBME) for medical students. However, the underlying influence of SBME on undergraduate students is yet to be investigated. This work is a single-center cohort study involving 1178 undergraduate students who were divided into a control group and an SBME group. All participants gave their written informed consent. We compared the theoretical and practical achievements of these 2 groups and distributed a feedback questionnaire. Results show that SBME significantly improves the practical or theoretical achievements of students (P < .001). The humanistic care (improvement rate: 69.2%) and doctor-patient communication (improvement rate: 56.3%) performances of these studies were vastly improved. The students in the SBME group tend to allocate more time to communicating with others. SBME is an effective teaching method that can improve the reflective capacity and communication skills of undergraduate medical students, thereby resulting in their relatively improved performance.

Factors Associated with Health Care Provider Attitudes, and Confidence for the Care of Women and Girls Affected by Female Genital Mutilation/Cutting

Background: Female genital mutilation/cutting (FGM/C) is a cultural practice that includes procedures that intentionally alter or cause harm to female genital organs for nonmedical reasons, affecting ∼200 million women and girls globally. Health care providers in the United States often lack confidence to provide appropriate FGM/C-related care, and experience attitudes that may negatively impact quality of care for FGM/C. Methods: We conducted a cross-sectional survey of health care providers to explore the associations between health care provider characteristics, awareness of health complications of FGM/C, attitudes, and confidence for FGM/C care. Results: Factors associated with more Confidence for Clinical FGM/C Care include awareness of health complications, ever cared for a woman with FGM/C, being a woman or person of color, and more than 5 years of clinical practice. Increased Confidence in Communication Skills for FGM/C Care was associated with awareness of more health complications for FGM/C. Women endorsed significantly less Negative Attitudes toward FGM/C compared with men; no other factors were associated with health care provider attitudes. Conclusion: Future research should further investigate factors associated with health care provider attitudes toward FGM/C and those affected by the practice to promote quality care. Health providers require adequate training for clinical FGM/C care and in the communication skills that promote patient/provider communication cross-culturally. Trial Registration: Clinical Trials.Gov ID no. NCT03249649, Study ID no. 5252. Public website: https://clinicaltrials.gov/ct2/show/NCT03249649.

Otoskills training during covid-19 pandemic: a before-after study

BACKGROUND

The ongoing COVID-19 pandemic has disrupted the surgical training of residents. There is a real concern that trainees will not be able to meet their training requirements. Low-fidelity surgical simulation appears to be an alternative for surgical training. The educational benefits of repeating ossiculoplasty simulations under a microscope have never been evaluated. With this study we aimed to evaluate the differences in performance scores and on a global rating scale before and after training on an ossiculoplasty simulator.

METHODS

In this quasi-experimental, prospective, single-centre, before-after study with blinded rater evaluation, residents performed five microscopic ossiculoplasty tasks with a difficulty gradient (sliding beads onto rods, the insertion of a partial prosthesis, the insertion of a total prosthesis, and the insertion of a stapedotomy piston under microscopic or endoscopic surgery) before and after training on the same simulator. Performance scores were defined for each task, and total performance scores (score/min) were calculated. All data were collected prospectively.

RESULTS

Six out of seven intermediate residents and 8/9 novices strongly agreed that the simulator was an effective training device and should be included in the ENT residency program. The mean effect of training was a significant increase in the total performance score (+ 0.52 points/min, [95 % CI, 0.40-0.64], p < 0.001), without a significant difference between novice and intermediate residents.

CONCLUSIONS

This preliminary study shows that techniques for middle-ear surgery can be acquired using a simulator, avoiding any risk for patients, even under lockdown measures.

Correcting the Fundamentals of Laparoscopic Surgery "Illusion of Validity" in Laparoscopic Vaginal Cuff Suturing

STUDY OBJECTIVE

The "illusion of validity" is a cognitive bias in which the ability to interpret and predict surgical performance accurately is overestimated. To address this bias, we assessed participants comparing fundamentals of laparoscopic surgery (FLS) and non-FLS tasks with cadaveric vaginal cuff suturing to determine the most representative simulation task for laparoscopic vaginal cuff suturing.

DESIGN

Validity (Messick framework) study comparing FLS and non-FLS tasks with cadaveric vaginal cuff suturing.

SETTING

Simulation center cadaver laboratory.

PARTICIPANTS

Obstetrics and gynecology residents (n = 21), minimally invasive gynecologic surgery fellows (n = 3), gynecologic surgical subspecialists (n = 4), general obstetrician/gynecologists (n = 10).

INTERVENTIONS

Tasks included a simulated vaginal cuff (ipsilateral port placement), needle passage through a metal eyelet loop (contralateral and ipsilateral), and intracorporeal knot tying (contralateral and ipsilateral). Simulation task times were compared with the placement of the first cadaveric vaginal cuff suture time, as well as the in-person and blinded Global Operative Assessment of Laparoscopic Skills (GOALS) score ("relations to other variables" validity evidence). Statistical analyses included Spearman's test of correlation (continuous and ordinal variables) or Wilcoxon rank sum test (categoric variables).

MEASUREMENTS AND MAIN RESULTS

There was a stronger association with cadaver cuff suturing time for simulated vaginal cuff suturing time (r = 0.73, p <.001) compared with FLS intracorporeal contralateral suturing time (r = 0.54, p <.001). Additional measures associated with cadaveric performance included subspecialty training (median: 82 vs 185 seconds, p = .002), number of total laparoscopic hysterectomies (r = -0.53, p <.001), number of laparoscopic cuff closures (r = -0.61, p <.001), number of simulated laparoscopic suturing experiences (r = -0.51, p <.001), and eyelet contralateral time (r = 0.52, p <.001). Strong agreement between the in-person and blinded GOALS (intraclass correlation coefficient = 0.80) supports response process evidence. Correlations of cadaver cuff time with in-person (Spearman's r = -0.84, p <.001) and blinded GOALS (r = -0.76, p <.001) supports relations to other variables evidence CONCLUSION: The weaker correlation between FLS suturing and cadaver cuff suturing compared with a simulated vaginal cuff model may lead to an "illusion of validity" for assessment in gynecology. Since gynecology specific validity evidence has not been well established for FLS, we recommend prioritizing the use of a simulated vaginal cuff suturing assessment in addition to FLS.

Procedural Competency Among Hospitalists: A Literature Review and Future Considerations

BACKGROUND

As general internists practicing in the inpatient setting, hospitalists at many institutions are expected to perform invasive bedside procedures, as defined by professional standards. In reality, hospitalists are doing fewer procedures and increasingly are referring to specialists, which threatens their ability to maintain procedural skills. The discrepancy between expectations and reality, especially when hospitalists may be fully credentialed to perform procedures, poses significant risks to patients because of morbidity and mortality associated with complications, some of which derive from practitioner inexperience.

METHODS

We performed a structured search of the peer-reviewed literature to identify articles focused on hospitalists performing procedures.

RESULTS

Our synthesis of the literature characterizes contributors to hospitalists' procedural competency and discusses: (1) temporal trends for procedures performed by hospitalists and their associated referral patterns, (2) data comparing use and clinical outcomes of procedures performed by hospitalists compared with specialists, (3) the lack of nationwide standardization of hospitalist procedural training and credentialing, and (4) the role of medical procedure services, although limited in supportive evidence, in concentrating procedural skill and mitigating risk in the hands of a few well-trained hospitalists.

CONCLUSION

We conclude with recommendations for hospital medicine groups to ensure the safety of hospitalized patients undergoing bedside procedures.

Validity and effectiveness of augmented reality in surgical education: A systematic review

BACKGROUND

Current challenges in surgical training have led to the investigation of augmented reality as a potential method of supplementary education. However, its value for this purpose remains uncertain. The aim of this study was to perform a systematic review of the published literature to evaluate the validity and effectiveness of augmented reality in surgical education, and to compare it with other simulation modalities.

METHODS

Electronic literature searches were performed in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines. Two authors independently extracted pertinent data and assessed study quality. The primary outcome measures of interest were the validity and effectiveness of augmented reality as an educational tool.

RESULTS

Of 6,500 articles, 24 studies met eligibility criteria for inclusion, of which 2 were randomized. Ten studies investigated validity, with 7 establishing both face and content validity and an additional 1 just content validity. Construct validity was demonstrated in 9 of 11 studies. Of the 11 studies that examined the effectiveness of augmented reality in skills acquisition, 9 demonstrated enhanced learning. Of the 5 studies in which the effectiveness of augmented reality as an educational tool was compared with other modes of simulation, augmented reality was found to be superior in 2 and equivalent in the others.

CONCLUSION

Overall, the majority, including 2 high-quality randomized controlled trials, demonstrated the validity and effectiveness of augmented reality in surgical education. However, the quality of published studies was poor with marked heterogeneity. Although these results are encouraging, additional high-quality studies, preferably in the real-life environment, are required before the widespread implementation of augmented reality within surgical curricula can be recommended.

Structured and evidence-based training of technical skills in respiratory medicine and thoracic surgery

Within the last decade, the number of technical procedures in respiratory medicine and thoracic surgery has grown and created a higher demand for effective and evidence-based education. Today, trainees are often allowed to perform procedures unsupervised on patients after obtaining a course certificate from a theoretic course and having performed a fixed number of supervised procedures. However, these methods do not ensure adequate competence. Well-structured and effective educational programmes including validated tests are needed to reduce economic expenses, optimize time spend, and ensure patient safety. The aim of this article is to summarize current state of educational strategies for technical procedures within respiratory medicine and thoracic surgery. Thus, to discuss future recommendations for curriculum development and assessment of competences based on Kern’s framework. The approach by Kern consists of six topics, which needs to be considered and evaluated and in order to educate physicians and surgeons most effective and evidence-based. We present a practical guide contributing to future educators’ considerations on (I) problem identification and general needs assessment, (II) targeted needs assessment, (III) goals and objectives, (IV) educational strategies, (V) implementation, and finally (VI) evaluation and feedback.