Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis

Simulation-Based Training Program for Peripherally Inserted Central Catheter Placement: Randomized Comparative Study of in-Person Training With Synchronous Feedback Versus Distance Training With Asynchronous Feedback

INTRODUCTION

Simulation training that includes deliberate practice is effective for procedural skill training. Delivering feedback remotely and asynchronously has been examined for more cost-efficient training. This prospective randomized study aimed to compare 2 feedback techniques for simulation training: synchronous direct feedback versus asynchronous distance feedback (ASYNC).

METHODS

Forty anesthesia and internal medicine residents were recruited after study approval by the institutional ethics committee. Residents reviewed instructional material on an online platform and performed a pretraining assessment (PRE) for peripherally inserted central catheter (PICC) placement. Each resident was then randomly assigned to 1 of 2 training types, practice with synchronous direct feedback (SYNC) or practice with ASYNC. Training consisted of four, 1-hour practice sessions; each was conducted once per week. Both groups underwent posttraining evaluation (POST). The PRE and POST assessments were videotaped and evaluated by 2 independent, blinded reviewers using a global rating scale.

RESULTS

Thirty-five residents completed the training program and both evaluations. Both groups had significantly improved global rating scale scores after 4 sessions. The SYNC group improved from 28 to 45 points ( P < 0.01); the ASYNC group improved from 26.5 to 46 points ( P < 0.01). We found no significant between-group differences for the PRE ( P = 0.42) or POST assessments ( P = 0.13).

CONCLUSION

This simulation-based training program significantly improved residents' peripherally inserted central venous catheter placement skills using either modality. With these results, we are unable to demonstrate the superiority of synchronous feedback over ASYNC. Asynchronous feedback training modality represents a new, innovative approach for health care procedural skills training.

Mastery Learning Guided by Artificial Intelligence Is Superior to Directed Self-Regulated Learning in Flexible Bronchoscopy Training: An RCT

INTRODUCTION

Simulation-based training has proven effective for learning flexible bronchoscopy. However, no studies have tested the efficacy of training toward established proficiency criteria, i.e., mastery learning (ML). We wish to test the effectiveness of ML compared to directed self-regulated learning (DSRL) on novice bronchoscopists' end-of-training performance.

METHODS

In a standardized simulated setting, novices without prior bronchoscopy experience were trained using an artificial intelligence (AI) guidance system that automatically recognizes the bronchial segments. They were randomized into two groups: the ML group and the DSRL group. The ML group was trained until they completed two procedures meeting the proficiency targets: 18 inspected segments, 18 structured progressions, <120-s procedure time. The DSRL group was trained until they no longer perceived any additional benefits from training. Both groups then did a finalizing test, without the AI guidance enabled.

RESULTS

A total of 24 participants completed the study, with 12 in each group. Both groups had a high mean number of inspected segments (ML = 17.2 segments, DSRL = 17.3 segments, p = 0.85) and structured progressions (ML = 15.5 progressions, DSRL = 14.8 progressions, p = 0.58), but the ML group performed the test procedure significantly faster (ML = 107 s, DSRL = 180 s, p < 0.001). The ML did not spend significantly longer time training (ML = 114 min, DSRL = 109 min, p = 0.84).

CONCLUSIONS

ML is a very efficient training form allowing novice trainees to learn how to perform a thorough, systematic, and quick flexible bronchoscopy. ML does not require longer time spent training compared to DSRL, and we therefore recommend training of future bronchoscopists by this method.

Colonoscopy Training on Virtual-Reality Simulators or Physical Model Simulators: A Randomized Controlled Trial

OBJECTIVE

This study employed a randomized controlled trial to assess the efficacy of virtual-reality (VR) simulators and physical model simulators on colonoscopy training to explore the optimal and evidence-based simulation training.

DESIGN

Forty participants were divided into 2 groups and randomized as dyads: the VR simulator group and the physical model simulator group. All the participants performed a baseline test through porcine colonoscopy. After a 6 h simulation training, each participant underwent a post-test on a pig after bowel preparation, and the procedures were video-recorded. Both the baseline test and the post-test were blindly assessed by 2 experienced assistant director physicians based on the GAGES-C scoring system.

SETTING

Simulation center, Renji Hospital, Shanghai Jiaotong University School of Medicine, Shanghai.

PARTICIPANTS

Forty surgical residents without colonoscopy experience.

RESULTS

Both the VR simulator group and the physical model simulator group improved significantly over the baseline test. The VR simulator group performed significantly better than the physical model simulator group, p=0.042. The participants in both groups expressed a high level of simulator satisfaction.

CONCLUSIONS

Novice residents can benefit from both VR simulators and physical model simulators. The VR simulator was shown to be more effective for colonoscopy training. VR simulators were more recommended for novices conducting basic colonoscopy training.

Educational outcomes of simulation-based training in regional anaesthesia: a scoping review

BACKGROUND

Structured training in regional anaesthesia includes pretraining on simulation-based educational platforms to establish a safe and controlled learning environment before learners are provided clinical exposure in an apprenticeship model. This scoping review was designed to appraise the educational outcomes of current simulation-based educational modalities in regional anaesthesia.

METHODS

This review conformed to PRISMA-ScR guidelines. Relevant articles were searched in PubMed, Scopus, Google Scholar, Web of Science, and EMBASE with no date restrictions, until November 2023. Studies included randomised controlled trials, pre-post intervention, time series, case control, case series, and longitudinal studies, with no restrictions to settings, language or ethnic groups. The Kirkpatrick framework was applied for extraction of educational outcomes.

RESULTS

We included 28 studies, ranging from 2009 to 2023, of which 46.4% were randomised controlled trials. The majority of the target population was identified as trainees or residents (46.4%). Higher order educational outcomes that appraised translation to real clinical contexts (Kirkpatrick 3 and above) were reported in 12 studies (42.9%). Two studies demonstrated translational patient outcomes (Level 4) with reduced incidence of paraesthesia and clinical complications. The majority of studies appraised Level 3 outcomes of performance improvements in either laboratory simulation contexts (42.9%) or demonstration of clinical performance improvements in regional anaesthesia (39.3%).

CONCLUSIONS

There was significant heterogeneity in the types of simulation modalities used, teaching interventions applied, study methodologies, assessment tools, and outcome measures studied. When improvisations were made to regional anaesthesia simulation platforms (hybrid simulation), there were sustained educational improvements beyond 6 months. Newer technology-enhanced innovations such as virtual, augmented, and mixed reality simulations are evolving, with early reports of educational effectiveness.

Educational and personal impacts of the COVID-19 pandemic on emergency medicine resident physicians: a qualitative study

BACKGROUND

The COVID-19 pandemic had a significant impact on both the clinical practice and the psychological states of frontline physicians in the emergency department. Trainees, at the beginning of their careers and thus still developing their practice styles and identities as physicians, were uniquely affected.

OBJECTIVE

In this qualitative study, we sought to explore how the pandemic environment shaped the experiences of emergency medicine resident physicians.

METHODS

This was a qualitative study. We conducted in-depth interviews with emergency medicine faculty, resident physicians, and staff at a single emergency department based at an urban academic institution in the northeastern United States. Interviews were audio recorded and transcribed, and transcripts were then analyzed in an iterative process by our coding team for recurring themes related to the resident experience.

RESULTS

We reached data saturation with 27 individuals. Of those who were interviewed, 10 were resident physicians [6 senior residents (PGY-3 or PGY-4) and 4 junior residents (PGY-1 or PGY-2)]. Three major recurring themes regarding resident physician experience emerged during our analysis of the interviews: (1) novel educational experiences dampened by negative structural forces from the pandemic, (2) fracturing of social interactions and mitigation through ad-hoc support systems and community of practice, and (3) development of negative emotions and psychological trauma including fear, resentment, and moral injury causing lasting harm.

CONCLUSIONS

Our results suggest that emergency medicine resident physicians training during the COVID-19 pandemic faced unique experiences concerning their education, social support systems, and emotional states. While the educational and social experiences were described as having both negative and positive impacts, the emotional experiences were largely negative. Residency program leadership may use these insights to improve resident preparation, wellness, and resilience in the face of future adverse events.

Mastery Learning in Preclinical Endodontics Using Customized 3D-Printed Tooth Models for Deliberate Practice: An Application of Educational Design Research

INTRODUCTION

This study, conducted within an educational design research (EDR) framework, assessed the use of customized 3D-printed tooth models at various difficulty levels to enhance mastery learning and deliberate practice in preclinical endodontic training.

METHODS

The EDR was conducted in a preclinical endodontic training involving 42 third-year students. The study focused on developing and evaluating 3D-printed tooth models customized into 3 difficulty levels to facilitate mastery learning for the access opening procedure on upper anterior teeth. To promote deliberate practice, we ensured ample availability of these models. The evaluation combined quantitative analysis, using Friedman and Wilcoxon Signed-Rank tests to assess practice volume and performance, with qualitative feedback from interviews and focus groups, analyzed via content analysis.

RESULTS

There was a significant reduction in practice time across the different model levels, with students using an average of 1.5-1.8 models per level. No significant differences in performance scores were observed across levels and natural tooth (P = .333). Feedback highlighted the models' educational value in enhancing dental training, indicating their effectiveness in improving learning experiences and skill development in endodontic education.

CONCLUSIONS

The innovative design of a 3D-printed tooth model system, which features 3 levels of difficulty and was developed within an EDR framework, allowed for tailored learning progressions and ample practice opportunities. This significantly enhanced the endodontic training experience and skill development by providing varied and realistic challenges.

Simulation-based education in ultrasound - diagnostic and interventional abdominal focus

Simulation-based training (SBT) is increasingly acknowledged worldwide and has become a popular tool for ultrasound education. Ultrasound simulation involves the use of technology and software to create a virtual training setting. Simulation-based training allows healthcare professionals to learn, practice, and improve their ultrasound imaging skills in a safe learning-based environment. SBT can provide a realistic and focused learning experience that creates a deep and immersive understanding of the complexity of ultrasound, including enhancing knowledge and confidence in specific areas of interest. Abdominal ultrasound simulation is a tool to increase patient safety and can be a cost-efficient training method. In this paper, we provide an overview of various types of abdominal ultrasound simulators, and the benefits, and challenges of SBT. We also provide examples of how to develop SBT programs and learning strategies including mastery learning. In conclusion, the growing demand for medical imaging increases the need for healthcare professionals to start using ultrasound simulators in order to keep up with the rising standards.

Automated performance metrics and surgical gestures: two methods for assessment of technical skills in robotic surgery

The objective of this study is to compare automated performance metrics (APM) and surgical gestures for technical skills assessment during simulated robot-assisted radical prostatectomy (RARP). Ten novices and six experienced RARP surgeons performed simulated RARPs on the RobotiX Mentor (Surgical Science, Sweden). Simulator APM were automatically recorded, and surgical videos were manually annotated with five types of surgical gestures. The consequences of the pass/fail levels, which were based on contrasting groups' methods, were compared for APM and surgical gestures. Intra-class correlation coefficient (ICC) analysis and a Bland-Altman plot were used to explore the correlation between APM and surgical gestures. Pass/fail levels for both APM and surgical gesture could fully distinguish between the skill levels of the surgeons with a specificity and sensitivity of 100%. The overall ICC (one-way, random) was 0.70 (95% CI: 0.34-0.88), showing moderate agreement between the methods. The Bland-Altman plot showed a high agreement between the two methods for assessing experienced surgeons but disagreed on the novice surgeons' skill level. APM and surgical gestures could both fully distinguish between novices and experienced surgeons in a simulated setting. Both methods of analyzing technical skills have their advantages and disadvantages and, as of now, those are only to a limited extent available in the clinical setting. The development of assessment methods in a simulated setting enables testing before implementing it in a clinical setting.

The State of Mastery Learning in Pediatric Graduate Medical Education: A Scoping Review

Objective

The aim of this study was to characterize the state of mastery learning interventions, identify gaps in current approaches, and highlight opportunities to improve the rigor of ML in pediatric graduate medical education (GME) training programs.

Methods

In October 2022, we searched Ovid, PubMed, Scopus, and ERIC. Two reviewers independently screened 165 citations and reviewed the full manuscripts of 20 studies. We developed a modified data abstraction tool based on the Recommendations for Reporting Mastery Education Research in Medicine (ReMERM) guidelines and extracted variables related to mastery learning curricular implementation and design and learner assessment.

Results

Eleven studies of ML approaches within pediatric GME were included in the full review, with over half published after 2020. ML interventions were used to teach both simple and complex tasks, often in heterogeneous learner groups. While deliberate practice and feedback were consistently reported features of ML in pediatrics, opportunities for improvement include clearly defining mastery, conducting learning over multiple sessions, presenting sufficient validity evidence for assessment tools, adhering to rigorous standard setting processes, and reporting psychometric data appropriate for ML.

Conclusion

In pediatric GME, ML approaches are in their infancy. By addressing common shortcomings in the existing literature, future efforts can improve the rigor of ML in pediatric training programs and its impact on learners and patients.

Procedural simulation in venipuncture for medical undergraduates and its transfer to the bedside: a cluster randomized study

Simulation is accepted as an effective method of learning procedural skills. However, the translational outcomes of skills acquired through simulation still warrants investigation. We designed this study to assess if skills laboratory training in addition to bedside learning (intervention group [IG]) would provide better learning results than bedside learning alone (control group [CG]) in the context of venipuncture training. This prospective, cluster-randomized, single-blind study took place at the Faculty of Medicine, University of Kelaniya, Sri Lanka. Seventeen clusters of second-year medical students were randomly assigned to either IG or CG. The IG trained on venipuncture in the skills laboratory, receiving instruction after modified Payton's Four Step Method. Following the training, students of both IG and CG underwent bedside learning for one month. Afterward, students of both groups performed venipuncture on actual patients in a clinical setting. An independent, blinded assessor scored students' performance using the Integrated Procedural Protocol Instrument (IPPI) and a checklist. Patients assessed students' performance with the Communication Assessment Tool (CAT). Eight and nine clusters were randomized to the intervention and control groups, respectively. IG completed significantly more single steps of the procedure correctly (IG: 19.36 ± 3.87 for checklist items; CG: 15.57 ± 4.95; p < 0.001). IG also scored significantly better on IPPI ratings (median: IG: 27 (12) vs. CG: 21 (8); p < 0.001). Rated by patients, students' communication skills did not significantly differ between the two groups. Simulation-based venipuncture training enabled students to perform the procedure on actual patients with a higher technical accuracy than students who learned venipuncture at the bedside. Students were able to transfer the skills acquired through venipuncture training at the skill laboratory to the bedside.

Development and Validation of the Needlestick Injury Prevention (N-SIP) Module

INTRODUCTION

Needlestick injuries (NSIs) pose a significant occupational hazard to healthcare workers (HCWs), with potential risks of exposure to bloodborne pathogens. The development of effective training modules is crucial to addressing NSI prevention and enhancing HCWs' knowledge and risk perception. This study aims to develop and validate the Needlestick Injury Prevention Module (N-SIP) using the ADDIE model (Florida State University, FL), which stands for Analysis, Design, Development, Implementation, and Evaluation, to improve NSI-related knowledge and risk perception among House Officers (HOs) in healthcare settings.

METHODS

The study utilized approaches comprising literature review, module development using the ADDIE model, content validation by experts, and face validation among HOs. The N-SIP module addressed various aspects of NSI prevention, including background information, bloodborne viral infections, infection prevention practices, and occupational safety. The evaluation involved content validation by expert panels and face validation by HOs.

RESULTS

The content validity of the N-SIP module was rigorously evaluated through expert review and validation by subject matter experts and HOs. The experts' feedback ensured the quality, relevance, and comprehensiveness of the module's instructional materials. Furthermore, face validity was assessed among HOs to ensure the module's clarity, appropriateness, and perceived effectiveness in addressing NSI prevention. The positive response from HOs indicated favorable perceptions of the module's content and instructional design, affirming its potential to effectively enhance perceptions related to NSI prevention among HCWs.

CONCLUSION

The development and evaluation of the N-SIP represent a significant advancement in addressing NSIs among HCWs. Through a structured approach informed by the ADDIE model, the N-SIP module offers a comprehensive and tailored learning experience aimed at enhancing NSI-related knowledge and risk perception among HOs. The study findings underscore the importance of effective training interventions in promoting a culture of safety and reducing occupational hazards in healthcare settings.

Using Simulation-Based Mastery Learning to Achieve Excellent Learning Outcomes in Physical Therapist Education

INTRODUCTION

The 2 aims of this observational study are (a) to describe the implementation and feasibility of a bed mobility skills simulation-based mastery learning (SBML) curricular module for physical therapist students and (b) to measure learning outcomes and student perceptions of this module.

REVIEW OF LITERATURE

Simulation-based mastery learning is an outcome-based educational approach that has been successful in other health professions but has not been explored in physical therapy education.

SUBJECTS

Eighty-seven students in a single cohort of a Doctor of Physical Therapy program.

METHODS

The SBML module in this pretest-posttest study included a pretest, instruction, initial posttest, and additional rounds of instruction and assessment as needed for all learners to achieve the minimum passing standard (MPS) set using the Mastery Angoff and Patient Safety methods. Outcome measures were bed mobility assessment pass rates and scores, additional student and faculty time compared with a traditional approach, and student perceptions of their self-confidence and the module.

RESULTS

All students achieved the MPS after 3 rounds of training and assessment beyond the initial posttest. Mean Total Scores improved from 67.6% (12.9%) at pretest to 91.4% (4.8%) at mastery posttest (P < .001, Cohen's d = 1.8, 95% CI [1.4-2.1]); mean Safety Scores improved from 75.2% (16.0%) at pretest to 100.0% (0.0%) at mastery posttest (P < .001, Cohen's d = 1.5, 95% CI [1.2-1.9]). Students who did not achieve the MPS at the initial posttest (n = 30) required a mean of 1.2 hours for additional instruction and assessment. Survey results revealed an increase in student confidence (P < .001) and positive student perceptions of the module.

DISCUSSION AND CONCLUSION

Implementation of this SBML module was feasible and resulted in uniformly high levels of bed mobility skill acquisition. Based on rigorous learning outcomes, feasible requirements for implementation, and increased student confidence, SBML offers a promising approach for wider implementation in physical therapy education.

Setting proficiency standards for simulation-based mastery learning of short antegrade femoral nail osteosynthesis: a multicenter study

BACKGROUND AND PURPOSE

Orthopedic trainees frequently perform short antegrade femoral nail osteosynthesis of trochanteric fractures, but virtual reality simulation-based training (SBT) with haptic feedback has been unavailable. We explored a novel simulator, with the aim of gathering validity evidence for an embedded test and setting a credible pass/fail standard allowing trainees to practice to proficiency.

PATIENTS AND METHODS

The research, conducted from May to September 2020 across 3 Danish simulation centers, utilized the Swemac TraumaVision simulator for short antegrade femoral nail osteosynthesis. The validation process adhered to Messick's framework, covering all 5 sources of validity evidence. Participants included novice groups, categorized by training to plateau (n = 14) or to mastery (n = 10), and experts (n = 9), focusing on their performance metrics and training duration.

RESULTS

The novices in the plateau group and experts had hands-on training for 77 (95% confidence interval [CI] 59-95) and 52 (CI 36-69) minutes while the plateau test score, defined as the average of the last 4 scores, was 75% (CI 65-86) and 96% (CI 94-98) respectively. The pass/fail standard was established at the average expert plateau test score of 96%. All novices in the mastery group could meet this standard and interestingly without increased hands-on training time (65 [CI 46-84] minutes).

CONCLUSION

Our study provides supporting validity evidence from all sources of Messick's framework for a simulation-based test in short antegrade nail osteosynthesis of intertrochanteric hip fracture and establishes a defensible pass/fail standard for mastery learning of SBT. Novices who practiced using mastery learning were able to reach the pre-defined pass/fail standard and outperformed novices without a set goal for external motivation.

Simulation-based learning in nephrology

Simulation is a technique to replace and amplify real experiences with guided ones that evoke or replicate substantial aspects of the real world in a fully interactive fashion. In nephrology (a particularly complex specialty), simulation can be used by patients, nurses, residents, and attending physicians alike. It allows one to learn techniques outside the stressful environment of care such as central venous catheter placement, arteriovenous fistula management, learning about peritoneal dialysis, or performing a kidney biopsy. Serious games and virtual reality are emerging methods that show promise. Simulation could also be important in relational aspects of working in a team or with the patient. The development of simulation as a teaching tool in nephrology allows for maintaining high-quality training for residents, tailored to their future practice, and minimizing risks for patients. Additionally, this education helps nephrologists maintain mastery of technical procedures, making the specialty attractive to younger generations. Unfortunately, the inclusion of simulation training programmes faces occasional logistical or funding limitations that universities must overcome with the assistance and innovation of teaching nephrologists. The impact of simulation-based teaching on clinical outcomes needs to be investigated in clinical studies.

Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR), Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) Guideline for difficult airway management. Part II

The Airway Management section of the Spanish Society of Anesthesiology, Resuscitation, and Pain Therapy (SEDAR), the Spanish Society of Emergency Medicine (SEMES), and the Spanish Society of Otorhinolaryngology and Head and Neck Surgery (SEORL-CCC) present the Guide for the comprehensive management of difficult airway in adult patients. Its principles are focused on the human factors, cognitive processes for decision-making in critical situations, and optimization in the progression of strategies application to preserve adequate alveolar oxygenation in order to enhance safety and the quality of care. The document provides evidence-based recommendations, theoretical-educational tools, and implementation tools, mainly cognitive aids, applicable to airway management in the fields of anesthesiology, critical care, emergencies, and prehospital medicine. For this purpose, an extensive literature search was conducted following PRISMA-R guidelines and was analyzed using the GRADE methodology. Recommendations were formulated according to the GRADE methodology. Recommendations for sections with low-quality evidence were based on expert opinion through consensus reached via a Delphi questionnaire.

Pediatric Critical Care Fellow Perception of Learning through Virtual Reality Bronchoscopy

Background

Virtual reality (VR) simulators have revolutionized training in bronchoscopy, offering unrestricted availability in a low-stakes learning environment and frequent assessments represented by automatic scoring. The VR assessments can be used to monitor and support learners' progression. How trainees perceive these assessments needs to be clarified.

Objective

The objective of this study was to examine what assessments learners select to document and receive feedback on and what influences their decisions.

Methods

We used a sequential explanatory mixed methods strategy. All participants were pediatric critical care medicine trainees requiring competency in bronchoscopy skills. During independent simulation practice, we collected the number of learning-focused practice attempts (scores not recorded), assessment-focused practice (scores recorded and reviewed by the instructor for feedback), and the amount of time each attempt lasted. After simulation training, we conducted interviews to explore learners' perceptions of assessment.

Results

There was no significant difference in the number of attempts for each practice type. The average time per learning-focused attempt was almost three times longer than the assessment-focused attempt (mean [standard deviation] 16 ± 1 min vs. 6 ± 3 min, respectively; P < 0.05). Learners perceived documentation of their scores as high stakes and only recorded their better scores. Learners felt safer experimenting if their assessments were not recorded.

Conclusion

During independent practice, learners took advantage of automatic assessments generated by the VR simulator to monitor their progression. However, the recording of scores from the simulation program to document learners' trajectory to a set goal was perceived as high stakes, discouraging learners from seeking supervisor feedback.

Effect of Polypectomy Simulation-Based Mastery Learning on Skill Retention Among Practicing Endoscopists

PURPOSE

Practicing endoscopists frequently perform and teach screening colonoscopies and polypectomies, but there is no standardized method to train and assess physicians who perform polypectomy procedures. The authors created a polypectomy simulation-based mastery learning (SBML) curriculum and hypothesized that completion of the curriculum would lead to immediate improvement in polypectomy skills and skill retention at 6 and 12 months after training.

METHOD

The authors performed a pretest-posttest cohort study with endoscopists who completed SBML and were randomized to follow-up at 6 or 12 months from May 2021 to August 2022. Participants underwent SBML training, including a pretest, a video lecture, deliberate practice, and a posttest. All learners were required to meet or exceed a minimum passing standard on a 17-item skills checklist before completing training and were randomized to follow-up at 6 or 12 months. The authors compared simulated polypectomy skills performance on the checklist from pretest to posttest and posttest to 6- or 12-month follow-up test.

RESULTS

Twenty-four of 30 eligible participants (80.0%) completed the SBML intervention, and 20 of 24 (83.3%) completed follow-up testing. The minimum passing standard was set at 93% of checklist items correct. The pretest passing rate was 4 of 24 participants (16.7%) compared with 24 of 24 participants (100%) at posttest ( P < .001). There were no significant differences in passing rates from posttest to combined 6- and 12-month posttest in which 18 of 20 participants (90.0%) passed.

CONCLUSIONS

Before training and despite years of clinical experience, practicing endoscopists demonstrated poor performance of polypectomy skills. SBML was an effective method for practicing endoscopists to acquire and maintain polypectomy skills during a 6- to 12-month period.

Education in Focused Assessment With Sonography for Trauma Using Immersive Virtual Reality: A Prospective, Interventional Cohort Study and Non-inferiority Analysis With a Historical Control

OBJECTIVE

Focused assessment with sonography for trauma (FAST) is a valuable ultrasound procedure in emergency settings, and there is a need for evidence-based education in FAST to ensure competencies. Immersive virtual reality (IVR) is a progressive training modality gaining traction in the field of ultrasound training. IVR holds several economic and practical advantages to the common instructor-led FAST courses using screen-based simulation (SBS).

METHODS

This prospective, interventional cohort study investigated whether training FAST using IVR unsupervised and out-of-hospital was non-inferior to a historical control group training at a 90 min SBS course in terms of developing FAST competencies in novices. Competencies were assessed in both groups using the same post-training simulation-based FAST test with validity evidence, and a non-inferiority margin of 2 points was chosen.

RESULTS

A total of 27 medical students attended the IVR course, and 27 junior doctors attended the SBS course. The IVR group trained for a median time of 117 min and scored a mean 14.2 ± 2.0 points, compared with a mean 13.7 ± 2.5 points in the SBS group. As the lower bound of the 95% confidence interval at 13.6 was within the range of the non-inferiority margin (11.7-13.7 points), training FAST in IVR for a median of 117 min was found non-inferior to training at a 90 min SBS course. No significant correlation was found between time spent in IVR and test scores.

CONCLUSION

Within the limitations of the use of a historical control group, the results suggest that IVR could be an alternative to SBS FAST training and suitable for unsupervised, out-of-hospital courses in basic FAST competencies.

How Many Operative Performance Ratings Does a Pediatric Surgery Fellow Need to Be Deemed Practice Ready?

PURPOSE

Identifying the number of cases required for a fellow to achieve competence has been challenging. Workplace-based assessment (WBA) systems make collecting performance data practical and create the opportunity to translate WBA ratings into probabilistic statements about a fellow's likelihood of performing to a given standard on a subsequent assessment opportunity.

METHODS

We compared data from two pediatric surgery training programs that used the performance rating scale from the Society for Improving Medical Professional Learning (SIMPL). We used a Bayesian generalized linear mixed effects model to examine the relationship past and future performance for three procedures: Laparoscopic Inguinal Hernia Repair, Laparoscopic Gastrostomy Tube Placement, and Pyloromyotomy.

RESULTS

For site one, 26 faculty assessed 9 fellows on 16 procedures yielding 1094 ratings, of which 778 (71%) earned practice-ready ratings. For site two, 25 faculty rated 3 fellows on 4 unique procedures yielding 234 ratings of which 151 (65%) were deemed practice-ready. We identified similar model-based future performance expectations, with prior practice-ready ratings having a similar average effect across both sites (Site one, B = 0.25; Site two, B = 0.25). Similar prior practice-ready ratings were needed for Laparoscopic G-Tube Placement (Site one = 13; Site two = 14), while greater differences were observed for Laparoscopic Inguinal Hernia Repair (Site one = 10; Site two = 15) and Pyloromyotomy (Site one = 10; Site two = 15).

CONCLUSION

Our approach to modeling operative performance data is effective at determining future practice readiness of pediatric surgery fellows across multiple faculty and fellow groups. This method could be used to establish minimum case number requirements.

TYPE OF STUDY

Original manuscript, Study of Diagnostic Test.

LEVEL OF EVIDENCE

II.

Distance simulation in the health professions: a scoping review

BACKGROUND

Distance simulation is defined as simulation experiences in which participants and/or facilitators are separated from each other by geographic distance and/or time. The use of distance simulation as an education technique expanded rapidly with the recent COVID-19 pandemic, with a concomitant increase in scholarly work.

METHODS

A scoping review was performed to review and characterize the distance simulation literature. With the assistance of an informationist, the literature was systematically searched. Each abstract was reviewed by two researchers and disagreements were addressed by consensus. Risk of bias of the included studies was evaluated using the Risk of Bias 2 (RoB 2) and Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) tools.

RESULTS

Six thousand nine hundred sixty-nine abstracts were screened, ultimately leading to 124 papers in the final dataset for extraction. A variety of simulation modalities, contexts, and distance simulation technologies were identified, with activities covering a range of content areas. Only 72 papers presented outcomes and sufficient detail to be analyzed for risk of bias. Most studies had moderate to high risk of bias, most commonly related to confounding factors, intervention classification, or measurement of outcomes.

CONCLUSIONS

Most of the papers reviewed during the more than 20-year time period captured in this study presented early work or low-level outcomes. More standardization around reporting is needed to facilitate a clear and shared understanding of future distance simulation research. As the broader simulation community gains more experience with distance simulation, more studies are needed to inform when and how it should be used.

Innovative Curriculum to Train Medical Students on Lateral Canthotomy and Cantholysis

PURPOSE

Orbital compartment syndrome (OCS) is an ocular emergency requiring prompt decompression with a lateral canthotomy and cantholysis (LCC) within 2 hours. This study evaluates the feasibility and effectiveness of a standardized LCC curriculum to train medical students to identify and treat OCS.

METHODS

This was a prospective, non-randomized, non-comparative cohort study of 39 novice first-year medical students with no prior LCC training who underwent a standardized LCC curriculum incorporating both didactic and hands-on procedural training. Didactic knowledge of orbital anatomy and OCS was evaluated with written pre- and post-knowledge testing. Expert performance criteria were determined by expert consensus based on the performance of three oculoplastic surgeons and were defined as correctly performing all 12 critical checklist steps of an LCC within 3 minutes twice consecutively on a Sonalyst LCC training system eye model. Utilizing the principles of mastery learning, participants learned how to perform an LCC in a classroom environment and were evaluated on a final test of proficiency in a training lane designed to simulate an austere military environment.

RESULTS

Participants required a median of 3.0 practice iterations to achieve expert performance in the classroom environment. During the testing phase, all participants correctly identified the eye with OCS, and 77% (n = 30) of learners successfully performed an LCC at the expert level within their first attempt. The median completion time of those who passed on their first testing was 130 seconds. The mean LCC knowledge test scores significantly improved from 48.7% to 71.2% (P < .001).

CONCLUSION

This study successfully developed a standardized LCC curriculum utilizing the principles of hands-on mastery learning to train novice learners to perform an LCC efficiently and effectively.

Using insights from cognitive science for the teaching of clinical skills: AMEE Guide No. 155

Students have to develop a wide variety of clinical skills, from cannulation to advanced life support, prior to entering clinical practice. An important challenge for health professions' educators is the implementation of strategies for effectively supporting students in their acquisition of different types of clinical skills and also to minimize skill decay over time. Cognitive science provides a unified approach that can inform how to maximize clinical skill acquisition and also minimize skill decay. The Guide discusses the nature of expertise and mastery development, the key insights from cognitive science for clinical skill development and skill retention, how these insights can be practically applied and integrated with current approaches used in clinical skills teaching.

Competency-based assessment in neonatal simulation-based training

Simulation is a cornerstone of training in neonatal clinical care, allowing learners to practice skills in a safe and controlled environment. Competency-based assessment provides a systematic approach to evaluating technical and behavioral skills observed in the simulation environment to ensure the learner is prepared to safely perform the skill in a clinical setting. Accurate assessment of competency requires the creation of tools with evidence of validity and reliability. There has been considerable work on the use of competency-based assessment in the field of neonatology. In this chapter, we review neonatal simulation-based training, examine competency-based assessment tools, explore methods to gather evidence of the validity and reliability, and review an evidence-based approach to competency-based assessment using simulation.

Pre-clerkship procedural training in venipuncture: a prospective cohort study on skills acquisition and durability

BACKGROUND

The effectiveness of simulation-based training for skill acquisition is widely recognized. However, the impact of simulation-based procedural training (SBPT) on pre-clerkship medical students and the retention of procedural skills learned through this modality are rarely investigated.

METHODS

A prospective cohort study was conducted among pre-clerkship medical students. Learners underwent SBPT in venipuncture in the skills laboratory. Assessments were conducted at two main points: 1) immediate assessment following the training and 2) delayed assessment one year after training. Learner self-assessments, independent assessor assessments for procedural competency, and communication skills assessments were conducted in both instances. The students were assessed for their competency in performing venipuncture by an independent assessor immediately following the training in the simulated setting and one-year post-training in the clinical setting, using the Integrated Procedural Protocol Instrument (IPPI). The student's communication skills were assessed by standardized patients (SP) and actual patients in the simulated and clinical settings, respectively, using the Communication Assessment Tool (CAT).

RESULTS

Fifty-five pre-clerkship medical students were recruited for the study. A significant increase was observed in self-confidence [mean: 2.89 SD (Standard Deviation) (0.69)] and self-perceived competency [mean: 2.42 SD (0.57)] in performing venipuncture, which further improved at the delayed assessment conducted in the clinical setting (p < 0.001). Similarly, the IPPI ratings showed an improvement [immediate assessment: mean: 2.25 SD (1.62); delayed assessment: mean: 2.78 SD (0.53); p < 0.01] in venipuncture skills when assessed by an independent assessor blinded to the study design. A significant difference (p < 0.01) was also observed in doctor-patient communication when evaluated by SPs [mean: 2.49 SD (0.57)] and patients [mean: 3.76 SD (0.74)].

CONCLUSION

Simulation-based venipuncture training enabled students to perform the procedure with confidence and technical accuracy. Improved rating scores received at a one-year interval denote the impact of clinical training on skills acquisition. The durability of skills learned via SBPT needs to be further investigated.

Gathering validity evidence for a 3D-printed simulator for training of myringotomy and ventilation tube insertion

Objectives

This study aimed to gather validity evidence according to Messick's framework for a novel 3D-printed simulator for myringotomy with ventilation tube insertion for use in technical skills training of otorhinolaryngology (ORL) residents.

Methods

The study included 15 junior ORL residents (trainees) and 13 experienced teaching otolaryngologists (experts). Experts and trainees first received an identically structured introduction to the procedure, simulator, and simulation setup. Five procedures performed by each participant were video-recorded and ordered randomly for blinded rating by two independent raters. The rating tools used were a global rating scale (GBRS) and a task-specific checklist. Validity evidence was collected according to Messick's framework. Differences in time consumption and performance scores were analyzed. Finally, a pass/fail standard was established using the contrasting groups' method.

Results

Trainees used significantly more time per procedure (109 s, 95% CI: 99-120) than experts (82 s, 95% CI: 71-93; p < .001). Adjusted for repetition and rater leniency, experts achieved an average GBRS score of 18.8 (95% CI: 18.3-19.2) out of 20 points, whereas trainees achieved an average of 17.1 points (95% CI: 16.6-17.5; p < .001). In contrast to the task-specific checklist, the GBRS score discriminated between repetition number and participant experience. The pass/fail standard for the GBRS was established at 18.4 points.

Conclusion

We established educational validity evidence for a novel 3D-printed model for simulation-based training of ventilation tube insertion and established a reliable pass/fail standard.

Level of Evidence

1b.

Fostering Adaptive Expertise Through Simulation

Technology-enhanced simulation has been used to tackle myriad challenges within health professions education. Recently, work has typically adopted a mastery learning orientation that emphasizes trainees' sequential mastery of increasingly complex material. Doing so has privileged a focus on performance and task completion, as captured by trainees' observable behaviors and actions. Designing simulation in these ways has provided important advances to education, clinical care, and patient safety, yet also placed constraints around how simulation-based activities were enacted and learning outcomes were measured. In tracing the contemporary manifestations of simulation in health professions education, this article highlights several unintended consequences of this performance orientation and draws from principles of adaptive expertise to suggest new directions. Instructional approaches grounded in adaptive expertise in other contexts suggest that uncertainty, struggle, invention, and even failure help learners to develop deeper conceptual understanding and learn innovative approaches to novel problems. Adaptive expertise provides a new lens for simulation designers to think intentionally around how idiosyncrasy, individuality, and inventiveness could be enacted as central design principles, providing learners with opportunities to practice and receive feedback around the kinds of complex problems they are likely to encounter in practice. Fostering the growth of adaptive expertise through simulation will require a fundamental reimagining of the design of simulation scenarios, embracing the power of uncertainty and ill-defined problem spaces, and focusing on the structure and pedagogical stance of debriefing. Such an approach may reveal untapped potential within health care simulation.

Virtual Reality Technology in Nursing Professional Skills Training: Bibliometric Analysis

BACKGROUND

Nursing professional skills training has undergone significant transformation due to the exponential growth of computer and medical technology. The innovative use of virtual reality (VR) in nursing education has emerged as a cutting-edge technical support technique that has gained attention as a highly effective method for improving nurse training quality.

OBJECTIVE

This study aims to review the current status of VR technology in nursing professional skills training, research hotspots, and emerging trends in the last 15 years.

METHODS

The Web of Science Core Collection database was used to search for literature on VR technology in nursing professional skills training covering the period from 2006 to 2022. Biblioshiny (K-Synth Srl) was used to import and convert the records to Bibliometrix (K-Synth Srl) for analysis, and R (R Core Team) was used for descriptive bibliometric analysis. VOSviewer (Leiden University) was used to cluster co-occurring keywords, and Scimago Graphica (version 1.0.16; Scimago Lab) was used to generate a geographical visualization of published countries and regions.

RESULTS

A total of 1073 papers were analyzed, indicating a surge in research on the application of VR in nursing professional skills training in recent years, as evidenced by a positive trend in annual publication of relevant literature. The majority of studies were from the United States (n=340) and Canada (n=107), and Margaret Verkuyl was the most prolific author, leading the way with 9 publications. Furthermore, "Computerized Virtual Patients in Health Professions Education: a Systematic Review and Meta-Analysis" was the most frequently cited reference. Keywords such as education, simulation, skills, students, and care were most commonly used by researchers.

CONCLUSIONS

The bibliometric analysis provides a comprehensive overview of the use of VR in nursing professional skills training, indicating that VR-based training is an effective means of improving the skills and competencies of nursing students and professionals alike. The COVID-19 pandemic has reinforced the importance of developing VR-based distance education, despite challenges such as integrating virtual and real-world training and mitigating safety risks.

Are we talking about practice? A randomized study comparing simulation-based deliberate practice and mastery learning to self-guided practice

OBJECTIVES

Simulation-based technical skills training is now ubiquitous in medicine, particularly for high acuity, low occurrence (HALO) procedures. Mastery learning and deliberate practice (ML + DP) are potentially valuable educational methods, however, they are resource intensive. We sought to compare the effect of deliberate practice and mastery learning versus self-guided practice on skill performance of the rare, life-saving procedure, a bougie-assisted cricothyroidotomy (BAC).

METHODS

We conducted a multi-center, randomized study at five North American emergency medicine (EM) residency programs. We randomly assigned 176 EM residents to either the ML + DP or self-guided practice groups. Three blinded airway experts independently evaluated BAC skill performance by video review before (pre-test), after (post-test) and 6-12 months (retention) after the training session. The primary outcome was post-test skill performance using a global rating score (GRS). Secondary outcomes included performance time and skill performance at the retention test.

RESULTS

Immediately following training, GRS scores were significantly higher as mean performance improved from pre-test, (22, 95% CI = 21-23) to post-test (27, 95% CI = 26-28), (p < 0.001) for all participants. However, there was no difference between the groups on GRS scores (p = 0.2) at the post-test or at the retention test (p = 0.2). At the retention test, participants in the ML + DP group had faster performance times (66 s, 95% CI = 57-74) compared to the self-guided group (77 s, 95% CI = 67-86), (p < 0.01).

CONCLUSIONS

There was no significant difference in skill performance between groups. Residents who received deliberate practice and mastery learning demonstrated an improvement in skill performance time.

Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial

Importance

In cardiac device implant training, there is no common system to objectively assess trainees' ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators' early learning curve.

Objective

To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT).

Design, Setting, and Participants

In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022.

Intervention

Training with PBP vs SBT.

Main Outcome and Measures

The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined.

Results

Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees.

Conclusions and Relevance

In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications.

Trial Registration

ClinicalTrials.gov Identifier: NCT05952908.

Learning insertion of a Resuscitative Endovascular Balloon Occlusion of the Aorta (REBOA) catheter: Is clinical experience necessary? A prospective trial

BACKGROUND

Resuscitative endovascular balloon occlusion of the aorta (REBOA) is an emerging and potentially life-saving procedure, necessitating qualified operators in an increasing number of centres. The procedure shares technical elements with other vascular access procedures using the Seldinger technique, which is mastered by doctors not only in endovascular specialties but also in trauma surgery, emergency medicine, and anaesthesiology. We hypothesised that doctors mastering the Seldinger technique (experienced anaesthesiologist) would learn the technical aspects of REBOA with limited training and remain technically superior to doctors unfamiliar with the Seldinger technique (novice residents) given similar training.

METHODS

This was a prospective trial of an educational intervention. Three groups of doctors were enroled: novice residents, experienced anaesthesiologists, and endovascular experts. The novices and the anaesthesiologists completed 2.5 h of simulation-based REBOA training. Their skills were tested before and 8-12 weeks after training using a standardised simulated scenario. The endovascular experts, constituting a reference group, were equivalently tested. All performances were video recorded and rated by three blinded experts using a validated assessment tool for REBOA (REBOA-RATE). Performances were compared between groups and with a previously published pass/fail cutoff.

RESULTS

Sixteen novices, 13 board-certified specialists in anaesthesiology, and 13 endovascular experts participated. Before training, the anaesthesiologists outperformed the novices by 30 percentage points of the maximum REBOA-RATE score (56% (SD 14.0) vs 26% (SD 17%), p<0.01). After training, there was no difference in skills between the two groups (78% (SD 11%) vs 78 (SD 14%), p = 0.93). Neither group reached the endovascular experts' skill level (89% (SD 7%), p<0.05).

CONCLUSION

For doctors mastering the Seldinger technique, there was an initial inter-procedural transfer of skills advantage when performing REBOA. However, after identical simulation-based training, novices performed equally well to anaesthesiologists, indicating that vascular access experience is not a prerequisite to learning the technical aspects of REBOA. Both groups would need more training to reach technical proficiency.

Comparing Simulator Metrics and Rater Assessment of Laparoscopic Suturing Skills

BACKGROUND

Laparoscopic intracorporeal suturing is important to master and competence should be ensured using an optimal method in a simulated environment before proceeding to real operations. The objectives of this study were to gather validity evidence for two tools for assessing laparoscopic intracorporeal knot tying and compare the rater-based assessment of laparoscopic intracorporeal suturing with the assessment based on simulator metrics.

METHODS

Twenty-eight novices and 19 experienced surgeons performed four laparoscopic sutures on a Simball Box simulator twice. Two surgeons used the Intracorporeal Suturing Assessment Tool (ISAT) for blinded video rating.

RESULTS

Composite Simulator Score (CSS) had higher test-retest reliability than the ISAT. The correlation between the number performed procedures including suturing and ISAT score was 0.51, p<0.001, and 0.59 p<0.001 for CSS. We found an inter-rater reliability (0.72, p<0.001 for test 1 and 0.53 p<0.001 for test 2). The pass/fail rates for ISAT and CSS were similar.

CONCLUSION

CSS and ISAT provide similar results for assessing laparoscopic suturing but assess different aspects of performance. Using simulator metrics and raters' assessments in combination should be considered for a more comprehensive evaluation of laparoscopic knot-tying competency.

Simulation-based Mastery Learning Improves Critical Care Skills of Advanced Practice Providers

Background

Advanced practice providers (APPs) are essential members of intensive care unit (ICU) interprofessional teams and are expected to be competent in performing procedures. There are no published criteria for establishing when APPs can independently perform procedures. Simulation-based mastery learning (SBML) is an effective strategy for improving critical care skills but has not been applied to practicing ICU APPs.

Objective

The purpose of this study was to evaluate if an SBML curriculum could improve the critical care skills and procedural self-confidence of ICU APPs.

Methods

We performed a pretest-posttest study of central venous catheter (CVC) insertion, thoracentesis, and mechanical ventilation (MV) management skills among ICU APPs who participated in an SBML course at an academic hospital. For each skill, APPs underwent baseline skills assessments (pretests) on a simulator using previously published checklists, followed by didactic sessions and deliberate practice with individualized feedback. Within 2 weeks, participants were required to meet or exceed previously established minimum passing standards (MPS) on simulated skills assessments (posttests) using the same checklists. Further deliberate practice was provided for those unable to meet the MPS until they retested and met this standard. We compared pretest to posttest skills checklist scores and procedural confidence.

Results

All 12 eligible ICU APPs participated in internal jugular CVC, subclavian CVC, and MV training. Five APPs participated in thoracentesis training. At baseline, no APPs met the MPS on all skills. At training completion, all APPs achieved the mastery standard. Internal jugular CVC pretest performance improved from a mean of 67.2% (standard deviation [SD], 28.8%) items correct to 97.1% (SD, 3.8%) at posttest (P = 0.005). Subclavian CVC pretest performance improved from 29.2% (SD, 32.7%) items correct to 93.1% (SD 3.9%) at posttest (P < 0.001). Thoracentesis pretest skill improved from 63.9% (SD, 30.6%) items correct to 99.2% (SD, 1.7%) at posttest (P = 0.054). Pretest MV skills improved from 54.8% (SD, 19.7%) items correct to 92.3% (SD, 5.0%) at posttest (P < 0.001). APP procedural confidence improved for each skill from pre to posttest.

Conclusion

SBML is effective for training APPs to perform ICU skills. Relying on traditional educational methods does not reliably ensure that APPs are adequately prepared to perform skills such as CVC insertion, thoracentesis, and MV management.

Simulation-based Mastery Learning Improves Emergency Medicine Residents' Ability to Perform Temporary Transvenous Cardiac Pacing

INTRODUCTION

Temporary transvenous cardiac pacing (TVP) is a critical intervention that emergency physicians perform infrequently in clinical practice. Prior simulation studies revealed that emergency medicine (EM) residents and board-certified emergency physicians perform TVP poorly during checklist-based assessments. Our objective in this report was to describe the design and implementation of a simulation-based mastery learning (SBML) curriculum and evaluate its impact on EM residents' ability to perform TVP.

METHODS

An expert panel of emergency physicians and cardiologists set a minimum passing standard (MPS) for a previously developed 30-item TVP checklist using the Mastery Angoff approach. Emergency medicine residents were assessed using this checklist and a high-fidelity TVP task trainer. Residents who did not meet the MPS during baseline testing viewed a procedure video and completed a 30-minute individual deliberate practice session before retesting. Residents who did not meet the MPS during initial post-testing completed additional deliberate practice and assessment until meeting or exceeding the MPS.

RESULTS

The expert panel set an MPS of correctly performing 28 (93.3%) checklist items. Fifty-seven EM residents participated. Mean checklist scores improved from 13.4 (95% CI 11.8-15.0) during baseline testing to 27.5 (95% CI 26.9-28.1) during initial post-testing (P < 0.01). No residents met the MPS at baseline testing. The 21 (36.8%) residents who did not meet the MPS during initial post-testing all met or exceeded the MPS after completing one additional 30-minute deliberate practice session.

CONCLUSION

Emergency medicine residents demonstrated significantly improved TVP performance with reduced variability in checklist scores after completing a simulation-based mastery learning curriculum.

Videos of demonstration versus text and image-based material for pre-skill conceptualisation in flipped newborn resuscitation training for medical students: a pilot study

BACKGROUND

The flipped skills lab is a student-centred approach which incorporates pre-class preparation (pre-skill conceptualization) followed by repeated, hands-on practice for practical skills training. Objective measures of skills acquisition in the flipped literature are few and conflicting. The importance of pre-skill conceptualization in flipped skills training suggests that pedagogically informed pre-skill conceptualization can enhance outcomes.

METHODS

A mixed quasi-experimental study was conducted on 41 final year medical students who followed a flipped newborn resuscitation skills lab. Pre-class preparatory material covered conceptual and procedural knowledge. Students in the traditional group (n = 19) and those in the interventionalmental group (n = 22) received identical reading material covering conceptual knowledge. Procedural knowledge was shared with the interventional group as demonstration videos, while the traditional group received a PowerPoint presentation with text and images covering the same material. Knowledge acquisition was assessed by 20 single best answer questions before and after hands-on practice in the skills lab and skill performance was tested post-intervention with a simulated scenario. Students' perceptions were collected by survey. Quantitative data was analysed using Wilcoxon Signed Ranks test and Mann-Whitney U test as appropriate. Qualitative data was analysed by thematic analysis.

RESULTS

Overall student rating of the intervention was positive with ratings of 4.54 and 4.46 out of 5 by the traditional group and the experimental group respectively. Post-intervention skill performance in the experimental group was significantly better (p < .05) in the interventional group (M = 87.86%, SD = 5.89) than in the traditional group (M = 83.44, SD = 5.30) with a medium effect size (r = .40). While both groups showed significant knowledge gains, only students in the experimental group showed a statistically significant gain in procedural knowledge (p < .05) following the flipped skills lab. Finally, while both groups self-reported feeling more knowledgeable and confident following the intervention, the level of confidence was superior in the experimental group.

CONCLUSIONS

Flipping the skills lab with pre-skill conceptualisation combining text-based conceptual knowledge and video-based procedural knowledge followed by simulation-based hands-on practice improves procedural knowledge and skills acquisition in newborn resuscitation training for medical students. This study shows that in addition to temporal benefits, pedagogically informed pre-skill conceptualization can confer procedure-specific cognitive and emotional benefits supporting skills acquisition.

Learning practical nursing skills in simulation centers - A narrative review

Background

Practical skills are complex procedures integrating communication and caring, as well as technical and manual aspects. Simulation at a simulation/skills center offers a wide range of learning activities and aims to imitate patient situations.

Objectives

To investigate the international research literature on practical skills learning in simulation/skills centers in nursing education. Research questions: 1. What are the range and type of practical skills studied? 2. What learning activities are focused on in the studies included in the review? 3. What are the learning outcomes and how are they assessed?

Design

Narrative review.

Methods

We searched electronically and included studies from Medline Ovid, CINAHL, Eric, Embase, Academic Search Premiere, and Cochrane. Unique indexing terms and search strategies were developed for each database. The criteria for inclusion were bachelor nursing students as the study population and practical nursing skills learning in simulation/skills centers. We used Rayyan QCRIt for the initial screening and the Mixed Method Appraisal Tool for quality assessment. We used a narrative approach to synthesize the diverse range of studies.

Findings

One hundred and twenty-one studies from 26 countries published between January 2013 and March 2022 were included. The amount of quantitative research was overwhelming (n = 108). A total of 50 different practical skills were represented. The studies focused on which learning modalities resulted in the best learning outcomes. Only 8.5% (n = 7) of the included studies concerned students' learning processes. Skill performance (n = 101), knowledge (n = 57), confidence (n = 34), and satisfaction (n = 32) were the main learning outcomes measured.

Discussion

The quality assessment indicated that 10 of the studies achieved 100% on the mixed method appraisal tool criteria. In many of the studies with quasi-experimental and randomized controlled trial designs, the intervention group received some form of educational treatment while the control group received no treatment. The choice of no treatment for the control group in pedagogical research seems to disregard the inherent purpose and effect of teaching and learning.

Conclusion

Heterogeneity in the use of learning modalities and measuring instruments precludes the possibility of building on other research. Technical skills were the preferred choice of skill, while skills that involved a fair measure of communication and collaboration were only sparingly studied. Students' learning processes were barely touched on in the included studies. More focus should be placed on this area in further research, since the choice of learning modalities may affect the students' learning processes in significant ways.

Development, implementation, and evaluation of a simulation-based educational curriculum for pediatric hospitalists

BACKGROUND AND OBJECTIVES

Minimal published simulation-based educational training exists for practicing pediatric hospitalists. Our aim was to determine specific pediatric hospital medicine (PHM) knowledge, skill, and competency needs aligned with our scope of practice and evaluate the impact of a simulation-based training curriculum.

DESIGN AND METHODS

Baseline and post-training surveys were administered to 48 physicians providing self-ratings on a 5-point scale from Novice to Expert on published PHM competencies. Results were used to develop a targeted simulation curriculum. Participants were considered competent in a domain if their mean score was 3 or greater. We categorized participant responses to individual questions into nine domain scores on survey self-assessments. Score analysis was performed using the signed-rank test and McNemar's test. Post-training evaluations solicited curriculum acceptance and perceived clinical value.

RESULTS

The baseline response rate was 98% and the post-training response rate was 85%. Areas with the lowest competency on baseline self-assessment included advanced airway management (38%), vascular access and emergency medications (38%), code cart skills (19%), team communication (51%), and medically complex care (49%). Post-training scores improved significantly for five of nine domains, with the largest gains in the "not competent" at baseline group. Percent competent (% with mean score >3) increased significantly in three domains (advanced airway management, code cart skills, and complex care). Participants rated educational sessions favorably (98%) and most (95%) reported using knowledge/skills learned for patient care.

CONCLUSION

Baseline self-assessment results were instrumental in curriculum design. Post-training analysis revealed gains in multiple domains and identified opportunities for future interventions. Most hospitalists reported participation positively impacted patient care with high learner satisfaction.

"Hands off hand hygiene training": Implementation of a COVID safe auditor training program

Optimal hand hygiene practices reduce the risk of healthcare-associated infections, especially in high-risk settings of immunocompromised patients. In 2020, face-to-face learning was disallowed in the environment of coronavirus disease 2019 transmission. We developed a revised learning program for hand hygiene auditors for our cancer care facility. The learning package resulted in a 2-fold increase in the number of participants, with effective promotion by managers, due in part to reduced time and resources for training, and flexibility for staff.

Assessing VATS competence based on simulated lobectomies of all five lung lobes

OBJECTIVES

To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS

Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.

RESULTS

Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).

CONCLUSION

We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.

Simulation-Based Mastery Learning to Facilitate Transition to Nursing Practice

BACKGROUND

Simulation-based mastery learning (SBML) was developed and implemented to facilitate nursing students' transition to clinical practice.

PURPOSE

To evaluate nursing students' perspectives on the impact of SBML on their transition to clinical practice.

METHOD

A descriptive qualitative study using focus group discussions was conducted with final-year nursing students who had completed their clinical practicum and SBML.

RESULTS

Three themes emerged: authenticity of simulations, whereby students were able to draw some parallels between their simulations and clinical experiences; self-efficacy in clinical practice, which was gained through deliberate practice; and application of cognitive tools, which provided mental frameworks to guide clinical performances.

CONCLUSION

This study highlights the need to ensure the fidelity of the simulations to achieve authentic experiences, provide deliberate practice to develop self-efficacy, and use cognitive tools for mental model formation to create a high-quality SBML program to foster better transfer of simulation learning to real-life setting.

Simulation-based skills training: a qualitative interview study exploring surgical trainees' experience of stress

INTRODUCTION

Stress can affect the ability to acquire technical skills. Simulation-based training (SBT) courses allow surgical trainees to train their technical skills away from stressful clinical environments. Trainees' subjective experiences of stress during SBT courses on laparoscopic surgery remains understudied. Here, we explored the subjective stress experiences of surgical trainees during mandatory laparoscopic SBT courses. We aimed to obtain a broader understanding of which factors of the simulation training the trainees perceived as eliciting stress.

METHODS

A qualitative study with semistructured individual interviews was undertaken to explore trainees' subjective experiences of stress. Twenty surgical trainees participated while attending courses at a national training center for advanced laparoscopic surgery. Questions explored trainees' stress experiences during the SBT courses with a focus on perceived stressors related to laparoscopic simulation training on two box-trainers and one virtual reality simulator. Interview data were analyzed using inductive, qualitative content analysis methods to identify codes, categories, and themes.

RESULTS

Findings indicated that trainees have a variety of stress experiences during laparoscopic SBT. Three main themes were identified to be related to stress experiences: simulation task requirements, psychomotor skill levels and internal pressures, with subcategories such as task difficulty and time requirements, unrealistic haptic feedback and realism of graphics, inconsistent and poor technical performance, and self-imposed pressures and socio-evaluative threats.

CONCLUSIONS

Insights into surgical trainees' experience of stress during laparoscopic SBT courses showed that some stress experiences were directly related to simulation training, while others were of psychological nature. The technical and efficiency requirements of simulation tasks elicited stress experiences among trainees with less laparoscopic experience and lower levels of psychomotor skills. Self-imposed pressures played an integral part in how trainees mobilized and performed during the courses, suggesting that levels of stress might enhance laparoscopic simulation performance. For course facilitators aiming at optimizing future laparoscopic SBT courses, attending to the realism, providing clarity about learning objectives, and having awareness of individual differences among trainees' technical level when designing the simulation tasks, would be beneficial. Equally important to the laparoscopic SBT is to create a psychological safe learning space in order to reduce the internal pressures of trainees.

A Bi-national needs assessment to identify and prioritise procedures in paediatric surgery for simulation-based training

BACKGROUND

Simulation-based education (SBE) has led to significant changes in healthcare education. However, SBE has often been based on available resources and local expertise rather than a systematic approach to curriculum development. The aim of this study was to perform a bi-national needs assessment to identify and prioritise procedures in a paediatric surgery curriculum that can be supported using SBE.

METHOD

A modified 3-round Delphi technique was used to gather consensus from education leaders and trainees in paediatric surgery in Australia and Aotearoa New Zealand (ANZ). Round 1 identified all procedures a newly specialised paediatric surgeon should be able to perform. In Round 2, each procedure was explored for the need for SBE using the Copenhagen Academy for Medical Education and Simulation (CAMES) Needs-Assessment Formula (NAF). This pre-prioritised list from Round 2 was sent back to participants for final exclusion and ranking in Round 3. Results 88 participants were identified and invited. From 174 procedures identified in Round 1, 71 procedures were grouped and categorised for Round 2 using the CAMES NAF. In Round 3, 17 procedures were eliminated resulting in 54 procedures. Appendicectomy, inguinal herniotomy, and central venous access were the highest rank procedures after prioritisation in Round 3. There was a strong correlation (r = 0.99) between the NAF score and the prioritised ranking, as well as between consultants and trainees (r = 0.92 in Round 2 and 0.98 in Round 3).

CONCLUSION

The prioritised list represents a consensus document decided upon by education leaders and stakeholders in paediatric surgery. These procedures should be an integral part of the SBE of paediatric surgeons in the region.

LEVEL OF EVIDENCE

Level V.

Surgical experience is predictive for bladder tumour resection quality

OBJECTIVES

To assess the resection quality of transurethral bladder tumour resection (TURBT) and the association to surgeon experience depending on the presence of detrusor muscle.

METHODS

A retrospective study on 640 TURBT procedures performed at Zealand University Hospital, Denmark, from 1 January 2015 - 31 December 2016. Data included patient characteristics, procedure type, surgeon category, supervisor presence, surgical report data, pathological data, complications data and recurrence data. Analysis was performed using simple and multiple logistic regression on the association between surgeon experience and the presence of detrusor muscle in resected tissue from TURBT.

RESULTS

Supervised junior residents had significant lower detrusor muscle presence (73%) compared with consultants (83%) (OR = 0.4, 95% CI = 0.21-0.83). Limitations were the retrospective design and the diversity of included TURBT.

CONCLUSIONS

It was found that surgical experience predicts detrusor muscle presence and supervised junior residents performing TURBT on patients resulted in less detrusor muscle than consultants.

Simulation-based mastery learning in gastrointestinal endoscopy training

Simulation-based mastery learning (SBML) is an emerging form of competency-based training that has been proposed as the next standard method for procedural task training, including that in gastrointestinal endoscopy. Current basic gastrointestinal endoscopy training relies on the number of procedures performed, and it has been criticized for its lack of objective standards that result in variable skills among trainees and its association with patient safety risk. Thus, incorporating simulators into a competency-based curriculum seems ideal for gastrointestinal endoscopy training. The curriculum for SBML in gastrointestinal endoscopy is currently being developed and has promising potential to translate into the clinical performance. Unlike the present apprenticeship model of “see one, do one, teach one,” SBML integrates a competency-based curriculum with specific learning objectives alongside simulation-based training. This allows trainees to practice essential skills repeatedly, receive feedback from experts, and gradually develop their abilities to achieve mastery. Moreover, trainees and trainers need to understand the learning targets of the program so that trainees can focus their learning on the necessary skills and trainers can provide structured feedback based on the expected outcomes. In addition to learning targets, an assessment plan is essential to provide trainees with future directions for their improvement and ensure patient safety by issuing a passing standard. Finally, the SBML program should be planned and managed by a specific team and conducted within a developed and tested curriculum. This review discusses the current state of gastrointestinal endoscopy training and the role of SBML in that field.

The impact of simulation-based mastery learning, booster session timing and clinical exposure on confidence in intercostal drain insertion: a survey of internal medicine trainees in Scotland

BACKGROUND

Intercostal chest drain (ICD) insertion is a skill that medical trainees lack confidence in performing. This study explores the impact of a national programme of Simulation-Based Mastery Learning (SBML) on procedural confidence, including the impact of time intervals between booster sessions and interim clinical experience.

METHODS

Internal Medicine Trainees in Scotland were surveyed about confidence and clinical experience with ICD insertion before and immediately after SBML and booster session. Data were matched and analysed using paired sample t-tests. Short interval and long interval groups were compared using Student's unpaired t-test. The impact of interim clinical experience was assessed using Analysis of Variance.

RESULTS

Mean confidence in ICD insertion rose following SBML, fell between initial and booster session, and increased again following booster session (P =  < 0.001). 33 of 74 trainees had successfully inserted an ICD between sessions. Fall in confidence was unaffected by the time interval between training sessions, but was mitigated by interim clinical experience.

CONCLUSIONS

SBML boosts trainee confidence in ICD insertion. However, there is evidence of confidence decay, possibly due to a lack of clinical experience between sessions. More research is needed to explore barriers to transfer of skills from simulated to real-world environments.

Analysis of a Near Peer Tutoring Program to Improve Medical Students' Note Writing Skills

Problem: The ability to document a patient encounter is integral for any physician. Previous studies indicate that medical students' note writing skills are poor due to a lack of formal clinical documentation instruction. Barriers to formally teaching students how to write patient notes include the significant time burden and variability in faculty feedback. Implementing a near-peer teaching program to teach students how to write a patient note can overcome these barriers and fill this curricular gap. Intervention: Fourth year medical students who completed a Teaching in Medicine elective course were trained to provide individual feedback to 1^st and 2^nd year students on note writing as a part of their Introduction to Clinical Medicine course. In order to determine the effect of this two year, near-peer feedback program on note writing skills, we analyzed students' scores on the note writing portion of two Objective Structured Clinical Exams that took place at the end of the 2^nd and 3^rd years of medical school. Context: The near-peer feedback sessions were implemented in the Fall of 2013 during the preclinical years of the medical school curriculum. Data from students who received near-peer feedback (N = 112) were compared to a historical control group who did not receive near-peer feedback on their note writing (N = 110). Objective Structured Clinical Exam scores that were specific to note writing skills, including the history, physical exam, and differential diagnosis subscales were examined. Impact: The near-peer feedback had a positive impact on the quality of patient notes. On the end of the 2^nd year Objective Structured Clinical Exam, the near-peer feedback intervention group outperformed the no feedback group on the history and physical exam subscale scores but not on the differential diagnosis subscale score. One year later, the near-peer feedback intervention group continued to outperform the no feedback group on the physical exam subscale score, but not the history or the differential diagnosis subscale scores. Lessons Learned: Near-peer teaching improves student documentation of the history and physical exam, however only the effects on the physical exam portion persist into the clinical years of training. Writing up a differential diagnosis is a skill that develops through the clerkship experience regardless of exposure to feedback in the preclinical years. Implementing near-peer teaching in the medical school setting is feasible and can provide students with valuable learning experiences without relying on clinical faculty.

On embedding assessments of self-regulated learning into licensure activities in the health professions: a call to action

How well have healthcare professionals and trainees been prepared for the inevitable demands for new learning that will arise in their future? Given the rapidity with which 'core healthcare knowledge' changes, medical educators have a responsibility to audit whether trainees have developed the capacity to effectively self-regulate their learning. Trainees who engage in effective self-regulated learning (SRL) skillfully monitor and control their cognition, motivation, behaviour, and environment to adaptively meet demands for new learning. However, medical curricula rarely assess trainees' capacity to engage in these strategic processes. In this position paper, we argue for a paradigm shift toward assessing SRL more deliberately in undergraduate and postgraduate programs, as well as in associated licensing activities. Specifically, we explore evidence supporting an innovative blend of principles from the science on SRL, and on preparation for future learning (PFL) assessments. We propose recommendations for how program designers, curriculum developers, and assessment leads in undergraduate and postgraduate training programs, and in licensing bodies can work together to develop integrated assessments that measure how and how well trainees engage in SRL. Claims about lifelong learning in health professions education have gone unmatched by responsive curricular changes for far too long. Further neglecting these important competencies represents a disservice to medical trainees and a potential risk to the future patients they will care for.

Treatment of status epilepticus in pediatrics: curriculum learning combined with in-situ simulations

BACKGROUND

Appropriate and timely treatment of status epilepticus (SE) reduces morbidity and mortality. Therefore, skill-based identification and management are critical for emergency physicians.

PURPOSE

To assess whether the ability of training physicians, residents, nurses, and others to respond to SE as a team could be improved by using curriculum learning [Strategies and Tools to Enhance Performance and Patient Safety of Team (TeamSTEPPS) course training] combined with in-situ simulations of emergency department (ED) staff.

APPROACH

A pre-training-post-training design was used on SE skills and teamwork skills. Emergency training, residents, and N1 and N2 nurses completed the SE skill and teamwork assessments (pre-training) through in-situ simulation. Next, the participating physicians and nurses attended the SE course [Strategies and Tools to Enhance Performance and Patient Safety of Team (TeamSTEPPS) course training], followed by conscious skill practice, including in-situ simulation drills every 20 days (eight times total) and deliberate practice in the simulator. The participants completed the SE skill and teamwork assessments (post-training) again in an in-situ simulation. Pre-training-post-training simulated SE skills and teamwork performance were assessed. The simulation training evaluation showed that the training process was reasonable, and the training medical staff had different degrees of benefit in increasing subject interest, improving operational skills, theoretical knowledge, and work self-confidence.

FINDINGS

Sixty doctors and nurses participated in the intervention. When comparing the SE skills of 10 regular training physicians pre-training and post-training, their performance improved from 40% (interquartile range (IQR): 0-1) before training to 100% (IQR: 80.00-100) after training (p < 0.001). The teamwork ability of the 10 teams improved from 2.43 ± 0.09 before training to 3.16 ± 0.08 after training (p < 0.001).

CONCLUSION

SE curriculum learning combined with in-situ simulation training provides the learners with SE identification and management knowledge in children and teamwork skills.

Development and evaluation of a simulation-based mastery learning maintenance of certification course

BACKGROUND/OBJECTIVES

To develop and evaluate a post-acute care simulation-based mastery learning (SBML) continuing medical education (CME)/maintenance of certification (MOC) procedure course.

DESIGN

Pretest-posttest study of the SBML intervention.

SETTING

A 2-day post-acute care procedures course.

PARTICIPANTS

Sixteen practicing clinicians (5 physicians,11 advanced practice providers). Participants engaged in a skills pretest on knee aspiration/injection, gastrostomy tube removal/replacement, tracheostomy tube exchange, and basic suturing using a checklist created for each procedure. Participants received a didactic on each procedure followed by deliberate practice with feedback. Using the same checklists, participants completed a skills posttest and were required to meet a minimum passing standard (MPS) to obtain CME/MOC credit.

MEASUREMENTS

The MPS for each skills checklist was determined by a multidisciplinary panel of 11 experts. Participants completed surveys on procedure self-confidence and a course evaluation.

RESULTS

There was statistically significant improvement between pre- and posttests for all four procedures (p < .001). All participants were able to meet or exceed the MPS for each skill during the 2-day course. Participants' self-confidence regarding each procedure improved significantly (p < .001).

CONCLUSION

An SBML training course granting CME/MOC credit for post-acute care providers significantly improves performance of knee aspiration/injection, gastrostomy tube removal/replacement, tracheostomy tube exchange, and basic suturing.

A New Immersive Virtual Reality Station for Cardiopulmonary Resuscitation Objective Structured Clinical Exam Evaluation

The Objective Structured Clinical Exam (OSCE) is an assessment tool used as a reliable method for clinical competence evaluation of students. This paper presents an investigation focused on the chain of survival, its related exploration, management, and technical skills, and how Virtual Reality (VR) can be used for the creation of immersive environments capable of evaluating students' performance while applying the correct protocols. In particular, the Cardiopulmonary Resuscitation (CPR) procedure is studied as an essential step in the development of the chain of survival. The paper also aims to highlight the limitations of traditional methods using mechanical mannequins and the benefits of the new approaches that involve the students in virtual, immersive, and dynamic environments. Furthermore, an immersive VR station is presented as a new technique for assessing CPR performance through objective data collection and posterior evaluation. A usability test was carried out with 33 clinicians and OSCE evaluators to test the viability of the presented scenario, reproducing conditions of a real examination. Results suggest that the environment is intuitive, quick, and easy to learn and could be used in clinical practice to improve CPR performance and OSCE evaluation.