Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis

Design and validation of a cross-specialty simulation-based training course in basic robotic surgical skills

BACKGROUND

The aim of this study was to design and validate a cross-specialty basic robotic surgical skills training program on the RobotiX Mentor virtual reality simulator.

METHODS

A Delphi panel reached consensus on six modules to include in the training program. Validity evidence was collected according to Messick's framework with three performances in each simulator module by 11 experienced robotic surgeons and 11 residents without robotic surgical experience.

RESULTS

For five of the six modules, a compound metrics-based score could significantly discriminate between the performances of novices and experienced robotic surgeons. Pass/fail levels were established, resulting in very few novices passing in their first attempt.

CONCLUSIONS

This validated course can be used for structured simulation-based basic robotic surgical skills training within a mastery learning framework where the individual trainee can practice each module until they achieve proficiency and can continue training on other modalities and more specific to their specialty.

A Functional Contextualist Approach to Mastery Learning in Vocational Education and Training

Along with technological progress, vocational education and training (VET) is consistently changing. Workforce disruption has serious consequences for workers and international economies, often requiring adults to transition into different occupations or to upskill to maintain employment. We review recent literature covering VET trends, theoretical considerations for the 21st century, and present an approach to workforce training to help workers not only learn necessary skills but also become adaptable to constant change. We suggest a functional contextualist approach to mastery learning achieves this aim. Specifically, we offer suggestions for pedagogy that not only develop skills but also encourage higher order thinking. Within a novice to expert continuum, we suggest deliberate practice, mental simulation, and reflective meaning making as methods to achieve efficiency and transfer-learning outcomes relevant to a changing workforce. This approach recognizes that learning is context bound and should promote broader human capabilities that support both employability and the continuing development of life literacies.

A Mastery Learning Module on Sterile Technique to Prepare Graduating Medical Students for Internship

INTRODUCTION

Sterile technique is a basic technical skill used for a number of bedside procedures. Proper use of sterile technique improves patient safety by reducing infection risk.

METHODS

We applied the principles of mastery learning to develop a simulation-based mastery learning module for sterile technique that was used as part of a 2-week internship preparatory course for fourth-year medical students. Forty-one medical students entering surgical or emergency medicine internships completed the module. Learners demonstrated baseline skills with a pretest, watched a didactic online video, participated in supervised deliberate practice sessions, and then completed a posttest. Physicians evaluated performance using a nine-item mastery checklist validated by a multispecialty panel of board-certified physicians. Learners who did not demonstrate mastery by correctly performing all nine checklist items received formative feedback and repeated the posttest as needed until mastery was achieved.

RESULTS

No learners demonstrated mastery of sterile technique during pretesting. A total of 100% of learners demonstrated mastery of sterile technique during either their first or second attempt of the posttest. The learners reported statistically significantly higher levels of confidence at the end of the module.

DISCUSSION

Our module highlights the skills gap that exists in the transition from undergraduate to graduate medical education and offers a cheap, effective, and easily reproducible curriculum for sterile technique that could be widely adopted for many learner populations.

Aha! Taking on the myth that simulation-derived surprise enhances learning

OBJECTIVES

This paper aims to discuss the recurring education-related issue of the high-fidelity simulation myth. In the current instantiation, educators erroneously believe that trainees benefit from authentic uncertainty and surprise in simulation-based training.

METHODS

We explore the origins of this myth within the experiential learning and social constructivism theories and propose an evidence-based solution of transparent and guided instruction in simulation.

RESULTS

Constructivist theories highlight meaning making as the benefit of inquiry and discovery learning strategies. Inappropriate translation of this epistemology into an element of curriculum design creates unfortunate unintended consequences.

CONCLUSIONS

We propose that the translation of constructivist theories of learning within simulation-based education has resulted in a pervasive myth, which decrees that scenarios must introduce realistic tension or surprises to encourage exploration and insightful problem solving. We argue that this myth is masquerading as experiential learning. In this narrative review, we interpret our experiences and observations of simulation-based education through our expertise in education science and curriculum design. We offer anecdotal evidence along with a review of selected literature to establish the presence of this previously undetected myth.

Delphi Method to Achieve Clinical Consensus for a BPMN Representation of the Central Venous Access Placement for Training Purposes

Proper teaching of the technical skills necessary to perform a medical procedure begins with its breakdown into its constituent steps. Currently available methodologies require substantial resources and their results may be biased. Therefore, it is difficult to generate the necessary breakdown capable of supporting a procedural curriculum. The aim of our work was to breakdown the steps required for ultrasound guided Central Venous Catheter (CVC) placement and represent this procedure graphically using the standard BPMN notation. Methods: We performed the first breakdown based on the activities defined in validated evaluation checklists, which were then graphically represented in BPMN. In order to establish clinical consensus, we used the Delphi method by conducting an online survey in which experts were asked to score the suitability of the proposed activities and eventually propose new activities. Results: Surveys were answered by 13 experts from three medical specialties and eight different institutions in two rounds. The final model included 28 activities proposed in the initial model and four new activities proposed by the experts; seven activities from the initial model were excluded. Conclusions: The proposed methodology proved to be simple and effective, generating a graphic representation to represent activities, decision points, and alternative paths. This approach is complementary to more classical representations for the development of a solid knowledge base that allows the standardization of medical procedures for teaching purposes.

Learning Challenges, Teaching Strategies, and Cognitive Load: Insights From the Experience of Seasoned Endoscopy Teachers

PURPOSE

Learners of medical procedures must develop, refine, and apply schemas for both cognitive and psychomotor constructs, which may strain working memory capacity. Procedures with limitations in visual and tactile information may add risk of cognitive overload. The authors sought to elucidate how experienced procedural teachers perceived learners' challenges and their own teaching strategies in the exemplar setting of gastrointestinal endoscopy.

METHOD

The authors interviewed 22 experienced endoscopy teachers in the United States, Canada, and the Netherlands between May 2016 and March 2019 and performed thematic analysis using template analysis method. Interviews addressed learner challenges and teaching strategies from the teacher participants' perspectives. Cognitive load theory informed data interpretation and analysis.

RESULTS

Participants described taking steps to "diagnose" trainee ability and identify struggling trainees. They described learning challenges related to trainees (performance over mastery goal orientation, low self-efficacy, lack of awareness), tasks (psychomotor challenges, mental model development, tactile understanding), teachers (teacher-trainee relationship, inadequate teaching, teaching variability), and settings (internal/external distractions, systems issues). Participants described employing strategies that could match intrinsic load to learners' levels (teaching along developmental continuum, motor instruction, technical assistance/takeover), minimize extraneous load (optimize environment, systems solutions, emotional support, define expectations), and optimize germane load (promote mastery, teach schemas, stop and focus).

CONCLUSIONS

Participants provided insight into possible challenges while learning complex medical procedures with limitations in sensory channels, as well as teaching strategies that may address these challenges at individual and systems levels. Using cognitive load theory, the authors provide recommendations for procedural teachers.

Educational Interventions to Enhance Situation Awareness: A Systematic Review and Meta-Analysis

STATEMENT

We conducted a systematic review to evaluate the comparative effectiveness of educational interventions on health care professionals' situation awareness (SA). We searched MEDLINE, CINAHL, HW Wilson, ERIC, Scopus, EMBASE, PsycINFO, psycARTICLES, Psychology and Behavioural Science Collection and the Cochrane library. Articles that reported a targeted SA intervention or a broader intervention incorporating SA, and an objective outcome measure of SA were included. Thirty-nine articles were eligible for inclusion, of these 4 reported targeted SA interventions. Simulation-based education (SBE) was the most prevalent educational modality (31 articles). Meta-analysis of trial designs (19 articles) yielded a pooled moderate effect size of 0.61 (95% confidence interval = 0.17 to 1.06, P = 0.007, I = 42%) in favor of SBE as compared with other modalities and a nonsignificant moderate effect in favor of additional nontechnical skills training (effect size = 0.54, 95% confidence interval = 0.18 to 1.26, P = 0.14, I = 63%). Though constrained by the number of articles eligible for inclusion, our results suggest that in comparison with other modalities, SBE yields better SA outcomes.

Consensus-Based Expert Development of Critical Items for Direct Observation of Point-of-Care Ultrasound Skills

BACKGROUND

Point-of-care ultrasound (POCUS) is increasingly used in a number of medical specialties. To support competency-based POCUS education, workplace-based assessments are essential.

OBJECTIVE

We developed a consensus-based assessment tool for POCUS skills and determined which items are critical for competence. We then performed standards setting to set cut scores for the tool.

METHODS

Using a modified Delphi technique, 25 experts voted on 32 items over 3 rounds between August and December 2016. Consensus was defined as agreement by at least 80% of the experts. Twelve experts then performed 3 rounds of a standards setting procedure in March 2017 to establish cut scores.

RESULTS

Experts reached consensus for 31 items to include in the tool. Experts reached consensus that 16 of those items were critically important. A final cut score for the tool was established at 65.2% (SD 17.0%). Cut scores for critical items are significantly higher than those for noncritical items (76.5% ± SD 12.4% versus 53.1% ± SD 12.2%, P < .0001).

CONCLUSIONS

We reached consensus on a 31-item workplace-based assessment tool for identifying competence in POCUS. Of those items, 16 were considered critically important. Their importance is further supported by higher cut scores compared with noncritical items.

DebriefLive: A Pilot Study of a Virtual Faculty Development Tool for Debriefing

INTRODUCTION

The quality of healthcare simulation learning relies heavily on effective debriefers. Traditional methods of faculty development in debriefing lack a structured approach to achieve expertise via graduated and reflective practice.

METHODS

The Simulation Learning, Education and Research Network (SimLEARN) developed DebriefLive, a virtual teaching environment, bringing together faculty and participant debriefers from across the Veterans Health Administration. Recorded simulation sessions were viewed followed by the opportunity for participant debriefers to debrief virtual learners. Participant debriefers were then provided structured and objective debriefings of the debriefings with the added opportunity for immediate practice. Program evaluation data for the pilot sessions were collected via electronic survey including a mix of Likert scale questions as well as short answer responses.

RESULTS

On a 7-point Likert scale, participant debriefers (n = 15) rated the content as effective (mean = 6.67, SD = 0.47) and appropriate to their level (mean = 6.47, SD = 0.47). The technology of video-based scenarios (mean = 6.6, SD = 0.61), followed by avatar-based debriefing sessions (mean = 6.6, SD = 0.8), was felt to be accurate and appropriate. All participants would agree or strongly agree with recommending this training program to colleagues.

CONCLUSIONS

Simulation instructors and fellows across the spectrum of the Veterans Health Administration found the innovative computer-based faculty development program DebriefLive acceptable as well as effective in increasing self-efficacy in debriefing. DebriefLive is an innovative and potentially disruptive tool, combining best practices in simulation theory and virtual technologies, for the training and assessment of debriefers.

Simulation-based mastery learning compared to standard education for discussing diagnostic uncertainty with patients in the emergency department: a randomized controlled trial

BACKGROUND

Diagnostic uncertainty occurs frequently in emergency medical care, with more than one-third of patients leaving the emergency department (ED) without a clear diagnosis. Despite this frequency, ED providers are not adequately trained on how to discuss diagnostic uncertainty with these patients, who often leave the ED confused and concerned. To address this training need, we developed the Uncertainty Communication Education Module (UCEM) to teach physicians how to discuss diagnostic uncertainty. The purpose of the study is to evaluate the effectiveness of the UCEM in improving physician communications.

METHODS

The trial is a multicenter, two-arm randomized controlled trial designed to teach communication skills using simulation-based mastery learning (SBML). Resident emergency physicians from two training programs will be randomly assigned to immediate or delayed receipt of the two-part UCEM intervention after completing a baseline standardized patient encounter. The two UCEM components are: 1) a web-based interactive module, and 2) a smart-phone-based game. Both formats teach and reinforce communication skills for patient cases involving diagnostic uncertainty. Following baseline testing, participants in the immediate intervention arm will complete a remote deliberate practice session via a video platform and subsequently return for a second study visit to assess if they have achieved mastery. Participants in the delayed intervention arm will receive access to UCEM and remote deliberate practice after the second study visit. The primary outcome of interest is the proportion of residents in the immediate intervention arm who achieve mastery at the second study visit.

DISCUSSION

Patients' understanding of the care they received has implications for care quality, safety, and patient satisfaction, especially when they are discharged without a definitive diagnosis. Developing a patient-centered diagnostic uncertainty communication strategy will improve safety of acute care discharges. Although use of SBML is a resource intensive educational approach, this trial has been deliberately designed to have a low-resource, scalable intervention that would allow for widespread dissemination and uptake.

TRIAL REGISTRATION

The trial was registered at clinicaltrials.gov (NCT04021771). Registration date: July 16, 2019.

Deaf ACCESS: Adapting Consent Through Community Engagement and State-of-the-Art Simulation

One of the most understudied health disparity populations in the United States is the Deaf community, a sociolinguistic minority group of more than 500,000 individuals who communicate via American Sign Language. Research on Deaf health disparities is lacking due to inaccessible recruitment, sampling, and data collection procedures, as well as the fundamental disconnect between medical and cultural views of Deaf people. A potential starting place for addressing inaccessible research methods and mistrust of the biomedical research community is the careful reconsideration of the traditional informed consent process, often a Deaf individual's first point of contact with the research world. Yet, most Deaf individuals experience obstacles to engaging in informed consent due to differences in language and development compared to hearing individuals. In response to these issues, our team led a three-phase, formative, community-engaged approach to adapt the informed consent process and train research staff in the updated method so that all required components are properly communicated and understood. The goals of our work were to promote Deaf engagement in research about the Deaf community, increase the number of Deaf individuals who participate in general population biomedical research, and generalize our findings to improve research accessibility for the general population.

Patient, Caregiver, and Clinician Perceptions of Ventricular Assist Device Self-care Education Inform the Development of a Simulation-based Mastery Learning Curriculum

BACKGROUND

Patients who undergo ventricular assist device (VAD) implantation and their caregivers must rapidly learn a significant amount of self-care skills and knowledge.

OBJECTIVE

The aim of this study was to explore patient, caregiver, VAD coordinator, and physician perspectives and perceptions of existing VAD self-care training to inform development of a simulation-based mastery learning (SBML) curriculum to teach patients and caregivers VAD self-care skills and knowledge.

METHODS

We conducted semistructured, in-person interviews with patients with a VAD, their caregivers, VAD coordinators, and physicians (cardiac surgeons, an infectious disease physician, and advanced heart failure cardiologists). We used a 2-cycle team-based iterative inductive approach to coding and analysis.

RESULTS

We interviewed 16 patients, 12 caregivers, 7 VAD coordinators, and 11 physicians. Seven major themes were derived from the interviews including (1) identification of critical curricular content, (2) need for standardization and assessment, (3) training modalities, (4) benefits of repetition, (5) piercing it all together, (6) need for refresher training, and (7) provision of training before implant.

CONCLUSIONS

Findings from this study suggest that SBML is a natural fit for the high-risk tasks needed to save VAD self-care. The 7 unique training-related themes derived from the qualitative data informed the design and development of a VAD SBML self-care curriculum.

Chest Tube Insertion Among Surgical and Nonsurgical Trainees: How Skilled Are Our Residents?

BACKGROUND

Competency-based medical education has renewed focus on the attainment and evaluation of resident skill. Proper evaluation is crucial to inform educational interventions and identify residents in need of increased training and supervision. Currently, there is a paucity of studies rigorously evaluating resident chest tube insertion skill.

MATERIALS AND METHODS

Residents of all training levels before their intensive care unit rotation or currently rotating through the intensive care unit were invited to participate. Trainees inserted a thoracostomy tube on a high-fidelity simulator. Their performances were recorded and scored by blinded raters using the validated TUBE-iCOMPT rubric. Surgical and nonsurgical residents were compared.

RESULTS

Forty-nine residents participated; 30 from nonsurgical and 19 from surgical training programs. Overall, trainees were most deficient in the "preprocedural checks" and "patient positioning and local anesthetic" domains. Surgical trainees demonstrated higher chest tube insertion skill than their nonsurgical peers (median total score 88 [interquartile range, 74-90] versus 75 [interquartile range, 66-85], respectively, P = 0.01), particularly in the "patient positioning" and "blunt dissection" domains (P = 0.01 and P = 0.03, respectively). These differences were no longer significant when controlled for experience and Advanced Trauma Life Support certification.

CONCLUSIONS

Overall, surgical residents were more skilled than nonsurgical residents in tube thoracostomy placement. Relative skill deficits within the domains of chest tube insertion have also been identified among residents of different specialties. These areas can be targeted with educational interventions to improve resident performance, and ultimately, patient safety.

Mastery learning in a bachelor's of nursing program: the Roseman University of Health Sciences experience

BACKGROUND

Roseman University of Health Sciences (RUHS) developed and delivers a mastery learning curriculum designed for students to acquire the knowledge and skills to become competent nurses. Despite a trend in nursing education to adopt competency-based education (CBE) models, there is little in the nursing literature about programs based on a mastery model. The aim of this study is to describe an undergraduate nursing program built on a mastery learning model and to report on program outcome measures.

METHODS

The 18-month BSN nursing program is divided into blocks, varying in length and focusing on a single subject. Students must demonstrate mastery, defined as ≥90% on an assessment, to pass a block. Recognizing the critical nature of health care, educators seek methods to assure that practitioners become competent to perform the services they provide.Program outcomes reported include comparisons to national standards and RUHS student exit survey data.

RESULTS

From 2013 to 2017 the RUHS College of Nursing students' pass rates ranged from 82 to 97% for the National Council Licensure Examination exam compared to national pass rates between 81.8-84.5% during the same time frame. The program completion rate ranged from 86 to 100% and employment rates exceeded accreditation standards. Students reported overall satisfaction with their education as 4.38 and with the block system as 4.74 (5 point Likert scale).

CONCLUSIONS

Roseman University's mastery learning model appears successful as measured by high levels of student satisfaction, outcomes on exams, and degree completion when compared to national averages. The results suggest that other nursing and health profession's programs can develop a successful mastery based learning model.

Simulation-Based Mastery Learning Improves Patient and Caregiver Ventricular Assist Device Self-Care Skills: A Randomized Pilot Trial

BACKGROUND

No recognized standards exist for teaching patients and their caregivers ventricular assist device (VAD) self-care skills. We compared the effectiveness of a VAD simulation-based mastery learning (SBML) self-care training curriculum with usual VAD self-care training.

METHODS AND RESULTS

VAD patients and their caregivers were randomized to SBML or usual training during their implant hospitalization. The SBML group completed a pretest on 3 VAD self-care skills (controller, power source, and dressing change), then viewed videos and participated in deliberate practice on a simulator. SBML participants took a posttest and were required to meet or exceed a minimum passing standard for each of the skills. The usual training group completed the existing institutional VAD self-care teaching protocol. Before hospital discharge, the SBML and usual training groups took the same 3 VAD self-care skills tests. We compared demographic and clinical information, self-confidence, total participant training time, and skills performance between groups. Forty participants completed the study in each group. There were no differences in demographic and clinical information, self-confidence, or training time between groups. More participants in the SBML group met the minimum passing standard compared with the usual training group for controller (37/40 [93%] versus 25/40 [63%]; P=0.001), power source (36/40 [90%] versus 9/40 [23%]; P<0.001), and dressing change skills (19/20 [95%] versus 0/20; P<0.001).

CONCLUSIONS

SBML provided superior VAD self-care skills learning outcomes compared with usual training. This study has important implications for patients due to the morbidity and mortality associated with improper VAD self-care.

CLINICAL TRIAL REGISTRATION

URL: http://www.clinicaltrials.gov. Unique identifier: NCT03073005.

A meta-analysis for comparing effective teaching in clinical education

Aim: Many factors affect learning outcomes, however studies comparing the effectiveness of different clinical teaching methods are limited. We utilize the list of influences on educational achievement compiled by John Hattie to inform a meta-analysis of learning effect sizes (ESs) associated with teaching-learning factors (TLFs) in clinical education. Methods: A literature search was conducted in PubMed to identify articles examining clinically relevant TLFs. Selection criteria were applied to identify learner-focused studies, with subsequent categorization by study design (pretest-posttest or controlled group). The Cohen's ES (d) for each TLF was extracted and a pooled ES determined. Results: From 3454 studies, 132 suitable articles enabled analysis of 16 TLFs' ESs. In general, ESs derived from pretest-posttest data were larger than those from controlled group designs, probably due to learner maturation effect. The TLFs of mastery learning, small group learning and goal settings possessed the largest ESs (d ≥ 0.8), while worked examples, play programs, questioning, concept mapping, meta-cognitive strategies, visual-perception programs and teaching strategies demonstrated ESs between 0.4 and 0.8. Conclusions: This is the first study to provide a rigorous and comprehensive overview of the effectiveness of TLFs in clinical education. We discuss the practical traits shared by effective TLFs which may assist teaching design.

Assessment of the Interrater Reliability of the Congress of Neurological Surgeons Microanastomosis Assessment Scale

BACKGROUND

The potential for simulation-based learning in neurosurgical training has led the Congress of Neurosurgical Surgeons to develop a series of simulation modules. The Northwestern Objective Microanastomosis Assessment Tool (NOMAT) was created as the corresponding assessment tool for the Congress of Neurosurgical Surgeons Microanastomosis Module. The face and construct validity of the NOMAT have been previously established.

OBJECTIVE

To further validate the NOMAT by determining its interrater reliability (IRR) between raters of varying levels of microsurgical expertise.

METHODS

The NOMAT was used to assess residents' performance in a microanastomosis simulation module in 2 settings: Northwestern University and the Society of Neurological Surgeons 2014 Boot Camp at the University of Indiana. At Northwestern University, participants were scored by 2 experienced microsurgeons. At the University of Indiana, participants were scored by 2 postdoctoral fellows and an experienced microsurgeon. The IRR of NOMAT was estimated by computing the intraclass correlation coefficient using SPSS v22.0 (IBM, Armonk, New York).

RESULTS

A total of 75 residents were assessed. At Northwestern University, 21 residents each performed microanastomosis on 2 model vessels of different sizes, one 3 mm and one 1 mm. At the University of Indiana, 54 residents performed a single microanastomosis procedure on 3-mm vessels. The intraclass correlation coefficient of the total NOMAT scores was 0.88 at Northwestern University and 0.78 at the University of Indiana.

CONCLUSION

This study indicates high IRR for the NOMAT. These results suggest that the use of raters with varying levels of expertise does not compromise the precision or validity of the scale. This allows for a wider adoption of the scale and, hence, a greater potential educational impact.

Apprentissage par simulation en pédiatrie : l’exemple de l’arrêt cardiorespiratoire de l’enfant

La simulation en santé s’est diffusée très rapidement en pédiatrie à partir des années 2000 aux États-Unis et au Canada, puis à partir des années 2010 en France. L’arrêt cardiorespiratoire (ACR) de l’enfant représente le meilleur exemple des bénéfices qui peuvent être apportés par la simulation. La simulation peut reproduire à l’infini cette situation exceptionnelle, offre un apprentissage sans risque pour le patient ni pour l’apprenant, permet de travailler ses aspects cognitifs, techniques et humains. La simulation est effectivement associée à un gain en connaissance et en compétence chez les apprenants et participe à l’amélioration du pronostic des patients. Afin de maximiser l’efficacité pédagogique de l’enseignement de la prise en charge de l’ACR de l’enfant, les responsables pédagogiques devraient sans cesse évaluer leurs programmes, privilégier des curriculums qui associent la simulation avec d’autres modalités d’apprentissage et avoir pour objectif une pédagogie de la maîtrise. Les mannequins haute fidélité, s’ils sont appréciés par les apprenants, restent pédagogiquement équivalents aux mannequins basse fidélité pour l’apprentissage de la prise en charge de l’ACR de l’enfant.

Debriefing for the Transfer of Learning: The Importance of Context

The advent of simulation-based education has caused a renewed interest in feedback and debriefing. However, little attention has been given to the issue of transfer of learning from the simulation environment to real-life and novel situations. In this article, the authors discuss the importance of context in learning, based on the frameworks of analogical transfer and situated cognition, and the limitations that context imposes on transfer. They suggest debriefing strategies to improve transfer of learning: positioning the lived situation within its family of situations and implementing the metacognitive strategies of contextualizing, decontextualizing, and recontextualizing. In contextualization, the learners' actions, cognitive processes, and frames of reference are discussed within the context of the lived experience, and their mental representation of the situation and context is explored. In decontextualization, the underlying abstract principles are extracted without reference to the situation, and in recontextualization, those principles are adapted and applied to new situations and to the real-life counterpart. This requires that the surface and deep features that characterize the lived situation be previously compared and contrasted with those of the same situation with hypothetical scenarios ("what if"), of new situations within the same family of situations, of the prototype situation, and of real-life situations. These strategies are integrated into a cyclical contextualization, decontextualization, and recontextualization model to enhance debriefing.

Simulation-based mastery learning in medical students: Skill retention at 1-year follow up

Context: Deficits in basic skill performance and long-term skill retention among medical students and novice doctors are a persistent problem. This controlled study tested whether the addition of a mastery learning component to simulation-based teaching is associated with long-term retention and performance of peripheral venous catheter insertion. Methods: Fourth-year medical students were assigned to receive either the control (simulation without mastery learning, n = 131) or the intervention (simulation + mastery learning, n = 133) instruction in peripheral venous catheter insertion. Performance was assessed at one year post-instruction. Eighty-four students from the control group and 71 from the intervention group participated in the assessment. Results: Students who received the mastery learning instruction achieved higher overall test scores than did controls (median mastery learning score: 20.0, IQR 2.0; median control score 19.0, IQR 3.0; Mann-Whitney U test, p < 0.001, effect size d = 0.82). Pass rates also differed significantly between the groups, with 74.5% (n = 53) of the intervention group passing compared with 33% (n = 28) of the control group (p < 0.001). Conclusions: Mastery learning is an effective means of teaching practical skills to medical students, and is associated with higher scores at a 1-year follow up.

Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring

Background

Simulation is increasingly being used in postgraduate medical education as an opportunity for competency assessment. However, there is limited direct evidence that supports performance in the simulation lab as a surrogate of workplace-based clinical performance for non-procedural tasks such as resuscitation in the emergency department (ED). We sought to directly compare entrustment scoring of resident performance in the simulation environment to clinical performance in the ED.

Methods

The resuscitation assessment tool (RAT) was derived from the previously implemented and studied Queen's simulation assessment tool (QSAT) via a modified expert review process. The RAT uses an anchored global assessment scale to generate an entrustment score and narrative comments. Emergency medicine (EM) residents were assessed using the RAT on cases in simulation-based examinations and in the ED during resuscitation cases from July 2016 to June 2017. Resident mean entrustment scores were compared using Pearson's correlation coefficient to determine the relationship between entrustment in simulation cases and in the ED. Inductive thematic analysis of written commentary was conducted to compare workplace-based with simulation-based feedback.

Results

There was a moderate, positive correlation found between mean entrustment scores in the simulated and workplace-based settings, which was statistically significant (r = 0.630, n = 17, p < 0.01). Further, qualitative analysis demonstrated overall management and leadership themes were more common narratives in the workplace, while more specific task-based feedback predominated in the simulation-based assessment. Both workplace-based and simulation-based narratives frequently commented on communication skills.

Conclusions

In this single-center study with a limited sample size, assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace. This study suggests that resuscitation performance in simulation settings may be an indicator of competence in the clinical setting. However, multiple factors contribute to this complicated and imperfect relationship. It is imperative to consider narrative comments in supporting the rationale for numerical entrustment scores in both settings and to include both simulation and workplace-based assessment in high-stakes decisions of progression.

Comparative evaluation of video-based on-line course versus serious game for training medical students in cardiopulmonary resuscitation: A randomised trial

Purpose

To estimate the effect size of a serious game for cardiopulmonary resuscitation (CPR) training in comparison with a video-based on-line course in terms of learning outcomes among medical students before simulation-based CPR using a manikin.

Methods

Participants were 45 first-year medical students randomly assigned to CPR self-training using either a video-based Apple Keynote presentation (n = 22) or a serious game developed in a 3D learning environment (n = 23) for up to 20 min. Each participant was evaluated on a written, multiple-choice test (theoretical test) and then on a scenario of cardiac arrest (practical test) before and after exposure to the self-learning methods. The primary endpoint was change in theoretical and practical baseline scores during simulated CPR. This study was conducted in 2017.

Results

Both groups improved scores after exposure. The video group had superior performance in both the theoretical test (7.56±0.21 vs 6.51±0.21 for the game group; p = 0.001) and the practical test (9.67±0.21 vs 8.40±0.21 for the game group; p < 0.001). However, students showed a preference for using games, as suggested by the longer time they remained interested in the method (18.57±0.66 min for the game group vs 7.41±0.43 for the video group; p < 0.001).

Conclusions

The self-training modality using a serious game, after a short period of exposure, resulted in inferior students’ performance in both theoretical and practical CPR tests compared to the video-based self-training modality. However, students showed a clear preference for using games rather than videos as a form of self-training.

Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance

BACKGROUND

Simulation-based medical education (SBME) is a cornerstone for procedural skill training in residency education. Multiple studies have concluded that SBME is highly effective, superior to traditional clinical education, and translates to improved patient outcomes. Additionally it is widely accepted that mastery learning, which comprises deliberate practice, is essential for expert level performance for routine skills; however, given that highly structured practice is more time and resource-intensive, it is important to assess its value for the acquisition of rarely performed technical skills. The bougie-assisted cricothyroidotomy (BAC), a rarely performed, lifesaving procedure, is an ideal skill for evaluating the utility of highly structured practice as it is relevant across many acute care specialties and rare - making it unlikely for learners to have had significant previous training or clinical experience. The purpose of this study is to compare a modified mastery learning approach with deliberate practice versus self-guided practice on technical skill performance using a bougie-assisted cricothyroidotomy model.

METHODS

A multi-centre, randomized study will be conducted at four Canadian and one American residency programs with 160 residents assigned to either mastery learning and deliberate practice (ML + DP), or self-guided practice for BAC. Skill performance, using a global rating scale, will be assessed before, immediately after practice, and 6 months later. The two groups will be compared to assess whether the type of practice impacts performance and skill retention.

DISCUSSION

Mastery learning coupled with deliberate practice provides systematic and focused feedback during skill acquisition. However, it is resource-intensive and its efficacy is not fully defined. This multi-centre study will provide generalizable data about the utility of highly structured practice for technical skill acquisition of a rare, lifesaving procedure within postgraduate medical education. Study findings will guide educators in the selection of an optimal training strategy, addressing both short and long term performance.

A Mastery Learning Capstone Course to Teach and Assess Components of Three Entrustable Professional Activities to Graduating Medical Students

PROBLEM

Thirteen measurable Entrustable Professional Activities (EPAs) have been proposed by the Association of American Medical Colleges as a means to operationalize medical school graduates' patient care qualifications. Mastery learning is an effective method for boosting clinical skills, but its applicability to the EPAs remains to be studied. The authors designed this study to evaluate a mastery learning intervention to teach and assess components of 3 of the 13 EPAs in a 4th-year capstone course.

INTERVENTION

The course featured mastery learning principles and addressed three EPA-based skills: (a) obtain informed consent, (b) develop a differential diagnosis and write admission orders, and (c) write discharge prescriptions. All students underwent a baseline skills assessment, received feedback, engaged in deliberate practice with actionable feedback, and completed a similar skills-based posttest assessment. Students continued with practice and testing until the minimum passing standards (MPSs) were reached for each posttest.

CONTEXT

All medical students at a single medical school (Northwestern University, Feinberg School of Medicine) who matriculated in 2012 and graduated with the class of 2016 participated in a required transition to residency course immediately prior to graduation.

OUTCOME

There were 134 students eligible to participate, and 130 (97.0%) completed all curricular requirements and assessments. All 130 medical students who completed the course met or exceeded the MPS for each of the three EPA-based clinical skills. Reliability coefficients for outcome data were uniformly high. Measures for each of the three clinical skills showed statistically significant improvement.

LESSONS LEARNED

The capstone course was an effective approach to teach and assess components of three EPA-based clinical skills to mastery learning standards in a 4th-year capstone course. We learned that this approach for implementation is feasible and results in significant improvement in components of EPA skill performance. Next steps will include developing assessments incorporating the mastery model into components of additional EPAs, identifying the best location within the curriculum to insert this content, and expanding the number of assessments as part of a larger assessment system.

A Scoping Review of Medical Education Research in Neurosurgery

BACKGROUND

Little is known about the state of medical education (ME) research in neurosurgery. As ME started to develop in neurosurgery in recent years, it is important to understand the current status and develop the theory for advancement in neurosurgery. The aim of this study was to undertake a scoping review of neurosurgery literature on ME research.

METHODS

MEDLINE, SCOPUS, and PubMed databases were searched. Inclusion criteria were full-text articles in English published from January 2006 to December 2017. Research aspects included country of publication, annual number of publications, journal types, type of participants, frequently researched topics, and research design. Search terms included neurosurgery, medical education, teaching, training, learning, and curriculum.

RESULTS

A total of 9863 references were found across 3 databases. After duplicate removal and further screening, 533 references remained for coding analysis. ME research activity in neurosurgery is increasing and commonly observed in Western countries. Identified articles were mostly quantitative, with curriculum, assessment (especially simulation), and teaching and learning being the most dominant research themes.

CONCLUSIONS

This study highlights the need for enhanced quantity and quality of ME research in neurosurgery. It identifies areas of highest priority and aspects to be improved and provides us with a rationale for future development in ME in neurosurgery. These findings reveal future education research direction and programmatic research areas, while also establishing a benchmark to assess changes in educational scholarship over time.

Simulation in endoscopy: Practical educational strategies to improve learning

In gastrointestinal endoscopy, simulation-based training can help endoscopists acquire new skills and accelerate the learning curve. Simulation creates an ideal environment for trainees, where they can practice specific skills, perform cases at their own pace, and make mistakes with no risk to patients. Educators also benefit from the use of simulators, as they can structure training according to learner needs and focus solely on the trainee. Not all simulation-based training, however, is effective. To maximize benefits from this instructional modality, educators must be conscious of learners' needs, the potential benefits of training, and associated costs. Simulation should be integrated into training in a manner that is grounded in educational theory and empirical data. In this review, we focus on four best practices in simulation-based education: deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational strategies. For each topic, we provide definitions, supporting evidence, and practical tips for implementation.

Simulators in the training of surgeons: is it worth the investment in money and time? 2018 Jules Gonin lecture of the Retina Research Foundation

This paper describes transfer of skills obtained from training with the EyeSI virtual reality simulator of ophthalmic surgery to real-life surgical performance. Skills in real-life phacoemulsification surgery were assessed by systematic blinded evaluation of surgical videos based on the OSACCS system. Nineteen Danish cataract surgeons with varying clinical experience levels had their cataract surgery skills evaluated before and after completing a standardized mastery learning program on the EyeSI. It was found that transfer of skills could be demonstrated only for surgeons with a real-life experience of less than 75 completed, independent cases. We could not demonstrate transfer of skills from the EyeSI cataract module to the EyeSI vitreoretinal module, so each subspecialty seems to require specific training. Finally, the discriminative power of EyeSI simulation between emerging surgeons and experts was found to reside only in the first training sessions. The EyeSI simulator in its current state of development, and our implementation of it, seems to require further development before it can be used as a tool to select residents for surgical training and to re-certify more senior surgeons.

Ensuring basic competency in chest tube insertion using a simulated scenario: an international validation study

Introduction

Chest tube insertion can be associated with serious complications. A structured training programme is needed to minimise complications and enhance patient safety. Novices should pass a reliable test with solid evidence of validity before performing the procedure supervised on patients. The aim of this study was to establish a credible pass/fail standard.

Methods

We used an established assessment tool the Chest Tube Insertion Competency Test (TUBE-iCOMPT). Validity evidence was explored according to Messick’s five sources of validity. Two methods were used to establish a credible pass/fail standard. Contrasting groups’ method: 34 doctors (23 novices and 11 experienced surgeons) performed the procedure twice and all procedures were video recorded, edited, blinded and rated by two independent, international raters. Modified Angoff method: seven thoracic surgeons individually determined the scores that defined the pass/fail criteria. The data was gathered in Copenhagen, Denmark and Riyadh, Saudi Arabia.

Results

Internal consistency reliability was calculated as Cronbach’s alpha to 0.94. The generalisability coefficient with two raters and two procedures was 0.91. Mean scores were 50.7 (SD±13.2) and 74.7 (SD±4.8) for novices and experienced surgeons, respectively (p<0.001). The pass/fail score of 62 points resulted in zero false negatives and only three false positives.

Discussion

We have gathered valuable additional validity evidence for the assessment tool TUBE-iCOMPT including establishment of a credible pass/fail score. The TUBE-iCOMPT can now be integrated in mastery learning programmes to ensure competency before independent practice.

Simulator training improves ultrasound scanning performance on patients: a randomized controlled trial

BACKGROUND

Simulation-based mastery training may improve clinical performance. The aim of this study was to determine the effect of simulation-based mastery training on clinical performance in abdominal diagnostic ultrasound for radiology residents.

METHOD

This study was a multicenter randomized controlled trial registered at clinicaltrials.gov (identifier: NCT02921867) and reported using the Consolidated Standards of Reporting Trials (CONSORT) statement. Twenty radiology residents from 10 different hospitals were included in the study. Participants were randomized into two groups: (1) simulator-based training until passing a validated test scored by a blinded reviewer or (2) no intervention prior to standard clinical ultrasound training on patients. All scans performed during the first 6 weeks of clinical ultrasound training were scored. The primary outcome was performance scores assessed using Objective Structured Assessment of Ultrasound Skills (OSAUS). An exponential learning curve was fitted for the OSAUS score for the two groups using non-linear regression with random variation. Confidence intervals were calculated based on the variation between individual learning curves.

RESULTS

After randomization, eleven residents completed the simulation intervention and nine received standard clinical training. The simulation group participants attended two to seven training sessions using between 6 and 17 h of simulation-based training. The performance score for the simulation group was significantly higher for the first 29 scans compared to that for the non-simulation group, such that scores reached approximately the same level after 49 and 77 scans, respectively.

CONCLUSION

We showed improved performance in diagnostic ultrasound scanning on patients after simulation-based mastery learning for radiology residents.

TRIAL REGISTRATION

NCT02921867 KEY POINTS: • Improvement in scanning performance on patients is seen after simulation-based mastery learning in diagnostic abdominal ultrasound. • Simulation-based mastery learning can prevent patients from bearing the burden of the initial steep part of trainees' learning curve.

Power of Judgment: The Significance of Kant's Philosophy for the Medical System Today

The ways of thinking in the manufacturing sciences are increasingly determining the rationality within medicine as a practical or action-based science. This "technological paradigm" infiltrates the field of medicine with the promise of increasing efficiency while simultaneously improving quality at various points in the system. Simple linear causal relationships generally need to be taken into account when manufacturing products. Even complex manufacturing processes can be broken down into the smallest units and, therefore, also be automated. The situation in complex systems such as the human body, however, is completely different. In order for doctors to be able to carry out their actions within this complex system, medicine as a science provides the physician with rules on the means that should be used to decide which remedy should be used, when and how. This judgment of which remedy should be used, when and how, what is known as the indication, is a central medical moment. This requires a power of judgment sharpened by experience. The indication, in turn, essentially determines the course of a disease and thus the quality of the treatment or the quality of result so often referred to these days.

The Effect of Judge Selection on Standard Setting Using the Mastery Angoff Method during Development of a Ventricular Assist Device Self-Care Curriculum

Background

Patients and caregivers need to perform ventricular assist device (VAD) self-care safely to help prevent complications (e.g., infection). We developed a VAD self-care simulation-based mastery learning (SBML) curriculum. We determined optimal minimum passing scores (MPSs) and evaluated effects of judge selection.

Methods

A multidisciplinary team created a VAD self-care SBML curriculum including simulated skills and knowledge examinations. Patients, caregivers, VAD coordinators, and physicians were expert judges who determined MPSs using the Mastery Angoff method.

Results

MPSs for the skills and knowledge examinations were high (range = 94-99% and 97% correct), respectively. Judges closely agreed on MPSs.

Conclusions

Stakeholders set stringent MPSs for high-stakes VAD self-care.

Simulation training streamlines the real-life performance in endovascular repair of ruptured abdominal aortic aneurysms

OBJECTIVE

Difficulties in distributing endovascular experience among all operating room (OR) personnel prevented full-scale use of endovascular aneurysm repair (EVAR) in emergencies. To streamline the procedure of EVAR for ruptured aneurysm (rEVAR) and to provide this method even to unstable patients, we initiated regular simulation training sessions.

METHODS

This is an observational study of 29 simulation sessions performed between January 2015 and December 2017. We analyzed the development of time from OR door to aortic balloon occlusion during simulations and OR door to needle times in real-life rEVARs as well as the outcome of the 185 ruptured abdominal aortic aneurysm (rAAA) patients who arrived at the university hospital between January 2013 and December 2017. A questionnaire was sent for simulation attendants before and after the simulation session.

RESULTS

In the first simulations, the door to occlusion time was 20 to 35 minutes. After adding a hemodynamic collapse to the simulation protocol, the time decreased to 10 to 13 minutes in the 10 recent simulations, including a 5-minute cardiopulmonary resuscitation (P = .01). The electronic questionnaire performed for attendees before and after the simulation session showed significant improvement in both confidence and knowledge of the OR staff regarding rEVAR procedure. In the real-life rEVARs, 75 of the 185 patients with rAAAs underwent EVAR. Among rEVAR patients, the median OR door to needle time was 65 minutes before and 16 minutes after the onset of simulations (P = .000). The overall 30-day mortality among all rAAA patients was 44.8% and 30.6% accordingly (P = .046). When patients who were turned down from the emergency surgery were excluded, the 30-day operative mortality was 39.2% and 25.1% during the periods, respectively (P = .051). The 30-day mortality was 16.2% after rEVAR and 40.6% after open surgery (P = .001).

CONCLUSIONS

Simulation training for rEVAR significantly improves the treatment process in real-life patients and may enhance the outcome of rAAA patients.

Developing and Gathering Validity Evidence for a Simulation-Based Test of Competencies in Lung Ultrasound

BACKGROUND

Clinical lung ultrasound (LUS) is a fast bedside diagnostic tool which can assist clinicians in decisions regarding the treatment and monitoring of patients with respiratory symptoms. LUS training and education differ widely, and is often done in a clinical setting, with potential risks for patients if decisions are made based on the wrong interpretations. No clear guidelines or recommendations for objective and standardized assessment of LUS skills exist, and those that do are often based on a fixed time-frame or an arbitrary number of examinations performed; this does not ensure adequate competencies.

OBJECTIVES

The study aimed to develop and gather validity evidence for a practical, simulation-based test in LUS.

METHODS

Nine cases were developed in collaboration with 3D Systems Healthcare, Littleton, CO, USA, representing the most common diagnosis and sonographic findings in patients with respiratory symptoms. Thirty-six participants with different levels of competence in LUS, completed the test. The participants were divided into groups, i.e., novices, intermediates, and experienced, according to their experience with LUS, the number of examinations they had performed, and any research they had conducted. Their answers were used for item analyses.

RESULTS

The intraclass correlation coefficient, Cronbachs' α, was 0.69 summarized, and there was a statistically significant difference (p < 0.001) between the novices and the trained participants (intermediates and experienced). A pass/fail score of 16 points was calculated according to the contrasting-groups method.

CONCLUSION

We developed a test for the assessment of clinical competencies in LUS. The test proved solid validity evidence, and a pass/fail standard without any false-negatives, and only 2 explained false-positives.

Learning Theory Foundations of Simulation-Based Mastery Learning

STATEMENT

Simulation-based mastery learning (SBML), like all education interventions, has learning theory foundations. Recognition and comprehension of SBML learning theory foundations are essential for thoughtful education program development, research, and scholarship. We begin with a description of SBML followed by a section on the importance of learning theory foundations to shape and direct SBML education and research. We then discuss three principal learning theory conceptual frameworks that are associated with SBML-behavioral, constructivist, social cognitive-and their contributions to SBML thought and practice. We then discuss how the three learning theory frameworks converge in the course of planning, conducting, and evaluating SBML education programs in the health professions. Convergence of these learning theory frameworks is illustrated by a description of an SBML education and research program in advanced cardiac life support. We conclude with a brief coda.

Pretest Scores Uniquely Predict 1-Year-Delayed Performance in a Simulation-Based Mastery Course for Central Line Insertion

INTRODUCTION

Within simulation-based mastery learning (SBML) courses, there is inconsistent inclusion of learner pretesting, which requires considerable resources and is contrary to popular instructional frameworks. However, it may have several benefits, including its direct benefit as a form of deliberate practice and its facilitation of more learner-specific subsequent deliberate practice. We consider an unexplored potential benefit of pretesting: its ability to predict variable long-term learner performance.

METHODS

Twenty-seven residents completed an SBML course in central line insertion. Residents were tested on simulated central line insertion precourse, immediately postcourse, and after between 64 and 82 weeks. We analyzed pretest scores' prediction of delayed test scores, above and beyond prediction by program year, line insertion experiences in the interim, and immediate posttest scores.

RESULTS

Pretest scores related strongly to delayed test scores (r = 0.59, P = 0.01; disattenuated ρ = 0.75). The number of independent central lines inserted also related to year-delayed test scores (r = 0.44, P = 0.02); other predictors did not discernibly relate. In a regression model jointly predicting delayed test scores, pretest was a significant predictor (β = 0.487, P = 0.011); number of independent insertions was not (β = 0.234, P = 0.198).

CONCLUSIONS

This study suggests that pretests can play a major role in predicting learner variance in learning gains from SBML courses, thus facilitating more targeted refresher training. It also exposes a risk in SBML courses that learners who meet immediate mastery standards may be incorrectly assumed to have equal long-term learning gains.

Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

OBJECTIVE

Cumulative sum (CuSum) is a real-time proficiency-monitoring tool adapted for simulation-based training. This study's objective was to investigate long-term outcomes of a double blinded, randomized control trial conducted with medical students assessing CuSum-guided curriculum against volume-based standards. The trial found a nearly 20% reduction in practice time to reach proficiency using the CuSum curriculum but long-term effects of decreased practice volume on proficiency is unknown.

DESIGN

Prior participants completed a survey assessing confidence, exposure, and feedback at 12 to 18 months following trial completion. They underwent retention testing of suturing, intubation, and central venous catheter placement (CVC), which was video-recorded and assessed by an expert evaluator. Baseline characteristics among repeat subjects were compared using chi-squared tests. Retention and initial trial outcome were compared using paired parametric statistical methods.

SETTING

The study was conducted at a major tertiary care center and training hospital.

PARTICIPANTS

Medical students, which completed the initial randomized control trial were eligible for enrollment. A total of 30/46(65%) responded to the survey, whereas 33/46(72%) completed retention testing.

RESULTS

Average scores and decay in procedural tasks over time for suturing, intubation and CVC were 91.6% (-4.7%), 86.1% (-4.1%), and 76.2% (-14.8%), respectively. Compared to the control group, the CuSum group mean difference in retention evaluation scores was -5.6% (p = 0.12). Confidence was not associated with initial or retention testing performance in any procedural task. Higher confidence was associated with additional exposure to the procedural task in suturing and intubation (p = 0.03 and p = 0.02, respectively). For intubation, higher confidence was reported by participants who received positive feedback (p = 0.01), and those assigned to the volume-based training arm (p = 0.03).

CONCLUSION

CuSum-guided training was equivalent to conventional training for suturing, intubation, and CVC. These findings importantly suggest medical students can retain competency in invasive surgical tasks with modest decay in proficiency over time regardless of initial training method.

Using virtual-reality simulation to ensure basic competence in hysteroscopy

BACKGROUND

Hysteroscopy is a technically challenging procedure. Specialty curricula of obstetrics and gynaecology appraise hysteroscopy for trainees but there is no present evidence-based training program that certifies the fundamental technical skills before performance on patients. The objectives of this study were to develop and gather validity evidence for a simulation-based test that can ensure basic competence in hysteroscopy.

METHODS

We used the virtual-reality simulator HystMentor™. Six experts evaluated the feasibility and clinical relevance of the simulator modules. Six modules were selected for the test and a pilot study was carried out. Subsequently, medical students, residents, and experienced gynaecologists were enrolled for testing. Outcomes were based on generated simulator metrics. Validity evidence was explored for all five sources of evidence (content, response process, internal structure, relations to other variables, consequences of testing).

RESULTS

Inter-case reliability was high for four out of five metrics (Cronbach's alpha ≥ 0.80). Significant differences were identified when comparing the three groups' performances (p values < 0.05). Participants' clinical experience was significantly correlated to their simulator test score (Pearson's r = 0.49, p < 0.001). A single medical student managed to achieve the established pass/fail score (6.7% false positive) and three experienced gynaecologists failed the test (27.3% false negative).

CONCLUSIONS

We developed a virtual-reality simulation-based test in hysteroscopy with supporting validity evidence. The test is intended to ensure competency in a mastery learning program where trainees practise on the simulator until they are able to pass before they proceed to supervised training on patients.

Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation: A randomised controlled study

BACKGROUND

Although both recorded lectures and serious games have been used to pretrain health professionals before simulation training on cardiopulmonary resuscitation, they have never been compared.

OBJECTIVE

The aim of this study was to compare an online course and a serious game for pretraining medical students before simulation-based mastery learning on the management of sudden cardiac arrest.

DESIGN

A randomised controlled trial. Participants were pretrained using the online course or the serious game on day 1 and day 7. On day 8, each participant was evaluated repeatedly on a scenario of cardiac arrest until reaching a minimum passing score.

SETTING

Department of Simulation in Healthcare in a French medical faculty.

PARTICIPANTS

Eighty-two volunteer second-year medical students participated between June and October 2016 and 79 were assessed for primary outcome.

INTERVENTIONS

The serious game used was Staying Alive, which involved a 3D realistic environment, and the online course involved a PowerPoint lecture.

MAIN OUTCOME MEASURES

The median total training time needed for students to reach the minimum passing score on day 8. This same outcome was also assessed 4 months later.

RESULTS

The median training time (interquartile range) necessary for students to reach the minimum passing score was similar between the two groups: 20.5 (15.8 to 30.3) minutes in the serious game group versus 23 (15 to 32) minutes in the online course group, P = 0.51. Achieving an appropriate degree of chest compression was the most difficult requirement to fulfil for students in both groups. Four months later, the median training time decreased significantly in both groups, but no correlation was found at an individual level with the training times observed on day 8.

CONCLUSION

The serious game used in this study was not superior to an online course to pretrain medical students in the management of a cardiac arrest. The absence of any correlation between the performances of students evaluated during two training sessions separated by 4 months suggests that some elements in the management of cardiac arrest such as compression depth can only be partially learned and retained after a simulation-based training.

TRIAL REGISTRATION

ClinicalTrials.gov-NCT02758119.

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure

BACKGROUND

The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study's objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.

METHODS

Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module 'Guided Vaginal Cuff Closure' six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach's alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups' method.

RESULTS

The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach's alpha). The experienced surgeons' mean composite score for all six repetitions were significantly better than the novice surgeons' (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives).

CONCLUSION

Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.

Physicians' roles in competency-based teaching: Do students recognize them?

Introduction: Imparting the concept of physician's roles with the help of new teaching formats is becoming increasingly important in medical education. The "ambulatory care simulation" was developed at the Medical Faculty Mannheim as a format to teach the roles of "medical expert," "communicator", "health advocate", "manager", "team member" and "professional" in the practical year. During the "ambulatory care simulation", students work through case scenarios with simulated patients focusing on the physician's roles and subsequently discussing and reflecting on their experiences. Several measures are designated to ensure that each role is covered by the new teaching format. The present study investigates whether the physician's roles are actually addressed during the "ambulatory care simulation" and whether the competency-based learning objectives are recognized by the students. Methods: All participants in 12 of the 38 obligatory "ambulatory care simulations" signed informed consents to be filmed during the "ambulatory care simulation". These videos were categorized using previously defined observation criteria. A total of 211 out of 224 students completed and handed in a one-minute paper at the end of the "ambulatory care simulation". The answers to the question, "What have you learned?" have been assigned to competency-based learning objectives. Results: Although instructors and students adhered to the guidelines in the recorded "ambulatory care simulations", the most frequently addressed roles were "medical expert" and "communicator." Two-thirds of the participants indicate learning outcomes that do not correspond to the previously defined learning objectives of the "ambulatory care simulation". Discussion: To ensure a thorough understanding and long-lasting appreciation of the physician's roles, longitudinal integration of teaching interventions into the curriculum is to be favored over single teaching units. Instructors need intensive preparation for the unfamiliar construct of physician's roles. The learning objectives must also be made more transparent. Conclusion: Especially complex teaching formats need to be evaluated for success if they are to achieve their aims. Formative evaluations enable verification of whether the learning objectives are addressed, recognized and, finally, achieved.

Status of Nordic research on simulation-based learning in healthcare: an integrative review

Background

Based on common geography, sociopolitics, epidemiology, and healthcare services, the Nordic countries could benefit from increased collaboration and uniformity in the development of simulation-based learning (SBL). To date, only a limited overview exists on the Nordic research literature on SBL and its progress in healthcare education. Therefore, the aim of this study is to fill that gap and suggest directions for future research.

Methods

An integrative review design was used. A search was conducted for relevant research published during the period spanning from 1966 to June 2016. Thirty-seven studies met the inclusion criteria. All included studies were appraised for quality and were analyzed using thematic analysis.

Results

The Nordic research literature on SBL in healthcare revealed that Finland has published the greatest number of qualitative studies, and only Sweden and Norway have published randomized control trials. The studies included interprofessional or uniprofessional teams of healthcare professionals and students. An assessment of the research design revealed that most studies used a qualitative or a descriptive design. The five themes that emerged from the thematic analysis comprised technical skills, non-technical skills, user experience, educational aspects, and patient safety.

Conclusion

This review has identified the research relating to the progress of SBL in the Nordic countries. Most Nordic research on SBL employs a qualitative or a descriptive design. Shortcomings in simulation research in the Nordic countries include a lack of well-designed randomized control trials or robust evidence that supports simulation as an effective educational method. In addition, there is also a shortage of studies focusing on patient safety, the primary care setting, or a combination of specialized and primary care settings. Suggested directions for future research include strengthening the design and methodology of SBL studies, incorporating a cross-country comparison of studies using simulation in the Nordic countries, and studies combining specialized and primary care settings.

Electronic supplementary material

The online version of this article (10.1186/s41077-018-0071-8) contains supplementary material, which is available to authorized users.