Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit*:

Impact of Simulation-based Mastery Learning on Management of Massive Hemoptysis

Background

The management of massive hemoptysis is a high-risk, low-volume procedure that is associated with high mortality rates, and pulmonary and critical care medicine (PCCM) fellows often lack training. Simulation-based mastery learning (SBML) is an educational strategy that improves skill but has not been applied to massive hemoptysis management.

Objective

This pilot study aimed to develop a high-fidelity simulator, implement an SBML curriculum, and evaluate the impact on PCCM fellows managing massive hemoptysis.

Methods

We modified a simulator to bleed from segmental airways. Next, we developed an SBML curriculum and a validated 26-item checklist and set a minimum passing standard (MPS) to assess massive hemoptysis management. A cohort of traditionally trained providers was assessed using the checklist. First-year PCCM fellows reviewed a lecture before a pretest on the simulator using the skills checklist and underwent rapid-cycle deliberate practice with feedback. Subsequently, fellows were posttested on the simulator, with additional training as necessary until the MPS was met. We compared pretest and posttest performance and also compared SBML-trained fellows versus traditionally trained providers.

Results

The MPS on the checklist was set at 88%. All first-year PCCM fellows (N = 5) completed SBML training. Mean checklist scores for SBML participants improved from 67.7 ± 8.4% (standard deviation) at pretest to 84.6 ± 6.7% at the initial posttest and 92.3 ± 5.4% at the final (mastery) posttest. Traditionally trained participants had a mean test score of 60.6 ± 13.1%.

Conclusion

The creation and implementation of a massive hemoptysis simulator and SBML curriculum was feasible and may address gaps in massive hemoptysis management training.

Use of a Novel Three-Dimensional Model to Teach Ultrasound-guided Subclavian Vein Cannulation

Background

Central venous cannulation is an essential skill in perioperative and critical care medicine. Ultrasound guidance is the standard of care for femoral and internal jugular vein access, with the subclavian vein being perceived to be less amenable to ultrasound-guided (UG) insertion, resulting in a lack of procedural competency and low cannulation rate. There is a paucity of resources and a lack of experience among staff physicians to effectively instruct trainees. Simulation-based medical education has the potential to help maintain high-stakes, infrequently performed skills and counteract possible unrecognized skill decline. We aimed to create a novel, low-cost, high-fidelity three-dimensional (3D) model for UG subclavian vein (UG-SCV) access with an accompanying curriculum to improve this important skill.

Methods

A curriculum was created consisting of preparatory material reviewing UG-SCV access, followed by an in-person didactic lecture focusing on ultrasound use and management of complications and a deliberate practice session scanning volunteers and practicing UG vascular puncture on a 3D model. A qualitative usability test design was used to assess the validity of the curriculum in trainees with advanced vascular access skills (anesthesiologists). Participants were second-year anesthesia residents, anesthesia fellows, and staff physicians. Focus groups conducted after each session explored the face validity of the model and curriculum. By applying a usability design, the curriculum was optimized and finalized.

Results

Between September 2020 and February 2021, 28 participants tested the curriculum. The focus groups ensured that the curriculum achieved its objective, with iterative changes made after each session in a quality improvement framework Plan-Do-Study-Act approach. After the third cycle, minimal changes were suggested, and the curriculum and 3D model were finalized. An additional group of participants was used to ensure that no new input would help improve the curriculum further.

Conclusions

A focused curriculum for enhancing skills in UG-SCV cannulation using a novel 3D model was successfully implemented and validated through a usability test design. This curriculum is better targeted for practitioners experienced in central venous access to master a subclavian approach and maintain their skill level.

Performance of peripheral catheters inserted with ultrasound guidance versus landmark technique after a simulation-based mastery learning intervention

PROBLEM

Ultrasound-guided peripheral intravenous catheter (USGPIV) insertion is an effective method to gain vascular access in patients with difficult intravenous access (DIVA). While USGPIV success rates are reported to be high, some studies have reported a concerning incidence of USGPIV premature failures.

AIMS

The purpose of this study was to compare differences in USGPIV and landmark peripheral intravenous catheter (PIV) utilization and failure following a hospital-wide USGPIV training program for nurses.

METHODS

The authors performed a retrospective, electronic medical record review of all USGPIVs and PIVs inserted at a tertiary, urban, academic medical center from September 1, 2018, through September 30, 2019. The primary outcome was differences between USGPIV and PIV time to failure.

RESULTS

A total of 43,470 short peripheral intravenous catheters (PIVCs) were inserted in 23,713 patients. Of these, 7972 (16.8%) were USGPIV. At 30 days of follow-up, for PIVCs with an indication for removal documented, USGPIVs had higher Kaplan-Meier survival probabilities than PIVs (p < 0.001).

CONCLUSIONS

The use of simulation-based mastery associated with USGPIVs, demonstrated lower failure rates than standard PIVs after 2 days and USGPIVs exhibited improved survival rates in patients with DIVA. These findings suggest that rigorous simulation-based insertion training demonstrates improved USGPIV survival when compared to traditional PIVCs. SBML is an extremely useful tool to ensure appropriately trained clinicians acquire the necessary knowledge and skillset to improve USGPIV outcomes.

The Resident Learning Journey in the Medical Intensive Care Unit

Background

The medical intensive care unit (MICU) offers rich resident learning opportunities, but traditional teaching strategies can be difficult to employ in this fast-paced, high-acuity environment. Resident perspectives of learning within this environment may improve our understanding of the common challenges residents face and inform novel approaches to transform the MICU educational experience.

Objective

We conducted a qualitative study of internal medicine residents to better understand their approach to learning the critical care activities that they are entrusted to perform in the MICU.

Methods

Using a thematic analysis approach, we conducted six focus group interviews with 15 internal medicine residents, separated by postgraduate year. A trained investigator led each interview, which was audio-recorded and transcribed verbatim for analysis. Our diverse research team, representing different career stages across the continuum of learning to minimize interpretive bias, identified codes and subsequent themes inductively. We refined these themes through group discussion and sensitizing social learning theory concepts using Wenger's community of practice and organized them to create learner archetypes and a conceptual framework of resident learning in the MICU.

Results

We identified three thematic resident learning categories: learning goals and motivation, clinical engagement, and interprofessional collaboration. We distinguished three learner archetypes, the novice, experiential learner, and practicing member, to describe progressive resident development within the interprofessional MICU team, the challenges they frequently encounter, and potential teaching strategies to facilitate learning.

Conclusion

We developed a conceptual framework that describes the resident's journey to becoming a trusted, collaborating member of the interprofessional MICU team. We identified common developmental challenges residents face and offer educational strategies that may support their progress. These findings should inform future efforts to develop novel teaching strategies to promote resident learning in the MICU.

Clinical Experience Is Not a Proxy for Competence: Comparing Fellow and Medical Student Performance in a Breaking Bad News Simulation-Based Mastery Learning Curriculum

BACKGROUND

It is unknown whether traditional medical education ensures competence among fellows in the key skill of breaking bad news (BBN). While simulation-based mastery learning (SBML) has been used to train fourth-year medical students (M4s) in BBN, it is unclear if it adds similar value for fellows.

OBJECTIVE

We examined the effect of traditional medical training on BBN skills by comparing baseline fellow and M4 skills and confidence and assessed the impact of a BBN SBML curriculum for fellows.

METHODS

Fellows training in six programs at Northwestern University from November 2018 to May 2019 were eligible for inclusion. Fellows completed a BBN SBML curriculum including a pretest, individualized feedback using a previously published assessment tool, and ongoing deliberate practice until all achieved a minimum passing standard (MPS). The primary outcomes were checklist and scaled item scores on the assessment tool. Fellow performance was compared to a historical M4 cohort.

RESULTS

Twenty-eight of 38 eligible fellows completed the curriculum and were included for analysis. Fellows reported significantly more experience and confidence in BBN compared to M4s, yet their pre-training performance was significantly worse on checklist (57.1% vs 65.0%, P = .02) and scaled items; only 4% reached the MPS. After training, fellow performance significantly improved on checklist (57.1% to 92.6%, SD = 5.2%, P < .001) and scaled items; all reached the MPS.

CONCLUSIONS

Despite higher confidence and BBN clinical experience, fellows performed worse than untrained M4s, confirming that experience is not a proxy for skill. Programs must develop competency-based assessments to ensure entrustment of communication skills.

Simulation-based Mastery Learning Improves Critical Care Skills of Advanced Practice Providers

Background

Advanced practice providers (APPs) are essential members of intensive care unit (ICU) interprofessional teams and are expected to be competent in performing procedures. There are no published criteria for establishing when APPs can independently perform procedures. Simulation-based mastery learning (SBML) is an effective strategy for improving critical care skills but has not been applied to practicing ICU APPs.

Objective

The purpose of this study was to evaluate if an SBML curriculum could improve the critical care skills and procedural self-confidence of ICU APPs.

Methods

We performed a pretest-posttest study of central venous catheter (CVC) insertion, thoracentesis, and mechanical ventilation (MV) management skills among ICU APPs who participated in an SBML course at an academic hospital. For each skill, APPs underwent baseline skills assessments (pretests) on a simulator using previously published checklists, followed by didactic sessions and deliberate practice with individualized feedback. Within 2 weeks, participants were required to meet or exceed previously established minimum passing standards (MPS) on simulated skills assessments (posttests) using the same checklists. Further deliberate practice was provided for those unable to meet the MPS until they retested and met this standard. We compared pretest to posttest skills checklist scores and procedural confidence.

Results

All 12 eligible ICU APPs participated in internal jugular CVC, subclavian CVC, and MV training. Five APPs participated in thoracentesis training. At baseline, no APPs met the MPS on all skills. At training completion, all APPs achieved the mastery standard. Internal jugular CVC pretest performance improved from a mean of 67.2% (standard deviation [SD], 28.8%) items correct to 97.1% (SD, 3.8%) at posttest (P = 0.005). Subclavian CVC pretest performance improved from 29.2% (SD, 32.7%) items correct to 93.1% (SD 3.9%) at posttest (P < 0.001). Thoracentesis pretest skill improved from 63.9% (SD, 30.6%) items correct to 99.2% (SD, 1.7%) at posttest (P = 0.054). Pretest MV skills improved from 54.8% (SD, 19.7%) items correct to 92.3% (SD, 5.0%) at posttest (P < 0.001). APP procedural confidence improved for each skill from pre to posttest.

Conclusion

SBML is effective for training APPs to perform ICU skills. Relying on traditional educational methods does not reliably ensure that APPs are adequately prepared to perform skills such as CVC insertion, thoracentesis, and MV management.

Who can do this procedure? Using entrustable professional activities to determine curriculum and entrustment in anesthesiology - An international survey

INTRODUCTION

As competency-based curricula get increasing attention in postgraduate medical education, Entrustable Professional Activities (EPAs) are gaining in popularity. The aim of this survey was to determine the use of EPAs in anesthesiology training programs across Europe and North America.

METHODS

A survey was developed and distributed to anesthesiology residency training program directors in Switzerland, Germany, Austria, Netherlands, USA and Canada. A convergent design mixed-methods approach was used to analyze both quantitative and qualitative data.

RESULTS

The survey response rate was 38% (108 of 284). Seven percent of respondents used EPAs for making entrustment decisions. Fifty-three percent of institutions have not implemented any specific system to make such decisions. The majority of respondents agree that EPAs should become an integral part of the training of residents in anesthesiology as they are universal and easy to use.

CONCLUSION

Although recommended by several national societies, EPAs are used in few anesthesiology training programs. Over half of responding programs have no specific system for making entrustment decisions. Although several countries are adopting or planning to adopt EPAs and national societies are recommending the use of EPAs as a framework in their competency-based programs, few are yet using these to make "competence" decisions.

Creation of an International Interprofessional Simulation-enhanced Mechanical Ventilation Course

Background

Evidence shows poor adherence to strategies for reducing morbidity and mortality in intensive care unit (ICU) patients receiving mechanical ventilation globally. Best practice management relies on training all members of the interprofessional ICU team, each with complementary roles in patient management.

Objectives

To develop and evaluate a novel two-phase, train-the-trainer, interprofessional and multicultural “Best Practice Management of the Ventilated ICU Patient” multimodality, simulation-enhanced curriculum for Thai education leaders in critical care.

Methods

In phase 1 (Oregon Health and Science University cohort), two groups of nine ICU nurses and one critical care physician representing experts in critical care and education from a large hospital system in Thailand participated in a weeklong, immersive course consisting of didactic, simulation, and in situ immersive sessions focused on best practice management of mechanically ventilated ICU patients, as well as training in our educational techniques. Outcomes were assessed with pre- and postcourse knowledge assessments and overall course evaluation. In phase 2 (Thai cohort), participants from phase 1 returned to Thailand and implemented a lower fidelity curriculum in two hospitals, using the same pre- and posttest knowledge assessment in 41 participants, before the onset of the coronavirus disease (COVID-19) 6 pandemic.

Results

In the Oregon Health and Science University cohort, the mean pretest knowledge score was 58.4 ± 13.2%, with a mean improvement to 82.5 ± 11.6% after completion of the course (P , 0.05). The greatest improvements were seen in respiratory physiology and advanced/disease-specific concepts, which demonstrated absolute improvements of 30.4% and 30.6%, respectively (P < 0.05). Participants had a high degree of satisfaction, with 90% rating the course as “excellent” and .90% reporting that the course “greatly improved” their understanding of best practices and comfort in managing mechanical ventilation. The Thai cohort had a mean baseline score of 45.4 ± 15.0% and a mean improvement to 70.3 ± 19.1% after training (P < 0.05). This cohort also saw the greatest improvement in respiratory physiology and advanced/disease-specific concepts, with 26.2% and 26.3% absolute improvements, respectively (P < 0.05).

Conclusion

A novel, two-phase, interprofessional, multicultural, simulation-enhanced train-the-trainer curriculum was feasible and effective in improving education in best practice management of mechanically ventilated patients and may be a useful model for improving the care of ICU patients across the world.

Research Hotspots and Trend Exploration on the Clinical Translational Outcome of Simulation-Based Medical Education: A 10-Year Scientific Bibliometric Analysis From 2011 to 2021

BACKGROUND

In recent decades, an increasing number of studies have focused on the clinical translational effect of simulation-based medical education (SBME). However, few scientific bibliometric studies have analyzed the research hotspots and publication trends. This study aimed to investigate research hotspots and future direction in the clinical translational outcome of SBME via bibliometrics.

METHOD

Relevant publications on the clinical translational outcomes of SBME from 2011 to 2021 were identified and retrieved from the Web of Science Core Collection (WOSCC). Software including VOSviewer (1.6.17) and CiteSpace (5.8R3) and a platform (bibliometric.com) were employed to conduct bibliographic and visualized analysis on the literature.

RESULTS

A total of 1,178 publications were enrolled. An increasing number of publications were observed in the past decades from 48 in 2011 to 175 in 2021. The United States accounted for the largest number of publications (488, 41.4%) and citations (10,432); the University of Toronto and Northwestern University were the leading institutions. Academic Medicine was the most productive journal concerning this field. McGaghie W C and Konge L were the most influential authors in this area. The hot topic of the translational outcome of SBME was divided into 3 stages, laboratory phase, individual skill improvement, and patient outcome involving both technical skills and non-technical skills. Translational research of comprehensive impact and collateral outcomes could be obtained in the future.

CONCLUSION

From the overall trend of 10 years of research, we can see that the research is roughly divided into three phases, from laboratory stage, individual skill improvement to the patient outcomes, and comprehensive impacts such as skill retention and collateral effect as cost-effectiveness is a major trend of future research. More objective evaluation measurement should be designed to assess the diverse impact and further meta-analysis and randomized controlled trials are needed to provide more clinical evidence of SBME as translational science.

Current practice, education, and recommendations for training of central line insertion for trainees and fellows in adult ICUs across Australia and New Zealand

BACKGROUND

The objective of this study was to explore variability in existing training and accreditation processes for Central venous access device (CVAD) insertion among different ICUs, current practices of CVAD insertion among fellows of the College of Intensive Care Medicine (CICM) working in Australia and New Zealand (ANZ) and their recommendations for improvement.

METHODS

A prospective cross-sectional web-based survey was sent via email and CICM e-Newsletter to intensivists and directors of intensive care units (ICUs) across ANZ. All responses were tabulated, post-hoc exploratory analysis using multivariable ordinal logistic regression was used and free texts were analysed thematically and summarized.

RESULTS

115 responses were received from various public and private ICUs from all states of ANZ. 32% of the participants did not have any accreditation process for CVAD insertion skill in their ICUs, whereas 91% of respondents revealed there were no processes to assess deskilling. Most intensivists recommended supervision, simulation, various education tools, and ultrasound training to improve training and assessment. 35% of the participants inserted 0 to 5 CVADs and more than half of the intensivists had inserted <10 CVADs in one-year period. Two-thirds of the respondents recommended inserting between 6 to 20 CVADs each year to maintain competence.

CONCLUSION

The study identified wide variability in current practice, training methods and accreditation process for CVAD insertion amongst intensivists and ICU trainees in ANZ. Policy makers should consider revising the current clinical practice and training policies to new policies for accreditation and ongoing assessment for CVAD insertions across ANZ ICUs. This article is protected by copyright. All rights reserved.

Training Pulmonary Critical Care Medicine Fellows in Thoracentesis Using a Head-Mounted Video Camera

BACKGROUND

Determination of competence to perform procedures during pulmonary critical care medicine fellowship training has traditionally been based on subjective faculty opinion and numerical requirements.

OBJECTIVE

To describe an objective means of assessing competence of fellows to perform thoracentesis using a head-mounted video camera with offline scoring of the thoracentesis performed on an actual patient.

METHODS

To test competence in performance of thoracentesis after a multimodality training program, a total of eight first-year fellows performed a thoracentesis on an actual patient while recording the procedure with a lightweight head-mounted video camera in 2017 and 2018. The recordings were scored offline by two faculty members using a 30-point checklist. The percentage agreement between scorers was measured, as was the opinion of the fellows and the scorers on the testing process. If a fellow failed completion of all checklist items, they were provided with further training and retested to assure competence. As part of their training, fellows reviewed the video record of their procedures.

RESULTS

Eight first-year fellows were tested, of whom seven successfully completed key checklist items as determined by the video scorers. One failing fellow passed after further training and testing. The percentage agreement between the scorers was high, and fellows indicated that the video device was useful for training.

CONCLUSION

This study supports the use of video-based testing for assessment of competence and for training in performance of thoracentesis by fellows.

System for central venous catheterization training using computer vision-based workflow feedback

OBJECTIVE

To develop a system for training central venous catheterization that does not require an expert observer. We propose a training system that uses video-based workflow recognition and electromagnetic tracking to provide trainees with real-time instruction and feedback.

METHODS

The system provides trainees with prompts about upcoming tasks and visual cues about workflow errors. Most tasks are recognized from a webcam video using a combination of a convolutional neural network and a recurrent neural network. We evaluate the systems ability to recognize tasks in the workflow by computing the percent of tasks that were recognized and the average signed transitional delay between the system and reviewers. We also evaluate the usability of the system using a participant questionnaire.

RESULTS

The system was able to recognize 86.2% of tasks in the workflow. The average signed transitional delay was -0.7 8.7s. The average score on the questionnaire was 4.7 out of 5 for the system overall. The participants found the interactive task list to be the most useful component of the system with an average score of 4.8 out of 5.

CONCLUSION

Overall, the participants were happy with the system and felt that it would improve central venous catheterization training. Our system provides trainees with meaningful instruction and feedback without needing an expert observer to be present.

SIGNIFICANCE

We are able to provide trainees with more opportunities to access instruction and meaningful feedback by using workflow recognition.

FUN AND GAMES: DESIGNING A GAMIFIED CENTRAL VENOUS CATHETERIZATION TRAINING SIMULATOR

Gamification, or adding elements of games to training systems, has the potential to increase learner engagement and information retention. However, the use of gamification has yet to be explored in Central Venous Catheterization (CVC) trainers which teach a commonly performed medical procedure with high incidence rates. In order to combat these errors, a Dynamic Haptic Robotic Trainer (DHRT) was developed, which focuses on vessel identification and access. A DHRT+ system is currently under development that focuses on whole procedure training (e.g. sterilization and catheter insertion), including a gamified Graphical User Interface. The goal of this paper was to (1) develop a game-like, patient-centered interface to foster personalized learning and (2) understand the perceived utility of gamification for CVC skill development with expert doctors. This paper outlines some of the potential benefits and deficits of the use of gamification in medical trainers that can be used to drive simulation design.

Novel cricothyrotomy assessment tool for attending physicians: A multicenter study of an error avoidance checklist

BACKGROUND

This study used existing literature and expert feedback to develop and pilot a novel error-avoidance checklist tool for cricothyrotomy in attending physicians. Prior literature has not focused on expert cricothyrotomy performance. While published checklists teach a specific procedural method, ideal for novice learners, this may hinder expert learners.

OBJECTIVES

We endeavored to create a succinct error-avoidance checklist for cricothyrotomy. We hypothesized that such a checklist would prove feasible and acceptable to attending physicians.

METHODS

This is a multicenter prospective checklist creation, evaluation, and feasibility study. Multiple experts pursued an iterative process to reach consensus on a 7-item error-avoidance checklist. The checklist was trialed for feasibility in pilot sessions at two sites by 45 attending emergency physicians who used the checklist for peer performance assessment and provided feedback.

RESULTS

During the pilot implementation, 94% of respondents completed the procedure within the allotted 120 s. Greater than 85% of respondents agreed that four of the five procedural errors on the checklist were very or somewhat critical to avoid, including cutting >2 cm from midline, creating a false passage, failing to continuously maintain an object in the trachea, and injuring oneself during the procedure. Only 66% of participants felt severing the cricoid cartilage was critical. Successful breath administration and time under 120 s were critical for 100% and 95% of participants, respectively. The checklist was rated "easy" or "very easy" to use by 93% of participants, and 95% found this checklist reasonable for evaluating attending physicians.

CONCLUSIONS

We present the multicenter development and implementation of a novel error-avoidance checklist tool for use in expert cricothyrotomy performance. Attending emergency medicine (EM) physicians rated our tool easy to use and agreed that most of the proposed errors were critical. Participants overwhelmingly agreed this tool would be reasonable for evaluation of cricothyrotomy performance among attending EM physicians.

Mastery learning improves simulated central venous catheter insertion by emergency medicine teaching faculty

OBJECTIVES

Routine competency assessments of procedure skills, such as central venous catheter (CVC) insertion, do not occur beyond residency training. Evidence suggests variable, suboptimal attending physician procedure skills. Our study aimed to assess CVC insertion skill by academic emergency physicians, determine whether a simulation-based mastery learning (SBML) intervention improves performance and investigate for variables that predict competence.

METHODS

This is a pretest-posttest study that evaluated simulated CVC insertion by emergency medicine (EM) faculty physicians. We assessed 44 volunteer participants at a large academic medical center over a 1-month period using a published 29-item checklist. Our primary outcome was the difference in assessment score before and after a SBML intervention. A secondary analysis evaluated predictors of pretest performance.

RESULTS

A total of 44 subjects participated. Only four of 44 (9.1%) of subjects met a predefined minimum passing score on pretest. Mean assessment scores increased by 21.5% following the SBML intervention (95% confidence interval [CI] of the difference = 18.1% to 24.8%, p < 0.001). In a regression model, pretest scores increased by 10.8% (95% CI = 2.9 to 18.7%, p = 0.009) if subjects completed postgraduate training within 5 years. Frequency of CVC insertion did not predict performance, but 25 of 44 (56.8%) faculty members had no documented performance or supervision of a CVC insertion within 1 year of assessment.

CONCLUSIONS

SBML is a promising method to assess and improve CVC insertion performance by EM faculty physicians. Recent completion of postgraduate training was a significant predictor of CVC insertion performance. Our results require validation in larger cohorts of EM physicians across other academic institutions.

Using a Simulated Model and Mastery Learning Approach to Teach the Ultrasound-guided Serratus Anterior Plane Block to Emergency Medicine Residents: A Pilot Study

BACKGROUND

The serratus anterior plane block (SAPB) is a safe, single-injection alternative for pain control in patients with rib fractures. This pilot study aims to teach the ultrasound-guided SAPB to emergency medicine (EM) residents using a mastery learning approach.

METHODS

A 19-item checklist was created and mastery was determined to be 17 of 19 items correct. This pass score was established using a Mastery Angoff standard-setting exercise with a group of EM experts. Learners participated in baseline testing on a simulated model and performance was assessed by two raters. Learners then watched an instructional video and participated in an individualized teaching session. Learners underwent deliberate practice followed by posttesting until mastery was achieved. Score differences in baseline testing and posttesting were analyzed using a paired t-test. Pre- and posttesting surveys were also completed by participants.

RESULTS

Twenty-eight PGY-1 to -4 residents volunteered to participate in the study. The range of reported SAPBs seen previously was 0 to 5. The mean (±SD) number of items correct on the checklist for initial testing was 8.5 of 19 (±2.7), while the mean (±SD) final score was 18 of 19 (±0.6; p < 0.001). All participants met mastery standards after the curriculum intervention. Median self-reported procedural confidence was 2 out of 5 on a 5-point Likert scale before the session and 5 out of 5 after the session (Z = -4.681, p < 0.001).

CONCLUSIONS

Using a mastery learning approach and simulated model, we were able to successfully train EM residents to perform the SAPB at a level of mastery and increase their overall confidence in executing this procedure.

A randomized controlled trial of simulation-based mastery learning to teach the extended focused assessment with sonography in trauma

BACKGROUND

Mastery learning has gained popularity for training residents in procedural skills due to its demonstrated superiority over traditional methods. However, no studies have compared the efficacy of traditional versus mastery learning methods in residency point-of-care ultrasound education. We hypothesized that mastery learning would improve residents' skills in performing the extended focused assessment with sonography in trauma (eFAST).

METHODS

All first-year emergency medicine (EM) resident physicians at a single university hospital underwent a crossover randomized controlled trial to receive mastery-learning eFAST training either at the beginning of the academic year or 6 months into intern year. Participants were taught using a checklist validated by a panel of experts using Mastery Angoff methods and were given feedback on missed tasks until each trainee completed the eFAST with a minimum passing standard (MPS). Our primary outcome was technical proficiency between the two groups for eFAST examinations performed in the emergency department during the academic year.

RESULTS

Sixteen interns were enrolled; eight were randomized to each group. The group that received mastery training at the beginning of the year had mean clinical eFAST proficiency scores above the MPS in the first two quarters of the academic year, while the control group did not. Once the control group underwent eFAST mastery training at the midpoint of the year, both groups had mean proficiency scores above the MPS for the remainder of the year.

CONCLUSION

Simulation-based mastery learning is an effective method of teaching the eFAST examination. This training during intern orientation conferred early proficiency in clinical performance of eFAST among EM residents. This difference in proficiency was no longer present after the control group received mastery learning education halfway through the academic year.

Value Analysis of Central Line Simulation-Based Education

BACKGROUND

Central line-associated bloodstream infections (CLABSIs) are preventable complications that pose a significant health risk to patients and place a financial burden on hospitals. Central line simulation-based education (SBE) efforts vary widely in the literature. The aim of this study was to perform a value analysis of published central line SBE and develop a refined method of studying central line SBE.

METHODS

A database search of PubMed Central and Cumulative Index to Nursing and Allied Health Literature (CINAHL) was performed for articles mentioning "Cost and CLABSI," "Cost and Central line Associated Bloodstream Infections," and "Cost and Central Line" in their abstract and article body. Articles chosen for qualitative synthesis mentioned "simulation" in their abstract and article body and were analyzed based on the following criteria: infection rate before vs. after SBE, cost of simulation, SBE design including simulator model used, and learner analysis.

RESULTS

Of 215 articles identified, 23 were analyzed, 10 (43.48%) discussed cost of central line simulation with varying criteria for cost reporting, 8 (34.8%) numerically discussed central line complication rates (7 CLABSIs and 1 pneumothorax), and only 3 (13%) discussed both (Figure). Only 1 addressed the true cost of simulation (including space rental, equipment startup costs, and faculty salary) and its longitudinal effect on CLABSIs.

CONCLUSION

Current literature on central line SBE efforts lacks value propositions. Due to the lack of value-based data in the area of central line SBE, the authors propose a cost reporting standard for use by future studies reporting central line SBE costs.

An effective procedure skills training programme for GIM registrars

Background

The Royal College of Physicians' Acute care toolkit 8 recommends procedural training for medical registrars at all hospitals. We aimed to determine the interest and need, and to pilot the delivery of such training in the procedures outlined by the Joint Royal Colleges of Physicians Training Board (2017).

Methods

An online survey was sent to general internal medicine (GIM) trainees within the Thames Valley Deanery in January 2019. This identified a need for procedure skills training. Ninety per cent of trainees felt simulation training would improve their confidence in the outlined procedures.

We trialled a simulation programme for GIM registrars between September 2019 and October 2019. Sessions lasted 3-3.5 hours and trainees rotated through four stations. Feedback was obtained from trainees and trainers during each pilot session.

Results

Thirty-two trainees attended across both sites. Excellent feedback was obtained and trainee confidence improved by visual analogue scale scoring post-training for all procedures. Almost 90% of trainees felt the sessions would improve safety on GIM on calls.

Conclusion

Simulation training is an effective way to improve trainee confidence in procedural skills and this pilot shows such training is desired and necessitated in higher specialty training. Further work will assess its impact on maintaining trainee skillsets and impact on patient safety.

Can we improve teaching and learning of percutaneous dilatational tracheostomy's bronchoscopic guidance?

Percutaneous dilatational tracheostomy has become the technique of choice in multiple intensive care units. Among innovations to improve procedural safety and success, bronchoscopic guidance of percutaneous dilatational tracheostomy has been advocated and successfully implemented by multiple groups. Most published literature focuses on the percutaneous dilatational tracheostomy operator, with scarce descriptions of the bronchoscopic particularities of the procedure. In this article, we provide 10 suggestions to enhance specific procedural aspects of bronchoscopic guidance of percutaneous dilatational tracheostomy, and strategies to optimize its teaching and learning, in order to promote learners' competence acquisition and increase patient safety.

Immersive Video Modeling Versus Traditional Video Modeling for Teaching Central Venous Catheter Insertion to Medical Residents

Background 

Central Venous Catheter (CVC) placement is a common critical care procedure. Simulated practice has been shown to reduce its iatrogenic complications. Video modeling (VM) is an instructional adjunct that improves the quality and success of CVC insertion. Immersive VM can improve recall and skill translation, but its role in teaching medical procedures is not established.

Research question/hypothesis

We hypothesized that, relative to traditional VM, immersive VM would decrease cognitive load and enhance ultrasound-guided CVC insertion skill acquisition.

Methods

Thirty-two resident physicians from four specialties were randomized into traditional (control) or immersive VM (intervention) groups for three CVC training sessions. Cognitive load was quantified via NASA Task Load Index (TLX). Mean (± standard deviations) values were compared using two-tailed t-tests. Skill acquisition was quantified by procedural time and the average 5-point [EM1] [TB2] entrustment score of three expert raters.

Results

Overall entrustment scores improved from the first (3.44±0.98) to the third (4.06±1.23; p<0.002) session but were not significantly different between the control and intervention groups. There were no significant differences between NASA TLX scores or procedural time.

Conclusion

We found no significant difference in entrustment, cognitive load, or procedural time. Immersive VM was not found to be superior to traditional VM for teaching CVC insertion.

Safety culture transformation-The impact of training on explicit and implicit safety attitudes

The present paper investigates the changeability of safety culture elements such as explicit and implicit safety attitudes by training. Therefore, three studies with different time frames, training durations, and settings will be presented. In the first study, the short-term attitude change of students from an international environmental sciences study program was measured after safety training in a chemical laboratory. In the second study, the medium-term attitude change was assessed after a Crew Resource Management training for German production workers in the automotive industry. In the third study, the long-term attitude changes were measured after safety ethics training in a sample of German occupational psychology and business students. Different self-report measures were used to evaluate the training effectiveness of explicit safety attitudes. The change of implicit safety attitudes was assessed by Implicit Association Tests. The results of all three studies revealed a significant training effect on the explicit safety attitudes, but not on the implicit ones. Besides the training effect on the explicit attitudes, there was no effect of time frame (short-, medium-, long-term), training duration (2 h, 2 days, 12 weeks), and setting (chemical laboratory, automotive industry, safety ethics study program) on the attitude change. Based on the results, conceptual, methodological, and practical implications for training effectiveness and safety culture transformation are discussed.

Impact of Simulation-based Mastery Learning on Resident Skill Managing Mechanical Ventilators

Background: Caring for patients requiring mechanical ventilation is complex, and residents may lack adequate skill for managing these patients. Simulation-based mastery learning (SBML) is an educational model that trains clinicians to a high standard and can reduce complications. The mastery learning model has not been applied to ventilator management. Objective: The purpose of this study was to determine whether SBML, as compared with traditional training, is an effective strategy for teaching residents the skills necessary to manage patients requiring mechanical ventilation. Methods: We developed an SBML curriculum and a 47-item skills checklist to test ventilator management for patients with normal, restricted, and obstructed lung physiology. A minimum passing standard (MPS) on the checklist was set using the Mastery Angoff method. Residents rotating through the medical intensive care unit in Academic Year 2017-2018 were assigned to SBML or traditional training based on their medical intensive care unit team. The SBML group was pretested on a ventilator simulator using the skills checklist. They then received a 1.5-hour session (45 min didactic and 45 min deliberate practice on the simulator with feedback). At rotation completion, they were posttested on the simulator using the checklist until the MPS was met. Both SBML-trained and traditionally trained groups received teaching during daily bedside rounds and twice weekly didactic lectures. At rotation completion, traditionally trained residents were tested using the same skills checklist on the simulator. We compared pretest and posttest performance among SBML-trained residents and end of the rotation test performances between the SBML-trained and traditionally trained residents. Results: The MPS was set at 87% on the checklist. Fifty-seven residents were assigned to the SBML-trained group and 49 were assigned to the traditionally trained group. Mean checklist scores for SBML-trained residents improved from 51.4% (standard deviation [SD] = 17.5%) at pretest to 86.1% (SD = 7.6%) at initial posttest and 92.5% (SD = 3.7%) at final (mastery) posttest (both P < 0.001). Forty-two percent of residents required more than one attempt at the posttest to meet or exceed the MPS. At rotation completion, the traditionally trained residents had a mean test score of 60.9% (SD = 13.3%). Conclusion: SBML is an effective strategy to train residents on mechanical ventilator management. An SBML curriculum may augment traditional training methods to further equip residents to safely manage ventilated patients.

Simulation-Based Medical Education Improves Procedural Confidence in Core Invasive Procedures for Military Internal Medicine Residents

Introduction

The American Board of Internal Medicine (ABIM) requires that trainees receive procedural training for certification; however, Internal Medicine (IM) residents perform a variable number of procedures throughout residency training. This results in differences in confidence levels as well as procedural competence. For active-duty military trainees, this is especially problematic, as these procedural skills are often required during deployment soon after residency graduation. This deficit can be improved through standardized simulation-based training.

Methods

All internal medicine residents at our institution were invited to participate in a standardized simulation-based training program for core internal medicine procedures (lumbar puncture, arterial line, central line, thoracentesis, paracentesis, and arthrocentesis). Residents were asked to qualitatively rate their perceived procedural confidence using a Likert scale ranging from 1 (not at all confident) to 5 (extremely confident) in their ability to independently perform core internal medicine procedures prior to the simulation exercise. Experienced senior residents and internal medicine faculty instructed and supervised each resident as they performed the procedures. Following the simulation exercise, the residents repeated the survey and were asked to report whether or not they found the exercise useful. 

Results

Of the 96 residents invited to participate, 49 completed the pre-simulation questionnaire and 36 completed the post-simulation questionnaire. The cumulative mean Likert scale confidence rating for all procedures showed a statistically significant improvement post-simulation as compared to pre-simulation, including lumbar puncture (2.45±1.1 vs. 3.42±0.87, p<0.05), arterial line (2.48±1.06 vs. 3.39±1.04, p < 0.05), central line (2.86±1.08 vs. 3.5±1.02, p < 0.05), thoracentesis (2.67±1.10 vs. 3.64±0.83, p < 0.05), paracentesis (3.1±1.08 vs. 3.82±0.74, p < 0.05), and arthrocentesis (2.56±1.07 vs. 3.67±0.80, p < 0.05). All (36/36) trainees reported that they perceived the simulation exercise as valuable.

Conclusion

Internal medicine residents across all post-graduate year (PGY) levels at our institution lacked confidence to independently perform core internal medicine procedures. Utilizing simulation-based medical education as an adjunct to clinical training is well accepted by internal medicine trainees, and resulted in significantly improved procedural confidence. This intervention was well received by trainees and could feasibly be replicated at other active-duty military internal medicine residency programs to assist with readiness. Research is currently in progress to correlate in-situ competency and evaluate clinical outcomes of this improved confidence.

Use of a simulation-based mastery learning curriculum to improve ultrasound-guided vascular access skills of pediatric anesthesiologists

BACKGROUND

Pediatric vascular access is inherently challenging due to the small caliber of children's vessels. Ultrasound-guided intravenous catheter insertion has been shown to increase success rates and decrease time to cannulation in patients with difficult intravenous access. Although proficiency in ultrasound-guided intravenous catheter insertion is a critical skill in pediatric anesthesia, there are no published competency-based training curricula.

AIMS

The objective of this study was to evaluate the performance of pediatric anesthesiologists who participated in a novel ultrasound-guided intravenous catheter insertion simulation-based mastery learning curriculum.

METHODS

Pediatric anesthesia attendings, fellows, and rotating residents participated in the ultrasound-guided intravenous catheter insertion simulation-based mastery learning curriculum from August 2019 to February 2020. The 2-hour curriculum consisted of participants first undergoing a simulated skills pretest followed by watching a video on ultrasound-guided intravenous catheter insertion and deliberate practice on a simulator. Subsequently, all participants took a post-test and were required to meet or exceed a minimum passing standard. Those who were unable to meet the minimum passing standard participated in further practice until they could be retested and met this standard. We compared pre to post-test ultrasound-guided intravenous catheter insertion skills and self-confidence before and after participation in the curriculum.

RESULTS

Twenty-six pediatric anesthesia attendings, 12 fellows, and 38 residents participated in the curriculum. At pretest, 16/76 (21%) participants were able to meet or exceed the minimum passing standard. The median score on the pretest was 21/25 skills checklist items correct and improved to 24/25 at post-test (95% CI 3.0-4.0, P < .01). Self-confidence significantly improved after the course from an average of 3.2 before the course to a postcourse score of 3.9 (95% CI 0.5-0.9, P < .01; 1 = Not all confident, 5 = Very confident).

CONCLUSIONS

Simulation-based mastery learning significantly improved anesthesiologists' ultrasound-guided intravenous catheter insertion performance in a simulated setting.

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy

Introduction

Emergency department thoracotomy (EDT) is a lifesaving procedure within the scope of practice of emergency physicians. Because EDT is infrequently performed, emergency medicine (EM) residents lack opportunities to develop procedural competency. There is no current mastery learning curriculum for residents to learn EDT. The purpose of this study was to develop and implement a simulation-based mastery learning curriculum to teach and assess EM residents’ performance of the EDT.

Methods

We developed an EDT curriculum using a mastery learning framework. The minimum passing standard (MPS) for a previously developed 22-item checklist was determined using the Mastery Angoff approach. EM residents at a four-year academic EM residency program underwent baseline testing in performing an EDT on a simulation trainer. Performance was scored by two raters using the checklist. Learners then participated in a novel mastery learning EDT curriculum that included an educational video, hands-on instruction, and deliberate practice. After a three-month period, residents then completed initial post testing. Residents who did not meet the minimum passing standard after post testing participated in additional deliberate practice until mastery was obtained. Baseline and post-test scores, and time to completion of the procedure were compared with paired t-tests.

Results

Of 56 eligible EM residents, 54 completed baseline testing. Fifty-two residents completed post-testing until mastery was reached. The minimum passing standard was 91.1%, (21/22 items correct on the checklist). No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS, with deliberate practice sessions not exceeding 40 minutes. Scores from baseline testing to post-testing significantly improved across all postgraduate years from a mean score of 10.2/22 to 21.4/22 (p <0.001). Mean time to complete the procedure improved from baseline testing (6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) to post-testing (5 min 19 seconds, interquartile range 4 min 17sec - 6 min 15 sec; p = 0.001).

Conclusion

This simulation-based mastery learning curriculum resulted in all residents performing an EDT at a level that met or exceeded the MPS with an overall decrease in time needed to perform the procedure.

Mastery Learning Ensures Correct Personal Protective Equipment Use in Simulated Clinical Encounters of COVID-19

INTRODUCTION

The correct use of personal protective equipment (PPE) limits transmission of serious communicable diseases to healthcare workers, which is critically important in the era of coronavirus disease 2019 (COVID-19). However, prior studies illustrated that healthcare workers frequently err during application and removal of PPE. The goal of this study was to determine whether a simulation-based, mastery learning intervention with deliberate practice improves correct use of PPE by physicians during a simulated clinical encounter with a COVID-19 patient.

METHODS

This was a pretest-posttest study performed in the emergency department at a large, academic tertiary care hospital between March 31-April 8, 2020. A total of 117 subjects participated, including 56 faculty members and 61 resident physicians. Prior to the intervention, all participants received institution-mandated education on PPE use via an online video and supplemental materials. Participants completed a pretest skills assessment using a 21-item checklist of steps to correctly don and doff PPE. Participants were expected to meet a minimum passing score (MPS) of 100%, determined by an expert panel using the Mastery Angoff and Patient Safety standard-setting techniques. Participants that met the MPS on pretest were exempt from the educational intervention. Testing occurred before and after an in-person demonstration of proper donning and doffing techniques and 20 minutes of deliberate practice. The primary outcome was a change in assessment scores of correct PPE use following our educational intervention. Secondary outcomes included differences in performance scores between faculty members and resident physicians, and differences in performance during donning vs doffing sequences.

RESULTS

All participants had a mean pretest score of 73.1% (95% confidence interval [CI], 70.9-75.3%). Faculty member and resident pretest scores were similar (75.1% vs 71.3%, p = 0.082). Mean pretest doffing scores were lower than donning scores across all participants (65.8% vs 82.8%, p<0.001). Participant scores increased 26.9% (95% CI of the difference 24.7-29.1%, p<0.001) following our educational intervention resulting in all participants meeting the MPS of 100%.

CONCLUSION

A mastery learning intervention with deliberate practice ensured the correct use of PPE by physician subjects in a simulated clinical encounter of a COVID-19 patient. Further study of translational outcomes is needed.

Improving cardiology fellow education of right heart catheterization using a simulation based curriculum

BACKGROUND

Medical procedures are traditionally taught informally at patients' bedside through observation and practice using the adage "see one, do one, teach one." This lack of formalized training can cause trainees to be unprepared to perform procedures independently. Simulation based education (SBE) increases competence, reduces complications, and decreases costs. We developed, implemented, and evaluated the efficacy of a right heart catheterization (RHC) SBE curriculum.

METHODS

The RHC curriculum consisted of a pretest, video didactics, deliberate practice, and a posttest. Pre-and posttest skills examinations consisted of a dichotomous 43-item checklist on RHC skills and a 14-item hemodynamic waveform quiz. We enrolled two groups of fellows: 6 first-year, novice cardiology fellows at Northwestern University in their first month of training, and 11 second- and third-year fellows who had completed traditional required, level I training in RHC. We trained the first-year fellows at the beginning of the 2018-2019 year using the SBE curriculum and compared them to the traditionally-trained cardiology fellows who did not complete SBE.

RESULTS

The SBE-trained fellows significantly improved RHC skills, hemodynamic knowledge, and confidence from pre- to posttesting. SBE-trained fellows performed similarly to traditionally-trained fellows on simulated RHC skills checklists (88.4% correct vs. 89.2%, p = .84), hemodynamic quizzes (94.0% correct vs. 86.4%, p = .12), and confidence (79.4 vs. 85.9 out of 100, p = .15) despite less clinical experience.

CONCLUSIONS

A SBE curriculum for RHC allowed novice cardiology fellows to achieve level I skills and knowledge at the beginning of fellowship and can train cardiology fellows before patient contact.

A Mastery Learning Module on Sterile Technique to Prepare Graduating Medical Students for Internship

INTRODUCTION

Sterile technique is a basic technical skill used for a number of bedside procedures. Proper use of sterile technique improves patient safety by reducing infection risk.

METHODS

We applied the principles of mastery learning to develop a simulation-based mastery learning module for sterile technique that was used as part of a 2-week internship preparatory course for fourth-year medical students. Forty-one medical students entering surgical or emergency medicine internships completed the module. Learners demonstrated baseline skills with a pretest, watched a didactic online video, participated in supervised deliberate practice sessions, and then completed a posttest. Physicians evaluated performance using a nine-item mastery checklist validated by a multispecialty panel of board-certified physicians. Learners who did not demonstrate mastery by correctly performing all nine checklist items received formative feedback and repeated the posttest as needed until mastery was achieved.

RESULTS

No learners demonstrated mastery of sterile technique during pretesting. A total of 100% of learners demonstrated mastery of sterile technique during either their first or second attempt of the posttest. The learners reported statistically significantly higher levels of confidence at the end of the module.

DISCUSSION

Our module highlights the skills gap that exists in the transition from undergraduate to graduate medical education and offers a cheap, effective, and easily reproducible curriculum for sterile technique that could be widely adopted for many learner populations.

Applying Educational Theory and Best Practices to Solve Common Challenges of Simulation-based Procedural Training in Emergency Medicine

OBJECTIVES

Procedural competency is an essential prerequisite for the independent practice of emergency medicine. Multiple studies demonstrate that simulation-based procedural training (SBPT) is an effective method for acquiring and maintaining procedural competency and preferred over traditional paradigms ("see one, do one, teach one"). Although newer paradigms informing SBPT have emerged, educators often face circumstances that challenge and undermine their implementation. The goal of this paper is to identify and report on best practices and theory-supported solutions to some of these challenges as derived using a process of expert consensus building and reviews of the existing literature on SBPT.

METHODS

The Society for Academic Emergency Medicine (SAEM) Simulation Academy SBPT Workgroup convened approximately 8 months prior to the 2019 SAEM Annual Meeting to perform a review of the literature and participate in a consensus-building process to identify solutions (in the form of best practices and educational theory) to these challenges faced by educators engaging in SBPT.

RESULTS AND ANALYSIS

Thirteen distinct educational challenges to SBPT emerged from the expert group's primary literature reviews and consensus-building processes. Three domains emerged upon further analysis of the 13 challenges: learner, educator, and curriculum. Six challenges within the "learner" domain were selected for comprehensive discussion in this paper, as they were deemed representative of the most common and most significant threats to ideal SBPT. Each of the six challenges aligns with one of the following themes: 1) maximizing active learning, 2) maintaining learner engagement, 3) embracing learner diversity, 4) optimizing cognitive load, 5) promoting mindfulness and reflection, and 6) emphasizing deliberate practice for mastery learning. Over 20 "special treatments" for mitigating the impact of the 13 challenges were derived from the secondary literature search and consensus-building process prior to and during the preconference workshop; 11 of these that best address the six learner-centered challenges are explored, including implications for educators involved in SBPT.

CONCLUSIONS/IMPLICATIONS FOR EDUCATORS

We propose multiple consensus-generated solutions (in the form of best practices and applied educational theory) that we believe are suitable and well aligned to overcome commonly encountered learner-centered challenges and threats to optimal SBPT.

Medical Decision-Making in the Physician Hierarchy: A Pilot Pedagogical Evaluation

PURPOSE

Recently, the American College of Graduate Medical Education included medical decision-making as a core competency in several specialties. To date, the ability to demonstrate and measure a pedagogical evolution of medical judgment in a medical education program has been limited. In this study, we aim to examine differences in medical decision-making of physician groups in distinctly different stages of their postgraduate career.

METHODS

The study recruited physicians with a wide spectrum of disciplines and levels of experience to take part in 4 medical simulations divided into 2 categories, abdominal pain (biliary colic [BC] and renal colic [RC]) or chest pain (cardiac ischemia with ST-segment elevation myocardial infarction [STEMI] and pneumothorax [PTX]). Evaluation of medical decision-making used the Medical Judgment Metric (MJM). The targeted selection criteria for the physician groups are administrative physicians (APs), representing those with the most experience but whose current duties are largely administrative; resident physicians (RPs), those enrolled in postgraduate medical or surgical training; and mastery level physicians (MPs), those deemed to have mastery level experience. The study measured participant demographics, physiological responses, medical judgment scores, and simulation time to case resolution. Outcome differences were analyzed using Fisher exact tests with post hoc Bonferroni-adjusted z tests and single-factor analysis of variance F tests with post hoc Tukey honestly significant difference, as appropriate. The significance threshold was set at P < .05. Effect sizes were determined and reported to inform future studies.

RESULTS

A total of n = 30 physicians were recruited for the study with n = 10 participants in each physician group. No significant differences were found in baseline demographics between groups. Analysis of simulations showed a significant (P = .002) interaction for total simulation time between groups RP: 6.2 minutes (±1.58); MP: 8.7 minutes (±2.46); and AP: 10.3 minutes (±2.78). The AP MJM scores, 12.3 (±2.66), for the RC simulation were significantly (P = .010) lower than the RP 14.7 (±1.15) and MP 14.7 (±1.15) MJM scores. Analysis of simulated patient outcomes showed that the AP group was significantly less likely to stabilize the participant in the RC simulation than MP and RP groups (P = .040). While not significant, all MJM scores for the AP group were lower in the BC, STEMI, and PTX simulations compared with the RP and MP groups.

CONCLUSIONS

Physicians in distinctly different stages of their respective postgraduate career differed in several domains when assessed through a consistent high-fidelity medical simulation program. Further studies are warranted to accurately assess pedagogical differences over the medical judgment lifespan of a physician.

Aiming Beyond Competent: The Application of the Taxonomy of Significant Learning to Medical Education

Issue: Competency-Based Medical Education (CBME) focuses on demonstrable outcomes, as well as upholding medical education's accountability to society. Despite calls for a robust, multifaceted approach to competency-based assessment (CBA), lingering critiques exist. These critiques include reductionism, reinforcement of an external locus of control within learners, an loss of focus on learner development. Both CBME and CBA may be strengthened if viewed through the lens of a complementary curriculum design framework that broadens the focus on the learner. Evidence: Researchers and physician organizations have articulated the need for medical practitioners trained to provide optimal care in the rapidly changing care environment. In the drive to ensure accountability to patients and society, CBME may overlook the duty of educators to foster the necessary intrinsic development of learners as holistic professionals. The focus of CBA on outcomes may reward memorization and rote performance but may fail to ensure the underlying comprehension or critical thinking necessary to adapt to the variability of real-life patient care. Learners focus on tasks chosen for assessment; thus, areas less easily assessed may be overlooked or deemed unimportant. Reinforcement for learner motivation becomes externalized in CBA, as opposed to being driven by the desire for self-improvement and self-actualization. A recently proposed framework that views learner development as a process-based improvement cycle, the "Master Adaptive Learner," may help remedy this issue. L. Dee Fink's Taxonomy of Significant Learning aims to create meaningful learning experiences in higher education. This taxonomy consists of six interwoven domains: (a) Learning How to Learn, (b) Foundational Knowledge, (c) Application, (d) Integration, (e) Human Dimension, and (f) Caring. Each domain encompasses a unique perspective on the learning process, and when collectively applied to curriculum design, significant learning occurs. This taxonomy has not been widely applied to medical education but may offer an important counterbalance to the outcomes-based focus of CBME. Implications: The outcomes-based focus of CBME is well suited for skill-based tasks, such as procedures, that are observable and measurable. However, other essential physician skills-such as critical thinking, reflection, empathy, and self-directed learning-are not easily assessed, and thus may receive little focus in an outcomes-based model. A holistic approach, such as the Taxonomy of Significant Learning, can counter the deficits of CBME and provide a balanced approach to education program design and assessment.

Consensus on technical procedures for simulation-based training in anaesthesiology: A Delphi-based general needs assessment

BACKGROUND

Anaesthesiologists are expected to master an increasing number of technical procedures. Simulation-based procedural training can supplement and, in some areas, replace the classical apprenticeship approach during patient care. However, simulation-based training is very resource-intensive and must be prioritised and optimised. Developing a curriculum for simulation-based procedural training should follow a systematic approach, eg the Six-Step Approach developed by Kern. The aim of this study was to conduct a national general needs assessment to identify and prioritise technical procedures for simulation-based training in anaesthesiology.

METHODS

A three-round Delphi process was completed with anaesthesiology key opinion leaders. In the first round, the participants suggested technical procedures relevant to simulation-based training. In the second round, a needs assessment formula was used to explore the procedures and produce a preliminary prioritised list. In the third round, participants evaluated the preliminary list by eliminating and re-prioritising the procedures.

RESULTS

All teaching departments in Denmark were represented with high response rates in all three rounds: 79%, 77%, and 75%, respectively. The Delphi process produced a prioritised list of 30 procedure groups suitable for simulation-based training from the initial 138 suggestions. Top-5 on the final list was cardiopulmonary resuscitation, direct- and video laryngoscopy, defibrillation, emergency cricothyrotomy, and fibreoptic intubation. The needs assessment formula predicted the final prioritisation to a great extent.

CONCLUSION

The Delphi process produced a prioritised list of 30 procedure groups that could serve as a guide in future curriculum development for the simulation-based training of technical procedures in anaesthesiology.

Common errors in temporary hemodialysis catheter insertion

Non-tunneled hemodialysis catheter (NTHC) insertion is an essential skill for nephrology practice and remains a requirement of training. However, improper insertion technique can increase the risk of potentially fatal infectious and mechanical complications. Evidence-based strategies can reduce the rates of such complications and should be integrated into practice and training. Ultrasound (US) guidance should routinely be used for NTHC insertion at the femoral and internal jugular sites (with avoidance of the subclavian site). Nephrologists should receive proper training in the use of US for line insertion. With respect to other aspects of the procedure, proper insertion technique readily prevents guidewire-induced arrhythmias. In addition, adherence to infection-control guidelines results in a sustainable reduction in bloodstream infections. All these aspects of NTHC insertion may be best taught and evaluated through a program that includes simulation-based mastery learning (SBML) training. As a separate issue, nephrologists (and intensivists) should be aware that a dysfunctional catheter should be replaced at a new site rather than being changed over a guidewire. This review of common errors related to NTHC insertion seeks to highlight evidence-based approaches to practice and training.

Iatrogenic Critical Care Procedure Complication Boot Camp: A Simulation-based Pilot Study

BACKGROUND

Traditional medical education strategies teach learners how to correctly perform procedures while neglecting to provide formal training on iatrogenic error management. Error management training (EMT) requires active exploration as well as explicit encouragement for learners to make and learn from errors during training. Simulation provides an excellent methodology to execute a curriculum on iatrogenic procedural complication management. We hypothesize that a standardized simulation-based EMT curriculum will improve learner's confidence, cognitive knowledge, and performance in iatrogenic injury management.

METHODS

This was a pilot, prospective, observational study performed in a simulation center using a curriculum developed to educate resident physicians on iatrogenic procedural complication management. Pre- and postintervention assessments included confidence surveys, cognitive questionnaires, and critical action checklists for six simulated procedure complications. Assessment data were analyzed using medians and interquartile ranges (IQRs), and the paired change scores were tested for median equality to zero via Wilcoxon signed rank tests with p < 0.05 considered statistically significant.

RESULTS

Eighteen residents participated in the study curriculum. The median (IQR) confidence increased significantly by a summed score of 12.5 (8.75-17.25; p < 0.001). Similarly, the median (IQR) knowledge significantly increased by 6 (3-8) points from the pre- to postintervention assessment (p < 0.001). For each of the simulation cases, the number of critical actions performed increased significantly (p < 0.001 to p = 0.002).

CONCLUSION

We demonstrated significant improvement in the confidence, clinical knowledge, and performance of critical actions after the completion of this curriculum. This pilot study provides evidence that a structured EMT curriculum is an effective method to teach management of iatrogenic injuries.

High-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students

BACKGROUND

Simulation has become integral to the training of both undergraduate medical students and medical professionals. Due to the increasing degree of realism and range of features, the latest mannequins are referred to as high-fidelity simulators. Whether increased realism leads to a general improvement in trainees' outcomes is currently controversial and there are few data on the effects of these simulators on participants' personal confidence and self-assessment.

METHODS

One-hundred-and-thirty-five fourth-year medical students were randomly allocated to participate in either a high- or a low-fidelity simulated Advanced Life Support training session. Theoretical knowledge and self-assessment pre- and post-tests were completed. Students' performance in simulated scenarios was recorded and rated by experts.

RESULTS

Participants in both groups showed a significant improvement in theoretical knowledge in the post-test as compared to the pre-test, without significant intergroup differences. Performance, as assessed by video analysis, was comparable between groups, but, unexpectedly, the low-fidelity group had significantly better results in several sub-items. Irrespective of the findings, participants of the high-fidelity group considered themselves to be advantaged, solely based on their group allocation, compared with those in the low-fidelity group, at both pre- and post-self-assessments. Self-rated confidence regarding their individual performance was also significantly overrated.

CONCLUSION

The use of high-fidelity simulation led to equal or even worse performance and growth in knowledge as compared to low-fidelity simulation, while also inducing undesirable effects such as overconfidence. Hence, in this study, it was not beneficial compared to low-fidelity, but rather proved to be an adverse learning tool.

Development and Utilization of 3D Printed Material for Thoracotomy Simulation

Medical simulation is a widely used training modality that is particularly useful for procedures that are technically difficult or rare. The use of simulations for educational purposes has increased dramatically over the years, with most emergency medicine (EM) programs primarily using mannequin-based simulations to teach medical students and residents. As an alternative to using mannequin, we built a 3D printed models for practicing invasive procedures. Repeated simulations may help further increase comfort levels in performing an emergency department (ED) thoracotomy in particular, and perhaps this can be extrapolated to all invasive procedures. Using this model, a simulation training conducted with EM residents at an inner city teaching hospital showed improved confidence. A total of 21 residents participated in each of the three surveys [(1) initially, (2) after watching the educational video, and (3) after participating in the simulation]. Their comfort levels increased from baseline after watching the educational video (9.5%). The comfort level further improved from baseline after performing the hands on simulation (71.4%).

Anatomic Site-Specific Complication Rates for Central Venous Catheter Insertions

BACKGROUND

Central venous catheter (CVC) complication rates reflecting the application of modern insertion techniques to a clinically heterogeneous patient populations are needed to better understand procedural risk attributable to the 3 common anatomic insertion sites: internal jugular, subclavian, and femoral veins. We sought to define site-specific mechanical and duration-associated CVC complication rates across all hospital inpatients.

METHODS

A retrospective chart review was conducted over 9 months at Georgetown University Hospital and Washington Hospital Center. Peripherally inserted central catheters and tunneled or fluoroscopically placed CVC's were excluded. Mechanical complications (retained guidewire, arterial injury, and pneumothorax) and duration-associated complications (deep vein thrombosis or pulmonary embolism, and central line-associated bloodstream infections) were identified.

RESULTS

In all, 1179 CVC insertions in 801 adult patients were analyzed. Approximately 32% of patients had multiple lines placed. Of 1179 CVCs, 73 total complications were recorded, giving a total rate of one or more complications occurring per CVC of 5.9%. There was no statistically significant difference between site-specific complications. A total of 19 mechanical complications were documented, with a 1.5% complication rate of one or more mechanical complications occurring. A total of 54 delayed complications were documented, with a 4.4% complication rate of 1 or more delayed complications occurring. There were no statistically significant differences between anatomic sites for either total mechanical or total delayed complications.

CONCLUSIONS

These results suggest that site-specific CVC complication rates may be less common than previously reported. These data further inform on the safety of modern CVC insertion techniques across all patient populations and clinical settings.

The effects of graduate competency-based education and mastery learning on patient care and return on investment: a narrative review of basic anesthetic procedures

BACKGROUND

Despite the widespread implementation of competency-based education, evidence of ensuing enhanced patient care and cost-benefit remains scarce. This narrative review uses the Kirkpatrick/Phillips model to investigate the patient-related and organizational effects of graduate competency-based medical education for five basic anesthetic procedures.

METHODS

The MEDLINE, ERIC, CINAHL, and Embase databases were searched for papers reporting results in Kirkpatrick/Phillips levels 3-5 from graduate competency-based education for five basic anesthetic procedures. A gray literature search was conducted by reference search in Google Scholar.

RESULTS

In all, 38 studies were included, predominantly concerning central venous catheterization. Three studies reported significant cost-effectiveness by reducing infection rates for central venous catheterization. Furthermore, the procedural competency, retention of skills and patient care as evaluated by fewer complications improved in 20 of the reported studies.

CONCLUSION

Evidence suggests that competency-based education with procedural central venous catheterization courses have positive effects on patient care and are both cost-effective. However, more rigorously controlled and reproducible studies are needed. Specifically, future studies could focus on organizational effects and the possibility of transferability to other medical specialties and the broader healthcare system.

Credentialing of Hospitalists in Ultrasound-Guided Bedside Procedures: A Position Statement of the Society of Hospital Medicine

Ultrasound guidance is used increasingly to perform the following 6 bedside procedures that are core competencies of hospitalists: abdominal paracentesis, arterial catheter placement, arthrocentesis, central venous catheter placement, lumbar puncture, and thoracentesis. Yet most hospitalists have not been certified to perform these procedures, whether using ultrasound guidance or not, by specialty boards or other institutions extramural to their own hospitals. Instead, hospital privileging committees often ask hospitalist group leaders to make ad hoc intramural certification assessments as part of credentialing. Given variation in training and experience, such assessments are not straightforward "sign offs." We thus convened a panel of experts to conduct a systematic review to provide recommendations for credentialing hospitalist physicians in ultrasound guidance of these 6 bedside procedures. Pathways for initial and ongoing credentialing are proposed. A guiding principle of both is that certification assessments for basic competence are best made through direct observation of performance on actual patients.

Central Line Proficiency Test Outcomes after Simulation Training versus Traditional Training to Competence

RATIONALE

Studies have shown the importance of simulation-based training on the outcomes of central venous catheter (CVC) insertion by trainees.

OBJECTIVES

To compare the performance of internal medicine trainees who underwent standardized simulation training of CVC insertion with that of internal medicine trainees who had traditional CVC training and were already deemed competent to perform the procedure during a proficiency evaluation using a training mannequin.

METHODS

Trainees who perform CVC insertion were enrolled in the institutional Central Line Workshop, which includes both an online and an experiential simulation component. The training is followed by a certification station proficiency assessment. Residents and fellows previously certified competent to perform CVC placement without supervision completed the online module, but they could opt out of the experiential component and proceed directly to the evaluation.

RESULTS

Forty-eight trainees participated in the study. Twenty-one (44%), 15 (31%), 6 (13%), 1 (2%), 2 (4%), and 3 (6%) were in postgraduate year 1 (PGY1), PGY2, PGY3, PGY4, PGY5, and PGY6, respectively. Twenty-nine completed the hands-on instruction, 28 (97%) of whom successfully passed the simulation-based assessment on their first attempt. Nineteen trainees previously credentialed to perform CVC placement without supervision opted out of the simulation-based experiential training. Of these, five (26%) failed in their first attempt (P = 0.02 vs. trainees who completed the simulation training).

CONCLUSIONS

Standardized simulation-based training can improve CVC insertion proficiency, even among trainees with previous experience sufficient to have been deemed competent in the procedure. Improved performance at simulation-based testing may translate to improved outcomes of CVC placement by trainees.

Growing Pains: a Simulation-Based Curriculum for Improving the Transition to Hematology/Oncology Fellowship

Trainee exposure to clinical oncology during residency training is heterogeneous and often modest. The steep learning curve upon entry into fellowship can result in undue stress for fellows and their patients. Simulation-based training has been shown to be superior to classical didactic approaches. We have introduced several innovative simulation-based workshops into the curriculum for the Johns Hopkins Hematology/Oncology Fellowship Training Program in order to address this unmet need. During the first months of training, fellows were engaged in activities emphasizing essential clinical and procedural skills. Specific workshops included the following: (1) chemotherapy writing, (2) cadaveric and simulation-based bone marrow biopsy and intrathecal chemotherapy administration, and (3) simulation-based communication skills training. All first-year fellows in our program participated in these exercises. Pre- and post-workshop surveys were administered to assess knowledge, attitudes, and behaviors; additional distant post-workshop evaluations were disseminated to assess the durability/impact of the curricula and for program evaluation. Overall, participating fellows indicated that the workshops improved patient care and comfort with procedures and patient-centered communication. Continued implementation of these workshops was recommended for program improvement. To the best of our knowledge, ours is amongst the first oncology fellowship training programs to systematically implement simulation-based curricula into our schema for fellowship training. We hypothesize that proactively introducing fellows to these high-yield activities will translate into improved patient care and reduced stress for trainees. Additional investigation into the long-term impact of such curricula remains an area of ongoing need.

Development, Validation, and Implementation of a Medical Judgment Metric

Background: Medical decision making is a critical, yet understudied, aspect of medical education. Aims: To develop the Medical Judgment Metric (MJM), a numerical rubric to quantify good decisions in practice in simulated environments; and to obtain initial preliminary evidence of reliability and validity of the tool. Methods: The individual MJM items, domains, and sections of the MJM were built based on existing standardized frameworks. Content validity was determined by a convenient sample of eight experts. The MJM instrument was pilot tested in four medical simulations with a team of three medical raters assessing 40 participants with four levels of medical experience and skill. Results: Raters were highly consistent in their MJM scores in each scenario (intraclass correlation coefficient 0.965 to 0.987) as well as their evaluation of the expected patient outcome (Fleiss’s Kappa 0.791 to 0.906). For each simulation scenario, average rater cut-scores significantly predicted expected loss of life or stabilization (Cohen’s Kappa 0.851 to 0.880). Discussion: The MJM demonstrated preliminary evidence of reliability and validity.

An institution-wide approach to submission, review, and funding of simulation-based curricula

This article describes the development, implementation, and modification of an institutional process to evaluate and fund graduate medical education simulation curricula. The goals of this activity were to (a) establish a standardized mechanism for proposal submission and evaluation, (b) identify simulation-based medical education (SBME) curricula that would benefit from mentored improvement before implementation, and (c) ensure that funding decisions were fair and defensible. Our intent was to develop a process that was grounded in sound educational principles, allowed for efficient administrative oversight, ensured approved courses were high quality, encouraged simulation education research and scholarship, and provided opportunities for medical specialties that had not previously used SBME to receive mentoring and faculty development.