Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit*:

Standardized, Interdepartmental, Simulation-Based Central Line Insertion Course Closes an Educational Gap and Improves Intern Comfort with the Procedure

Central line placement is a common procedure, routinely performed by junior residents in medical and surgical departments. Before this project, no standardized instructional course on the insertion of central lines existed at our institution, and few interns had received formal ultrasound training. Interns from five departments participated in a simulation-based central line insertion course. Intern familiarity with the procedure and with ultrasound, as well as their prior experience with line placement and their level of comfort, was assessed. Of the 99 interns in participating departments, 45 per cent had been trained as ofOctober 2015. Forty-one per cent were female. The majority (59.5%) had no prior formal ultrasound training, and 46.0 per cent had never placed a line as primary operator. Scores increased significantly, from a precourse score mean of 13.7 to a postcourse score mean of 16.1, P < 0.001. All three of the self-reported measures of comfort with ultrasound also improved significantly. All interns reported the course was “very much” helpful, and 100 per cent reported they felt “somewhat” or “much” more comfortable with the procedure after attendance. To our knowledge, this is the first hospital-wide, standardized, simulation-based central line insertion course in the United States. Preliminary results indicate overwhelming satisfaction with the course, better ultrasound preparedness, and improved comfort with central line insertion.

Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures

OBJECTIVES

Central venous catheter insertions may lead to preventable adverse events. Attending physicians' central venous catheter insertion skills are not assessed routinely. We aimed to compare attending physicians' simulated central venous catheterinsertion performance to published competency standards.

DESIGN

Prospective cohort study of attending physicians' simulated internal jugular and subclavian central venous catheter insertion skills versus a historical comparison group of residents who participated in simulation training.

SETTING

Fifty-eight Veterans Affairs Medical Centers from February 2014 to December 2014 during a 2-day simulation-based education curriculum and two academic medical centers in Chicago.

SUBJECTS

A total of 108 experienced attending physicians and 143 internal medicine and emergency medicine residents.

INTERVENTION

None.

MEASUREMENTS AND MAIN RESULTS

Using a previously published central venous catheter insertion skills checklist, we compared Veterans Affairs Medical Centers attending physicians' simulated central venous catheter insertion performance to the same simulated performance by internal medicine and emergency medicine residents from two academic centers. Attending physician performance was compared to residents' baseline and posttest (after simulation training) performance. Minimum passing scores were set previously by an expert panel. Attending physicians performed higher on the internal jugular (median, 75.86% items correct; interquartile range, 68.97-86.21) and subclavian (median, 83.00%; interquartile range, 59.00-86.21) assessments compared to residents' internal jugular (median, 37.04% items correct; interquartile range, 22.22-68.97) and subclavian (median, 33.33%; interquartile range, 0.00-70.37; both p < 0.001) baseline assessments. Overall simulated performance was poor because only 12 of 67 attending physicians (17.9%) met or exceeded the minimum passing score for internal jugular central venous catheter insertion and only 11 of 47 (23.4%) met or exceeded the minimum passing score for subclavian central venous catheter insertion. Resident posttest performance after simulation training was significantly higher than attending physician performance (internal jugular: median, 96%; interquartile range, 93.10-100.00; subclavian: median, 100%; interquartile range, 96.00-100.00; both p < 0.001).

CONCLUSIONS

This study demonstrates highly variable simulated central venous catheter insertion performance among a national cohort of experienced attending physicians. Hospitals, healthcare systems, and governing bodies should recognize that even experienced physicians require periodic clinical skill assessment and retraining.

Validation of a Low-cost Optic Nerve Sheath Ultrasound Phantom: An Educational Tool

Objective

To validate an ocular phantom as a realistic educational tool utilizing in vivo and phantom optic nerve sheath (ONS) images obtained by ultrasound.

Methods

This prospective study enrolled 51 resident physicians from the Denver Health Residency in Emergency Medicine (EM) and 10 ultrasound fellowship-trained EM attending physicians. Participants performed optic nerve sheath diameter (ONSD) measurements on five in vivo and five phantom ocular ultrasound images and rated the realism of each image on a 5-point Likert scale. Chi-square analysis was performed to evaluate the subjective “realness” of in vivo and phantom images.

Results

Sixty-one participants performed ONSD measurements. Mean Likert scale values were 3.43 (95% confidence interval: 3.31–3.55) for in vivo images and 3.41 (95% confidence interval: 3.28–3.54) for phantom images. There was no statistical difference in subjective “realness” between in vivo and phantom ONSD ultrasound images among EM residents. Ultrasound fellowship-trained EM attending physicians aptly differentiated between in vivo (p < 0.01) and phantom (p < 0.01) images, as compared with EM residents.

Conclusion

Our ocular phantom simulates in vivo posterior ocular anatomy. EM resident physicians found the phantom indistinguishable from in vivo images. Our ONS model provides an inexpensive and realistic educational tool to teach bedside ONSD sonography.

Simulation to Standardize Patient Care and Maintain Procedural Competency

Simulation-based training is a means to teach procedural skills and to help advanced practice providers maintain procedural competency and credentialing. There is growing recognition of the importance of requiring providers to demonstrate competency of invasive procedures in a simulated environment prior to performing these high-risk procedures on patients. This article describes the development and implementation of the simulation procedural program at the University of Rochester Medical Center. In addition to contributing to the education of our providers, such a program can lead to improved patient quality, safety, and outcomes through the standardization of patient care. The innovative use of simulation can lead to effective heath care education and improvement in patient safety.

Screening residents for infant lumbar puncture readiness with just-in-time simulation-based assessments

Background

Determining when to entrust trainees to perform procedures is fundamental to patient safety and competency development.

Objective

To determine whether simulation-based readiness assessments of first year residents immediately prior to their first supervised infant lumbar punctures (LPs) are associated with success.

Methods

This prospective cohort study enrolled paediatric and other first year residents who perform LPs at 35 academic hospitals from 2012 to 2014. Within a standardised LP curriculum, a validated 4-point readiness assessment of first year residents was required immediately prior to their first supervised LP. A score ≥3 was required for residents to perform the LP. The proportion of successful LPs (<1000 red blood cells on first attempt) was determined. Process measures included success on any attempt, number of attempts, analgesia usage and use of the early stylet removal technique.

Results

We analysed 726 LPs reported from 1722 residents (42%). Of the 432 who underwent readiness assessments, 174 (40%, 95% CI 36% to 45%) successfully performed their first LP. Those who were not assessed succeeded in 103/294 (35%, 95% CI 30% to 41%) LPs. Assessed participants reported more frequent direct attending supervision of the LP (diff 16%; 95% CI 8% to 22%), greater use of topical analgesia (diff 6%; 95% CI 1% to 12%) and greater use of the early stylet removal technique (diff 11%; 95% CI 4% to 19%) but no difference in number of attempts or overall procedural success.

Conclusions

Simulation-based readiness assessments performed in a point-of-care fashion were associated with several desirable behaviours but were not associated with greater clinical success with LP.

Achieving Procedural Competence during Nephrology Fellowship Training: Current Requirements and Educational Research

Concerns have previously been raised as to whether training programs are ensuring that nephrology fellows achieve competence in the procedural skills required for independent practice. We sought to review the current requirements for procedural training as well as educational research pertaining to achieving competence in the core nephrology procedures of nontunneled (temporary) hemodialysis catheter insertion and percutaneous kidney biopsy. At this time, there is no universal approach to procedural training and assessment during nephrology fellowship. Nonetheless, simulation-based mastery learning programs have been shown to be effective in improving fellows' skills in nontunneled (temporary) hemodialysis catheter insertion and should be provided by all nephrology training programs. For percutaneous kidney biopsy, the development and evaluation of inexpensive simulators are a promising starting point for future study. Current practice with respect to procedural training during nephrology fellowship remains imperfect; however, the ongoing shift toward competency-based evaluation provides opportunities to refocus on improvement.

A Pediatric Cardiology Fellowship Boot Camp improves trainee confidence

Introduction New paediatric cardiology trainees are required to rapidly assimilate knowledge and gain clinical skills to which they have limited or no exposure during residency. The Pediatric Cardiology Fellowship Boot Camp (PCBC) at Boston Children's Hospital was designed to provide incoming fellows with an intensive exposure to congenital cardiac pathology and a broad overview of major areas of paediatric cardiology practice.

METHODS

The PCBC curriculum was designed by core faculty in cardiac pathology, echocardiography, electrophysiology, interventional cardiology, exercise physiology, and cardiac intensive care. Individual faculty contributed learning objectives, which were refined by fellowship directors and used to build a programme of didactics, hands-on/simulation-based activities, and self-guided learning opportunities.

RESULTS

A total of 16 incoming fellows participated in the 4-week boot camp, with no concurrent clinical responsibilities, over 2 years. On the basis of pre- and post-PCBC surveys, 80% of trainees strongly agreed that they felt more prepared for clinical responsibilities, and a similar percentage felt that PCBC should be offered to future incoming fellows. Fellows showed significant increase in their confidence in all specific knowledge and skills related to the learning objectives. Fellows rated hands-on learning experiences and simulation-based exercises most highly.

CONCLUSIONS

We describe a novel 4-week-long boot camp designed to expose incoming paediatric cardiology fellows to the broad spectrum of knowledge and skills required for the practice of paediatric cardiology. The experience increased trainee confidence and sense of preparedness to begin fellowship-related responsibilities. Given that highly interactive activities were rated most highly, boot camps in paediatric cardiology should strongly emphasise these elements.

Neonatology faculty development using simulation

The goal of faculty development activities is to supply the public with knowledgeable, skilled, and competent physicians who are prepared for high performance in the dynamic and complex healthcare environment. Current faculty development programs lack evidence-based support and are not sufficient to meet the professional needs of practicing physicians. Simulation activities for faculty development offer an alternative to traditional, teacher-centric educational offerings. Grounded in adult learning theory, simulation is a learner-centric, interactive, efficient, and effective method to train busy professionals. Many of the faculty development needs of clinical neonatologists can be met by participating in simulation-based activities that focus on technical skills, teamwork, leadership, communication, and patient safety.

Mastery learning improves students skills in inserting intravenous access: a pre-post-study

OBJECTIVE

Inserting peripheral venous catheters (PVCs) has been identified as a core competency for medical students. Because the performance - even of hygienic standards - of both students and novice physicians is frequently inadequate, medical faculties must focus on competence-based learning objectives and deliberate practice, features that are combined in mastery learning. Our aim was to determine the competency of students in inserting PVCs before and after an educational intervention.

DESIGN

This study comprised a skills assessment with pre- and post-tests of a group of third-year students who received a simulation-based intervention. A newly established curriculum involved one hour of practice at inserting PVCs on simulators. Students were required to pass a test (total 21 points, pass mark 20 points) developed on the concept of mastery learning. An unannounced follow-up test was performed one week (8 days) after the intervention.

SETTING

The simulation center of the medical faculty in Muenster.

PARTICIPANTS

Third-year students who received the intervention.

RESULTS

One hundred and nine complete data sets were obtained from 133 students (82.5%). Most students (97.2%) passed the test after the intervention (mean score increase from 15.56 to 20.50, P<0.001). There was a significant decrease in students' performance after one week (8 days): only 74.5% of participants passed this retest (mean score reduction from 20.50 to 20.06, P<0.001).

CONCLUSION

Mastery learning is an effective form of teaching practical skills to medical students, allowing a thorough preparation for the challenges of daily clinical practice.

Making progress in the ethical treatment of medical trainees

There is an inherent conflict within clinician educators as we balance the roles of healthcare provider to patients in need of care with that of educator of learners in need of teaching. In this essay we use Beauchamp and Childress' principles of biomedical ethics as a framework to compare the relationship that clinician educators have with their patients and their learners, and suggest that while we typically apply ethical principles when addressing the needs of our patients, these principles are frequently lacking in our interactions with learners. This dichotomy reflects a person-by-situation interaction that may be partly explained by the expectations of the regulatory bodies that define how clinicians should interact with patients and how educators should interact with learners. The result is that we may fall short in applying respect for autonomy, beneficence/nonmaleficence, and justice when addressing the needs of our learners. Fortunately there are ways in which we can incorporate these ethical principles into our interactions with learners while still adhering to accreditation standards and institutional policy. These include flipped classrooms and simulated learning experiences, incorporating aspects of instructional design that have been shown to improve learning outcomes, providing additional resources to learners with greater needs, and organizing training curricula around entrustable professional activities. Although the consistent application of ethical principles with all learners during all learning experiences is likely unachievable, we can, and should, move towards more ethical treatment of our learners.

Advances in Pediatric Surgical Education: A Critical Appraisal of Two Consecutive Minimally Invasive Pediatric Surgery Training Courses

BACKGROUND

Mandates for improved patient safety and increasing work hour restrictions have resulted in changes in surgical education. Educational courses increasingly must meet those needs. We sought to determine the experience, skill level, and the impact of simulation-based education (SBE) on two cohorts of pediatric surgery trainees.

MATERIALS AND METHODS

After Institutional Review Board (IRB) exempt determination, a retrospective review was performed of evaluations for an annual advanced minimally invasive surgery (MIS) course over 2 consecutive years. The courses included didactic content and hands-on skills training. Simulation included neonatal/infant models for rigid bronchoscopy-airway foreign body retrieval, laparoscopic common bile duct exploration, and real tissue diaphragmatic hernia (DH), duodenal atresia (DA), pulmonary lobectomy, and tracheoesophageal fistula models. Categorical data were analyzed with chi-squared analyses with t-tests for continuous data.

RESULTS

Participants had limited prior advanced neonatal MIS experience, with 1.95 ± 2.84 and 1.16 ± 1.54 prior cases in the 2014 and 2015 cohorts, respectively. The 2015 cohort had significantly less previous experience in lobectomy (P = .04) and overall advanced MIS (P = .007). Before both courses, a significant percentage of participants were not comfortable with DH repair (39%-42%), DA repair (50%-74%), lobectomy (34%-43%), and tracheoesophageal fistula repair (54%-81%). After course completion, > 60% of participants reported improvement in comfort with procedures and over 90% reported that the course significantly improved their perceived ability to perform each operation safely.

CONCLUSION

Pediatric surgery trainees continue to have limited exposure to advanced MIS during clinical training. SBE results in significant improvement in both cognitive knowledge and trainee comfort with safe operative techniques for advanced MIS.

The Past, Present, and Future of Simulation-based Education for Pediatric Emergency Medicine

The pediatric emergency medicine (PEM) environment is well suited for simulation-based activities, be they educational interventions for PEM learners, evaluations of the interface between health providers and the environment that they work in, or research investigations using simulation as a tool to answer specific clinical questions. As such, PEM has been among the leaders in the integration of this modality for clinical training. Traditionally, simulation has been used extensively for the dissemination of clinical training in the areas of clinical knowledge and its application, and the clinical, technical, and teamwork skills involved in PEM care. Increasingly, simulation is being used in novel applications, including breaking bad news, disclosure of error, family-centered care, quality and patient safety education, and system-level integration. The future will look to further identify, measure, and inform the integration of simulation with new and innovative adjuncts in the clinical environment, as well as to determine the optimal timing and use of simulation-based education to enhance the quality of care delivered to patients by the interprofessional and multidisciplinary team.

Development and initial evaluation of a training program for peripherally inserted central catheter (PICC) placement for radiology residents and technicians

OBJECTIVES

The goals of this study were to develop and evaluate a joint theoretical/practical training course for radiology residents and technicians and to start a collaborative practice agreement enabling radiology technicians to perform PICC placement under the responsibility of an interventional radiologist.

MATERIALS AND METHODS

A joint training session based on literature evidences and international recommendations was designed. Participants were assessed before and after training, and were also asked to evaluate the program one month after completion of the training course. Practical post-training mentoring guidelines were laid down for radiologists supervising technicians.

RESULTS

From January to April 2014, 6 radiology residents and 12 radiology technicians from the two interventional radiology departments of the University hospitals in Marseille took part in the training program. For both residents and technicians, significant improvement was observed between pretraining and post-training assessment. The majority of participants were satisfied with the program.

CONCLUSION

Our experience suggests that combined theoretical and practical training in PICC placement allows improving technical skill and yields high degrees of satisfaction for both radiology residents and technicians. A collaborative practice agreement is now formally established to enable radiologists to delegate PICC placement procedures to radiology technicians.

Development and Evaluation of a Learning Intervention Targeting First-Year Resident Defibrillation Skills

OBJECTIVES

The aim of the study was to evaluate an educational intervention targeting the acquisition and retention of critical core skills of defibrillation in first-year pediatric residents using simulation-based training and deliberate practice.

METHODS

From January 2011 to April 2012, a total of 23 first-year pediatric residents participated in a pretest-posttest study. An initial survey evaluated previous experience, training, and comfort. The scoring tool was designed and validated using a standard setting procedure and 60% was determined to be the minimum passing score. The 1-hour educational intervention included a brief video describing the defibrillator, 10 to 15 minutes of hands-on time with the defibrillator, and 30 minutes of simulation-based scenarios using deliberate practice with real-time feedback.

RESULTS

The number of subjects who achieved competency in defibrillation skills increased from 8 to 16 of 23 (35% vs 70%, P < 0.05), pretest versus posttest, with the posttest occurring 4 months after the intervention. There was a significant improvement in checklist score (53% vs 68%, P < 0.05) and time to defibrillation (282-189 s, P < 0.05). For those who initiated chest compressions, there was a nonsignificant improvement in time to compressions (50 vs 33 s, P = 0.08). Previous Pediatric Advanced Life Support training was not associated with performance on pretest or posttest.

CONCLUSIONS

This brief educational intervention was shown to be effective 4 months after instruction in achieving and retaining competency of defibrillation skills by first-year pediatric residents. In the process, we uncovered educational gaps in cardiopulmonary resuscitation and other resuscitation skills that need to be addressed in future educational interventions and training.

Assessment of central venous catheterization in a simulated model using a motion-tracking device: an experimental validation study

Background

Central venous catheterization (CVC) is a basic requirement for many medical specialties. Simulated training in CVC may allow the acquisition of this competency but few reports have established a valid methodology for learning and acquiring procedural skills for CVC. This study aims to validate the use of a tracking motion device, the imperial college surgical assessment device (ICSAD), by comparing it with validated global rating scales (GRS) to measure CVC performance in a simulated torso.

Methods

Senior year medical students, first and last year residents (PGY1, LYR), and expert anesthesiologists performed a jugular CVC assessment in a simulated model (Laerdal IV Torso). A validated GRS for objective assessment of technical skills and motion analysis by ICSAD was used. Statistical analysis was performed through Mann–Whitney and Kruskal–Wallis tests for construct validity and Spearman correlation coefficients between the ICSAD and GRS scores for concurrent validity between both.

Results

32 subjects were recruited (10 medical students, 8 PGY1, 8 LYR and 8 experts). Total path length measured with ICSAD and GRS scores were significantly different between all groups, except for LYR compared to experts (p = 0.664 for GRS and p = 0.72 for ICSAD). Regarding jugular CVC procedural time, LYR and experts were faster than PGY1 and MS (p < 0.05). Spearman correlation coefficient was −0.684 (p < 0.001) between ICSAD and GRS scores.

Conclusions

ICSAD is a valid tool for assessment of jugular CVC since it differentiates between expert and novice subjects, and correlates with a validated GRS for jugular CVC in a simulated torso.

Simulation for Skills-based Education in Pulmonary and Critical Care Medicine

The clinical practice of pulmonary and critical care medicine requires procedural competence in many technical domains, including vascular access, airway management, basic and advanced bronchoscopy, pleural procedures, and critical care ultrasonography. Simulation provides opportunities for standardized training and assessment in procedures without placing patients at undue risk. A growing body of literature supports the use and effectiveness of low-fidelity and high-fidelity simulators for procedural training and assessment. In this manuscript by the Skills-based Working Group of the American Thoracic Society Education Committee, we describe the background, available technology, and current evidence related to simulation-based skills training within pulmonary and critical care medicine. We outline working group recommendations for key procedural domains.

Teaching basic lung isolation skills on human anatomy simulator: attainment and retention of lung isolation skills

Background

Lung isolation skills, such as correct insertion of double lumen endobronchial tube and bronchial blocker, are essential in anesthesia training; however, how to teach novices these skills is underexplored. Our aims were to determine (1) if novices can be trained to a basic proficiency level of lung isolation skills, (2) whether video-didactic and simulation-based trainings are comparable in teaching lung isolation basic skills, and (3) whether novice learners’ lung isolation skills decay over time without practice.

Methods

First, five board certified anesthesiologist with experience of more than 100 successful lung isolations were tested on Human Airway Anatomy Simulator (HAAS) to establish Expert proficiency skill level. Thirty senior medical students, who were naive to bronchoscopy and lung isolation techniques (Novice) were randomized to video-didactic and simulation-based trainings to learn lung isolation skills. Before and after training, Novices’ performances were scored for correct placement using pass/fail scoring and a 5-point Global Rating Scale (GRS); and time of insertion was recorded. Fourteen novices were retested 2 months later to assess skill decay.

Results

Experts’ and novices’ double lumen endobronchial tube and bronchial blocker passing rates showed similar success rates after training (P >0.99). There were no differences between the video-didactic and simulation-based methods. Novices’ time of insertion decayed within 2 months without practice.

Conclusion

Novices could be trained to basic skill proficiency level of lung isolation. Video-didactic and simulation-based methods we utilized were found equally successful in training novices for lung isolation skills. Acquired skills partially decayed without practice.

Electronic supplementary material

The online version of this article (doi:10.1186/s12871-015-0169-7) contains supplementary material, which is available to authorized users.

An Educational Intervention to Improve Ultrasound Competency in Ultrasound-Guided Central Venous Access

OBJECTIVE

Ultrasound (US) guidance for central venous catheter (CVC) placement results in improved success and overall safety, but is a new skill for pediatric emergency medicine (PEM) physicians. No study to date has used simulation-based learning to evaluate the ability of PEM providers to perform US-guided CVC placement.Our objective was to assess the competency of physicians in a rarely performed procedure, US-guided CVC placement, before and after an educational intervention using simulation-based mastery learning.

METHODS

We performed a prospective cohort study evaluating change in PEM physician competency in US-guided CVC placement before and after an educational intervention. Subjects participated in a curriculum composed of 3 sessions: an intervention session, a 2-month follow-up session, and a 12-month follow-up session. At each session, subjects were observed using US to guide CVC placement on a simulation model and technical skill was scored using a validated direct-observation checklist. Competency was defined as successfully completing 7 critical items on the checklist.

RESULTS

Of the 28 PEM physicians participating, competency improved from 32% at preintervention to 93% at 2-month follow-up (difference, 62%; 95% confidence interval, 36%-84%). At 12-month follow-up, competency remained high (85%; difference, 53%; 95% confidence interval, 32%-75%).

CONCLUSIONS

Physician competency in US-guided CVC placement improved with a simulation-based educational intervention, and the effect was maintained over time. This study may serve as a model for outcomes-based education and certification in rarely performed procedures in pediatrics.

Development of a PROficiency-Based StePwise Endovascular Curricular Training (PROSPECT) Program

OBJECTIVES

Focus on patient safety, work-hour limitations, and cost-effective education is putting pressure to improve curricula to acquire minimally invasive techniques during surgical training. This study aimed to design a structured training program for endovascular skills and validate its assessment methods.

DESIGN

A PROficiency-based StePwise Endovascular Curricular Training (PROSPECT) program was developed, consisting of e-learning and hands-on simulation modules, focusing on iliac and superficial femoral artery atherosclerotic disease. Construct validity was investigated. Performances were assessed using multiple-choice questionnaires, valid simulation parameters, global rating scorings, and examiner checklists. Feasibility was assessed by passage of 2 final-year medical students through this PROSPECT program.

SETTING

Ghent University Hospital, a tertiary clinical care and academic center in Belgium with general surgery residency program.

PARTICIPANTS

Senior-year medical students were recruited at Ghent University Hospital. Vascular surgeons were invited to participate during conferences and meetings if they had performed at least 100 endovascular procedures as the primary operator during the last 2 years.

RESULTS

Overall, 29 medical students and 20 vascular surgeons participated. Vascular surgeons obtained higher multiple-choice questionnaire scores (median: 24.5-22.0 vs. 15.0-12.0; p < 0.001). Students took significantly longer to treat any iliac or femoral artery stenosis (3.3-14.8 vs. 5.8-30.1min; p = 0.001-0.04), whereas in more complex cases, fluoroscopy time was significantly higher in students (8.3 vs. 21.3min; p = 0.002; 7.3 vs. 13.1min; p = 0.03). In all cases, vascular surgeons scored higher on global rating scorings (51.0-42.0 vs. 29.5-18.0; p < 0.001) and examiner checklist (81.5-75.0 vs. 54.5-43.0; p < 0.001). Hence, proficiency levels based on median expert scores could be determined. There were 2 students who completed the program and passed for each step within a 3-month period during their internships.

CONCLUSIONS

A feasible and construct validated surgical program to train cognitive, technical, and nontechnical endovascular skills was developed. A structured, stepwise, proficiency-based valid endovascular program to train cognitive, technical, and human factor skills has been developed and proven to be feasible. A randomized controlled trial has been initiated to investigate its effect on performances in real life, patient outcomes, and cost-effectiveness.

The PrOSTE: identifying key components of effective procedural teaching

RATIONALE

Novel approaches for faculty development and assessment of procedural teaching skills are needed to improve the procedural education of trainees. The Objective Structured Teaching Exercise (OSTE) entails a simulated encounter in which faculty are observed teaching a standardized student and has been used to evaluate teaching skills. Use of an OSTE to assess the teaching of central venous catheterization has not been reported.

OBJECTIVES

The purpose of this study was to develop a procedural OSTE for subclavian central venous catheter (CVC) insertion and to determine specific aspects of procedural teaching associated with improved skills in novices.

METHODS

Critical care faculty/fellows taught a standardized student to insert a CVC in a simulator. We assessed the instructor's teaching skills using rating scales to generate a procedural teaching score. After this encounter, the instructor taught novice medical students to place CVCs in simulators. Novices then independently placed catheters in simulators and were evaluated by trained observers using a checklist. Generalized estimating equations were used to examine the correlation between specific teaching behaviors and the novices' skills in CVC placement.

MEASUREMENTS AND MAIN RESULTS

We recruited 10 participants to serve as teachers and 30 preclinical medical students to serve as novice learners. The overall mean procedural teaching score was 85.5 (±15.4). Improved student performance was directly related to the degree to which the teacher "provided positive feedback" (β = 1.53, SE = 0.44, P = 0.001), "offered learner suggestions for improvement" (β = 1.40, SE = 0.35, P < 0.001), and "demonstrated the procedure in a step-by-step manner" (β = 2.50, SE = 0.45, P < 0.001). There was no significant correlation between total scores and student skills (β = 0.06, SE = 0.46, P = 0.18).

CONCLUSIONS

The OSTE is a standardized method to assess procedural teaching skills. Our findings suggest that specific aspects of procedural teaching should be emphasized to ensure effective transfer of psychomotor skills to trainees.

Procedural instruction in invasive bedside procedures: a systematic review and meta-analysis of effective teaching approaches

IMPORTANCE

Optimal approaches to teaching bedside procedures are unknown.

OBJECTIVE

To identify effective instructional approaches in procedural training.

DATA SOURCES

We searched PubMed, EMBASE, Web of Science and Cochrane Library through December 2014.

STUDY SELECTION

We included research articles that addressed procedural training among physicians or physician trainees for 12 bedside procedures. Two independent reviewers screened 9312 citations and identified 344 articles for full-text review.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers extracted data from full-text articles.

MAIN OUTCOMES AND MEASURES

We included measurements as classified by translational science outcomes T1 (testing settings), T2 (patient care practices) and T3 (patient/public health outcomes). Due to incomplete reporting, we post hoc classified study outcomes as 'negative' or 'positive' based on statistical significance. We performed meta-analyses of outcomes on the subset of studies sharing similar outcomes.

RESULTS

We found 161 eligible studies (44 randomised controlled trials (RCTs), 34 non-RCTs and 83 uncontrolled trials). Simulation was the most frequently published educational mode (78%). Our post hoc classification showed that studies involving simulation, competency-based approaches and RCTs had higher frequencies of T2/T3 outcomes. Meta-analyses showed that simulation (risk ratio (RR) 1.54 vs 0.55 for studies with vs without simulation, p=0.013) and competency-based approaches (RR 3.17 vs 0.89, p<0.001) were effective forms of training.

CONCLUSIONS AND RELEVANCE

This systematic review of bedside procedural skills demonstrates that the current literature is heterogeneous and of varying quality and rigour. Evidence is strongest for the use of simulation and competency-based paradigms in teaching procedures, and these approaches should be the mainstay of programmes that train physicians to perform procedures. Further research should clarify differences among instructional methods (eg, forms of hands-on training) rather than among educational modes (eg, lecture vs simulation).

Dissemination of an innovative mastery learning curriculum grounded in implementation science principles: a case study

Dissemination of a medical education innovation, such as mastery learning, from a setting where it has been used successfully to a new and different medical education environment is not easy. This article describes the uneven yet successful dissemination of a simulation-based mastery learning (SBML) curriculum on central venous catheter (CVC) insertion for internal medicine and emergency medicine residents across medical education settings. The dissemination program was grounded in implementation science principles. The article begins by describing implementation science which addresses the mechanisms of medical education and health care delivery. The authors then present a mastery learning case study in two phases: (1) the development, implementation, and evaluation of the SBML CVC curriculum at a tertiary care academic medical center; and (2) the dissemination of the SBML CVC curriculum to an academic community hospital setting. Contextual information about the drivers and barriers that affected the SBML CVC curriculum dissemination is presented. This work demonstrates that dissemination of mastery learning curricula, like all other medical education innovations, will fail without active educational leadership, personal contacts, dedication, hard work, rigorous measurement, and attention to implementation science principles. The article concludes by presenting a set of lessons learned about disseminating an SBML CVC curriculum across different medical education settings.

Recommendations for Reporting Mastery Education Research in Medicine (ReMERM)

Guidelines for reporting several types of medical studies have been described in the literature. However, there are no current guidelines to report studies on mastery learning curriculum development and trainee evaluation in medical education. Such guidelines will be important because medical education is moving toward a competency-based model. The authors sought to define standards for the evaluation of mastery learning curricula using previously published guidelines in related fields and expert review.The authors reviewed previously published guidelines from clinical medicine, medical education, and the social sciences. Six authors with expertise in mastery learning curricula, performance assessment, and medical education compiled and reached agreement about a list of guidelines. The authors later circulated the list to 12 other experts and made revisions. A final list of guidelines was established and received group consensus. The Reporting Mastery Education Research in Medicine (ReMERM) guidelines have 22 categories with 38 items considered to be imperative for reporting a mastery learning research study. Details about each item, with a specific focus on those unique to mastery learning, are discussed.The ReMERM guidelines highlight the importance of developing rigorous curricula that embody reliable measures which yield valid decisions about achievement among medical learners. These guidelines should improve the quality of reporting and help educators, authors, peer reviewers, journal editors, and readers to better understand and evaluate mastery learning research. With this shift to competency-based medical education, the ReMERM guidelines should help meet medical educators' needs to achieve these new goals.

Evaluation of a Standardized Program for Training Practicing Anesthesiologists in Ultrasound-Guided Regional Anesthesia Skills

OBJECTIVES

Practicing anesthesiologists have generally not received formal training in ultrasound-guided perineural catheter insertion. We designed this study to determine the efficacy of a standardized teaching program in this population.

METHODS

Anesthesiologists in practice for 10 years or more were recruited and enrolled to participate in a 1-day program: lectures and live-model ultrasound scanning (morning) and faculty-led iterative practice and mannequin-based simulation (afternoon). Participants were assessed and recorded while performing ultrasound-guided perineural catheter insertion at baseline, at midday (interval), and after the program (final). Videos were scored by 2 blinded reviewers using a composite tool and global rating scale. Participants were surveyed every 3 months for 1 year to report the number of procedures, efficacy of teaching methods, and implementation obstacles.

RESULTS

Thirty-two participants were enrolled and completed the program; 31 of 32 (97%) completed the 1-year follow-up. Final scores [median (10th-90th percentiles)] were 21.5 (14.5-28.0) of 30 points compared to 14.0 (9.0-20.0) at interval (P < .001 versus final) and 12.0 (8.5-17.5) at baseline (P < .001 versus final), with no difference between interval and baseline. The global rating scale showed an identical pattern. Twelve of 26 participants without previous experience performed at least 1 perineural catheter insertion after training (P < .001). However, there were no differences in the monthly average number of procedures or complications after the course when compared to baseline.

CONCLUSIONS

Practicing anesthesiologists without previous training in ultrasound-guided regional anesthesia can acquire perineural catheter insertion skills after a 1-day standardized course, but changing clinical practice remains a challenge.

Missed Central Venous Guide Wires: A Systematic Analysis of Published Case Reports

OBJECTIVE

The inadvertent loss of an entire guide wire during central venous catheterization can lead to serious patient harm and require additional investigations as well as retrieval procedures. We aimed to analyze globally published reports of lost wires during central venous catheterization to understand its possible etiology, presentation, treatment, and outcomes with an objective of finding solutions to make the procedure safer.

DATA SOURCES

MEDLINE, Scopus, and CINAHL, supplemented by scanning the reference lists of relevant publications.

STUDY SELECTION

All reports describing an inadvertent intravascular loss of a complete guide wire during placement of central venous catheters published up to December 2014 were included. Reports exclusively describing the 1) retrieval method, 2) partially retained guide wires, and 3) entrapped guide wires during withdrawal were excluded.

DATA EXTRACTION

In each instance, we collected data about the method of the missed guide wire detection, the time interval between the procedure and detection, the supplementary investigations performed to confirm the diagnosis, and the risk factors associated with such events as well as the complications, the final outcome, and the wire retrieval methods used.

DATA SYNTHESIS

A systematic analysis of the accessed publications was performed.

CONCLUSIONS

Over the last decade, the number of reported instances of lost guide wires during central venous catheterization has increased rapidly. Unsupervised or improperly supervised insertions of the central catheters by trainees, distractions during insertions, and high workload are the main risk factors. A retained guide wire increases the risk and cost of additional diagnostic and therapeutic interventions, as well as imposing many minor-to-serious life-threatening complications. Continuing education along with simulator-based skill development, vigilant supervision, and a shared workload during out of hours working are likely to prevent such occurrences.

Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists

BACKGROUND AND OBJECTIVES

Concerns have been raised about nephrology fellows' skills in inserting temporary hemodialysis catheters. Less is known about temporary hemodialysis catheter insertion skills of attending nephrologists supervising these procedures. The aim of this study was to compare baseline temporary hemodialysis catheter insertion skills of attending nephrologists with the skills of nephrology fellows before and after a simulation-based mastery learning (SBML) intervention.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

This pre- post-intervention study with a pretest-only comparison group was conducted at the University of Toronto in September of 2014. Participants were nephrology fellows and attending nephrologists from three university-affiliated academic hospitals who underwent baseline assessment of internal jugular temporary hemodialysis catheter insertion skills using a central venous catheter simulator. Fellows subsequently completed an SBML intervention, including deliberate practice with the central venous catheter simulator. Fellows were expected to meet or exceed a minimum passing score at post-test. Fellows who did not meet the minimum passing score completed additional deliberate practice. Attending nephrologist and fellow baseline performance on the temporary hemodialysis catheter skills assessment was compared. Fellows' pre- and post-test temporary hemodialysis catheter insertion performance was compared to assess the effectiveness of SBML. The skills assessment was scored using a previously published 28-item checklist. The minimum passing score was set at 79% of checklist items correct.

RESULTS

In total, 19 attending nephrologists and 20 nephrology fellows participated in the study. Mean attending nephrologist checklist scores (46.1%; SD=29.5%) were similar to baseline scores of fellows (41.1% items correct; SD=21.4%; P=0.55). Only two of 19 attending nephrologists (11%) met the minimum passing score at baseline. After SBML, fellows' mean post-test score improved to 91.3% (SD=6.9%; P<0.001). Median time between pre- and post-test was 24 hours.

CONCLUSIONS

Attending nephrologists' baseline temporary hemodialysis catheter insertion skills were highly variable and similar to nephrology fellows' skills, with only a small minority able to competently insert a temporary hemodialysis catheter. SBML was extremely effective for training fellows and should be considered for attending nephrologists who supervise temporary hemodialysis catheter insertions.

Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial

OBJECTIVE

To study the effect of initial simulation-based transvaginal sonography (TVS) training compared with clinical training only, on the clinical performance of residents in obstetrics and gynecology (Ob-Gyn), assessed 2 months into their residency.

METHODS

In a randomized study, new Ob-Gyn residents (n = 33) with no prior ultrasound experience were recruited from three teaching hospitals. Participants were allocated to either simulation-based training followed by clinical training (intervention group; n = 18) or clinical training only (control group; n = 15). The simulation-based training was performed using a virtual-reality TVS simulator until an expert performance level was attained, and was followed by training on a pelvic mannequin. After 2 months of clinical training, one TVS examination was recorded for assessment of each resident's clinical performance (n = 26). Two ultrasound experts blinded to group allocation rated the scans using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale.

RESULTS

During the 2 months of clinical training, participants in the intervention and control groups completed an average ± SD of 58 ± 41 and 63 ± 47 scans, respectively (P = 0.67). In the subsequent clinical performance test, the intervention group achieved higher OSAUS scores than did the control group (mean score, 59.1% vs 37.6%, respectively; P < 0.001). A greater proportion of the intervention group passed a pre-established pass/fail level than did controls (85.7% vs 8.3%, respectively; P < 0.001).

CONCLUSION

Simulation-based ultrasound training leads to substantial improvement in clinical performance that is sustained after 2 months of clinical training. © 2015 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Cultivating Medical Education Research Mentorship as a Pathway Towards High Quality Medical Education Research

The lack of effective and consistent research mentorship and research mentor training in both undergraduate medical education (UME) and graduate medical education (GME) is a critical constraint on the development of innovative and high quality medical education research. Clinical research mentors are often not familiar with the nuances and context of conducting education research. Clinician-educators, meanwhile, often lack the skills in developing and conducting rigorous research. Mentors who are not prepared to articulate potential scholarship pathways for their mentees risk limiting the mentee's progress in early stages of their career. In fact, the relative paucity of experienced medical education research mentors arguably contributes to the perpetuation of a cycle leading to fewer well-trained researchers in medical education, a lack of high quality medical education research, and relative stagnation in medical education innovation. There is a path forward, however. Integration of doctoral-level educators, structured inter-departmental efforts, and external mentorship provide opportunities for faculty to gain traction in their medical education research efforts. An investment in medical education research mentors will ensure rigorous research for high quality innovation in medical education and patient care.

A Survey of the First-Hour Basic Care Tasks of Severe Sepsis and Septic Shock in Pediatric Patients and an Evaluation of Medical Simulation on Improving the Compliance of the Tasks

BACKGROUND

Application of the sepsis resuscitation bundle is limited by clinician knowledge, skills, and experience. We used the adjusted first-hour basic care tasks in pediatric patients in three tertiary hospitals in Shanghai, China.

OBJECTIVE

The aim of this study is to survey the compliance of the adjusted tasks and to evaluate in situ simulation team training on improving the compliance.

METHODS

A prospective observational study was performed with the survey checklists from May 2011 to January 2012 in three pediatric intensive care units. A simulated case scenario was administered to the practitioners in one hospital.

RESULTS

Seventy-three patients were enrolled, including 47 patients in one simulation hospital (SH) and 26 patients in two nonsimulation hospitals (NSH). The total compliance of the tasks was 47.9% (35/73). The compliance in the SH was significantly higher compared to that in the NSHs (61.7% [29/47] vs. 23.1% [6/26], p < 0.01). Compared to the SH, the main problems in the NSH were giving intravenous or intraosseous fluid resuscitation in a longer time (35.3 min vs. 19.9 min, p = 0.000), a smaller percentage of measurement of accurate urine output (38.5% vs. 68.1%, p = 0.027), delivering high-flow oxygen (73.1% vs. 93.6%, p = 0.028), and measurement of lactate (69.2% vs. 100%, p = 0.000).

CONCLUSIONS

In situ simulation team training is an effective method of teaching the tasks of septic shock care to clinicians and nurses on the front line and of improving the compliance of the tasks.

Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine

Acquisition of competency in procedural skills is a fundamental goal of medical training. In this Perspective, the authors propose an evidence-based pedagogical framework for procedural skill training. The framework was developed based on a review of the literature using a critical synthesis approach and builds on earlier models of procedural skill training in medicine. The authors begin by describing the fundamentals of procedural skill development. Then, a six-step pedagogical framework for procedural skills training is presented: Learn, See, Practice, Prove, Do, and Maintain. In this framework, procedural skill training begins with the learner acquiring requisite cognitive knowledge through didactic education (Learn) and observation of the procedure (See). The learner then progresses to the stage of psychomotor skill acquisition and is allowed to deliberately practice the procedure on a simulator (Practice). Simulation-based mastery learning is employed to allow the trainee to prove competency prior to performing the procedure on a patient (Prove). Once competency is demonstrated on a simulator, the trainee is allowed to perform the procedure on patients with direct supervision, until he or she can be entrusted to perform the procedure independently (Do). Maintenance of the skill is ensured through continued clinical practice, supplemented by simulation-based training as needed (Maintain). Evidence in support of each component of the framework is presented. Implementation of the proposed framework presents a paradigm shift in procedural skill training. However, the authors believe that adoption of the framework will improve procedural skill training and patient safety.

Targeting clinical outcomes: Endovascular simulation improves diagnostic coronary angiography skills

OBJECTIVE

The purpose of this study is to determine the effects of simulation-based medical education (SBME) on the skills required to perform coronary angiography in the cardiac catheterization laboratory.

BACKGROUND

Cardiovascular fellows commonly learn invasive procedures on patients. Because this approach is not standardized, it can result in inconsistent skill acquisition through exclusion of concepts and skills. Also, the learning curve varies between trainees yielding variability in skill acquisition. Therefore, coronary angiography skills are an excellent target for SBME in an environment in which direct patient care is not jeopardized.

METHODS

From January 2013 to June 2013, 14 cardiovascular fellows entering the cardiac catheterization laboratory at a tertiary care teaching hospital were tested on an endovascular simulator to assess baseline skills. All fellows subsequently underwent didactic teaching and preceptor-lead training on the endovascular simulator. Topics included basic catheterization skills and a review of catheterization laboratory systems. Following training, all fellows underwent a post-training assessment on the endovascular simulator. Paired t tests were used to compare items on the skills checklist and simulator defined variables.

RESULTS

Cardiovascular fellows scored significantly higher on a diagnostic coronary angiography skills checklist following SBME using an endovascular simulator. The mean pretest score was 66.6% (SD = 9.7%) compared to 86.0% (SD = 6.3%) following simulator training (P < 0.001). Additional findings include significant reduction in procedure time and use of cine-fluoroscopy at posttest.

CONCLUSIONS

SBME significantly improved cardiovascular fellows' performance of simulated coronary angiography skills. Standardized simulation-based education is a valuable adjunct to traditional clinical education for cardiovascular fellows.

Simulation for Maintenance of Certification

Maintenance of certification (MOC) is a process through which practitioners are able to show continuing competence in their areas of expertise. Simulation plays an increasingly important role in the assessment of students and residents, as well as in the initial practice certification for health care professionals. The use of simulation as an assessment tool in MOC has been sluggish to be universally accepted. This article discusses the role of simulation in health care education, how simulation might be effectively applied in the MOC process, and the future role of simulation in the MOC process.

Simulation of cardiac emergencies with real patients

BACKGROUND

Simulation training with manikin simulators for medical emergencies is increasingly used in medical training. The assessment of a manikin, in particular history and examination, is very different to that of a real patient. We sought to combine aspects of traditional simulation training with the assessment of real hospital in-patients.

STUDY DESIGN

In-patients who had recently experienced a cardiac emergency were asked to recall their symptoms as if they were still present. Medical students assessed these patients in the role of foundation year-1 (FY1) doctors, supervised by core medical trainee (CMT) doctors, and were encouraged to formulate a differential diagnosis and initial management plan. The students filled in a questionnaire prior to, immediately after and 1 week after each simulation session. This included a self-assessment of confidence in managing cardiac emergencies, as well as knowledge-based questions on aspects of assessment and management of cardiac emergencies. We sought to combine aspects of traditional simulation training with the assessment of real hospital in-patients

RESULTS

Confidence in managing cardiac emergencies was initially low, but significantly increased after one simulation training session (p < 0.001). This increase was sustained on re-assessment 1 week after the training session (p < 0.001). In addition to the increase in confidence, a significant and sustained increase in knowledge score was also observed (p < 0.001).

CONCLUSION

Simulation training with real patients led to an immediate and sustained increase in self-assessed confidence. There was also an increase in medical knowledge of the assessment and management of cardiac emergencies. This simulation technique is inexpensive, easily reproducible and can be used to complement learning from traditional simulation training with manikins.

Do you see what I see? A randomised pilot study to evaluate the effectiveness and efficiency of simulation-based training with videolaryngoscopy for neonatal intubation

Introduction

Direct laryngoscopy (DL) and airway intubation are critical for neonatal resuscitation. A challenge in teaching DL is that the instructor cannot assess the learners’ airway view. Videolaryngoscopy (VL), which allows display of a patient's airway on a monitor, enables the instructor to view the airway during the procedure. This pilot study compared deliberate practice using either VL with instruction (I-VL) or traditional DL. We hypothesised that I-VL would improve the efficiency and effectiveness of neonatal intubation (NI) training.

Methods

Participants (students, paediatric interns and neonatal fellows) were randomised to I-VL or DL. Baseline technical skills were assessed using a skills checklist and global skills assessment. Following educational sessions, deliberate practice was performed on mannequins using the Storz C-MAC. With I-VL, the instructor could guide training using a real-time airway monitor view. With DL, feedback was based solely on technique or direct visual confirmation, but the instructor and learner views were not concurrent. During summative assessment, procedural skills checklists were used to evaluate intubation ability on a neonatal airway trainer. The duration of attempts was recorded, and recorded airway views were blindly reviewed for airway grade. ‘Effectiveness’ reflected achievement of the minimum passing score (MPS). ‘Efficiency’ was the duration of training for learners achieving the MPS.

Results

58 learners were randomised. Baseline demographics were similar. All participants had a significant improvement in knowledge, skills and comfort/confidence following training. There were no significant differences between randomised groups in efficiency or effectiveness, but trends towards improvement in each were noted. Fellows were more likely to achieve ‘competency’ postinstruction compared to non-fellows (p<0.001).

Conclusions

This educational intervention to teach NI increased the learner's knowledge, technical skills and confidence in procedural performance in both groups. I-VL did not improve training effectiveness. The small sample size and participant diversity may have limited findings, and future work is indicated.