Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures

Attaining expertise in regional anaesthesia training using a multifactorial approach incorporating deliberate practice

The most effective way of delivering regional anaesthesia training and the best means of demonstrating competency have not been established. Clinical competency, based on the Dreyfus and Dreyfus lexicon, appears unachievable using current training approaches. Lessons should be taken from the worlds of music, chess, and sports. Modern skills training programmes should be built on an explicit and detailed understanding with measurement of a variety of factors such as perception, attention, psychomotor and visuospatial function, and kinesthetics, coupled with quantitative, accurate, and reliable measurement of performance.

Impact of Simulation-based Mastery Learning on Management of Massive Hemoptysis

Background

The management of massive hemoptysis is a high-risk, low-volume procedure that is associated with high mortality rates, and pulmonary and critical care medicine (PCCM) fellows often lack training. Simulation-based mastery learning (SBML) is an educational strategy that improves skill but has not been applied to massive hemoptysis management.

Objective

This pilot study aimed to develop a high-fidelity simulator, implement an SBML curriculum, and evaluate the impact on PCCM fellows managing massive hemoptysis.

Methods

We modified a simulator to bleed from segmental airways. Next, we developed an SBML curriculum and a validated 26-item checklist and set a minimum passing standard (MPS) to assess massive hemoptysis management. A cohort of traditionally trained providers was assessed using the checklist. First-year PCCM fellows reviewed a lecture before a pretest on the simulator using the skills checklist and underwent rapid-cycle deliberate practice with feedback. Subsequently, fellows were posttested on the simulator, with additional training as necessary until the MPS was met. We compared pretest and posttest performance and also compared SBML-trained fellows versus traditionally trained providers.

Results

The MPS on the checklist was set at 88%. All first-year PCCM fellows (N = 5) completed SBML training. Mean checklist scores for SBML participants improved from 67.7 ± 8.4% (standard deviation) at pretest to 84.6 ± 6.7% at the initial posttest and 92.3 ± 5.4% at the final (mastery) posttest. Traditionally trained participants had a mean test score of 60.6 ± 13.1%.

Conclusion

The creation and implementation of a massive hemoptysis simulator and SBML curriculum was feasible and may address gaps in massive hemoptysis management training.

Effect of Polypectomy Simulation-Based Mastery Learning on Skill Retention Among Practicing Endoscopists

PURPOSE

Practicing endoscopists frequently perform and teach screening colonoscopies and polypectomies, but there is no standardized method to train and assess physicians who perform polypectomy procedures. The authors created a polypectomy simulation-based mastery learning (SBML) curriculum and hypothesized that completion of the curriculum would lead to immediate improvement in polypectomy skills and skill retention at 6 and 12 months after training.

METHOD

The authors performed a pretest-posttest cohort study with endoscopists who completed SBML and were randomized to follow-up at 6 or 12 months from May 2021 to August 2022. Participants underwent SBML training, including a pretest, a video lecture, deliberate practice, and a posttest. All learners were required to meet or exceed a minimum passing standard on a 17-item skills checklist before completing training and were randomized to follow-up at 6 or 12 months. The authors compared simulated polypectomy skills performance on the checklist from pretest to posttest and posttest to 6- or 12-month follow-up test.

RESULTS

Twenty-four of 30 eligible participants (80.0%) completed the SBML intervention, and 20 of 24 (83.3%) completed follow-up testing. The minimum passing standard was set at 93% of checklist items correct. The pretest passing rate was 4 of 24 participants (16.7%) compared with 24 of 24 participants (100%) at posttest ( P < .001). There were no significant differences in passing rates from posttest to combined 6- and 12-month posttest in which 18 of 20 participants (90.0%) passed.

CONCLUSIONS

Before training and despite years of clinical experience, practicing endoscopists demonstrated poor performance of polypectomy skills. SBML was an effective method for practicing endoscopists to acquire and maintain polypectomy skills during a 6- to 12-month period.

Powerful medical education improves health care quality and return on investment

INTRODUCTION

Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness.

METHODS

This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations.

RESULTS

PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment.

CONCLUSIONS

The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

Necessity of Pretests in Central Venous Catheter Insertion Simulation-Based Mastery Learning: A Randomized Controlled Trial

PURPOSE

Simulation-based mastery learning (SBML) is a rigorous form of competency-based learning. Components of SBML include a pretest, deliberate practice, and a posttest; all learners must meet or exceed a minimum passing standard (MPS) on the posttest before completing training. The authors aimed to explore whether a modified SBML curriculum (without a pretest assessment) was as effective as the standard SBML curriculum (with a pretest assessment).

METHOD

The authors performed a randomized controlled trial of internal medicine residents who participated in an internal jugular central venous catheter insertion SBML curriculum at a tertiary care academic medical center in Chicago, Illinois, from December 2018 through December 2021. Residents were randomly assigned to complete the usual SBML intervention (pretest group) or to complete a modified SBML intervention without a pretest (no pretest group). The authors compared initial posttest performance and training time between groups.

RESULTS

Eighty-nine of 120 eligible residents (74.1%) completed the study: 43 in the pretest group and 46 in the no pretest group. Median (IQR) initial posttest scores were not statistically different between the pretest group (96.6 [93.1-100]) and the no pretest group (96.6 [92.4-100]). However, all 43 residents (100%) in the pretest group reached the MPS at the initial posttest compared with 41 of the 46 (89%) in the no pretest group ( P = .06). Residents in the pretest group required 16.5 hours more faculty and learning time than the no pretest group.

CONCLUSIONS

More residents who completed a pretest reached the MPS at initial posttest. However, incorporating a pretest during the internal jugular central venous catheter SBML curriculum required substantially more learner and faculty time without clear performance benefits.

Clinical Experience Is Not a Proxy for Competence: Comparing Fellow and Medical Student Performance in a Breaking Bad News Simulation-Based Mastery Learning Curriculum

BACKGROUND

It is unknown whether traditional medical education ensures competence among fellows in the key skill of breaking bad news (BBN). While simulation-based mastery learning (SBML) has been used to train fourth-year medical students (M4s) in BBN, it is unclear if it adds similar value for fellows.

OBJECTIVE

We examined the effect of traditional medical training on BBN skills by comparing baseline fellow and M4 skills and confidence and assessed the impact of a BBN SBML curriculum for fellows.

METHODS

Fellows training in six programs at Northwestern University from November 2018 to May 2019 were eligible for inclusion. Fellows completed a BBN SBML curriculum including a pretest, individualized feedback using a previously published assessment tool, and ongoing deliberate practice until all achieved a minimum passing standard (MPS). The primary outcomes were checklist and scaled item scores on the assessment tool. Fellow performance was compared to a historical M4 cohort.

RESULTS

Twenty-eight of 38 eligible fellows completed the curriculum and were included for analysis. Fellows reported significantly more experience and confidence in BBN compared to M4s, yet their pre-training performance was significantly worse on checklist (57.1% vs 65.0%, P = .02) and scaled items; only 4% reached the MPS. After training, fellow performance significantly improved on checklist (57.1% to 92.6%, SD = 5.2%, P < .001) and scaled items; all reached the MPS.

CONCLUSIONS

Despite higher confidence and BBN clinical experience, fellows performed worse than untrained M4s, confirming that experience is not a proxy for skill. Programs must develop competency-based assessments to ensure entrustment of communication skills.

Simulation-based Mastery Learning Improves Critical Care Skills of Advanced Practice Providers

Background

Advanced practice providers (APPs) are essential members of intensive care unit (ICU) interprofessional teams and are expected to be competent in performing procedures. There are no published criteria for establishing when APPs can independently perform procedures. Simulation-based mastery learning (SBML) is an effective strategy for improving critical care skills but has not been applied to practicing ICU APPs.

Objective

The purpose of this study was to evaluate if an SBML curriculum could improve the critical care skills and procedural self-confidence of ICU APPs.

Methods

We performed a pretest-posttest study of central venous catheter (CVC) insertion, thoracentesis, and mechanical ventilation (MV) management skills among ICU APPs who participated in an SBML course at an academic hospital. For each skill, APPs underwent baseline skills assessments (pretests) on a simulator using previously published checklists, followed by didactic sessions and deliberate practice with individualized feedback. Within 2 weeks, participants were required to meet or exceed previously established minimum passing standards (MPS) on simulated skills assessments (posttests) using the same checklists. Further deliberate practice was provided for those unable to meet the MPS until they retested and met this standard. We compared pretest to posttest skills checklist scores and procedural confidence.

Results

All 12 eligible ICU APPs participated in internal jugular CVC, subclavian CVC, and MV training. Five APPs participated in thoracentesis training. At baseline, no APPs met the MPS on all skills. At training completion, all APPs achieved the mastery standard. Internal jugular CVC pretest performance improved from a mean of 67.2% (standard deviation [SD], 28.8%) items correct to 97.1% (SD, 3.8%) at posttest (P = 0.005). Subclavian CVC pretest performance improved from 29.2% (SD, 32.7%) items correct to 93.1% (SD 3.9%) at posttest (P < 0.001). Thoracentesis pretest skill improved from 63.9% (SD, 30.6%) items correct to 99.2% (SD, 1.7%) at posttest (P = 0.054). Pretest MV skills improved from 54.8% (SD, 19.7%) items correct to 92.3% (SD, 5.0%) at posttest (P < 0.001). APP procedural confidence improved for each skill from pre to posttest.

Conclusion

SBML is effective for training APPs to perform ICU skills. Relying on traditional educational methods does not reliably ensure that APPs are adequately prepared to perform skills such as CVC insertion, thoracentesis, and MV management.

Will Any Road Get You There? Examining Warranted and Unwarranted Variation in Medical Education

Undergraduate and graduate medical education have long embraced uniqueness and variability in curricular and assessment approaches. Some of this variability is justified (warranted or necessary variation), but a substantial portion represents unwarranted variation. A primary tenet of outcomes-based medical education is ensuring that all learners acquire essential competencies to be publicly accountable to meet societal needs. Unwarranted variation in curricular and assessment practices contributes to suboptimal and variable educational outcomes and, by extension, risks graduates delivering suboptimal health care quality. Medical education can use lessons from the decades of study on unwarranted variation in health care as part of efforts to continuously improve the quality of training programs. To accomplish this, medical educators will first need to recognize the difference between warranted and unwarranted variation in both clinical care and educational practices. Addressing unwarranted variation will require cooperation and collaboration between multiple levels of the health care and educational systems using a quality improvement mindset. These efforts at improvement should acknowledge that some aspects of variability are not scientifically informed and do not support desired outcomes or societal needs. This perspective examines the correlates of unwarranted variation of clinical care in medical education and the need to address the interdependency of unwarranted variation occurring between clinical and educational practices. The authors explore the challenges of variation across multiple levels: community, institution, program, and individual faculty members. The article concludes with recommendations to improve medical education by embracing the principles of continuous quality improvement to reduce the harmful effect of unwarranted variation.

Treatment of status epilepticus in pediatrics: curriculum learning combined with in-situ simulations

BACKGROUND

Appropriate and timely treatment of status epilepticus (SE) reduces morbidity and mortality. Therefore, skill-based identification and management are critical for emergency physicians.

PURPOSE

To assess whether the ability of training physicians, residents, nurses, and others to respond to SE as a team could be improved by using curriculum learning [Strategies and Tools to Enhance Performance and Patient Safety of Team (TeamSTEPPS) course training] combined with in-situ simulations of emergency department (ED) staff.

APPROACH

A pre-training-post-training design was used on SE skills and teamwork skills. Emergency training, residents, and N1 and N2 nurses completed the SE skill and teamwork assessments (pre-training) through in-situ simulation. Next, the participating physicians and nurses attended the SE course [Strategies and Tools to Enhance Performance and Patient Safety of Team (TeamSTEPPS) course training], followed by conscious skill practice, including in-situ simulation drills every 20 days (eight times total) and deliberate practice in the simulator. The participants completed the SE skill and teamwork assessments (post-training) again in an in-situ simulation. Pre-training-post-training simulated SE skills and teamwork performance were assessed. The simulation training evaluation showed that the training process was reasonable, and the training medical staff had different degrees of benefit in increasing subject interest, improving operational skills, theoretical knowledge, and work self-confidence.

FINDINGS

Sixty doctors and nurses participated in the intervention. When comparing the SE skills of 10 regular training physicians pre-training and post-training, their performance improved from 40% (interquartile range (IQR): 0-1) before training to 100% (IQR: 80.00-100) after training (p < 0.001). The teamwork ability of the 10 teams improved from 2.43 ± 0.09 before training to 3.16 ± 0.08 after training (p < 0.001).

CONCLUSION

SE curriculum learning combined with in-situ simulation training provides the learners with SE identification and management knowledge in children and teamwork skills.

Development and evaluation of a simulation-based mastery learning maintenance of certification course

BACKGROUND/OBJECTIVES

To develop and evaluate a post-acute care simulation-based mastery learning (SBML) continuing medical education (CME)/maintenance of certification (MOC) procedure course.

DESIGN

Pretest-posttest study of the SBML intervention.

SETTING

A 2-day post-acute care procedures course.

PARTICIPANTS

Sixteen practicing clinicians (5 physicians,11 advanced practice providers). Participants engaged in a skills pretest on knee aspiration/injection, gastrostomy tube removal/replacement, tracheostomy tube exchange, and basic suturing using a checklist created for each procedure. Participants received a didactic on each procedure followed by deliberate practice with feedback. Using the same checklists, participants completed a skills posttest and were required to meet a minimum passing standard (MPS) to obtain CME/MOC credit.

MEASUREMENTS

The MPS for each skills checklist was determined by a multidisciplinary panel of 11 experts. Participants completed surveys on procedure self-confidence and a course evaluation.

RESULTS

There was statistically significant improvement between pre- and posttests for all four procedures (p < .001). All participants were able to meet or exceed the MPS for each skill during the 2-day course. Participants' self-confidence regarding each procedure improved significantly (p < .001).

CONCLUSION

An SBML training course granting CME/MOC credit for post-acute care providers significantly improves performance of knee aspiration/injection, gastrostomy tube removal/replacement, tracheostomy tube exchange, and basic suturing.

Simulation-Based Mastery Learning Improves Ultrasound-Guided Peripheral Intravenous Catheter Insertion Skills of Practicing Nurses

INTRODUCTION

Difficult intravenous (IV) access (DIVA) is frequently encountered in the hospital setting. Ultrasound-guided peripheral IV catheter (USGPIV) insertion has emerged as an effective procedure to establish access in patients with DIVA. Despite the increased use of USGPIV, little is known about the optimal training paradigms for bedside nurses. Therefore, we developed and evaluated a novel, sustainable, USGPIV simulation-based mastery learning (SBML) curriculum for nurses.

METHODS

This is a prospective cohort study of an USGPIV SBML training program for bedside nurses over a 12-month period. We evaluated skills and self-confidence before and after training and measured the proportion of the nurses achieving independent, proctor, and instructor status. Procedure logs and surveys were used to explore the nurse experience and utilization of USGPIV on real patients with DIVA 3 months after the intervention.

RESULTS

Two hundred thirty-eight nurses enrolled in the study. The USGPIV skill checklist scores increased from median of 6.0 [interquartile range = 4.0-9.0 (pretest) to 29.0, interquartile range = 28-30 (posttest), P < 0.001]. The USGPIV confidence improved from before (mean = 2.32, SD = 1.17) to after (mean = 3.85, SD = 0.73, P < 0.001) training (5-point Likert scale). Sixty-two percent of the nurses enrolled achieved independent status, 47.5% became proctors, and 11.3% course trainers. At 3-month posttraining, the nurses had attempted a mean of 35.6 USGPIV insertions with an 89.5% success rate.

CONCLUSIONS

This novel USGPIV SBML curriculum improves nurses' insertion skills, self-confidence, and progresses patient care through USGPIV insertions on hospitalized patients with DIVA.

Old Dog, New Trick: Efficacy of Self-Directed Procedural Training for Attending Critical Care Physicians

BACKGROUND

In teaching hospitals, the majority of central venous lines (CVL) are placed by trainees, resulting in little opportunity for attending critical care physicians to maintain this procedural skill. Additionally, not all attending critical care physicians have been trained in the most up-to-date method of dynamic ultrasound (US) guided CVL placement. Furthermore, there is no standardized method to assess procedural competency of attending critical care physicians or to train them in the evolving practice of CVL placement. Despite these limitations, attending critical care physicians are ultimately responsible for supervision of CVL placement by trainees.

OBJECTIVE

To assess the utility of an instructional video to impact attending critical care physicians' competency and confidence in dynamic US guided CVL placement.

METHODS

A pre-post intervention study was conducted at an academic medical center. Attending critical care physicians were first asked to obtain CVL access on a gelatin model using US guidance. They then participated in the intervention, which consisted of watching a short instructional video demonstrating a method of dynamic US guided CVL placement. They were then asked to obtain access again, this time using the described method. All CVL placements were video recorded to assess competency in dynamic US guided CVL placement as well as the time required to obtain CVL access. Two blinded and independent reviewers evaluated each video with discrepancies resolved by a third reviewer. Participants were also surveyed pre and post intervention to assess their confidence in performing and supervising CVL placement.

RESULTS

A total of 21 attending critical care physicians were included. Pre-intervention, four used dynamic US guidance compared to 16 post-intervention (P < .001). Confidence in both CVL placement and supervision improved post-intervention (P = .03 each). Time required to obtain CVL access did not differ significantly pre and post intervention. The majority (20/21) believed there should be required competency testing for CVL placement.

CONCLUSIONS

Pre-intervention dynamic US guided CVL competency was poor in this sample of attending critical care physicians but improved significantly with an instructional video intervention. This study suggests there is a role for procedural competency testing among attending critical care physicians, and that significant improvement is achievable with relatively minimal instruction.

Psychometric Validation of Central Venous Catheter Insertion Mastery Learning Checklist Data and Decisions

INTRODUCTION

Resident physicians are expected to acquire competence at central venous catheter (CVC) insertion to a mastery standard. Valid competence decisions about resident physicians' CVC performance rely on reliable data and rigorous achievement standards. This study used data from 3 CVC simulation-based mastery learning studies involving internal medicine (IM) and emergency medicine (EM) residents to address 2 questions: What is the effectiveness of a CVC mastery learning education intervention? Are minimum passing standards (MPSs) set by faculty supported by item response theory (IRT) analyses?

METHODS

Pretraining and posttraining skills checklist data were drawn from 3 simulation-based mastery learning research reports about CVC internal jugular (IJ) and subclavian (SC) insertion skill acquisition. Residents were required to meet or exceed a posttest skills MPS. Generalized linear mixed effect models compared checklist performance from pre to postintervention. Minimum passing standards were determined by Angoff and Hofstee standard setting methods. Item response theory models were used for cut-score evaluation.

RESULTS

Internal medicine and EM residents improved significantly on every IJ and SC checklist item after mastery learning. Item response theory analyses support the IJ and SC MPSs.

CONCLUSIONS

Mastery learning is an effective education intervention to achieve clinical skill acquisition among IM and EM residents. Item response theory analyses reveal desirable measurement properties for the MPSs previously set by expert faculty panels. Item response theory analysis is useful for evaluating standards for mastery learning interventions. The CVC mastery learning curriculum, reliable outcome data, and high achievement standards together contribute to reaching valid decisions about the competence of resident physicians to perform the clinical procedure.

Barriers and Facilitators to Central Venous Catheter Insertion: A Qualitative Study

OBJECTIVES

The aims of the study were to identify perceived barriers and facilitators to central venous catheter (CVC) insertion among healthcare providers and to understand the extent to which an existing Simulation-Based Mastery Learning (SBML) program may address barriers and leverage facilitators.

METHODS

Providers participating in a CVC insertion SBML train-the-trainer program, in addition to intensive care unit nurse managers, were purposively sampled from Veterans Administration Medical Centers located in geographically diverse areas. We conducted semistructured interviews to assess perceptions of barriers and facilitators to CVC insertion. Deidentified transcripts were analyzed using a grounded theory approach and the constant comparative method. We subsequently mapped identified barriers and facilitators to our SBML curriculum to determine whether or not the curriculum addresses these factors.

RESULTS

We interviewed 28 providers at six Veterans Administration Medical Centers, identifying the following five overarching factors of perceived barriers to CVC insertion: (1) equipment, (2) personnel/staff, (3) setting or organizational context, (4) patient or provider, and (5) time-related barriers. Three overarching factors of facilitators emerged: (1) equipment, (2) personnel, and (3) setting or organizational context facilitators. The SBML curriculum seems to address most identified barriers, while leveraging many facilitators; building on the commonly identified facilitator of nursing staff contribution by expanding the curriculum to explicitly include nurse involvement could improve team efficiency and organizational culture of safety.

CONCLUSIONS

Many identified facilitators (e.g., ability to use ultrasound, personnel confidence/competence) were also identified as barriers. Evidence-based SBML programs have the potential to amplify these facilitators while addressing the barriers by providing an opportunity to practice and master CVC insertion skills.

Effectiveness of a simulation-based mastery learning to train clinicians on a novel cricothyrotomy procedure at an academic medical centre during a pandemic: a quasi-experimental cohort study

OBJECTIVES

To develop and evaluate a simulation-based mastery learning (SBML) curriculum for cricothyrotomy using wet towels to suppress aerosolisation during a pandemic.

DESIGN

Quasi-experimental, pre-post study.

SETTING

Tertiary care, academic medical centre in Chicago.

PARTICIPANTS

Ear, nose and throat and general surgery residents, fellows and attendings.

INTERVENTION

Cricothyroidotomy simulation-based mastery learning curriculum.

OUTCOMES MEASURE

Pretest to posttest simulated cricothyrotomy skills checklist performance.

RESULTS

37 of 41 eligible surgeons participated in the curriculum. Median pretest score was 72.5 (IQR 55.0-80.0) and 100.0 (IQR 98.8-100.0) for the posttest p<0.001. All participants scored at or above a minimum passing standard (93% checklist items correct) at posttest.

CONCLUSIONS

Using SBML is effective to quickly train clinicians to competently perform simulated cricothyrotomy during a pandemic.

Simulation-Based Education Enhances Patient Safety Behaviors During Central Venous Catheter Placement

OBJECTIVE

We describe the effect of simulation-based education on residents' adherence to protocols for and performance of central venous access.

METHODS

Internal medicine and emergency medicine residents underwent a central venous access course that included a lecture, video presentation, readings, and simulation demonstrations presented by faculty. Baseline data were collected before the course was initiated. After a skills session where they rehearsed their ultrasound-guided central venous access skills, residents were evaluated using a procedural checklist and written knowledge exam. Residents also completed questionnaires regarding confidence in performing ultrasound-guided central venous access and opinions about the training course.

RESULTS

Residents demonstrated significant improvement on the written knowledge exam (P < 0.0001) and Standard Protocol Checklist (P < 0.0001) after the training course. Training improved a number of patient safety elements, including adherence to sterile technique, transparent dressing, discarding sharps, and ordering postprocedure x-rays. However, a number of residents failed to wash their hands, prepare with chlorhexidine, drape the patient using a sterile technique, anesthetize the site, and perform a preprocedure time-out. Significant improvement in procedural skills was also noted for reduction in skin-to-vein time (P < 0.003) as well as a reduction in number of residents who punctured the carotid artery (P < 0.02).

CONCLUSIONS

Simulation-based education significantly improved residents' knowledge and procedural skills along with their confidence. Adherence to the protocol also improved. This study illustrates that simulation-based education can improve patient safety through training and protocols.

Mastery skill assessment in hepato-pancreato-biliary surgical ultrasound: It's a Matter of Entrustment

BACKGROUND

A formative hepato-pancreato-biliary (HPB) ultrasound (US) skills practicum is offered annually to graduating HPB fellows, using entrustment assessments for open (IOUS) and laparoscopic (LAPUS) US. It is hypothesized that validity evidence will support the use of these assessments to determine if graduating fellows are well prepared to perform HPB US independently.

METHODS

Expert faculty were surveyed to set Mastery Entrustment standards for fellow performance. Standards were applied to fellow performances during two annual US skills practicums.

RESULTS

11 faculty questionnaires were included. Mean Entrustment cut scores across all items were 4.9/5.0 and 4.8/5.0 and Global Entrustment cut scores were 5.0/5.0 and 4.8/5.0 for IOUS and LAPUS, respectively. 78.5% (29/37) fellows agreed to have their de-identified data evaluated. Mean fellow Entrustments (across all skills) were 4.1 (SD 0.6; 2.6-4.9) and 3.9 (SD 0.7; 2.7-5), while the Global Entrustments were 3.6 (SD 0.8; 2-5) and 3.5 (SD 1.0; 2-5) for IOUS and LAPUS, respectively.

CONCLUSIONS

Two cohorts of graduating HPB fellows are not meeting Mastery Standards for HPB US performance determined by a panel of expert faculty.

Mastery learning improves simulated central venous catheter insertion by emergency medicine teaching faculty

OBJECTIVES

Routine competency assessments of procedure skills, such as central venous catheter (CVC) insertion, do not occur beyond residency training. Evidence suggests variable, suboptimal attending physician procedure skills. Our study aimed to assess CVC insertion skill by academic emergency physicians, determine whether a simulation-based mastery learning (SBML) intervention improves performance and investigate for variables that predict competence.

METHODS

This is a pretest-posttest study that evaluated simulated CVC insertion by emergency medicine (EM) faculty physicians. We assessed 44 volunteer participants at a large academic medical center over a 1-month period using a published 29-item checklist. Our primary outcome was the difference in assessment score before and after a SBML intervention. A secondary analysis evaluated predictors of pretest performance.

RESULTS

A total of 44 subjects participated. Only four of 44 (9.1%) of subjects met a predefined minimum passing score on pretest. Mean assessment scores increased by 21.5% following the SBML intervention (95% confidence interval [CI] of the difference = 18.1% to 24.8%, p < 0.001). In a regression model, pretest scores increased by 10.8% (95% CI = 2.9 to 18.7%, p = 0.009) if subjects completed postgraduate training within 5 years. Frequency of CVC insertion did not predict performance, but 25 of 44 (56.8%) faculty members had no documented performance or supervision of a CVC insertion within 1 year of assessment.

CONCLUSIONS

SBML is a promising method to assess and improve CVC insertion performance by EM faculty physicians. Recent completion of postgraduate training was a significant predictor of CVC insertion performance. Our results require validation in larger cohorts of EM physicians across other academic institutions.

Vascular Access Device Care and Management: A Comprehensive Organizational Approach

The Infusion Nurses Society asserts that a comprehensive organizational approach to vascular access device (VAD) care and management is imperative to ensure safe and efficacious patient care. It is essential that each organization (1) develops policies and procedures to align VAD care and management with recognized standards of practice; (2) integrates unique aspects of organization-selected VAD care products into policies and procedures and establishes expectations for adherence to these organizational directives; (3) develops a framework for gathering and analyzing clinical data related to patient outcomes for VAD care and management; (4) utilizes quality outcome data to facilitate evidence-based best practices within the organization; and (5) evaluates and facilitates educational programming to validate clinician competency.

Comparison of Two Simulated Procedural Assessment Formats in Attending Emergency Physicians

Background

Emergency physicians must be proficient at inserting central venous catheters and performing lumbar punctures to provide life-saving therapies to critically ill patients. An assessment of procedural skill is rarely performed after an emergency physician has completed residency. Current board certification exams for emergency medicine focus only on verbal descriptions of procedures to assess skill. We compared two methods of procedural skill assessment, simulated task trainer and verbal description, to assess the range of skill in central venous catheter insertion and lumbar punctures of emergency attending physicians at a large, urban, academic tertiary care institution.

Methodology

This is a prospective cohort study of simulated internal jugular central venous catheter insertion and lumbar puncture skill by emergency attending physicians on a task trainer versus verbal description. A total of 17 attending emergency medicine physicians consented to participate in the study during a yearly procedural skills session. For each subject, two expert raters used previously published checklists to assess procedural skill and give a global rating score.

Results

More checklist items were performed correctly on the task trainer than on verbal assessment for central line (task trainer = 78.4% ± 8.32% and verbal = 68.26% ± 8.9%) and lumbar puncture (task trainer = 85.57% ± 7.6% and verbal = 73.53%4 ± 10.34%) procedures, both with significant differences (p < 0.001). Of the participants, 82% strongly preferred the task trainer format to the verbal description assessment format.

Conclusions

The higher scores on the simulated format compared to the current verbal format imply that a shift towards simulated procedural assessment techniques may benefit examinees. More work is needed to determine if objective checklist scores for practicing attending emergency physicians correlate with subjective expert assessments of their procedural skills.

See One, Do One, Forget One: Early Skill Decay After Paracentesis Training

INTRODUCTION

Internal medicine residents perform paracentesis, but programs lack standard methods for assessing competence or maintenance of competence and instead rely on number of procedures completed. This study describes differences in resident competence in paracentesis over time.

METHODS

From 2016 to 2017, internal medicine residents (n = 118) underwent paracentesis simulation training. Competence was assessed using the Paracentesis Competency Assessment Tool (PCAT), which combines a checklist, global scale, and entrustment score. The PCAT also delineates two categorical cut-point scores: the Minimum Passing Standard (MPS) and the Unsupervised Practice Standard (UPS). Residents were randomized to return to the simulation lab at 3 and 6 months (group A, n = 60) or only 6 months (group B, n = 58). At each session, faculty raters assessed resident performance. Data were analyzed to compare resident performance at each session compared with initial training scores, and performance between groups at 6 months.

RESULTS

After initial training, all residents met the MPS. The number achieving UPS did not differ between groups: group A = 24 (40%), group B = 20 (34.5%), p = 0.67. When group A was retested at 3 months, performance on each PCAT component significantly declined, as did the proportion of residents meeting the MPS and UPS. At the 6-month test, residents in group A performed significantly better than residents in group B, with 52 (89.7%) and 20 (34.5%) achieving the MPS and UPS, respectively, in group A compared with 25 (46.3%) and 2 (3.70%) in group B (p < .001 for both comparison).

DISCUSSION

Skill in paracentesis declines as early as 3 months after training. However, retraining may help interrupt skill decay. Only a small proportion of residents met the UPS 6 months after training. This suggests using the PCAT to objectively measure competence would reclassify residents from being permitted to perform paracentesis independently to needing further supervision.

Procedural Competency Among Hospitalists: A Literature Review and Future Considerations

BACKGROUND

As general internists practicing in the inpatient setting, hospitalists at many institutions are expected to perform invasive bedside procedures, as defined by professional standards. In reality, hospitalists are doing fewer procedures and increasingly are referring to specialists, which threatens their ability to maintain procedural skills. The discrepancy between expectations and reality, especially when hospitalists may be fully credentialed to perform procedures, poses significant risks to patients because of morbidity and mortality associated with complications, some of which derive from practitioner inexperience.

METHODS

We performed a structured search of the peer-reviewed literature to identify articles focused on hospitalists performing procedures.

RESULTS

Our synthesis of the literature characterizes contributors to hospitalists' procedural competency and discusses: (1) temporal trends for procedures performed by hospitalists and their associated referral patterns, (2) data comparing use and clinical outcomes of procedures performed by hospitalists compared with specialists, (3) the lack of nationwide standardization of hospitalist procedural training and credentialing, and (4) the role of medical procedure services, although limited in supportive evidence, in concentrating procedural skill and mitigating risk in the hands of a few well-trained hospitalists.

CONCLUSION

We conclude with recommendations for hospital medicine groups to ensure the safety of hospitalized patients undergoing bedside procedures.

Simulation-Based Assessments and Graduating Neurology Residents' Milestones: Status Epilepticus Milestones

BACKGROUND

The American Board of Psychiatry and Neurology and the Accreditation Council for Graduate Medical Education (ACGME) developed Milestones that provide a framework for residents' assessment. However, Milestones do not provide a description for how programs should perform assessments.

OBJECTIVES

We evaluated graduating residents' status epilepticus (SE) identification and management skills and how they correlate with ACGME Milestones reported for epilepsy and management/treatment by their program's clinical competency committee (CCC).

METHODS

We performed a cohort study of graduating neurology residents from 3 academic medical centers in Chicago in 2018. We evaluated residents' skills identifying and managing SE using a simulation-based assessment (26-item checklist). Simulation-based assessment scores were compared to experience (number of SE cases each resident reported identifying and managing during residency), self-confidence in identifying and managing these cases, and their end of residency Milestones assigned by a CCC based on end-of-rotation evaluations.

RESULTS

Sixteen of 21 (76%) eligible residents participated in the study. Average SE checklist score was 15.6 of 26 checklist items correct (60%, SD 12.2%). There were no significant correlations between resident checklist performance and experience or self-confidence. The average participant's level of Milestone for epilepsy and management/treatment was high at 4.3 of 5 (SD 0.4) and 4.4 of 5 (SD 0.4), respectively. There were no significant associations between checklist skills performance and level of Milestone assigned.

CONCLUSIONS

Simulated SE skills performance of graduating neurology residents was poor. Our study suggests that end-of-rotation evaluations alone are inadequate for assigning Milestones for high-stakes clinical skills such as identification and management of SE.

YouTube to Teach Central Lines, The Expert vs Learner Perspective

PURPOSE

Instructional videos of medical procedures can be a useful guide for learners, demonstrating proper and safe technique. Open publishing sites such as YouTube are readily accessible, however the content is not peer reviewed and quality of videos vary greatly. Our aim was to evaluate a learner's ability to interpret the quality of openly published content by comparing their rating of the most popular central line insertion videos on YouTube to expert evaluations.

METHOD

YouTube search results for "central line placement" sorted by views or relevance compiled a list of the four most common videos. A fifth gold standard video, published by the New England Journal, was included, however was not found in the top results. Eleven expert practitioners from varying medical specialties (Critical Care, Surgery, Anesthesia, & Emergency Medicine) evaluated the 5 videos, utilizing a 22-item Likert scaled questionnaire emphasizing: preparation, sterility, anatomy, technique, & complications. Videos were compared as a composite average of the individual items on the survey. The highest, lowest, and 3rd ranked videos were evaluated by 45 residents ("learners") in varying specialties (Internal Medicine, Emergency Medicine, Surgery, Anesthesia) and post graduate year (PGY). Learners assessed the videos using the same scale. A Welch T-test assessed statistical significance between the two groups. Subgroup analysis compared experts against different PGY and specialty cohorts.

RESULTS

The lowest scored video among the experts and learners was the most popular on YouTube, with 858,933 views at the time of inclusion. Though lowest in rank, this video was judged higher by learners than the experts (2.63/5 vs 2.18/5, P = 0.0029). The 3rd ranked video by experts with 249,746 views on YouTube, was also rated higher by learners (3.77/5 vs 3.45/5, P = 0.0084). The gold standard video by NEJM had 320,580 views and was rated highest by both the experts and learners (4.37/5 vs 4.28/5, P = 0.518). Subgroup analysis showed similar results with learners rating the videos overall better than experts, this was particularly true in the PGY-1 subgroup.

CONCLUSION

The most popular central line insertion video was the worst rated by both experts and learners. Learners rated all the videos better than the expert. YouTube videos demonstrating medical procedures including central line insertion should come from peer reviewed sources if they are to be incorporated into educational curriculum.

Short-term Retention of Patient and Caregiver Ventricular Assist Device Self-care Skills after Simulation-based Mastery Learning

Background

We developed a simulation-based mastery learning (SBML) curriculum that boosted self-care skills for patients with a ventricular assist device (VAD). In this study, we evaluated short-term skills retention.

Methods

We assessed skill retention among patients and caregivers who participated in VAD self-care SBML at a tertiary care center. We compared discharge skills tests (immediately after completing SBML) to 1- and 3-month follow-up tests to assess skill retention.

Results

Fifteen patients and 15 caregivers completed discharge and follow-up testing. Skills were largely retained at 1- and 3-months.

Conclusions

SBML results in short-term retention of VAD self-care skills.

Impact of Simulation-based Mastery Learning on Resident Skill Managing Mechanical Ventilators

Background: Caring for patients requiring mechanical ventilation is complex, and residents may lack adequate skill for managing these patients. Simulation-based mastery learning (SBML) is an educational model that trains clinicians to a high standard and can reduce complications. The mastery learning model has not been applied to ventilator management. Objective: The purpose of this study was to determine whether SBML, as compared with traditional training, is an effective strategy for teaching residents the skills necessary to manage patients requiring mechanical ventilation. Methods: We developed an SBML curriculum and a 47-item skills checklist to test ventilator management for patients with normal, restricted, and obstructed lung physiology. A minimum passing standard (MPS) on the checklist was set using the Mastery Angoff method. Residents rotating through the medical intensive care unit in Academic Year 2017-2018 were assigned to SBML or traditional training based on their medical intensive care unit team. The SBML group was pretested on a ventilator simulator using the skills checklist. They then received a 1.5-hour session (45 min didactic and 45 min deliberate practice on the simulator with feedback). At rotation completion, they were posttested on the simulator using the checklist until the MPS was met. Both SBML-trained and traditionally trained groups received teaching during daily bedside rounds and twice weekly didactic lectures. At rotation completion, traditionally trained residents were tested using the same skills checklist on the simulator. We compared pretest and posttest performance among SBML-trained residents and end of the rotation test performances between the SBML-trained and traditionally trained residents. Results: The MPS was set at 87% on the checklist. Fifty-seven residents were assigned to the SBML-trained group and 49 were assigned to the traditionally trained group. Mean checklist scores for SBML-trained residents improved from 51.4% (standard deviation [SD] = 17.5%) at pretest to 86.1% (SD = 7.6%) at initial posttest and 92.5% (SD = 3.7%) at final (mastery) posttest (both P < 0.001). Forty-two percent of residents required more than one attempt at the posttest to meet or exceed the MPS. At rotation completion, the traditionally trained residents had a mean test score of 60.9% (SD = 13.3%). Conclusion: SBML is an effective strategy to train residents on mechanical ventilator management. An SBML curriculum may augment traditional training methods to further equip residents to safely manage ventilated patients.

Use of a simulation-based mastery learning curriculum to improve ultrasound-guided vascular access skills of pediatric anesthesiologists

BACKGROUND

Pediatric vascular access is inherently challenging due to the small caliber of children's vessels. Ultrasound-guided intravenous catheter insertion has been shown to increase success rates and decrease time to cannulation in patients with difficult intravenous access. Although proficiency in ultrasound-guided intravenous catheter insertion is a critical skill in pediatric anesthesia, there are no published competency-based training curricula.

AIMS

The objective of this study was to evaluate the performance of pediatric anesthesiologists who participated in a novel ultrasound-guided intravenous catheter insertion simulation-based mastery learning curriculum.

METHODS

Pediatric anesthesia attendings, fellows, and rotating residents participated in the ultrasound-guided intravenous catheter insertion simulation-based mastery learning curriculum from August 2019 to February 2020. The 2-hour curriculum consisted of participants first undergoing a simulated skills pretest followed by watching a video on ultrasound-guided intravenous catheter insertion and deliberate practice on a simulator. Subsequently, all participants took a post-test and were required to meet or exceed a minimum passing standard. Those who were unable to meet the minimum passing standard participated in further practice until they could be retested and met this standard. We compared pre to post-test ultrasound-guided intravenous catheter insertion skills and self-confidence before and after participation in the curriculum.

RESULTS

Twenty-six pediatric anesthesia attendings, 12 fellows, and 38 residents participated in the curriculum. At pretest, 16/76 (21%) participants were able to meet or exceed the minimum passing standard. The median score on the pretest was 21/25 skills checklist items correct and improved to 24/25 at post-test (95% CI 3.0-4.0, P < .01). Self-confidence significantly improved after the course from an average of 3.2 before the course to a postcourse score of 3.9 (95% CI 0.5-0.9, P < .01; 1 = Not all confident, 5 = Very confident).

CONCLUSIONS

Simulation-based mastery learning significantly improved anesthesiologists' ultrasound-guided intravenous catheter insertion performance in a simulated setting.

Use of a simulation-based mastery learning curriculum for neurology residents to improve the identification and management of status epilepticus

BACKGROUND

Appropriate and timely treatment of status epilepticus (SE) decreases morbidity and mortality. Therefore, skill-based training in the identification and management of SE is crucial for clinicians.

OBJECTIVE

The objective of the study was to develop and evaluate the impact of a simulation-based mastery learning (SBML) curriculum to train neurology residents on the identification and management of SE.

METHODS

We used pretest-posttest design with a retention test on SE skills for this study. Neurology residents in the second postgraduate year (PGY-2) were eligible to participate in the SE SBML curriculum. Learners completed a baseline-simulated SE skills assessment (pretest) using a 26-item dichotomous skills checklist. Next, they participated in a didactic session about the identification and management of SE, followed by deliberate skills practice. Subsequently, participants completed another skills assessment (posttest) using the same 26-item checklist. All participants were required to meet or exceed a minimum passing standard (MPS) determined by a panel of 14 SE experts using the Mastery Angoff standard setting method. After meeting the MPS at posttest, participants were reassessed during an unannounced in situ simulation session on the medical wards. We compared pretest with posttest simulated SE skills performance and posttest with reassessment in situ performance.

RESULTS

The MPS was set at 88% (23/26) checklist items correct. Sixteen neurology residents participated in the intervention. Participant performance improved from a median of 44.23% (Interquartile range (IQR): 34.62-55.77) at pretest to 94.23% (IQR: 92.13-100) at the posttest after SBML (p < .001). There was no significant difference in scores between the posttest and in situ test up to 8 months later (94.23%; IQR: 92.31-100 vs. 92.31%; IQR: 88.46-96.15; p = .13).

CONCLUSIONS

Our SBML curriculum significantly improved residents' SE identification and management skills that were largely retained during an unannounced simulated encounter in the hospital setting.

Improving cardiology fellow education of right heart catheterization using a simulation based curriculum

BACKGROUND

Medical procedures are traditionally taught informally at patients' bedside through observation and practice using the adage "see one, do one, teach one." This lack of formalized training can cause trainees to be unprepared to perform procedures independently. Simulation based education (SBE) increases competence, reduces complications, and decreases costs. We developed, implemented, and evaluated the efficacy of a right heart catheterization (RHC) SBE curriculum.

METHODS

The RHC curriculum consisted of a pretest, video didactics, deliberate practice, and a posttest. Pre-and posttest skills examinations consisted of a dichotomous 43-item checklist on RHC skills and a 14-item hemodynamic waveform quiz. We enrolled two groups of fellows: 6 first-year, novice cardiology fellows at Northwestern University in their first month of training, and 11 second- and third-year fellows who had completed traditional required, level I training in RHC. We trained the first-year fellows at the beginning of the 2018-2019 year using the SBE curriculum and compared them to the traditionally-trained cardiology fellows who did not complete SBE.

RESULTS

The SBE-trained fellows significantly improved RHC skills, hemodynamic knowledge, and confidence from pre- to posttesting. SBE-trained fellows performed similarly to traditionally-trained fellows on simulated RHC skills checklists (88.4% correct vs. 89.2%, p = .84), hemodynamic quizzes (94.0% correct vs. 86.4%, p = .12), and confidence (79.4 vs. 85.9 out of 100, p = .15) despite less clinical experience.

CONCLUSIONS

A SBE curriculum for RHC allowed novice cardiology fellows to achieve level I skills and knowledge at the beginning of fellowship and can train cardiology fellows before patient contact.

Assessing Competence in Central Venous Catheter Placement by Pediatric Critical Care Fellows: A National Survey Study

OBJECTIVES

To describe the current approach to initial training, ongoing skill maintenance, and assessment of competence in central venous catheter placement by pediatric critical care medicine fellows, a subset of trainees in whom this skill is required.

DESIGN

Cross-sectional internet-based survey with deliberate sampling.

SETTING

United States pediatric critical care medicine fellowship programs.

SUBJECTS

Pediatric critical care medicine program directors of Accreditation Council for Graduate Medical Education-accredited fellowship programs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

A working group of the Education in Pediatric Intensive Care Investigators research collaborative conducted a national study to assess the degree of standardization of training and competence assessment of central venous catheter placement across pediatric critical care medicine fellowship programs. After piloting, the survey was sent to all program directors (n = 67) of Accreditation Council for Graduate Medical Education-accredited pediatric critical care medicine programs between July 2017 and September 2017. The response rate was 85% (57/67). Although 98% of programs provide formalized central venous catheter placement training for first-year fellows, only 42% of programs provide ongoing maintenance training as part of fellowship. Over half (55%) of programs use a global assessment tool and 33% use a checklist-based tool when evaluating fellow central venous catheter placement competence under direct supervision. Only two programs (4%) currently use an assessment tool previously published and validated by the Education in Pediatric Intensive Care group. A majority (82%) of responding program directors believe that a standardized approach to assessment of central venous catheter competency across programs is important.

CONCLUSIONS

Despite national mandates for skill competence by many accrediting bodies, no standardized system currently exists across programs for assessing central venous catheter placement. Most pediatric critical care medicine programs use a global assessment and decisions around the ability of a fellow to place a central venous catheter under indirect supervision are largely based upon subjective assessment of performance. Further investigation is needed to determine if this finding is consistent in other specialties/subspecialties, if utilization of standardized assessment methods can improve program directors' abilities to ensure trainee competence in central venous catheter insertion in the setting of variable training approaches, and if these findings are consistent with other procedures across critical care medicine training programs, adult and pediatric.

Physician Versus Nonphysician Instruction: Evaluating an Expert Curriculum-Competent Facilitator Model for Simulation-Based Central Venous Catheter Training

INTRODUCTION

Healthcare simulation supports educational opportunities while maintaining patient safety. To reduce costs and increase the availability of training, a randomized controlled study evaluated central venous catheter (CVC) insertion training in the simulation laboratory with nonphysician competent facilitators (NPCFs) as instructors.

METHOD

A group of learners naive to central line placement participated in a blended curriculum consisting of interactive online materials and simulation-based training. Learners were randomized to training with NPCFs or attending physician faculty. The primary outcome was simulated CVC insertion task performance, graded with a validated checklist by blinded physician reviewers. Learner knowledge and satisfaction were also evaluated. Analysis was conducted using noninferiority testing.

RESULTS

Eighty-five students, 11 attending physicians, and 7 NPCFs voluntarily participated. Noninferiority testing of the difference in CVC insertion performance between NPCF-trained learners versus physician-trained learners found no significant difference [rejecting the null hypothesis of inferiority using an 8% noninferiority margin (P < 0.01)]. In addition, there was no difference found between the 2 groups on pre/post knowledge scores, self-reported learner comfort, course satisfaction, or instructor satisfaction.

CONCLUSIONS

An introductory CVC curriculum can be taught to novice learners by carefully trained and supported NPCFs and achieve skill and knowledge outcomes similar to learners taught by physicians.

Patient, Caregiver, and Clinician Perceptions of Ventricular Assist Device Self-care Education Inform the Development of a Simulation-based Mastery Learning Curriculum

BACKGROUND

Patients who undergo ventricular assist device (VAD) implantation and their caregivers must rapidly learn a significant amount of self-care skills and knowledge.

OBJECTIVE

The aim of this study was to explore patient, caregiver, VAD coordinator, and physician perspectives and perceptions of existing VAD self-care training to inform development of a simulation-based mastery learning (SBML) curriculum to teach patients and caregivers VAD self-care skills and knowledge.

METHODS

We conducted semistructured, in-person interviews with patients with a VAD, their caregivers, VAD coordinators, and physicians (cardiac surgeons, an infectious disease physician, and advanced heart failure cardiologists). We used a 2-cycle team-based iterative inductive approach to coding and analysis.

RESULTS

We interviewed 16 patients, 12 caregivers, 7 VAD coordinators, and 11 physicians. Seven major themes were derived from the interviews including (1) identification of critical curricular content, (2) need for standardization and assessment, (3) training modalities, (4) benefits of repetition, (5) piercing it all together, (6) need for refresher training, and (7) provision of training before implant.

CONCLUSIONS

Findings from this study suggest that SBML is a natural fit for the high-risk tasks needed to save VAD self-care. The 7 unique training-related themes derived from the qualitative data informed the design and development of a VAD SBML self-care curriculum.

Simulation-Based Mastery Learning Improves Patient and Caregiver Ventricular Assist Device Self-Care Skills: A Randomized Pilot Trial

BACKGROUND

No recognized standards exist for teaching patients and their caregivers ventricular assist device (VAD) self-care skills. We compared the effectiveness of a VAD simulation-based mastery learning (SBML) self-care training curriculum with usual VAD self-care training.

METHODS AND RESULTS

VAD patients and their caregivers were randomized to SBML or usual training during their implant hospitalization. The SBML group completed a pretest on 3 VAD self-care skills (controller, power source, and dressing change), then viewed videos and participated in deliberate practice on a simulator. SBML participants took a posttest and were required to meet or exceed a minimum passing standard for each of the skills. The usual training group completed the existing institutional VAD self-care teaching protocol. Before hospital discharge, the SBML and usual training groups took the same 3 VAD self-care skills tests. We compared demographic and clinical information, self-confidence, total participant training time, and skills performance between groups. Forty participants completed the study in each group. There were no differences in demographic and clinical information, self-confidence, or training time between groups. More participants in the SBML group met the minimum passing standard compared with the usual training group for controller (37/40 [93%] versus 25/40 [63%]; P=0.001), power source (36/40 [90%] versus 9/40 [23%]; P<0.001), and dressing change skills (19/20 [95%] versus 0/20; P<0.001).

CONCLUSIONS

SBML provided superior VAD self-care skills learning outcomes compared with usual training. This study has important implications for patients due to the morbidity and mortality associated with improper VAD self-care.

CLINICAL TRIAL REGISTRATION

URL: http://www.clinicaltrials.gov. Unique identifier: NCT03073005.

The association between implementation of second-tier prevention practices and CLABSI incidence: A national survey

BACKGROUND

Prevention of central-line-associated bloodstream infection (CLABSI) represents a complex challenge for the teams involved in device insertion and maintenance. First-tier practices for CLABSI prevention are well established.

OBJECTIVE

We describe second-tier prevention practices in Israeli medical-surgical ICUs and assess their association with CLABSI rates.

METHODS

In June 2017, an online survey assessing infection prevention practices in general ICUs was sent to all Israeli acute-care hospitals. The survey comprised 14 prevention measures supplementary to the established measures that are standard of care for CLABSI prevention. These measures fall into 2 domains: technology and implementation. The association between the number of prevention measures and CLABSI rate during the first 6 months of 2017 was assessed using Spearman's correlation. We used negative binomial regression to calculate the incidence rate ratio (IRR) associated with the overall number of prevention measures and with each measure individually.

RESULTS

The CLABSI rates in 24 general ICUs varied between 0.0 and 17.0 per 1,000 central-line days. Greater use of preventive measures was associated with lower CLABSI rates (ρ, -0.70; P < .001). For each additional measure, the incidence of CLABSI decreased by 19% (IRR, 0.81; 95% CI, 0.73-0.89). Specific measures associated with lower rates were involvement of ward champions (IRR, 0.47; 95% CI, 0.31-0.71), auditing of insertions by infection control staff (IRR, 0.35; 95% CI, 0.19-0.64), and simulation-based training (IRR, 0.38; 95% CI, 0.22-0.64).

CONCLUSION

Implementation of second-tier preventive practices was protective against CLABSI. Use of more practices was correlated with lower rates.

Process-Oriented Feedback through Process Mining for Surgical Procedures in Medical Training: The Ultrasound-Guided Central Venous Catheter Placement Case

Developing high levels of competence in the execution of surgical procedures through training is a key factor for obtaining good clinical results in healthcare. To improve the effectiveness of the training, it is advisable to provide feedback to each student tailored to how the student has performed the procedure on each occasion. Current state-of-the-art feedback is based on Checklists and Global Rating Scales, which indicate whether all process steps have been carried out and the quality of each execution step. However, there is a process perspective that is not captured successfully by these instruments, e.g., steps performed, but in an undesired order, group of activities that are repeated an unnecessary number of times, or an excessive transition time between two consecutive steps. In this research, we propose a novel use of process mining techniques to effectively identify desired and undesired process patterns regarding rework, the order in which activities are performed, and time performance, in order to complement the tailored feedback for surgical procedures using a process perspective. The proposed approach was applied to analyze a real case of ultrasound-guided Central Venous Catheter placement training. It was quantitatively and qualitatively validated that the students who participated in the training program perceived the process-oriented feedback they received as favorable for their learning.

Common errors in temporary hemodialysis catheter insertion

Non-tunneled hemodialysis catheter (NTHC) insertion is an essential skill for nephrology practice and remains a requirement of training. However, improper insertion technique can increase the risk of potentially fatal infectious and mechanical complications. Evidence-based strategies can reduce the rates of such complications and should be integrated into practice and training. Ultrasound (US) guidance should routinely be used for NTHC insertion at the femoral and internal jugular sites (with avoidance of the subclavian site). Nephrologists should receive proper training in the use of US for line insertion. With respect to other aspects of the procedure, proper insertion technique readily prevents guidewire-induced arrhythmias. In addition, adherence to infection-control guidelines results in a sustainable reduction in bloodstream infections. All these aspects of NTHC insertion may be best taught and evaluated through a program that includes simulation-based mastery learning (SBML) training. As a separate issue, nephrologists (and intensivists) should be aware that a dysfunctional catheter should be replaced at a new site rather than being changed over a guidewire. This review of common errors related to NTHC insertion seeks to highlight evidence-based approaches to practice and training.

The Effect of Judge Selection on Standard Setting Using the Mastery Angoff Method during Development of a Ventricular Assist Device Self-Care Curriculum

Background

Patients and caregivers need to perform ventricular assist device (VAD) self-care safely to help prevent complications (e.g., infection). We developed a VAD self-care simulation-based mastery learning (SBML) curriculum. We determined optimal minimum passing scores (MPSs) and evaluated effects of judge selection.

Methods

A multidisciplinary team created a VAD self-care SBML curriculum including simulated skills and knowledge examinations. Patients, caregivers, VAD coordinators, and physicians were expert judges who determined MPSs using the Mastery Angoff method.

Results

MPSs for the skills and knowledge examinations were high (range = 94-99% and 97% correct), respectively. Judges closely agreed on MPSs.

Conclusions

Stakeholders set stringent MPSs for high-stakes VAD self-care.

Investigating the Effect of Simulator Functional Fidelity and Personalized Feedback on Central Venous Catheterization Training

OBJECTIVE

To compare the effect of simulator functional fidelity (manikin vs a Dynamic Haptic Robotic Trainer [DHRT]) and personalized feedback on surgical resident self-efficacy and self-ratings of performance during ultrasound-guided internal jugular central venous catheterization (IJ CVC) training. In addition, we seek to explore how self-ratings of performance compare to objective performance scores generated by the DHRT system.

DESIGN

Participants were randomly assigned to either manikin or DHRT IJ CVC training over a 6-month period. Self-efficacy surveys were distributed before and following training. Training consisted of a pretest, 22 practice IJ CVC needle insertion attempts, 2 full-line practice attempts, and a posttest. Participants provided self-ratings of performance for each needle insertion and were presented with feedback from either an upper level resident (manikin) or a personalized learning system (DHRT).

SETTING

A study was conducted from July 2016 to February 2017 through a surgical skills training program at Hershey Medical Center in Hershey, Pennsylvania.

PARTICIPANTS

Twenty-six first-year surgical residents were recruited for the study. Individuals were informed that IJ CVC training procedures would be consistent regardless of participation in the study and that participation was optional. All recruited residents opted to participate in the study.

RESULTS

Residents in both groups significantly improved their self-efficacy scores from pretest to posttest (p < 0.01). Residents in the manikin group consistently provided higher self-ratings of performance (p < 0.001). Residents in the DHRT group recorded more feedback on errors (228 instances) than the manikin group (144 instances). Self-ratings of performance on the DHRT system were able to significantly predict the objective score of the DHRT system (R2 = 0.223, p < 0.001).

CONCLUSION

Simulation training with the DHRT system and the personalized learning feedback can improve resident self-efficacy with IJ CVC procedures and provide sufficient feedback to allow residents to accurately assess their own performance.

A Comparison of Approaches for Mastery Learning Standard Setting

PURPOSE

Defensible minimum passing standards (MPSs) must be used to evaluate learner performance outcomes in health professions education. In this study, the authors compared the results of traditional Angoff and Hofstee standard-setting exercises with the Mastery Angoff and Patient-Safety approaches for central venous catheter (CVC) insertion skills examinations. The authors also evaluated how these standards affected the historical performance of residents who participated in a simulation-based mastery learning (SBML) curriculum for CVC insertion skills.

METHOD

In April and May 2015, 12 physicians with expertise in CVC insertion set MPSs for previously published internal jugular (IJ) and subclavian (SC) CVC insertion checklists using Angoff, Hofstee, Mastery Angoff, and Patient-Safety approaches. The resulting MPSs were compared using historical performance of internal medicine and emergency medicine residents who participated in CVC insertion SBML.

RESULTS

The MPSs were set as follows: Angoff: IJ 91% checklist items correct, SC 90%. Hofstee: IJ 88%, SC 90%. Mastery Angoff: IJ 98%, SC 98%. Patient-Safety: IJ 98%, SC 98%. Based on the historical performance of 143 residents assessed on IJ and SC insertion, applying the 98% MPS would result in additional practice and retesting of 55/123 residents (45%) who had previously passed the IJ examination and 36/130 residents (28%) who had passed the SC examination using the Angoff and Hofstee MPSs.

CONCLUSIONS

The Mastery Angoff and Patient-Safety standard-setting approaches resulted in higher CVC insertion SBML MPSs compared with traditional standard-setting methods. Further study should assess the impact of these more rigorous standards on patient outcomes.

Effect of Trainee Performance Data on Standard-Setting Judgments Using the Mastery Angoff Method

BACKGROUND

Mastery learning in health professions education requires learners to learn and undergo assessment until they demonstrate a high level of competence. Setting defensible standards is key to accurately assessing educational outcomes in mastery learning. The Mastery Angoff method was proposed recently to set a minimum passing standard (MPS) for mastery learning curricula. However, it is unknown whether prior knowledge of trainee performance affects judges' decisions about setting an MPS using the Mastery Angoff method.

OBJECTIVE

We sought to determine the effect of introducing baseline data about trainee performance on faculty judges' decisions about the Mastery Angoff MPS for a written examination.

METHODS

We developed a mastery learning curriculum to train internal medicine residents and cardiology fellows about the correct interpretation of inpatient telemetry monitoring. All learners were required to meet or exceed an MPS on a 35-item written examination at the end of training. The MPS was set in 2017 by judges who used the item-based Mastery Angoff method without prior examinee performance information. The judges subsequently reevaluated the test items after receiving baseline data about trainee performance collected during pilot testing. Mastery Angoff MPSs with and without baseline performance data were compared.

RESULTS

Twelve judges participated in the standard-setting exercise. The initial MPS was similar to the repeat MPS set after judges received trainee performance data (86.2% versus 86.9%, P = .23).

CONCLUSIONS

Prior knowledge about medical trainee performance data did not affect MPS as determined by the Mastery Angoff procedure.

Simulation-Based Mastery Learning for Thoracentesis Skills Improves Patient Outcomes: A Randomized Trial

PURPOSE

Physicians-in-training often perform bedside thoracenteses in academic medical centers, and complications are more common among less experienced clinicians. Simulation-based mastery learning (SBML) is one potential solution to this problem. This study evaluated the effects of a randomized trial of thoracentesis SBML on patient complications: iatrogenic pneumothorax (IP), hemothorax, and reexpansion pulmonary edema (REPE).

METHOD

The authors randomized internal medicine residents to undergo thoracentesis SBML at a tertiary care academic center from December 2012 to May 2016. They subsequently compared thoracentesis complications from procedures performed by SBML-trained residents, traditionally trained residents (no simulation training), and those referred to pulmonary medicine or interventional radiology (IR).

RESULTS

During the study period, 917 thoracenteses were performed on 709 patients. IP occurred in 60 (6.5%) procedures, of which 7 (11.6%) were clinically meaningful. SBML-trained residents performed procedures with a trend toward lower combined clinically meaningful complications (IP, hemothorax, REPE) compared with traditionally trained residents (7.9% vs. 0%; P = .06). SBML-trained residents caused fewer clinically meaningful IPs compared with traditionally trained residents, pulmonary, and IR referrals (P = .02). Hemothorax occurred after 4 (0.4%) thoracenteses, and SBML-trained residents had a trend toward lower hemothorax (0) compared with other groups (P = .07). REPE occurred after 3 (0.3%) procedures, with no differences between groups. SBML-trained residents performed procedures with lower combined clinically meaningful complications compared with other groups (P = .008).

CONCLUSIONS

Residents randomized to an SBML intervention performed thoracenteses with low rates of clinically meaningful complications. Rigorous education represents a successful quality improvement strategy.