Relationship Between Technical Errors and Decision-Making Skills in the Junior Resident

Development of Competencies in Oral and Maxillofacial Surgery Training

The goal of graduate surgical education is to ensure that the graduate is competent to practice in his or her chosen specialty. Traditionally, surgical learning has been based on an apprenticeship model; that is, the long-term observation and assessment of the trainee over a prolonged period of time. Patient expectations, work hour restrictions, and expectations of increased faculty oversight have led to decreased resident autonomy and independence. Graduates completing surgical training with less surgical autonomy may have lower clinical competence, which may affect patient safety, patient outcomes, and career satisfaction. This will require the modification of current assessment and training methods.

Surgical Skill Can be Objectively Measured From Fluoroscopic Images Using a Novel Image-based Decision Error Analysis (IDEA) Score

BACKGROUND

To advance orthopaedic surgical skills training and assessment, more rigorous and objective performance measures are needed. In hip fracture repair, the tip-apex distance is a commonly used summative performance metric with clear clinical relevance, but it does not capture the skill exercised during the process of achieving the final implant position. This study introduces and evaluates a novel Image-based Decision Error Analysis (IDEA) score that better captures performance during fluoroscopically-assisted wire navigation.

QUESTIONS/PURPOSES

(1) Can wire navigation skill be objectively measured from a sequence of fluoroscopic images? (2) Are skill behaviors observed in a simulated environment also exhibited in the operating room? Additionally, we sought to define an objective skill metric that demonstrates improvement associated with accumulated surgical experience.

METHODS

Performance was evaluated both on a hip fracture wire navigation simulator and in the operating room during actual fracture surgery. After examining fluoroscopic image sequences from 176 consecutive simulator trials (performed by 58 first-year orthopaedic residents) and 21 consecutive surgical procedures (performed by 19 different orthopaedic residents and one attending orthopaedic surgeon), three main categories of erroneous skill behavior were identified: off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling. Skill behaviors were measured by comparing wire adjustments made between consecutive images against the goal of targeting the apex of the femoral head as part of our new IDEA scoring methodology. Decision error metrics (frequency, magnitude) were correlated with other measures (image count and tip-apex distance) to characterize factors related to surgical performance on both the simulator and in the operating room. An IDEA composite score integrating decision errors (off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling) and the final tip-apex distance to produce a single metric of overall performance was created and compared with the number of hip wire navigation cases previously completed (such as surgeon experience levels).

RESULTS

The IDEA methodology objectively analyzed 37,000 images from the simulator and 688 images from the operating room. The number of decision errors (7 ± 5 in the operating room and 4 ± 3 on the simulator) correlated with fluoroscopic image count (33 ± 14 in the operating room and 20 ± 11 on the simulator) in both the simulator and operating room environments (R2 = 0.76; p < 0.001 and R2 = 0.71; p < 0.001, respectively). Decision error counts did not correlate with the tip-apex distance (16 ± 4 mm in the operating room and 12 ± 5 mm on the simulator) for either the simulator or the operating room (R2 = 0.08; p = 0.15 and R2 = 0.03; p = 0.47, respectively), indicating that the tip-apex distance is independent of decision errors. The IDEA composite score correlated with surgical experience (R2 = 0.66; p < 0.001).

CONCLUSION

The fluoroscopic images obtained in the course of placing a guide wire contain a rich amount of information related to surgical skill. This points the way to an objective measure of skill that also has potential as an educational tool for residents. Future studies should expand this analysis to the wide variety of procedures that rely on fluoroscopic images.

CLINICAL RELEVANCE

This study has shown how resident skill development can be objectively assessed from fluoroscopic image sequences. The IDEA scoring provides a basis for evaluating the competence of a resident. The score can be used to assess skill at key timepoints throughout residency, such as when rotating onto/off of a new surgical service and before performing certain procedures in the operating room, or as a tool for debriefing/providing feedback after a procedure is completed.

The Feasibility of Virtual Reality and Student-Led Simulation Training as Methods of Lumbar Puncture Instruction

BACKGROUND

There is limited data assessing simulation and virtual reality training as a standardized tool in medical education. This feasibility study aimed to evaluate the effectiveness of virtual reality training and a student-led simulation module in preparing medical students to perform a lumbar puncture.

METHODS

Twenty-five medical students completed a pre-intervention survey, and a baseline video recorded lumbar puncture procedure on a task trainer. Students were randomly distributed into the virtual reality group, or the curriculum's standard student-led procedural instruction group. Participants were then given 45 min to practice the lumbar puncture procedure. After the intervention, all participants were video recorded again as they performed a post-intervention lumbar puncture and completed a post-intervention survey. Pre- and post-intervention videos were scored using a critical action checklist in conjunction with time needed to complete the procedure to evaluate proficiency.

RESULTS

At baseline, there were no major statistically significant differences between groups. Assessing overall post-intervention performance, both groups showed improvement in aggregate score (p < 0.001) and time required to complete (p = 0.002) the lumbar puncture. Following interventions, the student-led group improved over the virtual reality group in a variety of metrics. The student-led group increased their aggregate score by 3.49 and decreased their time to completion by 34 s over the VR group when controlling for baseline measures.

CONCLUSIONS

Both virtual reality and student-led simulation training were useful training modalities, with hands-on simulation showing better results versus virtual reality training in this setting.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-020-01141-6.

Scoping Evidence Review on Training and Skills Assessment for Open Emergency Surgery

OBJECTIVE

Scope evidence on technical performance metrics for open emergency surgery. Identify surgical performance metrics and procedures used in trauma training courses.

DESIGN

Structured literature searches of electronic databases were conducted from January 2010 to December 2019 to identify systematic reviews of tools to measure surgical skills employed in vascular or trauma surgery evaluation and training.

SETTING AND PARTICIPANTS

Faculty of Shock Trauma Anesthesiology Research Center, University of Maryland School of Medicine, Uniformed Services University of Health Sciences, Bethesda, Maryland and Implementation Science, King's College, London.

RESULTS

The evidence from 21 systematic reviews including over 54,000 subjects enrolled into over 840 eligible studies, identified that the Objective Structured Assessment of Technical Skill was used for elective surgery not for emergency trauma and vascular control surgery procedures. The Individual Procedure Score (IPS), used to evaluate emergency trauma procedures performed before and after training, distinguished performance of residents from experts and practicing surgeons. IPS predicted surgeons who make critical errors and need remediation interventions. No metrics showed Kirkpatrick's Level 4 evidence of technical skills training benefit to emergency surgery outcomes.

CONCLUSIONS

Expert benchmarks, errors, complication rates, task completion time, task-specific checklists, global rating scales, Objective Structured Assessment of Technical Skills, and IPS were found to identify surgeons, at all levels of seniority, who are in need of remediation of technical skills for open surgical hemorrhage control. Large-scale, multicenter studies are needed to evaluate any benefit of trauma technical skills training on patient outcomes.

Chest Tube Insertion Among Surgical and Nonsurgical Trainees: How Skilled Are Our Residents?

BACKGROUND

Competency-based medical education has renewed focus on the attainment and evaluation of resident skill. Proper evaluation is crucial to inform educational interventions and identify residents in need of increased training and supervision. Currently, there is a paucity of studies rigorously evaluating resident chest tube insertion skill.

MATERIALS AND METHODS

Residents of all training levels before their intensive care unit rotation or currently rotating through the intensive care unit were invited to participate. Trainees inserted a thoracostomy tube on a high-fidelity simulator. Their performances were recorded and scored by blinded raters using the validated TUBE-iCOMPT rubric. Surgical and nonsurgical residents were compared.

RESULTS

Forty-nine residents participated; 30 from nonsurgical and 19 from surgical training programs. Overall, trainees were most deficient in the "preprocedural checks" and "patient positioning and local anesthetic" domains. Surgical trainees demonstrated higher chest tube insertion skill than their nonsurgical peers (median total score 88 [interquartile range, 74-90] versus 75 [interquartile range, 66-85], respectively, P = 0.01), particularly in the "patient positioning" and "blunt dissection" domains (P = 0.01 and P = 0.03, respectively). These differences were no longer significant when controlled for experience and Advanced Trauma Life Support certification.

CONCLUSIONS

Overall, surgical residents were more skilled than nonsurgical residents in tube thoracostomy placement. Relative skill deficits within the domains of chest tube insertion have also been identified among residents of different specialties. These areas can be targeted with educational interventions to improve resident performance, and ultimately, patient safety.

Pre- and post-review of a standardized ultrasound-guided central venous catheterization curriculum evaluating procedural skills acquisition and clinician confidence

Background:

To evaluate novice and expert clinicians’ procedural confidence utilizing a blended learning mixed fidelity simulation model when applying a standardized ultrasound-guided central venous catheterization curriculum.

Methods:

Simulation-based education and ultrasound-guided central venous catheter insertion aims to provide facility-wide efficiencies and improves patient safety through interdisciplinary collaboration. The objective of this quality improvement research was to evaluate both novice (<50) and expert (>50) clinicians’ confidence across 100 ultrasound-guided central venous catheter insertion courses were performed at a mixture of teaching and non-teaching hospitals across 26 states within the United States between April 2015 and April 2016. A total of 1238 attendees completed a pre- and post-survey after attending a mixed method clinical simulation course. Attendees completed a 4-h online didactic education module followed by 4 h of hands-on clinical simulation stations (compliance/sterile technique, needling techniques, vascular ultrasound assessment, and experiential complication management).

Results:

The use of a standardized evidence-based ultrasound-guided central venous catheter curriculum improved confidence and application to required clinical tasks and knowledge across all interdisciplinary specialties, regardless of level of experience. Both physician and non-physician groups resulted in statistically significant results in both procedural compliance ( p < 0.001) and ultrasound skills ( p < 0.001).

Conclusion:

The use of a standardized clinical simulation curriculum enhanced all aspects of ultrasound-guided central venous catheter insertion skills, knowledge, and improved confidence for all clinician types. Self-reported complications were reported at significantly higher rates than previously published evidence, demonstrating the need for ongoing procedural competencies. While there are growing benefits for the role of simulation-based programs, further evaluation is needed to explore its effectiveness in changing the quality of clinical outcomes within the healthcare setting.

Iatrogenic Critical Care Procedure Complication Boot Camp: A Simulation-based Pilot Study

BACKGROUND

Traditional medical education strategies teach learners how to correctly perform procedures while neglecting to provide formal training on iatrogenic error management. Error management training (EMT) requires active exploration as well as explicit encouragement for learners to make and learn from errors during training. Simulation provides an excellent methodology to execute a curriculum on iatrogenic procedural complication management. We hypothesize that a standardized simulation-based EMT curriculum will improve learner's confidence, cognitive knowledge, and performance in iatrogenic injury management.

METHODS

This was a pilot, prospective, observational study performed in a simulation center using a curriculum developed to educate resident physicians on iatrogenic procedural complication management. Pre- and postintervention assessments included confidence surveys, cognitive questionnaires, and critical action checklists for six simulated procedure complications. Assessment data were analyzed using medians and interquartile ranges (IQRs), and the paired change scores were tested for median equality to zero via Wilcoxon signed rank tests with p < 0.05 considered statistically significant.

RESULTS

Eighteen residents participated in the study curriculum. The median (IQR) confidence increased significantly by a summed score of 12.5 (8.75-17.25; p < 0.001). Similarly, the median (IQR) knowledge significantly increased by 6 (3-8) points from the pre- to postintervention assessment (p < 0.001). For each of the simulation cases, the number of critical actions performed increased significantly (p < 0.001 to p = 0.002).

CONCLUSION

We demonstrated significant improvement in the confidence, clinical knowledge, and performance of critical actions after the completion of this curriculum. This pilot study provides evidence that a structured EMT curriculum is an effective method to teach management of iatrogenic injuries.

A Simulation-based, cognitive assessment of resident decision making during complex urinary catheterization scenarios

BACKGROUND

This study explores general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios.

METHODS

40 residents were presented with two scenarios. Scenario A was a male with traumatic urethral injury and scenario B was a male with complete urinary blockage. Residents verbalized whether they would catheterize the patient and described the workup and management of suspected pathologies. Residents' decision paths were documented and analyzed.

RESULTS

In scenario A, 45% of participants chose to immediately consult Urology. 47.5% named five diagnostic tests to decide if catheterization was safe. In scenario B, 27% chose to catheterize with a 16 French Coude. When faced with catheterization failure, participants randomly upsized or downsized catheters. Chi-square analysis revealed no measurable consensus amongst participants.

CONCLUSIONS

Residents need more training in complex decision making for urinary catheterization. The decision trees generated in this study provide a useful blueprint of residents' learning needs.

SUMMARY

Exploration of general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios revealed major deficiencies. The resulting decision trees reveal residents' learning needs.