Assessing residents' operative skills for external ventricular drain placement and shunt surgery in pediatric neurosurgery

A Resident-Driven Mobile Evaluation System Can Be Used to Augment Traditional Surgery Rotation Evaluations

BACKGROUND

The Accreditation Council for Graduate Medical Education requires residents to receive milestone-based evaluations in key areas. Shortcomings of the traditional evaluation system (TES) are a low completion rate and delay in completion. We hypothesized that adoption of a mobile evaluation system (MES) would increase the number of evaluations completed and improve their timeliness.

METHODS

Traditional evaluations for a general surgery residency program were converted into a web-based form via a widely available, free, and secure application and implemented in August 2017. After 8 months, MES data were analyzed and compared to that of our TES.

RESULTS

122 mobile evaluations were completed; 20% were solicited by residents. Introduction of the MES resulted in an increased number of evaluations per resident (P = .0028) and proportion of faculty completing evaluations (P = .0220). Timeliness also improved, with 71% of evaluations being completed during one's clinical rotation.

CONCLUSIONS

A resident-driven MES is an inexpensive and effective method to augment traditional end-of-rotation evaluations.

Learning Curve of Closed Reduction and Internal Fixation for Supracondylar Fractures of the Humerus in Children

BACKGROUND

This study aimed to identify the threshold for success in supracondylar humeral fracture surgery by describing the learning curve for beginners and exploring the relationship between the learning curve and the prognosis of supracondylar fractures of the humerus.

METHODS

Surgical information was collected of the first 100 humeral fractures treated by four pediatric orthopedic surgeons. The relationship between operation time, wire placement success rate, and surgical experience was determined using the restricted cubic strip (RCS). The inflection point in the curve and other risk factors that may affect fracture prognosis were collected and subjected to multiple logistic regression to clarify the relationship between the learning curve and prognosis of supracondylar fractures of the humerus. After the training, the four fresh surgeons were interviewed in the form of questionnaires to get feedback from the trainees.

RESULTS

A total of 400 supracondylar fractures of the humerus from four pediatric orthopedists were included in the study. On an RCS analysis, 65 surgical experiences were the inflection point of the learning curve. Before and after these 65 surgical experiences, there were significant differences in the patients' anatomical reduction (186 vs. 122, P < 0.001), conversion to incision (33 vs. 6, P = 0.008), and supervising physician guidance (28 vs. 2, P < 0.001). In the multiple logistic regression analysis, functional recovery after supracondylar fractures of the humerus was significantly associated with surgical experience, intraoperative conversion to incision, and post-operative infection. Four surgeons and a supervisor were interviewed. They believed that self-confidence establishment requires the experience accumulation of about 30 operations. The most critical surgical technique is the reduction of fractures.

CONCLUSIONS

Although the accumulated experience of 30 operations can establish the self-confidence of trainers, fresh surgeons must accumulate experience with 65 operations to master closed reduction and internal fixation for supracondylar fractures. Surgical experience significantly impacts the post-operative recovery of patients with fractures.

LEVEL OF EVIDENCE

Level III.

Tracking Scan to Incision Time in Patients with Emergent Operative Traumatic Brain Injuries as a Measure for Systems-Based Practice in Neurosurgical Trainees

BACKGROUND

Evaluation of trainee performance remains a challenge in resident education, particularly for systems-based practice (SysBP) metrics including care coordination and interdisciplinary teamwork. Time to intervention is an important modifiable outcome variable in severe traumatic brain injury (TBI) and may serve as a trackable metric for SysBP evaluation.

METHODS

We retrospectively studied time from computed tomography head scan to surgical incision (CTH-INC, minutes) among neurosurgical trainees treating patients with emergently operative TBI as a proxy SysBP measure. Our institutional operative database was utilized to identify all emergent TBI cases between July 2015 and June 2020. Patients evaluated by program year (PGY)-2 residents proceeding directly to the operating room from the emergency department were included. Statistical analysis was performed using linear regression.

RESULTS

One hundred sixty-six cases triaged by 14 PGY-2 neurosurgical trainees were analyzed. Median CTH-INC was 104 minutes (interquartile range, 82-136 minutes). CTH-INC improved 20.1% over the academic year (95% confidence interval, 4.3%-33.2%, P = 0.015). Between the first and second 6-month periods, the rate of CTH-INC within 90 minutes (29% vs. 46%, P = 0.04) improved. On a per-individual PGY-2 basis, median CTH-INC ranged from 83-127 minutes, 25th percentile CTH-INC ranged from 62-108 minutes, and fastest CTH-INC ranged from 45-92 minutes.

CONCLUSIONS

CTH-INC is an objective and trackable proxy measure for evaluating SysBP during neurosurgical training. Use of CTH-INC or other time metrics in resident evaluations should not supersede the safe and effective delivery of patient care.

Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence

PURPOSE

Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries.

METHOD

Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries.

RESULTS

From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified.

CONCLUSIONS

This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.

A Comparison of Visual Rating Scales and Simulated Virtual Reality Metrics in Neurosurgical Training: A Generalizability Theory Study

BACKGROUND

Adequate assessment and feedback remains a cornerstone of psychomotor skills acquisition, particularly within neurosurgery where the consequence of adverse operative events is significant. However, a critical appraisal of the reliability of visual rating scales in neurosurgery is lacking. Therefore, we sought to design a study to compare visual rating scales with simulated metrics in a neurosurgical virtual reality task.

METHODS

Neurosurgical faculty rated anonymized participant video recordings of the removal of simulated brain tumors using a visual rating scale made up of seven composite elements. Scale reliability was evaluated using generalizability theory, and scale subcomponents were compared with simulated metrics using Pearson correlation analysis.

RESULTS

Four staff neurosurgeons evaluated 16 medical student neurosurgery applicants. Overall scale reliability and internal consistency were 0.73 and 0.90, respectively. Reliability of 0.71 was achieved with two raters. Individual participants, raters, and scale items accounted for 27%, 11%, and 0.6% of the data variability. The hemostasis scale component related to the greatest number of simulated metrics, whereas respect for no-go zones and tissue was correlated with none. Metrics relating to instrument force and patient safety (brain volume removed and blood loss) were captured by the fewest number of rating scale components.

CONCLUSIONS

To our knowledge, this is the first study comparing participant's ratings with simulated performance. Given rating scales capture less well instrument force, quantity of brain volume removed, and blood loss, we suggest adopting a hybrid educational approach using visual rating scales in an operative environment, supplemented by simulated sessions to uncover potentially problematic surgical technique.