Evidence that surgical performance predicts clinical outcomes

Surgeon Skill and Perioperative Outcomes in Robot-Assisted Partial Nephrectomy

Importance

Technical skill in complex surgical procedures may affect clinical outcomes, and there is growing interest in understanding the clinical implications of surgeon proficiency levels.

Objectives

To determine whether surgeon scores representing technical skills of robot-assisted kidney surgery are associated with patient outcomes.

Design, Setting, and Participants

This quality improvement study included 10 urological surgeons participating in a surgical collaborative in Michigan from July 2021 to September 2022. Each surgeon submitted up to 7 videos of themselves performing robot-assisted partial nephrectomy. Videos were segmented into 6 key steps, yielding 127 video clips for analysis. Each video clip was deidentified and distributed to at least 3 of the 24 blinded peer surgeons from the collaborative who also perform robot-assisted partial nephrectomy. Reviewers rated technical skill and provided written feedback. Statistical analysis was performed from May 2023 to January 2024.

Main Outcomes and Measures

Reviewers scored each video clip using a validated instrument to assess technical skill for partial nephrectomy on a scale of 1 to 5 (higher scores indicating greater skill). For all submitting surgeons, outcomes from a clinical registry were assessed for length of stay (LOS) greater than 3 days, estimated blood loss (EBL) greater than 500 mL, warm ischemia time (WIT) greater than 30 minutes, positive surgical margin (PSM), 30-day emergency department (ED) visits, and 30-day readmission.

Results

Among the 27 unique surgeons who participated in this study as reviewers and/or individuals performing the procedures, 3 (11%) were female, and the median age was 47 (IQR, 39-52) years. Risk-adjusted outcomes were associated with scores representing surgeon skills. The overall performance score ranged from 3.5 to 4.7 points with a mean (SD) of 4.1 (0.4) points. Greater skill was correlated with significantly lower rates of LOS greater than 3 days (-6.8% [95% CI, -8.3% to -5.2%]), EBL greater than 500 mL (-2.6% [95% CI, -3.0% to -2.1%]), PSM (-8.2% [95% CI, -9.2% to -7.2%]), ED visits (-3.9% [95% CI, -5.0% to -2.8%]), and readmissions (-5.7% [95% CI, -6.9% to -4.6%]) (P < .001 for all). Higher overall score was also associated with higher partial nephrectomy volume (β coefficient, 11.4 [95% CI, 10.0-12.7]; P < .001).

Conclusions and Relevance

In this quality improvement study on video-based evaluation of robot-assisted partial nephrectomy, higher technical skill was associated with lower rates of adverse clinical outcomes. These findings suggest that video-based evaluation plays a role in assessing surgical skill and can be used in quality improvement initiatives to improve patient care.

Training for endovascular therapy of acute arterial disease and procedure-related complication: An extracorporeally-perfused human cadaver model study

PURPOSE

The aim of this study was to evaluate the usability of a recently developed extracorporeally-perfused cadaver model for training the angiographic management of acute arterial diseases and periprocedural complications that may occur during endovascular therapy of the lower extremity arterial runoff.

MATERIALS AND METHODS

Continuous extracorporeal perfusion was established in three fresh-frozen body donors via inguinal and infragenicular access. Using digital subtraction angiography for guidance, both arterial embolization (e.g., embolization using coils, vascular plugs, particles, and liquid embolic agents) and endovascular recanalization procedures (e.g., manual aspiration or balloon-assisted embolectomy) as well as various embolism protection devices were tested. Furthermore, the management of complications during percutaneous transluminal angioplasty, such as vessel dissection and rupture, were exercised by implantation of endovascular dissection repair system or covered stents. Interventions were performed by two board-certified interventional radiologists and one resident with only limited angiographic experience.

RESULTS

Stable extracorporeal perfusion was successfully established on both thighs of all three body donors. Digital subtraction angiography could be performed reliably and resulted in realistic artery depiction. The model allowed for repeatable training of endovascular recanalization and arterial embolization procedures with typical tactile feedback in a controlled environment. Furthermore, the handling of more complex angiographic devices could be exercised. Whereas procedural success was be ascertained for most endovascular interventions, thrombectomies procedures were not feasible in some cases due to the lack of inherent coagulation.

CONCLUSION

The presented perfusion model is suitable for practicing time-critical endovascular interventions in the lower extremity runoff under realistic but controlled conditions.

Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery

PURPOSE

Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip's 3D position and introduced a novel metric for surgical skill assessment.

METHODS

We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated.

RESULTS

Our analyses showed that tooltip path length (p [Formula: see text] 0.007) and path length along the instrument axis (p [Formula: see text] 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p [Formula: see text] 0.025) and the rubber band translocation tasks (p [Formula: see text] 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p [Formula: see text] 0.012) and increased by skill level.

CONCLUSION

Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.

Motion Smoothness-Based Assessment of Surgical Expertise: The Importance of Selecting Proper Metrics

The smooth movement of hand/surgical instruments is considered an indicator of skilled, coordinated surgical performance. Jerky surgical instrument movements or hand tremors can cause unwanted damages to the surgical site. Different methods have been used in previous studies for assessing motion smoothness, causing conflicting results regarding the comparison among surgical skill levels. We recruited four attending surgeons, five surgical residents, and nine novices. The participants conducted three simulated laparoscopic tasks, including peg transfer, bimanual peg transfer, and rubber band translocation. Tooltip motion smoothness was computed using the mean tooltip motion jerk, logarithmic dimensionless tooltip motion jerk, and 95% tooltip motion frequency (originally proposed in this study) to evaluate their capability of surgical skill level differentiation. The results revealed that logarithmic dimensionless motion jerk and 95% motion frequency were capable of distinguishing skill levels, indicated by smoother tooltip movements observed in high compared to low skill levels. Contrarily, mean motion jerk was not able to distinguish the skill levels. Additionally, 95% motion frequency was less affected by the measurement noise since it did not require the calculation of motion jerk, and 95% motion frequency and logarithmic dimensionless motion jerk yielded a better motion smoothness assessment outcome in distinguishing skill levels than mean motion jerk.

A Probabilistic Approach To Surgical Tasks and Skill Metrics

Identifying and quantifying the activities that compose surgery is essential for effective interventions, computer-aided analyses and the advancement of surgical data science. For example, recent studies have shown that objective metrics (referred to as objective performance indicators, OPIs) computed during key surgical tasks correlate with surgeon skill and clinical outcomes. Unambiguous identification of these surgical tasks can be particularly challenging for both human annotators and algorithms. Each surgical procedure has multiple approaches, each surgeon has their own level of skill, and the initiation and termination of surgical tasks can be subject to interpretation. As such, human annotators and machine learning models face the same basic problem, accurately identifying the boundaries of surgical tasks despite variable and unstructured information. For use in surgeon feedback, OPIs should also be robust to the variability and diversity in this data. To mitigate this difficulty, we propose a probabilistic approach to surgical task identification and calculation of OPIs. Rather than relying on tasks that are identified by hard temporal boundaries, we demonstrate an approach that relies on distributions of start and stop times, for a probabilistic interpretation of when the task was performed. We first use hypothetical data to outline how this approach is superior to other conventional approaches. Then we present similar analyses on surgical data. We find that when surgical tasks are identified by their individual probabilities, the resulting OPIs are less sensitive to noise in the identification of the start and stop times. These results suggest that this probabilistic approach holds promise for the future of surgical data science.