Crowdsourcing Evaluation of Ureteroscopic Videos Using the Post-Ureteroscopic Lesion Scale to Assess Ureteral Injury

Introduction to surgical coaching

Surgical coaching is a new approach to continuous professional development (CPD). Advocates claim that coaching improves surgical and non-technical skills, leading to enhanced performance and reduced stress and burnout. Widespread acceptance of coaching in the surgical community is limited due to a paucity of evidence, surgery-specific obstacles and lack of resources. This narrative review introduces what 'surgical coaching' is and describes its origins. Coaching techniques are based on live or video observation of procedures, followed by collaborative analysis, reflection and goal planning in a non-competitive setting aided by an expert, peer or lay coach. Technical skills improvements have been reported using coaching techniques, yet long-term influence on non-technical competencies, overall performance and surgeon wellbeing are largely hypothetical. There are clear potential benefits in the application of surgical coaching to daily surgical practice. However, significant knowledge gaps remain. Dedicated research into the short-term applicability and the long-term effects of surgical coaching are required.

Crowd-Sourced Assessment of Surgical Skills of Urology Resident Applicants: Four-Year Experience

OBJECTIVE

To determine a) if surgical skills among urology resident applicants could be reliably assessed via crowdsourcing and b) to what extent surgical skills testing impacts resident selection.

DESIGN

Interviewees completed the following surgical skills tasks during their interview day: open knot tying (OKT), laparoscopic peg transfer (LPT), and robotic suturing (RS). Urology faculty and crowd-workers evaluated each applicant's video-recorded performance using validated scoring and were assessed for agreement using Cronbach's alpha. Applicants' USMLE scores, interview scores, and Jefferson Scale of Physician Empathy (JSPE-S) scores were assessed for correlation with skills testing scores and match rank. Additionally, a survey was distributed to interviewees assessing match outcomes.

SETTING

University of California Irvine Department of Urology, Surgical Skills Laboratory PARTICIPANTS: All 94 urology residency interviewees at the University of California Irvine Department of Urology from 2015-2018 were invited to complete the three surgical skills tasks on their interview day.

RESULTS

Survey responses were received from all 94 interviewees (100%). Crowd and expert agreement was good (α=0.88), fair (α=0.67), and poor (α=0.32) for LPT, RS, and OKT scores, respectively. The skills testing scores did not correlate with match rank, USMLE score, or JSPE-S score. On multivariate analysis, only interview score (r= -0.723; p<0.001) and faculty LPT score (r=-0.262; p=0.001) were significant predictors of match rank. Interviewees who reported matching into a top 3 residency choice had significantly higher faculty LPT scores than those who did not (11.9 vs. 9.7, p=0.03).

CONCLUSIONS

Surgical skills overall did not significantly impact match rank. Expert assessment of laparoscopic peg transfer skills and interview performance among urology resident applicants correlated with match rank.

Video-Based Coaching: Current Status and Role in Surgical Practice (Part 1) From the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee

Patient safety and outcomes are directly related to surgical performance. Surgical training emphasizes the importance of the surgeon in determining these outcomes. After training is complete, there is a lack of structured programs for surgeons to audit their skills and continue their individual development. There is a significant linear relationship between surgeon technical skill and surgical outcomes; however, measuring technical performance is difficult. Video-based coaching matches an individual surgeon in practice with a surgical colleague who has been trained in the core principles of coaching for individualizing instruction. It can provide objective assessment for teaching higher-level concepts, such as technical skills, cognitive skills, and decision-making. There are many benefits to video-based coaching. While the concept is gaining acceptance as a method of surgical education, it is still novel in clinical practice. As more surgeons look towards video-based coaching for quality improvement, a consistent definition of the program, goals, and metrics for assessment will be critical. This paper is a review on the status of the video-based coaching as it applies to practicing surgeons.

Expert Consensus Recommendations for Robotic Surgery Credentialing

OBJECTIVE

To define criteria for robotic credentialing using expert consensus.

BACKGROUND

A recent review of institutional robotic credentialing policies identified significant variability and determined current policies are largely inadequate to ensure surgeon proficiency and may threaten patient safety.

METHODS

28 national robotic surgery experts were invited to participate in a consensus conference. After review of available institutional policies and discussion, the group developed a 91 proposed criteria. Using a modified Delphi process the experts were asked to indicate their agreement with the proposed criteria in three electronic survey rounds after the conference. Criteria that achieved 80% or more in agreement (consensus) in all rounds were included in the final list.

RESULTS

All experts agreed that there is a need for standardized robotic surgery credentialing criteria across institutions that promote surgeon proficiency. 49 items reached consensus in the first round, 19 in the second, and 8 in the third for a total of 76 final items. Experts agreed that privileges should be granted based on video review of surgical performance and attainment of clearly defined objective proficiency benchmarks. Parameters for ongoing outcome monitoring were determined and recommendations for technical skills training, proctoring, and performance assessment were defined.

CONCLUSIONS

Using a systematic approach, detailed credentialing criteria for robotic surgery were defined. Implementation of these criteria uniformly across institutions will promote proficiency of robotic surgeons and has the potential to positively impact patient outcomes.

Correlation between operative time and crowd-sourced skills assessment for robotic bariatric surgery

INTRODUCTION

Operative time has been traditionally used as a proxy for surgical skill and is commonly utilized to measure the learning curve, assuming that faster operations indicate a more skilled surgeon. The Global Evaluative Assessment of Robotic Skills (GEARS) rubric is a validated Likert scale for evaluating technical skill. We hypothesize that operative time will not correlate with the GEARS score.

METHODS

Patients undergoing elective robotic sleeve gastrectomy at a single bariatric center of excellence hospital from January 2019 to March 2020 were captured in a prospectively maintained database. For step-specific scoring, videos were broken down into three steps: ligation of short gastric vessels, gastric transection, and oversewing the staple line. Overall and step-specific GEARS scores were assigned by crowd-sourced evaluators. Correlation between operative time and GEARS score was assessed with linear regression and calculation of the R² statistic.

RESULTS

Sixty-eight patients were included in the study, with a mean operative time of 112 ± 27.4 min. The mean GEARS score was 20.1 ± 0.81. Mean scores for the GEARS subcomponents were: bimanual dexterity 4.06 ± 0.17; depth perception 3.96 ± 0.24; efficiency 3.82 ± 0.19; force sensitivity 4.06 ± 0.20; robotic control 4.16 ± 0.21. Operative time and overall score showed no correlation (R² = 0.0146, p = 0.326). Step-specific times and scores showed weak correlation for gastric transection (R² = 0.0737, p = 0.028) and no correlation for ligation of short gastric vessels (R² = 0.0262, p = 0.209) or oversewing the staple line (R² = 0.0142, p = 0.344).

CONCLUSIONS

Operative time and crowd-sourced GEARS score were not correlated. Operative time and GEARS scores measure different performance characteristics, and future studies should consider using both a validated skills assessment tool and operative time for a more complete evaluation of skill.

Video review for measuring and improving skill in urological surgery

Interest is growing within the urological surgery community for objective assessments of technical skill. Surgical video review relies on the use of objective assessment tools to evaluate both global and procedure-specific skill. These evaluations provide structured feedback to surgeons with the aim of improving technique, which has been associated with patient outcomes. Currently, skill assessments can be performed by using expert peer-review, crowdsourcing or computer-based methods. Given the relationship between skill and patient outcomes, surgeons might be required in the future to provide empirical evidence of their technical skill for certification, employment, credentialing and quality improvement. Interventions such as coaching and skills workshops incorporating video review might help surgeons improve their skill, with the ultimate goal of improving patient outcomes.

Evidence that surgical performance predicts clinical outcomes

PURPOSE

Assessment of surgeon performance in the operating room has been identified as a direct method of measuring surgical quality. Studies published in urology and other surgical disciplines have investigated this link directly by measuring surgeon and team performance using methodology supported by validity evidence. This article highlights the key findings of these studies and associated underlying concepts.

METHODS

​Seminal literature from urology and related areas of research was used to inform this review of the performance-outcome relationship in surgery. Current efforts to further our understanding of this concept are discussed, including relevant quality improvement and educational interventions that utilize this relationship.

RESULTS

Evidence from multiple surgical specialties and procedures has established the association between surgeon skill and clinically significant patient outcomes. Novel methods of measuring performance utilize surgeon kinematics and artificial intelligence techniques to more reliably and objectively quantify surgical performance.

CONCLUSIONS

Future directions include the use of this data to create interventions for quality improvement, as well as innovate the credentialing and recertification process for practicing surgeons.

Emerging Mobile Platforms to Aid in Stone Management

Nephrolithiasis is an increasingly common condition worldwide and mobile technology is revolutionizing how patients with kidney stone are being diagnosed and managed. Emerging platforms include software applications to increase adherence to stone prevention, mobile compatible hardware, online social media communities, and telemedicine. Applications and hardware specifically relevant to increasing hydration, diet modification, medication adherence, and rapid diagnosis (ie, mobile ultrasound and endoscopy) have the greatest potential to reduce stone recurrence and expedite treatment. Social media and online communities have also been rapidly adopted by patients and providers to promote education and support.