Evaluating the Effect of Surgical Skill on Outcomes for Laparoscopic Sleeve Gastrectomy: A Video-based Study

Association of Crowd-Sourced Assessment of Technical Skills (CSATSTM) and Outcomes of Robotic Assisted Radical Prostatectomy

OBJECTIVE

To investigate if use of the Crowd-Sourced Assessment of Technical Skills (CSATSTM) platform and video peer review with constructive feedback is associated with improvement in technical skill and patient outcomes for robotic-assisted laparoscopic prostatectomy (RALP).

METHODS

Five fellowship-trained urologists voluntarily submitted RALP cases for CSATSTM Global Evaluative Assessment of Robotic Skills (GEARS) scoring and expert narrative review between April 15, 2022 - April 30, 2023. Surgeon-selected and randomly-selected cases were reviewed. Surgeons underwent local peer review of videos with constructive feedback. Change in GEARS scores and frequency of post-operative outcomes over the 12-month periods before and during the study were analyzed in logistic regression models. Bias was assessed with sensitivity analysis comparing surgeon-selected to randomly-selected cases.

RESULTS

GEARS scores for randomly-selected versus surgeon-selected cases did not differ significantly. Overall GEARS score correlated positively with annual surgical RALP volume (r=0.39, p=0.003) and negatively with years in practice (r=-0.34, p=0.01). After adjusting for confounders, there was no significant improvement in overall GEARS Score (0.01±0.02/month, p=0.48); but likelihood of sepsis (Odds Ratio 0.07, 95% CI 0.01-1.00, p=0.05) and pelvic fluid collection (Odds Ratio 0.09, 95% CI 0.01-0.99, p=0.049) were significantly decreased during the intervention period (n=165) compared to the prior 12 months (n=144). No outcome increased in likelihood (p>0.05).

CONCLUSIONS

Integration of CSATSTM and local video peer review is associated with significant improvement in patient outcomes after RALP, despite no significant change in surgeons' GEARS scores. This is the first study demonstrating improvement in patient RALP outcomes after implementation of such a paradigm in practicing surgeons.

Adoption of routine surgical video recording: a nationwide freedom of information act request across England and Wales

Background

Surgical video contains data with significant potential to improve surgical outcome assessment, quality assurance, education, and research. Current utilisation of surgical video recording is unknown and related policies/governance structures are unclear.

Methods

A nationwide Freedom of Information (FOI) request concerning surgical video recording, technology, consent, access, and governance was sent to all acute National Health Service (NHS) trusts/boards in England/Wales between 20th February and 20th March 2023.

Findings

140/144 (97.2%) trusts/boards in England/Wales responded to the FOI request. Surgical procedures were routinely recorded in 22 trusts/boards. The median estimate of consultant surgeons routinely recording their procedures was 20%. Surgical video was stored on internal systems (n = 27), third-party products (n = 29), and both (n = 9). 32/140 (22.9%) trusts/boards ask for consent to record procedures as part of routine care. Consent for recording included non-clinical purposes in 55/140 (39.3%) trusts/boards. Policies for surgeon/patient access to surgical video were available in 48/140 (34.3%) and 32/140 (22.9%) trusts/boards, respectively. Surgical video was used for non-clinical purposes in 64/140 (45.7%) trusts/boards. Governance policies covering surgical video recording, use, and/or storage were available from 59/140 (42.1%) trusts/boards.

Interpretation

There is significant heterogeneity in surgical video recording practices in England and Wales. A minority of trusts/boards routinely record surgical procedures, with large variation in recording/storage practices indicating scope for NHS-wide coordination. Revision of surgical video consent, accessibility, and governance policies should be prioritised by trusts/boards to protect key stakeholders. Increased availability of surgical video is essential for patients and surgeons to maximally benefit from the ongoing digital transformation of surgery.

Funding

KL is supported by an NIHR Academic Clinical Fellowship and acknowledges infrastructure support for this research from the National Institute for Health Research (NIHR) Imperial Biomedical Research Centre (BRC).

Appraisal of surgical outcomes and oncological efficiency of intraoperative adverse events in robotic radical gastrectomy for gastric cancer

BACKGROUND

Surgical quality control is a crucial determinant of evaluating the tumor efficacy.

OBJECTIVE

To assess the ClassIntra grade for quality control and oncological outcomes of robotic radical surgery for gastric cancer (GC).

METHODS

Data of patients undergoing robotic radical surgery for GC at a high-volume center were retrospectively analyzed. Patients were categorized into two groups, the intraoperative adverse event (iAE) group and the non-iAE group, based on the occurrence of intraoperative adverse events. The iAEs were further classified into five sublevels (ranging from I to V according to severity) based on the ClassIntra grade. Surgical performance was assessed using the Objective Structured Assessment of Technical Skill (OSATS) and the General Error Reporting Tool.

RESULTS

This study included 366 patients (iAE group: n = 72 [19.7%] and non-iAE group: n = 294 [80.3%]). The proportion of ClassIntra grade II patients was the highest in the iAE group (54.2%). In total and distal gastrectomies, iAEs occurred most frequently in the suprapancreatic area (50.0% and 54.8%, respectively). In total gastrectomy, grade IV iAEs were most common during lymph node dissection in the splenic hilum area (once for bleeding [grade IV] and once for injury [grade IV]). The overall survival (OS) and disease-free survival of the non-iAE group were significantly better than those of the iAE group (Log rank P < 0.001). Uni- and multi-variate analyses showed that iAEs were key prognostic indicators, independent of tumor stage and adjuvant chemotherapy (P < 0.001).

CONCLUSION

iAEs in patients who underwent robotic radical gastrectomy significantly correlated with the occurrence of postoperative complications and a poor long-term prognosis. Therefore, utilization and inclusion of ClassIntra grading as a crucial surgical quality control and prognostic indicator in the routine surgical quality evaluation system are recommended.

Nationwide Association of Surgical Performance of Minimally Invasive Esophagectomy With Patient Outcomes

Importance

Suboptimal surgical performance is hypothesized to be associated with less favorable patient outcomes in minimally invasive esophagectomy (MIE). Establishing this association may lead to programs that promote better surgical performance of MIE and improve patient outcomes.

Objective

To investigate associations between surgical performance and postoperative outcomes after MIE.

Design, Setting, and Participants

In this nationwide cohort study of 15 Dutch hospitals that perform more than 20 MIEs per year, 7 masked expert MIE surgeons assessed surgical performance using videos and a previously developed and validated competency assessment tool (CAT). Each hospital submitted 2 representative videos of MIEs performed between November 4, 2021, and September 13, 2022. Patients registered in the Dutch Upper Gastrointestinal Cancer Audit between January 1, 2020, and December 31, 2021, were included to examine patient outcomes.

Exposure

Hospitals were divided into quartiles based on their MIE-CAT performance score. Outcomes were compared between highest (top 25%) and lowest (bottom 25%) performing quartiles. Transthoracic MIE with gastric tube reconstruction.

Main Outcome and Measure

The primary outcome was severe postoperative complications (Clavien-Dindo ≥3) within 30 days after surgery. Multilevel logistic regression, with clustering of patients within hospitals, was used to analyze associations between performance and outcomes.

Results

In total, 30 videos and 970 patients (mean [SD] age, 66.6 [9.1] years; 719 men [74.1%]) were included. The mean (SD) MIE-CAT score was 113.6 (5.5) in the highest performance quartile vs 94.1 (5.9) in the lowest. Severe postoperative complications occurred in 18.7% (41 of 219) of patients in the highest performance quartile vs 39.2% (40 of 102) in the lowest (risk ratio [RR], 0.50; 95% CI, 0.24-0.99). The highest vs the lowest performance quartile showed lower rates of conversions (1.8% vs 8.9%; RR, 0.21; 95% CI, 0.21-0.21), intraoperative complications (2.7% vs 7.8%; RR, 0.21; 95% CI, 0.04-0.94), and overall postoperative complications (46.1% vs 65.7%; RR, 0.54; 95% CI, 0.24-0.96). The R0 resection rate (96.8% vs 94.2%; RR, 1.03; 95% CI, 0.97-1.05) and lymph node yield (mean [SD], 38.9 [14.7] vs 26.2 [9.0]; RR, 3.20; 95% CI, 0.27-3.21) increased with oncologic-specific performance (eg, hiatus dissection, lymph node dissection). In addition, a high anastomotic phase score was associated with a lower anastomotic leakage rate (4.6% vs 17.7%; RR, 0.14; 95% CI, 0.06-0.31).

Conclusions and Relevance

These findings suggest that better surgical performance is associated with fewer perioperative complications for patients with esophageal cancer on a national level. If surgical performance of MIE can be improved with MIE-CAT implementation, substantially better patient outcomes may be achievable.

Decoding surgical skill: an objective and efficient algorithm for surgical skill classification based on surgical gesture features -experimental studies

BACKGROUND

Various surgical skills lead to differences in patient outcomes and identifying poorly skilled surgeons with constructive feedback contributes to surgical quality improvement. The aim of the study was to develop an algorithm for evaluating surgical skills in laparoscopic cholecystectomy based on the features of elementary functional surgical gestures (Surgestures).

MATERIALS AND METHODS

Seventy-five laparoscopic cholecystectomy videos were collected from 33 surgeons in five hospitals. The phase of mobilization hepatocystic triangle and gallbladder dissection from the liver bed of each video were annotated with 14 Surgestures. The videos were grouped into competent and incompetent based on the quantiles of modified global operative assessment of laparoscopic skills (mGOALS). Surgeon-related information, clinical data, and intraoperative events were analyzed. Sixty-three Surgesture features were extracted to develop the surgical skill classification algorithm. The area under the receiver operating characteristic curve of the classification and the top features were evaluated.

RESULTS

Correlation analysis revealed that most perioperative factors had no significant correlation with mGOALS scores. The incompetent group has a higher probability of cholecystic vascular injury compared to the competent group (30.8 vs 6.1%, P =0.004). The competent group demonstrated fewer inefficient Surgestures, lower shift frequency, and a larger dissection-exposure ratio of Surgestures during the procedure. The area under the receiver operating characteristic curve of the classification algorithm achieved 0.866. Different Surgesture features contributed variably to overall performance and specific skill items.

CONCLUSION

The computer algorithm accurately classified surgeons with different skill levels using objective Surgesture features, adding insight into designing automatic laparoscopic surgical skill assessment tools with technical feedback.

Association Between Racial and Socioeconomic Disparities and Hospital Performance in Treatment and Outcomes for Patients with Colon Cancer

BACKGROUND

Disparities in colon cancer care and outcomes by race/ethnicity, socioeconomic status (SES), and insurance are well recognized; however, the extent to which inequalities are driven by patient factors versus variation in hospital performance remains unclear. We sought to compare disparities in care delivery and outcomes at low- and high-performing hospitals.

METHODS

We identified patients with stage I-III colon adenocarcinoma from the 2012-2017 National Cancer Database. Adequate lymphadenectomy and timely adjuvant chemotherapy administration defined hospital performance. Multilevel regression models evaluated disparities by race/ethnicity, SES, and insurance at the lowest- and highest-performance quartile hospitals.

RESULTS

Of 92,573 patients from 704 hospitals, 45,982 (49.7%) were treated at 404 low-performing hospitals and 46,591 (50.3%) were treated at 300 high-performing hospitals. Low-performing hospitals treated more non-Hispanic (NH) Black, Hispanic, low SES, and Medicaid patients (all p < 0.01). Among low-performing hospitals, patients with low versus high SES (odds ratio [OR] 0.87, 95% confidence interval [CI] 0.82-0.92), and Medicare (OR 0.90, 95% CI 0.85-0.96) and Medicaid (OR 0.88, 95% CI 0.80-0.96) versus private insurance, had decreased odds of receiving high-quality care. At high-performing hospitals, NH Black versus NH White patients (OR 0.83, 95% CI 0.72-0.95) had decreased odds of receiving high-quality care. Low SES, Medicare, Medicaid, and uninsured patients had worse overall survival at low- and high-performing hospitals (all p < 0.01).

CONCLUSION

Disparities in receipt of high-quality colon cancer care occurred by SES and insurance at low-performing hospitals, and by race at high-performing hospitals. However, survival disparities by SES and insurance exist irrespective of hospital performance. Future steps include improving low-performing hospitals and identifying mechanisms affecting survival disparities.

A novel high accuracy model for automatic surgical workflow recognition using artificial intelligence in laparoscopic totally extraperitoneal inguinal hernia repair (TEP)

INTRODUCTION

Artificial intelligence and computer vision are revolutionizing the way we perceive video analysis in minimally invasive surgery. This emerging technology has increasingly been leveraged successfully for video segmentation, documentation, education, and formative assessment. New, sophisticated platforms allow pre-determined segments chosen by surgeons to be automatically presented without the need to review entire videos. This study aimed to validate and demonstrate the accuracy of the first reported AI-based computer vision algorithm that automatically recognizes surgical steps in videos of totally extraperitoneal (TEP) inguinal hernia repair.

METHODS

Videos of TEP procedures were manually labeled by a team of annotators trained to identify and label surgical workflow according to six major steps. For bilateral hernias, an additional change of focus step was also included. The videos were then used to train a computer vision AI algorithm. Performance accuracy was assessed in comparison to the manual annotations.

RESULTS

A total of 619 full-length TEP videos were analyzed: 371 were used to train the model, 93 for internal validation, and the remaining 155 as a test set to evaluate algorithm accuracy. The overall accuracy for the complete procedure was 88.8%. Per-step accuracy reached the highest value for the hernia sac reduction step (94.3%) and the lowest for the preperitoneal dissection step (72.2%).

CONCLUSIONS

These results indicate that the novel AI model was able to provide fully automated video analysis with a high accuracy level. High-accuracy models leveraging AI to enable automation of surgical video analysis allow us to identify and monitor surgical performance, providing mathematical metrics that can be stored, evaluated, and compared. As such, the proposed model is capable of enabling data-driven insights to improve surgical quality and demonstrate best practices in TEP procedures.

Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery

BACKGROUND

Surgical assessment instruments are used for formative and summative trainee evaluations. To characterize the features of existing instruments and a novel 12-item objective, procedure-specific assessment tool for Roux-en-Y Gastric Bypass (RYGB-OPSA), we evaluated the progress of a single surgical fellow over 17 consecutive surgeries.

METHODS

Seventeen consecutive RYGB videos completed between 8/2021 and 1/2022 by an academic hospital surgical fellow were de-identified and assessed by four board-certified bariatric surgeons using Global Operative Assessment of Laparoscopic Skills (GOALS), General Assessment of Surgical Skill (GASS), and RYGB-OPSA which includes the reflection of transverse colon, identification of ligament of Treitz, biliopancreatic and Roux limbs orientation, jejunal division point selection, stapler use, mesentery division, bleeding control, jejunojejunostomy (JJ) anastomotic site selection, apposition of JJ anastomotic site, JJ creation, common enterotomy closure of JJ, and integrity of anastomosis. The GASS measured economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, and achievement of hemostasis. RYGB-OPSA and GASS items were scored "poor-unsafe," "acceptable-safe," or "good-safe." Change in performance was measured by linear trendline slope.

RESULTS

Over the course of 17 procedures, significant improvement was demonstrated by three GOALS items, GOALS overall score, GASS bimanual dexterity, and three RYGB-OPSA tasks: JJ creation, jejunal division point selection, and stapler use. Achievement of hemostasis declined but never rated "poor-unsafe." Overall RYGB-OPSA and GOALS trendlines documented significant increase across the 17 procedures.

CONCLUSION

This examination of a bariatric surgery fellow's operative training experience as measured by three surgical assessment instruments demonstrated anticipated improvements in general skills and safe completion of procedure-specific tasks. Effective surgical assessment instruments have enough sensitivity to show improvement to enable meaningful trainee feedback (low-stakes assessments) as well as the ability to determine safe surgical practice to enable promotion to greater autonomous practice.

Predictive Value of KLASS-02-QC Assessment Score on KLASS-02 Surgical Outcomes: Validation of Surgeon Quality Control and Standardization for D2 Lymphadenectomy

OBJECTIVE

The aim of this study was to audit the 22 items and assessed each item's predictive value on surgical outcomes.

BACKGROUND

The KLASS-02 trial revealed that the oncologic outcomes of laparoscopic distal gastrectomy are not inferior to open distal gastrectomy in patients with advanced gastric cancer. The surgeons participating in this trial were chosen based on the assessment scores from the KLASS-02-QC trial, which used 22 items for standardization of D2 lymphadenectomy and quality control.

METHODS

We reviewed proficiency scores (PSs) for 22 items for 20 surgeons who participated in KLASS-02. The surgeons were divided into 2 groups according to PS, and the perioperative outcomes of 924 patients enrolled in KLASS-02 were compared between groups. Each item's predictive value for perioperative outcome was then assessed using multivariable regression models.

RESULTS

Of the total 924 patients, 529 were operated on by high-score surgeons (high PS) and 395 were operated on by low-score surgeons (low-PS). High-PS group had less intraoperative blood loss, longer operation times, and fewer complications, major complications, reoperations, and shorter first flatus and hospital stay than low-PS group ( P =0.006, P <0.001, P <0.001, P <0.001, P =0.042, P =0.013, and P <0.001, respectively). Some items used in KLASS-02-QC predicted perioperative outcomes, such as intraoperative blood loss, major complications, reoperation, and hospital stay.

CONCLUSIONS

Although this study only analyzed data associated with qualified surgeons, the 22 items effectively assessed the surgeons based on PS. A high score was associated with longer operation times, but better perioperative outcomes.

The effect of bedside assistant technical performance on outcomes in robotic surgery

Technical performance in surgery has been associated with patient outcomes. Robotic surgery is unique in that both a console surgeon and bedside surgeon are required. A systematic review according to PRISMA guidelines identified all pertinent literature regarding skill level of the bedside assistant with regards to patient outcomes in robotic surgery. 10 studies met inclusion criteria. In all studies, the skill level of the assistant was based on experience, either by post-graduate year of the resident or number of cases previously performed by the assistant. Five studies reported significant, shorter operative times with increasing experience of the bedside assistant. No study reported a significant difference in postoperative outcomes. The existing literature fails to show improved patient outcomes with more experienced bedside assistants in robotic surgery. Metrics should be developed to measure actual technical performance of the bedside assistant rather than using arbitrary assessments of experience in future studies.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.

An American Board of Surgery Pilot of Video Assessment of Surgeon Technical Performance in Surgery

OBJECTIVE

The American Board of Surgery (ABS) sought to investigate the suitability of video-based assessment (VBA) as an adjunct to certification for assessing technical skills.

BACKGROUND

Board certification is based on the successful completion of a residency program coupled with knowledge and reasoning assessments. VBA is a new modality for evaluating operative skills that have been shown to correlate with patient outcomes after surgery.

METHODS

Diplomates of the ABS were initially assessed for background knowledge and interest in VBA. Surgeons were then solicited to participate in the pilot. Three commercially available VBA platforms were identified and used for the pilot assessment. All participants served as reviewers and reviewees for videos. After the interaction, participants were surveyed regarding their experiences and recommendations to the ABS.

RESULTS

To the initial survey, 4853/25,715 diplomates responded. The majority were neither familiar with VBA, nor the tools used for operative assessments. Two hundred seventy-four surgeons actively engaged in the subsequent pilot. One hundred sixty-nine surgeons completed the postpilot survey. Most participants found the process straightforward. Of the participants, 74% felt that the feedback would help their surgical practice. The majority (81%) remain interested in VBA for continuing medical education credits. Using VBA in continuous certification could improve surgeon skills felt by 70%. Two-thirds of participants felt VBA could help identify and remediate underperforming surgeons. Identified barriers to VBA included limitations for open surgery, privacy issues, and technical concerns.

CONCLUSIONS

VBA is promising as an adjunct to the current board certification process and should be further considered by the ABS.

Skill-qualified surgeons positively affect short-term outcomes after laparoscopic gastrectomy for gastric cancer: a survey of the National Clinical Database of Japan

BACKGROUND

Proficiency of the operating surgeon is one of the most critical factors potentially associated with reductions in complications and surgery-related mortality. With video-rating systems having shown potential for assessing laparoscopic surgeons' proficiency, the Endoscopic Surgical Skill Qualification System (ESSQS) was developed by the Japan Society for Endoscopic Surgery to subjectively assess the proficiency of laparoscopic surgeons by rating applicants' non-edited case videos. We conducted a study to evaluate how ESSQS skill-qualified (SQ) surgeon involvement influences short-term outcomes of laparoscopic gastrectomy performed for gastric cancer.

METHODS

Data from the National Clinical Database regarding laparoscopic distal and total gastrectomy performed for gastric cancer between January 2016 and December 2018 were analyzed. Operative mortality, defined as 30-day mortality or 90-day in-hospital mortality, and anastomotic leakage rates were compared per involvement vs. non-involvement of an SQ surgeon. Outcomes were also compared per involvement of a gastrectomy-, colectomy-, or cholecystectomy-qualified surgeon. The association between the area of qualification and operative mortality/anastomotic leakage was also analyzed with a generalized estimating equation logistic regression model used to account for patient-level risk factors and institutional differences.

RESULTS

Of 104,093 laparoscopic distal gastrectomies, 52,143 were suitable for inclusion in the study; 30,366 (58.2%) were performed by an SQ surgeon. Of 43,978 laparoscopic total gastrectomies, 10,326 were suitable for inclusion; 6501 (63.0%) were performed by an SQ surgeon. Gastrectomy-qualified surgeons outperformed non-SQ surgeons in terms of both operative mortality and anastomotic leakage. They also outperformed cholecystectomy- and colectomy-qualified surgeons in terms of operative mortality or anastomotic leakage in distal and total gastrectomy, respectively.

CONCLUSION

The ESSQS appears to discriminate laparoscopic surgeons who can be expected to achieve significantly improved gastrectomy outcomes.

Evolution of the digital operating room: the place of video technology in surgery

PURPOSE

The aim of this review was to collate current evidence wherein digitalisation, through the incorporation of video technology and artificial intelligence (AI), is being applied to the practice of surgery. Applications are vast, and the literature investigating the utility of surgical video and its synergy with AI has steadily increased over the last 2 decades. This type of technology is widespread in other industries, such as autonomy in transportation and manufacturing.

METHODS

Articles were identified primarily using the PubMed and MEDLINE databases. The MeSH terms used were "surgical education", "surgical video", "video labelling", "surgery", "surgical workflow", "telementoring", "telemedicine", "machine learning", "deep learning" and "operating room". Given the breadth of the subject and the scarcity of high-level data in certain areas, a narrative synthesis was selected over a meta-analysis or systematic review to allow for a focussed discussion of the topic.

RESULTS

Three main themes were identified and analysed throughout this review, (1) the multifaceted utility of surgical video recording, (2) teleconferencing/telemedicine and (3) artificial intelligence in the operating room.

CONCLUSIONS

Evidence suggests the routine collection of intraoperative data will be beneficial in the advancement of surgery, by driving standardised, evidence-based surgical care and personalised training of future surgeons. However, many barriers stand in the way of widespread implementation, necessitating close collaboration between surgeons, data scientists, medicolegal personnel and hospital policy makers.

SAGES video acquisition framework-analysis of available OR recording technologies by the SAGES AI task force

BACKGROUND

Surgical video recording provides the opportunity to acquire intraoperative data that can subsequently be used for a variety of quality improvement, research, and educational applications. Various recording devices are available for standard operating room camera systems. Some allow for collateral data acquisition including activities of the OR staff, kinematic measurements (motion of surgical instruments), and recording of the endoscopic video streams. Additional analysis through computer vision (CV), which allows software to understand and perform predictive tasks on images, can allow for automatic phase segmentation, instrument tracking, and derivative performance-geared metrics. With this survey, we summarize available surgical video acquisition technologies and associated performance analysis platforms.

METHODS

In an effort promoted by the SAGES Artificial Intelligence Task Force, we surveyed the available video recording technology companies. Of thirteen companies approached, nine were interviewed, each over an hour-long video conference. A standard set of 17 questions was administered. Questions spanned from data acquisition capacity, quality, and synchronization of video with other data, availability of analytic tools, privacy, and access.

RESULTS

Most platforms (89%) store video in full-HD (1080p) resolution at a frame rate of 30 fps. Most (67%) of available platforms store data in a Cloud-based databank as opposed to institutional hard drives. CV powered analysis is featured in some platforms: phase segmentation in 44% platforms, out of body blurring or tool tracking in 33%, and suture time in 11%. Kinematic data are provided by 22% and perfusion imaging in one device.

CONCLUSION

Video acquisition platforms on the market allow for in depth performance analysis through manual and automated review. Most of these devices will be integrated in upcoming robotic surgical platforms. Platform analytic supplementation, including CV, may allow for more refined performance analysis to surgeons and trainees. Most current AI features are related to phase segmentation, instrument tracking, and video blurring.

Is staple line reinforcement still needed on contemporary staplers? A benchtop analysis

BACKGROUND

Staple line reinforcement (SLR) is commonly used in bariatric surgeries to reduce leaks and bleeds. With the evolution of staplers, the need for buttressing with the latest surgical stapling technology is in question. The efficacy of GORE® SEAMGUARD® (G-SLR) to improve staple line strength based on an established measure of burst pressure was evaluated. A benchtop test on synthetic tissue evaluated the pressure required for staple line leak across surgical staplers with and without G-SLR.

METHODS

Staple lines on a consistent thickness synthetic bowel were pressurized to the point of failure (burst pressure) among Ethicon®, Intuitive®, and Medtronic® surgical staplers with and without G-SLR. Burst pressure and leak location (through the staple line [TTSL] or through the staple [TTS], on the anvil or cartridge side) were recorded. Visual confirmation of a leak concluded each test.

RESULTS

The pooled mean burst pressure for G-SLR was greater (p < 0.05) by 0.494 pounds/square inch compared with no reinforcement with no meaningful differences among staplers. Leak failures were primarily TTS (91.7%) and equally distributed between reinforcement groups with more leak failures on the cartridge side with G-SLR and on the anvil side for non-SLR group. Leaks occurred across the length of staple lines with no discernable pattern.

CONCLUSION

Employing a buttressing material strengthens the staple line, as measured by burst pressure, and may reduce the risk for staple line failure. This benchtop study of G-SLR with three commonly used surgical staplers demonstrated a significant increase in burst pressures among the studied stapling devices.

The Surgical Learning Curve: Does Robotic Technical Skill Explain Differences in Operative Performance?

Background: Prior studies on technical skills use small collections of videos for assessment. However, there is likely heterogeneity of performance among surgeons and likely improvement after training. If technical skill explains these differences, then it should vary among practicing surgeons and improve over time. Materials and Methods: Sleeve gastrectomy cases (n = 162) between July 2018 and January 2021 at one health system were included. Global evaluative assessment of robotic skills (GEARS) scores were assigned by crowdsourced evaluators. Videos were manually annotated. Analysis of variance was used to compare continuous variables between surgeons. Tamhane's post hoc test was used to define differences between surgeons with the eta-squared value for effect size. Linear regression was used for temporal changes. A P value <.05 was considered significant. Results: Variations in operative time discriminated between individuals (e.g., between 2 surgeons, means were 91 and 112 minutes, Tamhane's = 0.001). Overall, GEARS scores did not vary significantly (e.g., between those 2 surgeons, means were 20.32 and 20.6, Tamhane's = 0.151). Operative time and total GEARS score did not change over time (R² = 0.0001-0.096). Subcomponent scores showed idiosyncratic temporal changes, although force sensitivity increased among all (R² = 0.172-0.243). For a novice surgeon, phase-adjusted operative time (R² = 0.24), but not overall GEARS scores (R² = 0.04), improved over time. Conclusions: GEARS scores showed less variability and did not improve with time for a novice surgeon. Improved technical skill does not explain the learning curve of a novice surgeon or variation among surgeons. More work could define valid surrogate metrics for performance analysis.

Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy

BACKGROUND

The Japanese operative-rating scale for laparoscopic distal gastrectomy (JORS-LDG) was developed through cognitive task analysis together with the Delphi method to measure intraoperative performance during laparoscopic distal gastrectomy. This study aimed to investigate the value of this rating scale as an educational tool and a surgical outcome predictor in laparoscopic distal gastrectomy.

METHODS

The surgical performance of laparoscopic distal gastrectomy was assessed by the first assistant, through self-evaluation in the operating room and by video raters blind to the case. We evaluated inter-rater reliability, internal consistency, and correlations between the JORS-LDG scores and the evaluation methods, patient characteristics, and surgical outcomes.

RESULTS

Fifty-four laparoscopic distal gastrectomy procedures performed by 40 surgeons at 16 institutions were evaluated in the operating room and with video recordings using the proposed rating scale. The video inter-rater reliability was > 0.8. Participating surgeons were divided into the low, intermediate, and high groups based on their total scores. The number of laparoscopic surgeries and laparoscopic gastrectomy procedures performed differed significantly among the groups according to laparoscopic distal gastrectomy skill levels. The low, intermediate, and high groups also differed in terms of median operating times (311, 266, and 229 min, respectively, P < 0.001), intraoperative complication rates (27.8, 11.8, and 0%, respectively, P = 0.01), and postoperative complication rates (22.2, 0, and 0%, respectively, P = 0.002).

CONCLUSIONS

The JORS-LDG is a reliable and valid measure for laparoscopic distal gastrectomy training and could be useful in predicting surgical outcomes.

The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review

BACKGROUND

Efforts to improve surgical safety and outcomes have traditionally placed little emphasis on intraoperative performance, partly due to difficulties in measurement. Video-based assessment (VBA) provides an opportunity for blinded and unbiased appraisal of surgeon performance. Therefore, we aimed to systematically review the existing literature on the association between intraoperative technical performance, measured using VBA, and patient outcomes.

METHODS

Major databases (Medline, Embase, Cochrane Database, and Web of Science) were systematically searched for studies assessing the association of intraoperative technical performance measured by tools supported by validity evidence with short-term (≤ 30 days) and/or long-term postoperative outcomes. Study quality was assessed using the Newcastle-Ottawa Scale. Results were appraised descriptively as study heterogeneity precluded meta-analysis.

RESULTS

A total of 11 observational studies were identified involving 8 different procedures in foregut/bariatric (n = 4), colorectal (n = 4), urologic (n = 2), and hepatobiliary surgery (n = 1). The number of surgeons assessed ranged from 1 to 34; patient sample size ranged from 47 to 10,242. High risk of bias was present in 5 of 8 studies assessing short-term outcomes and 2 of 6 studies assessing long-term outcomes. Short-term outcomes were reported in 8 studies (i.e., morbidity, mortality, and readmission), while 6 reported long-term outcomes (i.e., cancer outcomes, weight loss, and urinary continence). Better intraoperative performance was associated with fewer postoperative complications (6 of 7 studies), reoperations (3 of 4 studies), and readmissions (1 of 4 studies). Long-term outcomes were less commonly investigated, with mixed results.

CONCLUSION

Current evidence supports an association between superior intraoperative technical performance measured using surgical videos and improved short-term postoperative outcomes. Intraoperative performance analysis using video-based assessment represents a promising approach to surgical quality-improvement.

Outcome prediction in bariatric surgery through video-based assessment

INTRODUCTION

The relationship between intraoperative surgical performance scores and patient outcomes has not been demonstrated at a single-case level. The GEARS score is a Likert-based scale that quantifies robotic surgical proficiency in 5 domains. Given that even highly skilled surgeons can have variability in their skill among their cases, we hypothesized that at a patient level, higher surgical skill as determined by the GEARS score will predict individual patient outcomes.

METHODS

Patients undergoing robotic sleeve gastrectomy between July 2018 and January 2021 at a single-health care system were captured in a prospective database. Bivariate Pearson's correlation was used to compare continuous variables, one-way ANOVA for categorical variables compared with a continuous variable, and chi-square for two categorical variables. Significant variables in the univariable screen were included in a multivariable linear regression model. Two-tailed p-value < 0.05 was considered significant.

RESULTS

Of 162 patients included, 9 patients (5.5%) experienced a serious morbidity within 30 days. The average excess weight loss (EWL) was 72 ± 12% at 6 months and 74 ± 15% at 12 months. GEARS score was not significantly correlated with EWL at 6 months (p = 0.349), 12 months (p = 0.468), or serious morbidity (p = 0.848) on unadjusted analysis. After adjusting, total GEARS score was not correlated with serious morbidity (p = 0.914); however, GEARS score did predict EWL at 6 (p < 0.001) and 12 months (p < 0.001). All GEARS subcomponent scores, bimanual dexterity, depth perception, efficiency, force sensitivity, and robotic control were predictive of EWL at 6 months (p < 0.001) and 12 months (p < 0.001) on multivariable analysis.

CONCLUSION

For patients undergoing sleeve gastrectomy, surgical skill as assessed by the GEARS score was correlated with EWL, suggesting that better performance of a sleeve gastrectomy can result in improved postoperative weight loss.

Identification of Main Influencers of Surgical Efficiency and Variability Using Task-Level Objective Metrics: A Five-Year Robotic Sleeve Gastrectomy Case Series

Objective

Surgical efficiency and variability are critical contributors to optimal outcomes, patient experience, care team experience, and total cost to treat per disease episode. Opportunities remain to develop scalable, objective methods to quantify surgical behaviors that maximize efficiency and reduce variability. Such objective measures can then be used to provide surgeons with timely and user-specific feedbacks to monitor performances and facilitate training and learning. In this study, we used objective task-level analysis to identify dominant contributors toward surgical efficiency and variability across the procedural steps of robotic-assisted sleeve gastrectomy (RSG) over a five-year period for a single surgeon. These results enable actionable insights that can both complement those from population level analyses and be tailored to an individual surgeon's practice and experience.

Methods

Intraoperative video recordings of 77 RSG procedures performed by a single surgeon from 2015 to 2019 were reviewed and segmented into surgical tasks. Surgeon-initiated events when controlling the robotic-assisted surgical system were used to compute objective metrics. A series of multi-staged regression analysis were used to determine: if any specific tasks or patient body mass index (BMI) statistically impacted procedure duration; which objective metrics impacted critical task efficiency; and which task(s) statistically contributed to procedure variability.

Results

Stomach dissection was found to be the most significant contributor to procedure duration (β = 0.344, p< 0.001; R = 0.81, p< 0.001) followed by surgical inactivity and stomach stapling. Patient BMI was not found to be statistically significantly correlated with procedure duration (R = −0.01, p = 0.90). Energy activation rate, a robotic system event-based metric, was identified as a dominant feature in predicting stomach dissection duration and differentiating earlier and later case groups. Reduction of procedure variability was observed between earlier (2015-2016) and later (2017-2019) groups (IQR = 14.20 min vs. 6.79 min). Stomach dissection was found to contribute most to procedure variability (β = 0.74, p < 0.001).

Conclusions

A surgical task-based objective analysis was used to identify major contributors to surgical efficiency and variability. We believe this data-driven method will enable clinical teams to quantify surgeon-specific performance and identify actionable opportunities focused on the dominant surgical tasks impacting overall procedure efficiency and consistency.

Variation in Surgeon Proficiency Scores and Association With Digit Replantation Outcomes

IMPORTANCE

Risk-adjusted variation in surgeon outcomes has been traditionally explained by surgeon volume and hospital infrastructure, yet it is unclear how a surgeon's operative proficiency directly contributes to their patients' outcomes.

OBJECTIVE

To assess the variation of surgeons' operative proficiency and investigate its association with surgical outcomes.

DESIGN, SETTING, AND PARTICIPANTS

This case series was a retrospective analysis of all digit replantations and revascularizations at a single US university medical center between January 2000 and August 2020. Surgeons were assigned a proficiency score based on the expected procedure difficulty and outcomes from a sample of their cases. Surgeon proficiency scores were then used to determine associations with outcomes from subsequent cases. The expected difficulty of each case was calculated using a novel scoring system that applied pooled relative risks from a meta-analysis of risk factors for replantation and revascularization failure.

EXPOSURES

Digit replantation and revascularization.

MAIN OUTCOMES AND MEASURES

Digit survival at 1-month follow up (case success) and number of complications.

RESULTS

A total of 145 patients and 226 digits were treated by 11 surgeons with training in hand or microsurgery (mean [SD] age, 41.9 [15.2] years; 204 [90%] men); there were 116 replantations and 110 revascularizations. Surgeon proficiency scores ranged from 1.3 to 5.7, with a mean (SD) of 3.4 (1.4). Case success rates among surgeons varied from 20.0% to 90.5%, with a mean (SD) of 64.9%. Higher proficiency scores were associated with fewer case failures: each point increase was associated with 40% decreased odds of failure (odds ratio, 0.60; 95% CI, 0.38-0.94). Every 3-point increase in proficiency score was associated with 1 less complication (effect estimate, -0.29; 95% CI, -0.56 to 0.02). Surgeon proficiency score had a greater association with case failure than surgeon volume (16.7% vs 12.0%). The final model's association with case failure had an area under the receiver operating characteristics curve of 0.93.

CONCLUSIONS AND RELEVANCE

Operative proficiency varied widely among practicing surgeons and accounted for 17% of estimative ability for success of digit replantation and revascularization. Greater surgeon proficiency was associated with better outcomes, indicating that the value of surgical care may be optimized by improving surgeon proficiency.

Video-Based Coaching: Current Status and Role in Surgical Practice (Part 1) From the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee

Patient safety and outcomes are directly related to surgical performance. Surgical training emphasizes the importance of the surgeon in determining these outcomes. After training is complete, there is a lack of structured programs for surgeons to audit their skills and continue their individual development. There is a significant linear relationship between surgeon technical skill and surgical outcomes; however, measuring technical performance is difficult. Video-based coaching matches an individual surgeon in practice with a surgical colleague who has been trained in the core principles of coaching for individualizing instruction. It can provide objective assessment for teaching higher-level concepts, such as technical skills, cognitive skills, and decision-making. There are many benefits to video-based coaching. While the concept is gaining acceptance as a method of surgical education, it is still novel in clinical practice. As more surgeons look towards video-based coaching for quality improvement, a consistent definition of the program, goals, and metrics for assessment will be critical. This paper is a review on the status of the video-based coaching as it applies to practicing surgeons.

A new basic thoracoscopic surgical skill training and assessment system using automatic scoring techniques

PURPOSE

We report a new thoracoscopic surgical skill training and assessment system with automatic scoring techniques, the Huaxi Intelligent Thoracoscopic Skill Training and Assessment (HITSTA) system. We also evaluated the discriminative ability of this system compared to our conventional scoring method at our institution.

METHODS

We retrospectively collected training data of thoracic board-certified thoracic surgeons at West China Hospital, Sichuan University from January 1, 2018 to January 1, 2019. Surgeons were assessed by HITSTA system and human examiners simultaneously. Total scores were summed from 3 tasks (grasping with delivery, pattern cutting, and suture with knot). Bland-Altman analysis was used to test agreement of scores made by HITSTA system (automatic scoring) and human examiners (manual scoring). Differentiation ability was also compared between the two scoring methods.

RESULTS

Thirty-nine surgeons were recruited. Scores made by HITSTA system and human examiners were not consistent. For suture with knot, automatic scoring method could detect the score differences between different training status (trained: 26.92 ± 12.04, untrained: 19.85 ± 11.12; p = 0.026) and training duration (< 10 h: 20.67 ± 15.23,  ≥ 10 h: 31.92 ± 5.56; p = 0.003). For total scores, automatic scoring approach could discriminate between different training status (trained: 71.90 ± 12.63; untrained: 61.41 ± 13.87; p = 0.016) and training duration (< 10 h: 65.23 ± 15.31;  ≥ 10 h 77.23 ± 6.94; p = 0.046).

CONCLUSION

HITSTA system could discriminate the different levels of thoracoscopic surgical skills better than the traditional manual scoring method. Larger prospective studies are warranted to validate the differentiation ability of HITSTA system.

Fellowship training influences learning curves for laparoscopic sleeve gastrectomy

BACKGROUND

Laparoscopic sleeve gastrectomy (LSG) is the most commonly performed bariatric surgical procedure. Little is known about how surgeon training background influences the learning curve of this procedure. We examined operating times (OT), weight loss outcomes, and 30-day complications between surgeons with and without fellowship training in LSG. We hypothesize that post-residency training specific to LSG influences learning curves.

METHODS

Surgeons from a single institution were split into two groups: those who had not completed fellowship training in LSG (NF, n = 3), and those who had completed LSG specific training in fellowship (SGF, n = 3). OTs, BMI changes at 1 year, and 30-day readmissions, reoperations, and complications were extracted for the first 100 LSG cases of each surgeon. Data were analyzed in bins of 20 cases. Comparisons were made between cohorts within a bin and between adjacent bins of the same surgeon cohort. Logistic regression analyses were performed of OT and weight loss outcomes.

RESULTS

SGF surgeons showed no difference in OTs over their first 100 cases. NF surgeons had statistically significant increased OTs compared to SGF surgeons during their first 60 cases and progressively shortened OTs during that interval (109 min to 78 min, p < 0.001 for NF surgeons vs. 73 min to 69 min, SGF surgeons). NF surgeons had a significantly steeper slope for improvement in OT over case number. There was no correlation between case number and weight loss outcomes in either group, and no differences in 30-day outcomes between groups.

CONCLUSION

Surgeons who trained to perform LSG in fellowship demonstrate faster and consistent OR times on their initial independent LSG cases compared to surgeons who did not, with no correlation between case number and weight loss outcomes or safety profiles for either group. This suggests that learning curves for LSG are achieved during formal case-specific fellowship training.

Associations Between Video Evaluations of Surgical Technique and Outcomes of Laparoscopic Sleeve Gastrectomy

Importance

In any surgical procedure, various aspects of technique may affect patient outcomes. As new procedures enter practice, it is difficult to evaluate the association of each aspect of technique with patient outcomes.

Objective

To examine the associations between technique and outcomes in laparoscopic sleeve gastrectomy.

Design, Setting, and Participants

In this cohort study of bariatric surgery programs participating in a statewide surgical quality improvement collaborative, 30 surgeons submitted intraoperative videos from representative sleeve gastrectomies performed on 6915 patients with morbid obesity. These videos were reviewed by blinded peer surgeons on key technical elements, and 605 reviews were linked to sleeve gastrectomy outcomes of all of the surgeons' patients from January 1, 2015, to December 31, 2016.

Exposures

Surgeons' technical approaches to 5 controversial aspects of laparoscopic sleeve gastrectomy: dissection of the proximal stomach, sleeve caliber, sleeve anatomy, staple line reinforcement, and leak testing.

Main Outcomes and Measures

The 30-day outcomes were rate of postoperative hemorrhage and staple line leak. The 1-year outcomes were percentage of total weight lost and reflux severity (Gastroesophageal Reflux Disease Health-Related Quality of Life instrument).

Results

A total of 30 surgeons submitted 46 videos of operations performed on 6915 patients (mean [SD] age, 45.4 [11.7] years; 5494 [79.5%] female; 4706 [68.1%] White). Complete dissection of the proximal stomach was associated with reduced hemorrhage rates (higher ratings for complete mobilization of fundus were associated with a decrease in hemorrhage rate from 2.1% [25th percentile] to 1.0% [75th percentile], P = .01; higher ratings for visualization of the left crus were associated with a decrease in hemorrhage rate from 1.5% to 0.94%, P = .006; and higher ratings for complete division of the short gastrics were associated with a decrease in hemorrhage rate from 2.8% to 1.2%, P = .03). The reduction in hemorrhage rates came at the expense of higher leak rates (higher ratings for complete mobilization of fundus were associated with an increase in leak rate from 0.05% [25th percentile] to 0.16% [75th percentile], P < .001; higher ratings for visualization of the left crus were associated with an increase in leak rate from 0.1% to 0.2%, P = .003; and higher ratings for complete division of the short gastrics were associated with an increase in leak rate from 0.02% to 0.1%, P = .01). Surgeons who stapled more tightly to the bougie had smaller decreases in reflux than those who stapled less tightly (-2.0 to -1.3 on a 50-point scale, P = .002). Staple line reinforcement (buttressing and oversewing) was associated with a small (2 of 1000 cases) decrease in hemorrhage rates. Staple line buttressing was also associated with a similarly small increase in leak rates (1 of 1000 cases). Leak testing was associated with a statistically insignificant change in the staple line leak rate (0.16%-0.22%, P = .47).

Conclusions and Relevance

Variations in surgical technique can be measured by video review and are associated with differences in patient outcomes.

Feasibility of Surgeon-Delivered Audit and Feedback Incorporating Peer Surgical Coaching to Reduce Fistula Incidence following Cleft Palate Repair: A Pilot Trial

BACKGROUND

Improving surgeons' technical performance may reduce their frequency of postoperative complications. The authors conducted a pilot trial to evaluate the feasibility of a surgeon-delivered audit and feedback intervention incorporating peer surgical coaching on technical performance among surgeons performing cleft palate repair, in advance of a future effectiveness trial.

METHODS

A nonrandomized, two-arm, unblinded pilot trial enrolled surgeons performing cleft palate repair. Participants completed a baseline audit of fistula incidence. Participants with a fistula incidence above the median were allocated to an intensive feedback intervention that included selecting a peer surgical coach, observing the coach perform palate repair, reviewing operative video of their own surgical technique with the coach, and proposing and implementing changes in their technique. All others were allocated to simple feedback (receiving audit results). Outcomes assessed were proportion of surgeons completing the baseline audit, disclosing their fistula incidence to peers, and completing the feedback intervention.

RESULTS

Seven surgeons enrolled in the trial. All seven completed the baseline audit and disclosed their fistula incidence to other participants. The median baseline fistula incidence was 0.4 percent (range, 0 to 10.5 percent). Two surgeons were unable to receive the feedback intervention. Of the five remaining surgeons, two were allocated to intensive feedback and three to simple feedback. All surgeons completed their assigned feedback intervention. Among surgeons receiving intensive feedback, fistula incidence was 5.9 percent at baseline and 0.0 percent following feedback (adjusted OR, 0.98; 95 percent CI, 0.44 to 2.17).

CONCLUSION

Surgeon-delivered audit and feedback incorporating peer coaching on technical performance was feasible for surgeons.