Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

Surgical quality assessment of critical view of safety in 283 laparoscopic cholecystectomy videos by surgical residents and surgeons

INTRODUCTION

Surgical quality assessment has improved the efficacy and efficiency of surgical training and has the potential to optimize the surgical learning curve. In laparoscopic cholecystectomy (LC), the critical view of safety (CVS) can be assessed with a 6-point competency assessment tool (CAT), a task commonly performed by experienced surgeons. The aim of this study is to determine the capability of surgical residents to perform this assessment.

METHODS

Both surgeons and surgical residents assessed unedited LC videos using a 6-point CVS, a CAT, using an online video assessment platform. The CAT consists of the following three criteria: 1. clearance of hepatocystic triangle, 2. cystic plate, and 3. two structures connect to the gallbladder, with a maximum of 2 points available for each criterion. A higher score indicates superior surgical performance. The intraclass correlation coefficient (ICC) was employed to assess the inter-rater reliability between surgeons and surgical residents.

RESULTS

In total, 283 LC videos were assessed by 19 surgeons and 31 surgical residents. The overall ICC for all criteria was 0.628. Specifically, the ICC scores were 0.504 for criterion 1, 0.639 for criterion 2, and 0.719 for the criterion involving the two structures connected to the gallbladder. Consequently, only the criterion regarding clearance of the hepatocystic triangle exhibited fair agreement, whereas the other two criteria, as well as the overall scores, demonstrated good agreement. In 71% of cases, both surgeons and surgical residents scored a total score either ranging from 0 to 4 or from 5 to 6.

CONCLUSION

Compared to the gold standard, i.e., the surgeons' assessments, surgical residents are equally skilled at assessing critical view of safety (CVS) in laparoscopic cholecystectomy (LC) videos. By incorporating video-based assessments of surgical procedures into their training, residents could potentially enhance their learning pace, which may result in better clinical outcomes.

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

Training and assessment for colorectal surgery and appendicectomy- a systematic review

AIM

There is currently an increased focus on competency-based training, in which training and assessment play a crucial role. The aim of this systematic review is to create an overview of hands-on training methods and assessment tools for appendicectomy and colon and rectal surgery procedures using either an open, laparoscopic or robot-assisted approach.

METHOD

A systematic review of Medline, Embase, Cochrane and Scopus databases was conducted following the PRISMA guidelines. We conducted the last search on 9 March 2023. All published papers describing hands-on training, evaluation of performance data and development of assessment tools were eligible. The quality of studies and the validity evidence of assessment tools are reported.

RESULTS

Fifty-one studies were identified. Laparoscopic assessment tools are abundant, but the literature still lacks good-quality assessment tools for open appendicectomy, robotic colectomy and open rectal surgery. Overall, there is a lack of discussion regarding the establishment of pass/fail standards and the consequences of assessment. Virtual reality simulation is used more for appendicectomy than colorectal procedures. Only a few of the studies investigating training were of acceptable quality. There is a need for high-quality studies in open and robotic-assisted colon surgery and all approaches to rectal surgery.

CONCLUSION

This review provides an overview of current training methods and assessment tools and identifies where more research is needed based on the quality of the studies and the current validity evidence.

Variation in hospital performances after colorectal cancer surgery: A case-mix adjusted Dutch population based study

BACKGROUND

This study aimed to investigate hospital variability in postoperative mortality and anastomotic leakage (AL) after colorectal cancer surgery, as well as the association with hospital volume and teaching status.

MATERIALS AND METHODS

This nationwide population based study derived data from CRC patients who underwent a surgical resection with primary anastomosis from the Netherlands Cancer Registry between 2015 and 2020. Primary outcomes were 90-day mortality and AL for colon cancer (CC) patients, and AL for rectal cancer (RC) patients. Logistic regression modelling was used to evaluate the association between case-mix factors and hospital volume. Variability in outcomes between hospitals was analysed with Poisson regression.

RESULTS

This study included 44,101 CRC patients, comprising 35,164 CC patients, and 8937 RC patients. In the CC cohort, the unadjusted rates of AL ranged from 2.6 % to 14.4 %, and the unadjusted 90-day mortality rates ranged from 0.0 % to 6.7 %. In the RC cohort, the unadjusted rates of AL ranged from 0.0 % to 28.6 %. After case-mix adjustment, two hospitals performed significantly worse than expected regarding 90-day mortality in the CC cohort, and in both CC and RC cohorts, significant outliers were observed concerning AL. Amongst CC patients, low case volume (OR 1.26 95%CI 1.08-1.46) was significantly associated with AL.

CONCLUSION

Statistically significant variations in hospital performance were observed among Dutch hospitals after CRC surgery, but this effect could not be entirely attributed to hospitals' teaching status. Nevertheless, concentrating care has the potential to improve outcomes by enhancing individual surgical performance and optimizing care pathways.

Nationwide standardization of minimally invasive right hemicolectomy for colon cancer and development and validation of a video-based competency assessment tool (the Right study)

BACKGROUND

Substantial variation exists when performing a minimally invasive right hemicolectomy (MIRH) due to disparities in training, expertise and differences in implementation of innovations. This study aimed to achieve national consensus on an optimal and standardized MIRH technique for colon cancer and to develop and validate a video-based competency assessment tool (CAT) for MIRH.

METHOD

Statements covering all elements of MIRH were formulated. Subsequently, the Delphi technique was used to reach consensus on a standardized MIRH among 76 colorectal surgeons from 43 different centres. A CAT was developed based on the Delphi results. Nine surgeons assessed the same 12 unedited full-length videos using the CAT, allowing evaluation of the intraclass correlation coefficient (ICC).

RESULTS

After three Delphi rounds, consensus (≥80% agreement) was achieved on 23 of the 24 statements. Consensus statements included the use of low intra-abdominal pressure, detailed anatomical outline how to perform complete mesocolic excision with central vascular ligation, the creation of an intracorporeal anastomosis, and specimen extraction through a Pfannenstiel incision using a wound protector. The CAT included seven consecutive steps to measure competency of the MIRH and showed high consistency among surgeons with an overall ICC of 0.923.

CONCLUSION

Nationwide consensus on a standardized and optimized technique of MIRH was reached. The CAT developed showed excellent interrater reliability. These achievements are crucial steps to an ongoing nationwide quality improvement project (the Right study).

Effectiveness of, and Satisfaction with, a Microsurgical Testicular Sperm Extraction Knowledge and Skills Masterclass for a World-Wide Audience

PURPOSE

This is the first study to assess the impact of an online microsurgical testicular sperm extraction (mTESE) masterclass. We: 1) describe the masterclass's scientific content; 2) appraise the participants' acquisition of knowledge; 3) gauge whether the extent of improvement of the participants' knowledge/skills was influenced by demographic/professional attributes; and 4) evaluate the participants' satisfaction.

MATERIALS AND METHODS

This masterclass comprised five didactic lectures followed by 4 case discussions. Online surveys assessed the above objectives using a baseline questionnaire including demographics and past mTESE experience/training, a 24-question pre- and post-quiz, and a satisfaction questionnaire.

RESULTS

Participants were between 20-70 years old, with 80.37% males, mainly from Asia, Africa, and Europe, from clinical backgrounds (69.3%), and in public practice (64.4%). Half the sample reported no past mTESE training and very low skills, ≈60% wanted considerably more training, and 50% felt that good training was not readily available. Satisfaction was 98% to >99%. Pre- and post-quiz comparisons confirmed remarkable improvements in knowledge/skills, exhibiting five striking characteristics. Improvements were a) Broad i.e., across 19 of the 24 mTESE questions; b) Deep, of magnitude, as pre-/post-quiz scores improved from mean 13.71±4.13 to 17.06±4.73; c) Highly significant, consistently with p-values <0.001; d) Inclusive i.e., all participants enhanced their mTESE knowledge/skills regardless of demographic/professional attributes; and, e) Differential, e.g., non-clinical/clinical participants improved, but the former improved relatively significantly more, those with ≤5-year experience improved significantly more than those with >5-year, those in public practice significantly more than private practice participants, and those with lower self-rating in performing mTESE significantly more than those with higher self-rating.

CONCLUSIONS

The masterclass was successful with very high satisfaction levels, and markedly improved mTESE knowledge/skills among the participants. Global Andrology Forum's model can be adopted by organizations with similar goals. Future research needs to evaluate such training to develop a practically non-existent evidence base.