Setting Benchmarks for the New User: Training on the Robotic Simulator

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

Standard Setting in Simulation-Based Training of Surgical Procedures: A Systematic Review

OBJECTIVE

This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures.

SUMMARY OF BACKGROUND

Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods.

METHODS

Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE.

RESULTS

Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence.

CONCLUSION

Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.

Fellowship training in robotic colorectal surgery within the current hospital setting: an achievable goal?

BACKGROUND

Although currently limited, the requirement for colorectal trainees to attain skills in robotic surgery is likely to increase due to further utilization of robotic platforms globally. The aim of the study is to describe the training programme utilized and assess outcomes of fellowship training in robotic colorectal surgery.

METHODS

A structured robotic training programme was generated across a tertiary hospital setting. Review of four prospectively maintained fellow operative logbooks was performed to assess caseload and skill acquisition. Operative and patient-related outcomes were compared with consultant trainer performed cases. Data were analysed using R with a P < 0.05 considered significant.

RESULTS

The structured robotic training scheme is a two-tiered system over a 12-month period. The trainer-directed pathway comprised of a robotic console safety course followed by cart-side assisting, a wet lab animal course, dual-console accreditation training course and onsite proctoring, prior to becoming an independent console surgeon. Over 2 years, 265 robotic (n = 143 primary/component surgeon) cases were undertaken with fellows A, B, C and D involved in 63, 77, 75 and 50 robotic colorectal cases, respectively. Individual learning curves revealed independent procedure competency at cases 11, 14, 15 and 12, respectively, for robotic anterior resection. There was no significant difference observed in operative time (P = 0.39), blood loss (P = 0.41), lymph node harvest (P = 0.35), conversion rates (2% versus 4%), anastomotic leaks (1% versus 3%) and R0 resection rates (100% versus 98% colonic, 96% versus 96% rectal, P = 0.48) between surgical fellows and consultant trainers. Clavien-Dindo(III-IV) complications were similar (10% versus 6%,P = 0.25) with no mortalities encountered.

CONCLUSION

It is feasible and safe to train fellows in robotic colorectal surgery without compromise of operative- and patient-related outcomes.

Are basic robotic surgical skills transferable from the simulator to the operating room? A randomized, prospective, educational study

INTRODUCTION

We aimed to assess the transferability of basic robotic skills from the simulator to the operating room (OR) while performing robotic-assisted radical prostatectomy (RARP).

METHODS

Fourteen urology residents were randomized into two groups: group A was required to practice three sessions (nine tasks each) on the simulator, whereas group B was required to practice (same nine tasks) until they reached competency. Both groups were recorded while practicing on the da Vinci Surgical Skills Simulator. Both groups were then recorded while performing bladder mobilization during RARP. Senior residents from both groups were also recorded while performing urethro-vesical anastomosis during RARP. Recordings were assessed blindly using the validated Global Evaluative Assessment of Robotic Skills (GEARS) tool by C-SATS. Spearman's correlation coefficient (rho) was used to assess correlation between GEARS scores from practice sessions on the da Vinci Simulator and the GEARS scores from bladder mobilization and urethro-vesical anastomosis during RARP.

RESULTS

There was no difference in total GEARS scores between the two groups in the OR. Total GEARS scores for "ring and rail 2" and "suture sponge" tasks correlated with the total GEARS scores during urethro-vesical anastomosis (rho=0.86, p=0.007; rho=0.90, p=0.002, respectively). GEARS' efficiency component during "energy and dissection" task on the da Vinci Simulator correlated with GEARS' efficiency component during bladder mobilization (rho=0.62, p=0.03). GEARS' force sensitivity component during "ring and rail 2" and "dots and needles" tasks on the da Vinci Simulator correlated with GEARS' force sensitivity component during bladder mobilization (rho=0.58, p=0.047; rho =0.65, p=0.02, respectively).

CONCLUSIONS

Objective assessments of urology residents on the da Vinci Surgical Skills Simulator tasks ring and rail 2 and suture sponge correlated with their objective assessments of bladder mobilization and urethro-vesical anastomosis. Therefore, basic robotic skills could be transferred from the simulator to the OR.