Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men

Current Standards for Training in Robot-assisted Surgery and Endourology: A Systematic Review

BACKGROUND AND OBJECTIVE

Different training programs have been developed to improve trainee outcomes in urology. However, evidence on the optimal training methodology is sparse. Our aim was to provide a comprehensive description of the training programs available for urological robotic surgery and endourology, assess their validity, and highlight the fundamental elements of future training pathways.

METHODS

We systematically reviewed the literature using PubMed/Medline, Embase, and Web of Science databases. The validity of each training model was assessed. The methodological quality of studies on metrics and curricula was graded using the MERSQI scale. The level of evidence (LoE) and level of recommendation for surgical curricula were awarded using the educational Oxford Centre for Evidence-Based Medicine classification.

KEY FINDINGS AND LIMITATIONS

A total of 75 studies were identified. Many simulators have been developed to aid trainees in mastering skills required for both robotic and endourology procedures, but only four demonstrated predictive validity. For assessment of trainee proficiency, we identified 18 in robotics training and six in endourology training; however, the majority are Likert-type scales. Although proficiency-based progression (PBP) curricula demonstrated superior outcomes to traditional training in preclinical settings, only four of six (67%) in robotics and three of nine (33%) in endourology are PBP-based. Among these, the Fundamentals of Robotic Surgery and the SIMULATE curricula have the highest LoE (level 1b). The lack of a quantitative synthesis is the main limitation of our study.

CONCLUSIONS AND CLINICAL IMPLICATIONS

Training curricula that integrate simulators and PBP methodology have been introduced to standardize trainee outcomes in robotics and endourology. However, evidence regarding their educational impact remains restricted to preclinical studies. Efforts should be made to expand these training programs to different surgical procedures and assess their clinical impact.

Measures of performance and proficiency in robotic assisted surgery: a systematic review

Robotic assisted surgery (RAS) has seen a global rise in adoption. Despite this, there is not a standardised training curricula nor a standardised measure of performance. We performed a systematic review across the surgical specialties in RAS and evaluated tools used to assess surgeons' technical performance. Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published on or after January 2020-January 2022. Observational studies and RCTs were included; review articles and systematic reviews were excluded. The papers' quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. The initial search yielded 1189 papers of which 72 fit the eligibility criteria. 27 unique performance metrics were identified. Global assessments were the most common tool of assessment (n = 13); the most used was GEARS (Global Evaluative Assessment of Robotic Skills). 11 metrics (42%) were objective tools of performance. Automated performance metrics (APMs) were the most widely used objective metrics whilst the remaining (n = 15, 58%) were subjective. The results demonstrate variation in tools used to assess technical performance in RAS. A large proportion of the metrics are subjective measures which increases the risk of bias amongst users. A standardised objective metric which measures all domains of technical performance from global to cognitive is required. The metric should be applicable to all RAS procedures and easily implementable. Automated performance metrics (APMs) have demonstrated promise in their wide use of accurate measures.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Development and Validation of an Objective Scoring Tool for Robot-Assisted Partial Nephrectomy: Scoring for Partial Nephrectomy

Objective: To develop a structured and objective scoring tool for assessment of robot-assisted partial nephrectomy (RAPN): Scoring for Partial Nephrectomy (SPaN). Materials and Methods: Content development: RAPN was deconstructed into 6 domains by a multi-institutional panel of 10 expert robotic surgeons. Performance on each domain was represented on a Likert scale of 1 to 5, with specific descriptions of anchors 1, 3, and 5. Content validation: The Delphi methodology was utilized to achieve consensus about the description of each anchor for each domain in terms of appropriateness of the skill assessed, objectiveness, clarity, and unambiguous wording. The content validity index (CVI) of ≥0.75 was set as cutoff for consensus. Reliability: 15 de-identified videos of RAPN were utilized to determine the inter-rater reliability using linearly weighted percent agreement, and Construct validation of SPaN was described in terms of median scores and odds ratios. Results: The expert panel reached consensus (CVI ≥0.75) after 2 rounds. Consensus was achieved for 36 (67%) statements in the first round and 18 (33%) after the second round. The final six-domain SPaN included Exposure of the kidney; Identification and dissection of the ureter and gonadal vessels; Dissection of the hilum; Tumor localization and exposure; Clamping and tumor resection; and Renorrhaphy. The linearly weighted percent agreement was >0.75 for all domains. There was no difference between median scores for any domain between attendings and trainees. Conclusion: Despite the lack of significant construct validity, SPaN is a structured, reliable, and procedure-specific tool that can objectively assesses technical proficiency for RAPN.

Innovations in Urologic Surgical Training

PURPOSE OF REVIEW

This review aims to summarize innovations in urologic surgical training in the past 5 years.

RECENT FINDINGS

Many assessment tools have been developed to objectively evaluate surgical skills and provide structured feedback to urologic trainees. A variety of simulation modalities (i.e., virtual/augmented reality, dry-lab, animal, and cadaver) have been utilized to facilitate the acquisition of surgical skills outside the high-stakes operating room environment. Three-dimensional printing has been used to create high-fidelity, immersive dry-lab models at a reasonable cost. Non-technical skills such as teamwork and decision-making have gained more attention. Structured surgical video review has been shown to improve surgical skills not only for trainees but also for qualified surgeons. Research and development in urologic surgical training has been active in the past 5 years. Despite these advances, there is still an unfulfilled need for a standardized surgical training program covering both technical and non-technical skills.

Training in robot-assisted surgery

PURPOSE OF REVIEW

The aim of this narrative review is to identify the currently available training programs in the field of robotic-assisted surgery.

RECENT FINDINGS

The lack of well trained robotic surgeons represents the main obstacle facing robotic surgery. This calls for development of structured robotic training programs in order to improve patient safety and outcomes. Structured curriculum should consist of theoretical training (e-learning, case observation), preclinical simulation-based training (virtual reality simulation, dry and wet lab), clinical modular training, and final evaluation. Tele-mentoring might reveal mandatory in future curricula development. To date, several training programs are currently available for robotic surgery. However, the majority lack of clinical modular training that consists of progressive, proficiency-based training through surgical steps with increasing levels of complexity. Moreover, many of these curricula lack validation.

SUMMARY

This recent literature analysis suggests that there is an urgent need to develop and validate new structured training curricula for robotic surgery. This allows to improve the skills of the surgeons and of their team and to prevent patients from being used as a training module optimizing their safety.

Quality Indicators for Bladder Cancer Services: A Collaborative Review

CONTEXT

There is a lack of accepted consensus on what should constitute appropriate quality-of-care indicators for bladder cancer.

OBJECTIVE

To evaluate the optimal management of bladder cancer and propose quality indicators (QIs).

EVIDENCE ACQUISITION

A systematic review was performed to identify literature on current optimal management and potential quality indicators for both non-muscle-invasive (NMIBC) and muscle-invasive (MIBC) bladder cancer. A panel of experts was convened to select a recommended list of QIs.

EVIDENCE SYNTHESIS

For NMIBC, preoperative QIs include tobacco cessation counselling and appropriate imaging before initial transurethral resection of bladder tumour (TURBT). Intraoperative QIs include administration of antibiotics, proper safe conduct of TURBT using a checklist, and performing restaging TURBT with biopsy of the prostatic urethra in appropriate cases. Postoperative QIs include appropriate receipt of perioperative adjuvant therapy, risk-stratified surveillance, and appropriate decision to change therapy when indicated (eg, bacillus Calmette-Guerin [BCG] unresponsive). For MIBC, preoperative QIs include multidisciplinary care, selection for candidates for continent urinary diversion, receipt of neoadjuvant cisplatin-based chemotherapy, time to commencing radical treatment, consideration of trimodal therapy as a bladder-sparing alternative in select patients, preoperative counselling with stoma marking, surgical volume of radical cystectomy, and enhanced recovery after surgery protocols. Intraoperative QIs include adequacy of lymphadenectomy, blood loss, and operative time. Postoperative QIs include prospective standardised monitoring of morbidity and mortality, negative surgical margins for pT2 disease, appropriate surveillance after primary treatment, and adjuvant cisplatin-based chemotherapy in appropriate cases. Participation in clinical trials was highlighted as an important component indicating high quality of care.

CONCLUSIONS

We propose a set of QIs for both NMIBC and MIBC based on established clinical guidelines and the available literature. Although there is currently a lack of level 1 evidence for the benefit of implementing these QIs, we believe that the measurement of these QIs could aid in the improvement and benchmarking of optimal care for bladder cancer.

PATIENT SUMMARY

After a systematic review of existing guidelines and literature, a panel of experts has recommended a set of quality indicators that can help providers and patients measure and strive towards optimal outcomes for bladder cancer care.