Validation of a Simulation Model for Robotic Sacrocolpopexy

Validation of a Simulation Model for Robotic Myomectomy

STUDY OBJECTIVE

Several simulation models have been evaluated for gynecologic procedures such as hysterectomy, but there are limited published data for myomectomy. This study aimed to assess the validity of a low-cost robotic myomectomy model for surgical simulation training.

DESIGN

Prospective cohort simulation study.

SETTING

Surgical simulation laboratory.

PARTICIPANTS

Twelve obstetrics and gynecology residents and 4 fellowship-trained minimally invasive gynecologic surgeons were recruited for a 3:1 novice-to-expert ratio.

INTERVENTIONS

A robotic myomectomy simulation model was constructed using <$5 worth of materials: a foam cylinder, felt, a stress ball, bandage wrap, and multipurpose sealing wrap. Participants performed a simulation task involving 2 steps: fibroid enucleation and hysterotomy repair. Video-recorded performances were timed and scored by 2 blinded reviewers using the validated Global Evaluative Assessment of Robotic Skills (GEARS) scale (5-25 points) and a modified GEARS scale (5-40 points), which adds 3 novel domains specific to robotic myomectomy. Performance was also scored using predefined task errors. Participants completed a post-task questionnaire assessing the model's realism and utility.

MEASUREMENTS AND MAIN RESULTS

Median task completion time was shorter for experts than novices (9.7 vs 24.6 min, p = .001). Experts scored higher than novices on both the GEARS scale (median 23 vs 12, p = .004) and modified GEARS scale (36 vs 20, p = .004). Experts made fewer task errors than novices (median 15.5 vs 37.5, p = .034). For interrater reliability of scoring, the intraclass correlation coefficient was calculated to be 0.91 for the GEARS assessment, 0.93 for the modified GEARS assessment, and 0.60 for task errors. Using the contrasting groups method, the passing mark for the simulation task was set to a minimum modified GEARS score of 28 and a maximum of 28 errors. Most participants agreed that the model was realistic (62.5%) and useful for training (93.8%).

CONCLUSION

We have demonstrated evidence supporting the validity of a low-cost robotic myomectomy model. This simulation model and the performance assessments developed in this study provide further educational tools for robotic myomectomy training.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Development of a low-cost congenital abdominal wall defect simulator (wall-go) for undergraduate medical education: a validation study

BACKGROUND

Congenital Anomalies were responsible for 303,000 deaths in the neonatal period, according to the WHO, they are among the world's top 20 causes of morbidity and mortality. Expensive simulators demonstrate several diseases, but few are related to congenital anomalies. This study aims to develop, validate, and evaluate low-cost simulator models (WALL-GO) of the most common abdominal wall defects, gastroschisis, and omphalocele, to enable diagnosis through an accessible tool with study value and amenable to replication.

METHODS

Market research was conducted to find materials to build low-cost models. The researchers built the model and underwent validation assessment of the selected experts who scored five or more in the adapted Fehring criteria. The experts were assessed through a 5-point Likert scale to 7 statements (S1-7). Statements were assigned values according to relevance in face and transfer validities. Concomitantly, the model was also evaluated by students from 1st to 5th year with the same instruments. Content Validity Indexes (CVIs) were considered validated between groups with concordance greater than 90%. Text feedback was also collected. Each statement was subjected to Fisher's Exact Test.

RESULTS

Gastroschisis and omphalocele model costs were US $15 and US $27, respectively. In total, there were 105 simulator evaluators. 15 experts were selected. Of the 90 students, there were 16 (1st year), 22 (2nd), 16 (3rd), 22 (4th), and 14 (5th). Students and experts obtained CVI = 96.4% and 94.6%, respectively. The CVIs of each statement were not significantly different between groups (p < 0,05).

CONCLUSIONS

The WALL-GO models are suitable for use and replicable at a manufacturable low cost. Mannequins with abdominal wall defects are helpful in learning to diagnose and can be applied in teaching and training health professionals in developing and low-income countries.

Surgical education in urogynecology from low fidelity to virtual reality: Systematic review

IMPORTANCE

Increase dissemination of educational tools in urogynecology.

OBJECTIVE

Describe the effectiveness and public availability of published educational tools for urogynecologic surgery.

STUDY DESIGN

A systematic review was conducted by searching MEDLINE, EMBASE, Cochrane Library, and Web of Science from 1946 to 2023 for articles describing educational tools in urogynecology. There were no restrictions on study design or language. Data were extracted in duplicate using a standardized piloted extraction form, and outcomes were combined descriptively.

RESULTS

2997 titles, 457 abstracts, and 97 full-text articles were analyzed. Of the 97 interventions included, 43 were manuscripts and 54 were conference abstracts. The median study quality was low, with a moderate risk of bias. Six intervention categories were identified: didactics, animal models, cadavers, static models, extended reality (XR), and multimodal workshops. Didactics were subjectively useful for teaching pelvic anatomy and diseases and improving surgical techniques. If good quality, animal models and cadavers provided visual and tactile learning and assessed performance in real-time. Animal models were also anatomically realistic and useful at half the cost of cadavers. Static models and XR improved confidence, knowledge, skills, and error rates despite lack of realism and accurate tissue texture in some models and steep learning curve with XR. Only four models were commercially accessible. Most studies did not assess long-term (>6 months) retention.

CONCLUSION

All educational modalities for urogynecologic surgery are largely realistic and increase participant satisfaction, preparedness, knowledge, skills, and likelihood of use. But only 40% advanced to manuscripts, and even fewer (<5%) were widely available.

What credentials are required for robotic-assisted surgery in reconstructive and functional urology?

Introduction

The increasing popularity of robotic assisted surgery (RAS) as it is implemented in to sub specialities poses many challenges to ensuring standards in quality and safety. The area of Reconstructive and Functional Urology (RFU) has a wide range and largely complex heterogeneous procedures. In recent years RFU has started to incorporate RAS as the primary method to undertake these procedures due to improved vision, dexterity, and access to deep cavities. To ensure patient safety majority of institutions maintain minimal requirements to operate using RAS however across specialities and institutions these greatly vary.

Methods

A narrative review of all the relevant papers known to the author was conducted.

Results

Specific challenges facing RFU is the inability to rely on case numbers as a surrogate means to measure competency as well the ongoing consideration of how to differentiate between surgeons with robotic training and those with the clinical experience specific to RFU.

Conclusion

This review explores current models of training and credentialling and assess how it can be adapted to suggest a standardised guideline for RFU to ensure the highest standards of patient care.