Virtually Competent: A Comparative Analysis of Virtual Reality and Dry-Lab Robotic Simulation Training

Robotic hepaticojejunostomy training in novices using robotic simulation and dry-lab suturing (ROSIM): randomized controlled crossover trial

BACKGROUND

Robotic suturing training is in increasing demand and can be done using suture-pads or robotic simulation training. Robotic simulation is less cumbersome, whereas a robotic suture-pad approach could be more effective but is more costly. A training curriculum with crossover between both approaches may be a practical solution. However, studies assessing the impact of starting with robotic simulation or suture-pads in robotic suturing training are lacking.

METHODS

This was a randomized controlled crossover trial conducted with 20 robotic novices from 3 countries who underwent robotic suturing training using an Intuitive Surgical® X and Xi system with the SimNow (robotic simulation) and suture-pads (dry-lab). Participants were randomized to start with robotic simulation (intervention group, n = 10) or suture-pads (control group, n = 10). After the first and second training, all participants completed a robotic hepaticojejunostomy (HJ) in biotissue. Primary endpoint was the objective structured assessment of technical skill (OSATS) score during HJ, scored by two blinded raters. Secondary endpoints were force measurements and a qualitative analysis. After training, participants were surveyed regarding their preferences.

RESULTS

Overall, 20 robotic novices completed both training sessions and performed 40 robotic HJs. After both trainings, OSATS was scored higher in the robotic simulation-first group (3.3 ± 0.9 vs 2.5 ± 0.8; p = 0.049), whereas the median maximum force (N) (5.0 [3.2-8.0] vs 3.8 [2.3-12.8]; p = 0.739) did not differ significantly between the groups. In the survey, 17/20 (85%) participants recommended to include robotic simulation training, 14/20 (70%) participants preferred to start with robotic simulation, and 20/20 (100%) to include suture-pad training.

CONCLUSION

Surgical performance during robotic HJ in robotic novices was significantly better after robotic simulation-first training followed by suture-pad training. A robotic suturing curriculum including both robotic simulation and dry-lab suturing should ideally start with robotic simulation.

Development and validation of metrics for a new RAPN training model

Robot-assisted partial nephrectomy (RAPN) is a complex and index procedure that urologists need to learn how to perform safely. No validated performance metrics specifically developed for a RAPN training model (TM) exist. A Core Metrics Group specifically adapted human RAPN metrics to be used in a newly developed RAPN TM, explicitly defining phases, steps, errors, and critical errors. A modified Delphi meeting concurred on the face and content validation of the new metrics. One hundred percent consensus was achieved by the Delphi panel on 8 Phases, 32 Steps, 136 Errors and 64 Critical Errors. Two trained assessors evaluated recorded video performances of novice and expert RAPN surgeons executing an emulated RAPN in the newly developed TM. There were no differences in procedure Steps completed by the two groups. Experienced RAPN surgeons made 34% fewer Total Errors than the Novice group. Performance score for both groups was divided at the median score using Total Error scores, into HiError and LoError subgroups. The LowErrs Expert RAPN surgeons group made 118% fewer Total Errors than the Novice HiErrs group. Furthermore, the LowErrs Expert RAPN surgeons made 77% fewer Total Errors than the HiErrs Expert RAPN surgeons. These results established construct and discriminative validity of the metrics. The authors described a novel RAPN TM and its associated performance metrics with evidence supporting their face, content, construct, and discriminative validation. This report and evidence support the implementation of a simulation-based proficiency-based progression (PBP) training program for RAPN.

Validation of a Simulation Model for Robotic Myomectomy

STUDY OBJECTIVE

Several simulation models have been evaluated for gynecologic procedures such as hysterectomy, but there are limited published data for myomectomy. This study aimed to assess the validity of a low-cost robotic myomectomy model for surgical simulation training.

DESIGN

Prospective cohort simulation study.

SETTING

Surgical simulation laboratory.

PARTICIPANTS

Twelve obstetrics and gynecology residents and 4 fellowship-trained minimally invasive gynecologic surgeons were recruited for a 3:1 novice-to-expert ratio.

INTERVENTIONS

A robotic myomectomy simulation model was constructed using <$5 worth of materials: a foam cylinder, felt, a stress ball, bandage wrap, and multipurpose sealing wrap. Participants performed a simulation task involving 2 steps: fibroid enucleation and hysterotomy repair. Video-recorded performances were timed and scored by 2 blinded reviewers using the validated Global Evaluative Assessment of Robotic Skills (GEARS) scale (5-25 points) and a modified GEARS scale (5-40 points), which adds 3 novel domains specific to robotic myomectomy. Performance was also scored using predefined task errors. Participants completed a post-task questionnaire assessing the model's realism and utility.

MEASUREMENTS AND MAIN RESULTS

Median task completion time was shorter for experts than novices (9.7 vs 24.6 min, p = .001). Experts scored higher than novices on both the GEARS scale (median 23 vs 12, p = .004) and modified GEARS scale (36 vs 20, p = .004). Experts made fewer task errors than novices (median 15.5 vs 37.5, p = .034). For interrater reliability of scoring, the intraclass correlation coefficient was calculated to be 0.91 for the GEARS assessment, 0.93 for the modified GEARS assessment, and 0.60 for task errors. Using the contrasting groups method, the passing mark for the simulation task was set to a minimum modified GEARS score of 28 and a maximum of 28 errors. Most participants agreed that the model was realistic (62.5%) and useful for training (93.8%).

CONCLUSION

We have demonstrated evidence supporting the validity of a low-cost robotic myomectomy model. This simulation model and the performance assessments developed in this study provide further educational tools for robotic myomectomy training.

Simulation-based training for endoscopic mitral valve repair: the impact on basic surgical skills for placement of sutures at mitral valve annulus during 2-h training workshop

OBJECTIVES

The aim of this study was to evaluate the impact of simulation-based training on surgical skills during 2-h learning labs during surgical annual meeting.

METHODS

During the 36th European Association of Cardiothoracic Surgery annual meeting a learning drylab for simulation-based training for endoscopic mitral valve repair was set up. For this purpose, a validated high-fidelity endoscopic mitral valve surgery simulator and a validated suturing map were used. The training lasted 2 h. Technical pre- and post-assessment were carried out based on time and accuracy to place a suture at the posterior mitral valve annulus. The suture had to be placed within 60 s. The suture was considered anatomically correct if it entered and exited the annulus at the designated place (on the posterior annulus) and accurate if placed within the right width (8-12 mm).

RESULTS

In total, 46 participants were included in this study, of whom 18 (38%) were experienced/staff surgeons, 23 (51%) fellows and 5 (11%) residents. Before the training, 48% of the participants failed to place any suture for pre-assessment. After completing the training, 100% of the participants succeeded in placing an anatomically correct suture. There was a significant improvement in the time taken [pre-assessment mean 45 (standard deviation: 25) s vs post-assessment mean 18 (standard deviation: 12) s, P < 0.001] and the accuracy to place a suture in the mitral valve annulus after completing the training (pre-assessment 32.6% vs post-assessment 65.2%, P < 0.001).

CONCLUSIONS

This study shows a significant improvement in endoscopic skills for mitral valve surgery after completing a 2-h training with a high-fidelity endoscopic mitral valve surgery simulator. This suggests that simulation trainings during scientific annual meetings are effective on surgical skills.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Does Extended Reality Simulation Improve Surgical/Procedural Learning and Patient Outcomes When Compared With Standard Training Methods?: A Systematic Review

INTRODUCTION

The use of extended reality (XR) technologies, including virtual, augmented, and mixed reality, has increased within surgical and procedural training programs. Few studies have assessed experiential learning- and patient-based outcomes using XR compared with standard training methods.

METHODS

As a working group for the Society for Simulation in Healthcare, we used Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and a PICO strategy to perform a systematic review of 4238 articles to assess the effectiveness of XR technologies compared with standard training methods. Outcomes were grouped into knowledge, time-to-completion, technical proficiency, reactions, and patient outcomes. Because of study heterogeneity, a meta-analysis was not feasible.

RESULTS

Thirty-two studies met eligibility criteria: 18 randomized controlled trials, 7 comparative studies, and 7 systematic reviews. Outcomes of most studies included Kirkpatrick levels of evidence I-III (reactions, knowledge, and behavior), while few reported level IV outcomes (patient). The overall risk of bias was low. With few exceptions, included studies showed XR technology to be more effective than standard training methods in improving objective skills and performance, shortening procedure time, and receiving more positive learner ratings. However, XR use did not show significant differences in gained knowledge.

CONCLUSIONS

Surgical or procedural XR training may improve technical skill development among trainees and is generally favored over standard training methods. However, there should be an additional focus on how skill development translates to clinically relevant outcomes. We recommend longitudinal studies to examine retention and transfer of training to clinical settings, methods to improve timely, adaptive feedback for deliberate practice, and cost analyses.

Transfer of acquired practical skills from dry lab into live surgery using the avatera robotic system: An experimental study

OBJECTIVE

To evaluate the transfer of the practical skills of robot-assisted surgery acquired in the dry-lab into a real live experimental setting for performing upper and lower urinary tract surgeries.

MATERIAL AND METHODS

An in vivo experimental study design was utilized. Six urology trainees and fellows; two 2nd year trainees with no previous exposure to laparoscopic surgery (Group 1), two 4th year residents with medium exposure to laparoscopic surgery (Group 2) and two fellows trained to perform laparoscopic surgeries (Group 3) performed ureteral reimplantation into the bladder, pyeloplasty, and radical nephrectomy on three female pigs under general anesthesia. Prior to performing the requested procedures, each participant completed 10-14 h dry-lab robotic training acquiring skills in basic surgical tasks, such as suturing, cutting and needle passage. The recorded variables were the successful completion of the procedures, the console time, and the time to perform different steps and major complications.

RESULTS

All procedures were completed successfully by all groups except the pyeloplasty by group 1 which was complicated by bleeding from the renal vein, and the procedure was abandoned. Group 3 achieved shorter console time for all successfully completed procedures and for separate surgical steps compared to all groups, followed by Group 2. The slowest group for all procedures and steps analyzed was Group 3.

CONCLUSIONS

Although further clinical evidence is needed, the robotic-assisted urological procedures and the most challenging steps could be performed safely and effectively after proper training in the dry lab under mentor supervision according to our study.

Extended reality in surgical education: A systematic review

BACKGROUND

This review aims to evaluate the effectiveness of extended reality-based training in surgical education.

METHODS

This systematic review was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

A total of 33 studies were included in the qualitative analysis. Nine studies evaluated the effectiveness of virtual reality-based training against no substitutional training. Seven studies looked at training for laparoscopic surgery, and the results were contradicting. Two studies focused on orthopedics training, and the outcomes were positive. Fourteen studies compared the outcomes of virtual reality-based training to conventional didactic teaching, all demonstrating superior outcomes for virtual reality-based training. Nine studies compared the outcomes of virtual reality simulation training to dry lab simulation training. The inferior outcomes of virtual reality simulation training were demonstrated by 5 studies for laparoscopic surgery, 1 study for arthroscopic procedures, 1 study for robotic surgery, and 1 study for dental procedures. One study found potential benefits of virtual reality simulation training on orthopedics surgeries. One study found virtual reality simulation training to be superior to cadaveric training, and 3 studies found augmented reality and virtual reality-based training to be comparable to supervised operative opportunities.

CONCLUSION

Extended reality-based training is a potentially useful modality to serve as an adjunct to the current physical surgical training.

Model Development of a Novel Robotic Surgery Training Exercise With Electrocautery

Robot-assisted surgery (RAS) has undergone rapid adoption in general surgery due to features such as three-dimensional visualization, wrist dexterity, improved precision of movement, and operator ergonomics. While many surgical trainees encounter RAS during their residency, robotic skills training programs and curricula vary across institutions and there is broad variation in graduating general surgeons’ robotic proficiency levels. Due to a need for a formalized process to achieve competence on the robotic platform, simulation-based training has become instrumental in closing this gap as it provides training in a low-stakes environment while allowing the trainee to improve their psychomotor and basic procedural skills.

Several different models of simulation training exist including virtual reality, animal, cadaveric, and inanimate tissue platforms. Each form of training has its own merits and limitations. While virtual reality platforms have been well evaluated for face, content, and construct validity, their initial set-up costs can be as high as $125,000. Similarly, animal and cadaveric models are not only costly but also have ethical considerations that may preclude participation. There is an unmet need in developing high-fidelity, cost-effective simulations for basic videoscopic skills such as cautery use.

We developed a cost-effective and high-fidelity inanimate tissue model that incorporates electrocautery. Using a double-layered bowel model secured to a moistened household sponge, this inanimate exercise simulates fundamental skills of robotic surgery such as tissue handling, camera control, suturing, and electrocautery.

Robotic Surgery: The Impact of Simulation and Other Innovative Platforms on Performance and Training

OBJECTIVE

To review the current status of robotic training and the impact of various training platforms on the performance of robotic surgical trainees.

DATA SOURCES

Literature review of Google Scholar and PubMed. The search terms included a combination of the following: "robotic training," "simulation," "robotic curriculum," "obgyn residency robotic training," "virtual reality robotic training," "DaVinci training," "surgical simulation," "gyn surgical training." The sources considered for inclusion included peer-reviewed articles, literature reviews, textbook chapters, and statements from various institutions involved in resident training.

METHODS OF STUDY SELECTION

A literature search of Google Scholar and PubMed using terms related to robotic surgery and robotics training, as mentioned in the "Data Sources" section.

RESULTS

Multiple novel platforms that use machine learning and real-time video feedback to teach and evaluate robotic surgical skills have been developed over recent years. Various training curricula, virtual reality simulators, and other robotic training tools have been shown to enhance robotic surgical education and improve surgical skills. The integration of didactic learning, simulation, and intraoperative teaching into more comprehensive training curricula shows positive effects on robotic skills proficiency. Few robotic surgery training curricula have been validated through peer-reviewed study, and there is more work to be completed in this area. In addition, there is a lack of information about how the skills obtained through robotics curricula and simulation translate into operating room performance and patient outcomes.

CONCLUSION

Data collected to date show promising advances in the training of robotic surgeons. A diverse array of curricula for training robotic surgeons continue to emerge, and existing teaching modalities are evolving to keep up with the rapidly growing demand for proficient robotic surgeons. Futures areas of growth include establishing competency benchmarks for existing training tools, validating existing curricula, and determining how to translate the acquired skills in simulation into performance in the operating room and patient outcomes. Many surgical training platforms are beginning to expand beyond discrete robotic skills training to procedure-specific and team training. There is still a wealth of research to be done to understand how to create an effective training experience for gynecologic surgical trainees and robotics teams.