A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes

Rigor of cardiac device implant training affects performance outcome

BACKGROUND

Cardiac Implantable Electronic Devices (CIED) training for novice implanters typically occurs in vivo, varies across institutions, and results in inconsistent skill levels. We previously demonstrated the superior effects of Proficiency-Based Progression (PBP) training on implanters' performance in a randomized controlled trial (RCT). Beforehand, we conducted a 'pilot' study to evaluate the robustness of PBP training, hypothesizing that a substantial majority of trainees would achieve the target proficiency after training.

METHODS

In this international, prospective, single-arm study, novice implanters completed a metrics-based simulation training curriculum, requiring proficiency benchmark demonstration at each stage to advance. Trainees ultimately performed a cardiac resynchronization therapy implant on virtual reality simulation which was video-recorded and then scored using validated Metrics (Steps, Critical Errors, Errors non-critical) by independent assessors. The primary outcome was the number of trainees meeting the benchmark. Findings informed corrective actions for the subsequent RCT training, whose effectiveness was verified by comparing PBP-trained participants from the pilot study (PBP-PILOT) and the RCT (PBP-RCT) on Steps completed, All Errors, and benchmark achievement.

RESULTS

Only 17 % achieved the proficiency benchmark. Investigations revealed insufficient faculty adherence to the PBP methodology resulting in premature trainee advancement. Corrective actions improved the subsequent RCT training: compared to the PBP-PILOT group, PBP-RCT trainees completed 15 % more Steps (p = .014), made 69 % fewer All Errors (p = .015), and 93 % met the benchmark (p < .001).

CONCLUSIONS

Pilot testing revealed deficiencies in PBP training execution, highlighting the importance of rigor and faculty compliance. Verifying novel curricula before large-scale implementation is crucial to ensuring robust training outcomes.

Simulation-based learning in orthopaedics: q s r

Introduction

Simulation-based learning has emerged as a transformative tool in orthopaedic education, significantly improving surgical training and patient safety. This systematic review examines the role of simulation in enhancing technical skills, decision-making, and clinical competence among orthopaedic trainees.

Methods

A systematic review was conducted to assess the effectiveness of simulation-based training in orthopaedics. Various simulation modalities, including virtual reality (VR), augmented reality (AR), haptic feedback systems, and task-based trainers, were analyzed for their impact on skill acquisition and retention. The study was registered with PROSPERO (ID: CRD420250652679).

Results

Key findings suggest that simulation-based training leads to reduced surgical errors, faster learning curves, and better skill retention. However, challenges such as high costs, limited access to advanced simulation tools, and difficulties in integrating these technologies into traditional curricula persist.

Conclusion

Simulation is expected to play a crucial role in modernizing orthopaedic education by providing safe, repeatable practice opportunities. Future directions include AI-driven training modules and collaborative VR platforms to further enhance training efficacy and patient outcomes.

Comparing outcomes of robotic-assisted radical prostatectomy by specialists and trainees using a modular training approach

Robotic-assisted laparoscopic prostatectomy (RALP) is the dominant surgical approach for prostate cancer worldwide. The steep learning curve in robotic surgery is eased by modular training and the da Vinci Surgical System© dual console, where supervisors can assume control of the robot from a secondary console if required. Here we evaluate the safety of robotic training by comparing pathological and peri-operative outcomes of RALPs performed predominantly by urology trainees supported by a modular training approach and dual console supervision with RALPs performed predominantly by specialist robotic surgeons. This prospective cohort study examines RALPs performed at a tertiary robotic training centre in Australia between February 2017 and August 2018. Each case was divided into 13 steps from port placement to specimen retrieval. A case was considered a 'trainee-lead case' if the trainee completed more than 75% of the operative steps. We compared patient demographics, operative parameters, peri-operative outcomes, and pathological outcomes between groups. Differences between groups were measured using Fisher's exact test for categorical data and the unpaired Student's t-test for continuous data. Of 126 cases in this study, 39 (31%) were trainee-led cases and 87 (69%) were specialist lead cases. There was no significant difference in operative or pathological outcomes between trainee-lead cases and specialist-lead cases. Our results compared favourably with local and international benchmarks. RALP performed by trainees using a modular training approach and supported by the dual console can have equivalent peri-operative and pathological outcomes to specialist-led cases. This is achieved by graded progression and dual console supervision.

Identifying key drivers affecting the future of virtual reality in medical education

BACKGROUND

Policymakers must have a forward-looking mindset. Future studies are valuable tools for monitoring upcoming developments and identifying the driving forces that shape the future. This research aimed to identify the drivers or key influencing factors in the future of virtual reality in medical education.

MATERIALS AND METHOD

This study was conducted in 2021-2023 in three phases: conducting literature review, interviewing with experts, and categorizing these results within the STEEPV (Social, Technological, Economic, Environmental, Political, and Values) framework.

RESULTS

Content analysis of the literature review resulted in 273 codes and seven categories. Interviews with 15 experts were analyzed, which led to 220 codes and 30 sub-categories. Ultimately, the results of the two aforementioned stages were categorized within the STEEPV framework, including six, seven, six, two, four, and five drivers in the categories of social, technological, economic, environmental, political, and values drivers, respectively.

CONCLUSION

To successfully introduce virtual reality in universities, it is essential to create a supportive environment, adjust policies, and establish ethical guidelines. This will require initial investments, cost-reduction strategies, and collaboration with knowledge-based companies. In addition, it is vital to leverage existing technology, provide training, and enhance digital literacy to ensure effective utilization. The future of virtual reality in medical education will be influenced by technological advancements, economic factors, environmental concerns, political frameworks, and value-driven approaches.

Training in the use of basic functions of the daVinci Xi® robot: a comparative study of residents' skills

The rapid spread of the robotic surgical system has not been accompanied by an equally rapid creation of standardized training courses for the use of this technology.The purpose of our study was to evaluate skill acquisition in the handling and use of the daVinci Xi by comparing two groups of surgical residents. Surgical residents from the University of Turin were enrolled. The participants were divided into two groups: Group A: residents who had participated in at least 8 robotic surgical procedures, and Group B: residents who had never attended robotic surgery. All were administered two instructional videos on the patient cart and console exercises to be performed. Subsequently, the residents were tested and recorded to be evaluated by a senior surgeon experienced in robotic surgery, according to a previously assessed evaluation score. The time of the procedure was also recorded for each test. Patient cart exercises were completed by all participants. We found statistically significant differences between two groups for the first (p = 0.0000) and third (p = 0.0002) patient cart tests and for every test on the surgeon's console except the endoscope handling exercise. Group A scored higher on the patient cart exercises, and the difference reached statistical significance (p = 0.0001). The placement of a single hand-sewn knot on the silicone suture pad was the only exercise that was not fully completed by all participants and showed no statistical difference. The correlation analysis between surgical experience and final score was significant in Group A. The daVinci Xi robotic platform can be properly operated in its basic functions by young surgeons after a short training program even in the absence of previous exposure to robotic clinical procedure.

Maximizing Surgical Success by Aligning Interventions to Outcomes: A Systematic Review

Objective

This study aimed to identify common intraoperative interventions in surgery and evaluate their effectiveness in improving surgical outcomes.

Background

Despite decades of efforts, surgical adverse events remain stubbornly high. There are concerns that too much responsibility is placed on individuals to create change (ie, person-based interventions) rather than adapting systems to support human performance (ie, system-based interventions). This focus may be due to our limited understanding of which interventions most effectively improve outcomes.

Methods

A 2-step search was conducted. Systematic and meta-analytic reviews of Medline, CINAHL, Embase, PsycINFO, Scopus, Cochrane Reviews, Cochrane Protocols and Cochrane Trials were identified, and individual studies within these reviews were selected. Qualitative content analysis categorized intervention and outcome types using inductive and deductive methods. Intervention details and directional findings for all outcomes were extracted.

Results

A total of 575 studies were included in the final analysis comprising 5,288,513 cases, 25,435 providers and patients, 2608 hospitals, across 50 countries, with 1221 outcomes extracted. Overall, the most common interventions were person-based, including education (38%) and policy (19%). Person-based interventions were more likely to improve interpersonal outcomes such as culture, professional development, and resilience. In contrast, system-based interventions, such as technology (15%), cognitive aids (11%), equipment (11%), standardization (4%), and environment redesign (2%), though less frequently implemented, were effective across all outcome types.

Conclusions

Although person-based interventions are widely implemented, system-based interventions generally have a greater impact on surgical outcomes. These results offer valuable insights for optimizing the alignment of interventions to outcomes.

European expert consensus on a structured approach to circular stapling anastomosis in minimally invasive left-sided colorectal resection

AIM

The aim of this work is to develop and operationally define performance metrics that characterize a reference approach to circular stapling anastomosis during minimally invasive left-sided colorectal resection and to obtain face and content validity through a consensus meeting.

METHOD

Three expert colorectal surgeons with advanced experience with minimally invasive surgery, a senior behavioural scientist and a research fellow with experience in performance metrics development formed the Metrics Group. Technical support was provided by device engineers. Published guidelines, training materials, manufacturers' instructions for use and unedited videos of circular stapling anastomosis in minimally invasive left-sided colorectal resection were used to deconstruct the task into defined, observable performance units or metrics (i.e. procedural phases, steps, errors and critical errors). The performance metrics were then subjected to detailed review by 16 expert colorectal surgeons in a modified Delphi process.

RESULTS

Performance metrics for circular stapling anastomosis during minimally invasive left-sided colorectal resection had three procedural phases with 32 steps, 40 errors and 38 critical errors. After the modified Delphi process the agreed performance metrics consisted of three procedural phases, 36 steps, 42 errors and 39 critical errors. A group of expert colorectal surgeons from Europe verified the face and content of these metrics. After discussion, all procedural phases received unanimous consensus by the Delphi panel.

CONCLUSION

Circular stapling anastomosis during the minimally invasive approach to left-sided colorectal resection can be broken down into procedural phases and steps, with errors and critical errors known as performance metrics. We consider the metrics essential for the development of structured training in using circular stapling anastomosis in the minimally invasive approach to left-sided colorectal resection.

Assessing Surgical Competency among Fellows in Vitreoretinal Surgery: A Survey of Fellowship Program Directors and Fellows

PURPOSE

We surveyed vitreoretinal (VR) fellowship program directors (PDs) to elucidate how they assess surgical competency among VR fellows. In addition, we also surveyed fellowship program graduates for the years 2022 and 2023 regarding assessment metrics and tools used during VR fellowship training.

DESIGN

Web-based, cross-sectional descriptive study.

SUBJECTS

Fellowship PDs and recent fellowship graduates in the United States and Canada METHODS: The survey, distributed via email, queried participants about several aspects of assessing surgical competency in VR training including surgical numbers, teaching/assessment methods used to assess fellow surgical competency, comfort of fellows for various surgical procedures, appropriate degree of supervision, and criteria for a hypothetical national competence standard. A Likert scale was utilized for questions capturing participants' opinion. Mean response scores were reported.

MAIN OUTCOME MEASURES

Comparison of responses between PDs and fellows.

RESULTS

Forty-two PDs (42.1%) (33 university-based and 9 private institutions) and 40 fellows (16.8%) (28 academic and 12 private institutions) responded to the survey. Fellows expect a higher minimum number of vitrectomies (≥300) by graduation than PDs (≥200). Both PDs and fellows ranked direct observation of fellow (4.95/4.75), discussion with other faculty on fellow surgical performance (3.93/3.60), and outcomes of fellow surgical cases (3.88/3.53) (P > 0.05) respectively, as top 3 teaching tools. Both PDs and fellows expressed high comfort levels with various complex surgeries, such as primary scleral buckle (4.45/4.60), proliferative vitreoretinopathy detachments (4.57/4.45), advanced diabetic retinal detachments (4.57/4.38), and giant retinal tears (4.64/4.50), by the time of graduation. Autonomy was also considered an important indicator of surgical competence by both groups. However, apart from direct surgical experience, fellows rated other educational tools lower than PDs. There was overall agreement between the groups on several aspects of a hypothetical national competence standard.

CONCLUSIONS

This survey identified that the key tools utilized to assess surgical proficiency were direct observation of surgery by attending physician, discussions about fellow performance among faculty, and outcomes of fellow surgical cases. Both groups emphasized that by graduation, fellows should be proficient in several complex VR surgeries. These findings suggest a need for a more systematic approach to assess surgical competency of VR fellows.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Improving Medical Student Performance With Unsupervised Simulation and Remote Asynchronous Feedback

OBJECTIVE

This study aims to assess the effectiveness of training medical students to perform two clinical procedures using unsupervised simulation with remote asynchronous feedback, compared to an intensive workshop with in-person feedback.

DESIGN, SETTING, AND PARTICIPANTS

Third-year medical students were recruited and randomized into 2 groups: Thoracentesis or paracentesis. Within each group, participants were further randomized into either unsupervised simulation with remote asynchronous feedback (experimental group; EG) or a 2-hour workshop (control group; CG). The EG underwent two unsupervised 20-minute training sessions and received remote asynchronous feedback. The CG had a 2-hour workshop where they received in-person feedback. After training, students were assessed using the objective structured assessment of technical skills (OSATS) scale. Twenty students in thoracentesis and 23 in paracentesis training completed the 2 training sessions with remote and asynchronous feedback, and 30 students for both thoracentesis and paracentesis groups completed the 2-hour workshop.

RESULTS

The EG achieved a significantly higher passing rate than the CG on both procedures (thoracentesis 80% vs. 43%, paracentesis 91% vs. 67%, p-value< 0.05).

CONCLUSION

The asynchronous educational method allowed EG students to achieve higher performance than CG students. This novel modality allowed students and instructors to train and assess at their own pace.

Deep learning prediction of error and skill in robotic prostatectomy suturing

BACKGROUND

Manual objective assessment of skill and errors in minimally invasive surgery have been validated with correlation to surgical expertise and patient outcomes. However, assessment and error annotation can be subjective and are time-consuming processes, often precluding their use. Recent years have seen the development of artificial intelligence models to work towards automating the process to allow reduction of errors and truly objective assessment. This study aimed to validate surgical skill rating and error annotations in suturing gestures to inform the development and evaluation of AI models.

METHODS

SAR-RARP50 open data set was blindly, independently annotated at the gesture level in Robotic-Assisted Radical Prostatectomy (RARP) suturing. Manual objective assessment tools and error annotation methodology, Objective Clinical Human Reliability Analysis (OCHRA), were used as ground truth to train and test vision-based deep learning methods to estimate skill and errors. Analysis included descriptive statistics plus tool validity and reliability.

RESULTS

Fifty-four RARP videos (266 min) were analysed. Strong/excellent inter-rater reliability (range r = 0.70-0.89, p < 0.001) and very strong correlation (r = 0.92, p < 0.001) between objective assessment tools was demonstrated. Skill estimation of OSATS and M-GEARS had a Spearman's Correlation Coefficient 0.37 and 0.36, respectively, with normalised mean absolute error representing a prediction error of 17.92% (inverted "accuracy" 82.08%) and 20.6% (inverted "accuracy" 79.4%) respectively. The best performing models in error prediction achieved mean absolute precision of 37.14%, area under the curve 65.10% and Macro-F1 58.97%.

CONCLUSIONS

This is the first study to employ detailed error detection methodology and deep learning models within real robotic surgical video. This benchmark evaluation of AI models sets a foundation and promising approach for future advancements in automated technical skill assessment.

International consensus panel for transurethral resection of bladder tumours metrics: assessment of face and content validity

OBJECTIVE

To develop performance metrics that objectively define a reference approach to a transurethral resection of bladder tumours (TURBT) procedure, seek consensus on the performance metrics from a group of international experts.

METHODS

The characterisation of a reference approach to a TURBT procedure was performed by identifying phases and explicitly defined procedure events (i.e., steps, errors, and critical errors). An international panel of experienced urologists (i.e., Delphi panel) was then assembled to scrutinise the metrics using a modified Delphi process. Based on the panel's feedback, the proposed metrics could be edited, supplemented, or deleted. A voting process was conducted to establish the consensus level on the metrics. Consensus was defined as the panel majority (i.e., >80%) agreeing that the metric definitions were accurate and acceptable. The number of metric units before and after the Delphi meeting were presented.

RESULTS

A core metrics group (i.e., characterisation group) deconstructed the TURBT procedure. The reference case was identified as an elective TURBT on a male patient, diagnosed after full diagnostic evaluation with three or fewer bladder tumours of ≤3 cm. The characterisation group identified six procedure phases, 60 procedure steps, 43 errors, and 40 critical errors. The metrics were presented to the Delphi panel which included 15 experts from six countries. After the Delphi, six procedure phases, 63 procedure steps, 47 errors, and 41 critical errors were identified. The Delphi panel achieved a 100% consensus.

CONCLUSION

Performance metrics to characterise a reference approach to TURBT were developed and an international panel of experts reached 100% consensus on them. This consensus supports their face and content validity. The metrics can now be used for a proficiency-based progression training curriculum for TURBT.

Structure, governance and delivery of specialist training programs in periodontology and implant dentistry

AIM

To update the competences and learning outcomes and their evaluation, educational methods and education quality assurance for the training of contemporary specialists in periodontology, including the impact of the 2018 Classification of Periodontal and Peri-implant Diseases and Conditions (2018 Classification hereafter) and the European Federation of Periodontology (EFP) Clinical Practice Guidelines (CPGs).

METHODS

Evidence was gathered through scientific databases and by searching for European policies on higher education. In addition, two surveys were designed and sent to program directors and graduates.

RESULTS

Program directors reported that curricula were periodically adapted to incorporate advances in diagnosis, classification, treatment guidelines and clinical techniques, including the 2018 Classification and the EFP CPGs. Graduates evaluated their overall training positively, although satisfaction was limited for training in mucogingival and surgical procedures related to dental implants. Traditional educational methods, such as didactic lectures, are still commonly employed, but they are now often associated with more interactive methods such as case-based seminars and problem-based and simulation-based learning. The evaluation of competences/learning outcomes should employ multiple methods of assessment.

CONCLUSION

An update of competences and learning outcomes of specialist training in periodontology is proposed, including knowledge and practical application of the 2018 Classification and CPGs. Harmonizing specialist training in periodontology is a critical issue at the European level.

Learning analytics and the future of postgraduate medical training

Confronted by the many barriers and deficiencies which currently face those responsible for the training of doctors, the concept of a logic model applied in real time may seem aspirational. However, several of the necessary of logic-based practices are already in place - these include quantified training effect and performance, learning analytics, and applied reflective practice. A nationally or internationally co-ordinated effort is required to harness these disciplines (which currently exist disparately) to create a sustainable and effective training system which is adaptive to its own performance and to society's changing needs. This will mean making better use of the data currently being generated by and around training, and its presentation in a timely and comprehensible form to the person(s) who is responsible, prepared, and able to use it to best effect.

Robotic kidney transplantation

Kidney transplantation is the best treatment option for patients with end-stage renal disease owing to improved survival and quality of life compared with dialysis. The surgical approach to kidney transplantation has been somewhat stagnant in the past 50 years, with the open approach being the only available option. In this scenario, evidence of reduced surgery-related morbidity after the introduction of robotics into several surgical fields has induced surgeons to consider robot-assisted kidney transplantation (RAKT) as an alternative approach to these fragile and immunocompromised patients. Since 2014, when the RAKT technique was standardized thanks to the pioneering collaboration between the Vattikuti Urology Institute and the Medanta hospital (Vattikuti Urology Institute-Medanta), several centres worldwide implemented RAKT programmes, providing interesting results regarding the safety and feasibility of this procedure. However, RAKT is still considered an alternative procedure to be offered mainly in the living donor setting, owing to various possible drawbacks such as prolonged rewarming time, demanding learning curve, and difficulties in carrying out this procedure in challenging scenarios (such as patients with obesity, severe atherosclerosis of the iliac vessels, deceased donor setting, or paediatric recipients). Nevertheless, the refinement of robotic platforms through the implementation of novel technologies as well as the encouraging results from multicentre collaborations under the umbrella of the European Association of Urology Robotic Urology Section are currently expanding the boundaries of RAKT, making this surgical procedure a real alternative to the open approach.

Surgical skill simulation training to proficiency reduces procedural errors among novice cardiac device implanters: a randomized study

AIMS

In cardiac device implantation, having both surgical skills and ability to manipulate catheter/lead/wire is crucial. Few cardiologists, however, receive formal surgical training prior to implanting. Skills are mostly acquired directly on-the-job and surgical technique varies across institutions; suboptimal approaches may increase complications. We investigated how novel proficiency-based progression (PBP) simulation training impacts the surgical quality of implantations, compared to traditional simulation (SIM) training.

METHODS AND RESULTS

In this international prospective study, novice implanters were randomized (blinded) 1:1 to participate in a simulation-based procedure training curriculum, with proficiency demonstration requirements for advancing (PBP approach) or without (SIM). Ultimately, trainees performed the surgical tasks of an implant on a porcine tissue that was video-recorded and then scored by two independent assessors (blinded to group), using previously validated performance metrics. Primary outcomes were the number of procedural Steps Completed, Critical Errors, Errors (non-critical), and All Errors Combined. Thirty novice implanters from 10 countries participated. Baseline experiences were similar between groups. Compared to SIM-trained, the PBP-trained group completed on average 11% more procedural Steps (P < 0.001) and made 61.2% fewer Critical Errors (P < 0.001), 57.1% fewer Errors (P = 0.140), and 60.7% fewer All Errors Combined (P = 0.001); 11/15 (73%) PBP trainees demonstrated the predefined target performance level vs. 3/15 SIM trainees (20%) in the video-recorded performance.

CONCLUSION

Proficiency-based progression training produces superior objectively assessed novice operators' surgical performance in device implantation compared with traditional (simulation) training. Systematic PBP incorporation into formal academic surgical skills training is recommended before in vivo device practice. Future studies will quantify PBP training's effect on surgery-related device complications.

Current Standards for Training in Robot-assisted Surgery and Endourology: A Systematic Review

BACKGROUND AND OBJECTIVE

Different training programs have been developed to improve trainee outcomes in urology. However, evidence on the optimal training methodology is sparse. Our aim was to provide a comprehensive description of the training programs available for urological robotic surgery and endourology, assess their validity, and highlight the fundamental elements of future training pathways.

METHODS

We systematically reviewed the literature using PubMed/Medline, Embase, and Web of Science databases. The validity of each training model was assessed. The methodological quality of studies on metrics and curricula was graded using the MERSQI scale. The level of evidence (LoE) and level of recommendation for surgical curricula were awarded using the educational Oxford Centre for Evidence-Based Medicine classification.

KEY FINDINGS AND LIMITATIONS

A total of 75 studies were identified. Many simulators have been developed to aid trainees in mastering skills required for both robotic and endourology procedures, but only four demonstrated predictive validity. For assessment of trainee proficiency, we identified 18 in robotics training and six in endourology training; however, the majority are Likert-type scales. Although proficiency-based progression (PBP) curricula demonstrated superior outcomes to traditional training in preclinical settings, only four of six (67%) in robotics and three of nine (33%) in endourology are PBP-based. Among these, the Fundamentals of Robotic Surgery and the SIMULATE curricula have the highest LoE (level 1b). The lack of a quantitative synthesis is the main limitation of our study.

CONCLUSIONS AND CLINICAL IMPLICATIONS

Training curricula that integrate simulators and PBP methodology have been introduced to standardize trainee outcomes in robotics and endourology. However, evidence regarding their educational impact remains restricted to preclinical studies. Efforts should be made to expand these training programs to different surgical procedures and assess their clinical impact.

Advancements in surgical education: exploring animal and simulation models in fetal and neonatal surgery training

Introduction

Surgical education is undergoing a transformation, moving away from traditional models towards more modern approaches that integrate experiential and didactic methods. This shift is particularly pertinent in the realm of fetal and neonatal surgery, where specialized training is crucial. Historical training methods, such as cadaveric dissection, have been prevalent for centuries, but newer innovations, including animal and non-animal simulation models, are gaining prominence. This manuscript aims to explore the use of both animal and non-animal models in surgical education, with a specific focus on fetal and neonatal surgery.

Animal models

The use of animal models in surgical training has a long history, dating back to Halsted's introduction in 1889. These models, often utilizing large animals like swine and dogs, offer valuable insights into fetal and neonatal surgeries. They allow for the study of long-term outcomes and the simulation of various diseases and anomalies, providing essential training experiences not readily available in human surgeries. However, there are notable limitations, including anatomical and physiological differences from humans, ethical considerations, and substantial infrastructure and maintenance costs.

Simulation models

Simulation-based training offers several benefits, including standardized and safe learning environments without risks to real patients. Bench models, using synthetic materials or non-living animal tissue, provide cost-effective options for skills development. Virtual reality and 3-D printing technologies further enhance simulation experiences, allowing for the replication of complex clinical scenarios and patient-specific anatomies. While these models offer significant advantages, they lack the complexity of biological systems found in animal models.

Conclusion

In conclusion, both animal and non-animal simulation models play crucial roles in enhancing surgical education, particularly in fetal and neonatal surgery. While advancements in non-animal technologies are important for ethical reasons, the continued necessity of animal models in certain areas should be acknowledged. By responsibly integrating these models into training programs, surgical education can be further enriched while upholding ethical standards and ensuring optimal patient outcomes.

Virtual reality simulation training in laparoscopic surgery - does it really matter, what simulator to use? Results of a cross-sectional study

BACKGROUND

Virtual reality simulation training plays a crucial role in modern surgical training, as it facilitates trainees to carry out surgical procedures or parts of it without the need for training "on the patient". However, there are no data comparing different commercially available high-end virtual reality simulators.

METHODS

Trainees of an international gastrointestinal surgery workshop practiced in different sequences on LaparoS® (VirtaMed), LapSim® (Surgical Science) and LapMentor III® (Simbionix) eight comparable exercises, training the same basic laparoscopic skills. Simulator based metrics were compared between an entrance and exit examination.

RESULTS

All trainees significantly improved their basic laparoscopic skills performance, regardless of the sequence in which they used the three simulators. Median path length was initially 830 cm and 463 cm on the exit examination (p < 0.001), median time taken improved from 305 to 167 s (p < 0.001).

CONCLUSIONS

All Simulators trained efficiently the same basic surgery skills, regardless of the sequence or simulator used. Virtual reality simulation training, regardless of the simulator used, should be incorporated in all surgical training programs. To enhance comparability across different types of simulators, standardized outcome metrics should be implemented.

Development and validation of metrics for a new RAPN training model

Robot-assisted partial nephrectomy (RAPN) is a complex and index procedure that urologists need to learn how to perform safely. No validated performance metrics specifically developed for a RAPN training model (TM) exist. A Core Metrics Group specifically adapted human RAPN metrics to be used in a newly developed RAPN TM, explicitly defining phases, steps, errors, and critical errors. A modified Delphi meeting concurred on the face and content validation of the new metrics. One hundred percent consensus was achieved by the Delphi panel on 8 Phases, 32 Steps, 136 Errors and 64 Critical Errors. Two trained assessors evaluated recorded video performances of novice and expert RAPN surgeons executing an emulated RAPN in the newly developed TM. There were no differences in procedure Steps completed by the two groups. Experienced RAPN surgeons made 34% fewer Total Errors than the Novice group. Performance score for both groups was divided at the median score using Total Error scores, into HiError and LoError subgroups. The LowErrs Expert RAPN surgeons group made 118% fewer Total Errors than the Novice HiErrs group. Furthermore, the LowErrs Expert RAPN surgeons made 77% fewer Total Errors than the HiErrs Expert RAPN surgeons. These results established construct and discriminative validity of the metrics. The authors described a novel RAPN TM and its associated performance metrics with evidence supporting their face, content, construct, and discriminative validation. This report and evidence support the implementation of a simulation-based proficiency-based progression (PBP) training program for RAPN.

The deconstructed procedural description in robotic colorectal surgery

Increasing robotic surgical utilisation in colorectal surgery internationally has strengthened the need for standardised training. Deconstructed procedural descriptions identify components of an operation that can be integrated into proficiency-based progression training. This approach allows both access to skill level appropriate training opportunities and objective and comparable assessment. Robotic colorectal surgery has graded difficulty of operative procedures lending itself ideally to component training. Developing deconstructed procedural descriptions may assist in the structure and progression components in robotic colorectal surgical training. There is no currently published guide to procedural descriptions in robotic colorectal surgical or assessment of their training utility. This scoping review was conducted in June 2022 following the PRISMA-ScR guidelines to identify which robotic colorectal surgical procedures have available component-based procedural descriptions. Secondary aims were identifying the method of development of these descriptions and how they have been adapted in a training context. 20 published procedural descriptions were identified covering 8 robotic colorectal surgical procedures with anterior resection the most frequently described procedure. Five publications included descriptions of how the procedural description has been utilised for education and training. From these publications terminology relating to using deconstructed procedural descriptions in robotic colorectal surgical training is proposed. Development of deconstructed robotic colorectal procedural descriptions (DPDs) in an international context may assist in the development of a global curriculum of component operating competencies supported by objective metrics. This will allow for standardisation of robotic colorectal surgical training and supports a proficiency-based training approach.

ESRU-ESU-YAU_UROTECH Survey on Urology Residents Surgical Training: Are We Ready for Simulation and a Standardized Program?

Background

Currently, the landscape of surgical training is undergoing rapid evolution, marked by the initial implementation of standardized surgical training programs, which are further facilitated by the emergence of new technologies. However, this proliferation is uneven across various countries and hospitals.

Objective

To offer a comprehensive overview of the existing surgical training programs throughout Europe, with a specific focus on the accessibility of simulation resources and standardized surgical programs.

Design setting and participants

A dedicated survey was designed and spread in May 2022 via the European Association of Urology (EAU) mail list, to Young Urologist Office (YUO), Junior membership, European Urology Residents Education Program participants between 2014 and 2022, and other urologists under 40 yr, and via the EAU Newsletter.

Intervention

A 64-item, online-based survey in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) using the platform of Survey Monkey (Portland, OR, USA) was realized.

Outcome measurements and statistical analysis

The study involved an assessment of the demographic characteristics. Additionally, it explored the type of center, availability of various surgical approaches, presence of training infrastructure, participation in courses, organization of training, and participants' satisfaction with the training program. The level of satisfaction was evaluated using a Likert-5 scale. The subsequent sections delved into surgical training within the realms of open, laparoscopic, robotic, and endoscopic surgery, each explored separately. Finally, the investigation encompassed the presence of a structured training course and the availability of a duly validated final evaluation process.

Results and limitations

There were 375 responders with a completion rate of 82%. Among them, 75% were identified as male, 50.6% were young urologists, 31.7% were senior residents, and 17.6% were junior residents. A significant majority of participants (69.6%) were affiliated with academic centers. Regarding the presence of dry lab training facilities, only 50.3% of respondents indicated its availability. Among these centers, 46.7% were primarily focused on laparoscopy training. The availability of virtual and wet lab training centers was even more limited, with rates of 31.5% and 16.2%, respectively. Direct patient involvement was reported in 80.5% of cases for open surgery, 58.8% for laparoscopy, 25.0% for robotics, and 78.6% for endourology. It is worth noting that in <25% of instances, training followed a well-defined standardized program comprising both preclinical and clinical modular phases. Finally, the analysis of participant feedback showed that 49.7% of respondents expressed a satisfaction rating of either 4 or 5 points with respect to the training program. The limitations of our study include the low response rate, predominance of participants from academic centers, and absence of responses from individuals not affiliated with the EAU network.

Conclusions

The current distribution of surgical training centers falls short of ensuring widespread access to standardized training programs. Although dry lab facilities are relatively well spread, the availability of wet lab resources remains restricted. Additionally, it appears that many trainees' initial exposure to surgery occurs directly with patients. There is a pressing need for continued endeavors to establish uniform training routes and assessment techniques across various surgical methodologies.

Patient summary

Nowadays, the surgical training landscape is heterogeneous across different countries. The implementation of a standardized training methodology to enhance the overall quality of surgical training and thereby improving patient outcomes is needed.

Measures of performance and proficiency in robotic assisted surgery: a systematic review

Robotic assisted surgery (RAS) has seen a global rise in adoption. Despite this, there is not a standardised training curricula nor a standardised measure of performance. We performed a systematic review across the surgical specialties in RAS and evaluated tools used to assess surgeons' technical performance. Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published on or after January 2020-January 2022. Observational studies and RCTs were included; review articles and systematic reviews were excluded. The papers' quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. The initial search yielded 1189 papers of which 72 fit the eligibility criteria. 27 unique performance metrics were identified. Global assessments were the most common tool of assessment (n = 13); the most used was GEARS (Global Evaluative Assessment of Robotic Skills). 11 metrics (42%) were objective tools of performance. Automated performance metrics (APMs) were the most widely used objective metrics whilst the remaining (n = 15, 58%) were subjective. The results demonstrate variation in tools used to assess technical performance in RAS. A large proportion of the metrics are subjective measures which increases the risk of bias amongst users. A standardised objective metric which measures all domains of technical performance from global to cognitive is required. The metric should be applicable to all RAS procedures and easily implementable. Automated performance metrics (APMs) have demonstrated promise in their wide use of accurate measures.

Society for Simulation in Healthcare Guidelines for Simulation Training

BACKGROUND

Simulation has become a staple in the training of healthcare professionals with accumulating evidence on its effectiveness. However, guidelines for optimal methods of simulation training do not currently exist.

METHODS

Systematic reviews of the literature on 16 identified key questions were conducted and expert panel consensus recommendations determined using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology.

OBJECTIVE

These evidence-based guidelines from the Society for Simulation in Healthcare intend to support healthcare professionals in decisions on the most effective methods for simulation training in healthcare.

RESULTS

Twenty recommendations on 16 questions were determined using GRADE. Four expert recommendations were also provided.

CONCLUSIONS

The first evidence-based guidelines for simulation training are provided to guide instructors and learners on the most effective use of simulation in healthcare.

Which, how, and what? Using digital tools to train surgical skills; a systematic review and meta-analysis

Background

Digital tools like digital box trainers and VR seem promising in delivering safe and tailored practice opportunities outside of the surgical clinic, yet understanding their efficacy and limitations is essential. This study investigated Which digital tools are available to train surgical skills, How these tools are used, How effective they are, and What skills they are intended to teach.

Methods

Medline, Embase, and Cochrane libraries were systematically reviewed for randomized trials, evaluating digital skill-training tools based on objective outcomes (skills scores and completion time) in surgical residents. Digital tools effectiveness were compared against controls, wet/dry lab training, and other digital tools. Tool and training factors subgroups were analysed, and studies were assessed on their primary outcomes: technical and/or non-technical.

Results

The 33 included studies involved 927 residents and six digital tools; digital box trainers, (immersive) virtual reality (VR) trainers, robot surgery trainers, coaching and feedback, and serious games. Digital tools outperformed controls in skill scores (SMD 1.66 [1.06, 2.25], P < 0.00001, I2 = 83 %) and completion time (SMD -1.05 [-1.72, -0.38], P = 0.0001, I2 = 71 %). There were no significant differences between digital tools and lab training, between tools, or in other subgroups. Only two studies focussed on non-technical skills.

Conclusion

While the efficacy of digital tools in enhancing technical surgical skills is evident - especially for VR-trainers -, there is a lack of evidence regarding non-technical skills, and need to improve methodological robustness of research on new (digital) tools before they are implemented in curricula.

Key message

This study provides critical insight into the increasing presence of digital tools in surgical training, demonstrating their usefulness while identifying current challenges, especially regarding methodological robustness and inattention to non-technical skills.

Surgical Phase Duration in Robot-Assisted Partial Nephrectomy: A Surgical Data Science Exploration for Clinical Relevance

(1) Background: Surgical phases form the basic building blocks for surgical skill assessment, feedback, and teaching. The phase duration itself and its correlation with clinical parameters at diagnosis have not yet been investigated. Novel commercial platforms provide phase indications but have not been assessed for accuracy yet. (2) Methods: We assessed 100 robot-assisted partial nephrectomy videos for phase durations based on previously defined proficiency metrics. We developed an annotation framework and subsequently compared our annotations to an existing commercial solution (Touch Surgery, Medtronic™). We subsequently explored clinical correlations between phase durations and parameters derived from diagnosis and treatment. (3) Results: An objective and uniform phase assessment requires precise definitions derived from an iterative revision process. A comparison to a commercial solution shows large differences in definitions across phases. BMI and the duration of renal tumor identification are positively correlated, as are tumor complexity and both tumor excision and renorrhaphy duration. (4) Conclusions: The surgical phase duration can be correlated with certain clinical outcomes. Further research should investigate whether the retrieved correlations are also clinically meaningful. This requires an increase in dataset sizes and facilitation through intelligent computer vision algorithms. Commercial platforms can facilitate this dataset expansion and help unlock the full potential, provided that the phase annotation details are disclosed.

A comparative analysis of the HUGOTM robot-assisted surgery system and the Da Vinci® Xi surgical system for robot-assisted sacrocolpopexy for pelvic organ prolapse treatment

BACKGROUND

We aimed to compare the outcomes of Robot-assisted sacrocolpopexy (RASC) performed using the novel HUGOTM Robot-Assisted Surgery (RAS) System with the Da Vinci® Xi surgical system.

METHODS

Data from 38 women undergoing RASC for a ≥ 2-grade pelvic organ prolapse were collected (2021-2023).

RESULTS

Overall, 23 (60.5%) and 15 (39.5%) procedures were performed using the DaVinci® Xi and the HUGOTM RAS system, respectively. The median total operative time was 123 (IQR:106.5-140.5) minutes for the DaVinci® Xi versus 120 (IQR:120-146) minutes for the HUGOTM RAS cases (p = 0.5). No conversion to open/laparoscopic surgery, perioperative complications, or system failures occurred. No differences were recorded according to day of catheter removal and length of stay.

CONCLUSIONS

This study represents the first worldwide comparison of RASC executed using the HUGOTM RAS versus the Da Vinci® Xi System. Our data suggest that RASC might be performed with both robotic platforms with similar perioperative outcomes.

Characterizing individual trainee learning curves in surgical training: Challenges and opportunities

The surgical learning curve is an observable and measurable phenomenon. In the era of competency-based approaches to surgical training, monitoring the trajectory of individual trainee competence attainment could represent a meaningful method of formative and summative assessment. While technology can assist this approach, a number of significant barriers to the implementation of such assessment methods remain, including: accurate data collection, standard setting, and reliable assessment. Translating individual learning curve data into quantifiable case minimum targets in training poses further difficulties, and may not be possible for all procedures, particularly those that are less frequently performed and assessed. In spite of these challenges, significant benefits could be realized through an individualized approach to competency assessment using trainee learning curve data. Tracking competence acquisition against criterion-referenced standards could allow for targeted training and remediation, conforming with modern theories of adult education and empowering trainees to take control of their own learning. Learning curve data could also be used to assess the effects of educational interventions such as simulation-based training on subsequent competence acquisition rates. Ultimately, the individual learning curves of trainees could be used to inform personalised decisions regarding entrustment, credentialing, and certification, allowing training programmes to move beyond minimum operative experience targets as a crude proxy measure of competence.

Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial

Objective

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity.

Background

GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations.

Methods

In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves.

Results

The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments.

Conclusions

Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

Objective assessment of intraoperative skills for robot-assisted partial nephrectomy (RAPN)

RAPN training usually takes place in-vivo and methods vary across countries/institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize the optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Seven Novice and 9 Experienced RAPN surgeons video recorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84-1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when the LoErr Expert RAPN surgeon's performance was compared with the HiErrs Novice RAPN surgeon's performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0-0.8), for scoring RAPN surgical performance. The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experiences. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.

The Surgical Learning Curve for Biochemical Recurrence After Robot-assisted Radical Prostatectomy

BACKGROUND

Improved cancer control with increasing surgical experience-the learning curve-was demonstrated for open and laparoscopic prostatectomy. In a prior single-center study, we found that this might not be the case for robot-assisted radical prostatectomy (RARP).

OBJECTIVE

To investigate the relationship between prior experience of a surgeon and biochemical recurrence (BCR) after RARP.

DESIGN, SETTING, AND PARTICIPANTS

We retrospectively analyzed the data of 8101 patients with prostate cancer treated with RARP by 46 surgeons at nine institutions between 2003 and 2021. Surgical experience was coded as the total number of robotic prostatectomies performed by the surgeon before the patient operation.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

We evaluated the relationship of prior surgeon experience with the probability of BCR adjusting for preoperative prostate-specific antigen, pathologic stage, grade, lymph-node involvement, and year of surgery.

RESULTS AND LIMITATIONS

Overall, 1047 patients had BCR. The median follow-up for patients without BCR was 33 mo (interquartile range: 14, 61). After adjusting for case mix, the relationship between surgical experience and the risk of BCR after surgery was not statistically significant (p = 0.2). The 5-yr BCR-free survival rates for a patient treated by a surgeon with prior 10, 250, and 1000 procedures performed were, respectively, 82.0%, 82.7%, and 84.8% (absolute difference between 10 and 1000 prior procedures: 1.6% [95% confidence interval: 0.4%, 3.3%). Results were robust to a number of sensitivity analyses.

CONCLUSIONS

These findings suggest that, as opposed to open and laparoscopic radical prostatectomy, surgeons performing RARP achieve adequate cancer control in the early phase of their career. Further research should explore why the learning curve for robotic surgery differs from prior findings for open and laparoscopic radical prostatectomy. We hypothesize that surgical education, including simulation training and the adoption of objective performance metrics, is an important mechanism for flattening the learning curve.

PATIENT SUMMARY

We investigated the relationship between biochemical recurrence after robot-assisted radical prostatectomy and surgeon's experience. Surgeons at an early stage of their career had similar outcomes to those of more experienced surgeons, and we hypothesized that surgical education in robotics might be an important determinant of such a finding.

Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial

Importance

In cardiac device implant training, there is no common system to objectively assess trainees' ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators' early learning curve.

Objective

To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT).

Design, Setting, and Participants

In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022.

Intervention

Training with PBP vs SBT.

Main Outcome and Measures

The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined.

Results

Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees.

Conclusions and Relevance

In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications.

Trial Registration

ClinicalTrials.gov Identifier: NCT05952908.

The First Entirely 3D-Printed Training Model for Robot-assisted Kidney Transplantation: The RAKT Box

Background

Robot-assisted kidney transplantation (RAKT) is increasingly performed at selected referral institutions worldwide. However, simulation and proficiency-based progression training frameworks for RAKT are still lacking, making acquisition of the RAKT-specific skill set a critical unmet need for future RAKT surgeons.

Objective

To develop and test the RAKT Box, the first entirely 3D-printed, perfused, hyperaccuracy simulator for vascular anastomoses during RAKT.

Design setting and participants

The project was developed in a stepwise fashion by a multidisciplinary team including urologists and bioengineers via an iterative process over a 3-yr period (November 2019-November 2022) using an established methodology. The essential and time-sensitive steps of RAKT were selected by a team of RAKT experts and simulated using the RAKT Box according to the principles of the Vattituki-Medanta technique. The RAKT Box was tested in the operating theatre by an expert RAKT surgeon and independently by four trainees with heterogeneous expertise in robotic surgery and kidney transplantation.

Surgical procedure

Simulation of RAKT.

Measurements

Video recordings of the trainees' performance of vascular anastomoses using the RAKT Box were evaluated blind by a senior surgeon according to the Global Evaluative Assessment of Robotic Skills (GEARS) and Assessment of Robotic Console Skills (ARCS) tools.

Results and limitations

All participants successfully completed the training session, confirming the technical reliability of the RAKT Box simulator. Tangible differences were observed among the trainees in both anastomosis time and performance metrics. Key limitations of the RAKT Box include lack of simulation of the ureterovesical anastomosis and the need for a robotic platform, specific training instruments, and disposable 3D-printed vessels.

Conclusions

The RAKT Box is a reliable educational tool to train novice surgeons in the key steps of RAKT and may represent the first step toward the definition of a structured surgical curriculum in RAKT.

Patient summary

We describe the first entirely 3D-printed simulator that allows surgeons to test the key steps of robot-assisted kidney transplantation (RAKT) in a training environment before performing the procedure in patients. The simulator, called the RAKT Box, has been successfully tested by an expert surgeon and four trainees. The results confirm its reliability and potential as an educational tool for training of future RAKT surgeons.

Transferability of robotic console skills by early robotic surgeons: a multi-platform crossover trial of simulation training

Robotic surgical training is undergoing a period of transition now that new robotic operating platforms are entering clinical practice. As this occurs, training will need to be adapted to include strategies to train across various consoles. These new consoles differ in multiple ways, with some new vendors using flat screen open source 3D enhanced vision with glasses and differences in design will require surgeons to learn new skills. This process has parallels with aviation credentialling across different aircraft described as type rating. This study was designed to test the hypothesis that technical robotic console operating skills are transferrable across different robotic operating platforms. Ten participants sequentially completed four Mimic®(Surgical Science) simulation exercises on two different robotic operating platforms (DaVinci®, Intuitive Surgical and HUGO™ RAS, Medtronic). Ethical approval and informed consent were obtained for this study. Groups were balanced for key demographics including previous robotic simulator experience. Data for simulation metrics and time to proficiency were collected for each attempt at the simulated exercise and analysed. Qualitative feedback on multi-platform learning was sought via unstructured interviews and a questionnaire. Participants were divided into two groups of 5. Group 1 completed the simulation exercises on console A first then repeated these exercises on console B. Group 2 completed the simulated exercises on console B first then repeated these exercises on console A. Group 1 candidates adapted quicker to the second console and Group 2 candidates reached proficiency faster on the first console. Participants were slower on the second attempt of the final exercise regardless of their allocated group. Quality and efficiency metrics and risk and safety metrics were equivalent across consoles. The data from this investigation suggests that console operating skills are transferrable across different platforms. Overall risk and safety metrics are within acceptable limits regardless of the order of progression of console indicating that training can safely occur across multiple consoles contemporaneously. This data has implications for the design of training and certification as new platforms progress to market and supports a proficiency-based approach.

Virtual classroom proficiency-based progression for robotic surgery training (VROBOT): a randomised, prospective, cross-over, effectiveness study

Robotic surgery training has lacked evidence-based standardisation. We aimed to determine the effectiveness of adjunctive interactive virtual classroom training (VCT) in concordance with the self-directed Fundamentals of Robotic Surgery (FRS) curriculum. The virtual classroom is comprised of a studio with multiple audio-visual inputs to which participants can connect remotely via the BARCO weConnect platform. Eleven novice surgical trainees were randomly allocated to two training groups (A and B). In week 1, both groups completed a robotic skills induction. In week 2, Group A received training with the FRS curriculum and adjunctive VCT; Group B only received access to the FRS curriculum. In week 3, the groups received the alternate intervention. The primary outcome was measured using the validated robotic-objective structured assessment of technical skills (R-OSAT) at the end of week 2 (time-point 1) and 3 (time-point 2). All participants completed the training curriculum and were included in the final analyses. At time-point 1, Group A achieved a statistically significant greater mean proficiency score compared to Group B (44.80 vs 35.33 points, p = 0.006). At time-point 2, there was no significant difference in mean proficiency score in Group A from time-point 1. In contrast, Group B, who received further adjunctive VCT showed significant improvement in mean proficiency by 9.67 points from time-point 1 (95% CI 5.18-14.15, p = 0.003). VCT is an effective, accessible training adjunct to self-directed robotic skills training. With the steep learning curve in robotic surgery training, VCT offers interactive, expert-led learning and can increase training effectiveness and accessibility.

Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence

INTRODUCTION

The LapSim (Surgical Science, Sweden) laparoscopic simulator is a high-fidelity virtual reality simulator for use in endoscopic surgical training. This review critiques the current validity evidence for the LapSim laparoscopic simulator, specifically with respect to its potential use as a tool and method of training and assessment in surgery.

METHODS

A scoping review of the MEDLINE (PubMed), EMBASE, Cochrane and Web of Science databases was conducted in accordance with PRISMA guidelines (2020)-scoping review extension. Articles were included if they presented validity evidence for the use of the LapSim in operative skill training or assessment, in accordance with Messick's validity framework. European Association of Endoscopic Surgeons (EAES) guidelines (2005) were used to provide recommendations for the use of the LapSim in operative performance training and assessments.

RESULTS

Forty-nine articles were included. An EAES level 2 recommendation was provided with regard to the internal consistency reliability of automated performance metrics in assessing performance. An EAES recommendation of 2 was awarded with respect to the ability of the LapSim to discriminate based on case volume and overall laparoscopic experience (relationships with other variables). Performance assessment metrics on the LapSim correlate with improved performance in the operating room (EAES level of recommendation 1-consequential validity).

CONCLUSION

The LapSim has accumulated substantial evidence supporting the validity of its use in surgical training and assessment. Future studies should explore the relationship between the achievement of performance benchmarks on the LapSim and subsequent patient outcomes, and interrogate the benefits of implementing virtual reality simulation training and assessment curricula in post-graduate surgical training programmes.

Training on Congenital 3D Cardiac Models - Will Models Improve Surgical Performance?

Technical skill development in congenital heart surgery (CHS) is challenging due to numerous factors which potentially limit the hands-on operative exposure in surgical training. These challenges have stimulated the growth of simulation-based training through the development of 3D-printed models, providing hands-on surgical training (HOST). From its inception in 2015, the models used in the HOST program have constantly improved, and now include valvar/subvalvar apparatus and better materials that mimic real tissue. Evidence shows that deliberate, regular simulation practice can improve a surgeon's technical skills across the spectrum of CHS. Furthermore, surgical trainees who undergo simulation training are able to translate this improved performance into the operative environment with improved patient outcomes. Despite evidence to support the incorporation of simulation methods into congenital training, its widespread adoption into training curricula remains low. This is due to numerous factors including funding, lack of dedicated time or proctorship and access to models-all of which can be overcome with the newer generation of models and committed trainers. Training programs should consider incorporating simulation-methods as a routine component of congenital training programs.

Development and evaluation of a portable and soft 3D-printed cast for laparoscopic choledochojejunostomy model in surgical training

BACKGROUND

Laparoscopic choledochojejunostomy (LCJ) is an essential basic skill for biliary surgeons. Therefore, we established a convenient and effective LCJ 3D printing model to evaluate whether the model could simulate the actual operation situation and determine its effectiveness and validity in surgical training.

METHODS

A 3D printing dry laboratory model was established to simulate LCJ. The face and content validity of the model were evaluated by six experienced biliary surgeons based on 5-point Likert scale questionnaires. A total of 15 surgeons with different levels of experience performed LCJ on the model and evaluated the structural validity of the model using the objective structured assessment of technical skills (OSATS). Simultaneously, the operation time of each surgery was also recorded. A study was also performed to further evaluate the learning curve of residents.

RESULTS

The operating space score of the model was 4.83 ± 0.41 points. The impression score of bile duct and intestinal canal was 4.33 ± 0.52 and 4.17 ± 0.41 points, respectively. The tactile sensation score of bile duct suture and intestinal canal suture was 4.00 ± 0.63 and 3.83 ± 0.41points, respectively. The OSATS score for model operation in the attending group was 29.20 ± 0.45 points, which was significantly higher than that in the fellow group (26.80 ± 1.10, P = 0.007) and the resident group (19.80 ± 1.30, P < 0.001). In addition, there was a statistical difference in operation time among surgeons of different experience levels (P < 0.05). Residents could significantly improve the surgical score and shorten the time of LCJ through repeated training.

CONCLUSIONS

The 3D printing LCJ model can simulate the real operation scenes and distinguish surgeons with different levels of experience. The model is expected to be one of the training methods for biliary tract surgery in the future.

A study of simulation training in laparoscopic bilioenteric anastomosis on a 3D-printed dry lab model

BACKGROUND

There are few studies on simulation training in laparoscopic bilioenteric anastomosis. There is also a lack of mature and reliable training models for bilioenteric anastomosis. In this study, we aimed to assess a feasible training model for bilioenteric anastomosis. Surgeons can improve their surgical ability by performing laparoscopic bilioenteric anastomosis on this model through repeated training.

METHOD

The original articles related to simulation training in surgical anastomosis were identified from January 2000 to November 2021 in the Clarivate Analytics Web of Science Core Collection database. We conducted a bibliometric analysis based on the country of these publications and the type of anastomosis. A 3D-printed bilioenteric anastomosis model was applied in this study. Baseline data of 15 surgeons (5 surgeons of Attendings, 5 surgeons of Fellows, and 5 surgeons of Residents) were collected. The bilioenteric anastomosis data, including the operation time and operation score, were recorded and analyzed. A study of the learning curve was also performed for further assessment.

RESULT

Surgeons at different levels of experience exhibited different levels of performance in conducting laparoscopic bilioenteric anastomosis on this model. Experienced surgeons completed their first training session in a shorter time and obtained a higher surgical score. In turn, repeated training significantly shortened the time of laparoscopic bilioenteric anastomosis for each trainer and improved the surgical score. Surgeons with different levels of experience needed different numbers of cases to reach the stable period of the learning curve. Experienced surgeons were able to reach a proficient level through fewer training cases.

CONCLUSION

A suitable biliary-enteric anastomosis model can help surgeons conduct simulation training and provide experience and skill accumulation for future real operations. Our training model performed well in this study and can effectively accomplish this goal.

Effectiveness of an immersive virtual reality environment on curricular training for complex cognitive skills in liver surgery: a multicentric crossover randomized trial

BACKGROUND

Virtual reality (VR) is increasingly used in surgical education, but evidence of its benefits in complex cognitive training compared to conventional 3-dimensional (3D) visualization methods is lacking. The objective of this study is to assess the impact of 3D liver models rendered visible by VR or desktop interfaces (DIs) on residents' performance in clinical decision-making.

METHOD

From September 2020 to April 2021, a single-blinded, crossover randomized educational intervention trial was conducted at two university hospitals in Belgium and Italy. A proficiency-based stepwise curriculum for preoperative liver surgery planning was developed for general surgery residents. After completing the training, residents were randomized in one of two assessment sequences to evaluate ten real clinical scenarios.

RESULTS

Among the 50 participants, 46 (23 juniors/23 seniors) completed the training and were randomized. Forty residents (86.96%) achieved proficiency in decision-making. The accuracy of virtual surgical planning using VR was higher than that using DI in both groups A (8.43 ± 1.03 vs 6.86 ± 1.79, p < 0.001) and B (8.08 ± 0.9 vs 6.52 ± 1.37, p < 0.001).

CONCLUSION

Proficiency-based curricular training for liver surgery planning successfully resulted in the acquisition of complex cognitive skills. VR was superior to DI visualization of 3D models in decision-making.

CLINICALTRIALS

GOV ID

NCT04959630.

The Intraoperative Complications Assessment and Reporting with Universal Standards (ICARUS) Global Surgical Collaboration Project: Development of Criteria for Reporting Adverse Events During Surgical Procedures and Evaluating Their Impact on the Postoperative Course

BACKGROUND

Intraoperative adverse events (iAEs) are surgical and anesthesiologic complications. Despite the availability of grading criteria, iAEs are infrequently reported in the surgical literature and in cases for which iAEs are reported, these events are described with significant heterogeneity.

OBJECTIVE

To develop Intraoperative Complications Assessment and Reporting with Universal Standards (ICARUS) Global Surgical Collaboration criteria to standardize the assessment, reporting, and grading of iAEs. The ultimate aim is to improve our understanding of the nature and frequency of iAEs and our ability to counsel patients regarding surgical procedures.

DESIGN, SETTING, AND PARTICIPANTS

The present study involved the following steps: (1) collecting criteria for assessing, reporting, and grading of iAEs via a comprehensive umbrella review; (2) collecting additional criteria via a survey of a panel of experienced surgeons (first round of a modified Delphi survey); (3) creating a comprehensive list of reporting criteria; (4) combining criteria acquired in the first two steps; and (5) establishing a consensus on clinical and quality assessment utility as determined in the second round of the Delphi survey.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

Panel inter-rater agreement and consistency were assessed as the overall percentage agreement and Cronbach's α.

RESULTS AND LIMITATIONS

The umbrella review led to nine common criteria for assessing, grading, and reporting iAEs, and review of iAE grading systems led to two additional criteria. In the first Delphi round, 35 surgeons responded and two criteria were added. In the second Delphi round, 13 common criteria met the threshold for final guideline inclusion. All 13 criteria achieved the consensus minimum of 70%, with agreement on the usefulness of the criteria for clinical and quality improvement ranging from 74% to 100%. The mean inter-rater agreement was 89.0% for clinical improvement and 88.6% for quality improvement.

CONCLUSIONS

The ICARUS Global Collaboration criteria might aid in identifying important criteria when reporting iAEs, which will support all those involved in patient care and scientific publishing.

PATIENT SUMMARY

We consulted a panel of experienced surgeons to develop a set of guidelines for academic surgeons to follow when publishing surgical studies. The surgeon panel proposed a list of 13 criteria that may improve global understanding of complications during specific procedures and thus improve the ability to counsel patients on surgical risk.

What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature

A crucial element of any surgical training program is the ability to provide procedure-specific, objective, and reliable measures of performance. During robotic surgery, objective clinically relevant performance metrics (CRPMs) can provide tailored contextual feedback and correlate with clinical outcomes. This review aims to define CRPMs, assess their validity in robotic surgical training and compare CRPMs to existing measures of robotic performance. A systematic search of Medline and Embase databases was conducted in May 2022 following the PRISMA guidelines. The search terms included Clinically Relevant Performance Metrics (CRPMs) OR Clinically Relevant Outcome Measures (CROMs) AND robotic surgery. The study settings, speciality, operative context, study design, metric details, and validation status were extracted and analysed. The initial search yielded 116 citations, of which 6 were included. Citation searching identified 3 additional studies, resulting in 9 studies included in this review. Metrics were defined as CRPMs, CROMs, proficiency-based performance metrics and reference-procedure metrics which were developed using a modified Delphi methodology. All metrics underwent both contents and construct validation. Two studies found a strong correlation with GEARS but none correlated their metrics with patient outcome data. CRPMs are a validated and objective approach for assessing trainee proficiency. Evaluating CRPMs with other robotic-assessment tools will facilitate a multimodal metric evaluation approach to robotic surgery training. Further studies should assess the correlation with clinical outcomes. This review highlights there is significant scope for the development and validation of CRPMs to establish proficiency-based progression curricula that can be translated from a simulation setting into clinical practice.

Proficiency-based progression training for robotic surgery skills training: a randomized clinical trial

OBJECTIVE

To determine whether proficiency-based progression (PBP) training leads to better robotic surgical performance compared to traditional training (TT), given that the value of PBP training for learning robotic surgical skills is unclear.

MATERIALS AND METHODS

The PROVESA trial is a multicentric, prospective, randomized and blinded clinical study comparing PBP training with TT for robotic suturing and knot-tying anastomosis skills. A total of 36 robotic surgery-naïve junior residents were recruited from 16 training sites and 12 residency training programmes. Participants were randomly allocated to metric-based PBP training or the current standard of care TT, and compared at the end of training. The primary outcome was percentage of participants reaching the predefined proficiency benchmark. Secondary outcomes were the numbers of procedure steps and errors made.

RESULTS

Of the group that received TT, 3/18 reached the proficiency benchmark versus 12/18 of the PBP group (i.e. the PBP group were ~10 times as likely to demonstrate proficiency [P = 0.006]). The PBP group demonstrated a 51% reduction in number of performance errors from baseline to the final assessment (18.3 vs 8.9). The TT group demonstrated a marginal improvement (15.94 vs 15.44) in errors made.

CONCLUSIONS

The PROVESA trial is the first prospective randomized controlled trial on basic skills training in robotic surgery. Implementation of a PBP training methodology resulted in superior surgical performance for robotic suturing and knot-tying anastomosis performance. Compared to TT, better surgical quality could be obtained by implementing PBP training for basic skills in robotic surgery.

Metric-Based Simulation Training to Proficiency for Endovascular Thrombectomy in Ischemic Stroke

Ischemic stroke is one of the leading causes of death and long-term disability in the West. Mechanical revascularization techniques are considered the standard of care for large vessel occlusive stroke. Traditional apprenticeship models involve doctors training their skills on patients. Simulation platforms have long been recognized as an alternative to this. There has however been very little robust assessment of the training outcomes achieved on some of these platforms. At best, these simulations increase understanding of the procedural process and may help improve some technical skills; at worst they may instill bad habits and poor technique. The prerequisite of any simulation process must be to teach what to do, with which devices, in the correct sequence as well as what not to do. It should provide valid metric-based feedback to the trainee that is objective, transparent, and fair for formative and summative performance feedback. It should deliver a training program that measures the performance progress of trainees against expert benchmarks—benchmarks that represent an evidence-based peer-reviewed standard. In this paper, we present a perspective for PBP training for thrombectomy based on our experience with the process of procedure characterization, metric validation, and early experience of using this approach for proficiency training. Patient outcomes are not only determined by optimal performance in the Angio Suite but also by an efficient patient procedure pathway. There will be value in utilizing the PBP training standard not only for the procedure itself but also for the constituent elements of the stroke pathway to further improve treatment outcomes for ischemic stroke patients.