Simulation in Urological Training and Education (SIMULATE): Protocol and curriculum development of the first multicentre international randomized controlled trial assessing the transferability of simulation-based surgical training

Urology boot camp for medical students: Using virtual technology to enhance undergraduate education

Objectives

The study aims to describe the methodology of converting the urology boot camp for medical students into a virtual course with key take home points for a successful conversion and to present quantitative and qualitative data demonstrating the impact of the boot camp on improving delegates' knowledge and clinical acumen.

Materials and methods

The face-to-face boot camp was converted to a virtual format employing a variety of techniques including; utilizing an online platform to deliver live screened lectures, using online polling software to foster an interactive learning environment and displaying pre-recorded videos to teach practical skills. Validated Multiple Choice Questionnaires (MCQs) were used prior to and after the course. This enabled the assessment of delegates' knowledge of urology according to the national undergraduate curriculum, and paired t tests were used to quantify the level of improvement. Thematic analysis was carried out on post-course delegate feedback to identify highlights of the course and ways of improving future iterations.

Results

In total, 131 delegates took part in the pilot virtual course. Of these, 105 delegates completed the pre- and post-course MCQs. There was a statistically significant improvement in the assessment following the course (p = <0.001) with mean score increasing from 47.5% pre-course to 65.8% post-course. All delegates who attended the most recent implementation of the virtual course (n = 31) felt it improved their knowledge and confidence in urology. Twenty delegates (64.5%) felt that it prepared them for both final year medical school examinations and working as a foundation year doctor. Positive themes in feedback were identified, which included the interactive nature of the course, the quality of teaching, the level and content of information provided and the high yield, concise organization of the teaching schedule.

Conclusion

Using virtual technology and innovative educational frameworks, we have demonstrated the successful conversion of the urology boot camp for medical students to a virtual format. At a national level, with support from the British Association of Urological Surgeons, the face-to-face component of the course will continue to run in parallel with the virtual course with the aim of standardizing and improving UK undergraduate urological education. The virtual course has been implemented on an international scale, and this has already shown promising results.

A Systematic Review of Simulation-Based Training in Vascular Surgery

Introduction

Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods

PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results

Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content (35), response processes (12), the internal structure (5), relations to other variables (57), and consequences (2). Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions

Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.

Effect of Simulation-based Training on Surgical Proficiency and Patient Outcomes: A Randomised Controlled Clinical and Educational Trial

BACKGROUND

It is hypothesised that simulation enhances progression along the initial phase of the surgical learning curve.

OBJECTIVE

To evaluate whether residents undergoing additional simulation, compared to conventional training, are able to achieve proficiency sooner with better patient outcomes.

DESIGN, SETTING, AND PARTICIPANTS

This international, multicentre, randomised controlled trial recruited 94 urology residents with experience of zero to ten procedures and no prior exposure to simulation in ureterorenoscopy, selected as an index procedure.

INTERVENTION

Participants were randomised to simulation or conventional operating room training, as is the current standard globally, and followed for 25 procedures or over 18 mo.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

The number of procedures required to achieve proficiency, defined as achieving a score of ≥28 on the Objective Structured Assessment of Technical Skill (OSATS) scale over three consecutive operations, was measured. Surgical complications were evaluated as a key secondary outcome. This trial is registered at www.isrctn.com as ISCRTN 12260261.

RESULTS AND LIMITATIONS

A total of 1140 cases were performed by 65 participants, with proficiency achieved by 21 simulation and 18 conventional participants over a median of eight and nine procedures, respectively (hazard ratio [HR] 1.41, 95% confidence interval [CI] 0.72-2.75). More participants reached proficiency in the simulation arm in flexible ureterorenoscopy, requiring a lower number of procedures (HR 0.89, 95% CI 0.39-2.02). Significant differences were observed in overall comparison of OSATS scores between the groups (mean difference 1.42, 95% CI 0.91-1.92; p < 0.001), with fewer total complications (15 vs 37; p = 0.003) and ureteric injuries (3 vs 9; p < 0.001) in the simulation group.

CONCLUSIONS

Although the number of procedures required to reach proficiency was similar, simulation-based training led to higher overall proficiency scores than for conventional training. Fewer procedures were required to achieve proficiency in the complex form of the index procedure, with fewer serious complications overall.

PATIENT SUMMARY

This study investigated the effect of simulation training in junior surgeons and found that it may improve performance in real operating settings and reduce surgical complications for complex procedures.

Aligning Urology Residency Training With Real-World Workforce Needs

OBJECTIVE

Research suggests recently graduated urology residents do not feel ready for independent practice. We conducted a study to determine if Accreditation Council for Graduate Medical Education (ACGME) minimum case requirements, resident case logs, and graduating resident perceived readiness for practice are aligned with the procedural demand and needs of the current urology workforce.

DESIGN

Correlative study comparing the association between (1) workforce demand and ACGME case requirements, and (2) workforce demand and perceived resident competency. Three distinct datasets were used; (1) the 2017 Medicare Part B National Summary Data File; (2) the 2017 National Data Report published by the ACGME; and (3) a graduating resident survey from Okhunov et al. SETTING: N/A.

PARTICIPANTS

N/A.

RESULTS

In 2017, there were a total of 6,784,696 urologic cases performed through Medicare. We found nonsignificant positive associations between resident case logs (rho = 0.16, p = 0.5784), ACGME minimum procedure requirements (rho = 0.42, p = 0.1255), and Medicare procedural demand. Our 15 index procedures accounted for 21.1% (n = 1,431,775) of all Medicare cases, with a median number of 7706 procedures. Endopyelotomy was the least common procedure (n = 98), while cystoscopy was the most common (n = 980,623). Medicare case volume was positively correlated with graduating residents' procedural confidence (r = 0.86, p < 0.0001). We identified four categories with varied alignment of training and demand: (1) high volume and high confidence, (2) high volume and low confidence, (3) low volume and high confidence, and (4) low volume and low confidence.

CONCLUSIONS

Optimizing urology residency training is time-sensitive and important. Using national Medicare data coupled with recently graduated urology resident survey results, we provide a guiding framework for improving the alignment of training with workforce demand. Informed by these results, we recommend altering training requirements to reflect these needs.

Innovations in Urologic Surgical Training

PURPOSE OF REVIEW

This review aims to summarize innovations in urologic surgical training in the past 5 years.

RECENT FINDINGS

Many assessment tools have been developed to objectively evaluate surgical skills and provide structured feedback to urologic trainees. A variety of simulation modalities (i.e., virtual/augmented reality, dry-lab, animal, and cadaver) have been utilized to facilitate the acquisition of surgical skills outside the high-stakes operating room environment. Three-dimensional printing has been used to create high-fidelity, immersive dry-lab models at a reasonable cost. Non-technical skills such as teamwork and decision-making have gained more attention. Structured surgical video review has been shown to improve surgical skills not only for trainees but also for qualified surgeons. Research and development in urologic surgical training has been active in the past 5 years. Despite these advances, there is still an unfulfilled need for a standardized surgical training program covering both technical and non-technical skills.

Editorial Commentary: Simulation-Based Training in Orthopaedic Surgery: Current Evidence and Limitations

Simulation-based training has been widely adopted by surgical educators and is now an essential component of the modern resident's skills acquisition pathway and career progression. The challenges faced by residents because of lack of exposure as a result of working-time directives-and now the COVID-19 (coronavirus disease 2019) pandemic limiting nonurgent and elective operating-reinforce the need for evidence-based simulation training. Although a wide range of training platforms have been developed, very few have shown transfer of skills. Simulation is thought to enhance the initial phase of the procedural learning curve; however, this hypothesis is yet to be tested in a high-quality study. Nevertheless, in light of the current evidence, simulation-based procedural curricula should be developed using the strengths of multiple different training platforms while incorporating the essential concept of nontechnical skills.

The SIMULATE ureteroscopy training curriculum: educational value and transfer of skills

Objective

Different simulation modalities may be utilised in a curricular fashion to benefit from the strengths of each training model. The aim of this study is to evaluate a novel multi-modality ureterorenoscopy (URS) simulation curriculum in terms of educational value, content validity, transfer of skills and inter-rater reliability.

Methods

This international prospective study recruited urology residents (n = 46) with ≤ 10 URS experience and no prior simulation training. Participants were guided through each phase of the expert-developed SIMULATE URS curriculum by trainers and followed-up in the operating room (OR). Video recordings were obtained during training. A post-training evaluation survey was distributed to evaluate content validity and educational value, using descriptive statistics. Performance was evaluated using the objective structured assessment of technical skills (OSATS) scale to measure improvement in scores throughout the curriculum. Pearson’s correlation coefficient and Cohen’s kappa tests were utilised to investigate correlation and agreement between raters.

Results

Participants reported gaining OR-transferrable skills (Mean: 4.33 ± 0.67) and demonstrated marked improvement in throughout the curriculum, transferred to the OR for both semi-rigid URS (p = 0.004) and flexible URS (p = 0.007). 70% of participants were successfully followed-up in the OR (n = 32). No differences were identified with the additional use of fresh frozen cadavers (p = 0.85, p = 0.90) and the URO Mentor VR simulator (p = 0.13, p = 0.22). A moderate level of correlation was noted on the video OSATS assessments, between two expert assessors (r = 0.70), but a poor agreement with the live rating.

Conclusion

The SIMULATE URS training curriculum received high educational value from participants, who demonstrated statistically significant improvement with consecutive cases throughout the curriculum and transferability of skills to the OR in both semi-rigid and flexible URS.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00345-021-03604-w.

Simulation-Based Training Models for Urolithiasis: A Systematic Review

Objectives: Urolithiasis is one of the most common presentations in urological practice and it is becoming increasingly important to provide structured, simulation-based training using validated training models. This systematic review aims to identify current simulation-based training models and to evaluate their validity and effectiveness. Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a comprehensive search was performed on the Medline and EMBASE databases for English language articles published between 2000 and 2018 describing and/or assessing validity of simulation models for urolithiasis procedures. Studies were qualitatively assessed for validity using the Messick validity framework and models were assigned levels of recommendation using the McGaghie model of translational outcomes. Results: A total of 98 studies were included in this study assessing 51 models, with 28 studies concerning models for urethrocystoscopy, 46 studies for ureterorenoscopy, and 39 studies for percutaneous access and/or nephrolithotomy. Only four models demonstrated a level of recommendation of 4. The most validated models were the URO/PERC-Mentor (Simbionix, Lod, Israel) with multiple studies for each across various procedural skills. Conclusion: There is a wide spectrum of simulation-based models currently available for urolithiasis procedures, mostly with limited validity evidence from small studies. Further research is required with higher levels of evidence including randomized controlled trials. In addition, long-term transfer of skills to the operating room should be assessed to establish whether there is genuine skill development and retention using simulation models and whether this helps to reduce surgical complications.

Non-technical skills for urological surgeons (NoTSUS): development and evaluation of curriculum and assessment scale

Objective

In the last decade non-technical skills (NTS) have emerged as a vital area for improvement within surgery. This study aims to develop and evaluate a Non-technical Skills for Urological Surgeons (NoTSUS) training curriculum and assessment scale.

Methods

This international, longitudinal and observational study began with a 3-round Delphi methodology to refine curriculum contents and rating scale. Sessions with up to four participants were delivered where each candidate undertook an independent scenario within the validated full immersion simulation environment. Candidates were assessed using both the NoTSS (Non-technical Skills for Surgeons) and NoTSUS rating scales by NTS-trained and non-trained experts. A post-training evaluation survey was distributed.

Results

62 participants comprising trainees (n = 43) and specialists (n = 19) undertook the NoTSUS course. The NoTSS and NoTSUS scales correlated well, with a mean difference of 3.3 in the overall total (p = 0.10, r = 0.53). However, there was significant differences in scores between the NoTSS-trained and non-trained raters (n = 28, p = 0.03). A one-way ANOVA test revealed significant improvement throughout the four simulation scenarios in each session (p = 0.02). The NoTSUS curriculum received positive feedback from participants and demonstrated educational value and acceptability.

Conclusions

The NoTSUS curriculum has demonstrated high educational value for NTS training aimed at urologists, with marked improvement throughout sessions. Correlation of NoTSUS and NoTSS scales proves its suitability for evaluating NTS in future training. Demonstration of inter-rater reliability indicates that the scale is reliable for use in assessment by expert faculty members. Furthermore, qualitative feedback from participants suggests gain of transferrable skills over the course.

Electronic supplementary material

The online version of this article (10.1007/s00345-020-03406-6) contains supplementary material, which is available to authorized users.