Validation of computer-based training in ureterorenoscopy

Evaluating the learning curve for flexible ureteroscopic thulium fiber laser lithotripsy using CUSUM analysis

OBJECTIVES

Effective training methods and learning curve (LC) assessment are crucial for more difficult endoscopic procedures. The present study sought to assess the LC of flexible ureteroscopes (fURS) for thulium fiber laser (TFL) lithotripsy and determine the effectiveness of using a porcine kidney model in training modality, to inform future training.

METHODS

Postgraduate medical students without experience in ureteroscopy were recruited, and surgical data were collected from 10 consecutive flexible ureteroscopic lithotripsy (fURL) procedures performed on our homemade porcine kidney training model. Cumulative sum (CUSUM) analysis and pooled mean CUSUM curves were applied to determine the LC turning points (TPs), and pre-LC and post-LC data were compared with that of an experienced attending physician.

RESULTS

Data from 110 surgeries were included in the analysis. The mean number of surgical units to overcome the LC for the duration of fURS TFL lithotripsy was 4. The operation time, number of tissue damage, and number of central visual shifts were significantly lower in students after the TP than before. No statistically significant difference in operation time was found between the students after the LC (817.50 [703.75-964.75]) and the surgeon group (732.50 [51.00-822.25]).

CONCLUSION

Overcoming the LC of fURS TFL lithotripsy requires four surgeries, and the developed ex vivo porcine kidney is a conveniently accessible and effective clinical training model.

Global viewpoints: proposals for the development of endourology in resource-limited countries in Africa

Endourology plays an important role in modern urological practice. Compared to open surgery, it offers many advantages. In Africa, endourology is not widely practiced or non-existent in some referral centres. Several factors have been linked to this challenge. This article explores and proposes strategies to improve endourology practice in the African context. Recognising the unique challenges and opportunities in the region, the document discusses key initiatives and recommendations to promote the growth and development of endourological practices, including the identification of local needs, training, technological adaptation, etc. It aims to provide valuable information on the advancement of endourology in Africa.

Education and training evolution in urolithiasis: A perspective from European School of Urology

Objective

With the worldwide increase in urolithiasis prevalence, the present study aimed to delineate and summarise recent evolutions in training for the management of urolithiasis.

Methods

A PubMed search using the key words "simulation", "training", "ureteroscopy", "RIRS", "URS", "percutaneous nephrolithotomy", "PCNL", "virtual reality", "augmented reality", "artificial intelligence", "healthcare", "curriculum", and "assessment" was used to examine how education and training in urolithiasis have adapted over recent years. Focus was placed on the role of high- and low-fidelity simulation models, virtual reality and artificial intelligence, and standardised assessment and curriculum.

Results

This review supports the necessity to incorporate technology, simulation, and other skill enhancement training modalities into surgical training. However, these cannot solely replace mentored training with an experienced professional supervisor. Whilst technological and simulation advancements are likely to prove increasingly important in urolithiasis training, it is just as important for stratification of robust curricula with validated assessment. We also propose a pathway for future training.

Conclusion

Endourology subspeciality training programmes have successfully incorporated simulation model-based training for skill acquisition, refinement, and improving operative outcomes. Success is achieved by maintaining a delicate balance between machine and in person mentor-based training. A trainee-centred model that follows the proposed curriculum could aid this balance for the future generation of trainees.

[Impact of simulation-based training in endourology: A systematic review of the literature]

INTRODUCTION

The impact of simulation on the acquisition of surgical skills as well as their transfer to the operating room is still debated. The objective was to assess these two specific points, focusing on the field of endourology.

METHODS

A systematic review of the literature, following the PRIMA statement, was performed using Medline database through September 2021 without time limit. Studies focusing on the impact of simulators on the acquisition of surgical technical skills as well as their transfer to the operating room in the field of endourology were included. The endourological procedures were: cystoscopy, ureteroscopy, percutaneous nephrolithotomy, endoscopic treatment of benign prostatic hyperplasia, endoscopic bladder resection.

RESULTS

Among the 11,442 publications identified, fifty-two studies were included in the analysis. The majority reported an improvement in procedure time of the requested tasks and dexterity of participants, regardless of the type of simulator and procedure. The level of evidence of included studies was often low. Few studies evaluated the transfer of acquired skills from the simulator to the operating room.

CONCLUSION

This review showed the positive impact of simulation on the acquisition of technical skills in endourology. However, in order to include proficiency-based progression in the curriculum of trainees, some parameters such as the choice of reference simulators, choice of tasks, and method of validation of acquired skills must be validated in a consensual manner to offer a quality training.

Effectiveness of a newly-developed training module using 3D printing for the navigation during retrograde intrarenal surgery

Purpose

This study aimed to evaluate the feasibility of the newly-developed three-dimensional (3D) printed training module for navigation during retrograde intrarenal surgery.

Materials and Methods

Two specialists provided orientation to all trainees. The 3D printing model consisted of eight calyces in each kidney. One navigation time started from the moment when the endoscope entered the ureter. After navigation was completed, the navigation time was recorded. The goal was to perform ten navigation times for each side, starting from the right or the left side at random. After the experiment, all trainees were asked to fill out a questionnaire.

Results

The average training period of all 17 trainees was 3.05±1.80 years. Eleven trainees (64.7%) had the experience of assisting surgery for <100 cases, and six trainees (35.3%) had the experience of assisting surgery for 100 to 500 cases. Nine trainees (52.9%) began training from the right, and eight trainees (47.1%) started from the left. The average navigation time of 308 trials was 153.4±92.6 seconds. The maximum and minimum navigation times were 354.3±177.2 seconds and 80.1±25.6 seconds. The mean navigation time of the first and the last trials of all trainees significantly decreased from 251.4±108.0 seconds to 93.9±33.2 seconds. The average reduction in navigation time was 201.3±133.3 seconds. Almost all trainees were satisfied with the training.

Conclusions

The newly-developed 3D printing navigation training module seems to be adequate to improve surgical skills of flexible ureteroscopy.

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal - a validation study

OBJECTIVE

The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning.

DESIGN

This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process.

RESULTS

Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach's alpha = 0.14, p = 0.15) and across the three modules (Cronbach's alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences.

CONCLUSION

Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

One- versus two-surgeon active stone retrieval procedures for flexible ureteroscopy: An off-site simulator comparative study

OBJECTIVES

To evaluate the efficacy of the one-surgeon basketing technique compared to the conventional two-surgeon method for flexible ureteroscopy.

METHODS

Seven urologists (three trained and four less-trained surgeons) extracted three renal stones using each technique with an off-site simulator. We analyzed the task completion time, the number of times the basket forceps were opened and closed, and unexpected events during stone-catching, for both techniques.

RESULTS

There was no significant difference in the total task completion time between the one-surgeon (102.0 [interquartile range 63.5-164.0] s) and two-surgeon (99.5 [interquartile range 75.0-145.3] s) techniques. However, the time it took the trained surgeons to extract all stones was significantly shorter with the one-surgeon than with the two-surgeon technique (19.0 [interquartile range 18.0-20.0] s vs 34.0 [interquartile range 25.0-40.0] s; P = 0.049). Among the less-trained surgeons, the opposite effect was observed, but this difference was not statistically significant (96.5 [interquartile range 61.0-134.5] s vs 64.0 [interquartile range 51.5-77.0] s; P = 0.489). The number of times the forceps were opened and closed during stone-catching was significantly lower with the one-surgeon technique than with the two-surgeon technique (5.0 [interquartile range 4.0-5.0] times vs 9.0 [interquartile range 6.0-14.5] times; P = 0.018).

CONCLUSIONS

The one-surgeon technique can offer comparable efficacy to the two-surgeon technique. If surgeons are sufficiently trained, a one-surgeon basketing technique for performing flexible ureteroscopy might become a new technical modality for extracting renal stones.

Endourological Training Using 3D-Printed Bladder Phantoms: Development and Prospective Evaluation

Background: To create and evaluate a realistic, anatomically accurate, and user-friendly bladder phantom for reproducible endourological training purposes and endoscope mastery. Materials and Methods: The anatomy of full bladders was mapped from human computed tomography datasets. After a 3D model development process, content evidence and response process evidence (RPE) of the phantom were evaluated using the system usability scale (SUS), 5-point Likert scale questionnaires, and task execution of experienced urologists (U) and endoscopy-naive medical students (MS) in two training sessions (first vs second). Required validation cohort sizes (1:10) of the evaluating urologists (n = 12) and students (n = 115) were precalculated. Time measurements were recorded. Students were additionally evaluated by a validated global psychomotor assessment score (GPSS). Group comparisons were calculated by the Mann-Whitney U test. All tests were two sided with p < 0.05 considered statistically significant. Results: Content evidence was assessed by urologists with an "excellent" SUS score of 89.4 ± 5.9 and an average "agreement" of ≥4 pts in the Likert scale questionnaires. RPE was assessed by intra- and intergroup time comparison for the execution of endoscopic tasks (cystoscopy [CY], guidewire insertion, and tumor biopsy). For CY, U: first 17.6 ± 4.4 seconds vs second 12.4 ± 2.0 seconds, p = 0.002; MS: first 56.6 ± 28.2 seconds vs second 28.6 ± 14.7 seconds, p < 0.001; U vs MS: first U 17.6 ± 4.4 seconds vs first MS 56.6 ± 28.2 seconds, p < 0.001, second U 12.4 ± 2.0 seconds vs second MS 28.6 ± 14.7 seconds, p < 0.001. Significant time differences were documented for all tasks and sessions (p < 0.001). Additionally, significant GPSS differences were recorded between the sessions (GPSS: first 20.4 ± 5.1 pts vs second 24.7 ± 4.0 pts, p < 0.001). Conclusions: Our low-fidelity 3D-printed bladder, called BladCap, is an easy-to-assemble, inexpensive, and robust phantom. We present data, which establish construct validity to support use as a clinical training device.

Simulation-Based Training Models for Urolithiasis: A Systematic Review

Objectives: Urolithiasis is one of the most common presentations in urological practice and it is becoming increasingly important to provide structured, simulation-based training using validated training models. This systematic review aims to identify current simulation-based training models and to evaluate their validity and effectiveness. Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a comprehensive search was performed on the Medline and EMBASE databases for English language articles published between 2000 and 2018 describing and/or assessing validity of simulation models for urolithiasis procedures. Studies were qualitatively assessed for validity using the Messick validity framework and models were assigned levels of recommendation using the McGaghie model of translational outcomes. Results: A total of 98 studies were included in this study assessing 51 models, with 28 studies concerning models for urethrocystoscopy, 46 studies for ureterorenoscopy, and 39 studies for percutaneous access and/or nephrolithotomy. Only four models demonstrated a level of recommendation of 4. The most validated models were the URO/PERC-Mentor (Simbionix, Lod, Israel) with multiple studies for each across various procedural skills. Conclusion: There is a wide spectrum of simulation-based models currently available for urolithiasis procedures, mostly with limited validity evidence from small studies. Further research is required with higher levels of evidence including randomized controlled trials. In addition, long-term transfer of skills to the operating room should be assessed to establish whether there is genuine skill development and retention using simulation models and whether this helps to reduce surgical complications.

Global Assessment of Urological Endoscopic Skills (GAUES): development and validation of a novel assessment tool to evaluate endourological skills

OBJECTIVE

To develop and evaluate an assessment tool for endourological skills during simulation including cystoscopy, ureteroscopy (URS) and transurethral resection (TUR) procedures.

METHODS

We designed a Global Assessment of Urological Endoscopic Skills (GAUES) tool, comprised of nine endourology task-specific and two global-rating skills items. The tool was developed through two rounds of the Delphi process. The GAUES tool was used to assess acquisition of URS and TUR skills of novices (Year 2 core surgical trainees, CT2) and intermediate level trainees (residents at the start of the UK higher surgical training programme in Urology, Speciality Trainee Year 3, ST3) at the Urology Simulation Boot Camp (USBC) between 2016 and 2018. Validity was evaluated by comparing scores between trainees with different levels of urological experience. Inter-rater reliability was also assessed.

RESULTS

We evaluated 130 residents, 52% of trainees were at an intermediate stage of training and 39% were novices. In all, 9% of the anonymous forms were missing demographics. The completion rate of the GAUES tool during the USBC for URS and TUR was 85% and 89%, respectively. Our analysis demonstrated a significant difference in all domains between intermediates and novices at assessment in URS, except for one domain more suited to clinical assessment (P = 0.226). There was excellent intraclass correlation (ICC) overall between the two experts' judgements, ICC = 0.841 (95% confidence interval 0.767-0.893; P < 0.001, n = 88).

CONCLUSIONS

We have developed the novel GAUES tool for cystoscopic, URS and TUR skills. Overall, we demonstrated good face, content and construct validity and excellent reliability, suggesting that the GAUES tool can be useful for endourological skills assessment.

Current Educational Interventions for Improving Technical Skills of Urology Trainees in Endourological Procedures: A Systematic Review

Objective: Endourology continues to grow with the introduction of new technologies into clinical practice. Simulators and training models have been developed to improve comfort and proficiency in endoscopic procedures. The purpose of this systematic review was to examine the current educational interventions utilized to improve the performance of endourology trainees and to critically appraise the strengths and limitations of each. Methods: A search of the Ovid MEDLINE, EMBASE, PsycINFO, and the Cochrane Library databases was performed to identify literature focused on current educational interventions for improving technical skills of trainees in endourologic procedures. The Medical Education Research Study Quality Instrument (MERSQI) was used to evaluate the methodological quality of the abstracted articles. Results: Of the 2236 articles identified, 22 met the inclusion criteria. The types of educational interventions included: bench/wet lab models, virtual reality simulators, and instructional courses. Metrics used to quantify the impact of these interventions include global rating scales, Objective Structured Assessment of Technical Skills (OSATS) scores, and task-specific checklists. The setting of these evaluations comprises both virtual reality simulators and live surgery. Conclusions: In the surgical education literature, simulation-based training and assessment continues to play a prominent role in urologic training. The educational interventions highlighted in this review address various aspects of endourology, from stone management to transurethral resection. Additional work is needed to correlate technical performance in clinical and nonclinical settings with patient outcomes and develop a focused approach to nontechnical skill training.

Assessment of urology postgraduate trainees' competencies in flexible ureteroscopic stone extraction

INTRODUCTION

We aimed to assess flexible ureteroscopic stone extraction skill of urology postgraduate trainees (PGTs) at an Objective Structured Clinical Examination (OSCE) and to determine whether previous experience in the operating theatre or practice on the simulator correlated with performance.

METHODS

After obtaining ethics approval, PGTs from postgraduate years (PGYs) 3-5 were recruited from all four Quebec urology training programs during an OSCE. After a short orientation to the UroMentor™ simulator, PGTs were asked to perform Task 10 for 15 minutes, where two small stones from the left proximal ureter and renal pelvis were extracted using a basket. Competency of PGTs in performing the task was assessed using objective assessment from the simulator and subjective evaluations using Ureteroscopy-Global Rating Scale (URS-GRS). Simulator performance reports and URS-GRS scores were analyzed.

RESULTS

Thirty PGTs (9 PGY-3, 11 PGY-4, 10 PGY-5) participated in this study. PGTs had performed a mean of 55.9 semi-rigid and 45.7 flexible ureteroscopies prior to the study. Mean URS-GRS score of the participants was 20.0±4.4. Using norm-referenced method with three experts, cutoff score of 19 on the URS-GRS was determined to indicate competency. Sixty percent (18/30) of PGTs were competent. All eight PGTs who had practiced on the simulator were competent. Previous experience in the operating theatre and PGY level did not correlate with performance.

CONCLUSIONS

This study confirmed the feasibility of incorporating the UroMentor at OSCEs to assess competency of urology PGTs in ureteroscopic stone extraction skill. PGTs who practiced on the simulator scored significantly higher than those who did not practice; however, the software needs to be updated to improve its face validity and to include more complex tasks, such as holmium laser lithotripsy. Future studies with larger sample sizes and more complex cases are needed to confirm these results.

A Mobile-Based Surgical Simulation Application: A Comparative Analysis of Efficacy Using a Carpal Tunnel Release Module

PURPOSE

The utilization of surgical simulation continues to grow in medical training. The TouchSurgery application (app) is a new interactive virtual reality smartphone- or tablet-based app that offers a step-by-step tutorial and simulation for the execution of various operations. The purpose of this study was to compare the efficacy of the app versus traditional teaching modalities utilizing the "Carpal Tunnel Surgery" module. We hypothesized that users of the app would score higher than those using the traditional education medium indicating higher understanding of the steps of surgery.

METHODS

A total of 100 medical students were recruited to participate. The control group (n = 50) consisted of students learning about carpal tunnel release surgery using a video lecture utilizing slides. The study group (n = 50) consisted of students learning the procedure through the app. The content covered was identical in both groups but delivered through the different mediums. Outcome measures included comparison of test scores and overall app satisfaction.

RESULTS

Test scores in the study group (89.3%) using the app were significantly higher than those in the control group (75.6%). Students in the study group rated the overall content validity, quality of graphics, ease of use, and usefulness to surgery preparation as very high (4.8 of 5).

CONCLUSIONS

Students utilizing the app performed better on a standardized test examining the steps of a carpal tunnel release than those using a traditional teaching modality. The study findings lend support for the use of the app for medical students to prepare for and learn the steps for various surgical procedures.

CLINICAL RELEVANCE

This study provides useful information on surgical simulation, which can be utilized to educate trainees for new procedures.

Initial Content Validation Results of a New Simulation Model for Flexible Ureteroscopy: The Key-Box

PURPOSE

We sought to test the content validity of a new training model for flexible ureteroscopy: the Key-Box.

MATERIAL AND METHODS

Sixteen medical students were randomized to undergo a 10-day training consisting of performing 10 different exercises aimed at learning specific movements with the flexible ureteroscope, and how to catch and release stones with a nitinol basket using the Key-Box (n = 8 students in the training group, n = 8 students in the nontraining control group). Subsequently, an expert endourologist (O.T.) blindly assessed skills acquired by the whole cohort of students through two exercises on ureteroscope manipulation and one exercise on stone capture selected among those used for the training. A performance scale (1-5) assessing different steps of the procedure was used to evaluate each student. Time to complete the exercises was measured. Mann-Whitney Rank Sum test was used for comparisons between the two groups.

RESULTS

Mean scores obtained by trained students were significantly higher compared with those obtained by nontrained students (all p < 0.001). All trained students were able to complete the two exercises on ureteroscope manipulation within 3 minutes, whereas two students (25%) were not able to finish the exercise on stone capture. Conversely, four (50%) and six (75%) nontrained students were not able to finish one out of the two exercises on ureteroscope manipulation and the exercise on stone capture, respectively. The mean time to complete the three exercises was 76.3, 69.9, and 107 and 172.5, 137.9, and 168 seconds in the trained and nontrained groups, respectively (all p < 0.001).

CONCLUSIONS

The K-Box^® seems to be a valid easy-to-use training model for initiating novel endoscopists to flexible ureteroscopy.

Validation of the PASSPORT V2 training environment for arthroscopic skills

PURPOSE

Virtual reality simulators used in the education of orthopaedic residents often lack realistic haptic feedback. To solve this, the (Practice Arthroscopic Surgical Skills for Perfect Operative Real-life Treatment) PASSPORT simulator was developed, which was subjected to fundamental changes: improved realism and user interface. The purpose was to demonstrate its face and construct validity.

METHODS

Thirty-one participants were divided into three groups having different levels of arthroscopic experience. Participants answered questions regarding general information and the outer appearance of the simulator for face validity. Construct validity was assessed with one standardized navigation task, which was timed. Face validity, educational value and user-friendliness were determined with two representative exercises and by asking participants to fill out the questionnaire. A value of 7 or greater was considered sufficient.

RESULTS

Construct validity was demonstrated between experts and novices. Median task time for the fifth trial was 55 s (range 17-139 s) for the novices, 33 s (range 17-59 s) for the intermediates, and 26 s (range 14-52 s) for the experts. Median task times of three trials were not significantly different between the novices and intermediates, and none of the trials between intermediates and experts. Face validity, educational value and user-friendliness were perceived as sufficient (median >7). The presence of realistic tactile feedback was considered the biggest asset of the simulator.

CONCLUSION

Proper preparation for arthroscopic operations will increase the quality of real-life surgery and patients' safety. The PASSPORT simulator can assist in achieving this, as it showed construct and face validity, and its physical nature offered adequate haptic feedback during training. This indicates that PASSPORT has potential to evolve as a valuable training modality.

Validating Touch Surgery™: A cognitive task simulation and rehearsal app for intramedullary femoral nailing

BACKGROUND

The role of simulation in orthopaedic surgical training is becoming increasingly evident, as simulation allows repeated sustained practice in an environment that does not harm the patient. Previous studies have shown that the cognitive aspects of surgery are of equal if not greater importance in developing decision making than the practical aspects.

AIM

To observe construct, content and face validity of four IFN modules on a cognitive simulator, Touch Surgery™.

METHODS

39 novices and 10 experts were recruited to complete four simulation modules on surgical decision-making that represented the procedural steps of preparing the patient and equipment, inserting and locking an intramedullary femoral nail. Real-time objective performance metrics were obtained, stored electronically and analysed using median and Bonett-Price 95% confidence intervals from the participant's primary attempt to assess for construct validity. The median score of a post-study questionnaire using 5-point Likert scales assessed face and content validity. Data was confirmed as non-parametric by the Kolmogorov-Smirnov test. Significance was calculated using the Mann-Whitney U test for independent data whilst the Wilcoxon signed ranked test was used for paired data. Significance was set as 2-tailed p-value<0.05.

RESULTS

Experts significantly outperformed novices in all four modules to demonstrate construct validity (p<0.001). Specifically, experts scored 32.5% higher for patient positioning and preparation (p<0.0001), 31.5% higher for femoral canal preparation (p<0.0001), 22.5% higher for proximal locking (p<0.0001) and 17% higher scores for distal locking and closure (p<0.001). Both cohorts either agreed or strongly agreed that the graphics, simulated environment and procedural steps were realistic. Also, both cohorts agreed that the app was useful for surgical training and rehearsal, should be implemented within the curriculum and would want to use it to learn other surgical procedures.

CONCLUSION

IFN on the Touch Surgery app demonstrated construct, face and content validity. Users can demonstrate cognitive competencies prior to performing surgical procedures in the operating room. The application is an effective adjunct to traditional learning methods and has potential for curricular implementation.

Validation of the ArthroS virtual reality simulator for arthroscopic skills

PURPOSE

Virtual reality simulator training has become important for acquiring arthroscopic skills. A new simulator for knee arthroscopy ArthroS™ has been developed. The purpose of this study was to demonstrate face and construct validity, executed according to a protocol used previously to validate arthroscopic simulators.

METHODS

Twenty-seven participants were divided into three groups having different levels of arthroscopic experience. Participants answered questions regarding general information and the outer appearance of the simulator for face validity. Construct validity was assessed with one standardized navigation task. Face validity, educational value and user friendliness were further determined by giving participants three exercises and by asking them to fill out the questionnaire.

RESULTS

Construct validity was demonstrated between experts and beginners. Median task times were not significantly different for all repetitions between novices and intermediates, and between intermediates and experts. Median face validity was 8.3 for the outer appearance, 6.5 for the intra-articular joint and 4.7 for surgical instruments. Educational value and user friendliness were perceived as nonsatisfactory, especially because of the lack of tactile feedback.

CONCLUSION

The ArthroS™ demonstrated construct validity between novices and experts, but did not demonstrate full face validity. Future improvements should be mainly focused on the development of tactile feedback. It is necessary that a newly presented simulator is validated to prove it actually contributes to proficiency of skills.

Do high-fidelity training models translate into better skill acquisition for an endourologist?

PURPOSE OF REVIEW

Nowadays, accessibility to the operative room is becoming more limited for medical students and residents, principally due to decreasing operative time, increasing waiting list, ethical consideration and legal issue in case of any complications. Simulation models have gained in popularity and are now considered a major component in the training and skill development of medical students and residents before coming to the operative room. In this review, we summarized and discussed the relevant aspect of ureteroscopy training models and gave an overview of the advantage in skill acquisition while training with a high-fidelity model.

RECENT FINDINGS

Currently, there is an increase in surgical programs trying to implement endourology training models into the curriculum. The training simulators that would allow the medical students and residents to rapidly reach an autonomous level are yet to be developed. Several ureteroscopy models have been described and validated; however, the transposition of skill acquisition into real-life surgery is not properly demonstrated.

SUMMARY

Training reduces the learning curve for novice medical students or residents. However, further studies are still needed to better define the impact of skill acquisition in real life and its sustainability.

Flexible ureteroscopy training for surgeons using isolated porcine kidneys in vitro

BACKGROUND

To evaluate the feasibility of flexible ureteroscopy training by using isolated porcine kidneys and ureters in vitro.

METHODS

Twenty young urologists were randomly divided into four groups. Overall performance was assessed based on a global rating scale, pass/fail rating, total time to complete task, learning curve, incidence of trauma, and perforations. The effect of training was determined by comparing their performance in baseline with that in the post-test.

RESULTS

After the training, average operation time significantly decreased from 18 ± 3.4 min to 11 ± 1.2 min (P < 0.05). The urologists exhibited a relatively stable performance level after the sixth operation. Significant differences were observed between pre-test and post-test with respect to the global rating scale and the pass/fail rating (P < 0.05). However, the incidence of mucosal trauma and perforations did not change significantly (P = 0.26 and 0.35, respectively).

CONCLUSIONS

The isolated porcine kidneys are convenient and intuitive models for young urologists to practice flexible ureteroscopy on.

Impact of surgical experience on stone-free rates of ureteroscopy for single urinary calculi of the upper urinary tract: a matched-paired analysis of 600 patients

INTRODUCTION

To evaluate the impact of surgical experience on ureteroscopic stone-free rates (SFR) and complication rates (CR) for the treatment of urinary calculi of the upper urinary tract.

METHODS

We evaluated retrospectively, patients (n=300) having undergone ureteroscopy (URS) for single urinary calculi treated by residents (n=12) at our department over a 6-year period. These patients were matched according to age, gender, body-mass index, and stone side/size/site with patients (n=300) treated by consultants (n=5) of our department during the same period. Patient data, primary SFR, and CR were compared.

RESULTS

The mean±standard deviation (range) stone size was 6.39±3.26 (2-20) mm. The primary SFR after one URS procedure was 95.2% and did not differ between residents and consultants (95% vs 95.3%, p=0.489). The SFR were 95.9% and 98.5% for ureteral stones (p=0.125) and 93.2% and 89.3% for kidney stones (p=0.298) in the resident and consultant group, respectively. The SFR differed significantly between ureteral and kidney stones (97.2% vs 91.3%, p≤0.001). Perioperative complications occurred in a total of 63 patients (10.5%): Clavien 1: 3.8%, Clavien 2: 2%, Clavien 3a: 1.8%, and Clavien 3b: 2.8%, respectively. There were no differences in the total CR between residents (12%) and consultants (9%) (p=0.2116). However, the ureteral perforation rate was significantly higher in residents compared with consultants (4.3% vs 1.3%, p≤0.027).

CONCLUSIONS

URS is a safe and efficacious procedure for the treatment of single urinary calculi. Resident status does not compromise the SFR after ureteroscopic treatment of single urinary calculi. However, the incidence of ureteral perforation was associated with surgeon's experience.

A novel cadaveric simulation program in urology

OBJECTIVE

To evaluate the urology human cadaver training program developed by the British Association of Urological Surgeons.

DESIGN

This prospective, observational comparative study recruited urology residents, with different levels of experience, in 2 sessions of a 3-day modular cadaveric operative urology training. Participants performed various procedures on fresh-frozen cadaveric specimens, as per module, supervised by certified urological surgeons. At the conclusion of each module, all residents and faculty were invited to complete an evaluation survey.

SETTING

The training days were hosted by the British Association of Urological Surgeons at the University of Manchester Surgical Skills and Simulation Centre.

PARTICIPANTS

A total of 81 urology residents were recruited, with a maximum of 14 participants attending each module, over 2 sessions. We allocated 2 participants to each cadaver with access to all necessary equipment and guidance.

RESULTS

A total of 102 evaluation surveys were received from the trainees and faculty; a response rate of 94%. All procedures scored a mean of 3 on 5 for face validity, which is higher than the acceptability range. Regarding content validity, participants and faculty rated all aspects ≥3 on 5. Respondents held a positive view of the cadaver sessions and believed them to be useful for learning anatomy and steps of an operation (mean = 4.54) and as a confidence booster for performing a procedure (mean = 4.33). Furthermore, it was thought that the training program significantly improved skills (mean = 4.11), gave transferrable skills for the operating room (mean = 4.21), and was feasible to be incorporated into training programs (mean = 4.29). Human cadaveric simulation was rated as the best mode of simulation-based training for all the procedures in the curriculum.

CONCLUSIONS

This study on cadaveric simulation training demonstrated face and content validities. It also showed feasibility, acceptability, a high value for educational influence and cost-effectiveness for cadaveric simulation. A simulation-based training pathway has been proposed for effective procedural training in urology.

Simulation-based ureteroscopy training: a systematic review

OBJECTIVE

Simulation is a common adjunct to operative training and various modalities exist for ureteroscopy. This systematic review aims the following: (1) to identify available ureteroscopy simulators, (2) to explore evidence for their effectiveness using characteristic criterion, and (3) to provide recommendations for simulation-based ureteroscopy training.

DESIGN

The preferred reporting items for systematic reviews and meta-analysis statement guidelines were used. A literature search was performed using the PubMed, EMBASE, and Cochrane Library databases.

RESULTS

In total, 20 articles concerning ureteroscopy simulators were included. Overall, 3 high-fidelity bench models are available. The Uro-Scopic Trainer has demonstrated face, construct, and concurrent validity, whereas the Scope Trainer has undergone content, construct, and predictive validation. The adult ureteroscopy trainer has demonstrated face, content, and construct validity. The URO Mentor is the only available ureteroscopy virtual-reality system; 10 studies were identified demonstrating its face, content, construct, concurrent, and predictive validity. The Uro-Scopic Trainer, the Scope Trainer, and the URO Mentor have demonstrated high educational impact. A noncommercially available, low-fidelity model has demonstrated effectiveness comparable to its high-fidelity counterpart at 185 times lesser than the price of the Uro-Scopic Trainer. The use of porcine models has also been described in 3 studies but require further study.

CONCLUSIONS

Valid models are available for simulation-based ureteroscopy training. However, there is a lack of many high-level studies conducted, and further investigation is required in this area. Furthermore, current research focuses on the technical skills acquisition with little research conducted on nontechnical skills acquisition within ureteroscopy. The next step for ureteroscopy training is a formalized and validated curriculum, incorporating simulation, training models, development of nontechnical skills, and real-life practice.

Face and content validation of the prostatic hyperplasia model and holmium laser surgery simulator

OBJECTIVE

Although a number of simulators have been introduced for prostate surgery, none have undergone validation for holmium laser enucleation of the prostate training. This study was carried out to assess the face and content validities as well as feasibility and acceptability of the new prostatic hyperplasia model and prostate surgery simulator for holmium laser enucleation of the prostate.

DESIGN

This is a prospective, observational, and comparative study. Participants were given a 30-minute video tutorial followed by a 45-minute simulation session, with one-to-one mentoring. A survey with qualitative and quantitative fields was used to evaluate their experience.

SETTING

This study was carried out in a 2-day modular teaching course hosted by the Holmium User Group at Cambridge University Hospitals, UK, and during the British Association of Urological Surgeons 2013 Annual Meeting.

PARTICIPANTS

A total of 36 participants comprising 13 urology trainees and 23 senior urologists of varying levels from all around the globe were recruited.

RESULTS

Overall, 87% of the participants believed that holmium laser enucleation of the prostate was an effective method of treatment, simulation-based training, and assessment essential for patient safety and 84% believed a validated simulator would be useful for training. Of the participants, 97% agreed that the simulation should be implemented into training programs and only 31% felt it should be part of accreditation. Participants ranked all components of the simulator greater than 7 of 10 on a global rating scale and believed it was a feasible and acceptable method of training and assessment.

CONCLUSIONS

The new simulator for holmium laser enucleation of the prostate has been demonstrated to be useful as a training tool. This study has established face and content validities of the simulator. Senior and trainee urologists believed the simulator was an acceptable tool for training and assessment and its use feasible for novice trainees to acquire skills and knowledge to a predetermined level of proficiency.

Training in ureteroscopy for urolithiasis

Objectives

To provide an insight into the current status of semi-rigid and flexible ureteroscopy, following new curricula for training methods, including training with models, virtual reality and active mentoring.

Methods

We systematically reviewed previous reports, including articles in English identified using the following strategy: (‘ureteroscopy’[Mesh]) or (‘urolithiasis’[Mesh]) AND (‘education’[Mesh]), or (‘teaching’[Mesh]). Abstracts submitted at congresses were not included. Relevant articles that were identified as references in the retrieved articles were also included.

Results

The terms (‘urolithiasis’[Mesh] AND ‘education’[Mesh]) retrieved 106 articles, of which five were included. The terms (‘urolithiasis’[Mesh] AND ‘teaching’[Mesh]) retrieved six articles, of which three were included. The terms (‘ureteroscopy’[Mesh] AND ‘education’[Mesh]) retrieved 29 articles, of which 21 were included. The terms (‘ureteroscopy’[Mesh] AND ‘teaching’[Mesh]) retrieved eight articles, of which seven were included. Remaining articles were found in the reference section of retrieved articles. Finally, 43 articles were included. Four randomised controlled trials with level 1b evidence were included. Currently there is no standard teaching method for ureteroscopy and the number of cases to reach competence has not yet been defined. However, simulation-based training has been shown to be effective, cost-effective, and to increase patient safety.

Conclusions

Simulators lead to a more rapid acquisition of skills in ureteroscopy than do conventional training methods, and improve the performance of future surgeons. Flexible ureteroscopy simulators are a promising tool for training, and have the advantage of minimising the need for learning the procedures on patients. A didactic and clinical curriculum, including surgical videotape reviews as well as operative mentoring, enables a rapid progression in already experienced endourologists. However, there are few reports specifically addressing the skills necessary for training.

Comparative performance of high-fidelity training models for flexible ureteroscopy: Are all models effective?

Objective:

We performed a comparative study of high-fidelity training models for flexible ureteroscopy (URS). Our objective was to determine whether high-fidelity non-virtual reality (VR) models are as effective as the VR model in teaching flexible URS skills.

Materials and Methods:

Twenty-one trained urologists without clinical experience of flexible URS underwent dry lab simulation practice. After a warm-up period of 2 h, tasks were performed on a high-fidelity non-VR (Uro-scopic Trainer™; Endo-Urologie-Modell™) and a high-fidelity VR model (URO Mentor™). The participants were divided equally into three batches with rotation on each of the three stations for 30 min. Performance of the trainees was evaluated by an expert ureteroscopist using pass rating and global rating score (GRS). The participants rated a face validity questionnaire at the end of each session.

Results:

The GRS improved statistically at evaluation performed after second rotation (P<0.001 for batches 1, 2 and 3). Pass ratings also improved significantly for all training models when the third and first rotations were compared (P<0.05). The batch that was trained on the VR-based model had more improvement on pass ratings on second rotation but could not achieve statistical significance. Most of the realistic domains were higher for a VR model as compared with the non-VR model, except the realism of the flexible endoscope.

Conclusions:

All the models used for training flexible URS were effective in increasing the GRS and pass ratings irrespective of the VR status.

Endourological simulator performance in female but not male medical students predicts written examination results in basic surgery

Objective.

The objective of this study was to examine the correlation between endourological simulator performance and demonstrated theoretical knowledge in the basic surgical sciences.

Material and methods.

In total, 158 fourth year medical students participated in the study, 83 females and 75 males, all surgical novices. All students performed the flexible endoscopic task Hall of Fame in the urological simulator URO Mentor^™. Later during the same semester all students took the final theoretical examination in surgery.

Results.

In female medical students a significant correlation was found between surgical simulator performance and the examination results (r = –0.22, p = 0.04). There was no statistically significant correlation when looking at the total study population (r = –0.04, p = 0.58) or when looking at male medical students (r = 0.01, p = 0.9).

Conclusion.

Female medical students completing an endourological simulator task more efficiently passed the theoretical examination in the basic surgical sciences with significantly higher scores than females with low efficiency in the urological simulator. There are likely to be several explanations for this correlation, such as motivation and a lower amount of current video gaming experience.

Update on resident training models for ureteroscopy

Surgical skills increasingly are taught in training laboratories outside the operating room. This change in paradigm has driven the development of surrogate training models for a wide range of surgical procedures. In urology, a growing number of procedures using minimally invasive techniques are performed. Consequently, the development of simulators for training and assessment represents a growing need. In this article, we review the currently available ureteroscopic simulators, including a critical assessment of relevant validation studies for each.

Development and validation of a virtual reality transrectal ultrasound guided prostatic biopsy simulator

OBJECTIVE

We present the design, reliability, face, content and construct validity testing of a virtual reality simulator for transrectal ultrasound (TRUS), which allows doctors-in-training to perform multiple different biopsy schemes.

METHODS

This biopsy system design uses a regular "end-firing" TRUS probe. Movements of the probe are tracked with a micro-magnetic sensor to dynamically slice through a phantom patient's 3D prostate volume to provide real-time continuous TRUS views. 3D TRUS scans during prostate biopsy clinics were recorded. Intrinsic reliability was assessed by comparing the left side of the prostate to the right side of the prostate for each biopsy. A content and face validity questionnaire was administered to 26 doctors to assess the simulator. Construct validity was assessed by comparing notes from experts and novices with regards to the time taken and the accuracy of each biopsy.

RESULTS

Imaging data from 50 patients were integrated into the simulator. The completed VR TRUS simulator uses real patient images, and is able to provide simulation for 50 cases, with a haptic interface that uses a standard TRUS probe and biopsy needle. Intrinsic reliability was successfully demonstrated by comparing results from the left and right sides of the prostate. Face and content validity respondents noted the realism of the simulator, and its appropriateness as a teaching model. The simulator was able to distinguish between experts and novices during construct validity testing.

CONCLUSIONS

A virtual reality TRUS simulator has successfully been created. It has promising face, content and construct validity results.

Validation of a high fidelity adult ureteroscopy and renoscopy simulator

PURPOSE

Surgical simulation technology may efficiently train and accurately assess the acquisition of many skills. Surgical simulators often lack realism and can be expensive at $3,000 to more than $60,000. We assessed the face, content and construct validity of a newly developed, anatomically accurate, reasonably priced high fidelity ureteroscopy and renoscopy trainer.

MATERIALS AND METHODS

A total of 46 participants, including attending urologists, urology residents, medical students and industry representatives, assessed the face and content validity of the simulator using a standard questionnaire. Ten experienced ureteroscopists with greater than 30 procedures per year and 10 novice ureteroscopists with 0 were assessed on the ability to perform flexible ureteroscopy, renoscopy and intrarenal basket extraction of a lower pole calculus using the adult ureteroscopy trainer (Ideal Anatomic Modeling, Holt, Michigan). Subject performance was assessed by an experienced ureteroscopist using a checklist, global rating scale and time to task completion.

RESULTS

Of participants 100% rated the trainer as realistic and easy to use, and thought it was a good training tool, 98% thought that it would serve as a good training format and 96% would recommend it to urology trainees. All participants recommended it for use in residency programs and 96% would or would have used it during residency. Only 37.5% vs 100% of experienced vs novice ureteroscopists would use it to practice. Of participants 9% foresaw a problem with the trainer. On the trainer experienced ureteroscopists scored significantly higher on the global rating scale (mean +/- SD 33.1 +/- 1.3 vs 15.0 +/- 2.7, p <0.0001) and checklist (4.1 +/- 1.0 vs 2.4 +/- 1.1, p = 0.004), and required less time to complete the task (141.2 +/- 40.1 vs 447.2 +/- 301.7 seconds, p = 0.01).

CONCLUSIONS

Our preliminary study suggests the face, content and construct validity of the adult ureteroscopy trainer as a high fidelity ureteroscopy and renoscopy trainer.

Validation and implementation of surgical simulators: a critical review of present, past, and future

BACKGROUND

In the past 20 years the surgical simulator market has seen substantial growth. Simulators are useful for teaching surgical skills effectively and with minimal harm and discomfort to patients. Before a simulator can be integrated into an educational program, it is recommended that its validity be determined. This study aims to provide a critical review of the literature and the main experiences and efforts relating to the validation of simulators during the last two decades.

METHODS

Subjective and objective validity studies between 1980 and 2008 were identified by searches in Pubmed, Cochrane, and Web of Science.

RESULTS

Although several papers have described definitions of various subjective types of validity, the literature does not offer any general guidelines concerning methods, settings, and data interpretation. Objective validation studies on endourological simulators were mainly characterized by a large variety of methods and parameters used to assess validity and in the definition and identification of expert and novice levels of performance.

CONCLUSION

Validity research is hampered by a paucity of widely accepted definitions and measurement methods of validity. It would be helpful to those considering the use of simulators in training programs if there were consensus on guidelines for validating surgical simulators and the development of training programs. Before undertaking a study to validate a simulator, researchers would be well advised to conduct a training needs analysis (TNA) to evaluate the existing need for training and to determine program requirements in a training program design (TPD), methods that are also used by designers of military simulation programs. Development and validation of training models should be based on a multidisciplinary approach involving specialists (teachers), residents (learners), educationalists (teaching the teachers), and industrial designers (providers of teaching facilities). In addition to technical skills, attention should be paid to contextual, interpersonal, and task-related factors.