Competency-Based Training and Simulation: Making a “Valid” Argument

Three Dimensional Printing Technology Used to Create a High-Fidelity Ureteroscopy Simulator: Development and Validity Assessment (Rein-3D-Print-UroCCR-39)

OBJECTIVE

To create and assess the validity of a high-fidelity, three dimensional (3D) printed, flexible ureteroscopy simulator resulting from a real case.

METHODS

A patient's CT scan was segmented to obtain a 3D model in .stl format, including the urinary bladder, ureter and renal cavities. The file was printed and a kidney stone was introduced into the cavities. The simulated surgery consisted of monobloc stone extraction. Nineteen participants split into 3 groups according to their level (6 medical students, 7 residents and 6 urology fellows) performed the procedure twice at a 1-month interval. They were rated according to a global score and a task-specific score, based on an anonymized, timed video recording.

RESULTS

Participants demonstrated a significant improvement between the 2 assessments, both on the global score (29.4 vs 21.9 points out of 35; P < .001) and the task-specific score (17.7 vs 14.7 points out of 20; P < .001) as well as procedure time (498.5 vs 700 seconds; P = .001). Medical students showed the greatest progress for the global score (+15.5 points (mean), P = .001) and the task-specific score (+6.5 points (mean), P < .001). 69.2% of participants considered the model as visually quite realistic or highly realistic and all of them judged it quite or extremely interesting for intern training purposes.

CONCLUSION

Our 3D printed ureteroscopy simulator was able to enhance the progress of medical students who are new to endoscopy, whilst being valid and reasonably priced. It could become part of a training program in urology, in line with the latest recommendations for surgical education.

Learning Surgical Skills Through Video-Based Education: A Systematic Review

BACKGROUND

Educational videos are a potent resource for the learning of surgical skills among different study cohorts. However, there is limited evidence on the effectiveness of different educational video interventions and their features.

A systematic search of MEDLINE (via PubMed), Embase (via OVID), Cochrane libraries and Clinicaltrials.gov was performed from inception to 28/02/21. Studies included were not limited by date of publication, studies aiming to assess the impact of video-based interventions in the direct acquisition of surgical skill were included. Eligible studies were analysed based on study type, type of video intervention, method of assessment and period of education. The educational impact of the studies was also assessed as per Messick's framework for testing validity of evaluation methods and McGhagie's model for analysing translational outcomes.

22 studies were deemed suitable for inclusion, of which 14/22 (63.6%) demonstrated a significant improvement in knowledge/skills following the video-based teaching interventions, 3/22 (13.6%) studies demonstrated an improvement in trainee satisfaction scores. A recurrent limitation of the included studies was the lack of validation of selected assessment methods. None of the included studies scored on all 5 parameters of validity as defined by Messicks validity framework. Furthermore, none of the included trials were conducted for long enough to indicate direct changes to patient outcomes resultant from educational methods.

CONCLUSION

Video-based surgical education is effective in learning surgical skills within different levels of surgical training; however, superior study quality and follow-up is required to determine which aspects of video-based interventions are most impactful.

Integrating Medical Simulation into Residency Programs in Kingdom of Saudi Arabia

PURPOSE

This study conducted a simulation needs assessment on diverse ongoing residency training programs supervised by the Saudi Commission for Health Specialties (SCFHS) in Saudi Arabia. The goal was to develop a standardized methodological approach to integrate simulation as a teaching tool for any ongoing training program.

METHODS

A mixed-methods approach is used in four steps to focus on top educational needs and integrate simulation into the curriculum. The first step was the selection of 38 residency training programs based on the scoring criteria tool. Of these, nine were selected as target programs. The next step was champion recruitment, where two faculty representatives from each specialty were trained to become specialty champions. The third step was a targeted audience needs assessment, consisting of four phases: curriculum review; a targeted audience survey; stakeholders' interview; and selection of top educational requirements generated by the first three phases. Lastly, the fourth step used an integration simulation sheet to build common themes for incorporating simulation into the program curriculum.

RESULTS

Out of 38 programs, the nine selected top-ranked specialties completed the process, and roadmaps were developed. Using the combined list of all skills and behaviors, the final score proportion was calculated and then ranked. A list of the top needed skills and behaviors was compiled as follows: Obstetrics and Gynecology 10/84, Emergency Medicine (ER) 80/242, Intensive Care Unit 20/139, Internal Medicine (IM) 37/102, Pediatric 82/135, Ear, Nose, and Throat (ENT) 49/125, General Surgery (GS) 55/114, Plastic Surgery 24/165, and Family Medicine (FM) 59/168.

CONCLUSION

Findings from this process could be used by the supervisory bodies at a country level and assist decision-makers to determine which criteria to use in the needs assessment to integrate simulation into any ongoing residency training program.

Perspectives on simulation-based assessment of operative skill in surgical training

INTRODUCTION

The perspectives of the wider surgical community toward simulation-based assessment (SBA) in training is a gap in the literature. This study aims to explore the factors associated with the acceptable use of SBA in surgical training, through the perceptions and experiences of a broad range of stakeholder representatives, building on findings from a review of the published literature.

MATERIALS AND METHODS

Ten semi-structured interviews were conducted, using a sequential transformative qualitative methods approach, with representatives from identified key stakeholder groups; executive management, risk management, a practicing surgeon, an anaesthesiologist, a theatre-nursing representative, a representative from simulation industry, a patient, a medical student, a junior surgical trainee, and a senior surgical trainee. Interview transcripts underwent reflexive thematic analysis using an inductive and constructivist framework (NVIVO software, NVIVO 12, QSR International).

RESULTS

Four themes emerged: the 'need' for SBA, the concept of a 'minimum standard', the 'optimum design' of an SBA framework, and 'fairness'. SBA is a potential solution to challenges in the current training environment. It emerged that it should not replace trainer judgement, but could ensure that trainees meet a minimum operative competency standard. SBA should be used to identify underperforming trainees early in training to provide targeted remediation. The application of SBA in high-stakes settings such as trainee selection, autonomy granting, and end-of training certification has perceived benefits over current assessment methods.

CONCLUSIONS

This study builds on findings from prior research to explore factors regarding the acceptable use of simulation as an assessment method in surgical training, including perspectives from a broad range of stakeholder representatives. Findings can inform the development of simulation-based assessment curricula in surgical training.

A Systematic Review of Simulation-Based Training in Vascular Surgery

Introduction

Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods

PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results

Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content (35), response processes (12), the internal structure (5), relations to other variables (57), and consequences (2). Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions

Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.

Resident Endoscopy Experience Correlates Poorly with Performance on a Virtual Reality Simulator

Background  Endoscopy training has become increasingly emphasized during general surgery residency as reflected by introduction of the Fundamentals of Endoscopic Surgery (FES) examination, which includes testing of skills on virtual reality (VR) simulators. Although studies exist to assess the ability of the simulator to differentiate between novices and experienced endoscopists, it is not well understood how simulators can differentiate skills among resident cohort.

Objective  To assess the utility of the VR simulator, we evaluated the correlation between resident endoscopy experience and performance on two VR simulator colonoscopy modules on the GI-BRONCH Mentor (Simbionix Ltd, Airport City, Israel).

Methods  Postgraduate years 2 to 5 residents completed “easy” and “difficult” VR colonoscopies, and performance metrics were recorded from October 2017 to February 2018 at Rutgers' two general surgery residency programs. Resident endoscopy experience was obtained through Accreditation Council for Graduate Medical Education case logs. Correlations between resident endoscopy experience and VR colonoscopy performance metrics were assessed using Spearman's rho (ρ) correlation statistic and bivariate logistic regression.

Results  Fifty-five residents out of 65 (84.6%) eligible participants completed the study. There were limited correlations found between resident endoscopy experience and FES performance metrics and no correlations were found between resident endoscopy experience and binary metrics of colonoscopy—ability to complete colonoscopy, ability to retroflex, and withdrawal time of less than 6 minutes.

Conclusion  The VR simulator may have a limited ability to discriminate between experience levels among resident cohort. Future studies are needed to further understand how well the VR simulator metrics correlate with resident endoscopy experience.

Validation of a novel assessment tool identifying proficiency in Transurethral Bladder Tumour Resection: The OSATURBS assessment tool

BACKGROUND

Competence in transurethral bladder tumour resection (TURB) is critical in bladder cancer management and should be ensured before independent practice.

OBJECTIVE

Develop an assessment tool for TURB and explore validity evidence in a clinical context.

DESIGN, SETTING, AND PARTICIPANTS

July 2019-March 2021, a total of 33 volunteer doctors from three hospitals were included. Participants performed two TURB procedures on patients with bladder tumours. A newly developed assessment tool (OSATURBS) was used for direct observation assessment, self-assessment, and blinded video-assessment. Outcome measurements and statistical analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t-test and independent samples t-test.

RESULTS AND LIMITATIONS

The internal consistency reliability across items Cronbach's alpha was 0.94 (n = 260, p < 0.001). Inter-rater reliability = 0.80 (n = 64, p < 0.001). Test-retest correlation was high, r = 0.71 (n = 32, p < 0.001). Relation to TURB experience was high, r = 0.71 (n = 32, p < 0.001). Pass/fail score = 19 points. Direct observation assessments were strongly correlated with video ratings (r = 0.85, p < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances.

CONCLUSIONS

OSATURBS assessment tool for TURB can be used for assessment of surgical proficiency in the clinical setting. Direct observation assessment and self-assessment are biased, and blinded video-assessment of TURB performances is advised.

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal - a validation study

OBJECTIVE

The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning.

DESIGN

This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process.

RESULTS

Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach's alpha = 0.14, p = 0.15) and across the three modules (Cronbach's alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences.

CONCLUSION

Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

Development and preliminary validation of the competency model for rural general practitioner in China

BACKGROUND

There is a shortage of rural general practitioners (GPs) in China. Training programs have been initiated to meet the needs of GP workforce in the rural areas; however, there is an absence of validated tool to assess their competencies.

OBJECTIVE

This study aimed to develop a competency model for rural general practitioner (CMRGP) after training in China and to examine its validity and reliability.

METHODS

A multistage process was adopted to develop the CMRGP comprised literature review, panel discussion and expert consultation, and the initial version of CMRGP was reduced from 10 domains and 77 items to 7 domains and 54 items. A pilot study was conducted among 202 rural GPs for the psychometric evaluation and application of the initial version of CMRGP, in which a questionnaire on the importance of items and self-evaluation was completed by the GPs.

RESULTS

In the pilot study, 132 completed questionnaires (65.3%) were returned. Acceptability and face validity of the CMRGP were supported by high importance scores of the items, in which 52 out of 54 items achieved score higher than 4.00 (possible score from 0 to 5). Factor analysis supported the construct validity. After the modification, the final version of CMRGP contained 6 domains and 47 items. Good reliability was supported by internal consistency reliability (Cronbach's α was 0.98) and split-half reliability (Spearman-Brown coefficient was 0.99).

CONCLUSIONS

The CMRGP demonstrated good reliability and validity. Pilot study showed its potential for application in the rural general practice and training program.

Simulation-Based Training Models for Urolithiasis: A Systematic Review

Objectives: Urolithiasis is one of the most common presentations in urological practice and it is becoming increasingly important to provide structured, simulation-based training using validated training models. This systematic review aims to identify current simulation-based training models and to evaluate their validity and effectiveness. Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a comprehensive search was performed on the Medline and EMBASE databases for English language articles published between 2000 and 2018 describing and/or assessing validity of simulation models for urolithiasis procedures. Studies were qualitatively assessed for validity using the Messick validity framework and models were assigned levels of recommendation using the McGaghie model of translational outcomes. Results: A total of 98 studies were included in this study assessing 51 models, with 28 studies concerning models for urethrocystoscopy, 46 studies for ureterorenoscopy, and 39 studies for percutaneous access and/or nephrolithotomy. Only four models demonstrated a level of recommendation of 4. The most validated models were the URO/PERC-Mentor (Simbionix, Lod, Israel) with multiple studies for each across various procedural skills. Conclusion: There is a wide spectrum of simulation-based models currently available for urolithiasis procedures, mostly with limited validity evidence from small studies. Further research is required with higher levels of evidence including randomized controlled trials. In addition, long-term transfer of skills to the operating room should be assessed to establish whether there is genuine skill development and retention using simulation models and whether this helps to reduce surgical complications.

Competency Based Medical Education-Towards the Development of a Standardized Pediatric Radiology Testing Module

RATIONALE AND OBJECTIVES

To introduce a process that allows for development of standardized competency based testing modules (CBTM) for evaluating resident progress and competence during their radiology training. This work focuses on the development of pediatric imaging CBTMs to be utilized during general radiology residency.

MATERIALS AND METHODS

Multiple in-patient and ER imaging request audits along with surveys of training programs and text recommendations were obtained. A list of 200 total diagnoses accumulated by review was distributed into one of four CBTM folders. Imaging cases which made ≥90% of the indications of the audits were added to Folder 1. Distribution of remaining imaging diagnoses was based on consensus by three subspecialists. A pilot study was performed with residents dictating selected imaging cases in their usual manner mimicking a typical rotation.

RESULTS

The pilot study demonstrated resident grading mean scores significantly associated with both the American College of Radiology (ACR) rank (rho = 0.636, p = 0.035) and the objective structured clinical examinations (OSCE) scores (rho = 0.694, p = 0.018). The mean scores positively associated with the ACR score (rho = 0.466), but fell short of statistical significance (p = 0.149). As expected, the ACR score, ACR rank and OSCE scores all significantly correlated with each other ( < 0.01). PGY also significantly correlated with the ACR score (rho = 0.683, p = 0.021) and the OSCE (rho = 0.767, p = 0.006) but not with the ACR rank (rho = 0.408, p = 0.213).

CONCLUSION

The process utilized to develop a standardized CBTM can be used as a simulation tool to assess radiology resident competence during their training. The format allows for assessment of resident reasoning skills and knowledge base, which provides documentation of progression and throughout residency.

A case for feedback and monitoring assessment in competency-based medical education

PURPOSE

Within competency-based medical education, self-regulated learning (SRL) requires residents to leverage self-assessment and faculty feedback. We sought to investigate the potential for competency-based assessments to foster SRL by quantifying the relationship between faculty feedback and entrustment ratings as well as the congruence between faculty assessment and resident self-assessment.

MATERIALS AND METHODS

We collected comments in (a) an emergency medicine objective structured clinical examination group (objective structured clinical examinations [OSCE] and emergency medicine OSCE group [EMOG]) and (b) a first-year resident multidisciplinary resuscitation "Nightmares" course assessment group (NCAG) and OSCE group (NOG). We assessed comments across five domains including Initial Assessment (IA), Diagnostic Action (DA), Therapeutic Action (TA), Communication (COM), and entrustment. Analyses included structured qualitative coding and (non)parametric and descriptive analyses.

RESULTS

In the EMOG, faculty's positive comments in the entrustment domain corresponded to lower entrustment score Mean Ranks (MRs) for IA (<11.1), DA (<11.2), and entrustment (<11.6). In NOG, faculty's negative comments resulted in lower entrustment score MRs for TA (<11.8 and <10) and DA (<12.4), and positive comments resulted in higher entrustment score MRs for IA (>15.4) and COM (>17.6). In the NCAG, faculty's positive IA comments were negatively correlated with entrustment scores (ρ = -.27, P = .04). Across programs, faculty and residents made similar domain-specific comments 13% of the time.

CONCLUSIONS

Minimal and inconsistent associations were found between narrative and numerical feedback. Performance monitoring accuracy and feedback should be included in assessment validation.

Validation of Training and Acquisition of Surgical Skills in Veterinary Laparoscopic Surgery: A Review

At present, veterinary laparoscopic surgery training is lacking in experiences that provide a controlled and safe environment where surgeons can practice specific techniques while receiving experts' feedback. Surgical skills acquired using simulators must be certified and transferable to the operating room. Most models for practicing laparoscopic skills in veterinary minimally invasive surgery are general task trainers and consist of boxes (simulators) designed for training human surgery. These simulators exhibit several limitations, including anatomic species and procedural differences, as well as general psychomotor training rather than in vivo skill recreation. In this paper, we review the existing methods of training, evaluation, and validation of technical skills in veterinary laparoscopic surgery. Content includes global and specific scales, and the conditions a structured curriculum should meet for improving the performance of novice surgeons during and after training. A focus on trainee-specific assessment and tailored-technical instruction should influence training programs. We provide a comprehensive analysis of current theories and concepts related to the evaluation and validation of simulators for training laparoscopic surgery in small animal surgery. We also highlight the need to develop new training models and complementary evaluation scales for the validation of training and acquisition of basic and advanced skills in veterinary laparoscopic surgery.

Design and validation of a low-cost, high-fidelity model for robotic pyeloplasty simulation training

INTRODUCTION/BACKGROUND

Owing to restrictions in operative experiences, urology residents can no longer solely rely on 'hands-on' operative time to master their surgical skills by the end of residency. Simulation training could help residents master basic surgical skills and steps of a procedure to maximize time in the operative room. However, simulators can be expensive or tedious to set up, limiting the availability to residents and training programs.

OBJECTIVE

The authors sought to develop and validate an inexpensive, high-fidelity training model for robotic pyeloplasty.

STUDY DESIGN

Pyeloplasty models were created using Dragon Skin® FX-Pro tissue-mimicking silicone cast over 3-dimensional molds. Urology faculty and trainees completed a demographic questionnaire. The participants viewed a brief instructional video and then independently performed robotic dismembered pyeloplasty on the model. Acceptability and content validity were evaluated via post-task evaluation of the model. Construct validity was evaluated by comparing procedure completion time, the Global Evaluative Assessment of Robotic Skills (GEARS) score, blinded subjective physical evaluation of repair quality (1-10 scale), and flow rate between experts and novices.

RESULTS

In total, 5 urology faculty, 6 fellows, and 14 residents participated. The median robotic console experience among faculty, fellows, and residents was 8 years (interquartile range [IQR] = 6-11), 3.5 years (IQR = 2-4 years), and 0 years (IQR = 0-0.5 years), respectively. The median procedure completion time was 29 min (IQR = 26-40 min), and the median flow rate was 1.11 mL/s (IQR = 0-1.34 mL/s). All faculty had flow rates >1.25 mL/s and procedure times <30 min compared with 2 of 6 fellows and none of the residents (P < 0.001). All faculty, half of the fellows, and none of the residents achieved a GEARS score ≥20, with a median resident score of 12.5 (IQR = 8-13) (P < 0.001). For repair quality, all faculty scored ≥9 (out of 10), all fellows scored ≥8, and the median score among residents was 6 (IQR = 2-6) (P < 0.001). The material cost was $1.32/model, and the average production time was 0.12 person-hours/model.

DISCUSSION AND CONCLUSION

This low-cost pyeloplasty model exhibits acceptability and content validity. Construct validity is supported by significant correlation between participant expertise and simulator performance across multiple assessment domains. The model has excellent potential to be used as a training tool in urology and allows for repetitive practice of pyeloplasty skills before live cases.

National Board of Medical Examiners and Curriculum Change: What Do Scores Tell Us? A Case Study at the University of Balamand Medical School

OBJECTIVES

This study describes the results of NBME (National Board of Medical Examiners) implementation in Balamand Medical School (BMS) from 2015 to 2019, after major curricular changes were introduced as of 2012. BMS students' performance was compared with the international USMLE step 1 (United States Medical Licensing Examination, herein referred to as step 1) cohorts' performances. The BMS students' NBME results were analyzed over the successive academic years to assess the impact of the serial curricular changes that were implemented.

METHODS

This longitudinal study describes the performance of BMS preclinical second year medicine (Med II) students on all their NBME exams over 4 academic years starting 2015-2016 to 2018-2019. These scores were compared with the step 1 comparison group scores using item difficulty. The t test was computed for each of the NBME exams to check whether the scores' differences were significant.

RESULTS

Results revealed that all BMS cohorts scored lower than the international USMLE step 1 comparison cohorts in all disciplines across the 4 academic years except Psychiatry. However, the results were progressively approaching step 1 results, and the difference between step 1 scores and BMS students' NBME scores became closer and not significant as of year 4.

CONCLUSIONS

The results of the study are promising. They show that the serial curricular changes enabled BMS Med II students' scores to reach the international cohorts' scores after 4 academic years. Moreover, the absence of statistical difference between cohort 4 scores and step 1 cohorts is not module dependent and applies to all clinical modules. Further studies should be conducted to assess whether the results obtained for cohort 4 can be maintained.

Benefits and Limitations of Transurethral Resection of the Prostate Training With a Novel Virtual Reality Simulator

PURPOSE

Profound endourological skills are required for optimal postoperative outcome parameters after transurethral resection of the prostate (TURP). We investigated the Karl Storz (Tuttlingen, Germany) UroTrainer for virtual simulation training of the TURP.

MATERIALS AND METHODS

Twenty urologists underwent a virtual reality (VR) TURP training. After a needs analysis, performance scores and self-rated surgical skills were compared before and after the curriculum, the realism of the simulator was assessed, and the optimal level of experience for VR training was evaluated. Statistical testing was done with SPSS 25.

RESULTS

Forty percent of participants indicated frequent intraoperative overload during real-life TURP and 80% indicated that VR training might be beneficial for endourological skills development, underlining the need to advance classical endourological training. For the complete cohort, overall VR performance scores (P = 0.022) and completeness of resection (P < 0.001) significantly improved. Self-rated parameters including identification of anatomical structures (P = 0.046), sparing the sphincter (P = 0.002), and handling of the resectoscope (P = 0.033) became significantly better during the VR curriculum. Participants indicated progress regarding handling of the resectoscope (70%), bleeding control (55%), and finding the correct resection depth (50%). Although overall realism and handling of the resectoscope was good, virtual bleeding control and correct tissue feedback should be improved in future VR simulators. Seventy percent of participants indicated 10 to 50 virtual TURP cases to be optimal and 80% junior residents to be the key target group for VR TURP training.

CONCLUSIONS

There is a need to improve training the TURP and VR simulators might be a valuable supplement, especially for urologists beginning with the endourological desobstruction of the prostate.

Evaluation of the Endo-Uro trainer for semi-rigid ureteroscopy training

Background:

The aim of this study was to evaluate the validity of evidence of the Endo-Uro Trainer (SAMED, Dresden, Germany) for semi-rigid ureteroscopy.

Methods:

Novice (n = 29), intermediate-level (n = 25), and expert (n = 24) urological surgeons were recruited to participate in the study. Novices were allocated randomly to Groups A and B, where A performed two set procedures using the already validated Uro-Scopic Trainer (Limbs and Things, Bristol, UK), and Group B used the Endo-Uro trainer. Subsequently they were crossed over to perform the same two procedures using the other model. Intermediate and expert groups performed the same procedure on the Endo-Uro trainer only. Objective Structured Assessment of Technical Skills (OSATS) and the procedural times were collected and analyzed. All participants were invited to complete a final evaluation survey.

Results:

The evaluation survey revealed a realism rating in all aspects, with a mean Likert rating of 4.04/5. Significant differences were observed in performance time between novices and experts (p = 0.0014), and between intermediates and experts (p = 0.0113). OSATS scores differed significantly between all groups (p < 0.0001). Group B novices showed statistically significant improvement in performance time (p = 0.0012) and OSATS scores (p = 0.0439) after the crossover. Significant differences in performance time (p = 0.0025) between groups A and B were also observed post-crossover.

Conclusions:

This study demonstrated content validity for the Endo-Uro Trainer model. In addition, the model was shown to be capable of differentiating levels of experience, which contributes to the acceptance of the validity hypothesis. Improvement in performance using the model demonstrates its effectiveness for training.

[Simulation-based training in urology: A systematic literature review]

CONTEXT

Simulation-based training is taking an increasingly important place in surgical training and is becoming mandatory with the latest reform of the French medical studies.

OBJECTIVES

The objectives of this work were to report the various simulation tools available for the surgical training in urology, along with their validation level, through a systematic literature review.

DOCUMENTARY SOURCES

A search was conducted using Medline^® with the terms "urology" and "simulator". Articles in English and French were selected.

STUDY SELECTION

Two hundred and ninety-one abstracts were read, allowing for the selection of 154 articles read to assess their eligibility. Studies whose main objective was not the validation of a simulator, studies describing animal models or studies whose full text was not available were excluded.

RESULTS

One hundred and six studies were analyzed in this review. The simulators described were classified in 7 categories: laparoscopic surgery, robotic surgery, ureteroscopy, percutaneous nephrolithotomy, endoscopic bladder and prostate surgery, basic skills in urology, and ultrasound-guided prostate interventions simulators. Apparent and content validity were demonstrated for most simulators, but construct and predictive validity were often lacking.

LIMITATIONS

We did not consider the use of simulators as competency evaluation tools. Besides, the latest terminology proposed to define the various validation steps was not taken into account.

CONCLUSION

Many simulators are available in the field of urology and allow the reproduction of a large variety of urological procedures. However, their validation level is inconsistent, and has to be taken into account when choosing a simulator for surgical training, along with its cost, the eagerness of students to use the simulator and its availability.

Validation of a High-End Virtual Reality Simulator for Training Transurethral Resection of Bladder Tumors

OBJECTIVE

The oncological outcome in patients with bladder cancer (BC) significantly correlates with the quality of transurethral resection of bladder tumors (TUR-BT). Virtual reality (VR) training simulators have been developed to improve surgical skills. We evaluated the advantages and limitations of the novel Uro Trainer (UT) (Karl Storz GmbH, Germany) with respect to training for TUR-BT.

DESIGN

Participants underwent VR training based on 4 different TUR-BT cases accompanied by self-assessment and evaluation questionnaires. Results were compared between experienced endourologists and novices, and furthermore, correlated with self-rated capabilities for content and construct validity. Student's t tests, Pearson's correlation, and chi-squared tests were performed for statistical evaluation.

SETTING

The study was performed at the tertiary care urological department of the Ludwig-Maximilians-University, Munich, Germany.

PARTICIPANTS

A total of 22 urological physicians, including residents and consultants, were included in the study.

RESULTS

There is a need to improve TUR-BT training as 27.3% of the participants had already experienced overtaxing situations during TUR-BT and only a few reported of high satisfaction with the classical "see one-do one" teaching mode. Construct validity was demonstrated, as consultants achieved significantly higher overall scores (p < 0.001) and safety (p = 0.004) and visualization (p = 0.001) subscores. Interestingly, the self-assessed capability to perform a TUR-BT correlated significantly (p = 0.01) with overall UT scores. Significant progress of self-rated abilities was shown for several parameters, including inspection (p = 0.046) and resection (p = 0.026). Although participants indicated improvements in several procedural skills and overall benefit of the VR training with the UT was rated 4.6 on a 5-point scale by consultants, limitations of the UT were demonstrated predominantly for tissue feedback and authenticity of different tissue layers.

CONCLUSIONS

The novel VR simulator showed a high face and construct validity, and therefore has a great potential to complement endourological training.

Introduction of an ex-vivo pig model for teaching percutaneous nephrolithotomy access techniques

INTRODUCTION

In North America, obtaining access for percutaneous nephrolithotomy (PCNL) is not often performed by urologists. Hands-on training sessions help to ensure this skill continues within the urological community. An ex-vivo pig kidney model was developed for simulation. This model uses porcine tissues with a fluoroscopic C-arm and standard PCNL equipment. The bullseye or triangulation techniques are both possible. We propose this as a high-fidelity tool for teaching PCNL access.

METHODS

The pig kidney, fat, ribs, flank, and skin were arranged anatomically on a table with fluoroscopy. Hands-on training was provided to residents and urologists using the ex-vivo pig model and a silicone-based percutaneous access model. Questionnaires were given at the end of the session.

RESULTS

There was a total 14 responders for each model, with incomplete responses on two surveys. A total of 15% of responders for the pig model and 7% of responders for the silicone model had previous percutaneous access experience. For the pig model, 93% of trainees agreed or strongly agreed that the model was easy to use, and 79% of the silicone model trainees felt the same. After the session, 50% of silicone model trainees and 86% of pig model trainees reported increased confidence in their ability to obtain PCNL access. All the pig model trainees and 71% of the silicone model trainees felt that the simulation activity was worthwhile.

CONCLUSIONS

The inexpensive but anatomically realistic ex-vivo pig model using real-world equipment provides trainees with an excellent tool to learn PCNL access.

Simulation-based training in urology residency programmes in the USA: Results of a nationwide survey

Objective

To evaluate the current usage of simulation in urological education in the USA and the barriers to incorporating a simulation-based educational curriculum, as the shift towards competency-based medical education has necessitated the introduction of simulation for training and assessing both non-technical and technical skills.

Materials and methods

Residency programme directors at Accreditation Council for Graduate Medical Education (ACGME)-accredited urology training programmes in the USA were invited to respond to an anonymous electronic survey. The study evaluated the programme directors’ experiences and opinions for the current usage of existing urology simulators. The survey also elicited receptiveness and the barriers for incorporating simulation-based training curricula within urology training programmes.

Results

In all, 43 completed surveys were received (35% response rate). Amongst responders, 97% (42/43) reported having access to a simulation education centre, and 60% (25/42) have incorporated simulation into their curriculum. A total of 87% (37/43) agreed that there is a role for a standardised simulator training curriculum, and 75% (30/40) agreed that simulators would improve operating room performance. A total of 64% (27/42) agreed that cost was a limiting factor, 12% (5/42) agreed on the cost-effectiveness of simulators, 35% (17/41) agreed there was an increased need for simulator education within work-hour limitations, and 38% (16/42) agreed a simulation programme would reduce patient risks and complications.

Conclusions

The majority of urology programme directors consider that there is a role for incorporating a simulation-based curriculum into urology training. Barriers to implementation include cost burden, need for constant technology updates, need for advanced planning, and willingness of faculty to participate in administration.