Determinants of performance on the Transfer Task of the Basic Laparoscopic Urologic Surgery (BLUS(©)) curriculum administered at objective structured clinical examinations

Global Assessment of Urological Endoscopic Skills (GAUES): development and validation of a novel assessment tool to evaluate endourological skills

OBJECTIVE

To develop and evaluate an assessment tool for endourological skills during simulation including cystoscopy, ureteroscopy (URS) and transurethral resection (TUR) procedures.

METHODS

We designed a Global Assessment of Urological Endoscopic Skills (GAUES) tool, comprised of nine endourology task-specific and two global-rating skills items. The tool was developed through two rounds of the Delphi process. The GAUES tool was used to assess acquisition of URS and TUR skills of novices (Year 2 core surgical trainees, CT2) and intermediate level trainees (residents at the start of the UK higher surgical training programme in Urology, Speciality Trainee Year 3, ST3) at the Urology Simulation Boot Camp (USBC) between 2016 and 2018. Validity was evaluated by comparing scores between trainees with different levels of urological experience. Inter-rater reliability was also assessed.

RESULTS

We evaluated 130 residents, 52% of trainees were at an intermediate stage of training and 39% were novices. In all, 9% of the anonymous forms were missing demographics. The completion rate of the GAUES tool during the USBC for URS and TUR was 85% and 89%, respectively. Our analysis demonstrated a significant difference in all domains between intermediates and novices at assessment in URS, except for one domain more suited to clinical assessment (P = 0.226). There was excellent intraclass correlation (ICC) overall between the two experts' judgements, ICC = 0.841 (95% confidence interval 0.767-0.893; P < 0.001, n = 88).

CONCLUSIONS

We have developed the novel GAUES tool for cystoscopic, URS and TUR skills. Overall, we demonstrated good face, content and construct validity and excellent reliability, suggesting that the GAUES tool can be useful for endourological skills assessment.

Adaptation of Medical Students During Clinical Training: Effects of Holistic Preclinical Education on Clerkship Performance

THEORY

Taiwan's medical undergraduate program at a university or medical center is a continuation of 12 years of compulsory citizenship education rooted in holistic philosophies. Students acquire both technical knowledge and nontechnical attributes, which are necessary for success in further work and life. The early clinical learning experiences of medical students are primarily acquired through clerkships. These clerkships require medical students to apply and extend what they learned during their preclinical education; however, previous studies have explored this issue through examining fragmentary factors such as preclinical course grades and traits but not undertaking comprehensive, whole-person investigations.

HYPOTHESES

To account for the potential benefits of a holistic approach in medical students' learning, we propose three hypotheses: Medical students' preclinical performance on Taiwan's technical and nontechnical higher education assessments are positively associated with their clinical competence (Hypothesis 1) and psychological well-being (Hypothesis 2) during clerkships, and medical students' psychological well-being during clerkships is positively associated with their clinical competence (Hypothesis 3).

METHOD

We studied a cohort of 65 medical students engaged in clerkships from September 2013 to April 2015. Their preclinical technical knowledge scores-formal curricular grades received from course instructors-were obtained from their medical school's archival dataset. Their nontechnical attributes-moral and social performance scores received from student mentors and physical performance scores received from course instructors-were also obtained from the school's archival data set. The medical students' competence in their 2-year clinical clerkships was measured using the objective structured clinical examination scores from the end of both clerkship years. The medical students' psychological well-being during their 2-year clerkships was measured according to burnout level, which was determined using routine online surveys that employed validated, structured, and self-administered questionnaires at each specialty rotation. Multiple regressions and linear mixed-effects model were employed for statistical analysis.

RESULTS

Our study revealed that higher preclinical technical knowledge predicted superior clinical competence and a higher level of burnout during clerkships. By contrast, higher preclinical nontechnical attributes (i.e., higher preclinical moral, social, and physical performance) predicted lower level of burnout. However, no relationship was discovered between clerkship burnout and the clinical competence of the medical students.

CONCLUSIONS

Our study verified the value of a holistic education that encompasses both technical knowledge and nontechnical attributes during the preclinical learning stage for medical students. Our findings can serve as a reference for medical educators designing preclinical educational programs for medical students.

Assessment of urology postgraduate trainees' competencies in flexible ureteroscopic stone extraction

INTRODUCTION

We aimed to assess flexible ureteroscopic stone extraction skill of urology postgraduate trainees (PGTs) at an Objective Structured Clinical Examination (OSCE) and to determine whether previous experience in the operating theatre or practice on the simulator correlated with performance.

METHODS

After obtaining ethics approval, PGTs from postgraduate years (PGYs) 3-5 were recruited from all four Quebec urology training programs during an OSCE. After a short orientation to the UroMentor™ simulator, PGTs were asked to perform Task 10 for 15 minutes, where two small stones from the left proximal ureter and renal pelvis were extracted using a basket. Competency of PGTs in performing the task was assessed using objective assessment from the simulator and subjective evaluations using Ureteroscopy-Global Rating Scale (URS-GRS). Simulator performance reports and URS-GRS scores were analyzed.

RESULTS

Thirty PGTs (9 PGY-3, 11 PGY-4, 10 PGY-5) participated in this study. PGTs had performed a mean of 55.9 semi-rigid and 45.7 flexible ureteroscopies prior to the study. Mean URS-GRS score of the participants was 20.0±4.4. Using norm-referenced method with three experts, cutoff score of 19 on the URS-GRS was determined to indicate competency. Sixty percent (18/30) of PGTs were competent. All eight PGTs who had practiced on the simulator were competent. Previous experience in the operating theatre and PGY level did not correlate with performance.

CONCLUSIONS

This study confirmed the feasibility of incorporating the UroMentor at OSCEs to assess competency of urology PGTs in ureteroscopic stone extraction skill. PGTs who practiced on the simulator scored significantly higher than those who did not practice; however, the software needs to be updated to improve its face validity and to include more complex tasks, such as holmium laser lithotripsy. Future studies with larger sample sizes and more complex cases are needed to confirm these results.

Determinants and effects of medical students' core self-evaluation tendencies on clinical competence and workplace well-being in clerkship

Core self-evaluation (CSE) is a personality trait that involves a person’s evaluation of his or her own worth, competence, and capability. The objective of this study was to determine whether medical students’ CSEs exert beneficial effects on their adaptation to their clerkship in terms of their clinical competence and workplace well-being and whether their preclinical academic performance can be a trait-relevant situation that enhances their CSE expression. In total, 127 medical students from 2 cohorts were included as participants in this study. We analyzed complete measures of personal background, objective and subjective preclinical academic performance (course evaluation grades and self-reported efficacy), CSE tendencies, and clinical competence (as objective structured clinical examination scores) and workplace well-being (as compassion satisfaction and burnout) during their 2-year clerkship. Mixed linear models for repeated measures and multiple regressions were employed. Participants’ CSE tendencies had positive effects on their workplace compassion satisfaction and burnout but not on their clinical competence during their clerkship. Additionally, using the objective and subjective preclinical academic performance of the medical students as indicators, we observed that neither could be trait-relevant situations to enhance their CSE expression. CSE personality tendencies might be key to medical students’ ability to noncognitively adapt to clinical training during their clerkships. These tendencies should be identified earlier so that mentors can provide prompt care and support to mentees (medical students) during clerkships.

Assessment of percutaneous renal access skills during Urology Objective Structured Clinical Examinations (OSCE)

INTRODUCTION

The first objective was to assess percutaneous renal access (PCA) skills of urology postgraduate trainees (PGTs) during the Objective Structured Clinical Examinations (OSCEs). The second objective was to determine whether previous experience with percutaneous nephrolithotomy (PCNL) improved performance.

METHODS

After obtaining ethics approval, we recruited PGTs from two urology programs in Quebec between postgraduate years (PGY-3 to PGY-5). Each trainee was asked to answer a short questionnaire regarding previous experience in endourologic procedures. After a 3-minute orientation on the PERC Mentor simulator (Simbionix, Cleveland, OH), each trainee was asked to perform task 4, where they had to correctly access all of the renal calyces and pop the balloons in a normal left kidney model. We collected and analyzed data from the questionnaire and the performance report generated by the simulator.

RESULTS

In total, 13 PGTs participated in this study. PGTs had performed a median of 200 (range: 50-1000) cystoscopies, 50 (range: 10-125) TURBTs, 30 (range: 0-100) TURPs, 5 (range: 0-50) laser prostatectomies, and 50 (range: 2-125) ureteroscopies prior to this OSCE. PGTs with previous PCNL experience (8/13) had performed a mean of 18.6 ± 6.3 PCNLs. PGTs with previous PCNL experience performed significantly better in terms of shorter fluoroscopy time (10 ± 1.5 vs. 5.1 ± 0.7 min; p = 0.04), fewer attempts required for successful puncture of the pelvi-calyceal system (PCS) (21 ± 2.3 vs. 13 ± 1.8; p = 0.02), and had significantly lower complications in terms of fewer infundibular injury (7.4 ± 1.5 vs. 2 ± 0.4; p = 0.004) and fewer PCS perforations (11 ± 1.7 vs. 4.5 ± 1.2; p = 0.01).

CONCLUSION

It is feasible to use the PERC Mentor simulator during OSCEs to assess PCA skills of urology PGTs. PGTs who had previous PCNL experience performed significantly better with fewer complications.

Assessment of photoselective vaporization of prostate skills during Urology Objective Structured Clinical Examinations (OSCE)

INTRODUCTION

We evaluated the use of the GreenLight Simulator (GL-SIM) (American Medical Systems, Guelph, ON) in the skill assessment of postgraduate trainees (PGTs) in photoselective vaporization of the prostate (PVP). We also sought to determine whether previous PVP experience or GL-SIM practice improved performance.

METHODS

PGTs in postgraduate years (PGY-3 to PGY-5) from all 4 Quebec urology training programs were recruited during 2 annual Objective Structured Clinical Examinations (OSCEs). During a 20-minute OSCE station, PGTs were asked to perform 2 exercises: (1) identification of endoscopic landmarks and (2) a PVP of a 30-g normal prostate. Grams vaporized, global scores, and number of correct anatomical landmarks were recorded and correlated with PGY level, practice on the GL-SIM, and previous PVP experience.

RESULTS

In total, 25 PGTs were recruited at each OSCE, with 13 PGTs participating in both OSCEs. When comparing scores from the first and second OSCEs, there was a significant improvement in the number of grams vaporized (2.9 vs. 4.3 g; p = 0.003) and global score (100 vs. 165; p = 0.03). There was good correlation between the number of previously performed PVPs and the global score (r = 0.4, p = 0.04). Similarly, PGTs with previous practice on the GL-SIM had significantly higher global score (100.6 vs. 162.6; p = 0.04) and grams vaporized (3.1 vs. 4.1 g; p = 0.04) when compared with those who did not practice on GL-SIM. Furthermore, there were significantly more competent PGTs among those who had previously practiced on the GL-SIM (32.7% vs. 10.2%; p = 0.009). PGY level did not significantly affect grams vaporized or global score (p > 0.05).

CONCLUSION

Performance on the GL-SIM at OSCEs significantly correlated with previous practice on the GL-SIM and previous PVP experience rather than PGY level. Furthermore, there were significantly more competent PGTs among those who had previously practiced on the GL-SIM.