Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc. 2010;24:1834-1841. [PMID: 20112113 DOI: 10.1007/s00464-010-0882-8]

Can Pediatric Endoscopists Accurately Assess Their Clinical Competency? A Comparison Across Skill Levels

BACKGROUND

Assessment is critical to support pediatric endoscopy training. Although trainee engagement in assessment is encouraged, the use of self-assessment and its accuracy among pediatric endoscopists is not well described. We aimed to determine the self-assessment accuracy of novice, intermediate, and experienced pediatric endoscopists.

METHODS

Novice (performed <50 previous colonoscopies), intermediate (50-500), and experienced (>1000) pediatric endoscopists from 3 North American academic teaching hospitals each performed a clinical colonoscopy. Endoscopists were assessed in real-time by 2 experienced endoscopists using the Gastrointestinal Endoscopy Competency Assessment Tool for Pediatric Colonoscopy (GiECATKIDS). In addition, participants self-assessed their performance using the same instrument. Self-assessment accuracy between the externally assessed and self-assessed scores was evaluated using absolute difference scores, intraclass correlation coefficients, and Bland-Altman analyses.

RESULTS

Forty-seven endoscopists participated (21 novices, 16 intermediates, and 10 experienced). Overall, there was moderate agreement of externally assessed and self-assessed GiECATKIDS total scores with an intraclass correlation coefficient of 0.72 (95% confidence interval, 0.55-0.83). The absolute difference scores among the 3 groups were significantly different (P = 0.005), with experienced endoscopists demonstrating a more accurate self-assessment compared to novices (P = 0.003). Bland-Altman plots revealed that novice endoscopists' self-assessed scores tended to be higher than their externally assessed scores, indicating they overestimated their performance.

CONCLUSIONS

We found that endoscopic experience was positively associated with self-assessment accuracy among pediatric endoscopists. Novices were inaccurate in assessing their endoscopic competence and were prone to overestimation of their performances. Our findings suggest novices may benefit from targeted interventions aimed at improving their insight and self-awareness.

Validity Evidence for Direct Observation of Procedural Skills in Paediatric Gastroscopy

OBJECTIVES

Direct observation of procedural skills (DOPS) are competence-assessment tools in endoscopy. Formative paediatric gastroscopy DOPS were implemented into the UK curriculum in 2016 but lack validity evidence; we aimed to assess validity evidence using a recognised contemporary validity framework.

METHODS

We performed a prospective UK-wide analysis of formative paediatric gastroscopy DOPS submitted to the e-Portfolio over 1 year. Internal structure validity was assessed using interitem correlations between DOPS items, average domain, and skillset scores and with the overall competency rating. Overall competence scores and mean DOPS scores were compared by trainee seniority and procedure count (discriminative validity). Receiver operating characteristic curve analysis was performed to explore if DOPS scores could be used to delineate procedural competency (consequential validity).

RESULTS

A total of 157 DOPS assessments were completed by 20 trainers for 17 trainees. Strengths of correlations varied between DOPS components, with overall competency correlating most with technical-predominant items, domains and skillsets. Both the overall assessor's rating and mean DOPS scores increased with trainee seniority (P < 0.001) and lifetime procedure count (P < 0.001). Overall competency could be delineated using mean DOPS scores (area under receiver operating characteristic curve 0.95, P < 0.001), with a threshold of 3.9 providing optimal sensitivity (94.4%) and specificity (89.7%).

CONCLUSIONS

Competencies in paediatric gastroscopy, as assessed using DOPS, vary in their correlation with overall competence and increase with trainee experience. Formative DOPS thresholds could be used to indicate readiness for summative assessment. Our study therefore provides evidence of internal structure, discriminative, and consequential validity in support of formative paediatric gastroscopy DOPS.

The assessment of endosonographers in training

Endosonography (EUS) has an estimated long learning curve including the acquisition of both technical and cognitive skills. Trainees in EUS must learn to master intraprocedural steps such as echoendoscope handling and ultrasonographic imaging with the interpretation of normal anatomy and any pathology. In addition, there is a need to understand the periprocedural parts of the EUS-examination such as the indications and contraindications for EUS and potential adverse events that could occur post-EUS. However, the learning process and progress vary widely among endosonographers in training. Consequently, the performance of a certain number of supervised procedures during training does not automatically guarantee adequate competence in EUS. Instead, the assessment of EUS-competence should preferably be performed by the use of an assessment tool developed specifically for the evaluation of endosonographers in training. Such a tool, covering all the different steps of the EUS-procedure, would better depict the individual learning curve and better reflect the true competence of each trainee. This mini-review will address the issue of clinical education in EUS with respect to the evaluation of endosonographers in training. The aim of the article is to provide an informative overview of the topic. The relevant literature of the field will be reviewed and discussed. The current knowledge on how to assess the skills and competence of endosonographers in training is presented in detail.

The FES Test: Are We Ready?

PURPOSE

The FES hands-on skills test is administered using a $100,000 computer-based simulator. Few of our trainees have practiced on this device. Our aim was to evaluate our GS residents' baseline endoscopic skills and eventually develop a simulation-based endoscopy curriculum and clarify performance-based assessment criteria.

METHODS

General surgery residents' colonoscopy skills were assessed using a computer-based endoscopy simulator (CBES) during their biannual simulation-based OSCE-type assessments. Trainees were asked to reach the ileum in <5 minutes with minimal patient pain and complications. Module 1 (easy) was assigned to PGY 1-4 residents and module 5 (hard) to both PGY 4s and 5s. The colonoscope insertion length, % time with no pain, % time in "red out", and complications were recorded. Performance grading criteria were driven by literature review and expert opinion. Residents were assessed in the fall 2017; they were then given scoring criteria, a step-by-step instruction manual, and a voluntary hands-on session with the CBES. Residents repeated the same assessment in the spring 2018.

RESULTS

30 PGY-1s, 12 PGY-2s, 8 PGY-3s, 9 PGY-4s and 7 PGY-5s GS residents participated in the fall colonoscopy assessment. In module 1, 66% of PGY-4s, 50% of PGY-3s, 8% of PGY-2s and 0% of the PGY-1s intubated the ileum (p<0.05). In module 5, 30% of PGY 5 and 22% of PGY 4 residents completed the task (p<0.05). 15 PGY-1s, 5 PGY-2s, 1 PGY-3, 2 PGY-4s, and 1 PGY-5 participated in the voluntary hands-on session. All residents completing the fall assessment undertook the same task in the spring. In module 1, 89% of PGY-4s, 100% of the PGY-3s, 75% of PGY-2s and 70% of the PGY-1s completed the task. In module 5, 30% of PGY 5 and 34 % of PGY 4 residents completed the task. Residents who participated in the voluntary hands-on session (n= 24, 96% task completion) outperformed residents (n= 42, 64% task completion) that did not participate (p<0.05).

CONCLUSIONS

Most of our GS residents could not initially intubate the ileum using the CBES. Prior experience with the CBES was the only factor strongly correlated with successful task completion. A voluntary hands-on teaching session allowed 96% of participating trainees to subsequently achieve CBES task completion. Developing a formal simulation-based curriculum suggests we can better prepare surgical trainees for the FES exam.

Effect of rater training on the reliability of technical skill assessments: a randomized controlled trial

BACKGROUND

Rater training improves the reliability of observational assessment tools but has not been well studied for technical skills. This study assessed whether rater training could improve the reliability of technical skill assessment.

METHODS

Academic and community surgeons in Royal College of Physicians and Surgeons of Canada surgical subspecialties were randomly allocated to either rater training (7-minute video incorporating frame-of-reference training elements) or no training. Participants then assessed trainees performing a suturing and knot-tying task using 3 assessment tools: a visual analogue scale, a task-specific checklist and a modified version of the Objective Structured Assessment of Technical Skill global rating scale (GRS). We measured interrater reliability (IRR) using intraclass correlation type 2.

RESULTS

There were 24 surgeons in the training group and 23 in the no-training group. Mean assessment tool scores were not significantly different between the 2 groups. The training group had higher IRR than the no-training group on the visual analogue scale (0.71 v. 0.46), task-specific checklist (0.46 v. 0.33) and GRS (0.71 v. 0.61). However, confidence intervals were wide and overlapping for all 3 tools.

CONCLUSION

For education purposes, the reliability of the visual analogue scale and GRS would be considered "good" for the training group but "moderate" for the no-training group. However, a significant difference in IRR was not shown, and reliability remained below the desired level of 0.8 for high-stakes testing. Training did not significantly improve assessment tool reliability. Although rater training may represent a way to improve reliability, further study is needed to determine effective training methods.

Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy

BACKGROUND

Endoscopy has traditionally been taught with novices practicing on real patients under the supervision of experienced endoscopists. Recently, the growing awareness of the need for patient safety has brought simulation training to the forefront. Simulation training can provide trainees with the chance to practice their skills in a learner-centred, risk-free environment. It is important to ensure that skills gained through simulation positively transfer to the clinical environment. This updated review was performed to evaluate the effectiveness of virtual reality (VR) simulation training in gastrointestinal endoscopy.

OBJECTIVES

To determine whether virtual reality simulation training can supplement and/or replace early conventional endoscopy training (apprenticeship model) in diagnostic oesophagogastroduodenoscopy, colonoscopy, and/or sigmoidoscopy for health professions trainees with limited or no prior endoscopic experience.

SEARCH METHODS

We searched the following health professions, educational, and computer databases until 12 July 2017: the Cochrane Central Register of Controlled Trials, Ovid MEDLINE, Ovid Embase, Scopus, Web of Science, BIOSIS Previews, CINAHL, AMED, ERIC, Education Full Text, CBCA Education, ACM Digital Library, IEEE Xplore, Abstracts in New Technology and Engineering, Computer and Information Systems Abstracts, and ProQuest Dissertations and Theses Global. We also searched the grey literature until November 2017.

SELECTION CRITERIA

We included randomised and quasi-randomised clinical trials comparing VR endoscopy simulation training versus any other method of endoscopy training with outcomes measured on humans in the clinical setting, including conventional patient-based training, training using another form of endoscopy simulation, or no training. We also included trials comparing two different methods of VR training.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed the eligibility and methodological quality of trials, and extracted data on the trial characteristics and outcomes. We pooled data for meta-analysis where participant groups were similar, studies assessed the same intervention and comparator, and had similar definitions of outcome measures. We calculated risk ratio for dichotomous outcomes with 95% confidence intervals (CI). We calculated mean difference (MD) and standardised mean difference (SMD) with 95% CI for continuous outcomes when studies reported the same or different outcome measures, respectively. We used GRADE to rate the quality of the evidence.

MAIN RESULTS

We included 18 trials (421 participants; 3817 endoscopic procedures). We judged three trials as at low risk of bias. Ten trials compared VR training with no training, five trials with conventional endoscopy training, one trial with another form of endoscopy simulation training, and two trials compared two different methods of VR training. Due to substantial clinical and methodological heterogeneity across our four comparisons, we did not perform a meta-analysis for several outcomes. We rated the quality of evidence as moderate, low, or very low due to risk of bias, imprecision, and heterogeneity.Virtual reality endoscopy simulation training versus no training: There was insufficient evidence to determine the effect on composite score of competency (MD 3.10, 95% CI -0.16 to 6.36; 1 trial, 24 procedures; low-quality evidence). Composite score of competency was based on 5-point Likert scales assessing seven domains: atraumatic technique, colonoscope advancement, use of instrument controls, flow of procedure, use of assistants, knowledge of specific procedure, and overall performance. Scoring range was from 7 to 35, a higher score representing a higher level of competence. Virtual reality training compared to no training likely provides participants with some benefit, as measured by independent procedure completion (RR 1.62, 95% CI 1.15 to 2.26; 6 trials, 815 procedures; moderate-quality evidence). We evaluated overall rating of performance (MD 0.45, 95% CI 0.15 to 0.75; 1 trial, 18 procedures), visualisation of mucosa (MD 0.60, 95% CI 0.20 to 1.00; 1 trial, 55 procedures), performance time (MD -0.20 minutes, 95% CI -0.71 to 0.30; 2 trials, 29 procedures), and patient discomfort (SMD -0.16, 95% CI -0.68 to 0.35; 2 trials, 145 procedures), all with very low-quality evidence. No trials reported procedure-related complications or critical flaws (e.g. bleeding, luminal perforation) (3 trials, 550 procedures; moderate-quality evidence).Virtual reality endoscopy simulation training versus conventional patient-based training: One trial reported composite score of competency but did not provide sufficient data for quantitative analysis. Virtual reality training compared to conventional patient-based training resulted in fewer independent procedure completions (RR 0.45, 95% CI 0.27 to 0.74; 2 trials, 174 procedures; low-quality evidence). We evaluated performance time (SMD 0.12, 95% CI -0.55 to 0.80; 2 trials, 34 procedures), overall rating of performance (MD -0.90, 95% CI -4.40 to 2.60; 1 trial, 16 procedures), and visualisation of mucosa (MD 0.0, 95% CI -6.02 to 6.02; 1 trial, 18 procedures), all with very low-quality evidence. Virtual reality training in combination with conventional training appears to be advantageous over VR training alone. No trials reported any procedure-related complications or critical flaws (3 trials, 72 procedures; very low-quality evidence).Virtual reality endoscopy simulation training versus another form of endoscopy simulation: Based on one study, there were no differences between groups with respect to composite score of competency, performance time, and visualisation of mucosa. Virtual reality training in combination with another form of endoscopy simulation training did not appear to confer any benefit compared to VR training alone.Two methods of virtual reality training: Based on one study, a structured VR simulation-based training curriculum compared to self regulated learning on a VR simulator appears to provide benefit with respect to a composite score evaluating competency. Based on another study, a progressive-learning curriculum that sequentially increases task difficulty provides benefit with respect to a composite score of competency over the structured VR training curriculum.

AUTHORS' CONCLUSIONS

VR simulation-based training can be used to supplement early conventional endoscopy training for health professions trainees with limited or no prior endoscopic experience. However, we found insufficient evidence to advise for or against the use of VR simulation-based training as a replacement for early conventional endoscopy training. The quality of the current evidence was low due to inadequate randomisation, allocation concealment, and/or blinding of outcome assessment in several trials. Further trials are needed that are at low risk of bias, utilise outcome measures with strong evidence of validity and reliability, and examine the optimal nature and duration of training.

Global Evaluative Assessment of Robotic Skills in Endoscopy (GEARS-E): objective assessment tool for master and slave transluminal endoscopic robot

BACKGROUND AND STUDY AIMS

 The Master and Slave Transluminal Endoscopic Robot (MASTER) is a novel robotic endosurgical system with two operating arms that offer multiple degrees of freedom. We developed a new assessment tool, the Global Evaluative Assessment of Robotic Skills in Endoscopy (GEARS-E), derived from existing tools in laparoscopic and robotic surgery, and evaluated its utility in the performance of procedures using MASTER.

METHODS

This was a pilot study conducted in vivo and ex vivo on animals. Three operators (Novice-1, Novice-2 and Expert) performed a total of five endoscopic submucosal dissections (ESD) using MASTER. Novice operators had no MASTER experience and the expert had previously performed eight MASTER ESDs. Operator performance was assessed by four independent evaluators using GEARS-E, which has a maximum score of 25 for five domains representing various skill-related variables (depth perception, bimanual dexterity, efficiency, tissue handling and autonomy).

RESULTS

The mean global rating scores for Novice-1 first attempt, Novice-1 second attempt, Novice-2 first attempt, Novice-2 second attempt and Expert's cases were 13.0, 16.0, 13.3, 15.5, and 21.5, respectively. The mean scores of each of the five domains were statistically higher for the second attempts compared to the first attempts for both Novice-1 and Novice-2. The mean scores of each of the five domains for the Expert's case were consistently higher than those for the two novice operators in both their first and second attempts.

CONCLUSION

Results using GEARS-E showed correlations between surgical experience and MASTER ESD. As an assessment tool for evaluation of surgical skills, GEARS-E has great potential for application in MASTER procedures.

Training in Endoscopy

PURPOSE OF THE REVIEW

Progress towards the goal of high-quality endoscopy across health economies has been founded on high-quality structured training programmes linked to credentialing practice and ongoing performance monitoring. This review appraises the recent literature on training interventions, which may benefit performance and competency acquisition in novice endoscopy trainees.

RECENT FINDINGS

Increasing data on the learning curves for different endoscopic procedures has highlighted variations in performance amongst trainees. These differences may be dependent on the trainee, trainer and training programme. Evidence of the benefit of knowledge-based training, simulation training, hands-on courses and clinical training is available to inform the planning of ideal training pathway elements. The validation of performance assessment measures and global competency tools now also provides evidence on the effectiveness of training programmes to influence the learning curve. The impact of technological advances and intelligent metrics from national databases is also predicted to drive improvements and efficiencies in training programme design and monitoring of post-training outcomes. Training in endoscopy may be augmented through a series of pre-training and in-training interventions. In conjunction with performance metrics, these evidence-based interventions could be implemented into training pathways to optimise and quality assure training in endoscopy.

The Dimensionless Squared Jerk: An Objective Parameter That Improves Assessment of Hand Motion Analysis during Simulated Shoulder Arthroscopy

PURPOSE

Attempts to quantify hand movements of surgeons during arthroscopic surgery faced limited progress beyond motion analysis of hands and/or instruments. Surrogate markers such as procedure time have been used. The dimensionless squared jerk (DSJ) is a measure of deliberate hand movements. This study tests the ability of DSJ to differentiate novice and expert surgeons (construct validity) whilst performing simulated arthroscopic shoulder surgical tasks.

METHODS

Six residents (novice group) and six consultants (expert group) participated in this study. Participants performed three validated tasks sequentially under the same experimental setup (one performance). Each participant had ten performances assessed. Hand movements were recorded with optical tracking system. The DSJ, time taken, total path length, multiple measures of acceleration, and number of movements were recorded.

RESULTS

There were significant differences between novices and experts when assessed using time, number of movements with average and minimal acceleration threshold, and DSJ. No significant differences were observed in maximum acceleration, total path length, and number of movements with 10m/s2 acceleration threshold.

CONCLUSION

DSJ is an objective parameter that can differentiate novice and expert surgeons' simulated arthroscopic performances. We propose DSJ as an adjunct to more conventional parameters for arthroscopic surgery skills assessment.

Methods for certification in colonoscopy - a systematic review

INTRODUCTION

Reliable, valid, and feasible assessment tools are essential to ensure competence in colonoscopy. This study aims to provide an overview of the existing assessment methods and the validity evidence that supports them.

METHODS

A systematic search was conducted in October 2016. Pubmed, EMBASE, and PsycINFO were searched for studies evaluating assessment methods to ensure competency in colonoscopy. Outcome variables were described and evidence of validity was explored using a contemporary framework.

RESULTS

Twenty-five observational studies were included in the systematic review. Most studies were based on small sample sizes. The studies were categorized after outcome measures into five groups: Clinical process related outcome metrics (n = 2), direct observational colonoscopy assessment (n = 8), simulator based metrics (n = 11), automatic computerized metrics (n = 2), and self-assessment (n = 1). Validity score varied among the studies and only five studies presented sufficient evidence to recommend the tool for clinical assessment.

CONCLUSIONS

The objectives vary throughout the presented tools. Some tools are global tools where others focus on procedural technical skill assessment or even part-task skills. There is a tendency in the most recent studies towards more specific assessment of technical skills. The majority of assessment methods lack sufficient validity evidence.

A prospective comparison of live and video-based assessments of colonoscopy performance

BACKGROUND AND AIMS

Colonoscopy performance is typically assessed by a supervisor in the clinical setting. There are limitations of this approach, however, because it allows for rater bias and increases supervisor workload demand during the procedure. Video-based assessment of recorded procedures has been proposed as a complementary means by which to assess colonoscopy performance. This study sought to investigate the reliability, validity, and feasibility of video-based assessments of competence in performing colonoscopy compared with live assessment.

METHODS

Novice (<50 previous colonoscopies), intermediate (50-500), and experienced (>1000) endoscopists from 5 hospitals participated. Two views of each colonoscopy were videotaped: an endoscopic (intraluminal) view and a recording of the endoscopist's hand movements. Recorded procedures were independently assessed by 2 blinded experts using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT), a validated procedure-specific assessment tool comprising a global rating scale (GRS) and checklist (CL). Live ratings were conducted by a non-blinded expert endoscopist. Outcomes included agreement between live and blinded video-based ratings of clinical colonoscopies, intra-rater reliability, inter-rater reliability and discriminative validity of video-based assessments, and perceived ease of assessment.

RESULTS

Forty endoscopists participated (20 novices, 10 intermediates, and 10 experienced). There was good agreement between the live and video-based ratings (total, intra-class correlation [ICC] = 0.847; GRS, ICC = 0.868; CL, ICC = 0.749). Intra-rater reliability was excellent (total, ICC = 0.99; GRS, ICC = 0.99; CL, ICC = 0.98). Inter-rater reliability between the 2 blinded video-based raters was high (total, ICC = 0.91; GRS, ICC = 0.918; CL, ICC = 0.862). GiECAT total, GRS, and CL scores differed significantly among novice, intermediate, and experienced endoscopists (P < .001). Video-based assessments were perceived as "fairly easy," although live assessments were rated as significantly easier (P < .001).

CONCLUSIONS

Video-based assessments of colonoscopy procedures using the GiECAT have strong evidence of reliability and validity. In addition, assessments using videos were feasible, although live assessments were easier.

Impact of cap-assisted colonoscopy on the learning curve and quality in colonoscopy: a randomized controlled trial

BACKGROUND AND AIMS

Colonoscopy competency assessment in trainees traditionally has been informal. Comprehensive metrics such as the Assessment of Competency in Endoscopy (ACE) tool suggest that competency thresholds are higher than assumed. Cap-assisted colonoscopy (CAC) may improve competency, but data regarding novice trainees are lacking. We compared CAC versus standard colonoscopy (SC) performed by novice trainees in a randomized controlled trial.

METHODS

All colonoscopies performed by 3 gastroenterology fellows without prior experience were eligible for the study. Exclusion criteria included patient age <18 or >90 years, pregnancy, prior colon resection, diverticulitis, colon obstruction, severe hematochezia, referral for EMR, or a procedure done without patient sedation. Patients were randomized to either CAC or SC in a 1:1 fashion. The primary outcome was the independent cecal intubation rate (ICIR). Secondary outcomes were cecal intubation time, polyp detection rate, polyp miss rate, adenoma detection rate, ACE tool scores, and cumulative summation learning curves.

RESULTS

A total of 203 colonoscopies were analyzed, 101 in CAC and 102 in SC. CAC resulted in a significantly higher cecal intubation rate, at 79.2% in CAC compared with 66.7% in SC (P = .04). Overall cecal intubation time was significantly shorter at 13.7 minutes for CAC versus 16.5 minutes for SC (P =.02). Cecal intubation time in the case of successful independent fellow intubation was not significantly different between CAC and SC (11.6 minutes vs 12.7 minutes; P = .29). Overall ACE tool motor and cognitive scores were higher with CAC. Learning curves for ICIR approached the competency threshold earlier with cap use but reached competency for only 1 fellow. The polyp detection rate, polyp miss rate, and adenoma detection rate were not significantly different between groups.

CONCLUSIONS

CAC resulted in significant improvement in ICIR, overall ACE tool scores, and trend toward competency on learning curves when compared with SC in colonoscopy trainees without prior colonoscopy experience. (Clinical trial registration number: NCT02472730.).

Impact of experience on self-assessment accuracy of clinical colonoscopy competence

BACKGROUND AND AIMS

Self-assessment is important for life-long learning and a recommended assessment method for endoscopy skills. Prior literature has not investigated self-assessment accuracy of colonoscopic competence in the clinical setting. This study aimed to determine the self-assessment accuracy of novice, intermediate, and experienced endoscopists.

METHODS

Novice (performed <50 previous colonoscopies), intermediate (50-500), and experienced (>1000) endoscopists from 5 hospitals each performed a clinical colonoscopy. Video recordings of procedures were independently assessed by 2 blinded expert endoscopists by using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Externally assessed and self-assessed GiECAT scores were defined as the mean of the 2 video-based ratings and as participants' own assigned ratings, respectively. Self-assessment accuracy between the externally assessed and self-assessed scores was evaluated by using absolute difference scores, intraclass correlation coefficients, and the Bland-Altman analysis.

RESULTS

Twenty novice, 10 intermediate, and 10 experienced endoscopists participated. There was moderate agreement of externally assessed and self-assessed GiECAT scores, with an intraclass correlation coefficient of 0.65 (95% confidence interval, 0.44-0.80). The absolute difference scores among the 3 groups were significantly different (P = .002), with experienced endoscopists demonstrating a more accurate self-assessment ability compared with novices (P = .002). Bland-Altman plots suggest that novice and experienced endoscopists tend to overrate and underrate their clinical competence, respectively; no specific trends were associated with intermediates.

CONCLUSION

Participants demonstrated moderate self-assessment accuracy of clinical competence. Endoscopist experience was positively associated with self-assessment accuracy; novices demonstrated lower self-assessment accuracy compared with experienced endoscopists. Moreover, novices tended to overestimate their performances. Novice endoscopists may benefit from targeted interventions to improve self-assessment accuracy.

The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy

BACKGROUND AND AIMS

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application.

METHODS

The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing.

RESULTS

Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P < .0001). The mean SAFE-T overall score decreased with increasing case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P < .0001), and average cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P < .0001). Correlation of the SAFE-T overall score with the MCSAT overall hands-on and individual motor scores was excellent (each r > 0.90, P < .0001).

CONCLUSIONS

We developed and validated the SAFE-T assessment tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy.

Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the Endoscopy Training System

BACKGROUND

The Fundamentals of Endoscopic Surgery (FES) exam is required for American Board of Surgery certification. The purpose of this study was to develop performance standards for a simulation-based mastery learning (SBML) curriculum for the FES performance exam using the Endoscopy Training System (ETS).

METHODS

Experienced endoscopists from multiple institutions and specialties performed each ETS task (scope manipulation (SM), tool targeting (TT), retroflexion (RF), loop management (LM), and mucosal inspection (MI)) with scores used to develop performance standards for a SBML training curriculum. Trainees completed the curriculum to determine feasibility, and effect on FES performance.

RESULTS

Task specific training standards were determined (SM-121sec, TT-243sec, RF-159sec, LM-261sec, MI-180-480sec, 7 polyps). Trainees required 29.5 ± 3.7 training trials over 2.75 ± 0.5 training sessions to complete the SBML curriculum. Despite high baseline FES performance, scores improved (pre 73.4 ± 7, post 78.1 ± 5.2; effect size = 0.76, p > 0.1), but this was not statistically discernable.

CONCLUSIONS

This SBML curriculum was feasible and improved FES scores in a group of high performers. This curriculum should be applied to novice endoscopists to determine effectiveness for FES exam preparation.

A proficiency-based virtual reality endoscopy curriculum improves performance on the fundamentals of endoscopic surgery examination

INTRODUCTION

The fundamentals of endoscopic surgery (FES) examination is a national test of knowledge and skill in flexible gastrointestinal endoscopy. The skill portion of the examination involves five tasks that assesses the following skills: scope navigation, loop reduction, mucosal inspection, retroflexion, and targeting. This project aimed to assess the efficacy of a proficiency-based virtual reality (VR) curriculum in preparing residents for the FES skills exam.

METHODS

Experienced (>100 career colonoscopies) and inexperienced endoscopists (<50 career colonoscopies) were recruited to participate. Six VR modules were identified as reflecting the skills tested in the exam. All participants were asked to perform each of the selected modules twice, and median performance was compared between the two groups. Inexperienced endoscopists were subsequently randomized in matched pairs into a repetition (10 repetitions of each task) or proficiency curriculum. After completion of the respective curriculum, FES scores and pass rates were compared to national data and historical institutional control data (endoscopy-rotation training alone).

RESULTS

Five experienced endoscopists and twenty-three inexperienced endoscopists participated. Construct valid metrics were identified for six modules and proficiency benchmarks were set at the median performance of experienced endoscopists. FES scores of inexperienced endoscopists in the proficiency group had significantly higher FES scores (530 ± 86) versus historical control (386.7 ± 92.2, p = 0.0003) and higher pass rate (proficiency: 100%, historical control 61.5%, p = 0.01).

CONCLUSION

Trainee engagement in a VR curriculum yields superior FES performance compared to an endoscopy rotation alone. Compared to the 2012-2016 national resident pass rate of 80, 100% of trainees in a proficiency-based curriculum passed the FES manual skills examination.

Objective Measurement of Clinical Competency in Surgical Education Using Electrodermal Activity

OBJECTIVE

Within the realm of surgical education, there is a need for objective means to determine surgical competence and resident readiness to operate independently. We propose a novel, objective method of assessing resident confidence and clinical competence based on measurement of electrodermal activity (EDA) during live surgical procedures. We hypothesized that with progressive training, EDA responses to the stress of performing surgery would exhibit decline, elucidating an objective correlate of clinical competence.

DESIGN

EDA was measured using galvanic skin response sensors worn by residents performing laparoscopic cholecystectomy on sequential live human patients over an 8-month period. Baseline, phasic (peak) and tonic EDA responses were measured as a fractional change from baseline.

SETTING

University of Missouri, Columbia, Missouri, an academic tertiary care facility.

PARTICIPANTS

Fourteen categorical general surgery residents and 5 faculty surgeons were voluntarily enrolled and participated through completion.

RESULTS

Tonic fractional change (FC_TONIC) was highest in PGY3 residents compared with postgraduate year (PGY) 1 and 2 residents (7.199 vs. 2.100, p = 0.004, 95% CI: 8.58-1.61 and PGY4 and 5 residents (7.199 vs. 2.079, p = 0.002, 95% CI: 8.38-0.29). Phasic fractional change in EDA (FC_PHASIC) exhibited a progressive decline across resident training levels, with PGY1 and 2 residents having the highest response, and faculty displaying the lowest FC_PHASIC responses. Statistical differences were seen between FC_PHASIC faculty and PGY4 and 5 (3.596 vs. 6.180, p = 0.004, 95% CI: 0.80-4.36), PGY4 and 5, and PGY3 (6.180 vs. 15.998, p = 0.003, 95% CI: 3.33-16.3), as well as among all residents and faculty (13.057 vs. 3.596, p = 0.004, 95% CI: 15.8-3.1).

CONCLUSION

Phasic EDA changes decrease with increasing clinical competence. For those participants with the lowest and highest levels of competence, tonic EDA changes are minimal. Tonic EDA changes follow an inverse-U shape with differing levels of clinical competence.

Learning curve for endoscopy training: Is it all about numbers?

Endoscopy training is an important component of postgraduate gastroenterology and general surgery programs. Proficiency in endoscopy requires the development of several tangible and intangible skills. Much attention has traditionally been paid to establishing a threshold, or minimum procedural volume during the training period, which is necessary for a trainee to achieve competence in endoscopy by the conclusion of his or her program. However, despite several attempts to characterize this target, it has become clear in recent years that training programs need to consider other factors rather than relying on this measure as the sole marker of trainee competency. Here, we present a review of general concepts in endoscopy skills acquisition that affect the learning curve, the evolving definition of competency as it relates to procedural volume, the role of simulation in endoscopy training, and the concept of massed versus spaced delivery of endoscopy training.

Assessments and skills improvement for endoscopists

Different countries employ a range of assessment methods to monitor trainees from novice to independent practice. The optimal method to monitor and assess individuals' training in endoscopy has not been formally determined. The UK has developed a competency based assessment training and certification (credentialing) programme. The tools developed to provide endoscopy work based assessments (DOPS) have been validated and are used for trainees and independent endoscopists, providing formative feedback for targeted training. Summative assessments are used for trainee certification and independent colonoscopists wishing to provide part of the National Bowel Cancer Screening Programme. The UK was able to develop both clinical standards and an endoscopy training and certification process applied to all individuals and monitored by a single professional body. The supporting IT system enabled a structured and robust quality assurance process to be applied to all individuals and endoscopy units. Assessment of practising endoscopists relies on the development and measurement of surrogate measures, which represent key performance indicators for those individuals. These surrogates for performance are still evolving although they are now well established for colonoscopy practice. Monitoring of independent practice is dependent on clinical audit of these key performance indicators. Feedback of data to individuals helps benchmarking and identification of those with sub-optimal performance. Independent endoscopists now recognize the benefit of on-going training to help both skills development and to address sub-optimal performance. This chapter describes how the UK developed a web-based integrated training and certification system.

In-training gastrointestinal endoscopy competency assessment tools: Types of tools, validation and impact

The ability to perform endoscopy procedures safely, effectively and efficiently is a core element of gastroenterology practice. Training programs strive to ensure learners demonstrate sufficient competence to deliver high quality endoscopic care independently at completion of training. In-training assessments are an essential component of gastrointestinal endoscopy education, required to support training and optimize learner's capabilities. There are several approaches to in-training endoscopy assessment from direct observation of procedural skills to monitoring of surrogate measures of endoscopy skills such as procedural volume and quality metrics. This review outlines the current state of evidence as it pertains to in-training assessment of competency in performing gastrointestinal endoscopy as part of an overall endoscopy quality and skills training program.

Training and competence assessment in GI endoscopy: a systematic review

INTRODUCTION

Training procedural skills in GI endoscopy once focused on threshold numbers. As threshold numbers poorly reflect individual competence, the focus gradually shifts towards a more individual approach. Tools to assess and document individual learning progress are being developed and incorporated in dedicated training curricula. However, there is a lack of consensus and training guidelines differ worldwide, which reflects uncertainties on optimal set-up of a training programme.

AIMS

The primary aim of this systematic review was to evaluate the currently available literature for the use of training and assessment methods in GI endoscopy. Second, we aimed to identify the role of simulator-based training as well as the value of continuous competence assessment in patient-based training. Third, we aimed to propose a structured training curriculum based on the presented evidence.

METHODS

A literature search was carried out in the available medical and educational literature databases. The results were systematically reviewed and studies were included using a predefined protocol with independent assessment by two reviewers and a final consensus round.

RESULTS

The literature search yielded 5846 studies. Ninety-four relevant studies on simulators, assessment methods, learning curves and training programmes for GI endoscopy met the inclusion criteria. Twenty-seven studies on simulator validation were included. Good validity was demonstrated for four simulators. Twenty-three studies reported on simulator training and learning curves, including 17 randomised control trials. Increased performance on a virtual reality (VR) simulator was shown in all studies. Improved performance in patient-based assessment was demonstrated in 14 studies. Four studies reported on the use of simulators for assessment of competence levels. Current simulators lack the discriminative power to determine competence levels in patient-based endoscopy. Eight out of 14 studies on colonoscopy, endoscopic retrograde cholangiopancreatography and endosonography reported on learning curves in patient-based endoscopy and proved the value of this approach for measuring performance. Ten studies explored the numbers needed to gain competence, but the proposed thresholds varied widely between them. Five out of nine studies describing the development and evaluation of assessment tools for GI endoscopy provided insight into the performance of endoscopists. Five out of seven studies proved that intense training programmes result in good performance.

CONCLUSIONS

The use of validated VR simulators in the early training setting accelerates the learning of practical skills. Learning curves are valuable for the continuous assessment of performance and are more relevant than threshold numbers. Future research will strengthen these conclusions by evaluating simulation-based as well as patient-based training in GI endoscopy. A complete curriculum with the assessment of competence throughout training needs to be developed for all GI endoscopy procedures.

Training and Assessment in Pediatric Endoscopy

A key aspect of pediatric gastroenterology practice is the ability to perform endoscopy procedures safely, effectively, and efficiently. Similar to adult endoscopy, performance of pediatric endoscopy requires the acquisition of related technical, cognitive, and integrative competencies to effectively diagnose and manage gastrointestinal disorders in children. However, the distinctive requirements of pediatric patients and their families and the differential spectrum of disease highlight the need for a pediatric-specific training curriculum and assessment framework to ensure endoscopic procedures are performed safely and successfully in children. This review outlines the current state of evidence as it pertains to pediatric endoscopy training and assessment.

A Review of Endoscopic Simulation: Current Evidence on Simulators and Curricula

Upper and lower endoscopy is an important tool that is being utilized more frequently by general surgeons. Training in therapeutic endoscopic techniques has become a mandatory requirement for general surgery residency programs in the United States. The Fundamentals of Endoscopic Surgery has been developed to train and assess competency in these advanced techniques. Simulation has been shown to increase the skill and learning curve of trainees in other surgical disciplines. Several types of endoscopy simulators are commercially available; mechanical trainers, animal based, and virtual reality or computer-based simulators all have their benefits and limitations. However they have all been shown to improve trainee's endoscopic skills. Endoscopic simulators will play a critical role as part of a comprehensive curriculum designed to train the next generation of surgeons. We reviewed recent literature related to the various types of endoscopic simulators and their use in an educational curriculum, and discuss the relevant findings.

Three-dimensional Technology Facilitates Surgical Performance of Novice Laparoscopy Surgeons: A Quantitative Assessment on a Porcine Kidney Model

OBJECTIVE

To determine whether the use of 3-dimensional (3D) imaging translates into a better surgical performance of naïve urologic laparoscopic surgeons during pyeloplasty (PY) and partial nephrectomy (PN) procedures.

MATERIALS AND METHODS

Eighteen surgeons without any previous laparoscopic experience were randomly assigned to perform PY and PN in a porcine model using initially 2-dimensional (2D) and 3D laparoscopy. A surgical performance score was rated by an "expert" tutor through a modified 5-item global rating scale contemplating operative field view, bimanual dexterity, efficiency, tissue handling, and autonomy. Overall surgical time, complications, subjective perception of participating surgeons, and inconveniences related to the 3D vision were recorded.

RESULTS

No difference in terms if operative time was found between 2D or 3D laparoscopy for both the PY (P = .51) and the PN (P = .28) procedures. A better rate in terms of surgical performance score was noted by the tutors when the study participants were using 3D vs 2D, for both PY (3.6 [0.8] vs 3.0 [0.4]; P = .034) and PN (3.6 [0.51] vs 3.15 [0.63]; P = .001). No complications occurred in any of the procedures. Most (77.2%) of the participating naïve laparoscopic surgeons had the perception that 3D laparoscopy was overall easier than 2D. Headache (18.1%), nausea (18.1%), and visual disturbance (18.1%) were the most common issues reported by the surgeons during 3D procedures.

CONCLUSION

Despite the absence of translation in a shorter operative time, the use of 3D technology seems to facilitate the surgical performance of naïve surgeons during laparoscopic kidney procedures on a porcine model.

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors

BACKGROUND

Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment.

METHODS

Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score.

RESULTS

Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors.

CONCLUSIONS

A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow-worn sensors were shown to achieve an accurate assessment of surgical dexterity and performance.

CLINICAL RELEVANCE

The validation of an entirely objective assessment of arthroscopic skill with wireless elbow-worn motion sensors introduces, for the first time, a feasible assessment system for the live operating theater with the added potential to be applied to other surgical and interventional specialties.

Status of Competency-Based Medical Education in Endoscopy Training: A Nationwide Survey of US ACGME-Accredited Gastroenterology Training Programs

OBJECTIVES

The Accreditation Council for Graduate Medical Education (ACGME) emphasizes the importance of medical trainees meeting specific performance benchmarks and demonstrating readiness for unsupervised practice. The aim of this study was to examine the readiness of Gastroenterology (GI) fellowship programs for competency-based evaluation in endoscopic procedural training.

METHODS

ACGME-accredited GI program directors (PDs) and GI trainees nationwide completed an online survey of domains relevant to endoscopy training and competency assessment. Participants were queried about current methods and perceived quality of endoscopy training and assessment of competence. Participants were also queried about factors deemed important in endoscopy competence assessment. Five-point Likert items were analyzed as continuous variables by an independent t-test and χ(2)-test was used for comparison of proportions.

RESULTS

Survey response rate was 64% (94/148) for PDs and 47% (546/1,167) for trainees. Twenty-three percent of surveyed PDs reported that they do not have a formal endoscopy curriculum. PDs placed less importance (1—very important to 5—very unimportant) on endoscopy volume (1.57 vs. 1.18, P<0.001), adenoma detection rate (2.00 vs. 1.53, P<0.001), and withdrawal times (1.96 vs. 1.68, P=0.009) in determining endoscopy competence compared with trainees. A majority of PDs report that competence is assessed by procedure volume (85%) and teaching attending evaluations (96%). Only a minority of programs use skills assessment tools (30%) or specific quality metrics (28%). Specific competencies are mostly assessed by individual teaching attending feedback as opposed to official documentation or feedback from a PD. PDs rate the overall quality of their endoscopy training and assessment of competence as better than overall ratings by trainees.

CONCLUSIONS

Although the majority of PDs and trainees nationwide believe that measuring specific metrics is important in determining endoscopy competence, most programs still rely on procedure volume and subjective attending evaluations to determine overall competence. As medical training transitions from an apprenticeship model to competency-based education, there is a need for improved endoscopy curricula which are better suited to demonstrate readiness for unsupervised practice.

The Relationship of Endoscopic Proficiency to Educational Expense for Virtual Reality Simulator Training Amongst Surgical Trainees

With the introduction of Fundamentals of Endoscopic Surgery™, training methods in flexible endoscopy are being augmented with simulation-based curricula. The investment for virtual reality simulators warrants further research into its training advantage. Trainees were randomized into bedside or simulator training groups (BED vs SIM). SIM participated in a proficiency-based virtual reality curriculum. Trainees’ endoscopic skills were rated using the Global Assessment of Gastrointestinal Endoscopic Skills (GAGES) in the patient care setting. The number of cases to reach 90 per cent of the maximum GAGES score and calculated costs of training were compared. Nineteen residents participated in the study. There was no difference in the average number of cases required to achieve 90 per cent of the maximum GAGES score for esophagogastroduodenoscopy, 13 (SIM) versus11 (BED) ( P = 0.63), or colonoscopy 21 (SIM) versus 4 (BED) ( P = 0.34). The average per case cost of training for esophagogastroduodenoscopy was $35.98 (SIM) versus $39.71 (BED) ( P = 0.50), not including the depreciation costs associated with the simulator ($715.00 per resident over six years). Use of a simulator appeared to increase the cost of training without accelerating the learning curve or decreasing faculty time spent in instruction. The importance of simulation in endoscopy training will be predicated on more cost-effective simulators.

Effectiveness, training and quality assurance of colonoscopy screening for colorectal cancer

Screening for colorectal cancer has been proven to be effective in reducing colorectal cancer incidence and mortality. While the precise benefit of screening exclusively by colonoscopy is not yet known, unarguably, the exam is central to the success of any screening programme. The test affords the opportunity to detect and resect neoplasia across the entire large bowel and is the definitive examination when other screening tests are positive. However, colonoscopy is invasive and often requires sedation as well as extensive bowel preparation, all of which puts the patient at risk. Furthermore, the test can technically be demanding and, unarguably, there is variation in how it is performed. This variation in performance has now been definitively linked to important outcome measures. For example, interval cancers are more common in low adenoma detectors as compared with high adenoma detectors. This review outlines the most current thinking regarding the effectiveness of colonoscopy as a screening tool. It also outlines key concepts to optimise its performance through robust quality assurance programmes and high-quality training.

Technical skills assessment toolbox: a review using the unitary framework of validity

OBJECTIVE

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity.

BACKGROUND

Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency.

METHODS

A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills.

RESULTS

A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables.

CONCLUSIONS

This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Evaluation of two flexible colonoscopy simulators and transfer of skills into clinical practice

INTRODUCTION

Surgical residents have learned flexible endoscopy by practicing on patients in hospital settings under the strict guidance of experienced surgeons. Simulation is often used to "pretrain" novices on endoscopic skills before real clinical practice; nonetheless, the optimal method of training remains unknown. The purpose of this study was to compare endoscopic virtual reality and physical model simulators and their respective roles in transferring skills to the clinical environment.

METHODS

At the beginning of a skills development rotation, 27 surgical postgraduate year 1 residents performed a baseline screening colonoscopy on a real patient under faculty supervision. Their performances were scored using the Global Assessment of Gastrointestinal Endoscopic Skills (GAGES). Subsequently, interns completed a 3-week flexible endoscopy curriculum developed at our institution. One-third of the residents were assigned to train with the GI Mentor simulator, one-third trained with the Kyoto simulator, and one-third of the residents trained using both simulators. At the end of their rotations, each postgraduate year 1 resident performed one posttest colonoscopy on a different patient and was again scored using GAGES by an experienced faculty.

RESULTS

A statistically significant improvement in the GAGES total score (p < 0.001) and on each of its subcomponents (p = 0.001) was observed from pretest to posttest for all groups combined. Subgroup analysis indicated that trainees in the GI Mentor or both simulators conditions showed significant improvement from pretest to posttest in terms of GAGES total score (p = 0.017 vs 0.024, respectively). This was not observed for those exclusively using the Kyoto platform (p = 0.072). Nonetheless, no single training condition was shown to be a better training modality when compared to others in terms of total GAGES score or in any of its subcomponents.

CONCLUSION

Colonoscopy simulator training with the GI Mentor platform exclusively or in combination with a physical model simulator improves skill performance in real colonoscopy cases when measured with the GAGES tool.

Endoscopy in Canada: Proceedings of the National Roundtable

This 2014 roundtable discussion, hosted by the Canadian Association of General Surgeons, brought together general surgeons and gastroenterologists with expertise in endoscopy from across Canada to discuss the state of endoscopy in Canada. The focus of the roundtable was the evaluation of the competence of general surgeons at endoscopy, reviewing quality assurance parameters for high-quality endoscopy, measuring and assessing surgical resident preparedness for endoscopy practice, evaluating credentialing programs for the endosuite and predicting the future of endoscopic services in Canada. The roundtable noted several important observations. There exist inadequacies in both resident training and the assessment of competency in endoscopy. From these observations, several collaborative recommendations were then stated. These included the need for a formal and standardized system of both accreditation and training endoscopists.

Development of the gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy (GiECAT KIDS)

OBJECTIVES

Many aspects of pediatric colonoscopy differ from adult practice. To date, there is no validated measure of endoscopic competence for use in pediatrics. Using Delphi methodology, we aimed to determine expert consensus regarding items required on a checklist and global rating scale designed to assess the competence of clinicians performing colonoscopy on pediatric patients.

METHODS

A total of 41 North American pediatric endoscopy experts rated potential checklist and global rating items for their importance as indicators of the competence of trainees learning to perform pediatric colonoscopy. Responses were analyzed and re-sent to the panel for further ratings until consensus was reached. Items that ≥ 80% of experts rated as ≥ 4 out of 5 were included in the final instrument. Consensus items were compared with those items deemed by adult endoscopy experts as fundamental to assessing the performance of adult colonoscopy.

RESULTS

Five rounds of surveys were completed with response rates ranging from 76% to 100%. Seventy-five checklist and 38 global rating items were reduced to 18 checklist and 7 global rating items that reached consensus. Three pediatric checklist items differed from those considered to be critical adult indicators, whereas 4 items on the latter did not reach consensus among pediatric experts.

CONCLUSIONS

Delphi methodology allowed for achievement of expert consensus regarding essential items to be included in the Gastrointestinal Endoscopy Competency Assessment Tool for Pediatric Colonoscopy (GiECATKIDS), a measure of endoscopic competence specific to performing pediatric colonoscopy. Key differences in the checklist items, compared with items reaching consensus during a separate adult Delphi process using the same indicators, emphasize the need for a pediatric-specific tool.

Assessment of competence in pediatric gastrointestinal endoscopy

Gastrointestinal endoscopy is an invaluable tool for the diagnosis and treatment of upper and lower gastrointestinal diseases in children. Pediatric and adult endoscopy differ in several respects including differences in procedural indications, sedation practices, pre-procedure preparation, equipment, and the importance of routine tissue sampling and terminal ileum intubation. In the same way that performance of endoscopy in children requires pediatric-specific training, assessment of pediatric endoscopists requires an approach that is tailored to pediatric practice and the use of assessment methods and measures that have been developed and validated specifically within the pediatric context.