Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc. 2010;24:1834-1841. [PMID: 20112113 DOI: 10.1007/s00464-010-0882-8]

Colonoscopy Training on Virtual-Reality Simulators or Physical Model Simulators: A Randomized Controlled Trial

OBJECTIVE

This study employed a randomized controlled trial to assess the efficacy of virtual-reality (VR) simulators and physical model simulators on colonoscopy training to explore the optimal and evidence-based simulation training.

DESIGN

Forty participants were divided into 2 groups and randomized as dyads: the VR simulator group and the physical model simulator group. All the participants performed a baseline test through porcine colonoscopy. After a 6 h simulation training, each participant underwent a post-test on a pig after bowel preparation, and the procedures were video-recorded. Both the baseline test and the post-test were blindly assessed by 2 experienced assistant director physicians based on the GAGES-C scoring system.

SETTING

Simulation center, Renji Hospital, Shanghai Jiaotong University School of Medicine, Shanghai.

PARTICIPANTS

Forty surgical residents without colonoscopy experience.

RESULTS

Both the VR simulator group and the physical model simulator group improved significantly over the baseline test. The VR simulator group performed significantly better than the physical model simulator group, p=0.042. The participants in both groups expressed a high level of simulator satisfaction.

CONCLUSIONS

Novice residents can benefit from both VR simulators and physical model simulators. The VR simulator was shown to be more effective for colonoscopy training. VR simulators were more recommended for novices conducting basic colonoscopy training.

Current status of resident simulation training curricula: pearls and pitfalls

BACKGROUND

Residency programs are required to incorporate simulation into their training program. Ideally, simulation provides a safe environment for a trainee to be exposed to both common and challenging clinical scenarios. The purpose of this review is to detail the current state of the most commonly used laparoscopic, endoscopic, and robotic surgery simulation programs in general surgery residency education, including resources required for successful implementation and benchmarks for evaluation.

MATERIALS AND METHODS

Members of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) Resident and Fellow Task Force (RAFT) Committee performed a literature review using PubMed and training websites. Information regarding the components of the most commonly used laparoscopic, endoscopic, and/or robotic simulation curriculum, including both formal and informal benchmarks for evaluating training competence, were collected.

RESULTS

Laparoscopic simulation revolves around the Fundamentals of Laparoscopic Surgery (FLS). Proficiency-based as well as virtual simulation have been utilized for FLS training curricula. Challenges include less direct translation to the technical complexities that can arise in laparoscopic surgery. Endoscopic simulation focuses on the Fundamentals of Endoscopic Surgery. There are virtual reality simulation platforms that can be used for skills assessment and training. Challenges include simulator types and access, as well as structured mentoring and feedback. Robotic simulation training curricula have not been standardized. Simulation includes one primary technology, which can be prohibitive based on cost and requirements for onboarding.

CONCLUSIONS

While surgical simulation seems to be a fundamental and integrated part of surgical training, it requires a significant number of resources, which can be daunting for residency training programs. Regardless of the barriers outlined, the need for surgical simulation in laparoscopy, endoscopy, and robotics at surgical education training programs is clear.

The Future of Artificial Intelligence in Surgery

Until recently, innovations in surgery were largely represented by extensions or augmentations of the surgeon's perception. This includes advancements such as the operating microscope, tumor fluorescence, intraoperative ultrasound, and minimally invasive surgical instrumentation. However, introducing artificial intelligence (AI) into the surgical disciplines represents a transformational event. Not only does AI contribute substantively to enhancing a surgeon's perception with such methodologies as three-dimensional anatomic overlays with augmented reality, AI-improved visualization for tumor resection, and AI-formatted endoscopic and robotic surgery guidance. What truly makes AI so different is that it also provides ways to augment the surgeon's cognition. By analyzing enormous databases, AI can offer new insights that can transform the operative environment in several ways. It can enable preoperative risk assessment and allow a better selection of candidates for procedures such as organ transplantation. AI can also increase the efficiency and throughput of operating rooms and staff and coordinate the utilization of critical resources such as intensive care unit beds and ventilators. Furthermore, AI is revolutionizing intraoperative guidance, improving the detection of cancers, permitting endovascular navigation, and ensuring the reduction in collateral damage to adjacent tissues during surgery (e.g., identification of parathyroid glands during thyroidectomy). AI is also transforming how we evaluate and assess surgical proficiency and trainees in postgraduate programs. It offers the potential for multiple, serial evaluations, using various scoring systems while remaining free from the biases that can plague human supervisors. The future of AI-driven surgery holds promising trends, including the globalization of surgical education, the miniaturization of instrumentation, and the increasing success of autonomous surgical robots. These advancements raise the prospect of deploying fully autonomous surgical robots in the near future into challenging environments such as the battlefield, disaster areas, and even extraplanetary exploration. In light of these transformative developments, it is clear that the future of surgery will belong to those who can most readily embrace and harness the power of AI.

Saliva stress biomarkers in ERCP trainees before and after familiarisation with ERCP on a virtual simulator

Background

Stress during the early ERCP learning curve may interfere with acquisition of skills during training. The purpose of this study was to compare stress biomarkers in the saliva of trainees before and after familiarisation with ERCP exercises on a virtual simulator.

Methods

Altogether 26 endoscopists under training, 14 women and 12 men, completed the three phases of this study: Phase 1. Three different ERCP procedures were performed on the simulator. Saliva for α-amylase (sAA), Chromogranin A (sCgA), and Cortisol (sC) were collected before (baseline), halfway through the exercise (ex.), and 10 min after completion of the exercise (comp.); Phase 2. A three-week familiarisation period where at least 30 different cases were performed on the virtual ERCP simulator; and Phase 3. Identical to Phase 1 where saliva samples were once again collected at baseline, during, and after the exercise. Percentage differences in biomarker levels between baseline and exercise (Diffex) and between baseline and completion (Diffcomp) during Phase 1 and Phase 3 were calculated for each stress marker.

Results

Mean % changes, Diffex and Diffcomp, were significantly positive (p < 0.05) for all markers in both Phase 1 and Phase 3. Diffex in Phase 1 was significantly greater than Diffex in Phase 3 (p < 0.05) for sAA and sCgA. Diffcomp for sAA in Phase 1 was significantly greater than Diffcomp in Phase 3 (p < 0.05). No significant differences were found in sC concentration between Phases 1 and 3.

Conclusion

This study shows that familiarisation with the ERCP simulator greatly reduced stress as measured by the three saliva stress biomarkers used with sAA being the best. It also suggests that familiarisation with an ERCP simulator might reduce stress in the clinical setting.

Development and initial validation of a video-based peroral endoscopic myotomy assessment tool

BACKGROUND AND AIMS

Video analysis has emerged as a potential strategy for performance assessment and improvement. We aimed to develop a video-based skill assessment tool for peroral endoscopic myotomy (POEM).

METHODS

POEM was deconstructed into basic procedural components through video analysis by an expert panel. A modified Delphi approach and 2 validation exercises were conducted to refine the POEM assessment tool (POEMAT). Twelve assessors used the final POEMAT version to grade 10 videos. Fully crossed generalizability (G) studies investigated the contributions of assessors, endoscopists' performance, and technical elements to reliability. G coefficients below .5 were considered unreliable, between .5 and .7 as modestly reliable, and above .7 as indicative of satisfactory reliability.

RESULTS

After task deconstruction, discussions, and the modified Delphi process, the final POEMAT comprised 9 technical elements. G analysis showed low variance for endoscopist performance (.8%-24.9%) and high interrater variability (range, 63.2%-90.1%). The G score was moderately reliable (≥.60) for "submucosal tunneling" and "myotomy" and satisfactorily reliable (≥.70) for "active hemostasis" and "mucosal closure."

CONCLUSIONS

We developed and established initial content and response process validity evidence for the POEMAT. Future steps include appraisal of the tool using a wider range of POEM videos to establish and improve the discriminative validity of this tool.

Training in basic gastrointestinal endoscopic procedures: a European Society of Gastrointestinal Endoscopy (ESGE) and European Society of Gastroenterology and Endoscopy Nurses and Associates (ESGENA) Position Statement

This ESGE Position Statement provides structured and evidence-based guidance on the essential requirements and processes involved in training in basic gastrointestinal (GI) endoscopic procedures. The document outlines definitions; competencies required, and means to their assessment and maintenance; the structure and requirements of training programs; patient safety and medicolegal issues. 1:  ESGE and ESGENA define basic endoscopic procedures as those procedures that are commonly indicated, generally accessible, and expected to be mastered (technically and cognitively) by the end of any core training program in gastrointestinal endoscopy. 2:  ESGE and ESGENA consider the following as basic endoscopic procedures: diagnostic upper and lower GI endoscopy, as well as a limited range of interventions such as: tissue acquisition via cold biopsy forceps, polypectomy for lesions ≤ 10 mm, hemostasis techniques, enteral feeding tube placement, foreign body retrieval, dilation of simple esophageal strictures, and India ink tattooing of lesion location. 3:  ESGE and ESGENA recommend that training in GI endoscopy should be subject to stringent formal requirements that ensure all ESGE key performance indicators (KPIs) are met. 4:  Training in basic endoscopic procedures is a complex process and includes the development and acquisition of cognitive, technical/motor, and integrative skills. Therefore, ESGE and ESGENA recommend the use of validated tools to track the development of skills and assess competence. 5:  ESGE and ESGENA recommend incorporating a multimodal approach to evaluating competence in basic GI endoscopic procedures, including procedural thresholds and the measurement and documentation of established ESGE KPIs. 7:  ESGE and ESGENA recommend the continuous monitoring of ESGE KPIs during GI endoscopy training to ensure the trainee's maintenance of competence. 9:  ESGE and ESGENA recommend that GI endoscopy training units fulfil the ESGE KPIs for endoscopy units and, furthermore, be capable of providing the dedicated personnel, infrastructure, and sufficient case volume required for successful training within a structured training program. 10:  ESGE and ESGENA recommend that trainers in basic GI endoscopic procedures should be endoscopists with formal educational training in the teaching of endoscopy, which allows them to successfully and safely teach trainees.

Exploring Endoscopic Competence in Gastroenterology Training: A Simulation-Based Comparative Analysis of GAGES, DOPS, and ACE Assessment Tools

Purpose

Accurate and convenient evaluation tools are essential to document endoscopic competence in Gastroenterology training programs. The Direct Observation of Procedural Skills (DOPS), Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), and Assessment of Endoscopic Competency (ACE) are widely used validated competency assessment tools for gastrointestinal endoscopy. However, studies comparing these 3 tools are lacking, leading to lack of standardization in this assessment. Through simulation, this study seeks to determine the most reliable, comprehensive, and user-friendly tool for standardizing endoscopy competency assessment.

Methods

A mixed-methods quantitative-qualitative approach was utilized with sequential deductive design. All nine trainees in a gastroenterology training program were assessed on endoscopic procedural competence using the Simbionix Gi-bronch-mentor high-fidelity simulator, with 2 faculty raters independently completing the 3 assessment forms of DOPS, GAGES, and ACE. Psychometric analysis was used to evaluate the tools' reliability. Additionally, faculty trainers participated in a focused group discussion (FGD) to investigate their experience in using the tools.

Results

For upper GI endoscopy, Cronbach's alpha values for internal consistency were 0.53, 0.8, and 0.87 for ACE, DOPS, and GAGES, respectively. Inter-rater reliability (IRR) scores were 0.79 (0.43-0.92) for ACE, 0.75 (-0.13-0.82) for DOPS, and 0.59 (-0.90-0.84) for GAGES. For colonoscopy, Cronbach's alpha values for internal consistency were 0.53, 0.82, and 0.85 for ACE, DOPS, and GAGES, respectively. IRR scores were 0.72 (0.39-0.96) for ACE, 0.78 (-0.12-0.86) for DOPS, and 0.53 (-0.91-0.78) for GAGES. The FGD yielded three key themes: the ideal tool should be scientifically sound, comprehensive, and user-friendly.

Conclusion

The DOPS tool performed favourably in both the qualitative assessment and psychometric evaluation to be considered the most balanced amongst the three assessment tools. We propose that the DOPS tool be used for endoscopic skill assessment in gastroenterology training programs. However, gastroenterology training programs need to match their learning outcomes with the available assessment tools to determine the most appropriate one in their context.

Optimal Timing of Fundamentals of Endoscopic Surgery (FES) Testing in General Surgery Residency: Early Is Better

BACKGROUND

Fundamentals of endoscopic surgery (FES) completion is mandatory for certification by the American Board of Surgery (ABS). As early simulation and competency assessment can bolster development of trainee proficiency, we sought to determine the optimal timing for FES examination by evaluating pass rates based on training level and previous endoscopic experience.

METHODS

PGY2-5 residents at a university-based medical center who were novice to FES were assigned to complete FES training and testing. Training year, prior endoscopic experience, and FES exam scores were recorded with pre- and post-test surveys.

RESULTS

Most residents in the program (88%) were able to complete FES training and testing within a single academic year. Most required only a single faculty-led session (88%) to feel confident to take the exam, augmented by varying numbers of independent practice sessions (50% 1-2, 19% 3-5, 27% > 5). After training, most (84%) felt confident that they would pass the exam, and 93% did so on their first attempt. While higher written exam scores were noted in the PGY5 group, there were no other statistically significant differences in overall pass rates or technical exam scores based on PGY level (p = 0.24). A number of previously completed endoscopic cases did not correlate with exam scores (p = 0.24 written, p = 0.91 technical).

CONCLUSION

Fundamentals of endoscopic surgery (FES) certification can be successfully completed by junior level general surgery residents regardless of previous endoscopic experience. Moving this exam to earlier training years can benefit resident development and preparedness in the clinical setting without negatively impacting pass rates.

Cholangioscopy under direct visualization: skill progress during a dedicated Image-Guided Surgery Course

AIMS

Training programs are essential to introduce new methods for bile duct clearance. Visual examination via cholangioscopy is ideal to diagnose and treat biliary tract diseases such as cancer and choledocholithiasis. However, surgeons rarely use cholangioscopes. Specific training is required to master laparoscopic and percutaneous cholangioscopy. This study aims to assess skill acquisition and retention during cholangioscopy training in the Image-Guided Therapies Masterclass.

METHODS

This prospective study enrolled 17 physicians undergoing training in interventional treatments of biliary diseases. A novel disposable cholangioscope and access kit were used with a biliary tract model including two simulated common bile duct (CBD) stones. The curriculum required visualization of all critical structures before removal of one stone with a Dormia basket. After informed consent, demographic data and time to exercise completion were recorded on each of two subsequent training days. Task-specific questions were measured at the completion of training using a Likert scale (strongly disagree to strongly agree, 1-5 points).

RESULTS

All participants successfully completed the task (6F/11 M, age 36 ± 5 years; 13 surgeons, 4 interventional radiologists; median experience with percutaneous procedures 2 years, range 0-20). Significant improvement in mean task completion time was observed (day 1: 172 ± 59 s, day 2: 89 ± 45 s; P < 0.0001). All task-specific questions were answered with a median rating of 5/5: "The platform facilitates cholangioscopy" and "This training method accelerates gain in proficiency and is useful for residents/fellows" (IQR 5-5), "This platform is useful to measure the proficiency level" and "There is an application for simulation in percutaneous surgery training" (IQR 4.5-5), "The platform is user-friendly" and "The model quality recreates realistic scenarios" (IQR 4-5).

CONCLUSION

Cholangioscopic bile duct exploration and stone retrieval were achieved by all participants using a dedicated training program and physical simulator. Significant skill progress was observed during 2 days of dedicated training.

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis

BACKGROUND

Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists.

METHODS

This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland-Altman studies.

RESULTS

After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (-41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (-45.0 % to 35.1 % and -54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (-14.2 % to 21.4 %).

CONCLUSIONS

GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

A Novel Thyroid Ultrasound Proficiency Metric Designed Through a Multidisciplinary Delphi Approach

BACKGROUND

Thyroid ultrasounds extend surgeons' outpatient capabilities and are essential for operative planning. However, most residents are not formally trained in thyroid ultrasound. The purpose of this study was to create a novel thyroid ultrasound proficiency metric through a collaborative Delphi approach.

METHODS

Clinical faculty experienced in thyroid ultrasound participated on a Delphi panel to design the thyroid Ultrasound Proficiency Scale (UPS-Thyroid). Participants proposed items under the categories of Positioning, Technique, Image Capture, Measurement, and Interpretation. In subsequent rounds, participants voted to retain, revise, or exclude each item. The process continued until all items had greater than 70% consensus for retention. The UPS-Thyroid was pilot tested across 5 surgery residents with moderate ultrasound experience. Learning curves were assessed with cumulative sum.

RESULTS

Three surgeons and 4 radiologists participated on the Delphi panel. Following 3 iterative Delphi rounds, the panel arrived at >70% consensus to retain 14 items without further revisions or additions. The metric included the following items on a 3-point scale for a maximum of 42 points: Positioning (1 item), Technique (4 items), Image Capture (2 items), Measurement (2 items), and Interpretation (5 items). A pilot group of 5 residents was scored against a proficiency threshold of 36 points. Learning curve inflection points were noted at between 4 to 7 repetitions.

CONCLUSIONS

A multidisciplinary Delphi approach generated consensus for a thyroid ultrasound proficiency metric (UPS-Thyroid). Among surgery residents with moderate ultrasound experience, basic proficiency at thyroid ultrasound is feasible within 10 repetitions.

"Into the fire" approach to teaching endoscopic foreign body removal using a modular simulation curriculum

BACKGROUND

As flexible endoscopy becomes an increasingly valuable minimally invasive approach to surgical challenges, an efficient and comprehensive training curriculum is needed to train surgeons in therapeutic endoscopy. We developed a modular curriculum utilizing a simulation-based, "into the fire" approach to endoscopic foreign body removal for practicing physicians with task performance pre- and post-testing.

METHODS

From 2020 to 2021, two sessions of our advanced flexible endoscopy course were taught by two expert surgical endoscopists using ex-vivo porcine models. The course focused on safe removal techniques for various foreign bodies as part of an overall endoscopy curriculum that uses hands-on simulation-based pre-testing, didactics, and mentored practice sessions, followed by post-course examination. Pre- and post-course assessments and surveys were used to evaluate knowledge, performance, and confidence of participants, and subsequently analyzed using the Wilcoxon-signed rank test.

RESULTS

Of the 16 practicing physicians who participated in the course, 43.8% were certified in Fundamentals of Endoscopic Surgery, and 62.5% had completed > 200 prior upper endoscopies. Upon course completion, scoring on knowledge-based written examinations improved from 3.4 ± 1.9 to 5.8 ± 2.0 (p < 0.001). Technical facility of each participant demonstrated significant overall improvement with post-course score increased from 15.8 ± 2.5 to 23.6 ± 1.6 (p < 0.001), with skill refinement noted in technical subcategories of appropriate instrument use (p < 0.001), foreign body manipulation (p < 0.001), and successful foreign body removal (p < 0.001). Confidence surveys likewise demonstrated significant increase in confidence after completion of the curriculum 11.6 ± 3.4 to 23.0 ± 5.5 (p < 0.001).

CONCLUSIONS

The "into the fire" approach to teaching endoscopic foreign body removal utilizing our simulation module provides an effective curriculum to improve knowledge, confidence, and overall technical performance. Our methodology utilizes hands-on, simulation-based pre-testing prior to instruction. This introduces clinical scenarios and technical challenges, while accounting for and tailoring to provider-specific variation in knowledge and experience, facilitating training efficiency.

Teaching endoscopic management of gastrointestinal hemorrhage using a modular simulation curriculum

BACKGROUND

As flexible endoscopy is increasingly adopted as a minimally invasive approach to surgical challenges, an efficient curriculum is needed to train surgeons in therapeutic endoscopy. We developed a simulation-based approach to teaching endoscopic management of gastrointestinal hemorrhage as part of a modular curriculum, complete with task performance pre- and post-testing.

METHODS

Two sessions of our advanced flexible endoscopy course were taught using ex vivo porcine models to simulate active gastrointestinal hemorrhage and allow for training in hands-on endoscopic management. The module is composed of hands-on pretesting, didactics, mentored practice sessions, and postcourse assessments. Pre- and postcourse tests and surveys evaluated knowledge, confidence, and performance of participants and results were analyzed using the paired t test.

RESULTS

Sixteen practicing surgeons participated in the course. After course completion, overall knowledge-based assessments improved from 3.4 (±1.9) to 5.8 (±2.0) (P < .001). Although participants with glove sizes >7.0 and ≥2 years in practice had higher pretest evaluator scores (P = .045 and P = .020), all participants demonstrated overall improvement in endoscopic management of hemorrhage, with postcourse evaluator score increases from 20.9 (±1.6) to 23.6 (±2.0) (P = .001) and specific improvements in identification of target bleeding (P = .015), endoscopic clip setup (P < .001), and clip deployment (P = .002). Surveys also found increased confidence in competency after curriculum completion, 11.6 (±3.4)-23.0 (±5.5) (P < .001).

CONCLUSION

Our simulation-based approach to teaching the endoscopic management of gastrointestinal bleeding emphasizes hands-on pretesting and provides an effective training model to improve the knowledge, confidence, and technical performance of practicing surgeons.

Visual gaze patterns in trainee endoscopists - a novel assessment tool

BACKGROUND

Colonoscopy proficiency is significantly influenced by skills achieved during training. Although assessment scores exist, they do not evaluate the impact of visual search strategies and their use is time and labour intensive. Eye-tracking has shown significant differences in visual gaze patterns (VGPs) between expert endoscopists with varying polyp detection rates, so may provide a means of automated assessment and guidance for trainees. This study aimed to assess the feasibility of eye-tracking as a novel assessment method for trainee endoscopists.

METHODS

Eye-tracking glasses were used to record 26 colonoscopies from 12 endoscopy trainees who were assessed with directly observed procedural scores (DOPS), devised by the Joint Advisory Group (JAG) on GI endoscopy, and a visual analogue score of overall competence. A 'total weighted procedure score' (TWPS) was calculated from 1 to 20. Primary outcomes of fixation duration (FixD) and fixation frequency (FixF) were analysed according to areas of interest (AOIs) with the bowel surface and lumen represented by three concentric rings. Correlation was assessed using Pearson's coefficient. Significance was set at p<.050.

RESULTS

Trainees displayed a significant positive correlation between TWPS and FixD (R = 0.943, p<.0001) and FixF (R = 0.936, p<.0001) in the anatomical bowel mucosa peripheries. Conversely, they had significant negative correlations between TWPS and the anatomical bowel lumen (FixD: R= -0.546, p=.004; FixF: R= -0.568, p=.002).

CONCLUSIONS

Higher objective performance scores were associated with VGPs focussing on bowel mucosa. This is consistent with prior analysis showing peripheral VGPs correspond with higher polyp detection rates. Analysis of VGPs, therefore, has potential for training and assessment in colonoscopy.

A modular simulation curriculum to teach endoscopic stenting to practicing surgeons: an "Into the fire" approach

BACKGROUND

Flexible endoscopy is a valuable tool for the gastrointestinal (GI) surgeon, creating a need for effective and efficient training curricula in therapeutic endoscopic techniques for trainees and practicing providers. Here, we present a simulation-based modular curriculum using an "into the fire" approach with hands-on pre- and post-testing to teach endoscopic stenting to practicing surgeons.

METHODS

Three advanced flexible endoscopy courses were taught by expert surgical endoscopists from 2018 to 2019. The stenting module involved using self-expandable metal stents to manage simulated esophageal and gastroduodenal strictures on a non-tissue GI model. Based on the educational theories of inquiry-based learning, the simulation curriculum was designed with a series of pre-tests, didactics, mentored hands-on instructions, and post-tests. Assessments included a confidence survey, knowledge-based written test, and evaluation form specific to the hands-on performance of endoscopic stenting.

RESULTS

Twenty-eight practicing surgeons with varying endoscopic experiences participated in the course. Most of the participants (67.9%) had completed over 100 upper endoscopic procedures and 57.1% were certified in Fundamentals of Endoscopic Surgery. After completing the modular curriculum, participant confidence survey scores improved from 11.4 ± 4.2 to 20.7 ± 4.0 (p < 0.001). Knowledge-based written test scores also improved from 7.1 ± 1.2 to 8.4 ± 0.9 (p < 0.001). In terms of technical performance, overall hands-on performance scores improved from 21.3 ± 2.7 to 28.9 ± 1.2 (p < 0.001) with significant improvement in each individual component of the assessment (all p values < 0.01) and the greatest improvement seen in equipment handling (88%) and flow of procedure (54%).

CONCLUSION

Our modular simulation curriculum using an "into the fire" approach to teach endoscopic stenting is effective in improving learner knowledge, confidence, and hands-on performance of endoscopic stenting. This approach to simulation is effective, efficient, and adaptable to teaching practicing surgeons with varying levels of experience.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training

BACKGROUND

Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.

METHODS

Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.

RESULTS

Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.

CONCLUSION

A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Defining the learning curves of colorectal surgical trainees in colonoscopy using the Assessment of Competency in Endoscopy tool

BACKGROUND AND AIMS

Gastroenterology fellows require on average 250 to 275 colonoscopies to achieve competency. For surgical trainees, 50 colonoscopies is deemed adequate. Two training pathways using different assessment methods make any direct comparison impossible. At the Mayo Clinic colonoscopy training of gastroenterology and colorectal surgery (CRS) fellows were merged in 2017, providing a unique opportunity to define the learning curves of CRS trainees using the Assessment of Competency in Endoscopy (ACE) evaluation tool.

METHODS

In a single-center retrospective descriptive study, ACE scores were collected on colonoscopies performed by CRS fellows over a period of 4 academic years. By calculating the average scores at every 25 procedures of experience, the CRS colonoscopy learning curves were described for each core cognitive and motor skill.

RESULTS

Twelve CRS fellows (men, 8; women, 4) had an average prior experience of 123 colonoscopies (range, 50-266) during the general surgical residency. During CRS fellowship, an average of 136 colonoscopies (range, 116-173) were graded per fellow. Although the competency goals for a few metrics were met earlier, most motor and cognitive ACE metrics reached the minimum competency thresholds at 275 to 300 procedures.

CONCLUSIONS

CRS fellows reached competency in colonoscopy at around 275 to 300 procedures of experience, a trajectory similar to previously reported data for gastroenterology fellows, suggesting little difference in the learning curves between these 2 groups. In addition, no trainee was deemed competent at the onset of training despite an average experience well over the 50 colonoscopies required during residency.

Peer evaluation and feedback for invasive medical procedures: a systematic review

BACKGROUND

There is significant variability in the performance and outcomes of invasive medical procedures such as percutaneous coronary intervention, endoscopy, and bronchoscopy. Peer evaluation is a common mechanism for assessment of clinician performance and care quality, and may be ideally suited for the evaluation of medical procedures. We therefore sought to perform a systematic review to identify and characterize peer evaluation tools for practicing clinicians, assess evidence supporting the validity of peer evaluation, and describe best practices of peer evaluation programs across multiple invasive medical procedures.

METHODS

A systematic search of Medline and Embase (through September 7, 2021) was conducted to identify studies of peer evaluation and feedback relating to procedures in the field of internal medicine and related subspecialties. The methodological quality of the studies was assessed. Data were extracted on peer evaluation methods, feedback structures, and the validity and reproducibility of peer evaluations, including inter-observer agreement and associations with other quality measures when available.

RESULTS

Of 2,135 retrieved references, 32 studies met inclusion criteria. Of these, 21 were from the field of gastroenterology, 5 from cardiology, 3 from pulmonology, and 3 from interventional radiology. Overall, 22 studies described the development or testing of peer scoring systems and 18 reported inter-observer agreement, which was good or excellent in all but 2 studies. Only 4 studies, all from gastroenterology, tested the association of scoring systems with other quality measures, and no studies tested the impact of peer evaluation on patient outcomes. Best practices included standardized scoring systems, prospective criteria for case selection, and collaborative and non-judgmental review.

CONCLUSIONS

Peer evaluation of invasive medical procedures is feasible and generally demonstrates good or excellent inter-observer agreement when performed with structured tools. Our review identifies common elements of successful interventions across specialties. However, there is limited evidence that peer-evaluated performance is linked to other quality measures or that feedback to clinicians improves patient care or outcomes. Additional research is needed to develop and test peer evaluation and feedback interventions.

Geographic distribution of colonoscopy providers in the United States: An analysis of medicare claims data

INTRODUCTION

Screening colonoscopy is one of the few procedures that can prevent cancer. While the majority of colonoscopies in the USA are performed by gastroenterologists, general surgeons play a key role in at-risk, rural populations. The aim of this study was to examine geographic practice patterns in colonoscopy using a nationwide Medicare claims database.

METHODS AND PROCEDURES

The 2017 Medicare Provider Utilization and Payment database was used to identify physicians performing colonoscopy. Providers were classified as gastroenterologists, surgeons, ambulatory surgical centers (ASCs), or other. Rural-Urban Commuting Area classification at the zip code level was used to determine whether the practice location for an individual provider was in a rural area/small town (< 10,000 people), micropolitan area (10-50,000 people), or metropolitan area (> 50,000 people).

RESULTS

Claims data from 3,861,187 colonoscopy procedures on Medicare patients were included. The majority of procedures were performed by gastroenterologists (57.2%) and ASCs (32.1%). Surgeons performed 6.8% of cases overall. When examined at a zip code level, surgeons performed 51.6% of procedures in small towns/rural areas and 21.7% of procedures in micropolitan areas. Individual surgeons performed fewer annual procedures as compared to gastroenterologists (median 51 vs. 187, p < 0.001).

CONCLUSIONS

Surgeons perform the majority of colonoscopies in rural zip codes on Medicare patients. High-quality, surgical training in endoscopy is essential to ensure access to colonoscopy for patients outside of major metropolitan areas.

Resident Endoscopy Experience Correlates Poorly with Performance on a Virtual Reality Simulator

Background  Endoscopy training has become increasingly emphasized during general surgery residency as reflected by introduction of the Fundamentals of Endoscopic Surgery (FES) examination, which includes testing of skills on virtual reality (VR) simulators. Although studies exist to assess the ability of the simulator to differentiate between novices and experienced endoscopists, it is not well understood how simulators can differentiate skills among resident cohort.

Objective  To assess the utility of the VR simulator, we evaluated the correlation between resident endoscopy experience and performance on two VR simulator colonoscopy modules on the GI-BRONCH Mentor (Simbionix Ltd, Airport City, Israel).

Methods  Postgraduate years 2 to 5 residents completed “easy” and “difficult” VR colonoscopies, and performance metrics were recorded from October 2017 to February 2018 at Rutgers' two general surgery residency programs. Resident endoscopy experience was obtained through Accreditation Council for Graduate Medical Education case logs. Correlations between resident endoscopy experience and VR colonoscopy performance metrics were assessed using Spearman's rho (ρ) correlation statistic and bivariate logistic regression.

Results  Fifty-five residents out of 65 (84.6%) eligible participants completed the study. There were limited correlations found between resident endoscopy experience and FES performance metrics and no correlations were found between resident endoscopy experience and binary metrics of colonoscopy—ability to complete colonoscopy, ability to retroflex, and withdrawal time of less than 6 minutes.

Conclusion  The VR simulator may have a limited ability to discriminate between experience levels among resident cohort. Future studies are needed to further understand how well the VR simulator metrics correlate with resident endoscopy experience.

JAG consensus statements for training and certification in oesophagogastroduodenoscopy

INTRODUCTION

Training and quality assurance in oesophagogastroduodenoscopy (OGD) is important to ensure competent practice. A national evidence-based review was undertaken to update and develop standards and recommendations for OGD training and certification.

METHODS

Under the oversight of the Joint Advisory Group (JAG), a modified Delphi process was conducted with stakeholder representation from British Society of Gastroenterology, Association of Upper Gastrointestinal Surgeons, trainees and trainers. Recommendations on OGD training and certification were formulated following literature review and appraised using Grading of Recommendations Assessment, Development and Evaluation. These were subjected to electronic voting to achieve consensus. Accepted statements were incorporated into the updated certification pathway.

RESULTS

In total, 32 recommendation statements were generated for the following domains: definition of competence (4 statements), acquisition of competence (12 statements), assessment of competence (10 statements) and post-certification support (6 statements). The consensus process led to following certification criteria: (1) performing ≥250 hands-on procedures; (2) attending a JAG-accredited basic skills course; (3) attainment of relevant minimal performance standards defined by British Society of Gastroenterology/Association of Upper Gastrointestinal Surgeons of Great Britain and Ireland, (4) achieving physically unassisted D2 intubation and J-manoeuvre in ≥95% of recent procedures, (5) satisfactory performance in formative and summative direct observation of procedural skills assessments.

CONCLUSION

The JAG standards for diagnostic OGD have been updated following evidence-based consensus. These standards are intended to support training, improve competency assessment to uphold standards of practice and provide support to the newly-independent practitioner.

Colonoscopy competence assessment tools: a systematic review of validity evidence

BACKGROUND

Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity.

METHODS

We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI).

RESULTS

From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5).

CONCLUSIONS

The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.

Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game

Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.

Use of saliva stress biomarkers to estimate novice male endoscopist's stress during training in a high-end simulator

OBJECTIVE

Simulated endoscopic training can be challenging and stressful for the novice trainee. The absence of a reliable stress detection method during simulated endoscopic training makes estimating trainees' mental stress difficult to quantify. This study concomitantly measures the responses of four saliva stress biomarkers and compares them to the video score (VS) achieved by novice endoscopists in a reproducibly stressful simulation environment.

METHODS

Thirty-six male endoscopy naïve surgery residents were enrolled. After an orientation phase, a saliva specimen was collected for cortisol (sC), alpha-amylase (sAA), Chromogranin A (sCgA), and immunoglobulin A (sIgA) measurements (baseline phase, BL). Thereafter, the simulation exercise phase (E) started, practicing in the Fundamentals of Endoscopic Surgery Skills module (GI-Bronch Mentor). Immediately after, a second saliva sample for measuring the above-cited biomarkers was collected. The whole experiment was videotaped, and the VS was calculated. The percentage (E-BL)_diff of each of the four saliva biomarkers was calculated and examined for correlation to VS.

RESULTS

sCgA_diff showed the best correlation with VS, followed by sAA_diff.

CONCLUSIONS

sCgA and sAA, are saliva stress biomarkers that are easy to collect non-invasively and showed the best correlation with novice endoscopist's performance in our simulation setting, and therefore, they could be used for monitoring stress.

Self-assessment of Competence in Endoscopy: Challenges and Insights

BACKGROUND

Endoscopists use self-assessment to monitor the development and maintenance of their skills. The accuracy of these self-assessments, which reflects how closely one's own rating corresponds to an external rating, is unclear.

METHODS

In this narrative review, we critically examine the current literature on self-assessment in gastrointestinal endoscopy with the aim of informing training and practice and identifying opportunities to improve the methodological rigor of future studies.

RESULTS

In the seven included studies, the evidence regarding self-assessment accuracy was mixed. When stratified by experience level, however, novice endoscopists were least accurate in their self-assessments and tended to overestimate their performance. Studies examining the utility of video-based interventions using observation of expert benchmark performances show promise as a mechanism to improve self-assessment accuracy among novices.

CONCLUSIONS

Based on the results of this review, we highlight problematic areas, identify opportunities to improve the methodological rigor of future studies on endoscopic self-assessment and outline potential avenues for further exploration.

GAPS-EUS: a new and reliable tool for the assessment of basic skills and performance in EUS among endosonography trainees

Objective

Endosonography (EUS) is a useful but complex diagnostic modality which requires advanced endoscopy training and guidance by a supervisor. Since learning curves vary among individuals, assessment of the actual competence among EUS trainees is important.

Design/methods

We designed a novel assessment tool entitled Global Assessment of Performance and Skills in EUS (GAPS-EUS) for assessing skills among EUS trainees. Five quality indicators were marked on a five-grade scale by the supervisor (Observer Score) and by the trainee (Trainee Score). Trainees were included in two high-volume centres (Gothenburg, Sweden, and Bologna, Italy). Outcomes were feasibility, patient safety, reliability, and validity of GAPS-EUS in trainee-performed EUS procedures.

Results

Twenty-two EUS-trainees were assessed in a total of 157 EUS procedures with a completion rate of 157/157 (100 %) and a patient adverse event rate of 2/157 (1.3 %; gastroenteritis n=1, fever n=1). GAPS-EUS showed a high measurement reliability (Cronbach’s alpha coefficient=0.87) and a high inter-rater reliability comparing the supervisor and the trainee (r=0.83, r²=0.69, p<0.001). The construct of GAPS-EUS was verified by comparing low-level and high-level performance procedures and the content validity by recording that the EUS-FNA manoeuvre resulted in a lower score than other aspects of EUS 3.07 (95% CI 2.91 to 3.23) vs 3.51 (95% CI 3.37 to 3.65) (p<0.001). External validity was confirmed via similar findings in both centres.

Conclusion

GAPS-EUS is an easy-to-use and reliable tool with a recorded high validity for the assessment of competence among trainees in EUS. It can be recommended to centres involved in the education of future endosonographers.

Trial registration number

NCT02455570.

Early implementation of Fundamentals of Endoscopic Surgery training using a simulation-based mastery learning curriculum

BACKGROUND

After the American Board of Surgery announcement of the Flexible Endoscopy Curriculum requirement in 2014, we implemented a dedicated endoscopy rotation at the post graduate year (PGY)2 level including a simulation curriculum for Fundamentals of Endoscopic Surgery skills. Here we evaluate the outcomes of this implementation.

METHODS

Beginning in 2015, we developed a clinical endoscopy and simulation-based rotation to prepare for Fundamentals of Endoscopic Surgery testing. Originally, our curriculum was based on the published Texas Association of Surgical Skills Laboratories curriculum using the GI Mentor and transitioned to a mastery learning curriculum using the Endoscopy Training System in 2016. We evaluated the success of the curriculum in terms of first-time pass rates, training time required, and comparison to previously published benchmarks based on clinical experience.

RESULTS

Since 2015, a total of 37 general surgery residents in our program were Fundamentals of Endoscopic Surgery tested (PGY2 = 24, PGY3 = 4, PGY5 = 9); 84% (31) completed the Endoscopy Training System curriculum. At the time of testing, 73% (27) had performed <25 esophagogastroduodenoscopies, and 46% had performed <25 colonoscopies. Ninety-two percent (34) spent 10 hours or less completing the curriculum. The first-time pass rate for those completing the Endoscopy Training System curriculum was 97% vs 67% for those not completing the Endoscopy Training System curriculum (P = .01). For residents completing the Endoscopy Training System curriculum, total Fundamentals of Endoscopic Surgery scores were discernibly higher (472 vs 389, P < .01), as were 3/5 task scores (Nav1 80 vs 67, P = .02; Loop2 36 vs 8, P = .02; Retro3 89 vs 71, P = .02). Despite clinical inexperience (<25 esophagogastroduodenoscopies and <50 colonoscopies), PGY2s yielded a mean score of 454 and a pass rate of 92%. This was similar to PGY5s (427, 89%; P = .3) and compares to benchmark data of endoscopists with >300 cases.

CONCLUSION

Early implementation of flexible endoscopy training with a simulation-based curriculum results in Fundamentals of Endoscopic Surgery performance equal to a clinical experience not often gained during surgical residency. Often requiring <10 hours, this represents a fantastic return on investment for this training.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Development and prospective validation of a scoring system for the Basic Endoscopic Skills Training (BEST) box

AIMS

The increased use of endoscopy as a minimally invasive therapeutic technique has created a great demand for endoscopic training. The Basic Endoscopic Skills Training (BEST) box provides a low-cost solution by adapting the Fundamentals of Laparoscopic Surgery (FLS) box for flexible endoscopic simulation. The BEST box consists of six endoscopic tasks with a 5-min time limit per task. This study aims to develop a scoring system for objective evaluation of user performance.

METHODS

A total of 165 participants were tested on the BEST box. Participants were divided into two groups: retrospective analysis (n = 100) and prospective analysis (n = 65). From the retrospective group, 55 individuals were also scored on the Global Assessment of Gastrointestinal Endoscopic Skills-Upper Endoscopy (GAGES-UE). Linear regression between user performance on BEST box and GAGES-UE was performed to develop the scoring system. Receiver Operating Characteristic curve was used to determine a threshold score to help users appreciate their endoscopic expertise. Prospective scoring of 65 individuals was then performed using the formula developed (20 experts and 45 trainees).

RESULTS

The minimum and maximum possible scores are 30 and 110, respectively. Retrospective analysis showed that the scoring system was able to distinguish between experts and trainees (p < 0.001), correlated with GAGES-UE (p < 0.001), and had a reliability constant of r = 0.765 (p < 0.001). On prospective testing using the scoring system the expert group received a final average score of 92, whereas the average score for the trainee group was 61 (p < 0.001).

CONCLUSIONS

The developed BEST box scoring system correlates with the experience level of the test taker as well as with the GAGES-UE scoring system. The results of this study add further evidence to the validity of the BEST box as an effective, low-cost endoscopic simulator with the scores used by trainees to track their performance level overtime.

Global Assessment of Urological Endoscopic Skills (GAUES): development and validation of a novel assessment tool to evaluate endourological skills

OBJECTIVE

To develop and evaluate an assessment tool for endourological skills during simulation including cystoscopy, ureteroscopy (URS) and transurethral resection (TUR) procedures.

METHODS

We designed a Global Assessment of Urological Endoscopic Skills (GAUES) tool, comprised of nine endourology task-specific and two global-rating skills items. The tool was developed through two rounds of the Delphi process. The GAUES tool was used to assess acquisition of URS and TUR skills of novices (Year 2 core surgical trainees, CT2) and intermediate level trainees (residents at the start of the UK higher surgical training programme in Urology, Speciality Trainee Year 3, ST3) at the Urology Simulation Boot Camp (USBC) between 2016 and 2018. Validity was evaluated by comparing scores between trainees with different levels of urological experience. Inter-rater reliability was also assessed.

RESULTS

We evaluated 130 residents, 52% of trainees were at an intermediate stage of training and 39% were novices. In all, 9% of the anonymous forms were missing demographics. The completion rate of the GAUES tool during the USBC for URS and TUR was 85% and 89%, respectively. Our analysis demonstrated a significant difference in all domains between intermediates and novices at assessment in URS, except for one domain more suited to clinical assessment (P = 0.226). There was excellent intraclass correlation (ICC) overall between the two experts' judgements, ICC = 0.841 (95% confidence interval 0.767-0.893; P < 0.001, n = 88).

CONCLUSIONS

We have developed the novel GAUES tool for cystoscopic, URS and TUR skills. Overall, we demonstrated good face, content and construct validity and excellent reliability, suggesting that the GAUES tool can be useful for endourological skills assessment.

Teaching peroral endoscopic pyloromyotomy (POP) to practicing endoscopists: An "into-the-fire" approach to simulation

BACKGROUND

Peroral endoscopic pyloromyotomy, or gastric peroral endoscopic myotomy, is a novel endoscopic procedure for the treatment of refractory gastroparesis. We present a simulation-based curriculum using an "into-the-fire" approach with hands-on pre- and post-tests to teach this procedure.

METHODS

Six, 1-day peroral endoscopic pyloromyotomy courses were taught by an expert surgical endoscopist in 2018 to 2019. The curriculum is composed of a series of pretraining tests, lectures, mentored hands-on instruction, and post-training tests using porcine models. Both pre- and post-testing included a confidence survey, a knowledge-based written test, and a specific assessment form for the peroral endoscopic pyloromyotomy procedure. Participants' scores were analyzed using paired t tests.

RESULTS

Twenty-eight practicing physicians participated. After completing the curriculum, participants had improved confidence scores (10.5 ± 5.2 vs 19.4 ± 3.6; P < .001), written test scores (6.8 ± 1.6 vs 8.0 ± 1.1; P < .001), and hands-on performance scores (23.6 ± 3.4 vs 29.3 ± 1.4; P < .001) with marked improvement in all components of the peroral endoscopic pyloromyotomy procedure. Postcourse surveys showed 93% of participants had performed or intended to perform the peroral endoscopic pyloromyotomy procedure within the next year.

CONCLUSION

Our simulation curriculum with an into-the-fire approach to teach peroral endoscopic pyloromyotomy is effective in improving practitioner knowledge, confidence, and technical skills, leading to an increase in the adoption of this procedure.

A curriculum to democratize and standardize flexible endoscopy fundamental knowledge and skills: a critical review of the first 5 years of a surgical endoscopy university diploma

BACKGROUND

As flexible endoscopy offers many advantages to patients, access to training should be aggressively encouraged. In 2014, the IRCAD-IHU-Strasbourg launched a year-long university diploma using advanced education methods to offer surgeons and gastroenterologists high-quality, personalized training in flexible endoscopy. This paper describes and critically reviews the first 5 years of the University Diploma in Surgical Endoscopy (UDSE).

METHODS

The UDSE aims to progressively transmit theoretical knowledge, clinical judgment, and practical skills on basic and advanced flexible endoscopy. The 300-h year-long curriculum is composed of 100 h of online lectures with tests, 150 h of clinical rotations and 50 h of hands-on sessions. The hands-on training is delivered through validated mechanical simulators, virtual reality simulators, and specifically designed ex vivo and in vivo animal models. Participants' demographics, training, and clinical experience were recorded. Trainees' evaluations of each online lecture, hands-on training, and clinical rotations were assessed using a Likert scale from 1 (not satisfactory) to 5 (outstanding). Trainees' skill progression was evaluated using the Global Assessment of Gastrointestinal Endoscopic Skills (GAGES) proficiency test. Finally, clinical uptake was surveyed.

RESULTS

162 (79.01% males) trainees from 38 countries enrolled and successfully completed the first 5 courses. The vast majority of the trainees were surgeons and 19.14% were gastroenterologist. Sixty-nine (42.59%) participants were residents and 97 (56.79%) had no prior experience in flexible endoscopy. The online lectures, on-site sessions, and clinical rotations were highly appreciated receiving an overall average score of 4.33/5, 4.56/5, 4.43/5, respectively. Trainees' endoscopic skills improved significantly (16.68 vs. 20.53 GAGES scores; p = 0.016). At an average of 18.83 months following the course, 31 alumni (77.50% of repliers) started to use a flexible endoscope in their practice.

CONCLUSIONS

Over its 5-year evolution, the UDSE has proven to be a valid means to ease access to the fundamental knowledge, practical skills, and clinical judgment necessary to achieve proficiency in surgical endoscopy.

The SAFE-T upper endoscopy tool: a web-based application for the point-of-care evaluation of gastroenterology fellow performance in upper endoscopy

Background

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise, web-based assessment tool to evaluate real-time fellow performance in upper endoscopy.

Methods

We developed the Skill Assessment in Fellow Endoscopy Training (SAFE-T) upper endoscopy tool to capture both summative and formative feedback in a concise, five-part questionnaire. The tool mirrors the previously validated SAFE-T colonoscopy tool and is administered electronically via a web-based application. We evaluated the tool in a prospective study of 15 gastroenterology fellows (5 fellows each from Years 1–3 of training) over the 2018–2019 academic year. An independent reviewer evaluated a subset of these procedures and completed both the SAFE-T and Assessment of Competency in Endoscopy (ACE) upper endoscopy forms for reliability testing.

Results

Twenty faculty completed 413 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score differentiated each sequential fellow year of training, with first-year cases having lower performance than second-year cases (3.31 vs 4.25, P < 0.001) and second-year cases having lower performance than third-year cases (4.25 vs 4.56, P < 0.001). The mean SAFE-T overall score decreased with increasing case-complexity score, with straightforward compared with average cases (3.98 vs 3.39, P < 0.001) and average compared with challenging cases (3.39 vs 2.84, P = 0.042). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a Kappa agreement statistic of 0.815 (P = 0.001). The SAFE-T overall score also highly correlated with the ACE upper endoscopy overall hands-on score (r = 0.76, P = 0.011).

Conclusions

We developed and validated the SAFE-T upper endoscopy tool—a concise and web-based means of assessing real-time gastroenterology fellow performance in upper endoscopy.

Development of Simulator Guidelines for Resident Assessment in Flexible Endoscopy

Virtual reality (VR) simulators may hold a role in the assessment of trainee abilities independent of their role as instructional instruments. Thus, we piloted a course in flexible endoscopy to surgical trainees who had met Accreditation Council for Graduate Medical Education endoscopy requirements to establish the relationship between metrics produced by a VR endoscopic simulator and trainee ability. After a didactic session, we provided faculty instruction to senior residents for Case 1 upper endoscopy and colonoscopy modules on the CAE Endoscopy VR. Course conclusion was defined as a trainee meeting all proficiency standards in basic endoscopic procedures on the simulator. Simulator metrics and course evaluation comprised data. Eleven and eight residents participated in the colonoscopy and upper endoscopy courses, respectively. Average time to reach proficiency standards for esophagogastroduodenoscopy was 6 and 13 minutes for colonoscopy after a median of one (range, one to two) and one (range, one to four) task repetitions, respectively. Faculty instruction averaged 7.5 minutes of instruction per repetition. A subjective course evaluation demonstrated that the course improved learners’ knowledge of the subject and comfort with endoscopic equipment. Within a VR-based curriculum, experienced residents rapidly achieved task proficiency. The resultant scores may be used as simulator guidelines for resident assessment and readiness to perform flexible endoscopy.

Development and validation of an endoscopic submucosal dissection video assessment tool

BACKGROUND

Despite a need for assessment of endoscopic submucosal dissection (ESD) skills in order to track progress and determine competence, there is no structured measure of assessing competency in ESD performance. The present study aims to develop and examine validity evidence for an assessment tool to evaluate the recorded performance of ESD for gastric neoplasms.

METHODS

The ESD video assessment tool (EVAT) was systematically developed by ESD experienced endoscopists. The EVAT consists of a 25-item global rating scale and 3-item checklist to assess competencies required to perform ESD. Five unedited videos were each evaluated by 2-blinded experienced ESD endoscopists to assess inter-rater reliability using intraclass correlation coefficients (ICC). Seventeen unedited videos in total were rated by 3 blinded experienced ESD endoscopists. Validity evidence for relationship to other variables was examined by comparing scores of inexperienced (fellows) and experienced endoscopists (attending staff), and by evaluating the relationship between the EVAT scores and ESD case experience. Internal consistency was evaluated using Cronbach's alpha.

RESULTS

The inter-rater reliability for the total score was high at 0.87 (95% confidence interval 0.11 to 0.99). The total score [median, interquartile range (IQR)] was significantly different between the inexperienced (71, 63-77) and experienced group (95, 91-97) (P = 0.005). The total scores demonstrated high correlation with the number of ESD cases (Spearman's ρ = 0.79, P < 0.01). The internal consistency was 0.97.

CONCLUSIONS

This study provides preliminary validity evidence for the assessment of video-recorded ESD performances for gastric neoplasms using EVAT.

Using computerized assessment in simulated colonoscopy: a validation study

Background and study aims  Patient safety during a colonoscopy highly depends on endoscopist competence. Endoscopic societies have been calling for an objective and regular assessment of the endoscopists, but existing assessment tools are time-consuming and prone to bias. We aimed to develop and gather evidence of validity for a computerized assessment tool delivering automatic and unbiased assessment of colonoscopy based on 3 dimensional coordinates from the colonoscope. Methods  Twenty-four participants were recruited and divided into two groups based on experience: 12 experienced and 12 novices. Participants performed twice on a physical phantom model with a standardized alpha loop in the sigmoid colon. Data was gathered directly from the Olympus ScopeGuide system providing XYZ-coordinates along the length of the colonoscope. Five different motor skill measures were developed based on the data, named: Travel Length, Tip Progression, Chase Efficiency, Shaft movement without tip progression, and Looping. Results  The experinced had a lower travel length ( P  < 0.001), tip progression ( P  < 0.001), chase efficiency ( P  = 0.001) and looping ( P  = 0.006), and a higher shaft movement without tip progression ( P  < 0.001) reaching the cecum compared with the novices. A composite score was developed based on the five measurements to create a combined score of progression, the 3D-Colonoscopy-Progression-Score (3D-CoPS). The 3D-CoPS revealed a significant difference between groups (experienced: 0.495 (SD 0.303) and novices -0.454 (SD 0.707), P  < 0.001). Conclusion  This study presents a novel, real-time computerized assessment tool for colonoscopy, and strong evidence of validity was gathered in a simulation-based setting. The system shows promising opportunities for automatic, unbiased and continuous assessment of colonoscopy performance.

Self-Assessment of Surgical Skills: A Systematic Review

OBJECTIVE

Self-assessment is fundamental in surgical training to enhance learning in the absence of trainer feedback. The primary objective of this review was to assess the factors that influence accuracy of self-assessment at technical skills across all surgical specialties. The secondary objective was to assess whether there are any innate factors or attributes to predict those that will carry out effective self-assessment.

DESIGN

A systematic review was carried out in accordance with PRISMA guidelines. A search strategy encompassing MEDLINE, EMBASE, ERIC, WHO, and the Cochrane database was conducted to identify studies investigating self-assessment at any surgical task. Quality was assessed using the Newcastle-Ottawa scale. A summary table was created to describe specialty, participants, task, setting, assessment tool, and correlation coefficient between self and expert assessment. The review protocol was registered in PROSPERO.

RESULTS

Of 24,638 citations, 40 met inclusion criteria. In total 1753 participants performed 68 procedures. Twenty-six studies investigated skills in general surgery with the remaining 14 in various other surgical specialties. Accuracy of self-assessment is superior in those with greater experience and age, and with use of retrospective video playback. Accuracy tends to be reflected by overestimation of performance. Stressful environments reduce accuracy. There is limited evidence in the literature regarding predicting traits for those who will carry out accurate self-assessment.

CONCLUSIONS

The ability to perform accurate self-assessment is an important skill in surgical training, with accuracy being influenced by a multitude of factors. The use of self-assessment from retrospective video playback may be of benefit in surgical training curricula to enhance learning of technical skills. Further studies are required to define predictors of good self-assessment, which will strengthen recruitment and mentoring to assist trainee learning.

Design, implementation, and evaluation of a novel curriculum to teach transurethral resection of the prostate (TURP): a 3-year experience of urology simulation bootcamp course

OBJECTIVES

To present the three-year experience of the multi-component TURP module at Urology Simulation Bootcamp Course (USBC) and demonstrate trainee's competence progression and satisfaction.

METHODS

During the USBC, a 4-h TURP module was developed and consisted of (a) familiarisation and assembly of resectoscope instrument, (b) didactic lecture on TURP operative techniques and postoperative complications, (c) learning hands-on resection on validated simulators [Samed, GmBH, Dresden, Germany; TURP Mentor™, Simbionix, Israel], and (d) practicing clot evacuation using the Ellik bladder Evacuator. Trainee's level of instrument knowledge, operative competence, and confidence were assessed pre- and post-course. Trainee's feedback was also collected.

RESULTS

One hundred thirty trainees participated in the USBC between 2016 and 2018. Eighty-seven percent of trainees scored themselves as 1-3 (low confidence in resection) on a 5-point Likert scale. Trainees significantly improved in their ability to perform resectoscope assembly for resection, coagulation and incision by 33.6% (p < 0.001), 28.1% (p < 0.001) and 34.0% (p < 0.001), respectively. There was a significant improvement in scores in itemised technical skill on the TURP simulator following completion of the TURP module (Mean difference = 3.4 points, 95% CI 2-4, p < 0.001). Ninety-one percent of trainees agreed that the TURP module was useful for their development in urological training.

CONCLUSION

Our results demonstrated that it is feasible to develop and implement a focussed module for teaching TURP with significant improvement in learning. Trainee feedback suggests that they were highly satisfied with the teaching provided and models used. This style of training can be implemented for other common surgical procedures.

Assessing the Quality of Polypectomy and Teaching Polypectomy

Ineffective polypectomy technique may lead to incomplete polyp resection, high complication rates, interval colorectal cancer, and costly referral to surgery. Despite its central importance to endoscopy, training in polypectomy is not standardized nor has the most effective training approach been defined. Polypectomy competence is rarely reported and quality metrics for this skill are lacking. Use of tools and measurements to assess polypectomy outcomes is low. There is a need for standardization of training and remediation in polypectomy; defining standards of competent polypectomy and how it is feasibly measured; and integration of polypectomy quality metrics into training programs and the accreditation process.

Endoscopic simulators

BACKGROUND AND AIMS

Simulation refers to educational tools that allow for repetitive instruction in a nonpatient care environment that is risk-free. In GI endoscopy, simulators include ex vivo animal tissue models, live animal models, mechanical models, and virtual reality (VR) computer simulators.

METHODS

After a structured search of the peer-reviewed medical literature, this document reviews commercially available GI endoscopy simulation systems and clinical outcomes of simulation in endoscopy.

RESULTS

Mechanical simulators and VR simulators are frequently used early in training, whereas ex vivo and in vivo animal models are more commonly used for advanced endoscopy training. Multiple studies and systematic reviews show that simulation-based training appears to provide novice endoscopists with some advantage over untrained peers with regard to endpoints such as independent procedure completion and performance time, among others. Data also suggest that simulation training may accelerate the acquisition of specific technical skills in colonoscopy and upper endoscopy early in training. However, the available literature suggests that the benefits of simulator training appear to attenuate and cease after a finite period. Further studies are needed to determine if meeting competency metrics using simulation will predict actual clinical competency.

CONCLUSIONS

Simulation training is a promising modality that may aid in endoscopic education. However, for widespread incorporation of simulators into gastroenterology training programs to occur, simulators must show a sustained advantage over traditional mentored teaching in a cost-effective manner. Because most studies evaluating simulation have focused on novice learners, the role of simulation training in helping practicing endoscopists gain proficiency using new techniques and devices should be further explored.

ASGE EndoVators Summit: simulators and the future of endoscopic training

Interest in the use of simulation for acquiring, maintaining, and assessing skills in GI endoscopy has grown over the past decade, as evidenced by recent American Society for Gastrointestinal Endoscopy (ASGE) guidelines encouraging the use of endoscopy simulation training and its incorporation into training standards by a key accreditation organization. An EndoVators Summit, partially supported by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) of the National Institutes of Health, (NIH) was held at the ASGE Institute for Training and Technology from November 19 to 20, 2017. The summit brought together over 70 thought leaders in simulation research and simulator development and key decision makers from industry. Proceedings opened with a historical review of the role of simulation in medicine and an outline of priority areas related to the emerging role of simulation training within medicine broadly. Subsequent sessions addressed the summit's purposes: to review the current state of endoscopy simulation and the role it could play in endoscopic training, to define the role and value of simulators in the future of endoscopic training and to reach consensus regarding priority areas for simulation-related education and research and simulator development. This white paper provides an overview of the central points raised by presenters, synthesizes the discussions on the key issues under consideration, and outlines actionable items and/or areas of consensus reached by summit participants and society leadership pertinent to each session. The goal was to provide a working roadmap for the developers of simulators, the investigators who strive to define the optimal use of endoscopy-related simulation and assess its impact on educational outcomes and health care quality, and the educators who seek to enhance integration of simulation into training and practice.

Development and validation of a video-based cold snare polypectomy assessment tool (with videos)

BACKGROUND AND AIMS

Polyps <1 cm are the most commonly found polyps during colonoscopy. Cold snare removal is preferred given the significantly higher rate of incomplete resection associated with piecemeal biopsy forceps resection. There are currently no standardized tools to assess competence in cold snare polypectomy. This study aimed to develop and validate a cold snare polypectomy assessment tool (CSPAT).

METHODS

Experts in cold snare polypectomy used a Delphi method to develop the CSPAT. Metrics with a greater than 85% agreement as being "important" or "very important" were included in the CSPAT. The tool included evaluation of polyp inspection, positioning, appropriate ensnarement of tissue to ensure a rim of normal tissue, tissue retrieval, and postpolypectomy site inspection. Experts in cold snare polypectomy used the CSPAT to evaluate preselected videos that were previously evaluated using the Direct Observation of Polypectomy Skills (DOPyS) tool. Interobserver agreement was evaluated. CSPAT scores were compared with DOPyS scores to assess content validity.

RESULTS

Sixteen experts developed the 12-item CSPAT, and 13 experts reviewed 55 videos. There was a moderate degree of agreement in 10 metrics (κ = .52-.59) and a substantial degree of agreement (κ = .61-.63) in the other 2. There was a strong correlation between the mean of individual metrics 1 to 12 compared with the global competence assessment (ρ = .88, P < .001). There was a moderate correlation between the average overall DOPyS score and the overall CSPAT competence score (ρ = .56, P < .001).

CONCLUSIONS

We have developed and validated a customized CSPAT that can be easily applied to video-based assessments to assess competence in training and among practicing endoscopists.

Development of a fundamentals of endoscopic surgery proficiency-based skills curriculum for general surgery residents

BACKGROUND

Fundamentals of Endoscopic Surgery (FES) has become a board certification requirement for general surgery residents. While the FES program provides a robust didactic curriculum, an endoscopic skills curriculum is lacking for this high stakes assessment. The aims of this study were to develop a proficiency-based endoscopic skills curriculum and assess its effectiveness on success in the FES exam.

METHODS

Endoscopy experts developed a multiple-choice questionnaire based on the FES online didactics. Five training cases from the GI Mentor II simulator were selected, and expert performance defined proficiency levels for each case. Participating surgery residents were required to review online didactics and achieve proficiency twice on selected simulator cases. The multiple-choice questionnaire, simulator-generated metrics of two endoscopy cases, Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), NASA-Task Load Index (TLX), and the manual portion of the FES exam were used for assessment before and after training. The curriculum was implemented either alongside a clinical endoscopy rotation or independent of the rotation. Clinical endoscopic skills of participants with a dedicated rotation were assessed using GAGES.

RESULTS

Twenty-eight general surgery residents (PGY 2-5) participated in the study, of which 25 (89%) completed the curriculum. Scores of the multiple-choice questionnaire and all simulator-generated metrics improved in the post-training assessment, with the exception of the percentage of mucosal surface examined, which was slightly decreased. Simulated and clinical GAGES scores and the NASA-TLX score improved after training. Performance scores on four of five FES exam tasks were significantly improved.

CONCLUSIONS

The proficiency-based endoscopic skills curriculum was successfully implemented both alongside the clinical endoscopy rotation and independent of the rotation. Participating residents acquired skills to pass the FES exam. This curriculum will be valuable to general surgery residency programs.

Direct observation of procedural skills (DOPS) assessment in diagnostic gastroscopy: nationwide evidence of validity and competency development during training

Background

Validated competency assessment tools and the data supporting milestone development during gastroscopy training are lacking. We aimed to assess the validity of the formative direct observation of procedural skills (DOPS) assessment tool in diagnostic gastroscopy and study competency development using DOPS.

Methods

This was a prospective multicentre (N = 275) analysis of formative gastroscopy DOPS assessments. Internal structure validity was tested using exploratory factor analysis and reliability estimated using generalisability theory. Item and global DOPS scores were stratified by lifetime procedure count to define learning curves, using a threshold determined from receiver operator characteristics (ROC) analysis. Multivariable binary logistic regression analysis was performed to identify independent predictors of DOPS competence.

Results

In total, 10086 DOPS were submitted for 987 trainees. Exploratory factor analysis identified three distinct item groupings, representing ‘pre-procedure’, ‘technical’, and ‘post-procedure non-technical’ skills. From generalisability analyses, sources of variance in overall DOPS scores included trainee ability (31%), assessor stringency (8%), assessor subjectivity (18%), and trainee case-to-case variation (43%). The combination of three assessments from three assessors was sufficient to achieve the reliability threshold of 0.70. On ROC analysis, a mean score of 3.9 provided optimal sensitivity and specificity for determining competency. This threshold was attained in the order of ‘pre-procedure’ (100–124 procedures), ‘technical’ (150–174 procedures), ‘post-procedure non-technical’ skills (200–224 procedures), and global competency (225–249 procedures). Higher lifetime procedure count, DOPS count, surgical trainees and assessors, higher trainee seniority, and lower case difficulty were significant multivariable predictors of DOPS competence.

Conclusion

This study establishes milestones for competency acquisition during gastroscopy training and provides validity and reliability evidence to support gastroscopy DOPS as a competency assessment tool.

Electronic supplementary material

The online version of this article (10.1007/s00464-019-06737-7) contains supplementary material, which is available to authorised users.

Improvement of arthroscopic surgical performance using a new wide-angle arthroscope in the surgical training

Background

We have developed a new arthroscope with a field of view of 150°. This arthroscope requires less motion to maneuver and exhibits reduced optical error. It also improves how novices learn arthroscopy. We hypothesized that the surgical performance with this arthroscope is superior to that with a conventional arthroscope. This study tested the hypothesis by using motion analysis and a new validated parameter, “dimensionless squared jerk” (DSJ).

Methods

We compared the surgical performance between the use of the wide-angle arthroscope and that of the conventional arthroscope among 14 novice orthopedic residents who performed 3 standardized tasks 3 times with each arthroscope. The tasks simulated the surgical skills in arthroscopic rotator cuff repair. The arthroscope motion was analyzed using an optical tracking system. The differences in performance parameters, such as the time taken to complete the tasks, average acceleration of the hands (m/s²), number of movements, and total path length (m) including DSJ between the 2 arthroscopes were investigated using paired t-tests.

Results

All estimated values for the tasks using the 150° arthroscope were lower than those for the tasks using the 105° arthroscope. Statistically significant differences in performance between the 2 arthroscopes were observed only for DSJ (p = 0.014) and average acceleration (p = 0.039).

Conclusions

DSJ and average acceleration are reliable parameters for representing hand-eye coordination. The surgical performance of novice arthroscopists was better with the new wide-angle arthroscope than with the conventional arthroscope.