Combining different methods improves assessment of competence in colonoscopy

Colonoscope retraction technique and predicting adenoma detection rate: a multicenter study

BACKGROUND AND AIMS

The success of preventing colorectal cancer relies on the expertise of the colonoscopists. Studies suggest that the retraction technique is a powerful indicator of expertise in distinguishing endoscopists with various adenoma detection rates (ADRs). We aimed to develop a retraction technique score and explore the correlation between endoscopists' retraction technique and their ADRs.

METHODS

In a prospective, multicenter study, 8 colonoscopist nurses and physicians with various ADRs were included. Data from patients admitted for a colonoscopy, as part of the Danish nationwide screening program, were gathered directly from the Olympus ScopeGuide system (UPD-3; Olympus Optical, Tokyo, Japan) providing XYZ-coordinates from the coils along the length of the colonoscope. Motor skill measures were developed based on tip retraction, retraction efficiency, and retraction distance. The principal component analysis was used to study the association among the 3 measures and the historical ADR to create a combined score, the colonoscopy retraction score (CoRS).

RESULTS

Three hundred thirty-three recordings were analyzed. We demonstrated a significant and strong correlation between CoRS and ADR (.90, P < .01). Conversely, withdrawal time did not correlate significantly with ADR (.33, P = .42). In procedures without polypectomies or biopsy sampling, a significant and strong correlation was found between CoRS and ADR (.88, P < .01) and between withdrawal time and ADR (.75, P = .03).

CONCLUSIONS

This study presents a novel, real-time computerized and unbiased assessment tool for colonoscopy withdrawal. CoRS strongly correlated with ADR with and without therapeutic interventions during withdrawal and could be used to ensure quality instead of minimal withdrawal time. (Clinical trial registration number: NCT03587935.).

Expertise in colonoscopy intubation does not predict diagnostic accuracy: a simulation-based study

Background and study aims  Studies have linked cecal intubation rate with adenoma detection rate; however, the direct association between technical performance during colonoscopy intubation and withdrawal has never been explored. Thus, it remains unclear whether gentle and efficient intubation predicts superior mucosal inspection. The aim of this study was to investigate the correlation between performance during intubation and withdrawal in a simulation-based setup. Methods  Twenty-four physicians with various experience in colonoscopy performed twice on the Endoscopy Training System (ETS). Intubation skills were evaluated by assessing tasks on the ETS related to intubation (scope manipulation and loop management) and use of a computerized assessment tool called the 3D-Colonoscopy Progression Score (3D-CoPS). Diagnostic accuracy was defined by the number of polyps found during the ETS task of mucosal inspection. Pearson's correlations were calculated to explore associations between intubation skill and diagnostic accuracy. Results  The correlation analysis between 3D-CoPS and number of polyps found during mucosal inspection revealed a weak and insignificant correlation (0.157, P  = 0.3). Likewise, an insignificant correlation was seen between ETS intubation and number of polyps found (0.149, P  = 0.32). Conclusions  We found no evidence to support that technical performance during intubation is correlated with mucosal inspection performance in a simulation-based setting.

Colonoscopy competence assessment tools: a systematic review of validity evidence

BACKGROUND

Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity.

METHODS

We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI).

RESULTS

From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5).

CONCLUSIONS

The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.

Automatic and Objective Assessment of Motor Skills Performance in Flexible Bronchoscopy

BACKGROUND

Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists.

OBJECTIVES

The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion.

METHODS

Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS).

RESULTS

Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson's correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson's correlation, r = -0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups' method, a passing score of -0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures.

CONCLUSIONS

MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

Surgical simulation: Current practices and future perspectives for technical skills training

Simulation-based training (SBT) has become a standard component of modern surgical education, yet successful implementation of evidence-based training programs remains challenging. In this narrative review, we use Kern's framework for curriculum development to describe where we are now and what lies ahead for SBT within surgery with a focus on technical skills in operative procedures. Despite principles for optimal SBT (proficiency-based, distributed, and deliberate practice) having been identified, massed training with fixed time intervals or a fixed number of repetitions is still being extensively used, and simulators are generally underutilized. SBT should be part of surgical training curricula, including theoretical, technical, and non-technical skills, and be based on relevant needs assessments. Furthermore, training should follow evidence-based theoretical principles for optimal training, and the effect of training needs to be evaluated using relevant outcomes. There is a larger, still unrealized potential of surgical SBT, which may be realized in the near future as simulator technologies evolve, more evidence-based training programs are implemented, and cost-effectiveness and impact on patient safety is clearly demonstrated.

Methods for certification in colonoscopy - a systematic review

INTRODUCTION

Reliable, valid, and feasible assessment tools are essential to ensure competence in colonoscopy. This study aims to provide an overview of the existing assessment methods and the validity evidence that supports them.

METHODS

A systematic search was conducted in October 2016. Pubmed, EMBASE, and PsycINFO were searched for studies evaluating assessment methods to ensure competency in colonoscopy. Outcome variables were described and evidence of validity was explored using a contemporary framework.

RESULTS

Twenty-five observational studies were included in the systematic review. Most studies were based on small sample sizes. The studies were categorized after outcome measures into five groups: Clinical process related outcome metrics (n = 2), direct observational colonoscopy assessment (n = 8), simulator based metrics (n = 11), automatic computerized metrics (n = 2), and self-assessment (n = 1). Validity score varied among the studies and only five studies presented sufficient evidence to recommend the tool for clinical assessment.

CONCLUSIONS

The objectives vary throughout the presented tools. Some tools are global tools where others focus on procedural technical skill assessment or even part-task skills. There is a tendency in the most recent studies towards more specific assessment of technical skills. The majority of assessment methods lack sufficient validity evidence.