Validity evidence for observational ERCP competency assessment tools: a systematic review

Fast-tracking ERCP learning with the Boškoski-Costamagna Trainer: results of a multicenter randomized clinical trial

BACKGROUND

Achieving competence in endoscopic retrograde cholangiopancreatography (ERCP) requires extensive training. Recognizing the potential of simulator-based education for safe and effective skill development, we aimed to assess whether initial training with the Boškoski-Costamagna ERCP Trainer (BCT) is beneficial compared with conventional training alone (i.e. predictive validity).

METHODS

A prospective multicenter randomized trial involving 16 novice ERCP trainees, randomly assigned to intervention or control groups, was performed. Both underwent hands-on training, with the intervention group receiving additional simulation training during the first 3 months. Each trainee was required to complete a minimum of 60 ERCPs in up to 1 year. The TEESAT score evaluated rates of global overall competence (primary outcome), biliary cannulation, and adverse events (AEs; secondary outcomes). Mixed-effect logistic regression models assessed differences in between-group ERCP procedure competence and success. Learning curves were generated cumulatively over the training period.

RESULTS

1106 ERCPs (562 simulator group; 544 control group) were included. Although no statistically significant difference in global overall competence was observed between the groups, possibly owing to data heterogeneity, simulation training demonstrated higher success for native biliary cannulation (52% vs. 42%; P<0.001) and faster median (interquartile range) biliary cannulation times (3 [6] vs. 5 [8] minutes; P<0.001). The simulator group also showed faster improvements in overall performance, native biliary cannulation, and sphincterotomy. No statistical difference was found in overall AEs between the groups.

CONCLUSION

Early simulation training with the BCT improved technical competence in native biliary cannulation and accelerated overall ERCP learning. This approach has the potential to enhance ERCP training programs.

Pilot Study of a Modified DOPS Scale for Insulin Pump and CGM Installation Training in Chinese Medical Students During Endocrinology Rotations

Background

Direct Observation of Procedural Skills (DOPS) is a clinical assessment tool that enables trainers to observe medical students' procedural abilities in real-time clinical settings. It assesses students' knowledge application, decision-making, and skill proficiency during clinical tasks.

Methods

This study modifies the DOPS to evaluate the operation of insulin pumps (PUMP) and continuous glucose monitoring systems (CGMS) in diabetes management. Key elements of the modified DOPS include 1) Knowledge Assessment: Evaluating understanding of PUMP and CGMS, including interpreting CGMS data for insulin adjustments; 2) Operational Skills: Assessing correct PUMP needle insertion, programming, and adjustments; 3) Patient Safety: Ensuring safe and aseptic procedures; 4) Feedback: Providing constructive feedback to help students improve their skills.

Results

Training through DOPS led to significant improvements in all domains, overall performance scores, and reduced execution time for each domain. Correlations between domains showed that PUMP indication scores were linked to all other domains and execution times, including re-evaluation. Communication skills and seeking assistance were crucial factors influencing other domains. Multilinear regression analysis revealed that while DOPS-CGMS (R square 1.0) fully explained performance scores, DOPS-PUMP (R square 0.984) indicated that additional personal qualities significantly impacted students' PUMP operation performance.

Conclusion

This customized DOPS form offers insights into students' abilities in managing diabetes with PUMP and CGMS, while emphasizing the need for training on both technical skills and interpersonal skills in future educational models.

Development of American Society for Gastrointestinal Endoscopy standards for training in advanced endoscopy within dedicated advanced endoscopy fellowship programs

BACKGROUND AND AIMS

Training in interventional endoscopy is offered by nonaccredited advanced endoscopy fellowship programs (AEFPs). The number of these programs has increased dramatically with a concurrent increase in the breadth and complexity of interventional endoscopy procedures. Accreditation is governed by competency-based education, yet what constitutes a "high-quality" nonaccredited AEFP has not been defined. Using an evidence-based consensus process, we aimed to establish standards for AEFPs.

METHODS

The RAND UCLA appropriateness method, a well-described modified Delphi process to develop quality indicators, was used. A task force established by the American Society for Gastrointestinal Endoscopy drafted potential quality indicators (structure, process, and outcome) in 6 categories: activity preceding training; structure of AEFPs; training in ERCP, EUS, and EMR; and luminal stent placement. Three rounds of iterative feedback from 20 experts were conducted. Round 0 involved discussion of project details. In round 1, experts independently ranked proposed quality indicators on a 9-point interval scale ranging from highly inappropriate (1) to highly appropriate (9). Next, proposed quality indicators were discussed and reworded in a group meeting followed by round 2, in which experts independently reranked proposed quality indicators and provided benchmarks (when applicable). The median score for each quality indicator was calculated. Mean absolute deviation from the median was calculated, and appropriateness of potential quality indicators was assessed using the BIOMED concerted action on appropriateness definition, P value method, and interpercentile range adjusted for symmetry definition. A quality indicator was deemed appropriate if the median score was ≥7 and met criteria for appropriateness using all 3 defined statistical methods.

RESULTS

Of 89 proposed quality indicators, 37 statements met criteria as appropriate for a quality indicator (activity preceding training, 2; structure of AEFPs, 10; training in ERCP, 7; training in EUS, 8; training in EMR, 7; luminal stent placement, 3). Minimum thresholds were defined for 19 relevant quality indicators for number of trainers, procedures during fellowship, and procedures before assessment of competence. Among the final appropriate quality indicators were that all trainees should undergo qualitative and quantitative competence assessments using validated tools at least quarterly with documented feedback throughout the training period and that trainees should track outcomes and relevant quality metrics for specific procedures.

CONCLUSIONS

This consensus process using validated methodology established standards for an AEFP in an effort to ensure adequate training in the most commonly taught interventional endoscopic procedures (ERCP, EUS, EMR, and luminal stent placement) during fellowship. An important component of an AEFP is the use of competency-based assessments that are compliant with the Accreditation Council for Graduate Medical Education's Next Accreditation System, with the goal of ensuring that trainees achieve specific milestones in their progression to achieving cognitive and technical competency.

Validity evidence for endoscopic ultrasound competency assessment tools: Systematic review

Background and study aims Competent endoscopic ultrasound (EUS) performance requires a combination of technical, cognitive, and non-technical skills. Direct observation assessment tools can be employed to enhance learning and ascertain clinical competence; however, there is a need to systematically evaluate validity evidence supporting their use. We aimed to evaluate the validity evidence of competency assessment tools for EUS and examine their educational utility. Methods We systematically searched five databases and gray literature for studies investigating EUS competency assessment tools from inception to May 2023. Data on validity evidence across five domains (content, response process, internal structure, relations to other variables, and consequences) were extracted and graded (maximum score 15). We evaluated educational utility using the Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI). Results From 2081 records, we identified five EUS assessment tools from 10 studies. All tools are formative assessments intended to guide learning, with four employed in clinical settings. Validity evidence scores ranged from 3 to 12. The EUS and ERCP Skills Assessment Tool (TEESAT), Global Assessment of Performance and Skills in EUS (GAPS-EUS), and the EUS Assessment Tool (EUSAT) had the strongest validity evidence with scores of 12, 10, and 10, respectively. Overall educational utility was high given ease of tool use. MERSQI scores ranged from 9.5 to 12 (maximum score 13.5). Conclusions The TEESAT, GAPS-EUS, and EUSAT demonstrate strong validity evidence for formative assessment of EUS and are easily implemented in educational settings to monitor progress and support learning.

Endoscopic retrograde cholangiopancreatography training and education

Endoscopic retrograde cholangiopancreatography (ERCP) is an advanced endoscopic technique used to diagnose and treat biliary and pancreatic diseases. It is one of the most technically demanding endoscopic procedures. ERCP training programs must ensure trainees have adequate knowledge of the anatomy and physiology associated with biliopancreatic diseases. The variety of ERCP procedures included in training programs should provide sufficient basic training for novice trainees and advanced training for experienced endoscopists. The main endoscopic procedures should be trained in ascending order of difficulty. Incorporating models capable of simulating various clinical and anatomical conditions could provide an effective means of fulfilling training requirements, although they are not easily available due to expensive facilities and void of standard assessment. Competency assessment is crucial in ERCP training to ensure trainees can independently and safely perform ERCP. Because of the rapid advancement of diagnostic and therapeutic methods, postgraduate training is critical for ERCP practitioners. Once certificates are attained, practitioners are solely responsible for maintaining their competency, credentialing, and quality.

Pediatric endoscopy: how can we improve patient outcomes and ensure best practices?

INTRODUCTION

Strategies to promote high-quality endoscopy in children require consensus around pediatric-specific quality standards and indicators. Using a rigorous guideline development process, the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) was developed to support continuous quality improvement efforts within and across pediatric endoscopy services.

AREAS COVERED

This review presents a framework, informed by the PEnQuIN guidelines, for assessing endoscopist competence, granting procedural privileges, audit and feedback, and for skill remediation, when required. As is critical for promoting quality, PEnQuIN indicators can be benchmarked at the individual endoscopist, endoscopy facility, and endoscopy community levels. Furthermore, efforts to incorporate technologies, including electronic medical records and artificial intelligence, into endoscopic quality improvement processes can aid in creation of large-scale networks to facilitate comparison and standardization of quality indicator reporting across sites.

EXPERT OPINION

PEnQuIN quality standards and indicators provide a framework for continuous quality improvement in pediatric endoscopy, benefiting individual endoscopists, endoscopy facilities, and the broader endoscopy community. Routine and reliable measurement of data, facilitated by technology, is required to identify and drive improvements in care. Engaging all stakeholders in endoscopy quality improvement processes is crucial to enhancing patient outcomes and establishing best practices for safe, efficient, and effective pediatric endoscopic care.

Promoting Research that Supports High-Quality Gastrointestinal Endoscopy in Children

PURPOSE OF REVIEW

Defining and measuring the quality of endoscopic care is a key component of performing gastrointestinal endoscopy in children. The purpose of this review is to discuss quality metrics for pediatric gastrointestinal endoscopy and identify where additional research is needed.

RECENT FINDINGS

Pediatric-specific standards and indicators were recently defined by the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) working group through a rigorous guideline consensus process. Although the aim of these guidelines is to facilitate best practices for safe and high-quality gastrointestinal endoscopy in children, they highlight the pressing need to expand upon the body of evidence supporting these standards and indicators as predictors of clinically relevant outcomes. In this review, we propose and discuss ideas for several high-yield research topics to engage pediatric endoscopists and promote best practices in pediatric endoscopy.

ERCP-related adverse events: incidence, mechanisms, risk factors, prevention, and management

INTRODUCTION

Endoscopic retrograde cholangiopancreatography (ERCP) is a commonly performed procedure for pancreaticobiliary disease. While ERCP is highly effective, it is also associated with the highest adverse event (AE) rates of all commonly performed endoscopic procedures. Thus, it is critical that endoscopists and caregivers of patients undergoing ERCP have clear understandings of ERCP-related AEs.

AREAS COVERED

This narrative review provides a comprehensive overview of the available evidence on ERCP-related AEs. For the purposes of this review, we subdivide the presentation of each ERCP-related AE according to the following clinically relevant domains: definitions and incidence, proposed mechanisms, risk factors, prevention, and recognition and management. The evidence informing this review was derived in part from a search of the electronic databases PubMed, Embase, and Cochrane, performed on 1 May 20231 May 2023.

EXPERT OPINION

Knowledge of ERCP-related AEs is critical not only given potential improvements in peri-procedural quality and related care that can ensue but also given the importance of reviewing these considerations with patients during informed consent. The ERCP community and researchers should aim to apply standardized definitions of AEs. Evidence-based knowledge of ERCP risk factors should inform patient care decisions during training and beyond.