Using structured progress to measure competence in flexible bronchoscopy

Artificial Intelligence Improves Novices' Bronchoscopy Performance: A Randomized Controlled Trial in a Simulated Setting

BACKGROUND

Navigating through the bronchial tree and visualizing all bronchial segments is the initial step toward learning flexible bronchoscopy. A novel bronchial segment identification system based on artificial intelligence (AI) has been developed to help guide trainees toward more effective training.

RESEARCH QUESTION

Does feedback from an AI-based automatic bronchial segment identification system improve novice bronchoscopists' end-of-training performance?

STUDY DESIGN AND METHODS

The study was conducted as a randomized controlled trial in a standardized simulated setting. Novices without former bronchoscopy experience practiced on a mannequin. The feedback group (n = 10) received feedback from the AI, and the control group (n = 10) trained according to written instructions. Each participant decided when to end training and proceed to performing a full bronchoscopy without any aids.

RESULTS

The feedback group performed significantly better on all three outcome measures (median difference, P value): diagnostic completeness (3.5 segments, P < .001), structured progress (13.5 correct progressions, P < .001), and procedure time (-214 seconds, P = .002).

INTERPRETATION

Training guided by this novel AI makes novices perform more complete, more systematic, and faster bronchoscopies. Future studies should examine its use in a clinical setting and its effects on more advanced learners.

Effectiveness of Flexible Bronchoscopy Simulation-Based Training: A Systematic Review

BACKGROUND

The implementation of simulation-based training (SBT) to teach flexible bronchoscopy (FB) skills to novice trainees has increased during the last decade. However, it is unknown whether SBT is effective to teach FB to novices and which instructional features contribute to training effectiveness.

RESEARCH QUESTION

How effective is FB SBT and which instructional features contribute to training effectiveness?

STUDY DESIGN AND METHODS

We searched Embase, PubMed, Scopus, and Web of Science for articles on FB SBT for novice trainees, considering all available literature until November 10, 2022. We assessed methodological quality of included studies using a modified version of the Medical Education Research Study Quality Instrument, evaluated risk of bias with relevant tools depending on study design, assessed instructional features, and intended to correlate instructional features to outcome measures.

RESULTS

We identified 14 studies from an initial pool of 544 studies. Eleven studies reported positive effects of FB SBT on most of their outcome measures. However, risk of bias was moderate or high in eight studies, and only six studies were of high quality (modified Medical Education Research Study Quality Instrument score ≥ 12.5). Moreover, instructional features and outcome measures varied highly across studies, and only four studies evaluated intervention effects on behavioral outcome measures in the patient setting. All of the simulation training programs in studies with the highest methodological quality and most relevant outcome measures included curriculum integration and a range in task difficulty.

INTERPRETATION

Although most studies reported positive effects of simulation training programs on their outcome measures, definitive conclusions regarding training effectiveness on actual bronchoscopy performance in patients could not be made because of heterogeneity of training features and the sparse evidence of training effectiveness on validated behavioral outcome measures in a patient setting.

TRIAL REGISTRATION

PROSPERO; No.: CRD42021262853; URL: https://www.crd.york.ac.uk/prospero/.

Assessment of esophagogastroduodenoscopy skills on simulators before real-life performance

Background and study aims  Operator competency is essential for esophagogastroduodenoscopy (EGD) quality, which makes appropriate training with a final test important. The aims of this study were to develop a test for assessing skills in performing EGD, gather validity evidence for the test, and establish a credible pass/fail score. Methods  An expert panel developed a practical test using the Simbionix GI Mentor II simulator (3 D Systems) and an EGD phantom (OGI 4, CLA Medical) with a diagnostic (DP) and a technical skills part (TSP) for a prospective validation study. During the test a supervisor measured: 1) total time; 2) degree of mucosal visualization; and 3) landmarks and pathology identification. The contrasting groups standard setting method was used to establish a pass/fail score. Results  We included 15 novices (N), 10 intermediates (I), and 10 experienced endoscopists (E). The internal structure was high with a Cronbach's alpha of 0.76 for TSP time consumption and 0.74 for the identification of landmarks. Mean total times, in minutes, for the DP were N 15.7, I 11.3, and E 7.0, and for TSP., they were N 7.9, I 8.9, and E 2.9. The total numbers of identified landmarks were N 26, I 41, and E 48. Mean visualization percentages were N 80, I 71, and E 71. A pass/fail standard was established requiring identification of all landmarks and performance of the TSP in < 5 minutes. All experienced endoscopists passed, while none of the endoscopists in the other categories did. Conclusions  We established a test that can distinguish between participants with different competencies. This enables an objective and evidence-based approach to assessment of competencies in EGD.