Computer-Aided Detection False Positives in Colonoscopy

Advanced Endoscopic Imaging for Assessing Mucosal Healing and Histologic Remission in Inflammatory Bowel Diseases

Recent advances in the field of endoscopy have found fertile ground for application in inflammatory bowel diseases (IBD). Mucosal healing is a primary goal of IBD therapy, and current evidence shows that histologic remission (HR) is an additional desirable outcome. However, with the use of standard endoscopy, a considerable number of patients with histologically active disease go unrecognized. This narrative article examines the role, current or potential, of each endoscopic technique, from standard white-light endoscopy to molecular imaging, in the assessment of mucosal healing and HR in IBD.

A prospective comparison of two computer aided detection systems with different false positive rates in colonoscopy

This study evaluated the impact of differing false positive (FP) rates in two computer-aided detection (CADe) systems on the clinical effectiveness of artificial intelligence (AI)-assisted colonoscopy. The primary outcomes were adenoma detection rate (ADR) and adenomas per colonoscopy (APC). The ADR in the control, system A (3.2% FP rate), and system B (0.6% FP rate) groups were 44.3%, 43.4%, and 50.4%, respectively, with system B showing a significantly higher ADR than the control group. The APC for the control, A, and B groups were 0.75, 0.83, and 0.90, respectively, with system B also showing a higher APC than the control. The non-true lesion resection rates were 23.8%, 29.2%, and 21.3%, with system B having the lowest. The system with lower FP rates demonstrated improved ADR and APC without increasing the resection of non-neoplastic lesions. These findings suggest that higher FP rates negatively affect the clinical performance of AI-assisted colonoscopy.

Assessment of the role of false-positive alerts in computer-aided polyp detection for assistance capabilities

BACKGROUND AND AIM

False positives (FPs) pose a significant challenge in the application of artificial intelligence (AI) for polyp detection during colonoscopy. The study aimed to quantitatively evaluate the impact of computer-aided polyp detection (CADe) systems' FPs on endoscopists.

METHODS

The model's FPs were categorized into four gradients: 0-5, 5-10, 10-15, and 15-20 FPs per minute (FPPM). Fifty-six colonoscopy videos were collected for a crossover study involving 10 endoscopists. Polyp missed rate (PMR) was set as primary outcome. Subsequently, to further verify the impact of FPPM on the assistance capability of AI in clinical environments, a secondary analysis was conducted on a prospective randomized controlled trial (RCT) from Renmin Hospital of Wuhan University in China from July 1 to October 15, 2020, with the adenoma detection rate (ADR) as primary outcome.

RESULTS

Compared with routine group, CADe reduced PMR when FPPM was less than 5. However, with the continuous increase of FPPM, the beneficial effect of CADe gradually weakens. For secondary analysis of RCT, a total of 956 patients were enrolled. In AI-assisted group, ADR is higher when FPPM ≤ 5 compared with FPPM > 5 (CADe group: 27.78% vs 11.90%; P = 0.014; odds ratio [OR], 0.351; 95% confidence interval [CI], 0.152-0.812; COMBO group: 38.40% vs 23.46%, P = 0.029; OR, 0.427; 95% CI, 0.199-0.916). After AI intervention, ADR increased when FPPM ≤ 5 (27.78% vs 14.76%; P = 0.001; OR, 0.399; 95% CI, 0.231-0.690), but no statistically significant difference was found when FPPM > 5 (11.90% vs 14.76%, P = 0.788; OR, 1.111; 95% CI, 0.514-2.403).

CONCLUSION

The level of FPs of CADe does affect its effectiveness as an aid to endoscopists, with its best effect when FPPM is less than 5.

Assessing clinical efficacy of polyp detection models using open-access datasets

Background

Ensuring accurate polyp detection during colonoscopy is essential for preventing colorectal cancer (CRC). Recent advances in deep learning-based computer-aided detection (CADe) systems have shown promise in enhancing endoscopists' performances. Effective CADe systems must achieve high polyp detection rates from the initial seconds of polyp appearance while maintaining low false positive (FP) detection rates throughout the procedure.

Method

We integrated four open-access datasets into a unified platform containing over 340,000 images from various centers, including 380 annotated polyps, with distinct data splits for comprehensive model development and benchmarking. The REAL-Colon dataset, comprising 60 full-procedure colonoscopy videos from six centers, is used as the fifth dataset of the platform to simulate clinical conditions for model evaluation on unseen center data. Performance assessment includes traditional object detection metrics and new metrics that better meet clinical needs. Specifically, by defining detection events as sequences of consecutive detections, we compute per-polyp recall at early detection stages and average per-patient FPs, enabling the generation of Free-Response Receiver Operating Characteristic (FROC) curves.

Results

Using YOLOv7, we trained and tested several models across the proposed data splits, showcasing the robustness of our open-access platform for CADe system development and benchmarking. The introduction of new metrics allows for the optimization of CADe operational parameters based on clinically relevant criteria, such as per-patient FPs and early polyp detection. Our findings also reveal that omitting full-procedure videos leads to non-realistic assessments and that detecting small polyp bounding boxes poses the greatest challenge.

Conclusion

This study demonstrates how newly available open-access data supports ongoing research progress in environments that closely mimic clinical settings. The introduced metrics and FROC curves illustrate CADe clinical efficacy and can aid in tuning CADe hyperparameters.

Long-term impact of artificial intelligence on colorectal adenoma detection in high-risk colonoscopy

BACKGROUND

Improved adenoma detection rate (ADR) has been demonstrated with artificial intelligence (AI)-assisted colonoscopy. However, data on the real-world application of AI and its effect on colorectal cancer (CRC) screening outcomes is limited.

AIM

To analyze the long-term impact of AI on a diverse at-risk patient population undergoing diagnostic colonoscopy for positive CRC screening tests or symptoms.

METHODS

AI software (GI Genius, Medtronic) was implemented into the standard procedure protocol in November 2022. Data was collected on patient demographics, procedure indication, polyp size, location, and pathology. CRC screening outcomes were evaluated before and at different intervals after AI introduction with one year of follow-up.

RESULTS

We evaluated 1008 colonoscopies (278 pre-AI, 255 early post-AI, 285 established post-AI, and 190 late post-AI). The ADR was 38.1% pre-AI, 42.0% early post-AI (P = 0.77), 40.0% established post-AI (P = 0.44), and 39.5% late post-AI (P = 0.77). There were no significant differences in polyp detection rate (PDR, baseline 59.7%), advanced ADR (baseline 16.2%), and non-neoplastic PDR (baseline 30.0%) before and after AI introduction.

CONCLUSION

In patients with an increased pre-test probability of having an abnormal colonoscopy, the current generation of AI did not yield enhanced CRC screening metrics over high-quality colonoscopy. Although the potential of AI in colonoscopy is undisputed, current AI technology may not universally elevate screening metrics across all situations and patient populations. Future studies that analyze different AI systems across various patient populations are needed to determine the most effective role of AI in optimizing CRC screening in clinical practice.

Evaluating false-positive detection in a computer-aided detection system for colonoscopy

BACKGROUND AND AIM

Computer-aided detection (CADe) systems can efficiently detect polyps during colonoscopy. However, false-positive (FP) activation is a major limitation of CADe. We aimed to compare the rate and causes of FP using CADe before and after an update designed to reduce FP.

METHODS

We analyzed CADe-assisted colonoscopy videos recorded between July 2022 and October 2022. The number and causes of FPs and excessive time spent by the endoscopist on FP (ET) were compared pre- and post-update using 1:1 propensity score matching.

RESULTS

During the study period, 191 colonoscopy videos (94 and 97 in the pre- and post-update groups, respectively) were recorded. Propensity score matching resulted in 146 videos (73 in each group). The mean number of FPs and median ET per colonoscopy were significantly lower in the post-update group than those in the pre-update group (4.2 ± 3.7 vs 18.1 ± 11.1; P < 0.001 and 0 vs 16 s; P < 0.001, respectively). Mucosal tags, bubbles, and folds had the strongest association with decreased FP post-update (pre-update vs post-update: 4.3 ± 3.6 vs 0.4 ± 0.8, 0.32 ± 0.70 vs 0.04 ± 0.20, and 8.6 ± 6.7 vs 1.6 ± 1.7, respectively). There was no significant decrease in the true positive rate (post-update vs pre-update: 95.0% vs 99.2%; P = 0.09) or the adenoma detection rate (post-update vs pre-update: 52.1% vs 49.3%; P = 0.87).

CONCLUSIONS

The updated CADe can reduce FP without impairing polyp detection. A reduction in FP may help relieve the burden on endoscopists.

Human-artificial intelligence interaction in gastrointestinal endoscopy

The number and variety of applications of artificial intelligence (AI) in gastrointestinal (GI) endoscopy is growing rapidly. New technologies based on machine learning (ML) and convolutional neural networks (CNNs) are at various stages of development and deployment to assist patients and endoscopists in preparing for endoscopic procedures, in detection, diagnosis and classification of pathology during endoscopy and in confirmation of key performance indicators. Platforms based on ML and CNNs require regulatory approval as medical devices. Interactions between humans and the technologies we use are complex and are influenced by design, behavioural and psychological elements. Due to the substantial differences between AI and prior technologies, important differences may be expected in how we interact with advice from AI technologies. Human–AI interaction (HAII) may be optimised by developing AI algorithms to minimise false positives and designing platform interfaces to maximise usability. Human factors influencing HAII may include automation bias, alarm fatigue, algorithm aversion, learning effect and deskilling. Each of these areas merits further study in the specific setting of AI applications in GI endoscopy and professional societies should engage to ensure that sufficient emphasis is placed on human-centred design in development of new AI technologies.

Artificial Intelligence Applied to Colonoscopy: Is It Time to Take a Step Forward?

Growing evidence indicates that artificial intelligence (AI) applied to medicine is here to stay. In gastroenterology, AI computer vision applications have been stated as a research priority. The two main AI system categories are computer-aided polyp detection (CADe) and computer-assisted diagnosis (CADx). However, other fields of expansion are those related to colonoscopy quality, such as methods to objectively assess colon cleansing during the colonoscopy, as well as devices to automatically predict and improve bowel cleansing before the examination, predict deep submucosal invasion, obtain a reliable measurement of colorectal polyps and accurately locate colorectal lesions in the colon. Although growing evidence indicates that AI systems could improve some of these quality metrics, there are concerns regarding cost-effectiveness, and large and multicentric randomized studies with strong outcomes, such as post-colonoscopy colorectal cancer incidence and mortality, are lacking. The integration of all these tasks into one quality-improvement device could facilitate the incorporation of AI systems in clinical practice. In this manuscript, the current status of the role of AI in colonoscopy is reviewed, as well as its current applications, drawbacks and areas for improvement.

Comparing the number and relevance of false activations between 2 artificial intelligence computer-aided detection systems: the NOISE study

BACKGROUND AND AIMS

Artificial intelligence has been shown to be effective in polyp detection, and multiple computer-aided detection (CADe) systems have been developed. False-positive (FP) activation emerged as a possible way to benchmark CADe performance in clinical practice. The aim of this study was to validate a previously developed classification of FPs comparing the performances of different brands of approved CADe systems.

METHODS

We compared 2 different consecutive video libraries (40 video per arm) collected at Humanitas Research Hospital with 2 different CADe system brands (CADe A and CADe B). For each video, the number of CADe false activations, cause, and time spent by the endoscopist to examine the area erroneously highlighted were reported. The FP activations were classified according to the previously developed classification of FPs (the NOISE classification) according to their cause and relevance.

RESULTS

In CADe A 1021 FP activations were registered across the 40 videos (25.5 ± 12.2 FPs per colonoscopy), whereas in CADe B 1028 were identified (25.7 ± 13.2 FPs per colonoscopy; P = .53). Among them, 22.9 ± 9.9 (89.8% in CADe A) and 22.1 ± 10.0 (86.0% in CADe B) were because of artifacts from the bowel wall. Conversely, 2.6 ± 1.9 (10.2% in CADe A) and 3.5 ± 2.1 (14% in CADe B) were caused by bowel content (P = .45). Within CADe A each false activation required .2 ± .9 seconds, with 1.6 ± 1.0 FPs (6.3%) requiring additional time for endoscopic assessment. Comparable results were reported within CADe B with .2 ± .8 seconds spent per false activation and 1.8 ± 1.2 FPs per colonoscopy requiring additional inspection.

CONCLUSIONS

The use of a standardized nomenclature provided comparable results with either of the 2 recently approved CADe systems. (Clinical trial registration number: NCT04399590.).