Addressing false-positive findings with artificial intelligence for polyp detection

Assessment of the role of false-positive alerts in computer-aided polyp detection for assistance capabilities

BACKGROUND AND AIM

False positives (FPs) pose a significant challenge in the application of artificial intelligence (AI) for polyp detection during colonoscopy. The study aimed to quantitatively evaluate the impact of computer-aided polyp detection (CADe) systems' FPs on endoscopists.

METHODS

The model's FPs were categorized into four gradients: 0-5, 5-10, 10-15, and 15-20 FPs per minute (FPPM). Fifty-six colonoscopy videos were collected for a crossover study involving 10 endoscopists. Polyp missed rate (PMR) was set as primary outcome. Subsequently, to further verify the impact of FPPM on the assistance capability of AI in clinical environments, a secondary analysis was conducted on a prospective randomized controlled trial (RCT) from Renmin Hospital of Wuhan University in China from July 1 to October 15, 2020, with the adenoma detection rate (ADR) as primary outcome.

RESULTS

Compared with routine group, CADe reduced PMR when FPPM was less than 5. However, with the continuous increase of FPPM, the beneficial effect of CADe gradually weakens. For secondary analysis of RCT, a total of 956 patients were enrolled. In AI-assisted group, ADR is higher when FPPM ≤ 5 compared with FPPM > 5 (CADe group: 27.78% vs 11.90%; P = 0.014; odds ratio [OR], 0.351; 95% confidence interval [CI], 0.152-0.812; COMBO group: 38.40% vs 23.46%, P = 0.029; OR, 0.427; 95% CI, 0.199-0.916). After AI intervention, ADR increased when FPPM ≤ 5 (27.78% vs 14.76%; P = 0.001; OR, 0.399; 95% CI, 0.231-0.690), but no statistically significant difference was found when FPPM > 5 (11.90% vs 14.76%, P = 0.788; OR, 1.111; 95% CI, 0.514-2.403).

CONCLUSION

The level of FPs of CADe does affect its effectiveness as an aid to endoscopists, with its best effect when FPPM is less than 5.

Improvement of adenoma detection rate by two computer-aided colonic polyp detection systems in high adenoma detectors: a randomized multicenter trial

BACKGROUND

This study aimed to evaluate the benefits of a self-developed computer-aided polyp detection system (SD-CADe) and a commercial system (CM-CADe) for high adenoma detectors compared with white-light endoscopy (WLE) as a control.

METHODS

Average-risk 50-75-year-old individuals who underwent screening colonoscopy at five referral centers were randomized to SD-CADe, CM-CADe, or WLE groups (1:1:1 ratio). Trainees and staff with an adenoma detection rate (ADR) of ≥35% were recruited. The primary outcome was ADR. Secondary outcomes were the proximal adenoma detection rate (pADR), advanced adenoma detection rate (AADR), and the number of adenomas, proximal adenomas, and advanced adenomas per colonoscopy (APC, pAPC, and AAPC, respectively).

RESULTS

The study enrolled 1200 participants. The ADR in the control, CM-CADe, and SD-CADe groups was 38.3%, 50.0%, and 54.8%, respectively. The pADR was 23.0%, 32.3%, and 38.8%, respectively. AADR was 6.0%, 10.3%, and 9.5%, respectively. After adjustment, the ADR and pADR in both intervention groups were significantly higher than in controls (all P<0.05). The APC in the control, CM-CADe, and SD-CADe groups was 0.66, 1.04, and 1.16, respectively. The pAPC was 0.33, 0.53, and 0.64, respectively, and the AAPC was 0.07, 0.12, and 0.10, respectively. Both CADe systems showed significantly higher APC and pAPC than WLE. AADR and AAPC were improved in both CADe groups versus control, although the differences were not statistically significant.

CONCLUSION

Even in high adenoma detectors, CADe significantly improved ADR and APC. The AADR tended to be higher with both systems, and this may enhance colorectal cancer prevention.

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

BACKGROUND/AIMS

This study aims to compare polyp detection performance of "Deep-GI," a newly developed artificial intelligence (AI) model, to a previously validated AI model computer-aided polyp detection (CADe) using various false positive (FP) thresholds and determining the best threshold for each model.

METHODS

Colonoscopy videos were collected prospectively and reviewed by three expert endoscopists (gold standard), trainees, CADe (CAD EYE; Fujifilm Corp.), and Deep-GI. Polyp detection sensitivity (PDS), polyp miss rates (PMR), and false-positive alarm rates (FPR) were compared among the three groups using different FP thresholds for the duration of bounding boxes appearing on the screen.

RESULTS

In total, 170 colonoscopy videos were used in this study. Deep-GI showed the highest PDS (99.4% vs. 85.4% vs. 66.7%, p<0.01) and the lowest PMR (0.6% vs. 14.6% vs. 33.3%, p<0.01) when compared to CADe and trainees, respectively. Compared to CADe, Deep-GI demonstrated lower FPR at FP thresholds of ≥0.5 (12.1 vs. 22.4) and ≥1 second (4.4 vs. 6.8) (both p<0.05). However, when the threshold was raised to ≥1.5 seconds, the FPR became comparable (2 vs. 2.4, p=0.3), while the PMR increased from 2% to 10%.

CONCLUSION

Compared to CADe, Deep-GI demonstrated a higher PDS with significantly lower FPR at ≥0.5- and ≥1-second thresholds. At the ≥1.5-second threshold, both systems showed comparable FPR with increased PMR.