Automated evaluation of colon capsule endoscopic severity of ulcerative colitis using ResNet50

Artificial intelligence image-based prediction models in IBD exhibit high risk of bias: A systematic review

BACKGROUND

There has been an increase in the development of both machine learning (ML) and deep learning (DL) prediction models in Inflammatory Bowel Disease. We aim in this systematic review to assess the methodological quality and risk of bias of ML and DL IBD image-based prediction studies.

METHODS

We searched three databases, PubMed, Scopus and Embase, to identify ML and DL diagnostic or prognostic predictive models using imaging data in IBD, to Dec 31, 2022. We restricted our search to include studies that primarily used conventional imaging data, were undertaken in human participants, and published in English. Two reviewers independently reviewed the abstracts. The methodological quality of the studies was determined, and risk of bias evaluated using the prediction risk of bias assessment tool (PROBAST).

RESULTS

Forty studies were included, thirty-nine developed diagnostic models. Seven studies utilized ML approaches, six were retrospective and none used multicenter data for model development. Thirty-three studies utilized DL approaches, ten were prospective, and twelve multicenter studies. Overall, all studies demonstrated high risk of bias. ML studies were evaluated in 4 domains all rated as high risk of bias: participants (6/7), predictors (1/7), outcome (3/7), and analysis (7/7), and DL studies evaluated in 3 domains: participants (24/33), outcome (10/33), and analysis (18/33). The majority of image-based studies used colonoscopy images.

CONCLUSION

The risk of bias was high in AI IBD image-based prediction models, owing to insufficient sample size, unreported missingness and lack of an external validation cohort. Models with a high risk of bias are unlikely to be generalizable and suitable for clinical implementation.

Artificial intelligence in endoscopy related to inflammatory bowel disease: A systematic review

BACKGROUND AND OBJECTIVES

In spite of rapid growth of artificial intelligence (AI) in digestive endoscopy in lesion detection and characterization, the role of AI in inflammatory bowel disease (IBD) endoscopy is not clearly defined. We aimed at systematically reviewing the role of AI in IBD endoscopy and identifying future research areas.

METHODS

We searched the PubMed and Embase database using keywords ("artificial intelligence" OR "machine learning" OR "computer-aided" OR "convolutional neural network") AND ("inflammatory bowel disease" OR "ulcerative colitis" OR "Crohn's") AND ("endoscopy" or "colonoscopy" or "capsule endoscopy" or "device assisted enteroscopy") between 1975 and September 2023 and identified 62 original articles for detailed review. Review articles, consensus guidelines, case reports/series, editorials, letter to the editor, non-peer-reviewed pre-prints and conference abstracts were excluded. The quality of the included studies was assessed using the MI-CLAIM checklist.

RESULTS

The accuracy of AI models (25 studies) to assess ulcerative colitis (UC) endoscopic activity ranged between 86.54% and 94.5%. AI-assisted capsule endoscopy reading (12 studies) substantially reduced analyzable images and reading time with excellent accuracy (90.5% to 99.9%). AI-assisted analysis of colonoscopic images can help differentiate IBD from non-IBD, UC from non-UC and UC from Crohn's disease (CD) (three studies) with 72.1%, 98.3% and > 90% accuracy, respectively. AI models based on non-invasive clinical and radiologic parameters could predict endoscopic activity (three studies). AI-assisted virtual chromoendoscopy (four studies) could predict histologic remission and long-term outcomes. Computer-assisted detection (CADe) of dysplasia (two studies) is feasible along with AI-based differentiation of high from low-grade IBD neoplasia (79% accuracy). AI is effective in linking electronic medical record data (two studies) with colonoscopic videos to facilitate widespread machine learning.

CONCLUSION

AI-assisted IBD endoscopy has the potential to impact clinical management by automated detection and characterization of endoscopic lesions. Large, multi-center, prospective studies and commercially available IBD-specific endoscopic AI algorithms are warranted.

Using Computer Vision to Improve Endoscopic Disease Quantification in Therapeutic Clinical Trials of Ulcerative Colitis

BACKGROUND & AIMS

Endoscopic assessment of ulcerative colitis (UC) typically reports only the maximum severity observed. Computer vision methods may better quantify mucosal injury detail, which varies among patients.

METHODS

Endoscopic video from the UNIFI clinical trial (A Study to Evaluate the Safety and Efficacy of Ustekinumab Induction and Maintenance Therapy in Participants With Moderately to Severely Active Ulcerative Colitis) comparing ustekinumab and placebo for UC were processed in a computer vision analysis that spatially mapped Mayo Endoscopic Score (MES) to generate the Cumulative Disease Score (CDS). CDS was compared with the MES for differentiating ustekinumab vs placebo treatment response and agreement with symptomatic remission at week 44. Statistical power, effect, and estimated sample sizes for detecting endoscopic differences between treatments were calculated using both CDS and MES measures. Endoscopic video from a separate phase 2 clinical trial replication cohort was performed for validation of CDS performance.

RESULTS

Among 748 induction and 348 maintenance patients, CDS was lower in ustekinumab vs placebo users at week 8 (141.9 vs 184.3; P < .0001) and week 44 (78.2 vs 151.5; P < .0001). CDS was correlated with the MES (P < .0001) and all clinical components of the partial Mayo score (P < .0001). Stratification by pretreatment CDS revealed ustekinumab was more effective than placebo (P < .0001) with increasing effect in severe vs mild disease (-85.0 vs -55.4; P < .0001). Compared with the MES, CDS was more sensitive to change, requiring 50% fewer participants to demonstrate endoscopic differences between ustekinumab and placebo (Hedges' g = 0.743 vs 0.460). CDS performance in the JAK-UC replication cohort was similar to UNIFI.

CONCLUSIONS

As an automated and quantitative measure of global endoscopic disease severity, the CDS offers artificial intelligence enhancement of traditional MES capability to better evaluate UC in clinical trials and potentially practice.

Machine and deep learning in inflammatory bowel disease

PURPOSE OF REVIEW

The Management of inflammatory bowel disease (IBD) has evolved with the introduction and widespread adoption of biologic agents; however, the advent of artificial intelligence technologies like machine learning and deep learning presents another watershed moment in IBD treatment. Interest in these methods in IBD research has increased over the past 10 years, and they offer a promising path to better clinical outcomes for IBD patients.

RECENT FINDINGS

Developing new tools to evaluate IBD and inform clinical management is challenging because of the expansive volume of data and requisite manual interpretation of data. Recently, machine and deep learning models have been used to streamline diagnosis and evaluation of IBD by automating review of data from several diagnostic modalities with high accuracy. These methods decrease the amount of time that clinicians spend manually reviewing data to formulate an assessment.

SUMMARY

Interest in machine and deep learning is increasing in medicine, and these methods are poised to revolutionize the way that we treat IBD. Here, we highlight the recent advances in using these technologies to evaluate IBD and discuss the ways that they can be leveraged to improve clinical outcomes.

A trial deep learning-based model for four-class histologic classification of colonic tumor from narrow band imaging

Narrow band imaging (NBI) has been extensively utilized as a diagnostic tool for colorectal neoplastic lesions. This study aimed to develop a trial deep learning (DL) based four-class classification model for low-grade dysplasia (LGD); high-grade dysplasia or mucosal carcinoma (HGD); superficially invasive submucosal carcinoma (SMs) and deeply invasive submucosal carcinomas (SMd) and evaluate its potential as a diagnostic tool. We collected a total of 1,390 NBI images as the dataset, including 53 LGD, 120 HGD, 20 SMs and 17 SMd. A total of 598,801 patches were trimmed from the lesion and background. A patch-based classification model was built by employing a residual convolutional neural network (CNN) and validated by three-fold cross-validation. The patch-based validation accuracy was 0.876, 0.957, 0.907 and 0.929 in LGD, HGD, SMs and SMd, respectively. The image-level classification algorithm was derived from the patch-based mapping across the entire image domain, attaining accuracies of 0.983, 0.990, 0.964, and 0.992 in LGD, HGD, SMs, and SMd, respectively. Our CNN-based model demonstrated high performance for categorizing the histological grade of dysplasia as well as the depth of invasion in routine colonoscopy, suggesting a potential diagnostic tool with minimal human inputs.

Deep learning as a novel method for endoscopic diagnosis of chronic atrophic gastritis: a prospective nested case-control study

Background and aims

Chronic atrophic gastritis (CAG) is a precancerous disease that often leads to the development of gastric cancer (GC) and is positively correlated with GC morbidity. However, the sensitivity of the endoscopic diagnosis of CAG is only 42%. Therefore, we developed a real-time video monitoring model for endoscopic diagnosis of CAG based on U-Net deep learning (DL) and conducted a prospective nested case–control study to evaluate the diagnostic evaluation indices of the model and its consistency with pathological diagnosis.

Methods

Our cohort consisted of 1539 patients undergoing gastroscopy from December 1, 2020, to July 1, 2021. Based on pathological diagnosis, patients in the cohort were divided into the CAG group or the chronic nonatrophic gastritis (CNAG) group, and we assessed the diagnostic evaluation indices of this model and its consistency with pathological diagnosis after propensity score matching (PSM) to minimize selection bias in the study.

Results

After matching, the diagnostic evaluation indices and consistency evaluation of the model were better than those of endoscopists [sensitivity (84.02% vs. 62.72%), specificity (97.04% vs. 81.95%), positive predictive value (96.60% vs. 77.66%), negative predictive value (85.86% vs. 68.73%), accuracy rate (90.53% vs. 72.34%), Youden index (81.06% vs. 44.67%), odd product (172.5 vs. 7.64), positive likelihood ratio (28.39 vs. 3.47), negative likelihood ratio (0.16 vs. 0.45), AUC (95% CI) [0.909 (0.884–0.934) vs. 0.740 (0.702–0.778)] and Kappa (0.852 vs. 0.558)].

Conclusions

Our prospective nested case–control study proved that the diagnostic evaluation indices and consistency evaluation of the real-time video monitoring model for endoscopic diagnosis of CAG based on U-Net DL were superior to those of endoscopists.

Trial registrationChiCTR2100044458, 18/03/2020.