Accurate diagnosis of endoscopic mucosal healing in ulcerative colitis using deep learning and machine learning

Deep mucosal healing in ulcerative colitis: how deep is better?

Ulcerative colitis (UC), characterized by its recurrent nature, imposes a significant disease burden and compromises the quality of life. Emerging evidence suggests that achieving clinical remission is not sufficient for long-term remission. In pursuit of a favorable prognosis, mucosal healing (MH) has been defined as the target of therapies in UC. This paradigm shift has given rise to the formulation of diverse endoscopic and histological scoring systems, providing distinct definitions for MH. Endoscopic remission (ER) has been widely employed in clinical practice, but it is susceptible to subjective factors related to endoscopists. And there's growing evidence that histological remission (HR) might be associated with a lower risk of disease flares, but the incorporation of HR as a routine therapeutic endpoint remains a debate. The integration of advanced technology has further enriched the definition of deep MH. Up to now, a universal standardized definition for deep MH in clinical practice is currently lacking. This review will focus on the definition of deep MH, from different dimensions, and analyze strengths and limitations, respectively. Subsequent multiple large-scale trials are needed to validate the concept of deep MH, offering valuable insights into potential benefits for UC patients.

The Role of Artificial Intelligence in the Diagnosis and Treatment of Ulcerative Colitis

BACKGROUND AND OBJECTIVES

This review aims to delve into the role of artificial intelligence in medicine. Ulcerative colitis (UC) is a chronic, inflammatory bowel disease (IBD) characterized by superficial mucosal inflammation, rectal bleeding, diarrhoea and abdominal pain. By identifying the challenges inherent in UC diagnosis, we seek to highlight the potential impact of artificial intelligence on enhancing both diagnosis and treatment methodologies for this condition.

METHOD

A targeted, non-systematic review of literature relating to ulcerative colitis was undertaken. The PubMed and Scopus databases were searched to categorize a well-rounded understanding of the field of artificial intelligence and its developing role in the diagnosis and treatment of ulcerative colitis. Articles that were thought to be relevant were included. This paper only included articles published in English.

RESULTS

Artificial intelligence (AI) refers to computer algorithms capable of learning, problem solving and decision-making. Throughout our review, we highlighted the role and importance of artificial intelligence in modern medicine, emphasizing its role in diagnosis through AI-assisted endoscopies and histology analysis and its enhancements in the treatment of ulcerative colitis. Despite these advances, AI is still hindered due to its current lack of adaptability to real-world scenarios and its difficulty in widespread data availability, which hinders the growth of AI-led data analysis.

CONCLUSIONS

When considering the potential of artificial intelligence, its ability to enhance patient care from a diagnostic and therapeutic perspective shows signs of promise. For the true utilization of artificial intelligence, some roadblocks must be addressed. The datasets available to AI may not truly reflect the real-world, which would prevent its impact in all clinical scenarios when dealing with a spectrum of patients with different backgrounds and presenting factors. Considering this, the shift in medical diagnostics and therapeutics is coinciding with evolving technology. With a continuous advancement in artificial intelligence programming and a perpetual surge in patient datasets, these networks can be further enhanced and supplemented with a greater cohort, enabling better outcomes and prediction models for the future of modern medicine.

Artificial intelligence image-based prediction models in IBD exhibit high risk of bias: A systematic review

BACKGROUND

There has been an increase in the development of both machine learning (ML) and deep learning (DL) prediction models in Inflammatory Bowel Disease. We aim in this systematic review to assess the methodological quality and risk of bias of ML and DL IBD image-based prediction studies.

METHODS

We searched three databases, PubMed, Scopus and Embase, to identify ML and DL diagnostic or prognostic predictive models using imaging data in IBD, to Dec 31, 2022. We restricted our search to include studies that primarily used conventional imaging data, were undertaken in human participants, and published in English. Two reviewers independently reviewed the abstracts. The methodological quality of the studies was determined, and risk of bias evaluated using the prediction risk of bias assessment tool (PROBAST).

RESULTS

Forty studies were included, thirty-nine developed diagnostic models. Seven studies utilized ML approaches, six were retrospective and none used multicenter data for model development. Thirty-three studies utilized DL approaches, ten were prospective, and twelve multicenter studies. Overall, all studies demonstrated high risk of bias. ML studies were evaluated in 4 domains all rated as high risk of bias: participants (6/7), predictors (1/7), outcome (3/7), and analysis (7/7), and DL studies evaluated in 3 domains: participants (24/33), outcome (10/33), and analysis (18/33). The majority of image-based studies used colonoscopy images.

CONCLUSION

The risk of bias was high in AI IBD image-based prediction models, owing to insufficient sample size, unreported missingness and lack of an external validation cohort. Models with a high risk of bias are unlikely to be generalizable and suitable for clinical implementation.

Artificial intelligence in endoscopy related to inflammatory bowel disease: A systematic review

BACKGROUND AND OBJECTIVES

In spite of rapid growth of artificial intelligence (AI) in digestive endoscopy in lesion detection and characterization, the role of AI in inflammatory bowel disease (IBD) endoscopy is not clearly defined. We aimed at systematically reviewing the role of AI in IBD endoscopy and identifying future research areas.

METHODS

We searched the PubMed and Embase database using keywords ("artificial intelligence" OR "machine learning" OR "computer-aided" OR "convolutional neural network") AND ("inflammatory bowel disease" OR "ulcerative colitis" OR "Crohn's") AND ("endoscopy" or "colonoscopy" or "capsule endoscopy" or "device assisted enteroscopy") between 1975 and September 2023 and identified 62 original articles for detailed review. Review articles, consensus guidelines, case reports/series, editorials, letter to the editor, non-peer-reviewed pre-prints and conference abstracts were excluded. The quality of the included studies was assessed using the MI-CLAIM checklist.

RESULTS

The accuracy of AI models (25 studies) to assess ulcerative colitis (UC) endoscopic activity ranged between 86.54% and 94.5%. AI-assisted capsule endoscopy reading (12 studies) substantially reduced analyzable images and reading time with excellent accuracy (90.5% to 99.9%). AI-assisted analysis of colonoscopic images can help differentiate IBD from non-IBD, UC from non-UC and UC from Crohn's disease (CD) (three studies) with 72.1%, 98.3% and > 90% accuracy, respectively. AI models based on non-invasive clinical and radiologic parameters could predict endoscopic activity (three studies). AI-assisted virtual chromoendoscopy (four studies) could predict histologic remission and long-term outcomes. Computer-assisted detection (CADe) of dysplasia (two studies) is feasible along with AI-based differentiation of high from low-grade IBD neoplasia (79% accuracy). AI is effective in linking electronic medical record data (two studies) with colonoscopic videos to facilitate widespread machine learning.

CONCLUSION

AI-assisted IBD endoscopy has the potential to impact clinical management by automated detection and characterization of endoscopic lesions. Large, multi-center, prospective studies and commercially available IBD-specific endoscopic AI algorithms are warranted.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

Endoscopic remission is an important therapeutic goal in ulcerative colitis (UC). The Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Mayo Endoscopic Score (MES) are the commonly used endoscopic scoring criteria. This systematic review and meta-analysis aimed to evaluate the accuracy of artificial intelligence (AI) in diagnosing endoscopic remission in UC. We also performed a meta-analysis of each of the four endoscopic remission criteria (UCEIS = 0, MES = 0, UCEIS = <1, MES = <1). Eighteen studies involving 13,687 patients were included. The combined sensitivity and specificity of AI for diagnosing endoscopic remission in UC was 87% (95% confidence interval [CI]:81-92%) and 92% (95% CI: 89-94%), respectively. The area under the curve (AUC) was 0.96 (95% CI: 0.94-0.97). The results showed that the AI model performed well regardless of which criteria were used to define endoscopic remission of UC.

Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning

BACKGROUND

Assessment of endoscopic activity in ulcerative colitis (UC) is important for treatment decisions and monitoring disease progress. However, substantial inter- and intraobserver variability in grading impairs the assessment. Our aim was to develop a computer-aided diagnosis system using deep learning to reduce subjectivity and improve the reliability of the assessment.

METHODS

The cohort comprises 11 276 images from 564 patients who underwent colonoscopy for UC. We propose a regression-based deep learning approach for the endoscopic evaluation of UC according to the Mayo endoscopic score (MES). Five state-of-the-art convolutional neural network (CNN) architectures were used for the performance measurements and comparisons. Ten-fold cross-validation was used to train the models and objectively benchmark them. Model performances were assessed using quadratic weighted kappa and macro F1 scores for full Mayo score classification and kappa statistics and F1 score for remission classification.

RESULTS

Five classification-based CNNs used in the study were in excellent agreement with the expert annotations for all Mayo subscores and remission classification according to the kappa statistics. When the proposed regression-based approach was used, (1) the performance of most of the models statistically significantly increased and (2) the same model trained on different cross-validation folds produced more robust results on the test set in terms of deviation between different folds.

CONCLUSIONS

Comprehensive experimental evaluations show that commonly used classification-based CNN architectures have successful performance in evaluating endoscopic disease activity of UC. Integration of domain knowledge into these architectures further increases performance and robustness, accelerating their translation into clinical use.

Diagnostic accuracy of convolutional neural network-based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis

BACKGROUND AND AIMS

Endoscopic assessment of ulcerative colitis (UC) can be performed by using the Mayo Endoscopic Score (MES) or the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). In this meta-analysis, we assessed the pooled diagnostic accuracy parameters of deep machine learning by means of convolutional neural network (CNN) algorithms in predicting UC severity on endoscopic images.

METHODS

Databases including MEDLINE, Scopus, and Embase were searched in June 2022. Outcomes of interest were the pooled accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Standard meta-analysis methods used the random-effects model, and heterogeneity was assessed using the I2statistics.

RESULTS

Twelve studies were included in the final analysis. The pooled diagnostic parameters of CNN-based machine learning algorithms in endoscopic severity assessment of UC were as follows: accuracy 91.5% (95% confidence interval [CI], 88.3-93.8; I2 = 84%), sensitivity 82.8% (95% CI, 78.3-86.5; I2 = 89%), specificity 92.4% (95% CI, 89.4-94.6; I2 = 84%), PPV 86.6% (95% CI, 82.3-90; I2 = 89%), and NPV 88.6% (95% CI, 85.7-91; I2 = 78%). Subgroup analysis revealed significantly better sensitivity and PPV with the UCEIS scoring system compared with the MES (93.6% [95% CI, 87.5-96.8; I2 = 77%] vs 82% [95% CI, 75.6-87; I2 = 89%], P = .003, and 93.6% [95% CI, 88.7-96.4; I2 = 68%] vs 83.6% [95% CI, 76.8-88.8; I2 = 77%], P = .007, respectively).

CONCLUSIONS

CNN-based machine learning algorithms demonstrated excellent pooled diagnostic accuracy parameters in the endoscopic severity assessment of UC. Using UCEIS scores in CNN training might offer better results than the MES. Further studies are warranted to establish these findings in real clinical settings.

New Technologies in Digestive Endoscopy for Ulcerative Colitis Patients

Ulcerative colitis (UC) is a chronic inflammatory bowel disease primarily affecting the colon and rectum. Endoscopy plays a crucial role in the diagnosis and management of UC. Recent advancements in endoscopic technology, including chromoendoscopy, confocal laser endomicroscopy, endocytoscopy and the use of artificial intelligence, have revolutionized the assessment and treatment of UC patients. These innovative techniques enable early detection of dysplasia and cancer, more precise characterization of disease extent and severity and more targeted biopsies, leading to improved diagnosis and disease monitoring. Furthermore, these advancements have significant implications for therapeutic decision making, empowering clinicians to carefully consider a range of treatment options, including pharmacological therapies, endoscopic interventions and surgical approaches. In this review, we provide an overview of the latest endoscopic technologies and their applications for diagnosing and monitoring UC. We also discuss their impact on treatment decision making, highlighting the potential benefits and limitations of each technique.

Deep learning model for distinguishing Mayo endoscopic subscore 0 and 1 in patients with ulcerative colitis

The aim of this study was to address the issue of differentiating between Mayo endoscopic subscore (MES) 0 and MES 1 using a deep learning model. A dataset of 492 ulcerative colitis (UC) patients who demonstrated MES improvement between January 2018 and December 2019 at Samsung Medical Center was utilized. Specifically, two representative images of the colon and rectum were selected from each patient, resulting in a total of 984 images for analysis. The deep learning model utilized in this study consisted of a convolutional neural network (CNN)-based encoder, with two auxiliary classifiers for the colon and rectum, as well as a final MES classifier that combined image features from both inputs. In the internal test, the model achieved an F1-score of 0.92, surpassing the performance of seven novice classifiers by an average margin of 0.11, and outperforming their consensus by 0.02. The area under the receiver operating characteristic curve (AUROC) was calculated to be 0.97 when considering MES 1 as positive, with an area under the precision-recall curve (AUPRC) of 0.98. In the external test using the Hyperkvasir dataset, the model achieved an F1-score of 0.89, AUROC of 0.86, and AUPRC of 0.97. The results demonstrate that the proposed CNN-based model, which integrates image features from both the colon and rectum, exhibits superior performance in accurately discriminating between MES 0 and MES 1 in patients with UC.

Artificial Intelligence in Inflammatory Bowel Disease Endoscopy: Advanced Development and New Horizons

Inflammatory bowel disease (IBD) is a complex chronic immune disease with two subtypes: Crohn's disease and ulcerative colitis. Considering the differences in pathogenesis, etiology, clinical presentation, and response to therapy among patients, gastroenterologists mainly rely on endoscopy to diagnose and treat IBD during clinical practice. However, as exemplified by the increasingly comprehensive ulcerative colitis endoscopic scoring system, the endoscopic diagnosis, evaluation, and treatment of IBD still rely on the subjective manipulation and judgment of endoscopists. In recent years, the use of artificial intelligence (AI) has grown substantially in various medical fields, and an increasing number of studies have investigated the use of this emerging technology in the field of gastroenterology. Clinical applications of AI have focused on IBD pathogenesis, etiology, diagnosis, and patient prognosis. Large-scale datasets offer tremendous utility in the development of novel tools to address the unmet clinical and practice needs for treating patients with IBD. However, significant differences among AI methodologies, datasets, and clinical findings limit the incorporation of AI technology into clinical practice. In this review, we discuss practical AI applications in the diagnosis of IBD via gastroenteroscopy and speculate regarding a future in which AI technology provides value for the diagnosis and treatment of IBD patients.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

Chemoprevention of Colitis-Associated Dysplasia or Cancer in Inflammatory Bowel Disease

The association between inflammatory bowel disease and colorectal cancer is well known. Although the overall incidence of inflammatory bowel disease has declined recently, patients with this disease still have a 1.7-fold increased risk of colorectal cancer. The risk factors for developing colorectal cancer include extensive colitis, young age at diagnosis, disease duration, primary sclerosing cholangitis, chronic colonic mucosal inflammation, dysplasia lesion, and post-inflammatory polyps. In patients with inflammatory bowel disease, control of chronic inflammation and surveillance colonoscopies are important for the prevention of colorectal cancer. The 2017 guidelines from the European Crohn's and Colitis Organisation suggest that colonoscopies to screen for colorectal cancer should be performed when inflammatory bowel disease symptoms have lasted for 8 years. Current evidence supports the use of chemoprevention therapy with mesalamine to reduce the risk of colorectal cancer in patients with ulcerative colitis. Other compounds, including thiopurine, folic acid, statin, and tumor necrosis factor-α inhibitor, are controversial. Large surveillance cohort studies with longer follow-up duration are needed to evaluate the impact of drugs on colorectal cancer risks.

Enhanced segmentation of gastrointestinal polyps from capsule endoscopy images with artifacts using ensemble learning

BACKGROUND

Endoscopy artifacts are widespread in real capsule endoscopy (CE) images but not in high-quality standard datasets.

AIM

To improve the segmentation performance of polyps from CE images with artifacts based on ensemble learning.

METHODS

We collected 277 polyp images with CE artifacts from 5760 h of videos from 480 patients at Guangzhou First People’s Hospital from January 2016 to December 2019. Two public high-quality standard external datasets were retrieved and used for the comparison experiments. For each dataset, we randomly segmented the data into training, validation, and testing sets for model training, selection, and testing. We compared the performance of the base models and the ensemble model in segmenting polyps from images with artifacts.

RESULTS

The performance of the semantic segmentation model was affected by artifacts in the sample images, which also affected the results of polyp detection by CE using a single model. The evaluation based on real datasets with artifacts and standard datasets showed that the ensemble model of all state-of-the-art models performed better than the best corresponding base learner on the real dataset with artifacts. Compared with the corresponding optimal base learners, the intersection over union (IoU) and dice of the ensemble learning model increased to different degrees, ranging from 0.08% to 7.01% and 0.61% to 4.93%, respectively. Moreover, in the standard datasets without artifacts, most of the ensemble models were slightly better than the base learner, as demonstrated by the IoU and dice increases ranging from -0.28% to 1.20% and -0.61% to 0.76%, respectively.

CONCLUSION

Ensemble learning can improve the segmentation accuracy of polyps from CE images with artifacts. Our results demonstrated an improvement in the detection rate of polyps with interference from artifacts.

Systematic review of artificial intelligence-based image diagnosis for inflammatory bowel disease

OBJECTIVES

Diagnosis of inflammatory bowel diseases (IBD) involves combining clinical, laboratory, endoscopic, histologic, and radiographic data. Artificial intelligence (AI) is rapidly being developed in various fields of medicine, including IBD. Because a key part in the diagnosis of IBD involves evaluating imaging data, AI is expected to play an important role in this aspect in the coming decades. We conducted a systematic literature review to highlight the current advancement of AI in diagnosing IBD from imaging data.

METHODS

We performed an electronic PubMed search of the MEDLINE database for studies up to January 2022 involving IBD and AI. Studies using imaging data as input were included, and nonimaging data were excluded.

RESULTS

A total of 27 studies are reviewed, including 18 studies involving endoscopic images and nine studies involving other imaging data.

CONCLUSION

We highlight in this review the recent advancement of AI in diagnosing IBD from imaging data by summarizing the relevant studies, and discuss the future role of AI in clinical practice.