Deep learning enabled classification of Mayo endoscopic subscore in patients with ulcerative colitis

Artificial Intelligence in Inflammatory Bowel Disease

Artificial intelligence (AI) is being increasingly studied and implemented in gastroenterology. In inflammatory bowel disease (IBD), numerous AI models are being developed to assist with IBD diagnosis, standardization of endoscopic and radiologic disease activity, and predicting outcomes. Further prospective, multicenter studies representing diverse populations and novel applications are needed prior to routine implementation in clinical practice and expected improved outcomes for clinicians and patients.

How Artificial Intelligence Will Transform Clinical Care, Research, and Trials for Inflammatory Bowel Disease

Artificial intelligence (AI) refers to computer-based methodologies that use data to teach a computer to solve pre-defined tasks; these methods can be applied to identify patterns in large multi-modal data sources. AI applications in inflammatory bowel disease (IBD) includes predicting response to therapy, disease activity scoring of endoscopy, drug discovery, and identifying bowel damage in images. As a complex disease with entangled relationships between genomics, metabolomics, microbiome, and the environment, IBD stands to benefit greatly from methodologies that can handle this complexity. We describe current applications, critical challenges, and propose future directions of AI in IBD.

Ulcerative Colitis, LAIR1 and TOX2 Expression, and Colorectal Cancer Deep Learning Image Classification Using Convolutional Neural Networks

BACKGROUND

Ulcerative colitis is a chronic inflammatory bowel disease of the colon mucosa associated with a higher risk of colorectal cancer.

OBJECTIVE

This study classified hematoxylin and eosin (H&E) histological images of ulcerative colitis, normal colon, and colorectal cancer using artificial intelligence (deep learning).

METHODS

A convolutional neural network (CNN) was designed and trained to classify the three types of diagnosis, including 35 cases of ulcerative colitis (n = 9281 patches), 21 colon control (n = 12,246), and 18 colorectal cancer (n = 63,725). The data were partitioned into training (70%) and validation sets (10%) for training the network, and a test set (20%) to test the performance on the new data. The CNNs included transfer learning from ResNet-18, and a comparison with other CNN models was performed. Explainable artificial intelligence for computer vision was used with the Grad-CAM technique, and additional LAIR1 and TOX2 immunohistochemistry was performed in ulcerative colitis to analyze the immune microenvironment.

RESULTS

Conventional clinicopathological analysis showed that steroid-requiring ulcerative colitis was characterized by higher endoscopic Baron and histologic Geboes scores and LAIR1 expression in the lamina propria, but lower TOX2 expression in isolated lymphoid follicles (all p values < 0.05) compared to mesalazine-responsive ulcerative colitis. The CNN classification accuracy was 99.1% for ulcerative colitis, 99.8% for colorectal cancer, and 99.1% for colon control. The Grad-CAM heatmap confirmed which regions of the images were the most important. The CNNs also differentiated between steroid-requiring and mesalazine-responsive ulcerative colitis based on H&E, LAIR1, and TOX2 staining. Additional classification of 10 new cases of colorectal cancer (adenocarcinoma) were correctly classified.

CONCLUSIONS

CNNs are especially suited for image classification in conditions such as ulcerative colitis and colorectal cancer; LAIR1 and TOX2 are relevant immuno-oncology markers in ulcerative colitis.

Artificial Intelligence in Coloproctology: A Review of Emerging Technologies and Clinical Applications

Artificial intelligence (AI) has emerged as a transformative tool across several specialties, namely gastroenterology, where it has the potential to optimize both diagnosis and treatment as well as enhance patient care. Coloproctology, due to its highly prevalent pathologies and tremendous potential to cause significant mortality and morbidity, has drawn a lot of attention regarding AI applications. In fact, its application has yielded impressive outcomes in various domains, colonoscopy being one prominent example, where it aids in the detection of polyps and early signs of colorectal cancer with high accuracy and efficiency. With a less explored path but equivalent promise, AI-powered capsule endoscopy ensures accurate and time-efficient video readings, already detecting a wide spectrum of anomalies. High-resolution anoscopy is an area that has been growing in interest in recent years, with efforts being made to integrate AI. There are other areas, such as functional studies, that are currently in the early stages, but evidence is expected to emerge soon. According to the current state of research, AI is anticipated to empower gastroenterologists in the decision-making process, paving the way for a more precise approach to diagnosing and treating patients. This review aims to provide the state-of-the-art use of AI in coloproctology while also reflecting on future directions and perspectives.

Artificial intelligence and endo-histo-omics: new dimensions of precision endoscopy and histology in inflammatory bowel disease

Integrating artificial intelligence into inflammatory bowel disease (IBD) has the potential to revolutionise clinical practice and research. Artificial intelligence harnesses advanced algorithms to deliver accurate assessments of IBD endoscopy and histology, offering precise evaluations of disease activity, standardised scoring, and outcome prediction. Furthermore, artificial intelligence offers the potential for a holistic endo-histo-omics approach by interlacing and harmonising endoscopy, histology, and omics data towards precision medicine. The emerging applications of artificial intelligence could pave the way for personalised medicine in IBD, offering patient stratification for the most beneficial therapy with minimal risk. Although artificial intelligence holds promise, challenges remain, including data quality, standardisation, reproducibility, scarcity of randomised controlled trials, clinical implementation, ethical concerns, legal liability, and regulatory issues. The development of standardised guidelines and interdisciplinary collaboration, including policy makers and regulatory agencies, is crucial for addressing these challenges and advancing artificial intelligence in IBD clinical practice and trials.

Can artificial intelligence replace endoscopists when assessing mucosal healing in ulcerative colitis? A systematic review and diagnostic test accuracy meta-analysis

BACKGROUNDS AND AIMS

Mucosal healing (MH) in inflammatory bowel diseases (IBD) is an important landmark for clinical decision making. Artificial intelligence systems (AI) that automatically deliver the grade of endoscopic inflammation may solve moderate interobserver agreement and the need of central reading in clinical trials.

METHODS

We performed a systematic review of EMBASE and MEDLINE databases up to 01/12/2022 following PRISMA and the Joanna Briggs Institute methodologies to answer the following question: "Can AI replace endoscopists when assessing MH in IBD?". The research was restricted to ulcerative colitis (UC), and a diagnostic odds ratio (DOR) meta-analysis was performed. Risk of bias was evaluated with QUADAS-2 tool.

RESULTS

A total of 21 / 739 records were selected for full text evaluation, and 12 were included in the meta-analysis. Deep learning algorithms based on convolutional neural networks architecture achieved a satisfactory performance in evaluating MH on UC, with sensitivity, specificity, DOR and SROC of respectively 0.91(CI95 %:0.86-0.95);0.89(CI95 %:0.84-0.93);92.42(CI95 %:54.22-157.53) and 0.957 when evaluating fixed images (n = 8) and 0.86(CI95 %:0.75-0.93);0.91(CI95 %:0.87-0.94);70.86(CI95 %:24.63-203.86) and 0.941 when evaluating videos (n = 6). Moderate-high levels of heterogeneity were noted, limiting the quality of the evidence.

CONCLUSIONS

AI systems showed high potential in detecting MH in UC with optimal diagnostic performance, although moderate-high heterogeneity of the data was noted. Standardised and shared AI training may reduce heterogeneity between systems.

Challenges in IBD Research 2024: Precision Medicine

Precision medicine is part of 5 focus areas of the Challenges in IBD Research 2024 research document, which also includes preclinical human IBD mechanisms, environmental triggers, novel technologies, and pragmatic clinical research. Building on Challenges in IBD Research 2019, the current Challenges aims to provide a comprehensive overview of current gaps in inflammatory bowel diseases (IBDs) research and deliver actionable approaches to address them with a focus on how these gaps can lead to advancements in interception, remission, and restoration for these diseases. The document is the result of multidisciplinary input from scientists, clinicians, patients, and funders, and represents a valuable resource for patient-centric research prioritization. In particular, the precision medicine section is focused on the main research gaps in elucidating how to bring the best care to the individual patient in IBD. Research gaps were identified in biomarker discovery and validation for predicting disease progression and choosing the most appropriate treatment for each patient. Other gaps were identified in making the best use of existing patient biosamples and clinical data, developing new technologies to analyze large datasets, and overcoming regulatory and payer hurdles to enable clinical use of biomarkers. To address these gaps, the Workgroup suggests focusing on thoroughly validating existing candidate biomarkers, using best-in-class data generation and analysis tools, and establishing cross-disciplinary teams to tackle regulatory hurdles as early as possible. Altogether, the precision medicine group recognizes the importance of bringing basic scientific biomarker discovery and translating it into the clinic to help improve the lives of IBD patients.

The Role of Artificial Intelligence in the Diagnosis and Treatment of Ulcerative Colitis

BACKGROUND AND OBJECTIVES

This review aims to delve into the role of artificial intelligence in medicine. Ulcerative colitis (UC) is a chronic, inflammatory bowel disease (IBD) characterized by superficial mucosal inflammation, rectal bleeding, diarrhoea and abdominal pain. By identifying the challenges inherent in UC diagnosis, we seek to highlight the potential impact of artificial intelligence on enhancing both diagnosis and treatment methodologies for this condition.

METHOD

A targeted, non-systematic review of literature relating to ulcerative colitis was undertaken. The PubMed and Scopus databases were searched to categorize a well-rounded understanding of the field of artificial intelligence and its developing role in the diagnosis and treatment of ulcerative colitis. Articles that were thought to be relevant were included. This paper only included articles published in English.

RESULTS

Artificial intelligence (AI) refers to computer algorithms capable of learning, problem solving and decision-making. Throughout our review, we highlighted the role and importance of artificial intelligence in modern medicine, emphasizing its role in diagnosis through AI-assisted endoscopies and histology analysis and its enhancements in the treatment of ulcerative colitis. Despite these advances, AI is still hindered due to its current lack of adaptability to real-world scenarios and its difficulty in widespread data availability, which hinders the growth of AI-led data analysis.

CONCLUSIONS

When considering the potential of artificial intelligence, its ability to enhance patient care from a diagnostic and therapeutic perspective shows signs of promise. For the true utilization of artificial intelligence, some roadblocks must be addressed. The datasets available to AI may not truly reflect the real-world, which would prevent its impact in all clinical scenarios when dealing with a spectrum of patients with different backgrounds and presenting factors. Considering this, the shift in medical diagnostics and therapeutics is coinciding with evolving technology. With a continuous advancement in artificial intelligence programming and a perpetual surge in patient datasets, these networks can be further enhanced and supplemented with a greater cohort, enabling better outcomes and prediction models for the future of modern medicine.

Artificial intelligence in endoscopy related to inflammatory bowel disease: A systematic review

BACKGROUND AND OBJECTIVES

In spite of rapid growth of artificial intelligence (AI) in digestive endoscopy in lesion detection and characterization, the role of AI in inflammatory bowel disease (IBD) endoscopy is not clearly defined. We aimed at systematically reviewing the role of AI in IBD endoscopy and identifying future research areas.

METHODS

We searched the PubMed and Embase database using keywords ("artificial intelligence" OR "machine learning" OR "computer-aided" OR "convolutional neural network") AND ("inflammatory bowel disease" OR "ulcerative colitis" OR "Crohn's") AND ("endoscopy" or "colonoscopy" or "capsule endoscopy" or "device assisted enteroscopy") between 1975 and September 2023 and identified 62 original articles for detailed review. Review articles, consensus guidelines, case reports/series, editorials, letter to the editor, non-peer-reviewed pre-prints and conference abstracts were excluded. The quality of the included studies was assessed using the MI-CLAIM checklist.

RESULTS

The accuracy of AI models (25 studies) to assess ulcerative colitis (UC) endoscopic activity ranged between 86.54% and 94.5%. AI-assisted capsule endoscopy reading (12 studies) substantially reduced analyzable images and reading time with excellent accuracy (90.5% to 99.9%). AI-assisted analysis of colonoscopic images can help differentiate IBD from non-IBD, UC from non-UC and UC from Crohn's disease (CD) (three studies) with 72.1%, 98.3% and > 90% accuracy, respectively. AI models based on non-invasive clinical and radiologic parameters could predict endoscopic activity (three studies). AI-assisted virtual chromoendoscopy (four studies) could predict histologic remission and long-term outcomes. Computer-assisted detection (CADe) of dysplasia (two studies) is feasible along with AI-based differentiation of high from low-grade IBD neoplasia (79% accuracy). AI is effective in linking electronic medical record data (two studies) with colonoscopic videos to facilitate widespread machine learning.

CONCLUSION

AI-assisted IBD endoscopy has the potential to impact clinical management by automated detection and characterization of endoscopic lesions. Large, multi-center, prospective studies and commercially available IBD-specific endoscopic AI algorithms are warranted.

Impact of artificial intelligence on prognosis, shared decision-making, and precision medicine for patients with inflammatory bowel disease: a perspective and expert opinion

INTRODUCTION

Artificial intelligence (AI) is expected to impact all facets of inflammatory bowel disease (IBD) management, including disease assessment, treatment decisions, discovery and development of new biomarkers and therapeutics, as well as clinician-patient communication.

AREAS COVERED

This perspective paper provides an overview of the application of AI in the clinical management of IBD through a review of the currently available AI models that could be potential tools for prognosis, shared decision-making, and precision medicine. This overview covers models that measure treatment response based on statistical or machine-learning methods, or a combination of the two. We briefly discuss a computational model that allows integration of immune/biological system knowledge with mathematical modeling and also involves a 'digital twin', which allows measurement of temporal trends in mucosal inflammatory activity for predicting treatment response. A viewpoint on AI-enabled wearables and nearables and their use to improve IBD management is also included.

EXPERT OPINION

Although challenges regarding data quality, privacy, and security; ethical concerns; technical limitations; and regulatory barriers remain to be fully addressed, a growing body of evidence suggests a tremendous potential for integration of AI into daily clinical practice to enable precision medicine and shared decision-making.

Application of deep learning in the diagnosis and evaluation of ulcerative colitis disease severity

Background

Achieving endoscopic and histological remission is a critical treatment objective in ulcerative colitis (UC). Nevertheless, interobserver variability can significantly impact overall assessment performance.

Objectives

We aimed to develop a deep learning algorithm for the real-time and objective evaluation of endoscopic disease activity and prediction of histological remission in UC.

Design

This is a retrospective diagnostic study.

Methods

Two convolutional neural network (CNN) models were constructed and trained using 12,257 endoscopic images and biopsy results sourced from 1124 UC patients who underwent colonoscopy at a single center from January 2018 to December 2022. Mayo Endoscopy Subscore (MES) and UC Endoscopic Index of Severity Score (UCEIS) assessments were conducted by two experienced and independent reviewers. Model performance was evaluated in terms of accuracy, sensitivity, and positive predictive value. The output of the CNN models was also compared with the corresponding histological results to assess histological remission prediction performance.

Results

The MES-CNN model achieved 97.04% accuracy in diagnosing endoscopic remission of UC, while the MES-CNN and UCEIS-CNN models achieved 90.15% and 85.29% accuracy, respectively, in evaluating endoscopic severity of UC. For predicting histological remission, the CNN models achieved accuracy and kappa values of 91.28% and 0.826, respectively, attaining higher accuracy than human endoscopists (87.69%).

Conclusion

The proposed artificial intelligence model, based on MES and UCEIS evaluations from expert gastroenterologists, offered precise assessment of inflammation in UC endoscopic images and reliably predicted histological remission.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

Endoscopic remission is an important therapeutic goal in ulcerative colitis (UC). The Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Mayo Endoscopic Score (MES) are the commonly used endoscopic scoring criteria. This systematic review and meta-analysis aimed to evaluate the accuracy of artificial intelligence (AI) in diagnosing endoscopic remission in UC. We also performed a meta-analysis of each of the four endoscopic remission criteria (UCEIS = 0, MES = 0, UCEIS = <1, MES = <1). Eighteen studies involving 13,687 patients were included. The combined sensitivity and specificity of AI for diagnosing endoscopic remission in UC was 87% (95% confidence interval [CI]:81-92%) and 92% (95% CI: 89-94%), respectively. The area under the curve (AUC) was 0.96 (95% CI: 0.94-0.97). The results showed that the AI model performed well regardless of which criteria were used to define endoscopic remission of UC.

Using a Deep Learning Model to Address Interobserver Variability in the Evaluation of Ulcerative Colitis (UC) Severity

The use of endoscopic images for the accurate assessment of ulcerative colitis (UC) severity is crucial to determining appropriate treatment. However, experts may interpret these images differently, leading to inconsistent diagnoses. This study aims to address the issue by introducing a standardization method based on deep learning. We collected 254 rectal endoscopic images from 115 patients with UC, and five experts in endoscopic image interpretation assigned classification labels based on the Ulcerative Colitis Endoscopic Index of Severity (UCEIS) scoring system. Interobserver variance analysis of the five experts yielded an intraclass correlation coefficient of 0.8431 for UCEIS scores and a kappa coefficient of 0.4916 when the UCEIS scores were transformed into UC severity measures. To establish a consensus, we created a model that considered only the images and labels on which more than half of the experts agreed. This consensus model achieved an accuracy of 0.94 when tested with 50 images. Compared with models trained from individual expert labels, the consensus model demonstrated the most reliable prediction results.

Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning

BACKGROUND

Assessment of endoscopic activity in ulcerative colitis (UC) is important for treatment decisions and monitoring disease progress. However, substantial inter- and intraobserver variability in grading impairs the assessment. Our aim was to develop a computer-aided diagnosis system using deep learning to reduce subjectivity and improve the reliability of the assessment.

METHODS

The cohort comprises 11 276 images from 564 patients who underwent colonoscopy for UC. We propose a regression-based deep learning approach for the endoscopic evaluation of UC according to the Mayo endoscopic score (MES). Five state-of-the-art convolutional neural network (CNN) architectures were used for the performance measurements and comparisons. Ten-fold cross-validation was used to train the models and objectively benchmark them. Model performances were assessed using quadratic weighted kappa and macro F1 scores for full Mayo score classification and kappa statistics and F1 score for remission classification.

RESULTS

Five classification-based CNNs used in the study were in excellent agreement with the expert annotations for all Mayo subscores and remission classification according to the kappa statistics. When the proposed regression-based approach was used, (1) the performance of most of the models statistically significantly increased and (2) the same model trained on different cross-validation folds produced more robust results on the test set in terms of deviation between different folds.

CONCLUSIONS

Comprehensive experimental evaluations show that commonly used classification-based CNN architectures have successful performance in evaluating endoscopic disease activity of UC. Integration of domain knowledge into these architectures further increases performance and robustness, accelerating their translation into clinical use.

CB-HRNet: A Class-Balanced High-Resolution Network for the evaluation of endoscopic activity in patients with ulcerative colitis

Endoscopic evaluation is the key to the management of ulcerative colitis (UC). However, there is interobserver variability in interpreting endoscopic images among gastroenterologists. Furthermore, it is time-consuming. Convolutional neural networks (CNNs) can help overcome these obstacles and has yielded preliminary positive results. We aimed to develop a new CNN-based algorithm to improve the performance for evaluation tasks of endoscopic images in patients with UC. A total of 12,163 endoscopic images from 308 patients with UC were collected from January 2014 to December 2021. The training set and test set images were randomly divided into 37,515 and 3191 after excluding possible interference and data augmentation. Mayo Endoscopic Subscores (MES) were predicted by different CNN-based models with different loss functions. Their performances were evaluated by several metrics. After comparing the results of different CNN-based models with different loss functions, High-Resolution Network with Class-Balanced Loss achieved the best performances in all MES classification subtasks. It was especially great at determining endoscopic remission in UC, which achieved a high accuracy of 95.07% and good performances in other evaluation metrics with sensitivity 92.87%, specificity 95.41%, kappa coefficient 0.8836, positive predictive value 93.44%, negative predictive value 95.00% and area value under the receiver operating characteristic curve 0.9834, respectively. In conclusion, we proposed a new CNN-based algorithm, Class-Balanced High-Resolution Network (CB-HRNet), to evaluate endoscopic activity of UC with excellent performance. Besides, we made an open-source dataset and it can be a new benchmark in the task of MES classification.

Diagnostic accuracy of convolutional neural network-based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis

BACKGROUND AND AIMS

Endoscopic assessment of ulcerative colitis (UC) can be performed by using the Mayo Endoscopic Score (MES) or the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). In this meta-analysis, we assessed the pooled diagnostic accuracy parameters of deep machine learning by means of convolutional neural network (CNN) algorithms in predicting UC severity on endoscopic images.

METHODS

Databases including MEDLINE, Scopus, and Embase were searched in June 2022. Outcomes of interest were the pooled accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Standard meta-analysis methods used the random-effects model, and heterogeneity was assessed using the I2statistics.

RESULTS

Twelve studies were included in the final analysis. The pooled diagnostic parameters of CNN-based machine learning algorithms in endoscopic severity assessment of UC were as follows: accuracy 91.5% (95% confidence interval [CI], 88.3-93.8; I2 = 84%), sensitivity 82.8% (95% CI, 78.3-86.5; I2 = 89%), specificity 92.4% (95% CI, 89.4-94.6; I2 = 84%), PPV 86.6% (95% CI, 82.3-90; I2 = 89%), and NPV 88.6% (95% CI, 85.7-91; I2 = 78%). Subgroup analysis revealed significantly better sensitivity and PPV with the UCEIS scoring system compared with the MES (93.6% [95% CI, 87.5-96.8; I2 = 77%] vs 82% [95% CI, 75.6-87; I2 = 89%], P = .003, and 93.6% [95% CI, 88.7-96.4; I2 = 68%] vs 83.6% [95% CI, 76.8-88.8; I2 = 77%], P = .007, respectively).

CONCLUSIONS

CNN-based machine learning algorithms demonstrated excellent pooled diagnostic accuracy parameters in the endoscopic severity assessment of UC. Using UCEIS scores in CNN training might offer better results than the MES. Further studies are warranted to establish these findings in real clinical settings.

Deep learning model for distinguishing Mayo endoscopic subscore 0 and 1 in patients with ulcerative colitis

The aim of this study was to address the issue of differentiating between Mayo endoscopic subscore (MES) 0 and MES 1 using a deep learning model. A dataset of 492 ulcerative colitis (UC) patients who demonstrated MES improvement between January 2018 and December 2019 at Samsung Medical Center was utilized. Specifically, two representative images of the colon and rectum were selected from each patient, resulting in a total of 984 images for analysis. The deep learning model utilized in this study consisted of a convolutional neural network (CNN)-based encoder, with two auxiliary classifiers for the colon and rectum, as well as a final MES classifier that combined image features from both inputs. In the internal test, the model achieved an F1-score of 0.92, surpassing the performance of seven novice classifiers by an average margin of 0.11, and outperforming their consensus by 0.02. The area under the receiver operating characteristic curve (AUROC) was calculated to be 0.97 when considering MES 1 as positive, with an area under the precision-recall curve (AUPRC) of 0.98. In the external test using the Hyperkvasir dataset, the model achieved an F1-score of 0.89, AUROC of 0.86, and AUPRC of 0.97. The results demonstrate that the proposed CNN-based model, which integrates image features from both the colon and rectum, exhibits superior performance in accurately discriminating between MES 0 and MES 1 in patients with UC.

Development and validation of a deep learning-based approach to predict the Mayo endoscopic score of ulcerative colitis

Background

The ulcerative colitis (UC) Mayo endoscopy score is a useful tool for evaluating the severity of UC in patients in clinical practice.

Objectives

We aimed to develop and validate a deep learning-based approach to automatically predict the Mayo endoscopic score using UC endoscopic images.

Design

A multicenter, diagnostic retrospective study.

Methods

We collected 15120 colonoscopy images of 768 UC patients from two hospitals in China and developed a deep model based on a vision transformer named the UC-former. The performance of the UC-former was compared with that of six endoscopists on the internal test set. Furthermore, multicenter validation from three hospitals was also carried out to evaluate UC-former's generalization performance.

Results

On the internal test set, the areas under the curve of Mayo 0, Mayo 1, Mayo 2, and Mayo 3 achieved by the UC-former were 0.998, 0.984, 0.973, and 0.990, respectively. The accuracy (ACC) achieved by the UC-former was 90.8%, which is higher than that achieved by the best senior endoscopist. For three multicenter external validations, the ACC was 82.4%, 85.0%, and 83.6%, respectively.

Conclusions

The developed UC-former could achieve high ACC, fidelity, and stability to evaluate the severity of UC, which may provide potential application in clinical practice.

Registration

This clinical trial was registered at the ClinicalTrials.gov (trial registration number: NCT05336773).

The Role of Artificial Intelligence in Monitoring Inflammatory Bowel Disease-The Future Is Now

Crohn's disease and ulcerative colitis remain debilitating disorders, characterized by progressive bowel damage and possible lethal complications. The growing number of applications for artificial intelligence in gastrointestinal endoscopy has already shown great potential, especially in the field of neoplastic and pre-neoplastic lesion detection and characterization, and is currently under evaluation in the field of inflammatory bowel disease management. The application of artificial intelligence in inflammatory bowel diseases can range from genomic dataset analysis and risk prediction model construction to the disease grading severity and assessment of the response to treatment using machine learning. We aimed to assess the current and future role of artificial intelligence in assessing the key outcomes in inflammatory bowel disease patients: endoscopic activity, mucosal healing, response to treatment, and neoplasia surveillance.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

PHF3 Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images

Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework (PHF3) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the PHF3 model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed PHF3 model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity.

Artificial intelligence in gastrointestinal endoscopy - Evolution to a new era

Artificial intelligence (AI) systems based on machine learning have evolved in the last few years with an increasing applicability in gastrointestinal endoscopy. Thanks to AI, an image (input) can be transformed into a clinical decision (output). Although AI systems have been initially studied to improve detection (CADe) and characterization of colorectal lesions (CADx), other indications are being currently investigated as detection of blind spots, scope guidance, or delineation/measurement of lesions. The objective of these review is to summarize the current evidence on applicability of AI systems in gastrointestinal endoscopy, highlight strengths and limitations of the technology and review regulatory and ethical aspects for its general implementation in gastrointestinal endoscopy.

Constrained multiple instance learning for ulcerative colitis prediction using histological images

BACKGROUND AND OBJECTIVE

Ulcerative colitis (UC) is an inflammatory bowel disease (IBD) affecting the colon and the rectum characterized by a remitting-relapsing course. To detect mucosal inflammation associated with UC, histology is considered the most stringent criteria. In turn, histologic remission (HR) correlates with improved clinical outcomes and has been recently recognized as a desirable treatment target. The leading biomarker for assessing histologic remission is the presence or absence of neutrophils. Therefore, the finding of this cell in specific colon structures indicates that the patient has UC activity. However, no previous studies based on deep learning have been developed to identify UC based on neutrophils detection using whole-slide images (WSI).

METHODS

The methodological core of this work is a novel multiple instance learning (MIL) framework with location constraints able to determine the presence of UC activity using WSI. In particular, we put forward an effective way to introduce constraints about positive instances to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. In addition, we propose a new weighted embedding to enlarge the relevance of the positive instances.

RESULTS

Extensive experiments on a multi-center dataset of colon and rectum WSIs, PICASSO-MIL, demonstrate that using the location information we can improve considerably the results at WSI-level. In comparison with prior MIL settings, our method allows for 10% improvements in bag-level accuracy.

CONCLUSION

Our model, which introduces a new form of constraints, surpass the results achieved from current state-of-the-art methods that focus on the MIL paradigm. Our method can be applied to other histological concerns where the morphological features determining a positive WSI are tiny and similar to others in the image.

Clinical application and diagnostic accuracy of artificial intelligence in colonoscopy for inflammatory bowel disease: systematic review

Background and aims  Artificial intelligence (AI) technology is being evaluated for its potential to improve colonoscopic assessment of inflammatory bowel disease (IBD), particularly with computer-aided image classifiers. This review evaluates the clinical application and diagnostic test accuracy (DTA) of AI algorithms in colonoscopy for IBD. Methods  A systematic review was performed on studies evaluating AI in colonoscopy of adult patients with IBD. MEDLINE, Embase, Emcare, PsycINFO, CINAHL, Cochrane Library and Clinicaltrials.gov databases were searched on 28 ^th April 2021 for English language articles published between January 1, 2000 and April 28, 2021. Risk of bias and applicability were assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool. Diagnostic accuracy was presented as median (interquartile range). Results  Of 1029 records screened, nine studies with 7813 patients were included for review. AI was used to predict endoscopic and histologic disease activity in ulcerative colitis, and differentiation of Crohn's disease from Behcet's disease and intestinal tuberculosis. DTA of AI algorithms ranged between 52-91 %. The sensitivity and specificity for AI algorithms predicting endoscopic severity of disease were 78 % (range 72-83, interquartile range 5.5) and 91 % (range 86-96, interquartile range 5), respectively. Conclusions  AI has been primarily used to assess disease activity in ulcerative colitis. The diagnostic performance is promising and suggests potential for other clinical application of AI in IBD colonoscopy such as dysplasia detection. However, current evidence is limited by retrospective data and models trained on still images only. Future prospective multicenter studies with full-motion videos are needed to replicate the real-world clinical setting.

Rapid development of accurate artificial intelligence scoring for colitis disease activity using applied data science techniques

Background and study aims  Scoring endoscopic disease activity in colitis represents a complex task for artificial intelligence (AI), but is seen as a worthwhile goal for clinical and research use cases. To date, development attempts have relied on large datasets, achieving reasonable results when comparing normal to active inflammation, but not when generating subscores for the Mayo Endoscopic Score (MES) or ulcerative colitis endoscopic index of severity (UCEIS). Patients and methods  Using a multi-task learning framework, with frame-by-frame analysis, we developed a machine-learning algorithm (MLA) for UCEIS trained on just 38,124 frames (73 patients with biopsy-proven ulcerative colitis). Scores generated by the MLA were compared to consensus scores from three independent human reviewers. Results  Accuracy and agreement (kappa) were calculated for the following differentiation tasks: (1) normal mucosa vs active inflammation (UCEIS 0 vs ≥ 1; accuracy 0.90, κ = 0.90); (2) mild inflammation vs moderate-severe (UCEIS 0-3 vs ≥ 4; accuracy 0.98, κ = 0.96); (3) generating total UCEIS score (κ = 0.92). Agreement for UCEIS subdomains was also high (κ = 0.80, 0.83 and 0.88 for vascular pattern, bleeding and erosions respectively). Conclusions  We have demonstrated that, using modified data science techniques and a relatively smaller datasets, it is possible to achieve high levels of accuracy and agreement with human reviewers (in some cases near-perfect), for AI in colitis scoring. Further work will focus on refining this technique, but we hope that it can be used in other tasks to facilitate faster development.

Artificial Endoscopy and Inflammatory Bowel Disease: Welcome to the Future

Artificial intelligence (AI) is assuming an increasingly important and central role in several medical fields. Its application in endoscopy provides a powerful tool supporting human experiences in the detection, characterization, and classification of gastrointestinal lesions. Lately, the potential of AI technology has been emerging in the field of inflammatory bowel disease (IBD), where the current cornerstone is the treat-to-target strategy. A sensible and specific tool able to overcome human limitations, such as AI, could represent a great ally and guide precision medicine decisions. Here we reviewed the available literature on the endoscopic applications of AI in order to properly describe the current state-of-the-art and identify the research gaps in IBD at the dawn of 2022.

Utility of artificial intelligence in colonoscopy

Colorectal cancer is one of the major causes of death worldwide. Colonoscopy is the most important tool that can identify neoplastic lesion in early stages and resect it in a timely manner which helps in reducing mortality related to colorectal cancer. However, the quality of colonoscopy findings depends on the expertise of the endoscopist and thus the rate of missed adenoma or polyp cannot be controlled. It is desirable to standardize the quality of colonoscopy by reducing the number of missed adenoma/polyps. Introduction of artificial intelligence (AI) in the field of medicine has become popular among physicians nowadays. The application of AI in colonoscopy can help in reducing miss rate and increasing colorectal cancer detection rate as per recent studies. Moreover, AI assistance during colonoscopy has also been utilized in patients with inflammatory bowel disease to improve diagnostic accuracy, assessing disease severity and predicting clinical outcomes. We conducted a literature review on the available evidence on use of AI in colonoscopy. In this review article, we discuss about the principles, application, limitations, and future aspects of AI in colonoscopy.

Artificial intelligence applications in inflammatory bowel disease: Emerging technologies and future directions

Inflammatory bowel disease (IBD) is a complex and multifaceted disorder of the gastrointestinal tract that is increasing in incidence worldwide and associated with significant morbidity. The rapid accumulation of large datasets from electronic health records, high-definition multi-omics (including genomics, proteomics, transcriptomics, and metagenomics), and imaging modalities (endoscopy and endomicroscopy) have provided powerful tools to unravel novel mechanistic insights and help address unmet clinical needs in IBD. Although the application of artificial intelligence (AI) methods has facilitated the analysis, integration, and interpretation of large datasets in IBD, significant heterogeneity in AI methods, datasets, and clinical outcomes and the need for unbiased prospective validations studies are current barriers to incorporation of AI into clinical practice. The purpose of this review is to summarize the most recent advances in the application of AI and machine learning technologies in the diagnosis and risk prediction, assessment of disease severity, and prediction of clinical outcomes in patients with IBD.

Artificial intelligence in inflammatory bowel disease endoscopy: current landscape and the road ahead

Inflammatory bowel disease is a complex chronic inflammatory disorder with challenges in diagnosis, choosing appropriate therapy, determining individual responsiveness, and prediction of future disease course to guide appropriate management. Artificial intelligence has been examined in the field of inflammatory bowel disease endoscopy with promising data in different domains of inflammatory bowel disease, including diagnosis, assessment of mucosal activity, and prediction of recurrence and complications. Artificial intelligence use during endoscopy could be a step toward precision medicine in inflammatory bowel disease care pathways. We reviewed available data on use of artificial intelligence for diagnosis of inflammatory bowel disease, grading of severity, prediction of recurrence, and dysplasia detection. We examined the potential role of artificial intelligence enhanced endoscopy in various aspects of inflammatory bowel disease care and future perspectives in this review.