Development and Validation of a Deep Neural Network for Accurate Evaluation of Endoscopic Images From Patients With Ulcerative Colitis

Rapid Endoscopic Diagnosis of Benign Ulcerative Colorectal Diseases With an Artificial Intelligence Contextual Framework

BACKGROUND & AIMS

Benign ulcerative colorectal diseases (UCDs) such as ulcerative colitis, Crohn's disease, ischemic colitis, and intestinal tuberculosis share similar phenotypes with different etiologies and treatment strategies. To accurately diagnose closely related diseases like UCDs, we hypothesize that contextual learning is critical in enhancing the ability of the artificial intelligence models to differentiate the subtle differences in lesions amidst the vastly divergent spatial contexts.

METHODS

White-light colonoscopy datasets of patients with confirmed UCDs and healthy controls were retrospectively collected. We developed a Multiclass Contextual Classification (MCC) model that can differentiate among the mentioned UCDs and healthy controls by incorporating the tissue object contexts surrounding the individual lesion region in a scene and spatial information from other endoscopic frames (video-level) into a unified framework. Internal and external datasets were used to validate the model's performance.

RESULTS

Training datasets included 762 patients, and the internal and external testing cohorts included 257 patients and 293 patients, respectively. Our MCC model provided a rapid reference diagnosis on internal test sets with a high averaged area under the receiver operating characteristic curve (image-level: 0.950 and video-level: 0.973) and balanced accuracy (image-level: 76.1% and video-level: 80.8%), which was superior to junior endoscopists (accuracy: 71.8%, P < .0001) and similar to experts (accuracy: 79.7%, P = .732). The MCC model achieved an area under the receiver operating characteristic curve of 0.988 and balanced accuracy of 85.8% using external testing datasets.

CONCLUSIONS

These results enable this model to fit in the routine endoscopic workflow, and the contextual framework to be adopted for diagnosing other closely related diseases.

Artificial intelligence and endo-histo-omics: new dimensions of precision endoscopy and histology in inflammatory bowel disease

Integrating artificial intelligence into inflammatory bowel disease (IBD) has the potential to revolutionise clinical practice and research. Artificial intelligence harnesses advanced algorithms to deliver accurate assessments of IBD endoscopy and histology, offering precise evaluations of disease activity, standardised scoring, and outcome prediction. Furthermore, artificial intelligence offers the potential for a holistic endo-histo-omics approach by interlacing and harmonising endoscopy, histology, and omics data towards precision medicine. The emerging applications of artificial intelligence could pave the way for personalised medicine in IBD, offering patient stratification for the most beneficial therapy with minimal risk. Although artificial intelligence holds promise, challenges remain, including data quality, standardisation, reproducibility, scarcity of randomised controlled trials, clinical implementation, ethical concerns, legal liability, and regulatory issues. The development of standardised guidelines and interdisciplinary collaboration, including policy makers and regulatory agencies, is crucial for addressing these challenges and advancing artificial intelligence in IBD clinical practice and trials.

How Artificial Intelligence Will Transform Clinical Care, Research, and Trials for Inflammatory Bowel Disease

Artificial intelligence (AI) refers to computer-based methodologies that use data to teach a computer to solve pre-defined tasks; these methods can be applied to identify patterns in large multi-modal data sources. AI applications in inflammatory bowel disease (IBD) includes predicting response to therapy, disease activity scoring of endoscopy, drug discovery, and identifying bowel damage in images. As a complex disease with entangled relationships between genomics, metabolomics, microbiome, and the environment, IBD stands to benefit greatly from methodologies that can handle this complexity. We describe current applications, critical challenges, and propose future directions of AI in IBD.

Deep Bayesian active learning-to-rank with relative annotation for estimation of ulcerative colitis severity

Automatic image-based severity estimation is an important task in computer-aided diagnosis. Severity estimation by deep learning requires a large amount of training data to achieve a high performance. In general, severity estimation uses training data annotated with discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult in images with ambiguous severity, and the annotation cost is high. In contrast, relative annotation, in which the severity between a pair of images is compared, can avoid quantizing severity and thus makes it easier. We can estimate relative disease severity using a learning-to-rank framework with relative annotations, but relative annotation has the problem of the enormous number of pairs that can be annotated. Therefore, the selection of appropriate pairs is essential for relative annotation. In this paper, we propose a deep Bayesian active learning-to-rank that automatically selects appropriate pairs for relative annotation. Our method preferentially annotates unlabeled pairs with high learning efficiency from the model uncertainty of the samples. We prove the theoretical basis for adapting Bayesian neural networks to pairwise learning-to-rank and demonstrate the efficiency of our method through experiments on endoscopic images of ulcerative colitis on both private and public datasets. We also show that our method achieves a high performance under conditions of significant class imbalance because it automatically selects samples from the minority classes.

Can artificial intelligence replace endoscopists when assessing mucosal healing in ulcerative colitis? A systematic review and diagnostic test accuracy meta-analysis

BACKGROUNDS AND AIMS

Mucosal healing (MH) in inflammatory bowel diseases (IBD) is an important landmark for clinical decision making. Artificial intelligence systems (AI) that automatically deliver the grade of endoscopic inflammation may solve moderate interobserver agreement and the need of central reading in clinical trials.

METHODS

We performed a systematic review of EMBASE and MEDLINE databases up to 01/12/2022 following PRISMA and the Joanna Briggs Institute methodologies to answer the following question: "Can AI replace endoscopists when assessing MH in IBD?". The research was restricted to ulcerative colitis (UC), and a diagnostic odds ratio (DOR) meta-analysis was performed. Risk of bias was evaluated with QUADAS-2 tool.

RESULTS

A total of 21 / 739 records were selected for full text evaluation, and 12 were included in the meta-analysis. Deep learning algorithms based on convolutional neural networks architecture achieved a satisfactory performance in evaluating MH on UC, with sensitivity, specificity, DOR and SROC of respectively 0.91(CI95 %:0.86-0.95);0.89(CI95 %:0.84-0.93);92.42(CI95 %:54.22-157.53) and 0.957 when evaluating fixed images (n = 8) and 0.86(CI95 %:0.75-0.93);0.91(CI95 %:0.87-0.94);70.86(CI95 %:24.63-203.86) and 0.941 when evaluating videos (n = 6). Moderate-high levels of heterogeneity were noted, limiting the quality of the evidence.

CONCLUSIONS

AI systems showed high potential in detecting MH in UC with optimal diagnostic performance, although moderate-high heterogeneity of the data was noted. Standardised and shared AI training may reduce heterogeneity between systems.

A novel artificial intelligence-assisted "vascular healing" diagnosis for prediction of future clinical relapse in patients with ulcerative colitis: a prospective cohort study (with video)

BACKGROUND AND AIMS

Image-enhanced endoscopy has attracted attention as a method for detecting inflammation and predicting outcomes in patients with ulcerative colitis (UC); however, the procedure requires specialist endoscopists. Artificial intelligence (AI)-assisted image-enhanced endoscopy may help nonexperts provide objective accurate predictions with the use of optical imaging. We aimed to develop a novel AI-based system using 8853 images from 167 patients with UC to diagnose "vascular-healing" and establish the role of AI-based vascular-healing for predicting the outcomes of patients with UC.

METHODS

This open-label prospective cohort study analyzed data for 104 patients with UC in clinical remission. Endoscopists performed colonoscopy using the AI system, which identified the target mucosa as AI-based vascular-active or vascular-healing. Mayo endoscopic subscore (MES), AI outputs, and histologic assessment were recorded for 6 colorectal segments from each patient. Patients were followed up for 12 months. Clinical relapse was defined as a partial Mayo score >2 RESULTS: The clinical relapse rate was significantly higher in the AI-based vascular-active group (23.9% [16/67]) compared with the AI-based vascular-healing group (3.0% [1/33)]; P = .01). In a subanalysis predicting clinical relapse in patients with MES ≤1, the area under the receiver operating characteristic curve for the combination of complete endoscopic remission and vascular healing (0.70) was increased compared with that for complete endoscopic remission alone (0.65).

CONCLUSIONS

AI-based vascular-healing diagnosis system may potentially be used to provide more confidence to physicians to accurately identify patients in remission of UC who would likely relapse rather than remain stable.

Artificial intelligence-assisted colonoscopy to identify histologic remission and predict the outcomes of patients with ulcerative colitis: A systematic review

This systematic review evaluated the current status of AI-assisted colonoscopy to identify histologic remission and predict the clinical outcomes of patients with ulcerative colitis. The use of artificial intelligence (AI) has increased substantially across several medical fields, including gastrointestinal endoscopy. Evidence suggests that it may be helpful to predict histologic remission and relapse, which would be beneficial because current histological diagnosis is limited by the inconvenience of obtaining biopsies and the high cost and time-intensiveness of pathological diagnosis. MEDLINE and the Cochrane Central Register of Controlled Trials were searched for studies published between January 1, 2000, and October 31, 2023. Nine studies fulfilled the selection criteria and were included; five evaluated the prediction of histologic remission, two assessed the prediction of clinical outcomes, and two evaluated both. Seven were prospective observational or cohort studies, while two were retrospective observational studies. No randomized controlled trials were identified. AI-assisted colonoscopy demonstrated sensitivity between 65 %-98 % and specificity values of 80 %-97 % for identifying histologic remission. Furthermore, it was able to predict future relapse in patients with ulcerative colitis. However, several challenges and barriers still exist to its routine clinical application, which should be overcome before the true potential of AI-assisted colonoscopy can be fully realized.

Effects of ultrasound with an automatic vessel detection system using artificial intelligence on the selection of puncture points among ultrasound beginner clinical nurses

BACKGROUND

Ultrasound guidance increases the success rate of peripheral intravenous catheter placement. However, the longer time required to obtain ultrasound-guided access poses difficulties for ultrasound beginners. Notably, interpretation of ultrasonographic images is considered as one of the main reasons of difficulty in using ultrasound for catheter placement. Therefore, an automatic vessel detection system (AVDS) using artificial intelligence was developed. This study aimed to investigate the effectiveness of AVDS for ultrasound beginners in selecting puncture points and determine suitable users for this system.

METHODS

In this crossover experiment involving the use of ultrasound with and without AVDS, we enrolled 10 clinical nurses, including 5 with some experience in peripheral intravenous catheterization using ultrasound-aided methods (categorized as ultrasound beginners) and 5 with no experience in ultrasound and less experience in peripheral intravenous catheterization using conventional methods (categorized as inexperienced). These participants chose two puncture points (those with the largest and second largest diameter) as ideal in each forearm of a healthy volunteer. The results of this study were the time required for the selection of puncture points and the vein diameter of the selected points.

RESULTS

Among ultrasound beginners, the time required for puncture point selection in the right forearm second candidate vein with a small diameter (<3 mm) was significantly shorter when using ultrasound with AVDS than when using it without AVDS (mean, 87 vs 247 s). Among inexperienced nurses, no significant difference in the time required for all puncture point selections was found between the use of ultrasound with and without AVDS. In the vein diameter, significant difference was shown only in the absolute difference at left second candidate among inexperienced participants.

CONCLUSION

Ultrasonography beginners needed less time to select the puncture points in a small diameter vein using ultrasound with AVDS than without AVDS.

Development of an artificial intelligent model for pre-endoscopic screening of precancerous lesions in gastric cancer

BACKGROUND

Given the high cost of endoscopy in gastric cancer (GC) screening, there is an urgent need to explore cost-effective methods for the large-scale prediction of precancerous lesions of gastric cancer (PLGC). We aim to construct a hierarchical artificial intelligence-based multimodal non-invasive method for pre-endoscopic risk screening, to provide tailored recommendations for endoscopy.

METHODS

From December 2022 to December 2023, a large-scale screening study was conducted in Fujian, China. Based on traditional Chinese medicine theory, we simultaneously collected tongue images and inquiry information from 1034 participants, considering the potential of these data for PLGC screening. Then, we introduced inquiry information for the first time, forming a multimodality artificial intelligence model to integrate tongue images and inquiry information for pre-endoscopic screening. Moreover, we validated this approach in another independent external validation cohort, comprising 143 participants from the China-Japan Friendship Hospital.

RESULTS

A multimodality artificial intelligence-assisted pre-endoscopic screening model based on tongue images and inquiry information (AITonguequiry) was constructed, adopting a hierarchical prediction strategy, achieving tailored endoscopic recommendations. Validation analysis revealed that the area under the curve (AUC) values of AITonguequiry were 0.74 for overall PLGC (95% confidence interval (CI) 0.71-0.76, p < 0.05) and 0.82 for high-risk PLGC (95% CI 0.82-0.83, p < 0.05), which were significantly and robustly better than those of the independent use of either tongue images or inquiry information alone. In addition, AITonguequiry has superior performance compared to existing PLGC screening methodologies, with the AUC value enhancing 45% in terms of PLGC screening (0.74 vs. 0.51, p < 0.05) and 52% in terms of high-risk PLGC screening (0.82 vs. 0.54, p < 0.05). In the independent external verification, the AUC values were 0.69 for PLGC and 0.76 for high-risk PLGC.

CONCLUSION

Our AITonguequiry artificial intelligence model, for the first time, incorporates inquiry information and tongue images, leading to a higher precision and finer-grained pre-endoscopic screening of PLGC. This enhances patient screening efficiency and alleviates patient burden.

Fecal calprotectin and endoscopic scores: The cornerstones in clinical practice for evaluating mucosal healing in inflammatory bowel disease

Managing inflammatory bowel disease (IBD) is becoming increasingly complex and personalized, considering the advent of new advanced therapies with distinct mechanisms of action. Achieving mucosal healing (MH) is a pivotal therapeutic goal in IBD management and can prevent IBD progression and reduce flares, hospitalization, surgery, intestinal damage, and colorectal cancer. Employing proactive disease and therapy assessment is essential to achieve better control of intestinal inflammation, even if subclinical, to alter the natural course of IBD. Periodic monitoring of fecal calprotectin (FC) levels and interval endoscopic evaluations are cornerstones for evaluating response/remission to advanced therapies targeting IBD, assessing MH, and detecting subclinical recurrence. Here, we comment on the article by Ishida et al Moreover, this editorial aimed to review the role of FC and endoscopic scores in predicting MH in patients with IBD. Furthermore, we intend to present some evidence on the role of these markers in future targets, such as histological and transmural healing. Additional prospective multicenter studies with a stricter MH criterion, standardized endoscopic and histopathological analyses, and virtual chromoscopy, potentially including artificial intelligence and other biomarkers, are desired.

The development of artificial intelligence in the histological diagnosis of Inflammatory Bowel Disease (IBD-AI)

BACKGROUND

Inflammatory bowel disease (IBD) includes Crohn's Disease (CD) and Ulcerative Colitis (UC). Correct diagnosis requires the identification of precise morphological features such basal plasmacytosis. However, histopathological interpretation can be challenging, and it is subject to high variability.

AIM

The IBD-Artificial Intelligence (AI) project aims at the development of an AI-based evaluation system to support the diagnosis of IBD, semi-automatically quantifying basal plasmacytosis.

METHODS

A deep learning model was trained to detect and quantify plasma cells on a public dataset of 4981 annotated images. The model was then tested on an external validation cohort of 356 intestinal biopsies of CD, UC and healthy controls. AI diagnostic performance was calculated compared to human gold standard.

RESULTS

The system correctly found that CD and UC samples had a greater prevalence of basal plasma cells with mean number of PCs within ROIs of 38.22 (95 % CI: 31.73, 49.04) for CD, 55.16 (46.57, 65.93) for UC, and 17.25 (CI: 12.17, 27.05) for controls. Overall, OR=4.968 (CI: 1.835, 14.638) was found for IBD compared to normal mucosa (CD: +59 %; UC: +129 %). Additionally, as expected, UC samples were found to have more plasma cells in colon than CD cases.

CONCLUSION

Our model accurately replicated human assessment of basal plasmacytosis, underscoring the value of AI models as a potential aid IBD diagnosis.

Challenges in IBD Research 2024: Precision Medicine

Precision medicine is part of 5 focus areas of the Challenges in IBD Research 2024 research document, which also includes preclinical human IBD mechanisms, environmental triggers, novel technologies, and pragmatic clinical research. Building on Challenges in IBD Research 2019, the current Challenges aims to provide a comprehensive overview of current gaps in inflammatory bowel diseases (IBDs) research and deliver actionable approaches to address them with a focus on how these gaps can lead to advancements in interception, remission, and restoration for these diseases. The document is the result of multidisciplinary input from scientists, clinicians, patients, and funders, and represents a valuable resource for patient-centric research prioritization. In particular, the precision medicine section is focused on the main research gaps in elucidating how to bring the best care to the individual patient in IBD. Research gaps were identified in biomarker discovery and validation for predicting disease progression and choosing the most appropriate treatment for each patient. Other gaps were identified in making the best use of existing patient biosamples and clinical data, developing new technologies to analyze large datasets, and overcoming regulatory and payer hurdles to enable clinical use of biomarkers. To address these gaps, the Workgroup suggests focusing on thoroughly validating existing candidate biomarkers, using best-in-class data generation and analysis tools, and establishing cross-disciplinary teams to tackle regulatory hurdles as early as possible. Altogether, the precision medicine group recognizes the importance of bringing basic scientific biomarker discovery and translating it into the clinic to help improve the lives of IBD patients.

Application of artificial intelligence in gastrointestinal endoscopy

Endoscopy is an important method for diagnosing gastrointestinal (GI) diseases. In this study, we provide an overview of the advances in artificial intelligence (AI) technology in the field of GI endoscopy over recent years, including esophagus, stomach, large intestine, and capsule endoscopy (small intestine). AI-assisted endoscopy shows high accuracy, sensitivity, and specificity in the detection and diagnosis of GI diseases at all levels. Hence, AI will make a breakthrough in the field of GI endoscopy in the near future. However, AI technology currently has some limitations and is still in the preclinical stages.

Artificial intelligence quantifying endoscopic severity of ulcerative colitis in gradation scale

OBJECTIVES

Existing endoscopic scores for ulcerative colitis (UC) objectively categorize disease severity based on the presence or absence of endoscopic findings; therefore, it may not reflect the range of clinical severity within each category. However, inflammatory bowel disease (IBD) expert endoscopists categorize the severity and diagnose the overall impression of the degree of inflammation. This study aimed to develop an artificial intelligence (AI) system that can accurately represent the assessment of the endoscopic severity of UC by IBD expert endoscopists.

METHODS

A ranking-convolutional neural network (ranking-CNN) was trained using comparative information on the UC severity of 13,826 pairs of endoscopic images created by IBD expert endoscopists. Using the trained ranking-CNN, the UC Endoscopic Gradation Scale (UCEGS) was used to express severity. Correlation coefficients were calculated to ensure that there were no inconsistencies in assessments of severity made using UCEGS diagnosed by the AI and the Mayo Endoscopic Subscore, and the correlation coefficients of the mean for test images assessed using UCEGS by four IBD expert endoscopists and the AI.

RESULTS

Spearman's correlation coefficient between the UCEGS diagnosed by AI and Mayo Endoscopic Subscore was approximately 0.89. The correlation coefficients between IBD expert endoscopists and the AI of the evaluation results were all higher than 0.95 (P < 0.01).

CONCLUSIONS

The AI developed here can diagnose UC severity endoscopically similar to IBD expert endoscopists.

Deep Learning Models Capture Histological Disease Activity in Crohn's Disease and Ulcerative Colitis with High Fidelity

BACKGROUND AND AIMS

Histological disease activity in inflammatory bowel disease [IBD] is associated with clinical outcomes and is an important endpoint in drug development. We developed deep learning models for automating histological assessments in IBD.

METHODS

Histology images of intestinal mucosa from phase 2 and phase 3 clinical trials in Crohn's disease [CD] and ulcerative colitis [UC] were used to train artificial intelligence [AI] models to predict the Global Histology Activity Score [GHAS] for CD and Geboes histopathology score for UC. Three AI methods were compared. AI models were evaluated on held-back testing sets, and model predictions were compared against an expert central reader and five independent pathologists.

RESULTS

The model based on multiple instance learning and the attention mechanism [SA-AbMILP] demonstrated the best performance among competing models. AI-modelled GHAS and Geboes subgrades matched central readings with moderate to substantial agreement, with accuracies ranging from 65% to 89%. Furthermore, the model was able to distinguish the presence and absence of pathology across four selected histological features, with accuracies for colon in both CD and UC ranging from 87% to 94% and for CD ileum ranging from 76% to 83%. For both CD and UC and across anatomical compartments [ileum and colon] in CD, comparable accuracies against central readings were found between the model-assigned scores and scores by an independent set of pathologists.

CONCLUSIONS

Deep learning models based upon GHAS and Geboes scoring systems were effective at distinguishing between the presence and absence of IBD microscopic disease activity.

Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review

BACKGROUND

Generative artificial intelligence tools and applications (GenAI) are being increasingly used in health care. Physicians, specialists, and other providers have started primarily using GenAI as an aid or tool to gather knowledge, provide information, train, or generate suggestive dialogue between physicians and patients or between physicians and patients' families or friends. However, unless the use of GenAI is oriented to be helpful in clinical service encounters that can improve the accuracy of diagnosis, treatment, and patient outcomes, the expected potential will not be achieved. As adoption continues, it is essential to validate the effectiveness of the infusion of GenAI as an intelligent technology in service encounters to understand the gap in actual clinical service use of GenAI.

OBJECTIVE

This study synthesizes preliminary evidence on how GenAI assists, guides, and automates clinical service rendering and encounters in health care The review scope was limited to articles published in peer-reviewed medical journals.

METHODS

We screened and selected 0.38% (161/42,459) of articles published between January 1, 2020, and May 31, 2023, identified from PubMed. We followed the protocols outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select highly relevant studies with at least 1 element on clinical use, evaluation, and validation to provide evidence of GenAI use in clinical services. The articles were classified based on their relevance to clinical service functions or activities using the descriptive and analytical information presented in the articles.

RESULTS

Of 161 articles, 141 (87.6%) reported using GenAI to assist services through knowledge access, collation, and filtering. GenAI was used for disease detection (19/161, 11.8%), diagnosis (14/161, 8.7%), and screening processes (12/161, 7.5%) in the areas of radiology (17/161, 10.6%), cardiology (12/161, 7.5%), gastrointestinal medicine (4/161, 2.5%), and diabetes (6/161, 3.7%). The literature synthesis in this study suggests that GenAI is mainly used for diagnostic processes, improvement of diagnosis accuracy, and screening and diagnostic purposes using knowledge access. Although this solves the problem of knowledge access and may improve diagnostic accuracy, it is oriented toward higher value creation in health care.

CONCLUSIONS

GenAI informs rather than assisting or automating clinical service functions in health care. There is potential in clinical service, but it has yet to be actualized for GenAI. More clinical service-level evidence that GenAI is used to streamline some functions or provides more automated help than only information retrieval is needed. To transform health care as purported, more studies related to GenAI applications must automate and guide human-performed services and keep up with the optimism that forward-thinking health care organizations will take advantage of GenAI.

AI-luminating Artificial Intelligence in Inflammatory Bowel Diseases: A Narrative Review on the Role of AI in Endoscopy, Histology, and Imaging for IBD

Endoscopy, histology, and cross-sectional imaging serve as fundamental pillars in the detection, monitoring, and prognostication of inflammatory bowel disease (IBD). However, interpretation of these studies often relies on subjective human judgment, which can lead to delays, intra- and interobserver variability, and potential diagnostic discrepancies. With the rising incidence of IBD globally coupled with the exponential digitization of these data, there is a growing demand for innovative approaches to streamline diagnosis and elevate clinical decision-making. In this context, artificial intelligence (AI) technologies emerge as a timely solution to address the evolving challenges in IBD. Early studies using deep learning and radiomics approaches for endoscopy, histology, and imaging in IBD have demonstrated promising results for using AI to detect, diagnose, characterize, phenotype, and prognosticate IBD. Nonetheless, the available literature has inherent limitations and knowledge gaps that need to be addressed before AI can transition into a mainstream clinical tool for IBD. To better understand the potential value of integrating AI in IBD, we review the available literature to summarize our current understanding and identify gaps in knowledge to inform future investigations.

Artificial intelligence image-based prediction models in IBD exhibit high risk of bias: A systematic review

BACKGROUND

There has been an increase in the development of both machine learning (ML) and deep learning (DL) prediction models in Inflammatory Bowel Disease. We aim in this systematic review to assess the methodological quality and risk of bias of ML and DL IBD image-based prediction studies.

METHODS

We searched three databases, PubMed, Scopus and Embase, to identify ML and DL diagnostic or prognostic predictive models using imaging data in IBD, to Dec 31, 2022. We restricted our search to include studies that primarily used conventional imaging data, were undertaken in human participants, and published in English. Two reviewers independently reviewed the abstracts. The methodological quality of the studies was determined, and risk of bias evaluated using the prediction risk of bias assessment tool (PROBAST).

RESULTS

Forty studies were included, thirty-nine developed diagnostic models. Seven studies utilized ML approaches, six were retrospective and none used multicenter data for model development. Thirty-three studies utilized DL approaches, ten were prospective, and twelve multicenter studies. Overall, all studies demonstrated high risk of bias. ML studies were evaluated in 4 domains all rated as high risk of bias: participants (6/7), predictors (1/7), outcome (3/7), and analysis (7/7), and DL studies evaluated in 3 domains: participants (24/33), outcome (10/33), and analysis (18/33). The majority of image-based studies used colonoscopy images.

CONCLUSION

The risk of bias was high in AI IBD image-based prediction models, owing to insufficient sample size, unreported missingness and lack of an external validation cohort. Models with a high risk of bias are unlikely to be generalizable and suitable for clinical implementation.

Present and future of endoscopy precision for inflammatory bowel disease

Several advanced imaging techniques are now available for endoscopists managing inflammatory bowel disease (IBD) patients. These tools, including dye-based and virtual chromoendoscopy, probe-based confocal laser endomicroscopy and endocytoscopy, are increasingly innovative applications in clinical practice. They allow for a more in-depth and refined evaluation of the mucosal and vascular bowel surface, getting closer to histology. They have demonstrated a remarkable ability in assessing intestinal inflammation, histologic remission, and predicting relapse and favorable long-term outcomes. In addition, the future application of molecular endoscopy to predict biological drug responses has yielded preliminary but encouraging results. Furthermore, these techniques are crucial in detecting and characterizing IBD-related dysplasia, assisting endoscopic mucosal resection and submucosal dissection towards a surgery-sparing approach. Artificial intelligence (AI) holds great potential in this promising landscape, as it can provide an objective and reproducible assessment of inflammation and dysplasia. Moreover, it can improve the prediction of outcomes and aid in subsequent therapeutic decision-making. This review aims to summarize the promising role of state-of-the-art advanced endoscopic techniques and related AI-enabled models for managing IBD, paving the way for precision medicine.

Artificial intelligence in endoscopy related to inflammatory bowel disease: A systematic review

BACKGROUND AND OBJECTIVES

In spite of rapid growth of artificial intelligence (AI) in digestive endoscopy in lesion detection and characterization, the role of AI in inflammatory bowel disease (IBD) endoscopy is not clearly defined. We aimed at systematically reviewing the role of AI in IBD endoscopy and identifying future research areas.

METHODS

We searched the PubMed and Embase database using keywords ("artificial intelligence" OR "machine learning" OR "computer-aided" OR "convolutional neural network") AND ("inflammatory bowel disease" OR "ulcerative colitis" OR "Crohn's") AND ("endoscopy" or "colonoscopy" or "capsule endoscopy" or "device assisted enteroscopy") between 1975 and September 2023 and identified 62 original articles for detailed review. Review articles, consensus guidelines, case reports/series, editorials, letter to the editor, non-peer-reviewed pre-prints and conference abstracts were excluded. The quality of the included studies was assessed using the MI-CLAIM checklist.

RESULTS

The accuracy of AI models (25 studies) to assess ulcerative colitis (UC) endoscopic activity ranged between 86.54% and 94.5%. AI-assisted capsule endoscopy reading (12 studies) substantially reduced analyzable images and reading time with excellent accuracy (90.5% to 99.9%). AI-assisted analysis of colonoscopic images can help differentiate IBD from non-IBD, UC from non-UC and UC from Crohn's disease (CD) (three studies) with 72.1%, 98.3% and > 90% accuracy, respectively. AI models based on non-invasive clinical and radiologic parameters could predict endoscopic activity (three studies). AI-assisted virtual chromoendoscopy (four studies) could predict histologic remission and long-term outcomes. Computer-assisted detection (CADe) of dysplasia (two studies) is feasible along with AI-based differentiation of high from low-grade IBD neoplasia (79% accuracy). AI is effective in linking electronic medical record data (two studies) with colonoscopic videos to facilitate widespread machine learning.

CONCLUSION

AI-assisted IBD endoscopy has the potential to impact clinical management by automated detection and characterization of endoscopic lesions. Large, multi-center, prospective studies and commercially available IBD-specific endoscopic AI algorithms are warranted.

Machine learning application in autoimmune diseases: State of art and future prospectives

Autoimmune diseases are a group of disorders resulting from an alteration of immune tolerance, characterized by the formation of autoantibodies and the consequent development of heterogeneous clinical manifestations. Diagnosing autoimmune diseases is often complicated, and the available prognostic tools are limited. Machine learning allows us to analyze large amounts of data and carry out complex calculations quickly and with minimal effort. In this work, we examine the literature focusing on the use of machine learning in the field of the main systemic (systemic lupus erythematosus and rheumatoid arthritis) and organ-specific autoimmune diseases (type 1 diabetes mellitus, autoimmune thyroid, gastrointestinal, and skin diseases). From our analysis, interesting applications of machine learning emerged for developing algorithms useful in the early diagnosis of disease or prognostic models (risk of complications, therapeutic response). Subsequent studies and the creation of increasingly rich databases to be supplied to the algorithms will eventually guide the clinician in the diagnosis, allowing intervention when the pathology is still in an early stage and immediately directing towards a correct therapeutic approach.

Comparison of Endoscopic and Artificial Intelligence Diagnoses for Predicting the Histological Healing of Ulcerative Colitis in a Real-World Clinical Setting

Background

Artificial intelligence (AI)-assisted colonoscopy systems with contact microscopy capabilities have been reported previously; however, no studies regarding the clinical use of a commercially available system in patients with ulcerative colitis (UC) have been reported. In this study, the diagnostic performance of an AI-assisted ultra-magnifying colonoscopy system for histological healing was compared with that of conventional light non-magnifying endoscopic evaluation in patients with UC.

Methods

The data of 52 patients with UC were retrospectively analyzed. The Mayo endoscopic score (MES) was determined by 3 endoscopists. Using the AI system, healing of the same spot assessed via MES was defined as a predicted Geboes score (GS) < 3.1. The GS was then determined using pathology specimens from the same site.

Results

A total of 191 sites were evaluated, including 159 with a GS < 3.1. The MES diagnosis identified 130 sites as MES0. A total of 120 sites were determined to have healed based on AI. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of MES0 for the diagnosis of GS < 3.1 were 79.2%, 90.6%, 97.7%, 46.8%, and 81.2%, respectively. The AI system performed similarly to MES for the diagnosis of GS < 3.1: sensitivity, 74.2%; specificity: 93.8%; PPV: 98.3%; NPV: 42.3%; and accuracy: 77.5%. The AI system also significantly identified a GS of < 3.1 in the setting of MES1 (P = .0169).

Conclusions

The histological diagnostic yield the MES- and AI-assisted diagnoses was comparable. Healing decisions using AI may avoid the need for histological examinations.

Using Computer Vision to Improve Endoscopic Disease Quantification in Therapeutic Clinical Trials of Ulcerative Colitis

BACKGROUND & AIMS

Endoscopic assessment of ulcerative colitis (UC) typically reports only the maximum severity observed. Computer vision methods may better quantify mucosal injury detail, which varies among patients.

METHODS

Endoscopic video from the UNIFI clinical trial (A Study to Evaluate the Safety and Efficacy of Ustekinumab Induction and Maintenance Therapy in Participants With Moderately to Severely Active Ulcerative Colitis) comparing ustekinumab and placebo for UC were processed in a computer vision analysis that spatially mapped Mayo Endoscopic Score (MES) to generate the Cumulative Disease Score (CDS). CDS was compared with the MES for differentiating ustekinumab vs placebo treatment response and agreement with symptomatic remission at week 44. Statistical power, effect, and estimated sample sizes for detecting endoscopic differences between treatments were calculated using both CDS and MES measures. Endoscopic video from a separate phase 2 clinical trial replication cohort was performed for validation of CDS performance.

RESULTS

Among 748 induction and 348 maintenance patients, CDS was lower in ustekinumab vs placebo users at week 8 (141.9 vs 184.3; P < .0001) and week 44 (78.2 vs 151.5; P < .0001). CDS was correlated with the MES (P < .0001) and all clinical components of the partial Mayo score (P < .0001). Stratification by pretreatment CDS revealed ustekinumab was more effective than placebo (P < .0001) with increasing effect in severe vs mild disease (-85.0 vs -55.4; P < .0001). Compared with the MES, CDS was more sensitive to change, requiring 50% fewer participants to demonstrate endoscopic differences between ustekinumab and placebo (Hedges' g = 0.743 vs 0.460). CDS performance in the JAK-UC replication cohort was similar to UNIFI.

CONCLUSIONS

As an automated and quantitative measure of global endoscopic disease severity, the CDS offers artificial intelligence enhancement of traditional MES capability to better evaluate UC in clinical trials and potentially practice.

Impact of artificial intelligence on prognosis, shared decision-making, and precision medicine for patients with inflammatory bowel disease: a perspective and expert opinion

INTRODUCTION

Artificial intelligence (AI) is expected to impact all facets of inflammatory bowel disease (IBD) management, including disease assessment, treatment decisions, discovery and development of new biomarkers and therapeutics, as well as clinician-patient communication.

AREAS COVERED

This perspective paper provides an overview of the application of AI in the clinical management of IBD through a review of the currently available AI models that could be potential tools for prognosis, shared decision-making, and precision medicine. This overview covers models that measure treatment response based on statistical or machine-learning methods, or a combination of the two. We briefly discuss a computational model that allows integration of immune/biological system knowledge with mathematical modeling and also involves a 'digital twin', which allows measurement of temporal trends in mucosal inflammatory activity for predicting treatment response. A viewpoint on AI-enabled wearables and nearables and their use to improve IBD management is also included.

EXPERT OPINION

Although challenges regarding data quality, privacy, and security; ethical concerns; technical limitations; and regulatory barriers remain to be fully addressed, a growing body of evidence suggests a tremendous potential for integration of AI into daily clinical practice to enable precision medicine and shared decision-making.

Application of deep learning in the diagnosis and evaluation of ulcerative colitis disease severity

Background

Achieving endoscopic and histological remission is a critical treatment objective in ulcerative colitis (UC). Nevertheless, interobserver variability can significantly impact overall assessment performance.

Objectives

We aimed to develop a deep learning algorithm for the real-time and objective evaluation of endoscopic disease activity and prediction of histological remission in UC.

Design

This is a retrospective diagnostic study.

Methods

Two convolutional neural network (CNN) models were constructed and trained using 12,257 endoscopic images and biopsy results sourced from 1124 UC patients who underwent colonoscopy at a single center from January 2018 to December 2022. Mayo Endoscopy Subscore (MES) and UC Endoscopic Index of Severity Score (UCEIS) assessments were conducted by two experienced and independent reviewers. Model performance was evaluated in terms of accuracy, sensitivity, and positive predictive value. The output of the CNN models was also compared with the corresponding histological results to assess histological remission prediction performance.

Results

The MES-CNN model achieved 97.04% accuracy in diagnosing endoscopic remission of UC, while the MES-CNN and UCEIS-CNN models achieved 90.15% and 85.29% accuracy, respectively, in evaluating endoscopic severity of UC. For predicting histological remission, the CNN models achieved accuracy and kappa values of 91.28% and 0.826, respectively, attaining higher accuracy than human endoscopists (87.69%).

Conclusion

The proposed artificial intelligence model, based on MES and UCEIS evaluations from expert gastroenterologists, offered precise assessment of inflammation in UC endoscopic images and reliably predicted histological remission.

Endoscopy-based IBD identification by a quantized deep learning pipeline

BACKGROUND

Discrimination between patients affected by inflammatory bowel diseases and healthy controls on the basis of endoscopic imaging is an challenging problem for machine learning models. Such task is used here as the testbed for a novel deep learning classification pipeline, powered by a set of solutions enhancing characterising elements such as reproducibility, interpretability, reduced computational workload, bias-free modeling and careful image preprocessing.

RESULTS

First, an automatic preprocessing procedure is devised, aimed to remove artifacts from clinical data, feeding then the resulting images to an aggregated per-patient model to mimic the clinicians decision process. The predictions are based on multiple snapshots obtained through resampling, reducing the risk of misleading outcomes by removing the low confidence predictions. Each patient's outcome is explained by returning the images the prediction is based upon, supporting clinicians in verifying diagnoses without the need for evaluating the full set of endoscopic images. As a major theoretical contribution, quantization is employed to reduce the complexity and the computational cost of the model, allowing its deployment on small power devices with an almost negligible 3% performance degradation. Such quantization procedure holds relevance not only in the context of per-patient models but also for assessing its feasibility in providing real-time support to clinicians even in low-resources environments. The pipeline is demonstrated on a private dataset of endoscopic images of 758 IBD patients and 601 healthy controls, achieving Matthews Correlation Coefficient 0.9 as top performance on test set.

CONCLUSION

We highlighted how a comprehensive pre-processing pipeline plays a crucial role in identifying and removing artifacts from data, solving one of the principal challenges encountered when working with clinical data. Furthermore, we constructively showed how it is possible to emulate clinicians decision process and how it offers significant advantages, particularly in terms of explainability and trust within the healthcare context. Last but not least, we proved that quantization can be a useful tool to reduce the time and resources consumption with an acceptable degradation of the model performs. The quantization study proposed in this work points up the potential development of real-time quantized algorithms as valuable tools to support clinicians during endoscopy procedures.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

Endoscopic remission is an important therapeutic goal in ulcerative colitis (UC). The Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Mayo Endoscopic Score (MES) are the commonly used endoscopic scoring criteria. This systematic review and meta-analysis aimed to evaluate the accuracy of artificial intelligence (AI) in diagnosing endoscopic remission in UC. We also performed a meta-analysis of each of the four endoscopic remission criteria (UCEIS = 0, MES = 0, UCEIS = <1, MES = <1). Eighteen studies involving 13,687 patients were included. The combined sensitivity and specificity of AI for diagnosing endoscopic remission in UC was 87% (95% confidence interval [CI]:81-92%) and 92% (95% CI: 89-94%), respectively. The area under the curve (AUC) was 0.96 (95% CI: 0.94-0.97). The results showed that the AI model performed well regardless of which criteria were used to define endoscopic remission of UC.

Using a Deep Learning Model to Address Interobserver Variability in the Evaluation of Ulcerative Colitis (UC) Severity

The use of endoscopic images for the accurate assessment of ulcerative colitis (UC) severity is crucial to determining appropriate treatment. However, experts may interpret these images differently, leading to inconsistent diagnoses. This study aims to address the issue by introducing a standardization method based on deep learning. We collected 254 rectal endoscopic images from 115 patients with UC, and five experts in endoscopic image interpretation assigned classification labels based on the Ulcerative Colitis Endoscopic Index of Severity (UCEIS) scoring system. Interobserver variance analysis of the five experts yielded an intraclass correlation coefficient of 0.8431 for UCEIS scores and a kappa coefficient of 0.4916 when the UCEIS scores were transformed into UC severity measures. To establish a consensus, we created a model that considered only the images and labels on which more than half of the experts agreed. This consensus model achieved an accuracy of 0.94 when tested with 50 images. Compared with models trained from individual expert labels, the consensus model demonstrated the most reliable prediction results.

Artificial intelligence for gastroenterology: Singapore artificial intelligence for Gastroenterology Working Group Position Statement

BACKGROUND

Successful implementation of artificial intelligence in gastroenterology and hepatology practice requires more than technology. There are ethical, legal, and social issues that need to be settled.

AIM

A group consisting of AI developers (engineer), AI users (gastroenterologist, hepatologist, and surgeon) and AI regulators (ethicist and administrator) formed a Working Group to draft these Positions Statements with the objective of arousing public and professional interest and dialogue, to promote ethical considerations when implementing AI technology, to suggest to policy makers and health authorities relevant factors to take into account when approving and regulating the use of AI tools, and to engage the profession in preparing for change in clinical practice.

STATEMENTS

These series of Position Statements point out the salient issues to maintain the trust between care provider and care receivers, and to legitimize the use of a non-human tool in healthcare delivery. It is based on fundamental principles such as respect, autonomy, privacy, responsibility, and justice. Enforcing the use of AI without considering these factor risk damaging the doctor-patient relationship.

Artificial intelligence in endoscopy: Overview, applications, and future directions

Since the emergence of artificial intelligence (AI) in medicine, endoscopy applications in gastroenterology have been at the forefront of innovations. The ever-increasing number of studies necessitates the need to organize and classify applications in a useful way. Separating AI capabilities by computer aided detection (CADe), diagnosis (CADx), and quality assessment (CADq) allows for a systematic evaluation of each application. CADe studies have shown promise in accurate detection of esophageal, gastric and colonic neoplasia as well as identifying sources of bleeding and Crohn's disease in the small bowel. While more advanced CADx applications employ optical biopsies to give further information to characterize neoplasia and grade inflammatory disease, diverse CADq applications ensure quality and increase the efficiency of procedures. Future applications show promise in advanced therapeutic modalities and integrated systems that provide multimodal capabilities. AI is set to revolutionize clinical decision making and performance of endoscopy.

Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning

BACKGROUND

Assessment of endoscopic activity in ulcerative colitis (UC) is important for treatment decisions and monitoring disease progress. However, substantial inter- and intraobserver variability in grading impairs the assessment. Our aim was to develop a computer-aided diagnosis system using deep learning to reduce subjectivity and improve the reliability of the assessment.

METHODS

The cohort comprises 11 276 images from 564 patients who underwent colonoscopy for UC. We propose a regression-based deep learning approach for the endoscopic evaluation of UC according to the Mayo endoscopic score (MES). Five state-of-the-art convolutional neural network (CNN) architectures were used for the performance measurements and comparisons. Ten-fold cross-validation was used to train the models and objectively benchmark them. Model performances were assessed using quadratic weighted kappa and macro F1 scores for full Mayo score classification and kappa statistics and F1 score for remission classification.

RESULTS

Five classification-based CNNs used in the study were in excellent agreement with the expert annotations for all Mayo subscores and remission classification according to the kappa statistics. When the proposed regression-based approach was used, (1) the performance of most of the models statistically significantly increased and (2) the same model trained on different cross-validation folds produced more robust results on the test set in terms of deviation between different folds.

CONCLUSIONS

Comprehensive experimental evaluations show that commonly used classification-based CNN architectures have successful performance in evaluating endoscopic disease activity of UC. Integration of domain knowledge into these architectures further increases performance and robustness, accelerating their translation into clinical use.

Artificial Intelligence in Inflammatory Bowel Disease Endoscopy: Implications for Clinical Trials

Artificial intelligence shows promise for clinical research in inflammatory bowel disease endoscopy. Accurate assessment of endoscopic activity is important in clinical practice and inflammatory bowel disease clinical trials. Emerging artificial intelligence technologies can increase efficiency and accuracy of assessing the baseline endoscopic appearance in patients with inflammatory bowel disease and the impact that therapeutic interventions may have on mucosal healing in both of these contexts. In this review, state-of-the-art endoscopic assessment of mucosal disease activity in inflammatory bowel disease clinical trials is described, covering the potential for artificial intelligence to transform the current paradigm, its limitations, and suggested next steps. Site-based artificial intelligence quality evaluation and inclusion of patients in clinical trials without the need for a central reader is proposed; for following patient progress, a second reading using AI alongside a central reader with expedited reading is proposed. Artificial intelligence will support precision endoscopy in inflammatory bowel disease and is on the threshold of advancing inflammatory bowel disease clinical trial recruitment.

CB-HRNet: A Class-Balanced High-Resolution Network for the evaluation of endoscopic activity in patients with ulcerative colitis

Endoscopic evaluation is the key to the management of ulcerative colitis (UC). However, there is interobserver variability in interpreting endoscopic images among gastroenterologists. Furthermore, it is time-consuming. Convolutional neural networks (CNNs) can help overcome these obstacles and has yielded preliminary positive results. We aimed to develop a new CNN-based algorithm to improve the performance for evaluation tasks of endoscopic images in patients with UC. A total of 12,163 endoscopic images from 308 patients with UC were collected from January 2014 to December 2021. The training set and test set images were randomly divided into 37,515 and 3191 after excluding possible interference and data augmentation. Mayo Endoscopic Subscores (MES) were predicted by different CNN-based models with different loss functions. Their performances were evaluated by several metrics. After comparing the results of different CNN-based models with different loss functions, High-Resolution Network with Class-Balanced Loss achieved the best performances in all MES classification subtasks. It was especially great at determining endoscopic remission in UC, which achieved a high accuracy of 95.07% and good performances in other evaluation metrics with sensitivity 92.87%, specificity 95.41%, kappa coefficient 0.8836, positive predictive value 93.44%, negative predictive value 95.00% and area value under the receiver operating characteristic curve 0.9834, respectively. In conclusion, we proposed a new CNN-based algorithm, Class-Balanced High-Resolution Network (CB-HRNet), to evaluate endoscopic activity of UC with excellent performance. Besides, we made an open-source dataset and it can be a new benchmark in the task of MES classification.

Diagnostic accuracy of convolutional neural network-based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis

BACKGROUND AND AIMS

Endoscopic assessment of ulcerative colitis (UC) can be performed by using the Mayo Endoscopic Score (MES) or the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). In this meta-analysis, we assessed the pooled diagnostic accuracy parameters of deep machine learning by means of convolutional neural network (CNN) algorithms in predicting UC severity on endoscopic images.

METHODS

Databases including MEDLINE, Scopus, and Embase were searched in June 2022. Outcomes of interest were the pooled accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Standard meta-analysis methods used the random-effects model, and heterogeneity was assessed using the I2statistics.

RESULTS

Twelve studies were included in the final analysis. The pooled diagnostic parameters of CNN-based machine learning algorithms in endoscopic severity assessment of UC were as follows: accuracy 91.5% (95% confidence interval [CI], 88.3-93.8; I2 = 84%), sensitivity 82.8% (95% CI, 78.3-86.5; I2 = 89%), specificity 92.4% (95% CI, 89.4-94.6; I2 = 84%), PPV 86.6% (95% CI, 82.3-90; I2 = 89%), and NPV 88.6% (95% CI, 85.7-91; I2 = 78%). Subgroup analysis revealed significantly better sensitivity and PPV with the UCEIS scoring system compared with the MES (93.6% [95% CI, 87.5-96.8; I2 = 77%] vs 82% [95% CI, 75.6-87; I2 = 89%], P = .003, and 93.6% [95% CI, 88.7-96.4; I2 = 68%] vs 83.6% [95% CI, 76.8-88.8; I2 = 77%], P = .007, respectively).

CONCLUSIONS

CNN-based machine learning algorithms demonstrated excellent pooled diagnostic accuracy parameters in the endoscopic severity assessment of UC. Using UCEIS scores in CNN training might offer better results than the MES. Further studies are warranted to establish these findings in real clinical settings.

New Technologies in Digestive Endoscopy for Ulcerative Colitis Patients

Ulcerative colitis (UC) is a chronic inflammatory bowel disease primarily affecting the colon and rectum. Endoscopy plays a crucial role in the diagnosis and management of UC. Recent advancements in endoscopic technology, including chromoendoscopy, confocal laser endomicroscopy, endocytoscopy and the use of artificial intelligence, have revolutionized the assessment and treatment of UC patients. These innovative techniques enable early detection of dysplasia and cancer, more precise characterization of disease extent and severity and more targeted biopsies, leading to improved diagnosis and disease monitoring. Furthermore, these advancements have significant implications for therapeutic decision making, empowering clinicians to carefully consider a range of treatment options, including pharmacological therapies, endoscopic interventions and surgical approaches. In this review, we provide an overview of the latest endoscopic technologies and their applications for diagnosing and monitoring UC. We also discuss their impact on treatment decision making, highlighting the potential benefits and limitations of each technique.

Spatiotemporal analysis of small bowel capsule endoscopy videos for outcomes prediction in Crohn's disease

Background

Deep learning techniques can accurately detect and grade inflammatory findings on images from capsule endoscopy (CE) in Crohn's disease (CD). However, the predictive utility of deep learning of CE in CD for disease outcomes has not been examined.

Objectives

We aimed to develop a deep learning model that can predict the need for biological therapy based on complete CE videos of newly-diagnosed CD patients.

Design

This was a retrospective cohort study. The study cohort included treatment-naïve CD patients that have performed CE (SB3, Medtronic) within 6 months of diagnosis. Complete small bowel videos were extracted using the RAPID Reader software.

Methods

CE videos were scored using the Lewis score (LS). Clinical, endoscopic, and laboratory data were extracted from electronic medical records. Machine learning analysis was performed using the TimeSformer computer vision algorithm developed to capture spatiotemporal characteristics for video analysis.

Results

The patient cohort included 101 patients. The median duration of follow-up was 902 (354-1626) days. Biological therapy was initiated by 37 (36.6%) out of 101 patients. TimeSformer algorithm achieved training and testing accuracy of 82% and 81%, respectively, with an Area under the ROC Curve (AUC) of 0.86 to predict the need for biological therapy. In comparison, the AUC for LS was 0.70 and for fecal calprotectin 0.74.

Conclusion

Spatiotemporal analysis of complete CE videos of newly-diagnosed CD patients achieved accurate prediction of the need for biological therapy. The accuracy was superior to that of the human reader index or fecal calprotectin. Following future validation studies, this approach will allow for fast and accurate personalization of treatment decisions in CD.

Artificial Intelligence in Pediatric Endoscopy: Current Status and Future Applications

The application of artificial intelligence (AI) has great promise for improving pediatric endoscopy. The majority of preclinical studies have been undertaken in adults, with the greatest progress being made in the context of colorectal cancer screening and surveillance. This development has only been possible with advances in deep learning, like the convolutional neural network model, which has enabled real-time detection of pathology. Comparatively, the majority of deep learning systems developed in inflammatory bowel disease have focused on predicting disease severity and were developed using still images rather than videos. The application of AI to pediatric endoscopy is in its infancy, thus providing an opportunity to develop clinically meaningful and fair systems that do not perpetuate societal biases. In this review, we provide an overview of AI, summarize the advances of AI in endoscopy, and describe its potential application to pediatric endoscopic practice and education.

A Review of Colonoscopy in Intestinal Diseases

Since the development of the fiberoptic colonoscope in the late 1960s, colonoscopy has been a useful tool to diagnose and treat various intestinal diseases. This article reviews the clinical use of colonoscopy for various intestinal diseases based on present and future perspectives. Intestinal diseases include infectious diseases, inflammatory bowel disease (IBD), neoplasms, functional bowel disorders, and others. In cases of infectious diseases, colonoscopy is helpful in making the differential diagnosis, revealing endoscopic gross findings, and obtaining the specimens for pathology. Additionally, colonoscopy provides clues for distinguishing between infectious disease and IBD, and aids in the post-treatment monitoring of IBD. Colonoscopy is essential for the diagnosis of neoplasms that are diagnosed through only pathological confirmation. At present, malignant tumors are commonly being treated using endoscopy because of the advancement of endoscopic resection procedures. Moreover, the characteristics of tumors can be described in more detail by image-enhanced endoscopy and magnifying endoscopy. Colonoscopy can be helpful for the endoscopic decompression of colonic volvulus in large bowel obstruction, balloon dilatation as a treatment for benign stricture, and colon stenting as a treatment for malignant obstruction. In the diagnosis of functional bowel disorder, colonoscopy is used to investigate other organic causes of the symptom.

Mise au point de l’utilisation de l’intelligence artificielle dans la prise en charge des maladies inflammatoires chroniques de l’intestin

Complexity of inflammatory bowel diseases (IBD) lies on their management and their biology. Clinics, blood and fecal samples tests, endoscopy and histology are the main tools guiding IBD treatment, but they generate a large amount of data, difficult to analyze by clinicians. Because of its capacity to analyze large number of data, artificial intelligence is currently generating enthusiasm in medicine, and this technology could be used to improve IBD management. In this review, after a short summary on IBD management and artificial intelligence, we will report pragmatic examples of artificial intelligence utilisation in IBD. Lastly, we will discuss the limitations of this technology.

Development of a tongue image-based machine learning tool for the diagnosis of gastric cancer: a prospective multicentre clinical cohort study

BACKGROUND

Tongue images (the colour, size and shape of the tongue and the colour, thickness and moisture content of the tongue coating), reflecting the health state of the whole body according to the theory of traditional Chinese medicine (TCM), have been widely used in China for thousands of years. Herein, we investigated the value of tongue images and the tongue coating microbiome in the diagnosis of gastric cancer (GC).

METHODS

From May 2020 to January 2021, we simultaneously collected tongue images and tongue coating samples from 328 patients with GC (all newly diagnosed with GC) and 304 non-gastric cancer (NGC) participants in China, and 16 S rDNA was used to characterize the microbiome of the tongue coating samples. Then, artificial intelligence (AI) deep learning models were established to evaluate the value of tongue images and the tongue coating microbiome in the diagnosis of GC. Considering that tongue imaging is more convenient and economical as a diagnostic tool, we further conducted a prospective multicentre clinical study from May 2020 to March 2022 in China and recruited 937 patients with GC and 1911 participants with NGC from 10 centres across China to further evaluate the role of tongue images in the diagnosis of GC. Moreover, we verified this approach in another independent external validation cohort that included 294 patients with GC and 521 participants with NGC from 7 centres. This study is registered at ClinicalTrials.gov, NCT01090362.

FINDINGS

For the first time, we found that both tongue images and the tongue coating microbiome can be used as tools for the diagnosis of GC, and the area under the curve (AUC) value of the tongue image-based diagnostic model was 0.89. The AUC values of the tongue coating microbiome-based model reached 0.94 using genus data and 0.95 using species data. The results of the prospective multicentre clinical study showed that the AUC values of the three tongue image-based models for GCs reached 0.88-0.92 in the internal verification and 0.83-0.88 in the independent external verification, which were significantly superior to the combination of eight blood biomarkers.

INTERPRETATION

Our results suggest that tongue images can be used as a stable method for GC diagnosis and are significantly superior to conventional blood biomarkers. The three kinds of tongue image-based AI deep learning diagnostic models that we developed can be used to adequately distinguish patients with GC from participants with NGC, even early GC and precancerous lesions, such as atrophic gastritis (AG).

FUNDING

The National Key R&D Program of China (2021YFA0910100), Program of Zhejiang Provincial TCM Sci-tech Plan (2018ZY006), Medical Science and Technology Project of Zhejiang Province (2022KY114, WKJ-ZJ-2104), Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer (JBZX-202006), Natural Science Foundation of Zhejiang Province (HDMY22H160008), Science and Technology Projects of Zhejiang Province (2019C03049), National Natural Science Foundation of China (82074245, 81973634, 82204828), and Chinese Postdoctoral Science Foundation (2022M713203).

The Role of Artificial Intelligence in Monitoring Inflammatory Bowel Disease-The Future Is Now

Crohn's disease and ulcerative colitis remain debilitating disorders, characterized by progressive bowel damage and possible lethal complications. The growing number of applications for artificial intelligence in gastrointestinal endoscopy has already shown great potential, especially in the field of neoplastic and pre-neoplastic lesion detection and characterization, and is currently under evaluation in the field of inflammatory bowel disease management. The application of artificial intelligence in inflammatory bowel diseases can range from genomic dataset analysis and risk prediction model construction to the disease grading severity and assessment of the response to treatment using machine learning. We aimed to assess the current and future role of artificial intelligence in assessing the key outcomes in inflammatory bowel disease patients: endoscopic activity, mucosal healing, response to treatment, and neoplasia surveillance.

Artificial Intelligence-The Rising Star in the Field of Gastroenterology and Hepatology

Artificial intelligence (AI) is a term that covers a multitude of techniques that are used in a manner that tries to reproduce human intelligence. AI is helpful in various medical specialties that use imaging for diagnostic purposes, and gastroenterology is no exception. In this field, AI has several applications, such as detecting and classifying polyps, detecting the malignancy in polyps, diagnosing Helicobacter pylori infection, gastritis, inflammatory bowel disease, gastric cancer, esophageal neoplasia, and pancreatic and hepatic lesions. The aim of this mini-review is to analyze the currently available studies regarding AI in the field of gastroenterology and hepatology and to discuss its main applications as well as its main limitations.

Novel deep learning-based computer-aided diagnosis system for predicting inflammatory activity in ulcerative colitis

BACKGROUND AND AIMS

Endoscopy is increasingly performed for evaluating patients with ulcerative colitis (UC). However, its diagnostic accuracy is largely affected by the subjectivity of endoscopists' experience and scoring methods, and scoring of selected endoscopic images cannot reflect the inflammation of the entire intestine. We aimed to develop an automatic scoring system using deep-learning technology for consistent and objective scoring of endoscopic images and full-length endoscopic videos of patients with UC.

METHODS

We collected 5875 endoscopic images and 20 full-length videos from 332 patients with UC who underwent colonoscopy between January 2017 and March 2021. We trained the artificial intelligence (AI) scoring system using these images, which was then used for full-length video scoring. To more accurately assess and visualize the full-length intestinal inflammation, we divided the large intestine into a fixed number of "areas" (cecum, 20; transverse colon, 20; descending colon, 20; sigmoid colon, 15; rectum, 10). The scoring system automatically scored inflammatory severity of 85 areas from every video and generated a visualized result of full-length intestinal inflammatory activity.

RESULTS

Compared with endoscopist scoring, the trained convolutional neural network achieved 86.54% accuracy in the Mayo-scored task, whereas the kappa coefficient was .813 (95% confidence interval [CI], .782-.844). The metrics of the Ulcerative Colitis Endoscopic Index of Severity-scored task were encouraging, with accuracies of 90.7%, 84.6%, and 77.7% and kappa coefficients of .822 (95% CI, .788-.855), .784 (95% CI, .744-.823), and .702 (95% CI, .612-.793) for vascular pattern, erosions and ulcers, and bleeding, respectively. The AI scoring system predicted each bowel segment's score and displayed distribution of inflammatory activity in the entire large intestine using a 2-dimensional colorized image.

CONCLUSIONS

We established a novel deep learning-based scoring system to evaluate endoscopic images from patients with UC, which can also accurately describe the severity and distribution of inflammatory activity through full-length intestinal endoscopic videos.

Evaluating the potential of artificial intelligence in ulcerative colitis

INTRODUCTION

Diagnosis and therapeutic management in ulcerative colitis (UC) relies on a combination of endoscopic and histological scorings which are difficult to objectively quantify. Artificial intelligence (AI) might overcome the current issues of inter-observer variability, repetitive need for biopsies and estimation of disease activity medicine currently encourages.

AREAS COVERED

With this narrative literature review we aim to provide a clear and critical overview of the recent evolutions in the field of AI and UC, based on a literature search performed on Pubmed, Embase and Cochrane Library. The major focus of this review is the use of AI for endoscopic assessment of disease activity and the correlation with histology and long-term outcome. Moreover, we elucidate on the more recent developments in the field of AI as support in histological disease assessment, surveillance, therapy monitoring and natural language processing.

EXPERT OPINION

UC management is evolving with AI impacting nearly every aspect of it. The immediate future influence of AI in UC management will be focussed on the collection, extraction and organization of particular clinical information. Expect is the transformation toward a real-time standardized, reproducible, objective and high-reliable disease grading, especially in endoscopy, histology and eventually radiology applications for UC.

Artificial-intelligence-based decision support tools for the differential diagnosis of colitis

BACKGROUND

Whereas Artificial Intelligence (AI) based tools have recently been introduced in the field of gastroenterology, application in inflammatory bowel disease (IBD) is in its infancies. We established AI-based algorithms to distinguish IBD from infectious and ischemic colitis using endoscopic images and clinical data.

METHODS

First, we trained and tested a Convolutional Neural Network (CNN) using 1796 real-world images from 494 patients, presenting with three diseases (IBD [n = 212], ischemic colitis [n = 157], and infectious colitis [n = 125]). Moreover, we evaluated a Gradient Boosted Decision Trees (GBDT) algorithm using five clinical parameters as well as a hybrid approach (CNN + GBDT). Patients and images were randomly split into two completely independent datasets. The proposed approaches were benchmarked against each other and three expert endoscopists on the test set.

RESULTS

For the image-based CNN, the GBDT algorithm and the hybrid approach global accuracies were .709, .792, and .766, respectively. Positive predictive values were .602, .702, and .657. Global areas under the receiver operating characteristics (ROC) and precision recall (PR) curves were .727/.585, .888/.823, and .838/.733, respectively. Global accuracy did not differ between CNN and endoscopists (.721), but the clinical parameter-based GBDT algorithm outperformed CNN and expert image classification.

CONCLUSIONS

Decision support systems exclusively based on endoscopic image analysis for the differential diagnosis of colitis, representing a complex clinical challenge, seem not yet to be ready for primetime and more diverse image datasets may be necessary to improve performance in future development. The clinical value of the proposed clinical parameters algorithm should be evaluated in prospective cohorts.

Artificial intelligence and inflammatory bowel disease: Where are we going?

Inflammatory bowel diseases, namely ulcerative colitis and Crohn’s disease, are chronic and relapsing conditions that pose a growing burden on healthcare systems worldwide. Because of their complex and partly unknown etiology and pathogenesis, the management of ulcerative colitis and Crohn’s disease can prove challenging not only from a clinical point of view but also for resource optimization. Artificial intelligence, an umbrella term that encompasses any cognitive function developed by machines for learning or problem solving, and its subsets machine learning and deep learning are becoming ever more essential tools with a plethora of applications in most medical specialties. In this regard gastroenterology is no exception, and due to the importance of endoscopy and imaging numerous clinical studies have been gradually highlighting the relevant role that artificial intelligence has in inflammatory bowel diseases as well. The aim of this review was to summarize the most recent evidence on the use of artificial intelligence in inflammatory bowel diseases in various contexts such as diagnosis, follow-up, treatment, prognosis, cancer surveillance, data collection, and analysis. Moreover, insights into the potential further developments in this field and their effects on future clinical practice were discussed.

Artificial Intelligence in Inflammatory Bowel Disease Endoscopy: Advanced Development and New Horizons

Inflammatory bowel disease (IBD) is a complex chronic immune disease with two subtypes: Crohn's disease and ulcerative colitis. Considering the differences in pathogenesis, etiology, clinical presentation, and response to therapy among patients, gastroenterologists mainly rely on endoscopy to diagnose and treat IBD during clinical practice. However, as exemplified by the increasingly comprehensive ulcerative colitis endoscopic scoring system, the endoscopic diagnosis, evaluation, and treatment of IBD still rely on the subjective manipulation and judgment of endoscopists. In recent years, the use of artificial intelligence (AI) has grown substantially in various medical fields, and an increasing number of studies have investigated the use of this emerging technology in the field of gastroenterology. Clinical applications of AI have focused on IBD pathogenesis, etiology, diagnosis, and patient prognosis. Large-scale datasets offer tremendous utility in the development of novel tools to address the unmet clinical and practice needs for treating patients with IBD. However, significant differences among AI methodologies, datasets, and clinical findings limit the incorporation of AI technology into clinical practice. In this review, we discuss practical AI applications in the diagnosis of IBD via gastroenteroscopy and speculate regarding a future in which AI technology provides value for the diagnosis and treatment of IBD patients.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

Predictive models for endoscopic disease activity in patients with ulcerative colitis: Practical machine learning-based modeling and interpretation

Background

Endoscopic disease activity monitoring is important for the long-term management of patients with ulcerative colitis (UC), there is currently no widely accepted non-invasive method that can effectively predict endoscopic disease activity. We aimed to develop and validate machine learning (ML) models for predicting it, which are desired to reduce the frequency of endoscopic examinations and related costs.

Methods

The patients with a diagnosis of UC in two hospitals from January 2016 to January 2021 were enrolled in this study. Thirty nine clinical and laboratory variables were collected. All patients were divided into four groups based on MES or UCEIS scores. Logistic regression (LR) and four ML algorithms were applied to construct the prediction models. The performance of models was evaluated in terms of accuracy, sensitivity, precision, F1 score, and area under the receiver-operating characteristic curve (AUC). Then Shapley additive explanations (SHAP) was applied to determine the importance of the selected variables and interpret the ML models.

Results

A total of 420 patients were entered into the study. Twenty four variables showed statistical differences among the groups. After synthetic minority oversampling technique (SMOTE) oversampling and RFE variables selection, the random forests (RF) model with 23 variables in MES and the extreme gradient boosting (XGBoost) model with 21 variables in USEIS, had the greatest discriminatory ability (AUC = 0.8192 in MES and 0.8006 in UCEIS in the test set). The results obtained from SHAP showed that albumin, rectal bleeding, and CRP/ALB contributed the most to the overall model. In addition, the above three variables had a more balanced contribution to each classification under the MES than the UCEIS according to the SHAP values.

Conclusion

This proof-of-concept study demonstrated that the ML model could serve as an effective non-invasive approach to predicting endoscopic disease activity for patients with UC. RF and XGBoost, which were first introduced into data-based endoscopic disease activity prediction, are suitable for the present prediction modeling.