Development of a deep learning-based software for calculating cleansing score in small bowel capsule endoscopy

A multi-label dataset and its evaluation for automated scoring system for cleanliness assessment in video capsule endoscopy

An automated scoring system for cleanliness assessment during video capsule endoscopy (VCE) is presently lacking. The present study focused on developing an approach to automatically assess the cleanliness in VCE frames as per the latest scoring i.e., Korea-Canada (KODA). Initially, an easy-to-use mobile application called artificial intelligence-KODA (AI-KODA) score was developed to collect a multi-label image dataset of twenty-eight patient capsule videos. Three readers (gastroenterology fellows), who had been trained in reading VCE, rated this dataset in a duplicate manner. The labels were saved automatically in real-time. Inter-rater and intra-rater reliability were checked. The developed dataset was then randomly split into train:validate:test ratio of 70:20:10 and 60:20:20. It was followed by a comprehensive benchmarking and evaluation of three multi-label classification tasks using ten machine learning and two deep learning algorithms. Reliability estimation was found to be overall good among the three readers. Overall, random forest classifier achieved the best evaluation metrics, followed by Adaboost, KNeighbours, and Gaussian naive bayes in the machine learning-based classification tasks. Deep learning algorithms outperformed the machine learning-based classification tasks for only VM labels. Thorough analysis indicates that the proposed approach has the potential to save time in cleanliness assessment and is user-friendly for research and clinical use. Further research is required for the improvement of intra-rater reliability of KODA, and the development of automated multi-task classification in this field.

Reading of small bowel capsule endoscopy after frame reduction using an artificial intelligence algorithm

OBJECTIVES

Poorly visualized images that appear during small bowel capsule endoscopy (SBCE) can confuse the interpretation of small bowel lesions and increase the physician's workload. Using a validated artificial intelligence (AI) algorithm that can evaluate the mucosal visualization, we aimed to assess whether SBCE reading after the removal of poorly visualized images could affect the diagnosis of SBCE.

METHODS

A study was conducted to analyze 90 SBCE cases in which a small bowel examination was completed. Two experienced endoscopists alternately performed two types of readings. They used the AI algorithm to remove poorly visualized images for the frame reduction reading (AI user group) and conducted whole frame reading without AI (AI non-user group) for the same patient. A poorly visualized image was defined as an image with < 50% mucosal visualization. The study outcomes were diagnostic concordance and reading time between the two groups. The SBCE diagnosis was classified as Crohn's disease, bleeding, polyp, angiodysplasia, and nonspecific finding.

RESULTS

The final SBCE diagnoses between the two groups showed statistically significant diagnostic concordance (k = 0.954, p < 0.001). The mean number of lesion images was 3008.5 ± 9964.9 in the AI non-user group and 1401.7 ± 4811.3 in the AI user group. There were no cases in which lesions were completely removed. Compared with the AI non-user group (120.9 min), the reading time was reduced by 35.6% in the AI user group (77.9 min).

CONCLUSIONS

SBCE reading after reducing poorly visualized frames using the AI algorithm did not have a negative effect on the final diagnosis. SBCE reading method integrated with frame reduction and mucosal visualization evaluation will help improve AI-assisted SBCE interpretation.

From Data to Insights: How Is AI Revolutionizing Small-Bowel Endoscopy?

The role of capsule endoscopy and enteroscopy in managing various small-bowel pathologies is well-established. However, their broader application has been hampered mainly by their lengthy reading times. As a result, there is a growing interest in employing artificial intelligence (AI) in these diagnostic and therapeutic procedures, driven by the prospect of overcoming some major limitations and enhancing healthcare efficiency, while maintaining high accuracy levels. In the past two decades, the applicability of AI to gastroenterology has been increasing, mainly because of the strong imaging component. Nowadays, there are a multitude of studies using AI, specifically using convolutional neural networks, that prove the potential applications of AI to these endoscopic techniques, achieving remarkable results. These findings suggest that there is ample opportunity for AI to expand its presence in the management of gastroenterology diseases and, in the future, catalyze a game-changing transformation in clinical activities. This review provides an overview of the current state-of-the-art of AI in the scope of small-bowel study, with a particular focus on capsule endoscopy and enteroscopy.

Disease surveillance evaluation of primary small-bowel follicular lymphoma using capsule endoscopy images based on a deep convolutional neural network (with video)

BACKGROUND AND AIMS

Capsule endoscopy (CE) is useful in evaluating disease surveillance for primary small-bowel follicular lymphoma (FL), but some cases are difficult to evaluate objectively. This study evaluated the usefulness of a deep convolutional neural network (CNN) system using CE images for disease surveillance of primary small-bowel FL.

METHODS

We enrolled 26 consecutive patients with primary small-bowel FL diagnosed between January 2011 and January 2021 who underwent CE before and after a watch-and-wait strategy or chemotherapy. Disease surveillance by the CNN system was evaluated by the percentage of FL-detected images among all CE images of the small-bowel mucosa.

RESULTS

Eighteen cases (69%) were managed with a watch-and-wait approach, and 8 cases (31%) were treated with chemotherapy. Among the 18 cases managed with the watch-and-wait approach, the outcome of lesion evaluation by the CNN system was almost the same in 13 cases (72%), aggravation in 4 (22%), and improvement in 1 (6%). Among the 8 cases treated with chemotherapy, the outcome of lesion evaluation by the CNN system was improvement in 5 cases (63%), almost the same in 2 (25%), and aggravation in 1 (12%). The physician and CNN system reported similar results regarding disease surveillance evaluation in 23 of 26 cases (88%), whereas a discrepancy between the 2 was found in the remaining 3 cases (12%), attributed to poor small-bowel cleansing level.

CONCLUSIONS

Disease surveillance evaluation of primary small-bowel FL using CE images by the developed CNN system was useful under the condition of excellent small-bowel cleansing level.

Machine learning-based prediction of symptomatic intracerebral hemorrhage after intravenous thrombolysis for stroke: a large multicenter study

Background

This study aimed to compare the performance of different machine learning models in predicting symptomatic intracranial hemorrhage (sICH) after thrombolysis treatment for ischemic stroke.

Methods

This multicenter study utilized the Shenyang Stroke Emergency Map database, comprising 8,924 acute ischemic stroke patients from 29 comprehensive hospitals who underwent thrombolysis between January 2019 and December 2021. An independent testing cohort was further established, including 1,921 patients from the First People's Hospital of Shenyang. The structured dataset encompassed 15 variables, including clinical and therapeutic metrics. The primary outcome was the sICH occurrence post-thrombolysis. Models were developed using an 80/20 split for training and internal validation. Performance was assessed using machine learning classifiers, including logistic regression with lasso regularization, support vector machine (SVM), random forest, gradient-boosted decision tree (GBDT), and multilayer perceptron (MLP). The model boasting the highest area under the curve (AUC) was specifically employed to highlight feature importance.

Results

Baseline characteristics were compared between the training cohort (n = 6,369) and the external validation cohort (n = 1,921), with the sICH incidence being slightly higher in the training cohort (1.6%) compared to the validation cohort (1.1%). Among the evaluated models, the logistic regression with lasso regularization achieved the highest AUC of 0.87 (95% confidence interval [CI]: 0.79-0.95; p < 0.001), followed by the MLP model with an AUC of 0.766 (95% CI: 0.637-0.894; p = 0.04). The reference model and SVM showed AUCs of 0.575 and 0.582, respectively, while the random forest and GBDT models performed less optimally with AUCs of 0.536 and 0.436, respectively. Decision curve analysis revealed net benefits primarily for the SVM and MLP models. Feature importance from the logistic regression model emphasized anticoagulation therapy as the most significant negative predictor (coefficient: -2.0833) and recombinant tissue plasminogen activator as the principal positive predictor (coefficient: 0.5082).

Conclusion

After a comprehensive evaluation, the MLP model is recommended due to its superior ability to predict the risk of symptomatic hemorrhage post-thrombolysis in ischemic stroke patients. Based on decision curve analysis, the MLP-based model was chosen and demonstrated enhanced discriminative ability compared to the reference. This model serves as a valuable tool for clinicians, aiding in treatment planning and ensuring more precise forecasting of patient outcomes.

Design of a Convolutional Neural Network as a Deep Learning Tool for the Automatic Classification of Small-Bowel Cleansing in Capsule Endoscopy

Background and objectives: Capsule endoscopy (CE) is a non-invasive method to inspect the small bowel that, like other enteroscopy methods, requires adequate small-bowel cleansing to obtain conclusive results. Artificial intelligence (AI) algorithms have been seen to offer important benefits in the field of medical imaging over recent years, particularly through the adaptation of convolutional neural networks (CNNs) to achieve more efficient image analysis. Here, we aimed to develop a deep learning model that uses a CNN to automatically classify the quality of intestinal preparation in CE. Methods: A CNN was designed based on 12,950 CE images obtained at two clinical centers in Porto (Portugal). The quality of the intestinal preparation was classified for each image as: excellent, ≥90% of the image surface with visible mucosa; satisfactory, 50-90% of the mucosa visible; and unsatisfactory, <50% of the mucosa visible. The total set of images was divided in an 80:20 ratio to establish training and validation datasets, respectively. The CNN prediction was compared with the classification established by consensus of a group of three experts in CE, currently considered the gold standard to evaluate cleanliness. Subsequently, how the CNN performed in diagnostic terms was evaluated using an independent validation dataset. Results: Among the images obtained, 3633 were designated as unsatisfactory preparation, 6005 satisfactory preparation, and 3312 with excellent preparation. When differentiating the classes of small-bowel preparation, the algorithm developed here achieved an overall accuracy of 92.1%, with a sensitivity of 88.4%, a specificity of 93.6%, a positive predictive value of 88.5%, and a negative predictive value of 93.4%. The area under the curve for the detection of excellent, satisfactory, and unsatisfactory classes was 0.98, 0.95, and 0.99, respectively. Conclusions: A CNN-based tool was developed to automatically classify small-bowel preparation for CE, and it was seen to accurately classify intestinal preparation for CE. The development of such a system could enhance the reproducibility of the scales used for such purposes.

Automatic Classification of GI Organs in Wireless Capsule Endoscopy Using a No-Code Platform-Based Deep Learning Model

The first step in reading a capsule endoscopy (CE) is determining the gastrointestinal (GI) organ. Because CE produces too many inappropriate and repetitive images, automatic organ classification cannot be directly applied to CE videos. In this study, we developed a deep learning algorithm to classify GI organs (the esophagus, stomach, small bowel, and colon) using a no-code platform, applied it to CE videos, and proposed a novel method to visualize the transitional area of each GI organ. We used training data (37,307 images from 24 CE videos) and test data (39,781 images from 30 CE videos) for model development. This model was validated using 100 CE videos that included "normal", "blood", "inflamed", "vascular", and "polypoid" lesions. Our model achieved an overall accuracy of 0.98, precision of 0.89, recall of 0.97, and F1 score of 0.92. When we validated this model relative to the 100 CE videos, it produced average accuracies for the esophagus, stomach, small bowel, and colon of 0.98, 0.96, 0.87, and 0.87, respectively. Increasing the AI score's cut-off improved most performance metrics in each organ (p < 0.05). To locate a transitional area, we visualized the predicted results over time, and setting the cut-off of the AI score to 99.9% resulted in a better intuitive presentation than the baseline. In conclusion, the GI organ classification AI model demonstrated high accuracy on CE videos. The transitional area could be more easily located by adjusting the cut-off of the AI score and visualization of its result over time.

Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement

This ESGE Position Statement defines the expected value of artificial intelligence (AI) for the diagnosis and management of gastrointestinal neoplasia within the framework of the performance measures already defined by ESGE. This is based on the clinical relevance of the expected task and the preliminary evidence regarding artificial intelligence in artificial or clinical settings. MAIN RECOMMENDATIONS:: (1) For acceptance of AI in assessment of completeness of upper GI endoscopy, the adequate level of mucosal inspection with AI should be comparable to that assessed by experienced endoscopists. (2) For acceptance of AI in assessment of completeness of upper GI endoscopy, automated recognition and photodocumentation of relevant anatomical landmarks should be obtained in ≥90% of the procedures. (3) For acceptance of AI in the detection of Barrett's high grade intraepithelial neoplasia or cancer, the AI-assisted detection rate for suspicious lesions for targeted biopsies should be comparable to that of experienced endoscopists with or without advanced imaging techniques. (4) For acceptance of AI in the management of Barrett's neoplasia, AI-assisted selection of lesions amenable to endoscopic resection should be comparable to that of experienced endoscopists. (5) For acceptance of AI in the diagnosis of gastric precancerous conditions, AI-assisted diagnosis of atrophy and intestinal metaplasia should be comparable to that provided by the established biopsy protocol, including the estimation of extent, and consequent allocation to the correct endoscopic surveillance interval. (6) For acceptance of artificial intelligence for automated lesion detection in small-bowel capsule endoscopy (SBCE), the performance of AI-assisted reading should be comparable to that of experienced endoscopists for lesion detection, without increasing but possibly reducing the reading time of the operator. (7) For acceptance of AI in the detection of colorectal polyps, the AI-assisted adenoma detection rate should be comparable to that of experienced endoscopists. (8) For acceptance of AI optical diagnosis (computer-aided diagnosis [CADx]) of diminutive polyps (≤5 mm), AI-assisted characterization should match performance standards for implementing resect-and-discard and diagnose-and-leave strategies. (9) For acceptance of AI in the management of polyps ≥ 6 mm, AI-assisted characterization should be comparable to that of experienced endoscopists in selecting lesions amenable to endoscopic resection.

Small bowel cleanliness in capsule endoscopy: a case-control study using validated artificial intelligence algorithm

Small bowel capsule endoscopy (SBCE) may need to be performed immediately after colonoscopy without additional bowel preparation if active small bowel diseases are suspected. However, it is unclear whether the small bowel cleanliness is adequately maintained even after SBCE is performed immediately after colonoscopy. We compared the small bowel cleanliness scores of the study group (SBCE immediately after colonoscopy) and control group (SBCE alone) using a validated artificial intelligence (AI) algorithm (cut-off score > 3.25 for adequate). Cases of SBCE in which polyethylene glycol was used were included retrospectively. Among 85 enrolled cases, 50 cases (58.8%) were the study group. The mean time from the last dose of purgative administration to SBCE was 6.86 ± 0.94 h in the study group and 3.00 ± 0.18 h in the control group. Seventy-five cases (88.2%) were adequate small bowel cleanliness, which was not different between the two groups. The mean small bowel cleanliness score for the study group was 3.970 ± 0.603, and for the control group was 3.937 ± 0.428. In the study group, better colon preparation resulted in a higher small bowel cleanliness score (p = 0.015). Small bowel cleanliness was also adequately maintained in SBCE immediately after colonoscopy. There was no difference between the time and volume of purgative administration and small bowel cleanliness.

Small Bowel Detection for Wireless Capsule Endoscopy Using Convolutional Neural Networks with Temporal Filtering

By automatically classifying the stomach, small bowel, and colon, the reading time of the wireless capsule endoscopy (WCE) can be reduced. In addition, it is an essential first preprocessing step to localize the small bowel in order to apply automated small bowel lesion detection algorithms based on deep learning. The purpose of the study was to develop an automated small bowel detection method from long untrimmed videos captured from WCE. Through this, the stomach and colon can also be distinguished. The proposed method is based on a convolutional neural network (CNN) with a temporal filtering on the predicted probabilities from the CNN. For CNN, we use a ResNet50 model to classify three organs including stomach, small bowel, and colon. The hybrid temporal filter consisting of a Savitzky–Golay filter and a median filter is applied to the temporal probabilities for the “small bowel” class. After filtering, the small bowel and the other two organs are differentiated with thresholding. The study was conducted on dataset of 200 patients (100 normal and 100 abnormal WCE cases), which was divided into a training set of 140 cases, a validation set of 20 cases, and a test set of 40 cases. For the test set of 40 patients (20 normal and 20 abnormal WCE cases), the proposed method showed accuracy of 99.8% in binary classification for the small bowel. Transition time errors for gastrointestinal tracts were only 38.8 ± 25.8 seconds for the transition between stomach and small bowel and 32.0 ± 19.1 seconds for the transition between small bowel and colon, compared to the ground truth organ transition points marked by two experienced gastroenterologists.

Recent developments in small bowel endoscopy: the "black box" is now open!

Over the last few years, capsule endoscopy has been established as a fundamental device in the practicing gastroenterologist’s toolbox. Its utilization in diagnostic algorithms for suspected small bowel bleeding, Crohn’s disease, and small bowel tumors has been approved by several guidelines. The advent of double-balloon enteroscopy has significantly increased the therapeutic possibilities and release of multiple devices (single-balloon enteroscopy and spiral enteroscopy) aimed at improving the performance of small bowel enteroscopy. Recently, some important innovations have appeared in the small bowel endoscopy scene, providing further improvement to its evolution. Artificial intelligence in capsule endoscopy should increase diagnostic accuracy and reading efficiency, and the introduction of motorized spiral enteroscopy into clinical practice could also improve the therapeutic yield. This review focuses on the most recent studies on artificial-intelligence-assisted capsule endoscopy and motorized spiral enteroscopy.

Semantic Segmentation Dataset for AI-Based Quantification of Clean Mucosa in Capsule Endoscopy

Background and Objectives: Capsule endoscopy (CE) for bowel cleanliness evaluation primarily depends on subjective methods. To objectively evaluate bowel cleanliness, we focused on artificial intelligence (AI)-based assessments. We aimed to generate a large segmentation dataset from CE images and verify its quality using a convolutional neural network (CNN)-based algorithm. Materials and Methods: Images were extracted and divided into 10 stages according to the clean regions in a CE video. Each image was classified into three classes (clean, dark, and floats/bubbles) or two classes (clean and non-clean). Using this semantic segmentation dataset, a CNN training was performed with 169 videos, and a clean region (visualization scale (VS)) formula was developed. Then, measuring mean intersection over union (mIoU), Dice index, and clean mucosal predictions were performed. The VS performance was tested using 10 videos. Results: A total of 10,033 frames of the semantic segmentation dataset were constructed from 179 patients. The 3-class and 2-class semantic segmentation’s testing performance was 0.7716 mIoU (range: 0.7031–0.8071), 0.8627 Dice index (range: 0.7846–0.8891), and 0.8927 mIoU (range: 0.8562–0.9330), 0.9457 Dice index (range: 0.9225–0.9654), respectively. In addition, the 3-class and 2-class clean mucosal prediction accuracy was 94.4% and 95.7%, respectively. The VS prediction performance for both 3-class and 2-class segmentation was almost identical to the ground truth. Conclusions: We established a semantic segmentation dataset spanning 10 stages uniformly from 179 patients. The prediction accuracy for clean mucosa was significantly high (above 94%). Our VS equation can approximately measure the region of clean mucosa. These results confirmed our dataset to be ideal for an accurate and quantitative assessment of AI-based bowel cleanliness.

Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges

Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.

Clinical feasibility of panintestinal (or panenteric) capsule endoscopy: a systematic review

In recent years, panintestinal capsule endoscopy (PCE) with double-headed capsules has been used to perform complete, single-sitting exploration of both small bowel and colon in different clinical conditions. Double-headed capsules for colonic examination (CCE) have been exploited first in this setting, followed by newer generations of capsules (i.e. PillCam Crohn, PCC) specifically engineered for this purpose. The aim of this study was to evaluate the feasibility of PCE in the form of a systematic review. We performed a comprehensive literature search to identify papers in which CE was specifically used for a PCE of the gastrointestinal tract. Data on CE, bowel preparation regimen, rate of cleanliness and completeness, and data on transit times were analyzed. The primary outcome was to assess the feasibility of a whole-gut exploration with CE. Sixteen (n = 16) studies including 915 CE procedures with CCE1 (n = 134), CCE2 (n = 357) and PCC (n = 424) were included. 13/16 studies were performed in the setting of Crohn's disease. Cleanliness and completeness rates were acceptable in all studies, ranging from 63.9% and 68.6% to 100%, respectively. In conclusion, PCE is a feasible technique, although further structured studies are needed to explore its full potential.

A Current and Newly Proposed Artificial Intelligence Algorithm for Reading Small Bowel Capsule Endoscopy

Small bowel capsule endoscopy (SBCE) is one of the most useful methods for diagnosing small bowel mucosal lesions. However, it takes a long time to interpret the capsule images. To solve this problem, artificial intelligence (AI) algorithms for SBCE readings are being actively studied. In this article, we analyzed several studies that applied AI algorithms to SBCE readings, such as automatic lesion detection, automatic classification of bowel cleanliness, and automatic compartmentalization of small bowels. In addition to automatic lesion detection using AI algorithms, a new direction of AI algorithms related to shorter reading times and improved lesion detection accuracy should be considered. Therefore, it is necessary to develop an integrated AI algorithm composed of algorithms with various functions in order to be used in clinical practice.

Development and Verification of a Deep Learning Algorithm to Evaluate Small-Bowel Preparation Quality

Capsule endoscopy (CE) quality control requires an objective scoring system to evaluate the preparation of the small bowel (SB). We propose a deep learning algorithm to calculate SB cleansing scores and verify the algorithm’s performance. A 5-point scoring system based on clarity of mucosal visualization was used to develop the deep learning algorithm (400,000 frames; 280,000 for training and 120,000 for testing). External validation was performed using additional CE cases (n = 50), and average cleansing scores (1.0 to 5.0) calculated using the algorithm were compared to clinical grades (A to C) assigned by clinicians. Test results obtained using 120,000 frames exhibited 93% accuracy. The separate CE case exhibited substantial agreement between the deep learning algorithm scores and clinicians’ assessments (Cohen’s kappa: 0.672). In the external validation, the cleansing score decreased with worsening clinical grade (scores of 3.9, 3.2, and 2.5 for grades A, B, and C, respectively, p < 0.001). Receiver operating characteristic curve analysis revealed that a cleansing score cut-off of 2.95 indicated clinically adequate preparation. This algorithm provides an objective and automated cleansing score for evaluating SB preparation for CE. The results of this study will serve as clinical evidence supporting the practical use of deep learning algorithms for evaluating SB preparation quality.