Deep learning-based endoscopic anatomy classification: an accelerated approach for data preparation and model validation

Artificial intelligence for gastric cancer in endoscopy: From diagnostic reasoning to market

Recognition of gastric conditions during endoscopy exams, including gastric cancer, usually requires specialized training and a long learning curve. Besides that, the interobserver variability is frequently high due to the different morphological characteristics of the lesions and grades of mucosal inflammation. In this sense, artificial intelligence tools based on deep learning models have been developed to support physicians to detect, classify, and predict gastric lesions more efficiently. Even though a growing number of studies exists in the literature, there are multiple challenges to bring a model to practice in this field, such as the need for more robust validation studies and regulatory hurdles. Therefore, the aim of this review is to provide a comprehensive assessment of the current use of artificial intelligence applied to endoscopic imaging to evaluate gastric precancerous and cancerous lesions and the barriers to widespread implementation of this technology in clinical routine.

Current status of quality control in screening esophagogastroduodenoscopy and the emerging role of artificial intelligence

Esophagogastroduodenoscopy (EGD) screening is being implemented in countries with a high incidence of upper gastrointestinal (UGI) cancer. High-quality EGD screening ensures the yield of early diagnosis and prevents suffering from advanced UGI cancer and minimal operational-related discomfort. However, performance varied dramatically among endoscopists, and quality control for EGD screening remains suboptimal. Guidelines have recommended potential measures for endoscopy quality improvement and research has been conducted for evidence. Moreover, artificial intelligence offers a promising solution for computer-aided diagnosis and quality control during EGD examinations. In this review, we summarized the key points for quality assurance in EGD screening based on current guidelines and evidence. We also outline the latest evidence, limitations, and future prospects of the emerging role of artificial intelligence in EGD quality control, aiming to provide a foundation for improving the quality of EGD screening.

Preparing Well for Esophageal Endoscopic Detection Using a Hybrid Model and Transfer Learning

Early detection of esophageal cancer through endoscopic imaging is pivotal for effective treatment. However, the intricacies of endoscopic diagnosis, contingent on the physician's expertise, pose challenges. Esophageal cancer features often manifest ambiguously, leading to potential confusions with other inflammatory esophageal conditions, thereby complicating diagnostic accuracy. In recent times, computer-aided diagnosis has emerged as a promising solution in medical imaging, particularly within the domain of endoscopy. Nonetheless, contemporary AI-based diagnostic models heavily rely on voluminous data sources, limiting their applicability, especially in scenarios with scarce datasets. To address this limitation, our study introduces novel data training strategies based on transfer learning, tailored to optimize performance with limited data. Additionally, we propose a hybrid model integrating EfficientNet and Vision Transformer networks to enhance prediction accuracy. Conducting rigorous evaluations on a carefully curated dataset comprising 1002 endoscopic images (comprising 650 white-light images and 352 narrow-band images), our model achieved exceptional outcomes. Our combined model achieved an accuracy of 96.32%, precision of 96.44%, recall of 95.70%, and f1-score of 96.04%, surpassing state-of-the-art models and individual components, substantiating its potential for precise medical image classification. The AI-based medical image prediction platform presents several advantageous characteristics, encompassing superior prediction accuracy, a compact model size, and adaptability to low-data scenarios. This research heralds a significant stride in the advancement of computer-aided endoscopic imaging for improved esophageal cancer diagnosis.

Chemoprevention of Colitis-Associated Dysplasia or Cancer in Inflammatory Bowel Disease

The association between inflammatory bowel disease and colorectal cancer is well known. Although the overall incidence of inflammatory bowel disease has declined recently, patients with this disease still have a 1.7-fold increased risk of colorectal cancer. The risk factors for developing colorectal cancer include extensive colitis, young age at diagnosis, disease duration, primary sclerosing cholangitis, chronic colonic mucosal inflammation, dysplasia lesion, and post-inflammatory polyps. In patients with inflammatory bowel disease, control of chronic inflammation and surveillance colonoscopies are important for the prevention of colorectal cancer. The 2017 guidelines from the European Crohn's and Colitis Organisation suggest that colonoscopies to screen for colorectal cancer should be performed when inflammatory bowel disease symptoms have lasted for 8 years. Current evidence supports the use of chemoprevention therapy with mesalamine to reduce the risk of colorectal cancer in patients with ulcerative colitis. Other compounds, including thiopurine, folic acid, statin, and tumor necrosis factor-α inhibitor, are controversial. Large surveillance cohort studies with longer follow-up duration are needed to evaluate the impact of drugs on colorectal cancer risks.

Development and validation of a deep learning-based algorithm for colonoscopy quality assessment

BACKGROUND

Quality indicators should be assessed and monitored to improve colonoscopy quality in clinical practice. Endoscopists must enter relevant information in the endoscopy reporting system to facilitate data collection, which may be inaccurate. The current study aimed to develop a full deep learning-based algorithm to identify and analyze intra-procedural colonoscopy quality indicators based on endoscopy images obtained during the procedure.

METHODS

A deep learning system for classifying colonoscopy images for quality assurance purposes was developed and its performance was assessed with an independent dataset. The system was utilized to analyze captured images and results were compared with those of real-world reports.

RESULTS

In total, 10,417 images from the hospital endoscopy database and 3157 from Hyper-Kvasir open dataset were utilized to develop the quality assurance algorithm. The overall accuracy of the algorithm was 96.72% and that of the independent test dataset was 94.71%. Moreover, 761 real-world reports and colonoscopy images were analyzed. The accuracy of electronic reports about cecal intubation rate was 99.34% and that of the algorithm was 98.95%. The agreement rate for the assessment of polypectomy rates using the electronic reports and the algorithm was 0.87 (95% confidence interval 0.83-0.90). A good correlation was found between the withdrawal time calculated using the algorithm and that entered by the physician (correlation coefficient r = 0.959, p < 0.0001).

CONCLUSION

We proposed a novel deep learning-based algorithm that used colonoscopy images for quality assurance purposes. This model can be used to automatically assess intra-procedural colonoscopy quality indicators in clinical practice.

Comparison Between Same-Day and Split-Dose Preparations with Sodium Picosulfate/Magnesium Citrate: A Randomized Noninferiority Study

BACKGROUND

Sodium picosulfate/magnesium citrate (SPMC) is a small-volume bowel cleansing agent with similar efficacy to and better tolerability than polyethylene glycol. However, we found no data on which SPMC preparation (same-day vs. split-dose) provides better bowel cleansing efficacy for afternoon colonoscopy.

AIMS

To compare bowel cleansing efficacy of different timing of the regimen.

METHODS

This randomized, single-center, endoscopist-blinded, noninferior study compared same-day and split-dose SPMC preparations for afternoon colonoscopy in 101 and 96 patients, respectively. We also included a prospective observation group of 100 patients receiving morning colonoscopy to compare bowel preparation between morning and afternoon colonoscopies. Bowel cleansing efficacy was then evaluated by the Aronchick Scale, Ottawa Bowel Preparation Scale (OBPS), Boston Bowel Preparation Scale (BBPS), and the Bubble Scale.

RESULTS

Same-day and split-dose preparations were similar in efficacy in all four scales. In the Aronchick Scale, the success rate (excellent and good cleanliness) was higher in same-day preparation than in split-dose preparation (100% vs. 92.8%). The same-day preparation also obtained a better OBPS score (1.4 vs. 2.1), but BBPS showed no difference between such groups (7.7 vs. 7.4).

CONCLUSION

Same-day preparation with SPMC is not inferior to split-dose preparation for afternoon colonoscopy.

Upper endoscopy photodocumentation quality evaluation with novel deep learning system

OBJECTIVES

Visualization and photodocumentation during endoscopy procedures are suggested to be one indicator for endoscopy performance quality. However, this indicator is difficult to measure and audit manually in clinical practice. Artificial intelligence (AI) is an emerging technology that may solve this problem.

METHODS

A deep learning model with an accuracy of 96.64% was developed from 15,305 images for upper endoscopy anatomy classification in the unit. Endoscopy images for asymptomatic patients receiving screening endoscopy were evaluated with this model to assess the completeness of photodocumentation rate.

RESULTS

A total of 15,723 images from 472 upper endoscopies performed by 12 endoscopists were enrolled. The complete photodocumentation rate from the pharynx to the duodenum was 53.8% and from the esophagus to the duodenum was 78.0% in this study. Endoscopists with a higher adenoma detection rate had a higher complete examination rate from the pharynx to duodenum (60.0% vs. 38.7%, P < 0.0001) and from esophagus to duodenum (83.0% vs. 65.7%, P < 0.0001) compared with endoscopists with lower adenoma detection rate. The pharynx, gastric angle, gastric retroflex view, gastric antrum, and the first portion of duodenum are likely to be missed by endoscopists with lower adenoma detection rates.

CONCLUSIONS

We report the use of a deep learning model to audit endoscopy photodocumentation quality in our unit. Endoscopists with better performance in colonoscopy had a better performance for this quality indicator. The use of such an AI system may help the endoscopy unit audit endoscopy performance.

Artificial Intelligence for Upper Gastrointestinal Endoscopy: A Roadmap from Technology Development to Clinical Practice

Stomach cancer is the third deadliest type of cancer in the world (0.86 million deaths in 2017). In 2035, a 20% increase will be observed both in incidence and mortality due to demographic effects if no interventions are foreseen. Upper GI endoscopy (UGIE) plays a paramount role in early diagnosis and, therefore, improved survival rates. On the other hand, human and technical factors can contribute to misdiagnosis while performing UGIE. In this scenario, artificial intelligence (AI) has recently shown its potential in compensating for the pitfalls of UGIE, by leveraging deep learning architectures able to efficiently recognize endoscopic patterns from UGIE video data. This work presents a review of the current state-of-the-art algorithms in the application of AI to gastroscopy. It focuses specifically on the threefold tasks of assuring exam completeness (i.e., detecting the presence of blind spots) and assisting in the detection and characterization of clinical findings, both gastric precancerous conditions and neoplastic lesion changes. Early and promising results have already been obtained using well-known deep learning architectures for computer vision, but many algorithmic challenges remain in achieving the vision of AI-assisted UGIE. Future challenges in the roadmap for the effective integration of AI tools within the UGIE clinical practice are discussed, namely the adoption of more robust deep learning architectures and methods able to embed domain knowledge into image/video classifiers as well as the availability of large, annotated datasets.

Forrest Classification for Bleeding Peptic Ulcer: A New Look at the Old Endoscopic Classification

The management of peptic ulcer bleeding is clinically challenging. For decades, the Forrest classification has been used for risk stratification for nonvariceal ulcer bleeding. The perception and interpretation of the Forrest classification vary among different endoscopists. The relationship between the bleeder and ulcer images and the different stages of the Forrest classification has not been studied yet. Endoscopic still images of 276 patients with peptic ulcer bleeding for the past 3 years were retrieved and reviewed. The intra-rater agreement and inter-rater agreement were compared. The obtained endoscopic images were manually drawn to delineate the extent of the ulcer and bleeding area. The areas of the region of interest were compared between the different stages of the Forrest classification. A total of 276 images were first classified by two experienced tutor endoscopists. The images were reviewed by six other endoscopists. A good intra-rater correlation was observed (0.92–0.98). A good inter-rater correlation was observed among the different levels of experience (0.639–0.859). The correlation was higher among tutor and junior endoscopists than among experienced endoscopists. Low-risk Forrest IIC and III lesions show distinct patterns compared to high-risk Forrest I, IIA, or IIB lesions. We found good agreement of the Forrest classification among different endoscopists in a single institution. This is the first study to quantitively analyze the obtained and explain the distinct patterns of bleeding ulcers from endoscopy images.

Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos

Esophagogastroduodenoscopy (EGD) is a critical step in the diagnosis of upper gastrointestinal disorders. However, due to inexperience or high workload, there is a wide variation in EGD performance by endoscopists. Variations in performance may result in exams that do not completely cover all anatomical locations of the stomach, leading to a potential risk of missed diagnosis of gastric diseases. Numerous guidelines or expert consensus have been proposed to assess and optimize the quality of endoscopy. However, there is a lack of mature and robust methods to accurately apply to real clinical real-time video environments. In this paper, we innovatively define the problem of recognizing anatomical locations in videos as a multi-label recognition task. This can be more consistent with the model learning of image-to-label mapping relationships. We propose a combined structure of a deep learning model (GL-Net) that combines a graph convolutional network (GCN) with long short-term memory (LSTM) networks to both extract label features and correlate temporal dependencies for accurate real-time anatomical locations identification in gastroscopy videos. Our methodological evaluation dataset is based on complete videos of real clinical examinations. A total of 29,269 images from 49 videos were collected as a dataset for model training and validation. Another 1736 clinical videos were retrospectively analyzed and evaluated for the application of the proposed model. Our method achieves 97.1% mean accuracy (mAP), 95.5% mean per-class accuracy and 93.7% average overall accuracy in a multi-label classification task, and is able to process these videos in real-time at 29.9 FPS. In addition, based on our approach, we designed a system to monitor routine EGD videos in detail and perform statistical analysis of the operating habits of endoscopists, which can be a useful tool to improve the quality of clinical endoscopy.

A Nomogram for Predicting Laparoscopic and Endoscopic Cooperative Surgery during the Endoscopic Resection of Subepithelial Tumors of the Upper Gastrointestinal Tract

Background: Considering the widespread use of esophagogastroduodenoscopy, the prevalence of upper gastrointestinal (GI) subepithelial tumors (SET) increases. For relatively safer removal of upper GI SETs, endoscopic submucosal dissection (ESD) has been developed as an alternative to surgery. This study aimed to analyze the outcome of endoscopic resection for SETs and develop a prediction model for the need for laparoscopic and endoscopic cooperative surgery (LECS) during the procedure. Method: We retrospectively analyzed 123 patients who underwent endoscopic resection for upper GI SETs between January 2012 and December 2020 at our institution. Intraoperatively, they underwent ESD or submucosal tunneling endoscopic resection (STER). Results: ESD and STER were performed in 107 and 16 patients, respectively. The median age was 55 years, and the average tumor size was 1.5 cm. En bloc resection was achieved in 114 patients (92.7%). The median follow-up duration was 242 days without recurrence. Perforation occurred in 47 patients (38.2%), and 30 patients (24.4%) underwent LECS. Most perforations occurred in the fundus. Through multivariable analysis, we built a nomogram that can predict LECS requirement according to tumor location, size, patient age, and sex. The prediction model exhibited good discrimination ability, with an area under the curve (AUC) of 0.893. Conclusions: Endoscopic resection is a noninvasive procedure for small upper-GI SETs. Most perforations can be successfully managed endoscopically. The prediction model for LECS requirement is useful in treatment planning.