Endoscopic detection and differentiation of esophageal lesions using a deep neural network

Surveillance for metachronous cancers after endoscopic resection of esophageal squamous cell carcinoma

The literature pertaining to surveillance following treatment for esophageal squamous cell carcinoma (SCC) was reviewed and summarized, encompassing the current status and future perspectives. Analysis of the standardized mortality and incidence ratios for these cancers indicates an elevated risk of cancer in the oral cavity, pharynx, larynx, and lungs among patients with esophageal SCC compared to the general population. To enhance the efficacy of surveillance for these metachronous cancers, risk stratification is needed. Various factors, including multiple Lugol-voiding lesions, multiple foci of dilated vascular areas, young age, and high mean corpuscular volume, have been identified as predictors of metachronous SCCs. Current practice involves stratifying the risk of metachronous esophageal and head/neck SCCs based on the presence of multiple Lugol-voiding lesions. Endoscopic surveillance, scheduled 6-12 months post-endoscopic resection, has demonstrated effectiveness, with over 90% of metachronous esophageal SCCs treatable through minimally invasive modalities. Narrow-band imaging emerges as the preferred surveillance method for esophageal and head/neck SCC based on comparative studies of various imaging techniques. Innovative approaches, such as artificial intelligence-assisted detection systems and radiofrequency ablation of high-risk background mucosa, may improve outcomes in patients following endoscopic resection.

Endoscopic Artificial Intelligence for Image Analysis in Gastrointestinal Neoplasms

BACKGROUND

Artificial intelligence (AI) using deep learning systems has recently been utilized in various medical fields. In the field of gastroenterology, AI is primarily implemented in image recognition and utilized in the realm of gastrointestinal (GI) endoscopy. In GI endoscopy, computer-aided detection/diagnosis (CAD) systems assist endoscopists in GI neoplasm detection or differentiation of cancerous or noncancerous lesions. Several AI systems for colorectal polyps have already been applied in colonoscopy clinical practices. In esophagogastroduodenoscopy, a few CAD systems for upper GI neoplasms have been launched in Asian countries. The usefulness of these CAD systems in GI endoscopy has been gradually elucidated.

SUMMARY

In this review, we outline recent articles on several studies of endoscopic AI systems for GI neoplasms, focusing on esophageal squamous cell carcinoma (ESCC), esophageal adenocarcinoma (EAC), gastric cancer (GC), and colorectal polyps. In ESCC and EAC, computer-aided detection (CADe) systems were mainly developed, and a recent meta-analysis study showed sensitivities of 91.2% and 93.1% and specificities of 80% and 86.9%, respectively. In GC, a recent meta-analysis study on CADe systems demonstrated that their sensitivity and specificity were as high as 90%. A randomized controlled trial (RCT) also showed that the use of the CADe system reduced the miss rate. Regarding computer-aided diagnosis (CADx) systems for GC, although RCTs have not yet been conducted, most studies have demonstrated expert-level performance. In colorectal polyps, multiple RCTs have shown the usefulness of the CADe system for improving the polyp detection rate, and several CADx systems have been shown to have high accuracy in colorectal polyp differentiation.

KEY MESSAGES

Most analyses of endoscopic AI systems suggested that their performance was better than that of nonexpert endoscopists and equivalent to that of expert endoscopists. Thus, endoscopic AI systems may be useful for reducing the risk of overlooking lesions and improving the diagnostic ability of endoscopists.

Esophageal cancer screening, early detection and treatment: Current insights and future directions

Esophageal cancer ranks among the most prevalent malignant tumors globally, primarily due to its highly aggressive nature and poor survival rates. According to the 2020 global cancer statistics, there were approximately 604000 new cases of esophageal cancer, resulting in 544000 deaths. The 5-year survival rate hovers around a mere 15%-25%. Notably, distinct variations exist in the risk factors associated with the two primary histological types, influencing their worldwide incidence and distribution. Squamous cell carcinoma displays a high incidence in specific regions, such as certain areas in China, where it meets the cost-effectiveness criteria for widespread endoscopy-based early diagnosis within the local population. Conversely, adenocarcinoma (EAC) represents the most common histological subtype of esophageal cancer in Europe and the United States. The role of early diagnosis in cases of EAC originating from Barrett's esophagus (BE) remains a subject of controversy. The effectiveness of early detection for EAC, particularly those arising from BE, continues to be a debated topic. The variations in how early-stage esophageal carcinoma is treated in different regions are largely due to the differing rates of early-stage cancer diagnoses. In areas with higher incidences, such as China and Japan, early diagnosis is more common, which has led to the advancement of endoscopic methods as definitive treatments. These techniques have demonstrated remarkable efficacy with minimal complications while preserving esophageal functionality. Early screening, prompt diagnosis, and timely treatment are key strategies that can significantly lower both the occurrence and death rates associated with esophageal cancer.

Ensembled CNN with artificial bee colony optimization method for esophageal cancer stage classification using SVM classifier

BACKGROUND

Esophageal cancer (EC) is aggressive cancer with a high fatality rate and a rapid rise of the incidence globally. However, early diagnosis of EC remains a challenging task for clinicians.

OBJECTIVE

To help address and overcome this challenge, this study aims to develop and test a new computer-aided diagnosis (CAD) network that combines several machine learning models and optimization methods to detect EC and classify cancer stages.

METHODS

The study develops a new deep learning network for the classification of the various stages of EC and the premalignant stage, Barrett's Esophagus from endoscopic images. The proposed model uses a multi-convolution neural network (CNN) model combined with Xception, Mobilenetv2, GoogLeNet, and Darknet53 for feature extraction. The extracted features are blended and are then applied on to wrapper based Artificial Bee Colony (ABC) optimization technique to grade the most accurate and relevant attributes. A multi-class support vector machine (SVM) classifies the selected feature set into the various stages. A study dataset involving 523 Barrett's Esophagus images, 217 ESCC images and 288 EAC images is used to train the proposed network and test its classification performance.

RESULTS

The proposed network combining Xception, mobilenetv2, GoogLeNet, and Darknet53 outperforms all the existing methods with an overall classification accuracy of 97.76% using a 3-fold cross-validation method.

CONCLUSION

This study demonstrates that a new deep learning network that combines a multi-CNN model with ABC and a multi-SVM is more efficient than those with individual pre-trained networks for the EC analysis and stage classification.

Detecting the location of lung cancer on thoracoscopic images using deep convolutional neural networks

OBJECTIVES

The prevalence of minimally invasive surgeries has increased the need for tumor detection using thoracoscopic images during lung cancer surgery. We conducted this study to analyze the efficacy of a deep convolutional neural network (DCNN) for tumor detection using recorded thoracoscopic images of pulmonary surfaces.

MATERIALS AND METHODS

We collected 644 intraoperative thoracoscopic images of changes in pulmonary appearance from 427 patients with lung cancer between 2012 and 2021. The lesion areas on the thoracoscopic images were detected by bounding boxes using an advanced version of YOLO, a well-known DCNN for object detection. The DCNN model was trained and evaluated by a 15-fold cross-validation scheme. Each predicted bounding box was considered successful detection when it overlapped more than 50% of the lesion areas annotated by board-certified surgeons.

RESULTS AND CONCLUSIONS

Precision, recall, and F1-measured values of 91.9%, 90.5%, and 91.1%, respectively, were obtained. The presence of lymphatic vessel invasion was associated with successful detection (p = 0.045). The presence of pathological pleural invasion also showed a tendency toward successful detection (p = 0.081). The proposed DCNN-based algorithm yielded an accuracy of more than 90% tumor detection. These algorithms will help surgeons detect lung cancer displayed on a screen automatically.

The role of artificial intelligence in the endoscopic diagnosis of esophageal cancer: a systematic review and meta-analysis

Early detection of esophageal cancer is limited by accurate endoscopic diagnosis of subtle macroscopic lesions. Endoscopic interpretation is subject to expertise, diagnostic skill, and thus human error. Artificial intelligence (AI) in endoscopy is increasingly bridging this gap. This systematic review and meta-analysis consolidate the evidence on the use of AI in the endoscopic diagnosis of esophageal cancer. The systematic review was carried out using Pubmed, MEDLINE and Ovid EMBASE databases and articles on the role of AI in the endoscopic diagnosis of esophageal cancer management were included. A meta-analysis was also performed. Fourteen studies (1590 patients) assessed the use of AI in endoscopic diagnosis of esophageal squamous cell carcinoma-the pooled sensitivity and specificity were 91.2% (84.3-95.2%) and 80% (64.3-89.9%). Nine studies (478 patients) assessed AI capabilities of diagnosing esophageal adenocarcinoma with the pooled sensitivity and specificity of 93.1% (86.8-96.4) and 86.9% (81.7-90.7). The remaining studies formed the qualitative summary. AI technology, as an adjunct to endoscopy, can assist in accurate, early detection of esophageal malignancy. It has shown superior results to endoscopists alone in identifying early cancer and assessing depth of tumor invasion, with the added benefit of not requiring a specialized skill set. Despite promising results, the application in real-time endoscopy is limited, and further multicenter trials are required to accurately assess its use in routine practice.

Application of convolutional neural network-based endoscopic imaging in esophageal cancer or high-grade dysplasia: A systematic review and meta-analysis

BACKGROUND

Esophageal cancer is the seventh-most common cancer type worldwide, accounting for 5% of death from malignancy. Development of novel diagnostic techniques has facilitated screening, early detection, and improved prognosis. Convolutional neural network (CNN)-based image analysis promises great potential for diagnosing and determining the prognosis of esophageal cancer, enabling even early detection of dysplasia.

AIM

To conduct a meta-analysis of the diagnostic accuracy of CNN models for the diagnosis of esophageal cancer and high-grade dysplasia (HGD).

METHODS

PubMed, EMBASE, Web of Science and Cochrane Library databases were searched for articles published up to November 30, 2022. We evaluated the diagnostic accuracy of using the CNN model with still image-based analysis and with video-based analysis for esophageal cancer or HGD, as well as for the invasion depth of esophageal cancer. The pooled sensitivity, pooled specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and area under the curve (AUC) were estimated, together with the 95% confidence intervals (CI). A bivariate method and hierarchical summary receiver operating characteristic method were used to calculate the diagnostic test accuracy of the CNN model. Meta-regression and subgroup analyses were used to identify sources of heterogeneity.

RESULTS

A total of 28 studies were included in this systematic review and meta-analysis. Using still image-based analysis for the diagnosis of esophageal cancer or HGD provided a pooled sensitivity of 0.95 (95%CI: 0.92-0.97), pooled specificity of 0.92 (0.89-0.94), PLR of 11.5 (8.3-16.0), NLR of 0.06 (0.04-0.09), DOR of 205 (115-365), and AUC of 0.98 (0.96-0.99). When video-based analysis was used, a pooled sensitivity of 0.85 (0.77-0.91), pooled specificity of 0.73 (0.59-0.83), PLR of 3.1 (1.9-5.0), NLR of 0.20 (0.12-0.34), DOR of 15 (6-38) and AUC of 0.87 (0.84-0.90) were found. Prediction of invasion depth resulted in a pooled sensitivity of 0.90 (0.87-0.92), pooled specificity of 0.83 (95%CI: 0.76-0.88), PLR of 7.8 (1.9-32.0), NLR of 0.10 (0.41-0.25), DOR of 118 (11-1305), and AUC of 0.95 (0.92-0.96).

CONCLUSION

CNN-based image analysis in diagnosing esophageal cancer and HGD is an excellent diagnostic method with high sensitivity and specificity that merits further investigation in large, multicenter clinical trials.

Blue light imaging and linked color imaging as a screening mode for esophageal squamous cell carcinoma in high-risk patients: Multicenter randomized trial

OBJECTIVES

Blue light imaging (BLI) and linked color imaging (LCI) are superior to conventional white light imaging for detecting esophageal squamous cell carcinoma (ESCC). Hence, we compared their diagnostic performances in ESCC screening.

METHODS

This open-labeled, randomized controlled trial was performed at seven hospitals. Patients with a high risk of ESCC were randomly assigned to the BLI group (BLI followed by LCI) and LCI group (LCI followed by BLI). The primary end-point was the detection rate of ESCC in the primary mode. The main secondary end-point was its miss rate in the primary mode.

RESULTS

In total, 699 patients were enrolled. The detection rate of ESCC did not significantly differ between the BLI and LCI groups (4.0% [14/351] vs. 4.9% [17/348]; P = 0.565); however, the number of patients with ESCC tended to be smaller in the BLI group (19 vs. 30). Notably, the miss rate of ESCC was lower in the BLI group (26.3% [5/19] vs. 63.3% [19/30]; P = 0.012) and LCI detected no ESCCs missed by BLI. The sensitivity was higher in BLI (75.0% vs. 47.6%; P = 0.042); on the other hand, the positive predictive value in BLI tended to be lower (28.8% vs. 45.5%; P = 0.092).

CONCLUSIONS

The detection rates of ESCC did not significantly differ between BLI and LCI. Although BLI may have the potential to be advantageous over LCI for the diagnosis of ESCC, it is still unclear whether BLI is superior to LCI, and a further large-scale study is needed.

TRIAL REGISTRATION

Japan Registry of Clinical Trials (jRCT1022190018-1).

The Advent of Domain Adaptation into Artificial Intelligence for Gastrointestinal Endoscopy and Medical Imaging

Artificial intelligence (AI) is a subfield of computer science that aims to implement computer systems that perform tasks that generally require human learning, reasoning, and perceptual abilities. AI is widely used in the medical field. The interpretation of medical images requires considerable effort, time, and skill. AI-aided interpretations, such as automated abnormal lesion detection and image classification, are promising areas of AI. However, when images with different characteristics are extracted, depending on the manufacturer and imaging environment, a so-called domain shift problem occurs in which the developed AI has a poor versatility. Domain adaptation is used to address this problem. Domain adaptation is a tool that generates a newly converted image which is suitable for other domains. It has also shown promise in reducing the differences in appearance among the images collected from different devices. Domain adaptation is expected to improve the reading accuracy of AI for heterogeneous image distributions in gastrointestinal (GI) endoscopy and medical image analyses. In this paper, we review the history and basic characteristics of domain shift and domain adaptation. We also address their use in gastrointestinal endoscopy and the medical field more generally through published examples, perspectives, and future directions.

The Importance of Artificial Intelligence in Upper Gastrointestinal Endoscopy

Recently, there has been a growing interest in the application of artificial intelligence (AI) in medicine, especially in specialties where visualization methods are applied. AI is defined as a computer's ability to achieve human cognitive performance, which is accomplished through enabling computer "learning". This can be conducted in two ways, as machine learning and deep learning. Deep learning is a complex learning system involving the application of artificial neural networks, whose algorithms imitate the human form of learning. Upper gastrointestinal endoscopy allows examination of the esophagus, stomach and duodenum. In addition to the quality of endoscopic equipment and patient preparation, the performance of upper endoscopy depends on the experience and knowledge of the endoscopist. The application of artificial intelligence in endoscopy refers to computer-aided detection and the more complex computer-aided diagnosis. The application of AI in upper endoscopy is aimed at improving the detection of premalignant and malignant lesions, with special attention on the early detection of dysplasia in Barrett's esophagus, the early detection of esophageal and stomach cancer and the detection of H. pylori infection. Artificial intelligence reduces the workload of endoscopists, is not influenced by human factors and increases the diagnostic accuracy and quality of endoscopic methods.

Lightweight deep learning model incorporating an attention mechanism and feature fusion for automatic classification of gastric lesions in gastroscopic images

Accurate diagnosis of various lesions in the formation stage of gastric cancer is an important problem for doctors. Automatic diagnosis tools based on deep learning can help doctors improve the accuracy of gastric lesion diagnosis. Most of the existing deep learning-based methods have been used to detect a limited number of lesions in the formation stage of gastric cancer, and the classification accuracy needs to be improved. To this end, this study proposed an attention mechanism feature fusion deep learning model with only 14 million (M) parameters. Based on that model, the automatic classification of a wide range of lesions covering the stage of gastric cancer formation was investigated, including non-neoplasm(including gastritis and intestinal metaplasia), low-grade intraepithelial neoplasia, and early gastric cancer (including high-grade intraepithelial neoplasia and early gastric cancer). 4455 magnification endoscopy with narrow-band imaging(ME-NBI) images from 1188 patients were collected to train and test the proposed method. The results of the test dataset showed that compared with the advanced gastric lesions classification method with the best performance (overall accuracy = 94.3%, parameters = 23.9 M), the proposed method achieved both higher overall accuracy and a relatively lightweight model (overall accuracy =95.6%, parameter = 14 M). The accuracy, sensitivity, and specificity of low-grade intraepithelial neoplasia were 94.5%, 93.0%, and 96.5%, respectively, achieving state-of-the-art classification performance. In conclusion, our method has demonstrated its potential in diagnosing various lesions at the stage of gastric cancer formation.

An Image Turing Test on Realistic Gastroscopy Images Generated by Using the Progressive Growing of Generative Adversarial Networks

Generative adversarial networks (GAN) in medicine are valuable techniques for augmenting unbalanced rare data, anomaly detection, and avoiding patient privacy issues. However, there were limits to generating high-quality endoscopic images with various characteristics, such as peristalsis, viewpoints, light sources, and mucous patterns. This study used the progressive growing of GAN (PGGAN) within the normal distribution dataset to confirm the ability to generate high-quality gastrointestinal images and investigated what barriers PGGAN has to generate endoscopic images. We trained the PGGAN with 107,060 gastroscopy images from 4165 normal patients to generate highly realistic 5122 pixel-sized images. For the evaluation, visual Turing tests were conducted on 100 real and 100 synthetic images to distinguish the authenticity of images by 19 endoscopists. The endoscopists were divided into three groups based on their years of clinical experience for subgroup analysis. The overall accuracy, sensitivity, and specificity of the 19 endoscopist groups were 61.3%, 70.3%, and 52.4%, respectively. The mean accuracy of the three endoscopist groups was 62.4 [Group I], 59.8 [Group II], and 59.1% [Group III], which was not considered a significant difference. There were no statistically significant differences in the location of the stomach. However, the real images with the anatomical landmark pylorus had higher detection sensitivity. The images generated by PGGAN showed highly realistic depictions that were difficult to distinguish, regardless of their expertise as endoscopists. However, it was necessary to establish GANs that could better represent the rugal folds and mucous membrane texture.

Global development of artificial intelligence in cancer field: a bibliometric analysis range from 1983 to 2022

Background

Artificial intelligence (AI) is widely applied in cancer field nowadays. The aim of this study is to explore the hotspots and trends of AI in cancer research.

Methods

The retrieval term includes four topic words ("tumor," "cancer," "carcinoma," and "artificial intelligence"), which were searched in the database of Web of Science from January 1983 to December 2022. Then, we documented and processed all data, including the country, continent, Journal Impact Factor, and so on using the bibliometric software.

Results

A total of 6,920 papers were collected and analyzed. We presented the annual publications and citations, most productive countries/regions, most influential scholars, the collaborations of journals and institutions, and research focus and hotspots in AI-based cancer research.

Conclusion

This study systematically summarizes the current research overview of AI in cancer research so as to lay the foundation for future research.

A single-center prospective study evaluating the usefulness of artificial intelligence for the diagnosis of esophageal squamous cell carcinoma in a real-time setting

BACKGROUND

Several pre-clinical studies have reported the usefulness of artificial intelligence (AI) systems in the diagnosis of esophageal squamous cell carcinoma (ESCC). We conducted this study to evaluate the usefulness of an AI system for real-time diagnosis of ESCC in a clinical setting.

METHODS

This study followed a single-center prospective single-arm non-inferiority design. Patients at high risk for ESCC were recruited and real-time diagnosis by the AI system was compared with that of endoscopists for lesions suspected to be ESCC. The primary outcomes were the diagnostic accuracy of the AI system and endoscopists. The secondary outcomes were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and adverse events.

RESULTS

A total of 237 lesions were evaluated. The accuracy, sensitivity, and specificity of the AI system were 80.6%, 68.2%, and 83.4%, respectively. The accuracy, sensitivity, and specificity of endoscopists were 85.7%, 61.4%, and 91.2%, respectively. The difference between the accuracy of the AI system and that of the endoscopists was - 5.1%, and the lower limit of the 90% confidence interval was less than the non-inferiority margin.

CONCLUSIONS

The non-inferiority of the AI system in comparison with endoscopists in the real-time diagnosis of ESCC in a clinical setting was not proven.

TRIAL REGISTRATION

Japan Registry of Clinical Trials (jRCTs052200015, 18/05/2020).

The current state of artificial intelligence in endoscopic diagnosis of early esophageal squamous cell carcinoma

Esophageal squamous cell carcinoma (ESCC) is a common malignant tumor of the digestive tract. The most effective method of reducing the disease burden in areas with a high incidence of esophageal cancer is to prevent the disease from developing into invasive cancer through screening. Endoscopic screening is key for the early diagnosis and treatment of ESCC. However, due to the uneven professional level of endoscopists, there are still many missed cases because of failure to recognize lesions. In recent years, along with remarkable progress in medical imaging and video evaluation technology based on deep machine learning, the development of artificial intelligence (AI) is expected to provide new auxiliary methods of endoscopic diagnosis and the treatment of early ESCC. The convolution neural network (CNN) in the deep learning model extracts the key features of the input image data using continuous convolution layers and then classifies images through full-layer connections. The CNN is widely used in medical image classification, and greatly improves the accuracy of endoscopic image classification. This review focuses on the AI-assisted diagnosis of early ESCC and prediction of early ESCC invasion depth under multiple imaging modalities. The excellent image recognition ability of AI is suitable for the detection and diagnosis of ESCC and can reduce missed diagnoses and help endoscopists better complete endoscopic examinations. However, the selective bias used in the training dataset of the AI system affects its general utility.

Artificial intelligence assisted detection of superficial esophageal squamous cell carcinoma in white-light endoscopic images by using a generalized system

BACKGROUND

The use of artificial intelligence (AI) assisted white light imaging (WLI) detection systems for superficial esophageal squamous cell carcinoma (SESCC) is limited by training with images from one specific endoscopy platform.

METHODS

In this study, we developed an AI system with a convolutional neural network (CNN) model using WLI images from Olympus and Fujifilm endoscopy platforms. The training dataset consisted of 5892 WLI images from 1283 patients, and the validation dataset included 4529 images from 1224 patients. We assessed the diagnostic performance of the AI system and compared it with that of endoscopists. We analyzed the system's ability to identify cancerous imaging characteristics and investigated the efficacy of the AI system as an assistant in diagnosis.

RESULTS

In the internal validation set, the AI system's per-image analysis had a sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) of 96.64%, 95.35%, 91.75%, 90.91%, and 98.33%, respectively. In patient-based analysis, these values were 90.17%, 94.34%, 88.38%, 89.50%, and 94.72%, respectively. The diagnostic results in the external validation set were also favorable. The CNN model's diagnostic performance in recognizing cancerous imaging characteristics was comparable to that of expert endoscopists and significantly higher than that of mid-level and junior endoscopists. This model was competent in localizing SESCC lesions. Manual diagnostic performances were significantly improved with the assistance by AI system, especially in terms of accuracy (75.12% vs. 84.95%, p = 0.008), specificity (63.29% vs. 76.59%, p = 0.017) and PPV (64.95% vs. 75.23%, p = 0.006).

CONCLUSIONS

The results of this study demonstrate that the developed AI system is highly effective in automatically recognizing SESCC, displaying impressive diagnostic performance, and exhibiting strong generalizability. Furthermore, when used as an assistant in the diagnosis process, the system improved manual diagnostic performance.

Artificial intelligence for detecting and delineating the extent of superficial esophageal squamous cell carcinoma and precancerous lesions under narrow-band imaging (with video)

BACKGROUND AND AIMS

Although narrow-band imaging (NBI) is a useful modality for detecting and delineating esophageal squamous cell carcinoma (ESCC), there is a risk of incorrectly determining the margins of some lesions even with NBI. This study aimed to develop an artificial intelligence (AI) system for detecting superficial ESCC and precancerous lesions and delineating the extent of lesions under NBI.

METHODS

Nonmagnified NBI images from 4 hospitals were collected and annotated. Internal and external image test datasets were used to evaluate the detection and delineation performance of the system. The delineation performance of the system was compared with that of endoscopists. Furthermore, the system was directly integrated into the endoscopy equipment, and its real-time diagnostic capability was prospectively estimated.

RESULTS

The system was trained and tested using 10,047 still images and 140 videos from 1112 patients and 1183 lesions. In the image testing, the accuracy of the system in detecting lesions in internal and external tests was 92.4% and 89.9%, respectively. The accuracy of the system in delineating extents in internal and external tests was 88.9% and 87.0%, respectively. The delineation performance of the system was superior to that of junior endoscopists and similar to that of senior endoscopists. In the prospective clinical evaluation, the system exhibited satisfactory performance, with an accuracy of 91.4% in detecting lesions and an accuracy of 85.9% in delineating extents.

CONCLUSIONS

The proposed AI system could accurately detect superficial ESCC and precancerous lesions and delineate the extent of lesions under NBI.

AI-assisted identification of intrapapillary capillary loops in magnification endoscopy for diagnosing early-stage esophageal squamous cell carcinoma: a preliminary study

Esophageal squamous cell carcinoma (ESCC) is one of the most common histological types of esophageal cancers. It can seriously affect public health, particularly in Eastern Asia. Early diagnosis and effective therapy of ESCC can significantly help improve patient prognoses. The visualization of intrapapillary capillary loops (IPCLs) under magnification endoscopy (ME) can greatly support the identification of ESCC occurrences by endoscopists. This paper proposes an artificial-intelligence-assisted endoscopic diagnosis approach using deep learning for localizing and identifying IPCLs to diagnose early-stage ESCC. An improved Faster region-based convolutional network (R-CNN) with a polarized self-attention (PSA)-HRNetV2p backbone was employed to automatically detect IPCLs in ME images. In our study, 2887 ME with blue laser imaging (ME-BLI) images of 246 patients and 493 ME with narrow-band imaging (ME-NBI) images of 81 patients were collected from multiple hospitals and used to train and test our detection model. The ME-NBI images were used as the external testing set to verify the generalizability of the model. The experimental evaluation revealed that the proposed method achieved a recall of 79.25%, precision of 75.54%, F1-score of 0.764 and mean average precision (mAP) of 74.95%. Our method outperformed other existing approaches in our evaluation. It can effectively improve the accuracy of ESCC detection and provide a useful adjunct to the assessment of early-stage ESCC for endoscopists.

ClusterNet: a clustering distributed prior embedded detection network for early-stage esophageal squamous cell carcinoma diagnosis

BACKGROUND

Early and accurate diagnosis of esophageal squamous cell carcinoma (ESCC) is important for reducing mortality. Analyzing intrapapillary capillary loops' (IPCLs) patterns on magnification endoscopy with narrow band imaging (ME-NBI) has been demonstrated effective in the diagnosis of early-stage ESCC. However, even experienced endoscopists may face difficulty in finding and classifying countless IPCLs on ME-NBI.

PURPOSE

We propose a novel clustering prior embedded detection network: ClusterNet. ClusterNet is capable of analyzing the distribution of IPCLs on ME-NBI automatically and enables endoscopists to overview multiple types of visualization. With ClusterNet assisting, endoscopists may observe ME-NBI images more efficiently, thus they may also predict the pathology and make medical decisions more easily.

METHODS

We propose the first large-scale ME-NBI dataset with fine-grained annotations by consensus of expert endoscopists. The dataset is splitted into a training set and an independent testing set based on patients. With two strategies for embedding, ClusterNet can automatically take the clustering effect into consideration. Prior to this work, none of the existing approaches take the clustering effect, which is rather important in classifying the IPCLs, into account.

RESULTS

ClusterNet achieves an average precision of 81.2% and an average recall of 90.0% for the detection of IPCLs patterns on each patient of the independent testing set. We also compare ClusterNet with other state-of-the-art detection approaches. The performance of ClusterNet with embedding strategies is consistently superior to that of other approaches in terms of average precision, recall and F2-Score.

CONCLUSIONS

Experiments demonstrate that our proposed method is able to detect almost all the IPCLs patterns on ME-NBI and classify them according to the Japanese Endoscopic Society (JES) classification accurately.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

Deep Learning for the Diagnosis of Esophageal Cancer in Endoscopic Images: A Systematic Review and Meta-Analysis

Esophageal cancer, one of the most common cancers with a poor prognosis, is the sixth leading cause of cancer-related mortality worldwide. Early and accurate diagnosis of esophageal cancer, thus, plays a vital role in choosing the appropriate treatment plan for patients and increasing their survival rate. However, an accurate diagnosis of esophageal cancer requires substantial expertise and experience. Nowadays, the deep learning (DL) model for the diagnosis of esophageal cancer has shown promising performance. Therefore, we conducted an updated meta-analysis to determine the diagnostic accuracy of the DL model for the diagnosis of esophageal cancer. A search of PubMed, EMBASE, Scopus, and Web of Science, between 1 January 2012 and 1 August 2022, was conducted to identify potential studies evaluating the diagnostic performance of the DL model for esophageal cancer using endoscopic images. The study was performed in accordance with PRISMA guidelines. Two reviewers independently assessed potential studies for inclusion and extracted data from retrieved studies. Methodological quality was assessed by using the QUADAS-2 guidelines. The pooled accuracy, sensitivity, specificity, positive and negative predictive value, and the area under the receiver operating curve (AUROC) were calculated using a random effect model. A total of 28 potential studies involving a total of 703,006 images were included. The pooled accuracy, sensitivity, specificity, and positive and negative predictive value of DL for the diagnosis of esophageal cancer were 92.90%, 93.80%, 91.73%, 93.62%, and 91.97%, respectively. The pooled AUROC of DL for the diagnosis of esophageal cancer was 0.96. Furthermore, there was no publication bias among the studies. The findings of our study show that the DL model has great potential to accurately and quickly diagnose esophageal cancer. However, most studies developed their model using endoscopic data from the Asian population. Therefore, we recommend further validation through studies of other populations as well.

Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement

This ESGE Position Statement defines the expected value of artificial intelligence (AI) for the diagnosis and management of gastrointestinal neoplasia within the framework of the performance measures already defined by ESGE. This is based on the clinical relevance of the expected task and the preliminary evidence regarding artificial intelligence in artificial or clinical settings. MAIN RECOMMENDATIONS:: (1) For acceptance of AI in assessment of completeness of upper GI endoscopy, the adequate level of mucosal inspection with AI should be comparable to that assessed by experienced endoscopists. (2) For acceptance of AI in assessment of completeness of upper GI endoscopy, automated recognition and photodocumentation of relevant anatomical landmarks should be obtained in ≥90% of the procedures. (3) For acceptance of AI in the detection of Barrett's high grade intraepithelial neoplasia or cancer, the AI-assisted detection rate for suspicious lesions for targeted biopsies should be comparable to that of experienced endoscopists with or without advanced imaging techniques. (4) For acceptance of AI in the management of Barrett's neoplasia, AI-assisted selection of lesions amenable to endoscopic resection should be comparable to that of experienced endoscopists. (5) For acceptance of AI in the diagnosis of gastric precancerous conditions, AI-assisted diagnosis of atrophy and intestinal metaplasia should be comparable to that provided by the established biopsy protocol, including the estimation of extent, and consequent allocation to the correct endoscopic surveillance interval. (6) For acceptance of artificial intelligence for automated lesion detection in small-bowel capsule endoscopy (SBCE), the performance of AI-assisted reading should be comparable to that of experienced endoscopists for lesion detection, without increasing but possibly reducing the reading time of the operator. (7) For acceptance of AI in the detection of colorectal polyps, the AI-assisted adenoma detection rate should be comparable to that of experienced endoscopists. (8) For acceptance of AI optical diagnosis (computer-aided diagnosis [CADx]) of diminutive polyps (≤5 mm), AI-assisted characterization should match performance standards for implementing resect-and-discard and diagnose-and-leave strategies. (9) For acceptance of AI in the management of polyps ≥ 6 mm, AI-assisted characterization should be comparable to that of experienced endoscopists in selecting lesions amenable to endoscopic resection.

Endocytoscopic Observation of Esophageal Lesions: Our Own Experience and a Review of the Literature

This review outlines the process of the development of the endocytoscope (EC) with reference to previously reported studies including our own. The EC is an ultra-high-magnification endoscope capable of imaging at the cellular level. The esophagus is the most suitable site for EC observation because it is amenable to vital staining. The diagnosis of esophageal lesions using EC is based on nuclear density and nuclear abnormality, allowing biopsy histology to be omitted. The observation of nuclear abnormality requires a magnification of ×600 or higher using digital technology. Several staining methods have been proposed, but single staining with toluidine blue or methylene blue is most suitable because the contrast at the border of a cancerous area can be easily identified. A three-tier classification of esophageal lesions visualized by EC is proposed: Type 1 (non-cancerous), Type 2 (endocytoscopic borderline), and Type 3 (cancerous). Since characteristic EC images reflecting pathology can be obtained from non-cancerous esophageal lesions, a modified form of classification with four additional characteristic non-cancerous EC features has also been proposed. Recently, deep-learning AI for analysis of esophageal EC images has revealed that its diagnostic accuracy is comparable to that of expert pathologists.

AI analysis and modified type classification for endocytoscopic observation of esophageal lesions

Endocytoscopy (EC) facilitates real-time histological diagnosis of esophageal lesions in vivo. We developed a deep-learning artificial intelligence (AI) system for analysis of EC images and compared its diagnostic ability with that of an expert pathologist and nonexpert endoscopists. Our new AI was based on a vision transformer model (DeiT) and trained using 7983 EC images of the esophagus (2368 malignant and 5615 nonmalignant). The AI evaluated 114 randomly arranged EC pictures (33 ESCC and 81 nonmalignant lesions) from 38 consecutive cases. An expert pathologist and two nonexpert endoscopists also analyzed the same image set according to the modified type classification (adding four EC features of nonmalignant lesions to our previous classification). The area under the curve calculated from the receiver-operating characteristic curve for the AI analysis was 0.92. In per-image analysis, the overall accuracy of the AI, pathologist, and two endoscopists was 91.2%, 91.2%, 85.9%, and 83.3%, respectively. The kappa value between the pathologist and the AI, and between the two endoscopists and the AI showed moderate concordance; that between the pathologist and the two endoscopists showed poor concordance. In per-patient analysis, the overall accuracy of the AI, pathologist, and two endoscopists was 94.7%, 92.1%, 86.8%, and 89.5%, respectively. The modified type classification aided high overall diagnostic accuracy by the pathologist and nonexpert endoscopists. The diagnostic ability of the AI was equal or superior to that of the experienced pathologist. AI is expected to support endoscopists in diagnosing esophageal lesions based on EC images.

Artificial intelligence in gastroenterology and hepatology: how to advance clinical practice while ensuring health equity

Artificial intelligence (AI) and machine learning (ML) systems are increasingly used in medicine to improve clinical decision-making and healthcare delivery. In gastroenterology and hepatology, studies have explored a myriad of opportunities for AI/ML applications which are already making the transition to bedside. Despite these advances, there is a risk that biases and health inequities can be introduced or exacerbated by these technologies. If unrecognised, these technologies could generate or worsen systematic racial, ethnic and sex disparities when deployed on a large scale. There are several mechanisms through which AI/ML could contribute to health inequities in gastroenterology and hepatology, including diagnosis of oesophageal cancer, management of inflammatory bowel disease (IBD), liver transplantation, colorectal cancer screening and many others. This review adapts a framework for ethical AI/ML development and application to gastroenterology and hepatology such that clinical practice is advanced while minimising bias and optimising health equity.

A prediction model for pathological findings after neoadjuvant chemoradiotherapy for resectable locally advanced esophageal squamous cell carcinoma based on endoscopic images using deep learning

OBJECTIVES

To propose deep-learning (DL)-based predictive model for pathological complete response rate for resectable locally advanced esophageal squamous cell carcinoma (SCC) after neoadjuvant chemoradiotherapy (NCRT) with endoscopic images.

METHODS AND MATERIAL

This retrospective study analyzed 98 patients with locally advanced esophagus cancer treated by preoperative chemoradiotherapy followed by surgery from 2004 to 2016. The patient data were split into two sets: 72 patients for the training of models and 26 patients for testing of the model. Patients was classified into two groups with the LC (Group I: responder and Group II: non-responder). The scanned images were converted into joint photographic experts group (JPEG) format and resized to 150 × 150 pixels. The input image without imaging filter (w/o filter) and with Laplacian, Sobel, and wavelet imaging filters deep-learning model to predict the pathological CR with a convolution neural network (CNN). The accuracy, sensitivity, and specificity, the area under the curve (AUC) of the receiver operating characteristic were evaluated.

RESULTS

The average of accuracy for the cross-validation was 0.64 for w/o filter, 0.69 for Laplacian filter, 0.71 for Sobel filter, and 0.81 for wavelet filter, respectively. The average of sensitivity for the cross-validation was 0.80 for w/o filter, 0.81 for Laplacian filter, 0.67 for Sobel filter, and 0.80 for wavelet filter, respectively. The average of specificity for the cross-validation was 0.37 for w/o filter, 0.55 for Laplacian filter, 0.68 for Sobel filter, and 0.81 for wavelet filter, respectively. From the ROC curve, the average AUC for the cross-validation was 0.58 for w/o filter, 0.67 for Laplacian filter, 0.73 for Sobel filter, and 0.83 for wavelet filter, respectively.

CONCLUSIONS

The current study proposed the improvement the accuracy of the DL-based prediction model with the imaging filters. With the imaging filters, the accuracy was significantly improved. The model can be supported to assist clinical oncologists to have a more accurate expectations of the treatment outcome.

ADVANCES IN KNOWLEDGE

The accuracy of the prediction for the local control after radiotherapy can improve with the input image with the imaging filter for deep learning.

Advances and horizons for artificial intelligence of endoscopic screening and surveillance of gastric and esophageal disease

The development of artificial intelligence in endoscopic assessment of the gastrointestinal tract has shown progressive enhancement in diagnostic acuity. This review discusses the expanding applications for gastric and esophageal diseases. The gastric section covers the utility of AI in detecting and characterizing gastric polyps and further explores prevention, detection, and classification of gastric cancer. The esophageal discussion highlights applications for use in screening and surveillance in Barrett's esophagus and in high-risk conditions for esophageal squamous cell carcinoma. Additionally, these discussions highlight applications for use in assessing eosinophilic esophagitis and future potential in assessing esophageal microbiome changes.

Development of a Deep Learning System to Detect Esophageal Cancer by Barium Esophagram

Background

Implementation of deep learning systems (DLSs) for analysis of barium esophagram, a cost-effective diagnostic test for esophageal cancer detection, is expected to reduce the burden to radiologists while ensuring the accuracy of diagnosis.

Objective

To develop an automated DLS to detect esophageal cancer on barium esophagram.

Methods

This was a retrospective study using deep learning for esophageal cancer detection. A two-stage DLS (including a Selection network and a Classification network) was developed. Five datasets based on barium esophagram were used for stepwise training, validation, and testing of the DLS. Datasets 1 and 2 were used to respectively train and test the Selection network, while Datasets 3, 4, and 5 were respectively used to train, validate, and test the Classification network. Finally, a positioning box with a probability value was outputted by the DLS. A region of interest delineated by experienced radiologists was selected as the ground truth to evaluate the detection and classification efficiency of the DLS. Standard machine learning metrics (accuracy, recall, precision, sensitivity, and specificity) were calculated. A comparison with the conventional visual inspection approach was also conducted.

Results

The accuracy, sensitivity, and specificity of our DLS in detecting esophageal cancer were 90.3%, 92.5%, and 88.7%, respectively. With the aid of DLS, the radiologists’ interpretation time was significantly shortened (Reader1, 45.7 s vs. 72.2 s without DLS aid; Reader2, 54.1 s vs. 108.7 s without DLS aid). Respective diagnostic efficiencies for Reader1 with and without DLS aid were 96.8% vs. 89.3% for accuracy, 97.5% vs. 87.5% for sensitivity, 96.2% vs. 90.6% for specificity, and 0.969 vs. 0.890 for AUC. Respective diagnostic efficiencies for Reader2 with and without DLS aid were 95.7% vs. 88.2% for accuracy, 92.5% vs. 77.5% for sensitivity, 98.1% vs. 96.2% for specificity, and 0.953 vs. 0.869 for AUC. Of note, the positioning boxes outputted by the DLS almost overlapped with those manually labeled by the radiologists on Dataset 5.

Conclusions

The proposed two-stage DLS for detecting esophageal cancer on barium esophagram could effectively shorten the interpretation time with an excellent diagnostic performance. It may well assist radiologists in clinical practice to reduce their burden.

Scoping out the future: The application of artificial intelligence to gastrointestinal endoscopy

Artificial intelligence (AI) is a quickly expanding field in gastrointestinal endoscopy. Although there are a myriad of applications of AI ranging from identification of bleeding to predicting outcomes in patients with inflammatory bowel disease, a great deal of research has focused on the identification and classification of gastrointestinal malignancies. Several of the initial randomized, prospective trials utilizing AI in clinical medicine have centered on polyp detection during screening colonoscopy. In addition to work focused on colorectal cancer, AI systems have also been applied to gastric, esophageal, pancreatic, and liver cancers. Despite promising results in initial studies, the generalizability of most of these AI systems have not yet been evaluated. In this article we review recent developments in the field of AI applied to gastrointestinal oncology.

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Application of Convolutional Neural Networks for Diagnosis of Eosinophilic Esophagitis Based on Endoscopic Imaging

Subjective symptoms associated with eosinophilic esophagitis (EoE), such as dysphagia, are not specific, thus the endoscopic identification of suggestive EoE findings is quite important for facilitating endoscopic biopsy sampling. However, poor inter-observer agreement among endoscopists regarding diagnosis has become a complicated issue, especially with inexperienced practitioners. Therefore, we constructed a computer-assisted diagnosis (CAD) system using a convolutional neural network (CNN) and evaluated its performance as a diagnostic utility. A CNN-based CAD system was developed based on ResNet50 architecture. The CNN was trained using a total of 1192 characteristic endoscopic images of 108 patients histologically proven to be in an active phase of EoE (≥15 eosinophils per high power field) as well as 1192 normal esophagus images. To evaluate diagnostic accuracy, an independent test set of 756 endoscopic images from 35 patients with EoE and 96 subjects with a normal esophagus was examined with the constructed CNN. The CNN correctly diagnosed EoE in 94.7% using a diagnosis per image analysis, with an overall sensitivity of 90.8% and specificity of 96.6%. For each case, the CNN correctly diagnosed 37 of 39 EoE cases with overall sensitivity and specificity of 94.9% and 99.0%, respectively. These findings indicate the usefulness of CNN for diagnosing EoE, especially for aiding inexperienced endoscopists during medical check-up screening.

Utility of an artificial intelligence system for classification of esophageal lesions when simulating its clinical use

Previous reports have shown favorable performance of artificial intelligence (AI) systems for diagnosing esophageal squamous cell carcinoma (ESCC) compared with endoscopists. However, these findings don’t reflect performance in clinical situations, as endoscopists classify lesions based on both magnified and non-magnified videos, while AI systems often use only a few magnified narrow band imaging (NBI) still images. We evaluated the performance of the AI system in simulated clinical situations. We used 25,048 images from 1433 superficial ESCC and 4746 images from 410 noncancerous esophagi to construct our AI system. For the validation dataset, we took NBI videos of suspected superficial ESCCs. The AI system diagnosis used one magnified still image taken from each video, while 19 endoscopists used whole videos. We used 147 videos and still images including 83 superficial ESCC and 64 non-ESCC lesions. The accuracy, sensitivity and specificity for the classification of ESCC were, respectively, 80.9% [95% CI 73.6–87.0], 85.5% [76.1–92.3], and 75.0% [62.6–85.0] for the AI system and 69.2% [66.4–72.1], 67.5% [61.4–73.6], and 71.5% [61.9–81.0] for the endoscopists. The AI system correctly classified all ESCCs invading the muscularis mucosa or submucosa and 96.8% of lesions ≥ 20 mm, whereas even the experts diagnosed some of them as non-ESCCs. Our AI system showed higher accuracy for classifying ESCC and non-ESCC than endoscopists. It may provide valuable diagnostic support to endoscopists.

Endoscopists' diagnostic accuracy in detecting upper gastrointestinal neoplasia in the framework of artificial intelligence studies

BACKGROUND

Estimates on miss rates for upper gastrointestinal neoplasia (UGIN) rely on registry data or old studies. Quality assurance programs for upper GI endoscopy are not fully established owing to the lack of infrastructure to measure endoscopists' competence. We aimed to assess endoscopists' accuracy for the recognition of UGIN exploiting the framework of artificial intelligence (AI) validation studies.

METHODS

Literature searches of databases (PubMed/MEDLINE, EMBASE, Scopus) up to August 2020 were performed to identify articles evaluating the accuracy of individual endoscopists for the recognition of UGIN within studies validating AI against a histologically verified expert-annotated ground-truth. The main outcomes were endoscopists' pooled sensitivity, specificity, positive and negative predictive value (PPV/NPV), and area under the curve (AUC) for all UGIN, for esophageal squamous cell neoplasia (ESCN), Barrett esophagus-related neoplasia (BERN), and gastric adenocarcinoma (GAC).

RESULTS

Seven studies (2 ESCN, 3 BERN, 1 GAC, 1 UGIN overall) with 122 endoscopists were included. The pooled endoscopists' sensitivity and specificity for UGIN were 82 % (95 % confidence interval [CI] 80 %-84 %) and 79 % (95 %CI 76 %-81 %), respectively. Endoscopists' accuracy was higher for GAC detection (AUC 0.95 [95 %CI 0.93-0.98]) than for ESCN (AUC 0.90 [95 %CI 0.88-0.92]) and BERN detection (AUC 0.86 [95 %CI 0.84-0.88]). Sensitivity was higher for Eastern vs. Western endoscopists (87 % [95 %CI 84 %-89 %] vs. 75 % [95 %CI 72 %-78 %]), and for expert vs. non-expert endoscopists (85 % [95 %CI 83 %-87 %] vs. 71 % [95 %CI 67 %-75 %]).

CONCLUSION

We show suboptimal accuracy of endoscopists for the recognition of UGIN even within a framework that included a higher prevalence and disease awareness. Future AI validation studies represent a framework to assess endoscopist competence.

Implementation of artificial intelligence in upper gastrointestinal endoscopy

The application of artificial intelligence (AI) using deep learning has significantly expanded in the field of esophagogastric endoscopy. Recent studies have shown promising results in detecting and differentiating early gastric cancer using AI tools built using white light, magnified, or image-enhanced endoscopic images. Some studies have reported the use of AI tools to predict the depth of early gastric cancer based on endoscopic images. Similarly, studies based on using AI for detecting early esophageal cancer have also been reported, with an accuracy comparable to that of endoscopy specialists. Moreover, an AI system, developed to diagnose pharyngeal cancer, has shown promising performance with high sensitivity. These reports suggest that, if introduced for regular use in clinical settings, AI systems can significantly reduce the burden on physicians. This review summarizes the current status of AI applications in the upper gastrointestinal tract and presents directions for clinical practice implementation and future research.

Artificial Intelligence and Deep Learning for Upper Gastrointestinal Neoplasia

Upper gastrointestinal (GI) neoplasia account for 35% of GI cancers and 1.5 million cancer-related deaths every year. Despite its efficacy in preventing cancer mortality, diagnostic upper GI endoscopy is affected by a substantial miss rate of neoplastic lesions due to failure to recognize a visible lesion or imperfect navigation. This may be offset by the real-time application of artificial intelligence (AI) for detection (computer-aided detection [CADe]) and characterization (computer-aided diagnosis [CADx]) of upper GI neoplasia. Stand-alone performance of CADe for esophageal squamous cell neoplasia, Barrett's esophagus-related neoplasia, and gastric cancer showed promising accuracy, sensitivity ranging between 83% and 93%. However, incorporation of CADe/CADx in clinical practice depends on several factors, such as possible bias in the training or validation phases of these algorithms, its interaction with human endoscopists, and clinical implications of false-positive results. The aim of this review is to guide the clinician across the multiple steps of AI development in clinical practice.

Systematic review with meta-analysis: artificial intelligence in the diagnosis of oesophageal diseases

BACKGROUND

Artificial intelligence (AI) has recently been applied to endoscopy and questionnaires for the evaluation of oesophageal diseases (ODs).

AIM

We performed a systematic review with meta-analysis to evaluate the performance of AI in the diagnosis of malignant and benign OD.

METHODS

We searched MEDLINE, EMBASE, EMBASE Classic and the Cochrane Library. A bivariate random-effect model was used to calculate pooled diagnostic efficacy of AI models and endoscopists. The reference tests were histology for neoplasms and the clinical and instrumental diagnosis for gastro-oesophageal reflux disease (GERD). The pooled area under the summary receiver operating characteristic (AUROC), sensitivity, specificity, positive and negative likelihood ratio (PLR and NLR) and diagnostic odds ratio (DOR) were estimated.

RESULTS

For the diagnosis of Barrett's neoplasia, AI had AUROC of 0.90, sensitivity 0.89, specificity 0.86, PLR 6.50, NLR 0.13 and DOR 50.53. AI models' performance was comparable with that of endoscopists (P = 0.35). For the diagnosis of oesophageal squamous cell carcinoma, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.97, 0.95, 0.92, 12.65, 0.05 and DOR 258.36, respectively. In this task, AI performed better than endoscopists although without statistically significant differences. In the detection of abnormal intrapapillary capillary loops, the performance of AI was: AUROC 0.98, sensitivity 0.94, specificity 0.94, PLR 14.75, NLR 0.07 and DOR 225.83. For the diagnosis of GERD based on questionnaires, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.99, 0.97, 0.97, 38.26, 0.03 and 1159.6, respectively.

CONCLUSIONS

AI demonstrated high performance in the clinical and endoscopic diagnosis of OD.

Application of convolutional neural networks for evaluating the depth of invasion of early gastric cancer based on endoscopic images

BACKGROUND AND AIM

Recently, artificial intelligence (AI) has been used in endoscopic examination and is expected to help in endoscopic diagnosis. We evaluated the feasibility of AI using convolutional neural network (CNN) systems for evaluating the depth of invasion of early gastric cancer (EGC), based on endoscopic images.

METHODS

This study used a deep CNN model, ResNet152. From patients who underwent treatment for EGC at our hospital between January 2012 and December 2016, we selected 100 consecutive patients with mucosal (M) cancers and 100 consecutive patients with cancers invading the submucosa (SM cancers). A total of 3508 non-magnifying endoscopic images of EGCs, including white-light imaging, linked color imaging, blue laser imaging-bright, and indigo-carmine dye contrast imaging, were included in this study. A total of 2288 images from 132 patients served as the development dataset, and 1220 images from 68 patients served as the testing dataset. Invasion depth was evaluated for each image and lesion. The majority vote was applied to lesion-based evaluation.

RESULTS

The sensitivity, specificity, and accuracy for diagnosing M cancer were 84.9% (95% confidence interval [CI] 82.3%-87.5%), 70.7% (95% CI 66.8%-74.6%), and 78.9% (95% CI 76.6%-81.2%), respectively, for image-based evaluation, and 85.3% (95% CI 73.4%-97.2%), 82.4% (95% CI 69.5%-95.2%), and 83.8% (95% CI 75.1%-92.6%), respectively, for lesion-based evaluation.

CONCLUSIONS

The application of AI using CNN to evaluate the depth of invasion of EGCs based on endoscopic images is feasible, and it is worth investing more effort to put this new technology into practical use.

Artificial Intelligence for Detecting and Delineating Margins of Early ESCC Under WLI Endoscopy

INTRODUCTION

Conventional white light imaging (WLI) endoscopy is the most common screening technique used for detecting early esophageal squamous cell carcinoma (ESCC). Nevertheless, it is difficult to detect and delineate margins of early ESCC using WLI endoscopy. This study aimed to develop an artificial intelligence (AI) model to detect and delineate margins of early ESCC under WLI endoscopy.

METHODS

A total of 13,083 WLI images from 1,239 patients were used to train and test the AI model. To evaluate the detection performance of the model, 1,479 images and 563 images were used as internal and external validation data sets, respectively. For assessing the delineation performance of the model, 1,114 images and 211 images were used as internal and external validation data sets, respectively. In addition, 216 images were used to compare the delineation performance between the model and endoscopists.

RESULTS

The model showed an accuracy of 85.7% and 84.5% in detecting lesions in internal and external validation, respectively. For delineating margins, the model achieved an accuracy of 93.4% and 95.7% in the internal and external validation, respectively, under an overlap ratio of 0.60. The accuracy of the model, senior endoscopists, and expert endoscopists in delineating margins were 98.1%, 78.6%, and 95.3%, respectively. The proposed model achieved similar delineating performance compared with that of expert endoscopists but superior to senior endoscopists.

DISCUSSION

We successfully developed an AI model, which can be used to accurately detect early ESCC and delineate the margins of the lesions under WLI endoscopy.

Artificial intelligence for detecting superficial esophageal squamous cell carcinoma under multiple endoscopic imaging modalities: A multicenter study

BACKGROUND AND AIM

Diagnosis of esophageal squamous cell carcinoma (ESCC) is complicated and requires substantial expertise and experience. This study aimed to develop an artificial intelligence (AI) system for detecting superficial ESCC under multiple endoscopic imaging modalities.

METHODS

Endoscopic images were retrospectively collected from West China Hospital, Sichuan University as a training dataset and an independent internal validation dataset. Images from other four hospitals were used as an external validation dataset. The AI system was compared with 11 experienced endoscopists. Furthermore, videos were collected to assess the performance of the AI system.

RESULTS

A total of 53 933 images from 2621 patients and 142 videos from 19 patients were used to develop and validate the AI system. In the internal and external validation datasets, the performance of the AI system under all or different endoscopic imaging modalities was satisfactory, with sensitivity of 92.5-99.7%, specificity of 78.5-89.0%, and area under the receiver operating characteristic curves of 0.906-0.989. The AI system achieved comparable performance with experienced endoscopists. Regarding superficial ESCC confined to the epithelium, the AI system was more sensitive than experienced endoscopists on white-light imaging (90.8% vs 82.5%, P = 0.022). Moreover, the AI system exhibited good performance in videos, with sensitivity of 89.5-100% and specificity of 73.7-89.5%.

CONCLUSIONS

We developed an AI system that showed comparable performance with experienced endoscopists in detecting superficial ESCC under multiple endoscopic imaging modalities and might provide valuable support for inexperienced endoscopists, despite requiring further evaluation.

Artificial Intelligence in the Diagnosis of Upper Gastrointestinal Diseases

Artificial intelligence (AI) has enormous potential to support clinical routine workflows and therefore is gaining increasing popularity among medical professionals. In the field of gastroenterology, investigations on AI and computer-aided diagnosis (CAD) systems have mainly focused on the lower gastrointestinal (GI) tract. However, numerous CAD tools have been tested also in upper GI disorders showing encouraging results. The main application of AI in the upper GI tract is endoscopy; however, the need to analyze increasing loads of numerical and categorical data in short times has pushed researchers to investigate applications of AI systems in other upper GI settings, including gastroesophageal reflux disease, eosinophilic esophagitis, and motility disorders. AI and CAD systems will be increasingly incorporated into daily clinical practice in the coming years, thus at least basic notions will be soon required among physicians. For noninsiders, the working principles and potential of AI may be as fascinating as obscure. Accordingly, we reviewed systematic reviews, meta-analyses, randomized controlled trials, and original research articles regarding the performance of AI in the diagnosis of both malignant and benign esophageal and gastric diseases, also discussing essential characteristics of AI.

Application of artificial intelligence-assisted endoscopic detection of early esophageal cancer

In recent years, artificial intelligence (AI) combined with endoscopy has made an appearance in the diagnosis of early esophageal cancer (EC) and achieved satisfactory results. Due to the rapid progression and poor prognosis of EC, the early detection and diagnosis of EC are of great value for patient prognosis improvement. AI has been applied in the screening of early EC and has shown advantages; notably, it is more accurate than less-experienced endoscopists. In China, the detection of early EC depends on endoscopist expertise and is inevitably subject to interobserver variability. The excellent imaging recognition ability of AI is very suitable for the diagnosis and recognition of EC, thereby reducing the missed diagnosis and helping physicians to perform endoscopy better. This paper reviews the application and relevant progress of AI in the field of endoscopic detection of early EC (including squamous cell carcinoma and adenocarcinoma), with a focus on diagnostic performance of AI to identify different types of endoscopic images, such as sensitivity and specificity.

Artificial Intelligence in the Analysis of Upper Gastrointestinal Disorders

In the past, conventional machine learning was applied to analyze tabulated medical data while deep learning was applied to study afflictions such as gastrointestinal disorders. Neural networks were used to detect, classify, and delineate various images of lesions because the local feature selection and optimization of the deep learning model enabled accurate image analysis. With the accumulation of medical records, the evolution of computational power and graphics processing units, and the widespread use of open-source libraries in large-scale machine learning processes, medical artificial intelligence (AI) is overcoming its limitations. While early studies prioritized the automatic diagnosis of cancer or pre-cancerous lesions, the current expanded scope of AI includes benign lesions, quality control, and machine learning analysis of big data. However, the limited commercialization of medical AI and the need to justify its application in each field of research are restricting factors. Modeling assumes that observations follow certain statistical rules, and external validation checks whether assumption is correct or generalizable. Therefore, unused data are essential in the training or internal testing process to validate the performance of the established AI models. This article summarizes the studies on the application of AI models in upper gastrointestinal disorders. The current limitations and the perspectives on future development have also been discussed.

Detection of multiple lesions of gastrointestinal tract for endoscopy using artificial intelligence model: a pilot study

BACKGROUND

This study was aimed to develop a computer-aided diagnosis (CAD) system with deep-learning technique and to validate its efficiency on detecting the four categories of lesions such as polyps, advanced cancer, erosion/ulcer and varices at endoscopy.

METHODS

A deep convolutional neural network (CNN) that consists of more than 50 layers were trained with a big dataset containing 327,121 white light images (WLI) of endoscopy from 117,005 cases collected from 2012 to 2017. Two CAD models were developed using images with or without annotation of the training dataset. The efficiency of the CAD system detecting the four categories of lesions was validated by another dataset containing consecutive cases from 2018 to 2019.

RESULTS

A total of 1734 cases with 33,959 images were included in the validation datasets which containing lesions of polyps 1265, advanced cancer 500, erosion/ulcer 486, and varices 248. The CAD system developed in this study may detect polyps, advanced cancer, erosion/ulcer and varices as abnormality with the sensitivity of 88.3% and specificity of 90.3%, respectively, in 0.05 s. The training datasets with annotation may enhance either sensitivity or specificity about 20%, p = 0.000. The sensitivities and specificities for polyps, advanced cancer, erosion/ulcer and varices reached about 90%, respectively. The detect efficiency for the four categories of lesions reached to 89.7%.

CONCLUSION

The CAD model for detection of multiple lesions in gastrointestinal lumen would be potentially developed into a double check along with real-time assessment and interpretation of the findings encountered by the endoscopists and may be a benefit to reduce the events of missing lesions.

Scope of Artificial Intelligence in Gastrointestinal Oncology

Gastrointestinal cancers are among the leading causes of death worldwide, with over 2.8 million deaths annually. Over the last few decades, advancements in artificial intelligence technologies have led to their application in medicine. The use of artificial intelligence in endoscopic procedures is a significant breakthrough in modern medicine. Currently, the diagnosis of various gastrointestinal cancer relies on the manual interpretation of radiographic images by radiologists and various endoscopic images by endoscopists. This can lead to diagnostic variabilities as it requires concentration and clinical experience in the field. Artificial intelligence using machine or deep learning algorithms can provide automatic and accurate image analysis and thus assist in diagnosis. In the field of gastroenterology, the application of artificial intelligence can be vast from diagnosis, predicting tumor histology, polyp characterization, metastatic potential, prognosis, and treatment response. It can also provide accurate prediction models to determine the need for intervention with computer-aided diagnosis. The number of research studies on artificial intelligence in gastrointestinal cancer has been increasing rapidly over the last decade due to immense interest in the field. This review aims to review the impact, limitations, and future potentials of artificial intelligence in screening, diagnosis, tumor staging, treatment modalities, and prediction models for the prognosis of various gastrointestinal cancers.

Usefulness of an artificial intelligence system for the detection of esophageal squamous cell carcinoma evaluated with videos simulating overlooking situation

OBJECTIVES

Artificial intelligence (AI) systems have shown favorable performance in the detection of esophageal squamous cell carcinoma (ESCC). However, previous studies were limited by the quality of their validation methods. In this study, we evaluated the performance of an AI system with videos simulating situations in which ESCC has been overlooked.

METHODS

We used 17,336 images from 1376 superficial ESCCs and 1461 images from 196 noncancerous and normal esophagi to construct the AI system. To record validation videos, the endoscope was passed through the esophagus at a constant speed without focusing on the lesion to simulate situations in which ESCC has been missed. Validation videos were evaluated by the AI system and 21 endoscopists.

RESULTS

We prepared 100 video datasets, including 50 superficial ESCCs, 22 noncancerous lesions, and 28 normal esophagi. The AI system had sensitivity of 85.7% (54 of 63 ESCCs) and specificity of 40%. Initial evaluation by endoscopists conducted with plain video (without AI support) had average sensitivity of 75.0% (47.3 of 63 ESCC) and specificity of 91.4%. Subsequent evaluation by endoscopists was conducted with AI assistance, which improved their sensitivity to 77.7% (P = 0.00696) without changing their specificity (91.6%, P = 0.756).

CONCLUSIONS

Our AI system had high sensitivity for the detection of ESCC. As a support tool, the system has the potential to enhance detection of ESCC without reducing specificity. (UMIN000039645).

Real-time artificial intelligence for endoscopic diagnosis of early esophageal squamous cell cancer (with video)

BACKGROUND AND AIMS

Endoscopic diagnosis of early esophageal squamous cell cancer (ESCC) is complicated and dependent on operators' experience. This study aimed to develop an artificial intelligence (AI) model for automatic diagnosis of early ESCC.

METHODS

Non-magnifying and magnifying endoscopic images of normal/noncancerous lesions, early ESCC, and advanced esophageal cancer (AEC) were retrospectively obtained from Qilu Hospital of Shandong University. A total of 10,988 images from 5075 cases were chosen for training and validation. Another 2309 images from 1055 cases were collected for testing. One hundred and four real-time videos were also collected to evaluate the diagnostic performance of the AI model. The diagnostic performance of the AI model was compared with endoscopists by magnifying images and the assistant efficiency of the AI model for novices was evaluated.

RESULTS

The AI diagnosis for non-magnifying images showed a per-patient accuracy, sensitivity, and specificity of 99.5%, 100%, 99.5% for white light imaging, and 97.0%, 97.2%, 96.4% for optical enhancement/iodine straining images. Regarding diagnosis for magnifying images, the per-patient accuracy, sensitivity, and specificity were 88.1%, 90.9%, and 85.0%. The diagnostic accuracy of the AI model was similar to experts (84.5%, P = 0.205) and superior to novices (68.5%, P = 0.005). The diagnostic performance of novices was significantly improved by AI assistance. When it comes to the diagnosis for real-time videos, the AI model showed acceptable performance as well.

CONCLUSIONS

The AI model could accurately recognize early ESCC among noncancerous mucosa and AEC. It could be a potential assistant for endoscopists, especially for novices.

Artificial intelligence and early esophageal cancer

The development of esophageal cancer (EC) from early to advanced stage results in a high mortality rate and poor prognosis. Advanced EC not only poses a serious threat to the life and health of patients but also places a heavy economic burden on their families and society. Endoscopy is of great value for the diagnosis of EC, especially in the screening of Barrett’s esophagus and early EC. However, at present, endoscopy has a low diagnostic rate for early tumors. In recent years, artificial intelligence (AI) has made remarkable progress in the diagnosis of digestive system tumors, providing a new model for clinicians to diagnose and treat these tumors. In this review, we aim to provide a comprehensive overview of how AI can help doctors diagnose early EC and precancerous lesions and make clinical decisions based on the predicted results. We analyze and summarize the recent research on AI and early EC. We find that based on deep learning (DL) and convolutional neural network methods, the current computer-aided diagnosis system has gradually developed from in vitro image analysis to real-time detection and diagnosis. Based on powerful computing and DL capabilities, the diagnostic accuracy of AI is close to or better than that of endoscopy specialists. We also analyze the shortcomings in the current AI research and corresponding improvement strategies. We believe that the application of AI-assisted endoscopy in the diagnosis of early EC and precancerous lesions will become possible after the further advancement of AI-related research.