Early esophageal adenocarcinoma detection using deep learning methods. Int J Comput Assist Radiol Surg. 2019;14:611-621. [PMID: 30666547 PMCID: PMC6420905 DOI: 10.1007/s11548-019-01914-4]

DeepCSFusion: Deep Compressive Sensing Fusion for Efficient COVID-19 Classification

Worldwide, the COVID-19 epidemic, which started in 2019, has resulted in millions of deaths. The medical research community has widely used computer analysis of medical data during the pandemic, specifically deep learning models. Deploying models on devices with constrained resources is a significant challenge due to the increased storage demands associated with larger deep learning models. Accordingly, in this paper, we propose a novel compression strategy that compresses deep features with a compression ratio of 10 to 90% to accurately classify the COVID-19 and non-COVID-19 computed tomography scans. Additionally, we extensively validated the compression using various available deep learning methods to extract the most suitable features from different models. Finally, the suggested DeepCSFusion model compresses the extracted features and applies fusion to achieve the highest classification accuracy with fewer features. The proposed DeepCSFusion model was validated on the publicly available dataset "SARS-CoV-2 CT" scans composed of 1252 CT. This study demonstrates that the proposed DeepCSFusion reduced the computational time with an overall accuracy of 99.3%. Also, it outperforms state-of-the-art pipelines in terms of various classification measures.

Lesion-Decoupling-Based Segmentation With Large-Scale Colon and Esophageal Datasets for Early Cancer Diagnosis

Lesions of early cancers often show flat, small, and isochromatic characteristics in medical endoscopy images, which are difficult to be captured. By analyzing the differences between the internal and external features of the lesion area, we propose a lesion-decoupling-based segmentation (LDS) network for assisting early cancer diagnosis. We introduce a plug-and-play module called self-sampling similar feature disentangling module (FDM) to obtain accurate lesion boundaries. Then, we propose a feature separation loss (FSL) function to separate pathological features from normal ones. Moreover, since physicians make diagnoses with multimodal data, we propose a multimodal cooperative segmentation network with two different modal images as input: white-light images (WLIs) and narrowband images (NBIs). Our FDM and FSL show a good performance for both single-modal and multimodal segmentations. Extensive experiments on five backbones prove that our FDM and FSL can be easily applied to different backbones for a significant lesion segmentation accuracy improvement, and the maximum increase of mean Intersection over Union (mIoU) is 4.58. For colonoscopy, we can achieve up to mIoU of 91.49 on our Dataset A and 84.41 on the three public datasets. For esophagoscopy, mIoU of 64.32 is best achieved on the WLI dataset and 66.31 on the NBI dataset.

Endoscopic Artificial Intelligence for Image Analysis in Gastrointestinal Neoplasms

BACKGROUND

Artificial intelligence (AI) using deep learning systems has recently been utilized in various medical fields. In the field of gastroenterology, AI is primarily implemented in image recognition and utilized in the realm of gastrointestinal (GI) endoscopy. In GI endoscopy, computer-aided detection/diagnosis (CAD) systems assist endoscopists in GI neoplasm detection or differentiation of cancerous or noncancerous lesions. Several AI systems for colorectal polyps have already been applied in colonoscopy clinical practices. In esophagogastroduodenoscopy, a few CAD systems for upper GI neoplasms have been launched in Asian countries. The usefulness of these CAD systems in GI endoscopy has been gradually elucidated.

SUMMARY

In this review, we outline recent articles on several studies of endoscopic AI systems for GI neoplasms, focusing on esophageal squamous cell carcinoma (ESCC), esophageal adenocarcinoma (EAC), gastric cancer (GC), and colorectal polyps. In ESCC and EAC, computer-aided detection (CADe) systems were mainly developed, and a recent meta-analysis study showed sensitivities of 91.2% and 93.1% and specificities of 80% and 86.9%, respectively. In GC, a recent meta-analysis study on CADe systems demonstrated that their sensitivity and specificity were as high as 90%. A randomized controlled trial (RCT) also showed that the use of the CADe system reduced the miss rate. Regarding computer-aided diagnosis (CADx) systems for GC, although RCTs have not yet been conducted, most studies have demonstrated expert-level performance. In colorectal polyps, multiple RCTs have shown the usefulness of the CADe system for improving the polyp detection rate, and several CADx systems have been shown to have high accuracy in colorectal polyp differentiation.

KEY MESSAGES

Most analyses of endoscopic AI systems suggested that their performance was better than that of nonexpert endoscopists and equivalent to that of expert endoscopists. Thus, endoscopic AI systems may be useful for reducing the risk of overlooking lesions and improving the diagnostic ability of endoscopists.

Advancement of artificial intelligence systems for surveillance endoscopy of Barrett's esophagus

Barrett's esophagus (BE) is a precursor disease for esophageal adenocarcinoma. Timely detection and treatment has significant influence on patient outcomes. Over the last years, several artificial intelligence (AI) systems have emerged to assist the endoscopist. The primary focus of research has been computer aided detection (CADe). Several groups have succeeded in developing competitive models for neoplasia detection. Additionally, computer aided diagnosis (CADx) models have been developed for subsequent lesion characterization and assistance in clinical decision making. Future studies should focus on bridging the domain gap between academic development and integration in daily practice.

HRU-Net: A high-resolution convolutional neural network for esophageal cancer radiotherapy target segmentation

BACKGROUND AND OBJECTIVE

The effective segmentation of esophageal squamous carcinoma lesions in CT scans is significant for auxiliary diagnosis and treatment. However, accurate lesion segmentation is still a challenging task due to the irregular form of the esophagus and small size, the inconsistency of spatio-temporal structure, and low contrast of esophagus and its peripheral tissues in medical images. The objective of this study is to improve the segmentation effect of esophageal squamous cell carcinoma lesions.

METHODS

It is critical for a segmentation network to effectively extract 3D discriminative features to distinguish esophageal cancers from some visually closed adjacent esophageal tissues and organs. In this work, an efficient HRU-Net architecture (High-Resolution U-Net) was exploited for esophageal cancer and esophageal carcinoma segmentation in CT slices. Based on the idea of localization first and segmentation later, the HRU-Net locates the esophageal region before segmentation. In addition, an Resolution Fusion Module (RFM) was designed to integrate the information of adjacent resolution feature maps to obtain strong semantic information, as well as preserve the high-resolution features.

RESULTS

Compared with the other five typical methods, the devised HRU-Net is capable of generating superior segmentation results.

CONCLUSIONS

Our proposed HRU-NET improves the accuracy of segmentation for squamous esophageal cancer. Compared to other models, our model performs the best. The designed method may improve the efficiency of clinical diagnosis of esophageal squamous cell carcinoma lesions.

Hospital volume-mortality association after esophagectomy for cancer: a systematic review and meta-analysis

BACKGROUND

Postoperative mortality plays an important role in evaluating the surgical safety of esophagectomy. Although postoperative mortality after esophagectomy is partly influenced by the yearly hospital surgical case volume (hospital volume), this association remains unclear.

METHODS

Studies assessing the association between hospital volume and postoperative mortality in patients who underwent esophagectomy for esophageal cancer were searched for eligibility. Odds ratios were pooled for the highest versus lowest categories of hospital volume using a random effects model. The dose-response association between hospital volume and the risk of postoperative mortality was analyzed. The study protocol was registered with PROSPERO.

RESULTS

Fifty-six studies including 385 469 participants were included. A higher-volume hospital significantly reduced the risk of postesophagectomy mortality by 53% compared with their lower-volume counterparts (odds ratio, 0.47; 95% CI: 0.42-0.53). Similar results were found in subgroup analyses. Volume-outcome analysis suggested that postesophagectomy mortality rates remained roughly stable after the hospital volume reached a plateau of 45 esophagectomies per year.

CONCLUSIONS

Higher-volume hospitals had significantly lower postesophagectomy mortality rates in patients with esophageal cancer, with a threshold of 45 esophagectomies per year for a high-volume hospital. This remarkable negative correlation showed the benefit of a better safety in centralization of esophagectomy to a high-volume hospital.

Semi-automatic standardized analysis method to objectively evaluate near-infrared fluorescent dyes in image-guided surgery

Significance

Near-infrared fluorescence imaging still lacks a standardized, objective method to evaluate fluorescent dye efficacy in oncological surgical applications. This results in difficulties in translation between preclinical to clinical studies with fluorescent dyes and in the reproduction of results between studies, which in turn hampers further clinical translation of novel fluorescent dyes.

Aim

Our aim is to develop and evaluate a semi-automatic standardized method to objectively assess fluorescent signals in resected tissue.

Approach

A standardized imaging procedure was designed and quantitative analysis methods were developed to evaluate non-targeted and tumor-targeted fluorescent dyes. The developed analysis methods included manual selection of region of interest (ROI) on white light images, automated fluorescence signal ROI selection, and automatic quantitative image analysis. The proposed analysis method was then compared with a conventional analysis method, where fluorescence signal ROIs were manually selected on fluorescence images. Dice similarity coefficients and intraclass correlation coefficients were calculated to determine the inter- and intraobserver variabilities of the ROI selections and the determined signal- and tumor-to-background ratios.

Results

The proposed non-targeted fluorescent dyes analysis method showed statistically significantly improved variabilities after application on indocyanine green specimens. For specimens with the targeted dye SGM-101, the variability of the background ROI selection was statistically significantly improved by implementing the proposed method.

Conclusion

Semi-automatic methods for standardized quantitative analysis of fluorescence images were successfully developed and showed promising results to further improve the reproducibility and standardization of clinical studies evaluating fluorescent dyes.

A deep learning system for detection of early Barrett's neoplasia: a model development and validation study

BACKGROUND

Computer-aided detection (CADe) systems could assist endoscopists in detecting early neoplasia in Barrett's oesophagus, which could be difficult to detect in endoscopic images. The aim of this study was to develop, test, and benchmark a CADe system for early neoplasia in Barrett's oesophagus.

METHODS

The CADe system was first pretrained with ImageNet followed by domain-specific pretraining with GastroNet. We trained the CADe system on a dataset of 14 046 images (2506 patients) of confirmed Barrett's oesophagus neoplasia and non-dysplastic Barrett's oesophagus from 15 centres. Neoplasia was delineated by 14 Barrett's oesophagus experts for all datasets. We tested the performance of the CADe system on two independent test sets. The all-comers test set comprised 327 (73 patients) non-dysplastic Barrett's oesophagus images, 82 (46 patients) neoplastic images, 180 (66 of the same patients) non-dysplastic Barrett's oesophagus videos, and 71 (45 of the same patients) neoplastic videos. The benchmarking test set comprised 100 (50 patients) neoplastic images, 300 (125 patients) non-dysplastic images, 47 (47 of the same patients) neoplastic videos, and 141 (82 of the same patients) non-dysplastic videos, and was enriched with subtle neoplasia cases. The benchmarking test set was evaluated by 112 endoscopists from six countries (first without CADe and, after 6 weeks, with CADe) and by 28 external international Barrett's oesophagus experts. The primary outcome was the sensitivity of Barrett's neoplasia detection by general endoscopists without CADe assistance versus with CADe assistance on the benchmarking test set. We compared sensitivity using a mixed-effects logistic regression model with conditional odds ratios (ORs; likelihood profile 95% CIs).

FINDINGS

Sensitivity for neoplasia detection among endoscopists increased from 74% to 88% with CADe assistance (OR 2·04; 95% CI 1·73-2·42; p<0·0001 for images and from 67% to 79% [2·35; 1·90-2·94; p<0·0001] for video) without compromising specificity (from 89% to 90% [1·07; 0·96-1·19; p=0·20] for images and from 96% to 94% [0·94; 0·79-1·11; ] for video; p=0·46). In the all-comers test set, CADe detected neoplastic lesions in 95% (88-98) of images and 97% (90-99) of videos. In the benchmarking test set, the CADe system was superior to endoscopists in detecting neoplasia (90% vs 74% [OR 3·75; 95% CI 1·93-8·05; p=0·0002] for images and 91% vs 67% [11·68; 3·85-47·53; p<0·0001] for video) and non-inferior to Barrett's oesophagus experts (90% vs 87% [OR 1·74; 95% CI 0·83-3·65] for images and 91% vs 86% [2·94; 0·99-11·40] for video).

INTERPRETATION

CADe outperformed endoscopists in detecting Barrett's oesophagus neoplasia and, when used as an assistive tool, it improved their detection rate. CADe detected virtually all neoplasia in a test set of consecutive cases.

FUNDING

Olympus.

The role of artificial intelligence in the endoscopic diagnosis of esophageal cancer: a systematic review and meta-analysis

Early detection of esophageal cancer is limited by accurate endoscopic diagnosis of subtle macroscopic lesions. Endoscopic interpretation is subject to expertise, diagnostic skill, and thus human error. Artificial intelligence (AI) in endoscopy is increasingly bridging this gap. This systematic review and meta-analysis consolidate the evidence on the use of AI in the endoscopic diagnosis of esophageal cancer. The systematic review was carried out using Pubmed, MEDLINE and Ovid EMBASE databases and articles on the role of AI in the endoscopic diagnosis of esophageal cancer management were included. A meta-analysis was also performed. Fourteen studies (1590 patients) assessed the use of AI in endoscopic diagnosis of esophageal squamous cell carcinoma-the pooled sensitivity and specificity were 91.2% (84.3-95.2%) and 80% (64.3-89.9%). Nine studies (478 patients) assessed AI capabilities of diagnosing esophageal adenocarcinoma with the pooled sensitivity and specificity of 93.1% (86.8-96.4) and 86.9% (81.7-90.7). The remaining studies formed the qualitative summary. AI technology, as an adjunct to endoscopy, can assist in accurate, early detection of esophageal malignancy. It has shown superior results to endoscopists alone in identifying early cancer and assessing depth of tumor invasion, with the added benefit of not requiring a specialized skill set. Despite promising results, the application in real-time endoscopy is limited, and further multicenter trials are required to accurately assess its use in routine practice.

Application of convolutional neural network-based endoscopic imaging in esophageal cancer or high-grade dysplasia: A systematic review and meta-analysis

BACKGROUND

Esophageal cancer is the seventh-most common cancer type worldwide, accounting for 5% of death from malignancy. Development of novel diagnostic techniques has facilitated screening, early detection, and improved prognosis. Convolutional neural network (CNN)-based image analysis promises great potential for diagnosing and determining the prognosis of esophageal cancer, enabling even early detection of dysplasia.

AIM

To conduct a meta-analysis of the diagnostic accuracy of CNN models for the diagnosis of esophageal cancer and high-grade dysplasia (HGD).

METHODS

PubMed, EMBASE, Web of Science and Cochrane Library databases were searched for articles published up to November 30, 2022. We evaluated the diagnostic accuracy of using the CNN model with still image-based analysis and with video-based analysis for esophageal cancer or HGD, as well as for the invasion depth of esophageal cancer. The pooled sensitivity, pooled specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and area under the curve (AUC) were estimated, together with the 95% confidence intervals (CI). A bivariate method and hierarchical summary receiver operating characteristic method were used to calculate the diagnostic test accuracy of the CNN model. Meta-regression and subgroup analyses were used to identify sources of heterogeneity.

RESULTS

A total of 28 studies were included in this systematic review and meta-analysis. Using still image-based analysis for the diagnosis of esophageal cancer or HGD provided a pooled sensitivity of 0.95 (95%CI: 0.92-0.97), pooled specificity of 0.92 (0.89-0.94), PLR of 11.5 (8.3-16.0), NLR of 0.06 (0.04-0.09), DOR of 205 (115-365), and AUC of 0.98 (0.96-0.99). When video-based analysis was used, a pooled sensitivity of 0.85 (0.77-0.91), pooled specificity of 0.73 (0.59-0.83), PLR of 3.1 (1.9-5.0), NLR of 0.20 (0.12-0.34), DOR of 15 (6-38) and AUC of 0.87 (0.84-0.90) were found. Prediction of invasion depth resulted in a pooled sensitivity of 0.90 (0.87-0.92), pooled specificity of 0.83 (95%CI: 0.76-0.88), PLR of 7.8 (1.9-32.0), NLR of 0.10 (0.41-0.25), DOR of 118 (11-1305), and AUC of 0.95 (0.92-0.96).

CONCLUSION

CNN-based image analysis in diagnosing esophageal cancer and HGD is an excellent diagnostic method with high sensitivity and specificity that merits further investigation in large, multicenter clinical trials.

Deep Learning in Barrett's Esophagus Diagnosis: Current Status and Future Directions

Barrett's esophagus (BE) represents a pre-malignant condition characterized by abnormal cellular proliferation in the distal esophagus. A timely and accurate diagnosis of BE is imperative to prevent its progression to esophageal adenocarcinoma, a malignancy associated with a significantly reduced survival rate. In this digital age, deep learning (DL) has emerged as a powerful tool for medical image analysis and diagnostic applications, showcasing vast potential across various medical disciplines. In this comprehensive review, we meticulously assess 33 primary studies employing varied DL techniques, predominantly featuring convolutional neural networks (CNNs), for the diagnosis and understanding of BE. Our primary focus revolves around evaluating the current applications of DL in BE diagnosis, encompassing tasks such as image segmentation and classification, as well as their potential impact and implications in real-world clinical settings. While the applications of DL in BE diagnosis exhibit promising results, they are not without challenges, such as dataset issues and the "black box" nature of models. We discuss these challenges in the concluding section. Essentially, while DL holds tremendous potential to revolutionize BE diagnosis, addressing these challenges is paramount to harnessing its full capacity and ensuring its widespread application in clinical practice.

Role of artificial intelligence in Barrett’s esophagus

The application of artificial intelligence (AI) in gastrointestinal endoscopy has gained significant traction over the last decade. One of the more recent applications of AI in this field includes the detection of dysplasia and cancer in Barrett’s esophagus (BE). AI using deep learning methods has shown promise as an adjunct to the endoscopist in detecting dysplasia and cancer. Apart from visual detection and diagnosis, AI may also aid in reducing the considerable interobserver variability in identifying and distinguishing dysplasia on whole slide images from digitized BE histology slides. This review aims to provide a comprehensive summary of the key studies thus far as well as providing an insight into the future role of AI in Barrett’s esophagus.

Machine learning applications for early detection of esophageal cancer: a systematic review

INTRODUCTION

Esophageal cancer (EC) is a significant global health problem, with an estimated 7th highest incidence and 6th highest mortality rate. Timely diagnosis and treatment are critical for improving patients' outcomes, as over 40% of patients with EC are diagnosed after metastasis. Recent advances in machine learning (ML) techniques, particularly in computer vision, have demonstrated promising applications in medical image processing, assisting clinicians in making more accurate and faster diagnostic decisions. Given the significance of early detection of EC, this systematic review aims to summarize and discuss the current state of research on ML-based methods for the early detection of EC.

METHODS

We conducted a comprehensive systematic search of five databases (PubMed, Scopus, Web of Science, Wiley, and IEEE) using search terms such as "ML", "Deep Learning (DL (", "Neural Networks (NN)", "Esophagus", "EC" and "Early Detection". After applying inclusion and exclusion criteria, 31 articles were retained for full review.

RESULTS

The results of this review highlight the potential of ML-based methods in the early detection of EC. The average accuracy of the reviewed methods in the analysis of endoscopic and computed tomography (CT (images of the esophagus was over 89%, indicating a high impact on early detection of EC. Additionally, the highest percentage of clinical images used in the early detection of EC with the use of ML was related to white light imaging (WLI) images. Among all ML techniques, methods based on convolutional neural networks (CNN) achieved higher accuracy and sensitivity in the early detection of EC compared to other methods.

CONCLUSION

Our findings suggest that ML methods may improve accuracy in the early detection of EC, potentially supporting radiologists, endoscopists, and pathologists in diagnosis and treatment planning. However, the current literature is limited, and more studies are needed to investigate the clinical applications of these methods in early detection of EC. Furthermore, many studies suffer from class imbalance and biases, highlighting the need for validation of detection algorithms across organizations in longitudinal studies.

ClusterNet: a clustering distributed prior embedded detection network for early-stage esophageal squamous cell carcinoma diagnosis

BACKGROUND

Early and accurate diagnosis of esophageal squamous cell carcinoma (ESCC) is important for reducing mortality. Analyzing intrapapillary capillary loops' (IPCLs) patterns on magnification endoscopy with narrow band imaging (ME-NBI) has been demonstrated effective in the diagnosis of early-stage ESCC. However, even experienced endoscopists may face difficulty in finding and classifying countless IPCLs on ME-NBI.

PURPOSE

We propose a novel clustering prior embedded detection network: ClusterNet. ClusterNet is capable of analyzing the distribution of IPCLs on ME-NBI automatically and enables endoscopists to overview multiple types of visualization. With ClusterNet assisting, endoscopists may observe ME-NBI images more efficiently, thus they may also predict the pathology and make medical decisions more easily.

METHODS

We propose the first large-scale ME-NBI dataset with fine-grained annotations by consensus of expert endoscopists. The dataset is splitted into a training set and an independent testing set based on patients. With two strategies for embedding, ClusterNet can automatically take the clustering effect into consideration. Prior to this work, none of the existing approaches take the clustering effect, which is rather important in classifying the IPCLs, into account.

RESULTS

ClusterNet achieves an average precision of 81.2% and an average recall of 90.0% for the detection of IPCLs patterns on each patient of the independent testing set. We also compare ClusterNet with other state-of-the-art detection approaches. The performance of ClusterNet with embedding strategies is consistently superior to that of other approaches in terms of average precision, recall and F2-Score.

CONCLUSIONS

Experiments demonstrate that our proposed method is able to detect almost all the IPCLs patterns on ME-NBI and classify them according to the Japanese Endoscopic Society (JES) classification accurately.

Role of artificial intelligence in diagnosing Barrett's esophagus-related neoplasia

Barrett's esophagus is associated with an increased risk of adenocarcinoma. Thorough screening during endoscopic surveillance is crucial to improve patient prognosis. Detecting and characterizing dysplastic or neoplastic Barrett's esophagus during routine endoscopy are challenging, even for expert endoscopists. Artificial intelligence-based clinical decision support systems have been developed to provide additional assistance to physicians performing diagnostic and therapeutic gastrointestinal endoscopy. In this article, we review the current role of artificial intelligence in the management of Barrett's esophagus and elaborate on potential artificial intelligence in the future.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

Pattern Classification Using Quantized Neural Networks for FPGA-Based Low-Power IoT Devices

With the recent growth of the Internet of Things (IoT) and the demand for faster computation, quantized neural networks (QNNs) or QNN-enabled IoT can offer better performance than conventional convolution neural networks (CNNs). With the aim of reducing memory access costs and increasing the computation efficiency, QNN-enabled devices are expected to transform numerous industrial applications with lower processing latency and power consumption. Another form of QNN is the binarized neural network (BNN), which has 2 bits of quantized levels. In this paper, CNN-, QNN-, and BNN-based pattern recognition techniques are implemented and analyzed on an FPGA. The FPGA hardware acts as an IoT device due to connectivity with the cloud, and QNN and BNN are considered to offer better performance in terms of low power and low resource use on hardware platforms. The CNN and QNN implementation and their comparative analysis are analyzed based on their accuracy, weight bit error, RoC curve, and execution speed. The paper also discusses various approaches that can be deployed for optimizing various CNN and QNN models with additionally available tools. The work is performed on the Xilinx Zynq 7020 series Pynq Z2 board, which serves as our FPGA-based low-power IoT device. The MNIST and CIFAR-10 databases are considered for simulation and experimentation. The work shows that the accuracy is 95.5% and 79.22% for the MNIST and CIFAR-10 databases, respectively, for full precision (32-bit), and the execution time is 5.8 ms and 18 ms for the MNIST and CIFAR-10 databases, respectively, for full precision (32-bit).

Dexterous Identification of Carcinoma through ColoRectalCADx with Dichotomous Fusion CNN and UNet Semantic Segmentation

Human colorectal disorders in the digestive tract are recognized by reference colonoscopy. The current system recognizes cancer through a three-stage system that utilizes two sets of colonoscopy data. However, identifying polyps by visualization has not been addressed. The proposed system is a five-stage system called ColoRectalCADx, which provides three publicly accessible datasets as input data for cancer detection. The three main datasets are CVC Clinic DB, Kvasir2, and Hyper Kvasir. After the image preprocessing stages, system experiments were performed with the seven prominent convolutional neural networks (CNNs) (end-to-end) and nine fusion CNN models to extract the spatial features. Afterwards, the end-to-end CNN and fusion features are executed. These features are derived from Discrete Wavelet Transform (DWT) and Vector Support Machine (SVM) classification, that was used to retrieve time and spatial frequency features. Experimentally, the results were obtained for five stages. For each of the three datasets, from stage 1 to stage 3, end-to-end CNN, DenseNet-201 obtained the best testing accuracy (98%, 87%, 84%), ((98%, 97%), (87%, 87%), (84%, 84%)), ((99.03%, 99%), (88.45%, 88%), (83.61%, 84%)). For each of the three datasets, from stage 2, CNN DaRD-22 fusion obtained the optimal test accuracy ((93%, 97%) (82%, 84%), (69%, 57%)). And for stage 4, ADaRDEV2-22 fusion achieved the best test accuracy ((95.73%, 94%), (81.20%, 81%), (72.56%, 58%)). For the input image segmentation datasets CVC Clinc-Seg, KvasirSeg, and Hyper Kvasir, malignant polyps were identified with the UNet CNN model. Here, the loss score datasets (CVC clinic DB was 0.7842, Kvasir2 was 0.6977, and Hyper Kvasir was 0.6910) were obtained.

AI-based analysis of oral lesions using novel deep convolutional neural networks for early detection of oral cancer

Artificial intelligence (AI) applications in oncology have been developed rapidly with reported successes in recent years. This work aims to evaluate the performance of deep convolutional neural network (CNN) algorithms for the classification and detection of oral potentially malignant disorders (OPMDs) and oral squamous cell carcinoma (OSCC) in oral photographic images. A dataset comprising 980 oral photographic images was divided into 365 images of OSCC, 315 images of OPMDs and 300 images of non-pathological images. Multiclass image classification models were created by using DenseNet-169, ResNet-101, SqueezeNet and Swin-S. Multiclass object detection models were fabricated by using faster R-CNN, YOLOv5, RetinaNet and CenterNet2. The AUC of multiclass image classification of the best CNN models, DenseNet-196, was 1.00 and 0.98 on OSCC and OPMDs, respectively. The AUC of the best multiclass CNN-base object detection models, Faster R-CNN, was 0.88 and 0.64 on OSCC and OPMDs, respectively. In comparison, DenseNet-196 yielded the best multiclass image classification performance with AUC of 1.00 and 0.98 on OSCC and OPMD, respectively. These values were inline with the performance of experts and superior to those of general practictioners (GPs). In conclusion, CNN-based models have potential for the identification of OSCC and OPMDs in oral photographic images and are expected to be a diagnostic tool to assist GPs for the early detection of oral cancer.

A prediction model for pathological findings after neoadjuvant chemoradiotherapy for resectable locally advanced esophageal squamous cell carcinoma based on endoscopic images using deep learning

OBJECTIVES

To propose deep-learning (DL)-based predictive model for pathological complete response rate for resectable locally advanced esophageal squamous cell carcinoma (SCC) after neoadjuvant chemoradiotherapy (NCRT) with endoscopic images.

METHODS AND MATERIAL

This retrospective study analyzed 98 patients with locally advanced esophagus cancer treated by preoperative chemoradiotherapy followed by surgery from 2004 to 2016. The patient data were split into two sets: 72 patients for the training of models and 26 patients for testing of the model. Patients was classified into two groups with the LC (Group I: responder and Group II: non-responder). The scanned images were converted into joint photographic experts group (JPEG) format and resized to 150 × 150 pixels. The input image without imaging filter (w/o filter) and with Laplacian, Sobel, and wavelet imaging filters deep-learning model to predict the pathological CR with a convolution neural network (CNN). The accuracy, sensitivity, and specificity, the area under the curve (AUC) of the receiver operating characteristic were evaluated.

RESULTS

The average of accuracy for the cross-validation was 0.64 for w/o filter, 0.69 for Laplacian filter, 0.71 for Sobel filter, and 0.81 for wavelet filter, respectively. The average of sensitivity for the cross-validation was 0.80 for w/o filter, 0.81 for Laplacian filter, 0.67 for Sobel filter, and 0.80 for wavelet filter, respectively. The average of specificity for the cross-validation was 0.37 for w/o filter, 0.55 for Laplacian filter, 0.68 for Sobel filter, and 0.81 for wavelet filter, respectively. From the ROC curve, the average AUC for the cross-validation was 0.58 for w/o filter, 0.67 for Laplacian filter, 0.73 for Sobel filter, and 0.83 for wavelet filter, respectively.

CONCLUSIONS

The current study proposed the improvement the accuracy of the DL-based prediction model with the imaging filters. With the imaging filters, the accuracy was significantly improved. The model can be supported to assist clinical oncologists to have a more accurate expectations of the treatment outcome.

ADVANCES IN KNOWLEDGE

The accuracy of the prediction for the local control after radiotherapy can improve with the input image with the imaging filter for deep learning.

Artificial Intelligence-Assisted Endoscopic Diagnosis of Early Upper Gastrointestinal Cancer: A Systematic Review and Meta-Analysis

Objective

The aim of this study was to assess the diagnostic ability of artificial intelligence (AI) in the detection of early upper gastrointestinal cancer (EUGIC) using endoscopic images.

Methods

Databases were searched for studies on AI-assisted diagnosis of EUGIC using endoscopic images. The pooled area under the curve (AUC), sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with 95% confidence interval (CI) were calculated.

Results

Overall, 34 studies were included in our final analysis. Among the 17 image-based studies investigating early esophageal cancer (EEC) detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.98, 0.95 (95% CI, 0.95–0.96), 0.95 (95% CI, 0.94–0.95), 10.76 (95% CI, 7.33–15.79), 0.07 (95% CI, 0.04–0.11), and 173.93 (95% CI, 81.79–369.83), respectively. Among the seven patient-based studies investigating EEC detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.98, 0.94 (95% CI, 0.91–0.96), 0.90 (95% CI, 0.88–0.92), 6.14 (95% CI, 2.06–18.30), 0.07 (95% CI, 0.04–0.11), and 69.13 (95% CI, 14.73–324.45), respectively. Among the 15 image-based studies investigating early gastric cancer (EGC) detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.94, 0.87 (95% CI, 0.87–0.88), 0.88 (95% CI, 0.87–0.88), 7.20 (95% CI, 4.32–12.00), 0.14 (95% CI, 0.09–0.23), and 48.77 (95% CI, 24.98–95.19), respectively.

Conclusions

On the basis of our meta-analysis, AI exhibited high accuracy in diagnosis of EUGIC.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO/, identifier PROSPERO (CRD42021270443).

Predictive effect of J waves on cardiac compression and clinical prognosis of esophageal tumors: a retrospective study

Background

The J wave syndromes (JWS) could be observed in patients with mediastinal tumors, though few studies have verified the statistical correlation between J waves and cardiac compression by tumors. This study aimed to investigate the relationship between J waves and cardiac compression by esophageal tumor and to compare the prediction of J waves on clinical prognosis with that of cardiac compression by esophageal tumor.

Methods

We enrolled 273 patients (228 males, 45 females; mean 63.8±7.5 years) with esophageal tumors admitted to Shanghai Chest Hospital between August 2016 and November 2020. The J wave was defined as a J-point elevation of ≥0.1 mV in a 12-lead electrocardiogram (ECG) and classified into multiple types. Chest computed tomography (CT) was reviewed to clarify the anatomical relationship between the heart and the esophageal tumor. The prognosis of severe cardiac events and survival status were followed up through medical history, examination records and telephone records.

Results

J waves were present in 141 patients among all 273 cases. The sensitivity and specificity of cardiac compression by the tumor for J waves were 78.1% and 67.3%, respectively. The odds ratio (OR) of cardiac compression by the tumor to J waves was 7.33 [95% confidence interval (CI): 4.21-12.74; P<0.001]. The Kappa coefficient between J waves and cardiac compression was 0.44±0.05. The significance association between J waves and cardiac compression was independent from other clinical variables (P<0.001). Decreased J wave amplitude was correlated with the disappearance of cardiac compression during follow-up (P=0.03). Patients with J waves had a higher risk of severe cardiac events than those without J waves (OR =2.84, 95% CI: 1.22-6.63; P=0.01). During the follow-up period, we found that the presence of J waves [hazard ratio (HR) =2.28; 95% CI: 1.35-3.84; P=0.002] and cardiac compression by the tumor (HR =2.51; 95% CI: 1.51-4.17; P<0.001) were both negatively correlated with the survival time of patients.

Conclusions

The presence of J waves could be used as an effective mean to predict the mechanical impact of esophageal tumor on the heart, and played an important role in predicting the survival of patients.

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Artificial Intelligence in the Management of Barrett’s Esophagus and Early Esophageal Adenocarcinoma

Simple Summary

Esophageal adenocarcinoma is increasing in incidence and is the most common subtype of esophageal cancer in Western societies. AI systems are currently under development and validation in many fields of gas-troenterology.

Abstract

Esophageal adenocarcinoma is increasing in incidence and is the most common subtype of esophageal cancer in Western societies. The stepwise progression of Barrett´s metaplasia to high-grade dysplasia and invasive adenocarcinoma provides an opportunity for screening and surveillance. There are important unresolved issues, which include (i) refining the definition of the screening population in order to avoid unnecessary invasive diagnostics, (ii) a more precise prediction of the (very heterogeneous) individual progression risk from metaplasia to invasive cancer in order to better tailor surveillance recommendations, (iii) improvement of the quality of endoscopy in order to reduce the high miss rate for early neoplastic lesions, and (iv) support for the diagnosis of tumor infiltration depth in order to guide treatment decisions. Artificial intelligence (AI) systems might be useful as a support to better solve the above-mentioned issues.

Artificial Intelligence and Deep Learning for Upper Gastrointestinal Neoplasia

Upper gastrointestinal (GI) neoplasia account for 35% of GI cancers and 1.5 million cancer-related deaths every year. Despite its efficacy in preventing cancer mortality, diagnostic upper GI endoscopy is affected by a substantial miss rate of neoplastic lesions due to failure to recognize a visible lesion or imperfect navigation. This may be offset by the real-time application of artificial intelligence (AI) for detection (computer-aided detection [CADe]) and characterization (computer-aided diagnosis [CADx]) of upper GI neoplasia. Stand-alone performance of CADe for esophageal squamous cell neoplasia, Barrett's esophagus-related neoplasia, and gastric cancer showed promising accuracy, sensitivity ranging between 83% and 93%. However, incorporation of CADe/CADx in clinical practice depends on several factors, such as possible bias in the training or validation phases of these algorithms, its interaction with human endoscopists, and clinical implications of false-positive results. The aim of this review is to guide the clinician across the multiple steps of AI development in clinical practice.

Advanced imaging and artificial intelligence for Barrett's esophagus: What we should and soon will do

Barrett’s esophagus (BE) is a well-established risk factor for esophageal adenocarcinoma. It is recommended that patients have regular endoscopic surveillance, with the ultimate goal of detecting early-stage neoplastic lesions before they can progress to invasive carcinoma. Detection of both dysplasia or early adenocarcinoma permits curative endoscopic treatments, and with this aim, thorough endoscopic assessment is crucial and improves outcomes. The burden of missed neoplasia in BE is still far from being negligible, likely due to inappropriate endoscopic surveillance. Over the last two decades, advanced imaging techniques, moving from traditional dye-spray chromoendoscopy to more practical virtual chromoendoscopy technologies, have been introduced with the aim to enhance neoplasia detection in BE. As witnessed in other fields, artificial intelligence (AI) has revolutionized the field of diagnostic endoscopy and is set to cover a pivotal role in BE as well. The aim of this commentary is to comprehensively summarize present evidence, recent research advances, and future perspectives regarding advanced imaging technology and AI in BE; the combination of computer-aided diagnosis to a widespread adoption of advanced imaging technologies is eagerly awaited. It will also provide a useful step-by-step approach for performing high-quality endoscopy in BE, in order to increase the diagnostic yield of endoscopy in clinical practice.

Systematic review with meta-analysis: artificial intelligence in the diagnosis of oesophageal diseases

BACKGROUND

Artificial intelligence (AI) has recently been applied to endoscopy and questionnaires for the evaluation of oesophageal diseases (ODs).

AIM

We performed a systematic review with meta-analysis to evaluate the performance of AI in the diagnosis of malignant and benign OD.

METHODS

We searched MEDLINE, EMBASE, EMBASE Classic and the Cochrane Library. A bivariate random-effect model was used to calculate pooled diagnostic efficacy of AI models and endoscopists. The reference tests were histology for neoplasms and the clinical and instrumental diagnosis for gastro-oesophageal reflux disease (GERD). The pooled area under the summary receiver operating characteristic (AUROC), sensitivity, specificity, positive and negative likelihood ratio (PLR and NLR) and diagnostic odds ratio (DOR) were estimated.

RESULTS

For the diagnosis of Barrett's neoplasia, AI had AUROC of 0.90, sensitivity 0.89, specificity 0.86, PLR 6.50, NLR 0.13 and DOR 50.53. AI models' performance was comparable with that of endoscopists (P = 0.35). For the diagnosis of oesophageal squamous cell carcinoma, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.97, 0.95, 0.92, 12.65, 0.05 and DOR 258.36, respectively. In this task, AI performed better than endoscopists although without statistically significant differences. In the detection of abnormal intrapapillary capillary loops, the performance of AI was: AUROC 0.98, sensitivity 0.94, specificity 0.94, PLR 14.75, NLR 0.07 and DOR 225.83. For the diagnosis of GERD based on questionnaires, the AUROC, sensitivity, specificity, PLR, NLR and DOR were 0.99, 0.97, 0.97, 38.26, 0.03 and 1159.6, respectively.

CONCLUSIONS

AI demonstrated high performance in the clinical and endoscopic diagnosis of OD.

Application of artificial intelligence-assisted endoscopic detection of early esophageal cancer

In recent years, artificial intelligence (AI) combined with endoscopy has made an appearance in the diagnosis of early esophageal cancer (EC) and achieved satisfactory results. Due to the rapid progression and poor prognosis of EC, the early detection and diagnosis of EC are of great value for patient prognosis improvement. AI has been applied in the screening of early EC and has shown advantages; notably, it is more accurate than less-experienced endoscopists. In China, the detection of early EC depends on endoscopist expertise and is inevitably subject to interobserver variability. The excellent imaging recognition ability of AI is very suitable for the diagnosis and recognition of EC, thereby reducing the missed diagnosis and helping physicians to perform endoscopy better. This paper reviews the application and relevant progress of AI in the field of endoscopic detection of early EC (including squamous cell carcinoma and adenocarcinoma), with a focus on diagnostic performance of AI to identify different types of endoscopic images, such as sensitivity and specificity.

Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network

We propose a novel shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection (ESD) surgery. This task is of great clinical significance but extremely challenging due to bleeding, lighting reflection, and motion blur in the complicated surgical environment. Compared with existing solutions, which either neglect geometric relationships among targeting objects or capture the relationships by using complicated aggregation schemes, the proposed network is capable of achieving satisfactory accuracy while maintaining real-time performance by taking full advantage of the spatial relations among landmarks. We first devise an algorithm to automatically generate relation keypoint heatmaps, which are able to intuitively represent the prior knowledge of spatial relations among landmarks without using any extra manual annotation efforts. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process. While one scheme introduces pixel-level regularization by multi-task learning, the other integrates global-level regularization by harnessing a newly designed grouped consistency evaluator, which adds relation constraints to the proposed network in an adversarial manner. Both schemes are beneficial to the model in training, and can be readily unloaded in inference to achieve real-time detection. We establish a large in-house dataset of ESD surgery for esophageal cancer to validate the effectiveness of our proposed method. Extensive experimental results demonstrate that our approach outperforms state-of-the-art methods in terms of accuracy and efficiency, achieving better detection results faster. Promising results on two downstream applications further corroborate the great potential of our method in ESD clinical practice.

Early gastrointestinal cancer: The application of artificial intelligence

Early gastrointestinal (GI) cancer has been the core of clinical endoscopic work. Its early detection and treatment are tightly associated with patients’ prognoses. As a novel technology, artificial intelligence has been improved and applied in the field of endoscopy. Studies on detection, diagnosis, risk, and prognosis evaluation of diseases in the GI tract have been in development, including precancerous lesions, adenoma, early GI cancers, and advanced GI cancers. In this review, research on esophagus, stomach, and colon was concluded, and associated with the process from precancerous lesions to early GI cancer, such as from Barrett’s esophagus to early esophageal cancer, from dysplasia to early gastric cancer, and from adenoma to early colonic cancer. A status quo of research on early GI cancers and artificial intelligence was provided.

Artificial intelligence in endoscopy: The challenges and future directions

Artificial intelligence based approaches, in particular deep learning, have achieved state-of-the-art performance in medical fields with increasing number of software systems being approved by both Europe and United States. This paper reviews their applications to early detection of oesophageal cancers with a focus on their advantages and pitfalls. The paper concludes with future recommendations towards the development of a real-time, clinical implementable, interpretable and robust diagnosis support systems.

Automatic classification and detection of oral cancer in photographic images using deep learning algorithms

BACKGROUND

Oral cancer is a deadly disease among the most common malignant tumors worldwide, and it has become an increasingly important public health problem in developing and low-to-middle income countries. This study aims to use the convolutional neural network (CNN) deep learning algorithms to develop an automated classification and detection model for oral cancer screening.

METHODS

The study included 700 clinical oral photographs, collected retrospectively from the oral and maxillofacial center, which were divided into 350 images of oral squamous cell carcinoma and 350 images of normal oral mucosa. The classification and detection models were created by using DenseNet121 and faster R-CNN, respectively. Four hundred and ninety images were randomly selected as training data. In addition, 70 and 140 images were assigned as validating and testing data, respectively.

RESULTS

The classification accuracy of DenseNet121 model achieved a precision of 99%, a recall of 100%, an F1 score of 99%, a sensitivity of 98.75%, a specificity of 100%, and an area under the receiver operating characteristic curve of 99%. The detection accuracy of a faster R-CNN model achieved a precision of 76.67%, a recall of 82.14%, an F1 score of 79.31%, and an area under the precision-recall curve of 0.79.

CONCLUSION

The DenseNet121 and faster R-CNN algorithm were proved to offer the acceptable potential for classification and detection of cancerous lesions in oral photographic images.

Automated detection of brain metastases on non-enhanced CT using single-shot detectors

PURPOSE

To develop and investigate deep learning-based detectors for brain metastases detection on non-enhanced (NE) CT.

METHODS

The study included 116 NECTs from 116 patients (81 men, age 66.5 ± 10.6 years) to train and test single-shot detector (SSD) models using 89 and 27 cases, respectively. The annotation was performed by three radiologists using bounding-boxes defined on contrast-enhanced CT (CECT) images. NECTs were coregistered and resliced to CECTs. The detection performance was evaluated at the SSD's 50% confidence threshold using sensitivity, positive-predictive value (PPV), and the false-positive rate per scan (FPR). For false negatives and true positives, binary logistic regression was used to examine the possible contributing factors.

RESULTS

For lesions 6 mm or larger, the SSD achieved a sensitivity of 35.4% (95% confidence interval (CI): [32.3%, 33.5%]); 51/144) with an FPR of 14.9 (95% CI [12.4, 13.9]). The overall sensitivity was 23.8% (95% CI: [21.3%, 22.8%]; 55/231) and PPV was 19.1% (95% CI: [18.5%, 20.4%]; 98/ of 513), with an FPR of 15.4 (95% CI [12.9, 14.5]). Ninety-five percent of the lesions that SSD failed to detect were also undetectable to radiologists (168/176). Twenty-four percent of the lesions (13/50) detected by the SSD were undetectable to radiologists. Logistic regression analysis indicated that density, necrosis, and size contributed to the lesions' visibility for radiologists, while for the SSD, the surrounding edema also enhanced the detection performance.

CONCLUSION

The SSD model we developed could detect brain metastases larger than 6 mm to some extent, a quarter of which were even retrospectively unrecognizable to radiologists.

[Deep Learning in Upper Gastrointestinal Disorders: Status and Future Perspectives]

Artificial intelligence using deep learning has been applied to gastrointestinal disorders for the detection, classification, and delineation of various lesion images. With the accumulation of enormous medical records, the evolution of computation power with graphic processing units, and the widespread use of open-source libraries in large-scale machine learning processes, medical artificial intelligence is overcoming its traditional limitations. This paper explains the basic concepts of deep learning model establishment and summarizes previous studies on upper gastrointestinal disorders. The limitations and perspectives on future development are also discussed.

Computer-aided diagnosis of esophageal cancer and neoplasms in endoscopic images: a systematic review and meta-analysis of diagnostic test accuracy

BACKGROUND AND AIMS

Diagnosis of esophageal cancer or precursor lesions by endoscopic imaging depends on endoscopist expertise and is inevitably subject to interobserver variability. Studies on computer-aided diagnosis (CAD) using deep learning or machine learning are on the increase. However, studies with small sample sizes are limited by inadequate statistical strength. Here, we used a meta-analysis to evaluate the diagnostic test accuracy (DTA) of CAD algorithms of esophageal cancers or neoplasms using endoscopic images.

METHODS

Core databases were searched for studies based on endoscopic imaging using CAD algorithms for the diagnosis of esophageal cancer or neoplasms and presenting data on diagnostic performance, and a systematic review and DTA meta-analysis were performed.

RESULTS

Overall, 21 and 19 studies were included in the systematic review and DTA meta-analysis, respectively. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD algorithms for the diagnosis of esophageal cancer or neoplasms in the image-based analysis were 0.97 (95% confidence interval [CI], 0.95-0.99), 0.94 (95% CI, 0.89-0.96), 0.88 (95% CI, 0.76-0.94), and 108 (95% CI, 43-273), respectively. Meta-regression showed no heterogeneity, and no publication bias was detected. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD algorithms for the diagnosis of esophageal cancer invasion depth were 0.96 (95% CI, 0.86-0.99), 0.90 (95% CI, 0.88-0.92), 0.88 (95% CI, 0.83-0.91), and 138 (95% CI, 12-1569), respectively.

CONCLUSIONS

CAD algorithms showed high accuracy for the automatic endoscopic diagnosis of esophageal cancer and neoplasms. The limitation of a lack in performance in external validation and clinical applications should be overcome.

The Role and Impact of Deep Learning Methods in Computer-Aided Diagnosis Using Gastrointestinal Endoscopy

At present, the application of artificial intelligence (AI) based on deep learning in the medical field has become more extensive and suitable for clinical practice compared with traditional machine learning. The application of traditional machine learning approaches to clinical practice is very challenging because medical data are usually uncharacteristic. However, deep learning methods with self-learning abilities can effectively make use of excellent computing abilities to learn intricate and abstract features. Thus, they are promising for the classification and detection of lesions through gastrointestinal endoscopy using a computer-aided diagnosis (CAD) system based on deep learning. This study aimed to address the research development of a CAD system based on deep learning in order to assist doctors in classifying and detecting lesions in the stomach, intestines, and esophagus. It also summarized the limitations of the current methods and finally presented a prospect for future research.

Artificial intelligence-assisted endoscopic detection of esophageal neoplasia in early stage: The next step?

Esophageal cancer (EC) is a common malignant tumor of the digestive tract and originates from the epithelium of the esophageal mucosa. It has been confirmed that early EC lesions can be cured by endoscopic therapy, and the curative effect is equivalent to that of surgical operation. Upper gastrointestinal endoscopy is still the gold standard for EC diagnosis. The accuracy of endoscopic examination results largely depends on the professional level of the examiner. Artificial intelligence (AI) has been applied in the screening of early EC and has shown advantages; notably, it is more accurate than less-experienced endoscopists. This paper reviews the application of AI in the field of endoscopic detection of early EC, including squamous cell carcinoma and adenocarcinoma, and describes the relevant progress. Although up to now most of the studies evaluating the clinical application of AI in early EC endoscopic detection are focused on still images, AI-assisted real-time detection based on live-stream video may be the next step.

GASTRO-CADx: a three stages framework for diagnosing gastrointestinal diseases

Gastrointestinal (GI) diseases are common illnesses that affect the GI tract. Diagnosing these GI diseases is quite expensive, complicated, and challenging. A computer-aided diagnosis (CADx) system based on deep learning (DL) techniques could considerably lower the examination cost processes and increase the speed and quality of diagnosis. Therefore, this article proposes a CADx system called Gastro-CADx to classify several GI diseases using DL techniques. Gastro-CADx involves three progressive stages. Initially, four different CNNs are used as feature extractors to extract spatial features. Most of the related work based on DL approaches extracted spatial features only. However, in the following phase of Gastro-CADx, features extracted in the first stage are applied to the discrete wavelet transform (DWT) and the discrete cosine transform (DCT). DCT and DWT are used to extract temporal-frequency and spatial-frequency features. Additionally, a feature reduction procedure is performed in this stage. Finally, in the third stage of the Gastro-CADx, several combinations of features are fused in a concatenated manner to inspect the effect of feature combination on the output results of the CADx and select the best-fused feature set. Two datasets referred to as Dataset I and II are utilized to evaluate the performance of Gastro-CADx. Results indicated that Gastro-CADx has achieved an accuracy of 97.3% and 99.7% for Dataset I and II respectively. The results were compared with recent related works. The comparison showed that the proposed approach is capable of classifying GI diseases with higher accuracy compared to other work. Thus, it can be used to reduce medical complications, death-rates, in addition to the cost of treatment. It can also help gastroenterologists in producing more accurate diagnosis while lowering inspection time.

Learning Spatiotemporal Features for Esophageal Abnormality Detection From Endoscopic Videos

Esophageal cancer is categorized as a type of disease with a high mortality rate. Early detection of esophageal abnormalities (i.e. precancerous and early cancerous) can improve the survival rate of the patients. Recent deep learning-based methods for selected types of esophageal abnormality detection from endoscopic images have been proposed. However, no methods have been introduced in the literature to cover the detection from endoscopic videos, detection from challenging frames and detection of more than one esophageal abnormality type. In this paper, we present an efficient method to automatically detect different types of esophageal abnormalities from endoscopic videos. We propose a novel 3D Sequential DenseConvLstm network that extracts spatiotemporal features from the input video. Our network incorporates 3D Convolutional Neural Network (3DCNN) and Convolutional Lstm (ConvLstm) to efficiently learn short and long term spatiotemporal features. The generated feature map is utilized by a region proposal network and ROI pooling layer to produce a bounding box that detects abnormality regions in each frame throughout the video. Finally, we investigate a post-processing method named Frame Search Conditional Random Field (FS-CRF) that improves the overall performance of the model by recovering the missing regions in neighborhood frames within the same clip. We extensively validate our model on an endoscopic video dataset that includes a variety of esophageal abnormalities. Our model achieved high performance using different evaluation metrics showing 93.7% recall, 92.7% precision, and 93.2% F-measure. Moreover, as no results have been reported in the literature for the esophageal abnormality detection from endoscopic videos, to validate the robustness of our model, we have tested the model on a publicly available colonoscopy video dataset, achieving the polyp detection performance in a recall of 81.18%, precision of 96.45% and F-measure 88.16%, compared to the state-of-the-art results of 78.84% recall, 90.51% precision and 84.27% F-measure using the same dataset. This demonstrates that the proposed method can be adapted to different gastrointestinal endoscopic video applications with a promising performance.

Artificial intelligence for the detection of esophageal and esophagogastric junctional adenocarcinoma

BACKGROUND AND AIM

Conventional endoscopy for the early detection of esophageal and esophagogastric junctional adenocarcinoma (E/J cancer) is limited because early lesions are asymptomatic, and the associated changes in the mucosa are subtle. There are no reports on artificial intelligence (AI) diagnosis for E/J cancer from Asian countries. Therefore, we aimed to develop a computerized image analysis system using deep learning for the detection of E/J cancers.

METHODS

A total of 1172 images from 166 pathologically proven superficial E/J cancer cases and 2271 images of normal mucosa in esophagogastric junctional from 219 cases were used as the training image data. A total of 232 images from 36 cancer cases and 43 non-cancerous cases were used as the validation test data. The same validation test data were diagnosed by 15 board-certified specialists (experts).

RESULTS

The sensitivity, specificity, and accuracy of the AI system were 94%, 42%, and 66%, respectively, and that of the experts were 88%, 43%, and 63%, respectively. The sensitivity of the AI system was favorable, while its specificity for non-cancerous lesions was similar to that of the experts. Interobserver agreement among the experts for detecting superficial E/J was fair (Fleiss' kappa = 0.26, z = 20.4, P < 0.001).

CONCLUSIONS

Our AI system achieved high sensitivity and acceptable specificity for the detection of E/J cancers and may be a good supporting tool for the screening of E/J cancers.

Artificial Intelligence and Its Role in Identifying Esophageal Neoplasia

Randomized trials have demonstrated that ablation of dysplastic Barrett's esophagus can reduce the risk of progression to cancer. Endoscopic resection for early stage esophageal adenocarcinoma and squamous cell carcinoma can significantly reduce postoperative morbidity compared to esophagectomy. Unfortunately, current endoscopic surveillance technologies (e.g., high-definition white light, electronic, and dye-based chromoendoscopy) lack sensitivity at identifying subtle areas of dysplasia and cancer. Random biopsies sample only approximately 5% of the esophageal mucosa at risk, and there is poor agreement among pathologists in identifying low-grade dysplasia. Machine-based deep learning medical image and video assessment technologies have progressed significantly in recent years, enabled in large part by advances in computer processing capabilities. In deep learning, sequential layers allow models to transform input data (e.g., pixels for imaging data) into a composite representation that allows for classification and feature identification. Several publications have attempted to use this technology to help identify dysplasia and early esophageal cancer. The aims of this reviews are as follows: (a) discussing limitations in our current strategies to identify esophageal dysplasia and cancer, (b) explaining the concepts behind deep learning and convolutional neural networks using language appropriate for clinicians without an engineering background, (c) systematically reviewing the literature for studies that have used deep learning to identify esophageal neoplasia, and (d) based on the systemic review, outlining strategies on further work necessary before these technologies are ready for "prime-time," i.e., use in routine clinical care.

Standalone performance of artificial intelligence for upper GI neoplasia: a meta-analysis

OBJECTIVE

Artificial intelligence (AI) may reduce underdiagnosed or overlooked upper GI (UGI) neoplastic and preneoplastic conditions, due to subtle appearance and low disease prevalence. Only disease-specific AI performances have been reported, generating uncertainty on its clinical value.

DESIGN

We searched PubMed, Embase and Scopus until July 2020, for studies on the diagnostic performance of AI in detection and characterisation of UGI lesions. Primary outcomes were pooled diagnostic accuracy, sensitivity and specificity of AI. Secondary outcomes were pooled positive (PPV) and negative (NPV) predictive values. We calculated pooled proportion rates (%), designed summary receiving operating characteristic curves with respective area under the curves (AUCs) and performed metaregression and sensitivity analysis.

RESULTS

Overall, 19 studies on detection of oesophageal squamous cell neoplasia (ESCN) or Barrett's esophagus-related neoplasia (BERN) or gastric adenocarcinoma (GCA) were included with 218, 445, 453 patients and 7976, 2340, 13 562 images, respectively. AI-sensitivity/specificity/PPV/NPV/positive likelihood ratio/negative likelihood ratio for UGI neoplasia detection were 90% (CI 85% to 94%)/89% (CI 85% to 92%)/87% (CI 83% to 91%)/91% (CI 87% to 94%)/8.2 (CI 5.7 to 11.7)/0.111 (CI 0.071 to 0.175), respectively, with an overall AUC of 0.95 (CI 0.93 to 0.97). No difference in AI performance across ESCN, BERN and GCA was found, AUC being 0.94 (CI 0.52 to 0.99), 0.96 (CI 0.95 to 0.98), 0.93 (CI 0.83 to 0.99), respectively. Overall, study quality was low, with high risk of selection bias. No significant publication bias was found.

CONCLUSION

We found a high overall AI accuracy for the diagnosis of any neoplastic lesion of the UGI tract that was independent of the underlying condition. This may be expected to substantially reduce the miss rate of precancerous lesions and early cancer when implemented in clinical practice.

Artificial intelligence-assisted esophageal cancer management: Now and future

Esophageal cancer poses diagnostic, therapeutic and economic burdens in high-risk regions. Artificial intelligence (AI) has been developed for diagnosis and outcome prediction using various features, including clinicopathologic, radiologic, and genetic variables, which can achieve inspiring results. One of the most recent tasks of AI is to use state-of-the-art deep learning technique to detect both early esophageal squamous cell carcinoma and esophageal adenocarcinoma in Barrett’s esophagus. In this review, we aim to provide a comprehensive overview of the ways in which AI may help physicians diagnose advanced cancer and make clinical decisions based on predicted outcomes, and combine the endoscopic images to detect precancerous lesions or early cancer. Pertinent studies conducted in recent two years have surged in numbers, with large datasets and external validation from multi-centers, and have partly achieved intriguing results of expert’s performance of AI in real time. Improved pre-trained computer-aided diagnosis algorithms in the future studies with larger training and external validation datasets, aiming at real-time video processing, are imperative to produce a diagnostic efficacy similar to or even superior to experienced endoscopists. Meanwhile, supervised randomized controlled trials in real clinical practice are highly essential for a solid conclusion, which meets patient-centered satisfaction. Notably, ethical and legal issues regarding the black-box nature of computer algorithms should be addressed, for both clinicians and regulators.

The Impact of Artificial Intelligence in the Endoscopic Assessment of Premalignant and Malignant Esophageal Lesions: Present and Future

In the gastroenterology field, the impact of artificial intelligence was investigated for the purposes of diagnostics, risk stratification of patients, improvement in quality of endoscopic procedures and early detection of neoplastic diseases, implementation of the best treatment strategy, and optimization of patient prognosis. Computer-assisted diagnostic systems to evaluate upper endoscopy images have recently emerged as a supporting tool in endoscopy due to the risks of misdiagnosis related to standard endoscopy and different expertise levels of endoscopists, time-consuming procedures, lack of availability of advanced procedures, increasing workloads, and development of endoscopic mass screening programs. Recent research has tended toward computerized, automatic, and real-time detection of lesions, which are approaches that offer utility in daily practice. Despite promising results, certain studies might overexaggerate the diagnostic accuracy of artificial systems, and several limitations remain to be overcome in the future. Therefore, additional multicenter randomized trials and the development of existent database platforms are needed to certify clinical implementation. This paper presents an overview of the literature and the current knowledge of the usefulness of different types of machine learning systems in the assessment of premalignant and malignant esophageal lesions via conventional and advanced endoscopic procedures. This study makes a presentation of the artificial intelligence terminology and refers also to the most prominent recent research on computer-assisted diagnosis of neoplasia on Barrett’s esophagus and early esophageal squamous cell carcinoma, and prediction of invasion depth in esophageal neoplasms. Furthermore, this review highlights the main directions of future doctor–computer collaborations in which machines are expected to improve the quality of medical action and routine clinical workflow, thus reducing the burden on physicians.

Automatic detection of periodontal compromised teeth in digital panoramic radiographs using faster regional convolutional neural networks

Purpose

Periodontal disease causes tooth loss and is associated with cardiovascular diseases, diabetes, and rheumatoid arthritis. The present study proposes using a deep learning-based object detection method to identify periodontally compromised teeth on digital panoramic radiographs. A faster regional convolutional neural network (faster R-CNN) which is a state-of-the-art deep detection network, was adapted from the natural image domain using a small annotated clinical data- set.

Materials and Methods

In total, 100 digital panoramic radiographs of periodontally compromised patients were retrospectively collected from our hospital's information system and augmented. The periodontally compromised teeth found in each image were annotated by experts in periodontology to obtain the ground truth. The Keras library, which is written in Python, was used to train and test the model on a single NVidia 1080Ti GPU. The faster R-CNN model used a pretrained ResNet architecture.

Results

The average precision rate of 0.81 demonstrated that there was a significant region of overlap between the predicted regions and the ground truth. The average recall rate of 0.80 showed that the periodontally compromised teeth regions generated by the detection method excluded healthiest teeth areas. In addition, the model achieved a sensitivity of 0.84, a specificity of 0.88 and an F-measure of 0.81.

Conclusion

The faster R-CNN trained on a limited amount of labeled imaging data performed satisfactorily in detecting periodontally compromised teeth. The application of a faster R-CNN to assist in the detection of periodontally compromised teeth may reduce diagnostic effort by saving assessment time and allowing automated screening documentation.

Artificial intelligence using convolutional neural networks for real-time detection of early esophageal neoplasia in Barrett's esophagus (with video)

BACKGROUND AND AIMS

The visual detection of early esophageal neoplasia (high-grade dysplasia and T1 cancer) in Barrett's esophagus (BE) with white-light and virtual chromoendoscopy still remains challenging. The aim of this study was to assess whether a convolutional neural artificial intelligence network can aid in the recognition of early esophageal neoplasia in BE.

METHODS

Nine hundred sixteen images from 65 patients of histology-proven early esophageal neoplasia in BE containing high-grade dysplasia or T1 cancer were collected. The area of neoplasia was masked using image annotation software. Nine hundred nineteen control images were collected of BE without high-grade dysplasia. A convolutional neural network (CNN) algorithm was pretrained on ImageNet and then fine-tuned with the goal of providing the correct binary classification of "dysplastic" or "nondysplastic." We developed an object detection algorithm that drew localization boxes around regions classified as dysplasia.

RESULTS

The CNN analyzed 458 test images (225 dysplasia and 233 nondysplasia) and correctly detected early neoplasia with sensitivity of 96.4%, specificity of 94.2%, and accuracy of 95.4%. With regard to the object detection algorithm for all images in the validation set, the system was able to achieve a mean average precision of .7533 at an intersection over union of .3 CONCLUSIONS: In this pilot study, our artificial intelligence model was able to detect early esophageal neoplasia in BE images with high accuracy. In addition, the object detection algorithm was able to draw a localization box around the areas of dysplasia with high precision and at a speed that allows for real-time implementation.