Application of artificial intelligence using a convolutional neural network for diagnosis of early gastric cancer based on magnifying endoscopy with narrow-band imaging

Accuracy of artificial intelligence-assisted endoscopy in the diagnosis of gastric intestinal metaplasia: A systematic review and meta-analysis

BACKGROUND AND AIMS

Gastric intestinal metaplasia is a precancerous disease, and a timely diagnosis is essential to delay or halt cancer progression. Artificial intelligence (AI) has found widespread application in the field of disease diagnosis. This study aimed to conduct a comprehensive evaluation of AI's diagnostic accuracy in detecting gastric intestinal metaplasia in endoscopy, compare it to endoscopists' ability, and explore the main factors affecting AI's performance.

METHODS

The study followed the PRISMA-DTA guidelines, and the PubMed, Embase, Web of Science, Cochrane, and IEEE Xplore databases were searched to include relevant studies published by October 2023. We extracted the key features and experimental data of each study and combined the sensitivity and specificity metrics by meta-analysis. We then compared the diagnostic ability of the AI versus the endoscopists using the same test data.

RESULTS

Twelve studies with 11,173 patients were included, demonstrating AI models' efficacy in diagnosing gastric intestinal metaplasia. The meta-analysis yielded a pooled sensitivity of 94% (95% confidence interval: 0.92-0.96) and specificity of 93% (95% confidence interval: 0.89-0.95). The combined area under the receiver operating characteristics curve was 0.97. The results of meta-regression and subgroup analysis showed that factors such as study design, endoscopy type, number of training images, and algorithm had a significant effect on the diagnostic performance of AI. The AI exhibited a higher diagnostic capacity than endoscopists (sensitivity: 95% vs. 79%).

CONCLUSIONS

AI-aided diagnosis of gastric intestinal metaplasia using endoscopy showed high performance and clinical diagnostic value. However, further prospective studies are required to validate these findings.

Early gastric cancer detection and lesion segmentation based on deep learning and gastroscopic images

Gastric cancer is a highly prevalent disease that poses a serious threat to public health. In clinical practice, gastroscopy is frequently used by medical practitioners to screen for gastric cancer. However, the symptoms of gastric cancer at different stages of advancement vary significantly, particularly in the case of early gastric cancer (EGC). The manifestations of EGC are often indistinct, leading to a detection rate of less than 10%. In recent years, researchers have focused on leveraging deep learning algorithms to assist medical professionals in detecting EGC and thereby improve detection rates. To enhance the ability of deep learning to detect EGC and segment lesions in gastroscopic images, an Improved Mask R-CNN (IMR-CNN) model was proposed. This model incorporates a "Bi-directional feature extraction and fusion module" and a "Purification module for feature channel and space" based on the Mask R-CNN (MR-CNN). Our study includes a dataset of 1120 images of EGC for training and validation of the models. The experimental results indicate that the IMR-CNN model outperforms the original MR-CNN model, with Precision, Recall, Accuracy, Specificity and F1-Score values of 92.9%, 95.3%, 93.9%, 92.5% and 94.1%, respectively. Therefore, our proposed IMR-CNN model has superior detection and lesion segmentation capabilities and can effectively aid doctors in diagnosing EGC from gastroscopic images.

Advanced diagnostic endoscopy in the upper gastrointestinal tract: Review of the Japan Gastroenterological Endoscopic Society core sessions

The Japan Gastroenterological Endoscopy Society (JGES) held four serial symposia between 2021 and 2022 on state-of-the-art issues related to advanced diagnostic endoscopy of the upper gastrointestinal tract. This review summarizes the four core sessions and presents them as a conference report. Eleven studies were discussed in the 101st JGES Core Session, which addressed the challenges and prospects of upper gastroenterological endoscopy. Ten studies were also explored in the 102nd JGES Core Session on advanced upper gastrointestinal endoscopic diagnosis for decision-making regarding therapeutic strategies. Moreover, eight studies were presented during the 103rd JGES Core Session on the development and evaluation of endoscopic artificial intelligence in the field of upper gastrointestinal endoscopy. Twelve studies were also discussed in the 104th JGES Core Session, which focused on the evidence and new developments related to the upper gastrointestinal tract. The endoscopic diagnosis of upper gastrointestinal diseases using image-enhanced endoscopy and AI is one of the most recent topics and has received considerable attention. These four core sessions enabled us to grasp the current state-of-the-art in upper gastrointestinal endoscopic diagnostics and identify future challenges. Based on these studies, we hope that an endoscopic diagnostic system useful in clinical practice is established for each field of upper gastrointestinal endoscopy.

Development of an Artificial Intelligence Diagnostic System Using Linked Color Imaging for Barrett's Esophagus

Background: Barrett's esophagus and esophageal adenocarcinoma cases are increasing as gastroesophageal reflux disease increases. Using artificial intelligence (AI) and linked color imaging (LCI), our aim was to establish a method of diagnosis for short-segment Barrett's esophagus (SSBE). Methods: We retrospectively selected 624 consecutive patients in total at our hospital, treated between May 2017 and March 2020, who experienced an esophagogastroduodenoscopy with white light imaging (WLI) and LCI. Images were randomly chosen as data for learning from WLI: 542 (SSBE+/- 348/194) of 696 (SSBE+/- 444/252); and LCI: 643 (SSBE+/- 446/197) of 805 (SSBE+/- 543/262). Using a Vision Transformer (Vit-B/16-384) to diagnose SSBE, we established two AI systems for WLI and LCI. Finally, 126 WLI (SSBE+/- 77/49) and 137 LCI (SSBE+/- 81/56) images were used for verification purposes. The accuracy of six endoscopists in making diagnoses was compared to that of AI. Results: Study participants were 68.2 ± 12.3 years, M/F 330/294, SSBE+/- 409/215. The accuracy/sensitivity/specificity (%) of AI were 84.1/89.6/75.5 for WLI and 90.5/90.1/91.1/for LCI, and those of experts and trainees were 88.6/88.7/88.4, 85.7/87.0/83.7 for WLI and 93.4/92.6/94.6, 84.7/88.1/79.8 for LCI, respectively. Conclusions: Using AI to diagnose SSBE was similar in accuracy to using a specialist. Our finding may aid the diagnosis of SSBE in the clinic.

Convolutional neural network applied to preoperative venous-phase CT images predicts risk category in patients with gastric gastrointestinal stromal tumors

OBJECTIVE

The risk category of gastric gastrointestinal stromal tumors (GISTs) are closely related to the surgical method, the scope of resection, and the need for preoperative chemotherapy. We aimed to develop and validate convolutional neural network (CNN) models based on preoperative venous-phase CT images to predict the risk category of gastric GISTs.

METHOD

A total of 425 patients pathologically diagnosed with gastric GISTs at the authors' medical centers between January 2012 and July 2021 were split into a training set (154, 84, and 59 with very low/low, intermediate, and high-risk, respectively) and a validation set (67, 35, and 26, respectively). Three CNN models were constructed by obtaining the upper and lower 1, 4, and 7 layers of the maximum tumour mask slice based on venous-phase CT Images and models of CNN_layer3, CNN_layer9, and CNN_layer15 established, respectively. The area under the receiver operating characteristics curve (AUROC) and the Obuchowski index were calculated to compare the diagnostic performance of the CNN models.

RESULTS

In the validation set, CNN_layer3, CNN_layer9, and CNN_layer15 had AUROCs of 0.89, 0.90, and 0.90, respectively, for low-risk gastric GISTs; 0.82, 0.83, and 0.83 for intermediate-risk gastric GISTs; and 0.86, 0.86, and 0.85 for high-risk gastric GISTs. In the validation dataset, CNN_layer3 (Obuchowski index, 0.871) provided similar performance than CNN_layer9 and CNN_layer15 (Obuchowski index, 0.875 and 0.873, respectively) in prediction of the gastric GIST risk category (All P >.05).

CONCLUSIONS

The CNN based on preoperative venous-phase CT images showed good performance for predicting the risk category of gastric GISTs.

Development and evaluation of a program based on a generative pre-trained transformer model from a public natural language processing platform for efficiency enhancement in post-procedural quality control of esophageal endoscopic submucosal dissection

BACKGROUND

Post-procedural quality control of endoscopic submucosal dissection (ESD) is emphasized in guidelines. However, this process can be tedious and time-consuming. Recently, a pre-training model called generative pre-trained transformer (GPT) on a public natural language processing platform has emerged and garnered significant attention, whose capabilities align well with the post-procedural quality control process and have the potential to streamline it. Therefore, we developed a simple program utilizing this platform and evaluated its performance.

METHODS

Esophageal ESDs were retrospectively included. The manual quality control process was performed and act as reference standard. GPT's prompt was optimized through multiple iterations. A Python program was developed to automatically submit prompt with pathological report of each ESD procedure and collect quality control information provided by GPT. Its performance on quality control was evaluated with accuracy, precision, recall, and F-1 score.

RESULTS

165 cases were involved into the dataset, of which 5 were utilized as the prompt optimization dataset and 160 as the validation dataset. Definitive prompt was achieved through seven iterations. Time spent on the validation dataset by GPT was 13.47 ± 2.43 min. Accuracies of pathological diagnosis, invasion depth, horizontal margin, vertical margin, vascular invasion, and lymphatic invasion of the quality control program were (0.940, 0.952) (95% CI), (0.925, 0.945) (95% CI), 0.931, 1.0, and 1.0, respectively. Precisions were (0.965, 0.969) (95% CI), (0.934, 0.954) (95% CI), and 0.957 for pathological diagnosis, invasion depth, and horizontal margin, respectively. Recalls were (0.940, 0.952) (95% CI), (0.925, 0.945) (95% CI), and 0.931 for factors as mentioned, respectively. F1-score were (0.945, 0.957) (95% CI), (0.928, 0.948) (95% CI), and 0.941 for factors as mentioned, respectively.

CONCLUSIONS

This quality control program was qualified of post-procedural quality control of esophageal ESDs. GPT can be easily applied to this quality control process and reduce workload of the endoscopists.

Convolutional neural network misclassification analysis in oral lesions: an error evaluation criterion by image characteristics

OBJECTIVE

This retrospective study analyzed the errors generated by a convolutional neural network (CNN) when performing automated classification of oral lesions according to their clinical characteristics, seeking to identify patterns in systemic errors in the intermediate layers of the CNN.

STUDY DESIGN

A cross-sectional analysis nested in a previous trial in which automated classification by a CNN model of elementary lesions from clinical images of oral lesions was performed. The resulting CNN classification errors formed the dataset for this study. A total of 116 real outputs were identified that diverged from the estimated outputs, representing 7.6% of the total images analyzed by the CNN.

RESULTS

The discrepancies between the real and estimated outputs were associated with problems relating to image sharpness, resolution, and focus; human errors; and the impact of data augmentation.

CONCLUSIONS

From qualitative analysis of errors in the process of automated classification of clinical images, it was possible to confirm the impact of image quality, as well as identify the strong impact of the data augmentation process. Knowledge of the factors that models evaluate to make decisions can increase confidence in the high classification potential of CNNs.

The effect of incorporating domain knowledge with deep learning in identifying benign and malignant gastric whitish lesions: A retrospective study

BACKGROUND AND AIM

Early whitish gastric neoplasms can be easily misdiagnosed; differential diagnosis of gastric whitish lesions remains a challenge. We aim to build a deep learning (DL) model to diagnose whitish gastric neoplasms and explore the effect of adding domain knowledge in model construction.

METHODS

We collected 4558 images from two institutions to train and test models. We first developed two sole DL models (1 and 2) using supervised and semi-supervised algorithms. Then we selected diagnosis-related features through literature research and developed feature-extraction models to determine features including boundary, surface, roundness, depression, and location. Then predictions of the five feature-extraction models and sole DL model were combined and inputted into seven machine-learning (ML) based fitting-diagnosis models. The optimal model was selected as ENDOANGEL-WD (whitish-diagnosis) and compared with endoscopists.

RESULTS

Sole DL 2 had higher sensitivity (83.12% vs 68.67%, Bonferroni adjusted P = 0.024) than sole DL 1. Adding domain knowledge, the decision tree performed best among the seven ML models, achieving higher specificity than DL 1 (84.38% vs 72.27%, Bonferroni adjusted P < 0.05) and higher accuracy than DL 2 (80.47%, Bonferroni adjusted P < 0.001) and was selected as ENDOANGEL-WD. ENDOANGEL-WD showed better accuracy compared with 10 endoscopists (75.70%, P < 0.001).

CONCLUSIONS

We developed a novel system ENDOANGEL-WD combining domain knowledge and traditional DL to detect gastric whitish neoplasms. Adding domain knowledge improved the performance of traditional DL, which provided a novel solution for establishing diagnostic models for other rare diseases potentially.

The value of machine learning approaches in the diagnosis of early gastric cancer: a systematic review and meta-analysis

BACKGROUND

The application of machine learning (ML) for identifying early gastric cancer (EGC) has drawn increasing attention. However, there lacks evidence-based support for its specific diagnostic performance. Hence, this systematic review and meta-analysis was implemented to assess the performance of image-based ML in EGC diagnosis.

METHODS

We performed a comprehensive electronic search in PubMed, Embase, Cochrane Library, and Web of Science up to September 25, 2022. QUADAS-2 was selected to judge the risk of bias of included articles. We did the meta-analysis using a bivariant mixed-effect model. Sensitivity analysis and heterogeneity test were performed.

RESULTS

Twenty-one articles were enrolled. The sensitivity (SEN), specificity (SPE), and SROC of ML-based models were 0.91 (95% CI: 0.87-0.94), 0.85 (95% CI: 0.81-0.89), and 0.94 (95% CI: 0.39-1.00) in the training set and 0.90 (95% CI: 0.86-0.93), 0.90 (95% CI: 0.86-0.92), and 0.96 (95% CI: 0.19-1.00) in the validation set. The SEN, SPE, and SROC of EGC diagnosis by non-specialist clinicians were 0.64 (95% CI: 0.56-0.71), 0.84 (95% CI: 0.77-0.89), and 0.80 (95% CI: 0.29-0.97), and those by specialist clinicians were 0.80 (95% CI: 0.74-0.85), 0.88 (95% CI: 0.85-0.91), and 0.91 (95% CI: 0.37-0.99). With the assistance of ML models, the SEN of non-specialist physicians in the diagnosis of EGC was significantly improved (0.76 vs 0.64).

CONCLUSION

ML-based diagnostic models have greater performance in the identification of EGC. The diagnostic accuracy of non-specialist clinicians can be improved to the level of the specialists with the assistance of ML models. The results suggest that ML models can better assist less experienced clinicians in diagnosing EGC under endoscopy and have broad clinical application value.

Applications of artificial intelligence in dentomaxillofacial imaging-a systematic review

BACKGROUND

Artificial intelligence (AI) technology has been increasingly developed in oral and maxillofacial imaging. The aim of this systematic review was to assess the applications and performance of the developed algorithms in different dentomaxillofacial imaging modalities.

STUDY DESIGN

A systematic search of PubMed and Scopus databases was performed. The search strategy was set as a combination of the following keywords: "Artificial Intelligence," "Machine Learning," "Deep Learning," "Neural Networks," "Head and Neck Imaging," and "Maxillofacial Imaging." Full-text screening and data extraction were independently conducted by two independent reviewers; any mismatch was resolved by discussion. The risk of bias was assessed by one reviewer and validated by another.

RESULTS

The search returned a total of 3,392 articles. After careful evaluation of the titles, abstracts, and full texts, a total number of 194 articles were included. Most studies focused on AI applications for tooth and implant classification and identification, 3-dimensional cephalometric landmark detection, lesion detection (periapical, jaws, and bone), and osteoporosis detection.

CONCLUSION

Despite the AI models' limitations, they showed promising results. Further studies are needed to explore specific applications and real-world scenarios before confidently integrating these models into dental practice.

Application of artificial intelligence for diagnosis of early gastric cancer based on magnifying endoscopy with narrow-band imaging

Although magnifying endoscopy with narrow-band imaging is the standard diagnostic test for gastric cancer, diagnosing gastric cancer using this technology requires considerable skill. Artificial intelligence has superior image recognition, and its usefulness in endoscopic image diagnosis has been reported in many cases. The diagnostic performance (accuracy, sensitivity, and specificity) of artificial intelligence using magnifying endoscopy with narrow band still images and videos for gastric cancer was higher than that of expert endoscopists, suggesting the usefulness of artificial intelligence in diagnosing gastric cancer. Histological diagnosis of gastric cancer using artificial intelligence is also promising. However, previous studies on the use of artificial intelligence to diagnose gastric cancer were small-scale; thus, large-scale studies are necessary to examine whether a high diagnostic performance can be achieved. In addition, the diagnosis of gastric cancer using artificial intelligence has not yet become widespread in clinical practice, and further research is necessary. Therefore, in the future, artificial intelligence must be further developed as an instrument, and its diagnostic performance is expected to improve with the accumulation of numerous cases nationwide.

Identification of gastric cancer types based on hyperspectral imaging technology

Gastric cancer is becoming the second biggest cause of death from cancer. Treatment and prognosis of different types of gastric cancer vary greatly. However, the routine pathological examination is limited to the tissue level and is easily affected by subjective factors. In our study, we examined gastric mucosal samples from 50 normal tissue and 90 cancer tissues. Hyperspectral imaging technology was used to obtain spectral information. A two-classification model for normal tissue and cancer tissue identification and a four-classification model for cancer type identification are constructed based on the improved deep residual network (IDRN). The accuracy of the two-classification model and four-classification model are 0.947 and 0.965. Hyperspectral imaging technology was used to extract molecular information to realize real-time diagnosis and accurate typing. The results show that hyperspectral imaging technique has good effect on diagnosis and type differentiation of gastric cancer, which is expected to be used in auxiliary diagnosis and treatment.

Application of artificial intelligence in endoscopic gastrointestinal tumors

With an increasing number of patients with gastrointestinal cancer, effective and accurate early diagnostic clinical tools are required provide better health care for patients with gastrointestinal cancer. Recent studies have shown that artificial intelligence (AI) plays an important role in the diagnosis and treatment of patients with gastrointestinal tumors, which not only improves the efficiency of early tumor screening, but also significantly improves the survival rate of patients after treatment. With the aid of efficient learning and judgment abilities of AI, endoscopists can improve the accuracy of diagnosis and treatment through endoscopy and avoid incorrect descriptions or judgments of gastrointestinal lesions. The present article provides an overview of the application status of various artificial intelligence in gastric and colorectal cancers in recent years, and the direction of future research and clinical practice is clarified from a clinical perspective to provide a comprehensive theoretical basis for AI as a promising diagnostic and therapeutic tool for gastrointestinal cancer.

Deep Learning and Gastric Cancer: Systematic Review of AI-Assisted Endoscopy

BACKGROUND

Gastric cancer (GC), a significant health burden worldwide, is typically diagnosed in the advanced stages due to its non-specific symptoms and complex morphological features. Deep learning (DL) has shown potential for improving and standardizing early GC detection. This systematic review aims to evaluate the current status of DL in pre-malignant, early-stage, and gastric neoplasia analysis.

METHODS

A comprehensive literature search was conducted in PubMed/MEDLINE for original studies implementing DL algorithms for gastric neoplasia detection using endoscopic images. We adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The focus was on studies providing quantitative diagnostic performance measures and those comparing AI performance with human endoscopists.

RESULTS

Our review encompasses 42 studies that utilize a variety of DL techniques. The findings demonstrate the utility of DL in GC classification, detection, tumor invasion depth assessment, cancer margin delineation, lesion segmentation, and detection of early-stage and pre-malignant lesions. Notably, DL models frequently matched or outperformed human endoscopists in diagnostic accuracy. However, heterogeneity in DL algorithms, imaging techniques, and study designs precluded a definitive conclusion about the best algorithmic approach.

CONCLUSIONS

The promise of artificial intelligence in improving and standardizing gastric neoplasia detection, diagnosis, and segmentation is significant. This review is limited by predominantly single-center studies and undisclosed datasets used in AI training, impacting generalizability and demographic representation. Further, retrospective algorithm training may not reflect actual clinical performance, and a lack of model details hinders replication efforts. More research is needed to substantiate these findings, including larger-scale multi-center studies, prospective clinical trials, and comprehensive technical reporting of DL algorithms and datasets, particularly regarding the heterogeneity in DL algorithms and study designs.

The adoption of artificial intelligence assisted endoscopy in the Middle East: challenges and future potential

The use of artificial intelligence (AI) in endoscopy has shown immense potential to enhance diagnostic accuracy, streamline procedures, and improve patient outcomes. There are potential uses in every field of endoscopy, from improving adenoma detection rate (ADR) in colonoscopy to reducing read time in capsule endoscopy or minimizing blind spots in gastroscopy. Indeed, some of these systems are already licensed and in commercial use across the world. In the Middle East, where healthcare systems are rapidly evolving, there is a growing interest in adopting AI technologies to revolutionise endoscopic practices. This article provides an overview of the advancements, potential opportunities and challenges associated with the implementation of AI in endoscopy within the Middle East region. Our aim is to contribute to the ongoing dialogue surrounding the implementation of AI in endoscopy and consider some of the factors that are particularly relevant in the Middle Eastern context, including the need to train the models for local populations, cost and training, as well as trying to ensure equity of access for patients. It provides valuable insights for healthcare professionals, policymakers, and researchers interested in leveraging AI to enhance endoscopic procedures, improve patient care, and address the unique healthcare needs of the Middle East population.

Four Markers Useful for the Distinction of Intrauterine Growth Restriction in Sheep

Intrauterine growth restriction (IUGR) is a common perinatal complication in animal reproduction, with long-lasting negative effects on neonates and postnatal animals, which seriously negatively affects livestock production. In this study, we aimed to identify potential genes associated with the diagnosis of IUGR through bioinformatics analysis. Based on the 73 differentially expressed related genes obtained by differential analysis and weighted gene co-expression network analysis, we used three machine learning algorithms to identify 4 IUGR-related hub genes (IUGR-HGs), namely, ADAM9, CRYL1, NDP52, and SERPINA7, whose ROC curves showed that they are a good diagnostic target for IUGR. Next, we identified two molecular subtypes of IUGR through consensus clustering analysis and constructed a gene scoring system based on the IUGR-HGs. The results showed that the IUGR score was positively correlated with the risk of IUGR. The AUC value of IUGR scoring accuracy was 0.970. Finally, we constructed a new artificial neural network model based on the four IUGR-HGs to diagnose sheep IUGR, and its accuracy reached 0.956. In conclusion, the IUGR-HGs we identified provide new potential molecular markers and models for the diagnosis of IUGR in sheep; they can better diagnose whether sheep have IUGR. The present findings provide new perspectives on the diagnosis of IUGR.

A Contrast-Enhanced CT-Based Deep Learning System for Preoperative Prediction of Colorectal Cancer Staging and RAS Mutation

PURPOSE

This study aimed to build a deep learning system using enhanced computed tomography (CT) portal-phase images for predicting colorectal cancer patients' preoperative staging and RAS gene mutation status.

METHODS

The contrast-enhanced CT image dataset comprises the CT portal-phase images from a retrospective cohort of 231 colorectal cancer patients. The deep learning system was developed via migration learning for colorectal cancer detection, staging, and RAS gene mutation status prediction. This study used pre-trained Yolov7, vision transformer (VIT), swin transformer (SWT), EfficientNetV2, and ConvNeXt. 4620, and contrast-enhanced CT images and annotated tumor bounding boxes were included in the tumor identification and staging dataset. A total of 19,700 contrast-enhanced CT images comprise the RAS gene mutation status prediction dataset.

RESULTS

In the validation cohort, the Yolov7-based detection model detected and staged tumors with a mean accuracy precision (IoU = 0.5) (mAP_0.5) of 0.98. The area under the receiver operating characteristic curve (AUC) in the test set and validation set for the VIT-based prediction model in predicting the mutation status of the RAS genes was 0.9591 and 0.9554, respectively. The detection network and prediction network of the deep learning system demonstrated great performance in explaining contrast-enhanced CT images.

CONCLUSION

In this study, a deep learning system was created based on the foundation of contrast-enhanced CT portal-phase imaging to preoperatively predict the stage and RAS mutation status of colorectal cancer patients. This system will help clinicians choose the best treatment option to increase colorectal cancer patients' chances of survival and quality of life.

Artificial intelligence in endoscopy: Overview, applications, and future directions

Since the emergence of artificial intelligence (AI) in medicine, endoscopy applications in gastroenterology have been at the forefront of innovations. The ever-increasing number of studies necessitates the need to organize and classify applications in a useful way. Separating AI capabilities by computer aided detection (CADe), diagnosis (CADx), and quality assessment (CADq) allows for a systematic evaluation of each application. CADe studies have shown promise in accurate detection of esophageal, gastric and colonic neoplasia as well as identifying sources of bleeding and Crohn's disease in the small bowel. While more advanced CADx applications employ optical biopsies to give further information to characterize neoplasia and grade inflammatory disease, diverse CADq applications ensure quality and increase the efficiency of procedures. Future applications show promise in advanced therapeutic modalities and integrated systems that provide multimodal capabilities. AI is set to revolutionize clinical decision making and performance of endoscopy.

Application of artificial intelligence in endoscopic image analysis for the diagnosis of a gastric cancer pathogen-Helicobacter pylori infection

Helicobacter pylori (H. pylori) infection is the principal cause of chronic gastritis, gastric ulcers, duodenal ulcers, and gastric cancer. In clinical practice, diagnosis of H. pylori infection by a gastroenterologists' impression of endoscopic images is inaccurate and cannot be used for the management of gastrointestinal diseases. The aim of this study was to develop an artificial intelligence classification system for the diagnosis of H. pylori infection by pre-processing endoscopic images and machine learning methods. Endoscopic images of the gastric body and antrum from 302 patients receiving endoscopy with confirmation of H. pylori status by a rapid urease test at An Nan Hospital were obtained for the derivation and validation of an artificial intelligence classification system. The H. pylori status was interpreted as positive or negative by Convolutional Neural Network (CNN) and Concurrent Spatial and Channel Squeeze and Excitation (scSE) network, combined with different classification models for deep learning of gastric images. The comprehensive assessment for H. pylori status by scSE-CatBoost classification models for both body and antrum images from same patients achieved an accuracy of 0.90, sensitivity of 1.00, specificity of 0.81, positive predictive value of 0.82, negative predicted value of 1.00, and area under the curve of 0.88. The data suggest that an artificial intelligence classification model using scSE-CatBoost deep learning for gastric endoscopic images can distinguish H. pylori status with good performance and is useful for the survey or diagnosis of H. pylori infection in clinical practice.

Developing neural network diagnostic models and potential drugs based on novel identified immune-related biomarkers for celiac disease

BACKGROUND

As one of the most common intestinal inflammatory diseases, celiac disease (CD) is typically characterized by an autoimmune disorder resulting from ingesting gluten proteins. Although the incidence and prevalence of CD have increased over time, the diagnostic methods and treatment options are still limited. Therefore, it is urgent to investigate the potential biomarkers and targeted drugs for CD.

METHODS

Gene expression data was downloaded from GEO datasets. Differential gene expression analysis was performed to identify the dysregulated immune-related genes. Multiple machine algorithms, including randomForest, SVM-RFE, and LASSO, were used to select the hub immune-related genes (HIGs). The immune-related genes score (IG score) and artificial neural network (ANN) were constructed based on HIGs. Potential drugs targeting HIGs were identified by using the Enrichr platform and molecular docking method.

RESULTS

We identified the dysregulated immune-related genes at a genome-wide level and demonstrated their roles in CD-related immune pathways. The hub genes (MR1, CCL25, and TNFSF13B) were further screened by integrating several machine algorithms. Meanwhile, the CD patients were divided into distinct subtypes with either high- or low-immunoactivity using single-sample gene set enrichment analysis (ssGSEA) and consensus clustering. By constructing IG score based on HIGs, we found that patients with high IG score were mainly attributed to high-immunoactivity subgroups, which suggested a strong link between HIGs and immunoactivity of CD patients. In addition, the novel constructed ANN model showed the sound diagnostic ability of HIGs. Mechanistically, we validated that the HIGs play pivotal roles in regulating CD's immune and inflammatory state. Through targeting the HIGs, we also found potential drugs for anti-CD treatment by using the Enrichr platform and molecular docking method.

CONCLUSIONS

This study unveils the HIGs and elucidates the networks regulated by these genes in the context of CD. It underscores the pivotal significance of HIGs in accurately predicting the presence or absence of CD in patients. Consequently, this research offers promising prospects for the development of diagnostic biomarkers and therapeutic targets for CD.

Artificial Intelligence in the Pathology of Gastric Cancer

Recent advances in artificial intelligence (AI) have provided novel tools for rapid and precise pathologic diagnosis. The introduction of digital pathology has enabled the acquisition of scanned slide images that are essential for the application of AI. The application of AI for improved pathologic diagnosis includes the error-free detection of potentially negligible lesions, such as a minute focus of metastatic tumor cells in lymph nodes, the accurate diagnosis of potentially controversial histologic findings, such as very well-differentiated carcinomas mimicking normal epithelial tissues, and the pathological subtyping of the cancers. Additionally, the utilization of AI algorithms enables the precise decision of the score of immunohistochemical markers for targeted therapies, such as human epidermal growth factor receptor 2 and programmed death-ligand 1. Studies have revealed that AI assistance can reduce the discordance of interpretation between pathologists and more accurately predict clinical outcomes. Several approaches have been employed to develop novel biomarkers from histologic images using AI. Moreover, AI-assisted analysis of the cancer microenvironment showed that the distribution of tumor-infiltrating lymphocytes was related to the response to the immune checkpoint inhibitor therapy, emphasizing its value as a biomarker. As numerous studies have demonstrated the significance of AI-assisted interpretation and biomarker development, the AI-based approach will advance diagnostic pathology.

Spatiotemporal analysis of small bowel capsule endoscopy videos for outcomes prediction in Crohn's disease

Background

Deep learning techniques can accurately detect and grade inflammatory findings on images from capsule endoscopy (CE) in Crohn's disease (CD). However, the predictive utility of deep learning of CE in CD for disease outcomes has not been examined.

Objectives

We aimed to develop a deep learning model that can predict the need for biological therapy based on complete CE videos of newly-diagnosed CD patients.

Design

This was a retrospective cohort study. The study cohort included treatment-naïve CD patients that have performed CE (SB3, Medtronic) within 6 months of diagnosis. Complete small bowel videos were extracted using the RAPID Reader software.

Methods

CE videos were scored using the Lewis score (LS). Clinical, endoscopic, and laboratory data were extracted from electronic medical records. Machine learning analysis was performed using the TimeSformer computer vision algorithm developed to capture spatiotemporal characteristics for video analysis.

Results

The patient cohort included 101 patients. The median duration of follow-up was 902 (354-1626) days. Biological therapy was initiated by 37 (36.6%) out of 101 patients. TimeSformer algorithm achieved training and testing accuracy of 82% and 81%, respectively, with an Area under the ROC Curve (AUC) of 0.86 to predict the need for biological therapy. In comparison, the AUC for LS was 0.70 and for fecal calprotectin 0.74.

Conclusion

Spatiotemporal analysis of complete CE videos of newly-diagnosed CD patients achieved accurate prediction of the need for biological therapy. The accuracy was superior to that of the human reader index or fecal calprotectin. Following future validation studies, this approach will allow for fast and accurate personalization of treatment decisions in CD.

A deep-learning based system using multi-modal data for diagnosing gastric neoplasms in real-time (with video)

BACKGROUND

White light (WL) and weak-magnifying (WM) endoscopy are both important methods for diagnosing gastric neoplasms. This study constructed a deep-learning system named ENDOANGEL-MM (multi-modal) aimed at real-time diagnosing gastric neoplasms using WL and WM data.

METHODS

WL and WM images of a same lesion were combined into image-pairs. A total of 4201 images, 7436 image-pairs, and 162 videos were used for model construction and validation. Models 1-5 including two single-modal models (WL, WM) and three multi-modal models (data fusion on task-level, feature-level, and input-level) were constructed. The models were tested on three levels including images, videos, and prospective patients. The best model was selected for constructing ENDOANGEL-MM. We compared the performance between the models and endoscopists and conducted a diagnostic study to explore the ENDOANGEL-MM's assistance ability.

RESULTS

Model 4 (ENDOANGEL-MM) showed the best performance among five models. Model 2 performed better in single-modal models. The accuracy of ENDOANGEL-MM was higher than that of Model 2 in still images, real-time videos, and prospective patients. (86.54 vs 78.85%, P = 0.134; 90.00 vs 85.00%, P = 0.179; 93.55 vs 70.97%, P < 0.001). Model 2 and ENDOANGEL-MM outperformed endoscopists on WM data (85.00 vs 71.67%, P = 0.002) and multi-modal data (90.00 vs 76.17%, P = 0.002), significantly. With the assistance of ENDOANGEL-MM, the accuracy of non-experts improved significantly (85.75 vs 70.75%, P = 0.020), and performed no significant difference from experts (85.75 vs 89.00%, P = 0.159).

CONCLUSIONS

The multi-modal model constructed by feature-level fusion showed the best performance. ENDOANGEL-MM identified gastric neoplasms with good accuracy and has a potential role in real-clinic.

Artificial intelligence in pancreatic cancer: diagnosis, limitations, and the future prospects-a narrative review

PURPOSE

This review aims to explore the role of AI in the application of pancreatic cancer management and make recommendations to minimize the impact of the limitations to provide further benefits from AI use in the future.

METHODS

A comprehensive review of the literature was conducted using a combination of MeSH keywords, including "Artificial intelligence", "Pancreatic cancer", "Diagnosis", and "Limitations".

RESULTS

The beneficial implications of AI in the detection of biomarkers, diagnosis, and prognosis of pancreatic cancer have been explored. In addition, current drawbacks of AI use have been divided into subcategories encompassing statistical, training, and knowledge limitations; data handling, ethical and medicolegal aspects; and clinical integration and implementation.

CONCLUSION

Artificial intelligence (AI) refers to computational machine systems that accomplish a set of given tasks by imitating human intelligence in an exponential learning pattern. AI in gastrointestinal oncology has continued to provide significant advancements in the clinical, molecular, and radiological diagnosis and intervention techniques required to improve the prognosis of many gastrointestinal cancer types, particularly pancreatic cancer.

A convolutional neural network-based system for detecting early gastric cancer in white-light endoscopy

BACKGROUND

White-light endoscopy (WLE) is a main and standard modality for detection of early gastric cancer (EGC). The detection rate of EGC is not satisfactory so far. In this single-center retrospective study we developed a convolutional neural network (CNN)-based system to automatically detect EGC in WLE images.

METHODS

An EGC detecting system was constructed based on the CNN architecture EfficientDet. We trained our system with a data set including 4527 images from 130 cases (cancerous images, 1737; noncancerous images, 2790). Then we tested its performance with a data set including 1243 images from 64 cases (cancerous images, 445; noncancerous images, 798).

RESULTS

For case-based analysis, our system successfully detected EGC in 63 of 64 cases and the sensitivity was 98.4%. For image-based analysis, the accuracy was 88.3%. The sensitivity, specificity, positive predictive value and negative predictive value were 84.5%, 90.5%, 83.2% and 91.3%, respectively. The most common cause for false positives was gastritis (57.9%). The most common cause for false negatives was that the lesion was too small with a diameter of 10 mm or less (44.9%).

CONCLUSION

Our CNN-based EGC detecting system was able to achieve satisfactory sensitivity for detecting EGC in WLE images and shows great potential in assisting endoscopists with the detection of EGC.

Diagnosis and segmentation effect of the ME-NBI-based deep learning model on gastric neoplasms in patients with suspected superficial lesions - a multicenter study

Background

Endoscopically visible gastric neoplastic lesions (GNLs), including early gastric cancer and intraepithelial neoplasia, should be accurately diagnosed and promptly treated. However, a high rate of missed diagnosis of GNLs contributes to the potential risk of the progression of gastric cancer. The aim of this study was to develop a deep learning-based computer-aided diagnosis (CAD) system for the diagnosis and segmentation of GNLs under magnifying endoscopy with narrow-band imaging (ME-NBI) in patients with suspected superficial lesions.

Methods

ME-NBI images of patients with GNLs in two centers were retrospectively analysed. Two convolutional neural network (CNN) modules were developed and trained on these images. CNN1 was trained to diagnose GNLs, and CNN2 was trained for segmentation. An additional internal test set and an external test set from another center were used to evaluate the diagnosis and segmentation performance.

Results

CNN1 showed a diagnostic performance with an accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 90.8%, 92.5%, 89.0%, 89.4% and 92.2%, respectively, and an area under the curve (AUC) of 0.928 in the internal test set. With CNN1 assistance, all endoscopists had a higher accuracy than for an independent diagnosis. The average intersection over union (IOU) between CNN2 and the ground truth was 0.5837, with a precision, recall and the Dice coefficient of 0.776, 0.983 and 0.867, respectively.

Conclusions

This CAD system can be used as an auxiliary tool to diagnose and segment GNLs, assisting endoscopists in more accurately diagnosing GNLs and delineating their extent to improve the positive rate of lesion biopsy and ensure the integrity of endoscopic resection.

Deep learning for real-time detection of breast cancer presenting pathological nipple discharge by ductoscopy

Objective

As a common breast cancer-related complaint, pathological nipple discharge (PND) detected by ductoscopy is often missed diagnosed. Deep learning techniques have enabled great advances in clinical imaging but are rarely applied in breast cancer with PND. This study aimed to design and validate an Intelligent Ductoscopy for Breast Cancer Diagnostic System (IDBCS) for breast cancer diagnosis by analyzing real-time imaging data acquired by ductoscopy.

Materials and methods

The present multicenter, case-control trial was carried out in 6 hospitals in China. Images for consecutive patients, aged ≥18 years, with no previous ductoscopy, were obtained from the involved hospitals. All individuals with PND confirmed from breast lesions by ductoscopy were eligible. Images from Beijing Chao-Yang Hospital were randomly assigned (8:2) to the training (IDBCS development) and internal validation (performance evaluation of the IDBCS) datasets. Diagnostic performance was further assessed with internal and prospective validation datasets from Beijing Chao-Yang Hospital; further external validation was carried out with datasets from 5 primary care hospitals. Diagnostic accuracies, sensitivities, specificities, and positive and negative predictive values for IDBCS and endoscopists (expert, competent, or trainee) in the detection of malignant lesions were obtained by the Clopper-Pearson method.

Results

Totally 11305 ductoscopy images in 1072 patients were utilized for developing and testing the IDBCS. Area under the curves (AUCs) in breast cancer detection were 0·975 (95%CI 0·899-0·998) and 0·954 (95%CI 0·925-0·975) in the internal validation and prospective datasets, respectively, and ranged between 0·922 (95%CI 0·866-0·960) and 0·965 (95%CI 0·892-0·994) in the 5 external validation datasets. The IDBCS had superior diagnostic accuracy compared with expert (0.912 [95%CI 0.839-0.959] vs 0.726 [0.672-0.775]; p<0.001), competent (0.699 [95%CI 0.645-0.750], p<0.001), and trainee (0.703 [95%CI 0.648-0.753], p<0.001) endoscopists.

Conclusions

IDBCS outperforms clinical oncologists, achieving high accuracy in diagnosing breast cancer with PND. The novel system could help endoscopists improve their diagnostic efficacy in breast cancer diagnosis.

Deep learning-assisted diagnosis of chronic atrophic gastritis in endoscopy

Background

Chronic atrophic gastritis (CAG) is a precancerous condition. It is not easy to detect CAG in endoscopy. Improving the detection rate of CAG under endoscopy is essential to reduce or interrupt the occurrence of gastric cancer. This study aimed to construct a deep learning (DL) model for CAG recognition based on endoscopic images to improve the CAG detection rate during endoscopy.

Methods

We collected 10,961 endoscopic images and 118 video clips from 4,050 patients. For model training and testing, we divided them into two groups based on the pathological results: CAG and chronic non-atrophic gastritis (CNAG). We compared the performance of four state-of-the-art (SOTA) DL networks for CAG recognition and selected one of them for further improvement. The improved network was called GAM-EfficientNet. Finally, we compared GAM-EfficientNet with three endoscopists and analyzed the decision basis of the network in the form of heatmaps.

Results

After fine-tuning and transfer learning, the sensitivity, specificity, and accuracy of GAM-EfficientNet reached 93%, 94%, and 93.5% in the external test set and 96.23%, 89.23%, and 92.37% in the video test set, respectively, which were higher than those of the three endoscopists.

Conclusions

The CAG recognition model based on deep learning has high sensitivity and accuracy, and its performance is higher than that of endoscopists.

Artificial intelligence in gastroenterology: A narrative review

Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.

A Framework to Predict Gastric Cancer Based on Tongue Features and Deep Learning

Gastric cancer has become a global health issue, severely disrupting daily life. Early detection in gastric cancer patients and immediate treatment contribute significantly to the protection of human health. However, routine gastric cancer examinations carry the risk of complications and are time-consuming. We proposed a framework to predict gastric cancer non-invasively and conveniently. A total of 703 tongue images were acquired using a bespoke tongue image capture instrument, then a dataset containing subjects with and without gastric cancer was created. As the images acquired by this instrument contain non-tongue areas, the Deeplabv3+ network was applied for tongue segmentation to reduce the interference in feature extraction. Nine tongue features were extracted, relationships between tongue features and gastric cancer were explored by using statistical methods and deep learning, finally a prediction framework for gastric cancer was designed. The experimental results showed that the proposed framework had a strong detection ability, with an accuracy of 93.6%. The gastric cancer prediction framework created by combining statistical methods and deep learning proposes a scheme for exploring the relationships between gastric cancer and tongue features. This framework contributes to the effective early diagnosis of patients with gastric cancer.

Effect of adding acetic acid when performing magnifying endoscopy with narrow band imaging for diagnosis of Barrett's esophageal adenocarcinoma

Background and study aims  Magnifying endoscopy with narrow band imaging (M-NBI) was developed to diagnose Barrett's esophageal adenocarcinoma (BEA); however, this method remains challenging for inexperienced endoscopists. We aimed to evaluate a modified M-NBI technique that included spraying acetic acid (M-AANBI). Patients and methods  Eight endoscopists retrospectively examined 456 endoscopic images obtained from 28 patients with 29 endoscopically resected BEA lesions using three validation schemes: Validation 1 (260 images), wherein the diagnostic performances of M-NBI and M-AANBI were compared - the dataset included 65 images each of BEA and non-neoplastic Barrett's esophagus (NNBE) obtained using each modality; validation 2 (112 images), wherein 56 pairs of M-NBI and M-AANBI images were prepared from the same BEA and NNBE lesions, and diagnoses derived using M-NBI alone were compared to those obtained using both M-NBI and M-AANBI; and validation 3 (84 images), wherein the ease of identifying the BEA demarcation line (DL) was scored via a visual analog scale in 28 patients using magnifying endoscopy with white-light imaging (M-WLI), M-NBI, and M-AANBI. Results  For validation 1, M-AANBI was superior to M-NBI in terms of sensitivity (90.8 % vs. 64.6 %), specificity (98.5 % vs. 76.9 %), and accuracy (94.6 % vs. 70.4 %) (all P  < 0.05). For validation 2, the accuracy of M-NBI alone was significantly improved when combined with M-AANBI (from 70.5 % to 89.3 %; P  < 0.05). For validation 3, M-AANBI had the highest mean score for ease of DL recognition (8.75) compared to M-WLI (3.63) and M-NBI (6.25) (all P <  0.001). Conclusions  Using M-AANBI might improve the accuracy of BEA diagnosis.

Current Status of Artificial Intelligence-Based Computer-Assisted Diagnosis Systems for Gastric Cancer in Endoscopy

Artificial intelligence (AI) is gradually being utilized in various fields as its performance has been improving with the development of deep learning methods, availability of big data, and the progression of computer processing units. In the field of medicine, AI is mainly implemented in image recognition, such as in radiographic and pathologic diagnoses. In the realm of gastrointestinal endoscopy, although AI-based computer-assisted detection/diagnosis (CAD) systems have been applied in some areas, such as colorectal polyp detection and diagnosis, so far, their implementation in real-world clinical settings is limited. The accurate detection or diagnosis of gastric cancer (GC) is one of the challenges in which performance varies greatly depending on the endoscopist's skill. The diagnosis of early GC is especially challenging, partly because early GC mimics atrophic gastritis in the background mucosa. Therefore, several CAD systems for GC are being actively developed. The development of a CAD system for GC is considered challenging because it requires a large number of GC images. In particular, early stage GC images are rarely available, partly because it is difficult to diagnose gastric cancer during the early stages. Additionally, the training image data should be of a sufficiently high quality to conduct proper CAD training. Recently, several AI systems for GC that exhibit a robust performance, owing to being trained on a large number of high-quality images, have been reported. This review outlines the current status and prospects of AI use in esophagogastroduodenoscopy (EGDS), focusing on the diagnosis of GC.

Application of artificial intelligence in the endoscopic diagnosis of early gastric cancer, atrophic gastritis, and Helicobacter pylori infection

Gastric cancer (GC) is one of the most serious health problems worldwide. Chronic atrophic gastritis (CAG) is most commonly caused by Helicobacter pylori (H. pylori) infection. Currently, endoscopic detection of early gastric cancer (EGC) and CAG remains challenging for endoscopists, and the diagnostic accuracy of H. pylori infection by endoscopy is approximately 70%. Artificial intelligence (AI) can assist endoscopic diagnosis including detection, prediction of depth of invasion, boundary delineation, and anatomical location of EGC, and has achievable diagnostic ability even comparable to experienced endoscopists. In this review we summarized various AI-assisted systems in the diagnosis of EGC, CAG, and H. pylori infection.

Accurate preoperative staging and HER2 status prediction of gastric cancer by the deep learning system based on enhanced computed tomography

PURPOSE

To construct the deep learning system (DLS) based on enhanced computed tomography (CT) images for preoperative prediction of staging and human epidermal growth factor receptor 2 (HER2) status in gastric cancer patients.

METHODS

The raw enhanced CT image dataset consisted of CT images of 389 patients in the retrospective cohort, The Cancer Imaging Archive (TCIA) cohort, and the prospective cohort. DLS was developed by transfer learning for tumor detection, staging, and HER2 status prediction. The pre-trained Yolov5, EfficientNet, EfficientNetV2, Vision Transformer (VIT), and Swin Transformer (SWT) were studied. The tumor detection and staging dataset consisted of 4860 enhanced CT images and annotated tumor bounding boxes. The HER2 state prediction dataset consisted of 38900 enhanced CT images.

RESULTS

The DetectionNet based on Yolov5 realized tumor detection and staging and achieved a mean Average Precision (IoU=0.5) (mAP_0.5) of 0.909 in the external validation cohort. The VIT-based PredictionNet performed optimally in HER2 status prediction with the area under the receiver operating characteristics curve (AUC) of 0.9721 and 0.9995 in the TCIA cohort and prospective cohort, respectively. DLS included DetectionNet and PredictionNet had shown excellent performance in CT image interpretation.

CONCLUSION

This study developed the enhanced CT-based DLS to preoperatively predict the stage and HER2 status of gastric cancer patients, which will help in choosing the appropriate treatment to improve the survival of gastric cancer patients.

The investigation of construction and clinical application of image recognition technology assisted bronchoscopy diagnostic model of lung cancer

Background

The incidence and mortality of lung cancer ranks first in China. Bronchoscopy is one of the most common diagnostic methods for lung cancer. In recent years, image recognition technology(IRT) has been more and more widely studied and applied in the medical field. We developed a diagnostic model of lung cancer under bronchoscopy based on deep learning method and tried to classify pathological types.

Methods

A total of 2238 lesion images were collected retrospectively from 666 cases of lung cancer diagnosed by pathology in the bronchoscopy center of the Third Xiangya Hospital from Oct.01 2017 to Dec.31 2020 and 152 benign cases from Jun.01 2015 to Dec.31 2020. The benign and malignant images were divided into training, verification and test set according to 7:1:2 respectively. The model was trained and tested based on deep learning method. We also tried to classify different pathological types of lung cancer using the model. Furthermore, 9 clinicians with different experience were invited to diagnose the same test images and the results were compared with the model.

Results

The diagnostic model took a total of 30s to diagnose 467 test images. The overall accuracy, sensitivity, specificity and area under curve (AUC) of the model to differentiate benign and malignant lesions were 0.951, 0.978, 0.833 and 0.940, which were equivalent to the judgment results of 2 doctors in the senior group and higher than those of other doctors. In the classification of squamous cell carcinoma (SCC) and adenocarcinoma (AC), the overall accuracy was 0.745, including 0.790 for SCC, 0.667 for AC and AUC was 0.728.

Conclusion

The performance of our diagnostic model to distinguish benign and malignant lesions in bronchoscopy is roughly the same as that of experienced clinicians and the efficiency is much higher than manually. Our study verifies the possibility of applying IRT in diagnosis of lung cancer during white light bronchoscopy.

Automatic detection of early gastric cancer in endoscopy based on Mask region-based convolutional neural networks (Mask R-CNN)(with video)

The artificial intelligence (AI)-assisted endoscopic detection of early gastric cancer (EGC) has been preliminarily developed. The currently used algorithms still exhibit limitations of large calculation and low-precision expression. The present study aimed to develop an endoscopic automatic detection system in EGC based on a mask region-based convolutional neural network (Mask R-CNN) and to evaluate the performance in controlled trials. For this purpose, a total of 4,471 white light images (WLIs) and 2,662 narrow band images (NBIs) of EGC were obtained for training and testing. In total, 10 of the WLIs (videos) were obtained prospectively to examine the performance of the RCNN system. Furthermore, 400 WLIs were randomly selected for comparison between the Mask R-CNN system and doctors. The evaluation criteria included accuracy, sensitivity, specificity, positive predictive value and negative predictive value. The results revealed that there were no significant differences between the pathological diagnosis with the Mask R-CNN system in the WLI test (χ² = 0.189, P=0.664; accuracy, 90.25%; sensitivity, 91.06%; specificity, 89.01%) and in the NBI test (χ² = 0.063, P=0.802; accuracy, 95.12%; sensitivity, 97.59%). Among 10 WLI real-time videos, the speed of the test videos was up to 35 frames/sec, with an accuracy of 90.27%. In a controlled experiment of 400 WLIs, the sensitivity of the Mask R-CNN system was significantly higher than that of experts (χ² = 7.059, P=0.000; 93.00% VS 80.20%), and the specificity was higher than that of the juniors (χ² = 9.955, P=0.000, 82.67% VS 71.87%), and the overall accuracy rate was higher than that of the seniors (χ² = 7.009, P=0.000, 85.25% VS 78.00%). On the whole, the present study demonstrates that the Mask R-CNN system exhibited an excellent performance status for the detection of EGC, particularly for the real-time analysis of WLIs. It may thus be effectively applied to clinical settings.

Application of deep learning in the real-time diagnosis of gastric lesion based on magnifying optical enhancement videos

Background and aim

Magnifying image-enhanced endoscopy was demonstrated to have higher diagnostic accuracy than white-light endoscopy. However, differentiating early gastric cancers (EGCs) from benign lesions is difficult for beginners. We aimed to determine whether the computer-aided model for the diagnosis of gastric lesions can be applied to videos rather than still images.

Methods

A total of 719 magnifying optical enhancement images of EGCs, 1,490 optical enhancement images of the benign gastric lesions, and 1,514 images of background mucosa were retrospectively collected to train and develop a computer-aided diagnostic model. Subsequently, 101 video segments and 671 independent images were used for validation, and error frames were labeled to retrain the model. Finally, a total of 117 unaltered full-length videos were utilized to test the model and compared with those diagnostic results made by independent endoscopists.

Results

Except for atrophy combined with intestinal metaplasia (IM) and low-grade neoplasia, the diagnostic accuracy was 0.90 (85/94). The sensitivity, specificity, PLR, NLR, and overall accuracy of the model to distinguish EGC from non-cancerous lesions were 0.91 (48/53), 0.78 (50/64), 4.14, 0.12, and 0.84 (98/117), respectively. No significant difference was observed in the overall diagnostic accuracy between the computer-aided model and experts. A good level of kappa values was found between the model and experts, which meant that the kappa value was 0.63.

Conclusions

The performance of the computer-aided model for the diagnosis of EGC is comparable to that of experts. Magnifying the optical enhancement model alone may not be able to deal with all lesions in the stomach, especially when near the focus on severe atrophy with IM. These results warrant further validation in prospective studies with more patients. A ClinicalTrials.gov registration was obtained (identifier number: NCT04563416).

Clinical Trial Registration

ClinicalTrials.gov, identifier NCT04563416.

Convolutional neural network-based artificial intelligence for the diagnosis of early esophageal cancer based on endoscopic images: A meta-analysis

BACKGROUND

Early screening and treatment of esophageal cancer (EC) is particularly important for the survival and prognosis of patients. However, early EC is difficult to diagnose by a routine endoscopic examination. Therefore, convolutional neural network (CNN)-based artificial intelligence (AI) has become a very promising method in the diagnosis of early EC using endoscopic images. The aim of this study was to evaluate the diagnostic performance of CNN-based AI for detecting early EC based on endoscopic images.

METHODS

A comprehensive search was performed to identify relevant English articles concerning CNN-based AI in the diagnosis of early EC based on endoscopic images (from the date of database establishment to April 2022). The pooled sensitivity (SEN), pooled specificity (SPE), positive likelihood ratio (LR+), negative likelihood ratio (LR-), diagnostic odds ratio (DOR) with 95% confidence interval (CI), summary receiver operating characteristic (SROC) curve, and area under the curve (AUC) for the accuracy of CNN-based AI in the diagnosis of early EC based on endoscopic images were calculated. We used the I² test to assess heterogeneity and investigated the source of heterogeneity by performing meta-regression analysis. Publication bias was assessed using Deeks' funnel plot asymmetry test.

RESULTS

Seven studies met the eligibility criteria. The SEN and SPE were 0.90 (95% confidence interval [CI]: 0.82-0.94) and 0.91 (95% CI: 0.79-0.96), respectively. The LR+ of the malignant ultrasonic features was 9.8 (95% CI: 3.8-24.8) and the LR- was 0.11 (95% CI: 0.06-0.21), revealing that CNN-based AI exhibited an excellent ability to confirm or exclude early EC on endoscopic images. Additionally, SROC curves showed that the AUC of the CNN-based AI in the diagnosis of early EC based on endoscopic images was 0.95 (95% CI: 0.93-0.97), demonstrating that CNN-based AI has good diagnostic value for early EC based on endoscopic images.

CONCLUSIONS

Based on our meta-analysis, CNN-based AI is an excellent diagnostic tool with high sensitivity, specificity, and AUC in the diagnosis of early EC based on endoscopic images.

A Non-Aligned Deep Representation to Enhance Standard Colonoscopy Observations from Vascular Narrow Band Polyp Patterns

Colorectal cancer (CRC) was responsible during 2020 for about one million deaths worldwide. Polyps are protuberance masses, observed in routine colonoscopies, that constitute the main CRC biomarker. Nonetheless, one of the best alternatives to the polyp malignancy classification is the vascular pattern analysis, typically observed from specialized narrow-band images (NBI). Even worst, these patterns are only characterized from gastroenterologist observations, introducing subjectivity and being prone to diagnostic errors, with misclassi-fications ranging from 59.5 % to 84.2 %. This work introduces a non-aligned and bi-directional deep projection between optical colonoscopy (OC) and NBI sequences, to recover enhanced OC sequences, integrating vascular patterns, that allow better dis-crimination among adenomas, hyperplastic and serrated polyps. This self-supervised representation help with misclassification in standard OC observations. The validation was performed on a total of 76 OC and 76 NBI sequences, achieving a gain of 22.34% w.r.t descriptors computed from raw OC. Clinical relevance- A deep representation that enhances standard OC observations associating vascularity to the polyps to discriminate among adenomas hyperplastic and serrated polyps.

Artificial Intelligence-Assisted Endoscopic Diagnosis of Early Upper Gastrointestinal Cancer: A Systematic Review and Meta-Analysis

Objective

The aim of this study was to assess the diagnostic ability of artificial intelligence (AI) in the detection of early upper gastrointestinal cancer (EUGIC) using endoscopic images.

Methods

Databases were searched for studies on AI-assisted diagnosis of EUGIC using endoscopic images. The pooled area under the curve (AUC), sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with 95% confidence interval (CI) were calculated.

Results

Overall, 34 studies were included in our final analysis. Among the 17 image-based studies investigating early esophageal cancer (EEC) detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.98, 0.95 (95% CI, 0.95–0.96), 0.95 (95% CI, 0.94–0.95), 10.76 (95% CI, 7.33–15.79), 0.07 (95% CI, 0.04–0.11), and 173.93 (95% CI, 81.79–369.83), respectively. Among the seven patient-based studies investigating EEC detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.98, 0.94 (95% CI, 0.91–0.96), 0.90 (95% CI, 0.88–0.92), 6.14 (95% CI, 2.06–18.30), 0.07 (95% CI, 0.04–0.11), and 69.13 (95% CI, 14.73–324.45), respectively. Among the 15 image-based studies investigating early gastric cancer (EGC) detection, the pooled AUC, sensitivity, specificity, PLR, NLR, and DOR were 0.94, 0.87 (95% CI, 0.87–0.88), 0.88 (95% CI, 0.87–0.88), 7.20 (95% CI, 4.32–12.00), 0.14 (95% CI, 0.09–0.23), and 48.77 (95% CI, 24.98–95.19), respectively.

Conclusions

On the basis of our meta-analysis, AI exhibited high accuracy in diagnosis of EUGIC.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO/, identifier PROSPERO (CRD42021270443).

A deep learning-based model improves diagnosis of early gastric cancer under narrow band imaging endoscopy

BACKGROUND

Diagnosis of early gastric cancer (EGC) under narrow band imaging endoscopy (NBI) is dependent on expertise and skills. We aimed to elucidate whether artificial intelligence (AI) could diagnose EGC under NBI and evaluate the diagnostic assistance of the AI system.

METHODS

In this retrospective diagnostic study, 21,785 NBI images and 20 videos from five centers were divided into a training dataset (13,151 images, 810 patients), an internal validation dataset (7057 images, 283 patients), four external validation datasets (1577 images, 147 patients), and a video validation dataset (20 videos, 20 patients). All the images were labeled manually and used to train an AI system using You look only once v3 (YOLOv3). Next, the diagnostic performance of the AI system and endoscopists were compared and the diagnostic assistance of the AI system was assessed. The accuracy, sensitivity, specificity, and AUC were primary outcomes.

RESULTS

The AI system diagnosed EGCs on validation datasets with AUCs of 0.888-0.951 and diagnosed all the EGCs (100.0%) in video dataset. The AI system achieved better diagnostic performance (accuracy, 93.2%, 95% CI, 90.0-94.9%) than senior (85.9%, 95% CI, 84.2-87.4%) and junior (79.5%, 95% CI, 77.8-81.0%) endoscopists. The AI system significantly enhanced the performance of endoscopists in senior (89.4%, 95% CI, 87.9-90.7%) and junior (84.9%, 95% CI, 83.4-86.3%) endoscopists.

CONCLUSION

The NBI AI system outperformed the endoscopists and exerted potential assistant impact in EGC identification. Prospective validations are needed to evaluate the clinical reinforce of the system in real clinical practice.

Hybrid and Deep Learning Approach for Early Diagnosis of Lower Gastrointestinal Diseases

Every year, nearly two million people die as a result of gastrointestinal (GI) disorders. Lower gastrointestinal tract tumors are one of the leading causes of death worldwide. Thus, early detection of the type of tumor is of great importance in the survival of patients. Additionally, removing benign tumors in their early stages has more risks than benefits. Video endoscopy technology is essential for imaging the GI tract and identifying disorders such as bleeding, ulcers, polyps, and malignant tumors. Videography generates 5000 frames, which require extensive analysis and take a long time to follow all frames. Thus, artificial intelligence techniques, which have a higher ability to diagnose and assist physicians in making accurate diagnostic decisions, solve these challenges. In this study, many multi-methodologies were developed, where the work was divided into four proposed systems; each system has more than one diagnostic method. The first proposed system utilizes artificial neural networks (ANN) and feed-forward neural networks (FFNN) algorithms based on extracting hybrid features by three algorithms: local binary pattern (LBP), gray level co-occurrence matrix (GLCM), and fuzzy color histogram (FCH) algorithms. The second proposed system uses pre-trained CNN models which are the GoogLeNet and AlexNet based on the extraction of deep feature maps and their classification with high accuracy. The third proposed method uses hybrid techniques consisting of two blocks: the first block of CNN models (GoogLeNet and AlexNet) to extract feature maps; the second block is the support vector machine (SVM) algorithm for classifying deep feature maps. The fourth proposed system uses ANN and FFNN based on the hybrid features between CNN models (GoogLeNet and AlexNet) and LBP, GLCM and FCH algorithms. All the proposed systems achieved superior results in diagnosing endoscopic images for the early detection of lower gastrointestinal diseases. All systems produced promising results; the FFNN classifier based on the hybrid features extracted by GoogLeNet, LBP, GLCM and FCH achieved an accuracy of 99.3%, precision of 99.2%, sensitivity of 99%, specificity of 100%, and AUC of 99.87%.

Development and evaluation of a deep learning model to improve the usability of polyp detection systems during interventions

Background

The efficiency of artificial intelligence as computer‐aided detection (CADe) systems for colorectal polyps has been demonstrated in several randomized trials. However, CADe systems generate many distracting detections, especially during interventions such as polypectomies. Those distracting CADe detections are often induced by the introduction of snares or biopsy forceps as the systems have not been trained for such situations. In addition, there are a significant number of non‐false but not relevant detections, since the polyp has already been previously detected. All these detections have the potential to disturb the examiner's work.

Objectives

Development and evaluation of a convolutional neuronal network that recognizes instruments in the endoscopic image, suppresses distracting CADe detections, and reliably detects endoscopic interventions.

Methods

A total of 580 different examination videos from 9 different centers using 4 different processor types were screened for instruments and represented the training dataset (519,856 images in total, 144,217 contained a visible instrument). The test dataset included 10 full‐colonoscopy videos that were analyzed for the recognition of visible instruments and detections by a commercially available CADe system (GI Genius, Medtronic).

Results

The test dataset contained 153,623 images, 8.84% of those presented visible instruments (12 interventions, 19 instruments used). The convolutional neuronal network reached an overall accuracy in the detection of visible instruments of 98.59%. Sensitivity and specificity were 98.55% and 98.92%, respectively. A mean of 462.8 frames containing distracting CADe detections per colonoscopy were avoided using the convolutional neuronal network. This accounted for 95.6% of all distracting CADe detections.

Conclusions

Detection of endoscopic instruments in colonoscopy using artificial intelligence technology is reliable and achieves high sensitivity and specificity. Accordingly, the new convolutional neuronal network could be used to reduce distracting CADe detections during endoscopic procedures. Thus, our study demonstrates the great potential of artificial intelligence technology beyond mucosal assessment.

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Usefulness of artificial intelligence in early gastric cancer

Gastric cancer (GC) is a major cancer worldwide, with high mortality and morbidity. Endoscopy, important for the early detection of GC, requires trained skills, high-quality technologies, surveillance and screening programs. Early diagnosis allows a better prognosis, through surgical or curative endoscopic therapy. Magnified endoscopy with virtual chromoendoscopy remarkably improve the detection of early gastric cancer (EGC) when endoscopy is performed by expert endoscopists. Artificial intelligence (AI) has also been introduced to GC diagnostics to increase diagnostic efficiency. AI improves the early detection of gastric lesions because it supports the non-expert and experienced endoscopist in defining the margins of the tumor and the depth of infiltration. AI increases the detection rate of EGC, reduces the rate of missing tumors, and characterizes EGCs, allowing clinicians to make the best therapeutic decision, that is, one that ensures curability. AI has had a remarkable evolution in medicine in recent years, moving from the research phase to clinical practice. In addition, the diagnosis of GC has markedly progressed. We predict that AI will allow great evolution in the diagnosis and treatment of EGC by overcoming the variability in performance that is currently a limitation of chromoendoscopy.

Multi-label recognition of cancer-related lesions with clinical priors on white-light endoscopy

Deep learning-based computer-aided diagnosis techniques have demonstrated encouraging performance in endoscopic lesion identification and detection, and have reduced the rate of missed and false detections of disease during endoscopy. However, the interpretability of the model-based results has not been adequately addressed by existing methods. This phenomenon is directly manifested by a significant bias in the representation of feature localization. Good recognition models experience severe feature localization errors, particularly for lesions with subtle morphological features, and such unsatisfactory performance hinders the clinical deployment of models. To effectively alleviate this problem, we proposed a solution to optimize the localization bias in feature representations of cancer-related recognition models that is difficult to accurately label and identify in clinical practice. Optimization was performed in the training phase of the model through the proposed data augmentation method and auxiliary loss function based on clinical priors. The data augmentation method, called partial jigsaw, can "break" the spatial structure of lesion-independent image blocks and enrich the data feature space to decouple the interference of background features on the space and focus on fine-grained lesion features. The annotation-based auxiliary loss function used class activation maps for sample distribution correction and led the model to present localization representation converging on the gold standard annotation of visualization maps. The results show that with the improvement of our method, the precision of model recognition reached an average of 92.79%, an F1-score of 92.61%, and accuracy of 95.56% based on a dataset constructed from 23 hospitals. In addition, we quantified the evaluation representation of visualization feature maps. The improved model yielded significant offset correction results for visualized feature maps compared with the baseline model. The average visualization-weighted positive coverage improved from 51.85% to 83.76%. The proposed approach did not change the deployment capability and inference speed of the original model and can be incorporated into any state-of-the-art neural network. It also shows the potential to provide more accurate localization inference results and assist in clinical examinations during endoscopies.

Artificial Intelligence and Deep Learning for Upper Gastrointestinal Neoplasia

Upper gastrointestinal (GI) neoplasia account for 35% of GI cancers and 1.5 million cancer-related deaths every year. Despite its efficacy in preventing cancer mortality, diagnostic upper GI endoscopy is affected by a substantial miss rate of neoplastic lesions due to failure to recognize a visible lesion or imperfect navigation. This may be offset by the real-time application of artificial intelligence (AI) for detection (computer-aided detection [CADe]) and characterization (computer-aided diagnosis [CADx]) of upper GI neoplasia. Stand-alone performance of CADe for esophageal squamous cell neoplasia, Barrett's esophagus-related neoplasia, and gastric cancer showed promising accuracy, sensitivity ranging between 83% and 93%. However, incorporation of CADe/CADx in clinical practice depends on several factors, such as possible bias in the training or validation phases of these algorithms, its interaction with human endoscopists, and clinical implications of false-positive results. The aim of this review is to guide the clinician across the multiple steps of AI development in clinical practice.