Artificial intelligence in endoscopy: Present and future perspectives

Application value of an artificial intelligence-based diagnosis and recognition system in gastroscopy training for graduate students in gastroenterology: a preliminary study

OBJECTIVE

This study aimed to discuss the application value of an artificial intelligence-based diagnosis and recognition system (AIDRS) in the teaching activities for Bachelor of Medicine and Bachelor of Surgery (MBBS) in China. The learning performance of graduate students in gastroenterology during gastroscopy training with and without AIDRS was assessed.

METHODS

The study recruited 32 graduate students of the gastroenterology program at Jiangsu province hospital of Chinese medicine and Xiangyang No. 1 People's Hospital from March 2018 to March 2022 and randomly divided them into AIDRS (n = 16) and non-AIDRS (n = 16) groups. The AIDRS software was used for real-time monitoring of blind spots of gastroscopy to aid in lesion diagnosis and recognition in the AIDRS group. Only a conventional gastroscopic procedure was implemented in the non-AIDRS group. The final performance score, success rate of gastroscopy, lesion detection rate, and pain score of patients were compared between the two groups during gastroscopy. A self-prepared teaching and learning satisfaction questionnaire was administered to the two groups of students.

RESULTS

The AIDRS group had a higher final performance score (92.60 ± 2.83 vs. 89.21 ± 3.57, t = 2.98, P < 0.05), a higher success rate of gastroscopy (448/480 vs. 417/480, χ2 = 11.23, P < 0.05), and a higher detection rate of lesions (51/52 vs. 41/53, χ2 = 8.56, P < 0.05) compared with the non-AIDRS group. The pain scores of patients were lower in the AIDRS group than in the non-AIDRS group (3.40 [2.23, 3.98] vs. 4.45 [3.72, 4.75], Z = 3.04, P < 0.05). Besides, the average time for gastroscopy was lower in the AIDRS group than in the non-AIDRS group (7.15 ± 1.24 vs. 8.21 ± 1.26, t = 2.38, P = 0.02). The overall satisfaction level with the teaching program was higher in the AIDRS group (43.51 ± 2.29 vs. 40.93 ± 2.07, t = 3.33, P < 0.05).

CONCLUSION

In the context of medicine-education cooperation, AIDRS offered valuable assistance in gastroscopy training and increased the success rate of gastroscopy and teaching and learning satisfaction. AIDRS is worthy of wider-scale promotion.

The Detection of Nasopharyngeal Carcinomas Using a Neural Network Based on Nasopharyngoscopic Images

OBJECTIVE

To construct and validate a deep convolutional neural network (DCNN)-based artificial intelligence (AI) system for the detection of nasopharyngeal carcinoma (NPC) using archived nasopharyngoscopic images.

METHODS

We retrospectively collected 14107 nasopharyngoscopic images (7108 NPCs and 6999 noncancers) to construct a DCNN model and prepared a validation dataset containing 3501 images (1744 NPCs and 1757 noncancers) from a single center between January 2009 and December 2020. The DCNN model was established using the You Only Look Once (YOLOv5) architecture. Four otolaryngologists were asked to review the images of the validation set to benchmark the DCNN model performance.

RESULTS

The DCNN model analyzed the 3501 images in 69.35 s. For the validation dataset, the precision, recall, accuracy, and F1 score of the DCNN model in the detection of NPCs on white light imaging (WLI) and narrow band imaging (NBI) were 0.845 ± 0.038, 0.942 ± 0.021, 0.920 ± 0.024, and 0.890 ± 0.045, and 0.895 ± 0.045, 0.941 ± 0.018, and 0.975 ± 0.013, 0.918 ± 0.036, respectively. The diagnostic outcome of the DCNN model on WLI and NBI images was significantly higher than that of two junior otolaryngologists (p < 0.05).

CONCLUSION

The DCNN model showed better diagnostic outcomes for NPCs than those of junior otolaryngologists. Therefore, it could assist them in improving their diagnostic level and reducing missed diagnoses.

LEVEL OF EVIDENCE

3 Laryngoscope, 134:127-135, 2024.

Deep learning for real-time detection of nasopharyngeal carcinoma during nasopharyngeal endoscopy

Nasopharyngeal carcinoma (NPC) is known for high curability during early stage of the disease, and early diagnosis relies on nasopharyngeal endoscopy and subsequent pathological biopsy. To enhance the early diagnosis rate by aiding physicians in the real-time identification of NPC and directing biopsy site selection during endoscopy, we assembled a dataset comprising 2,429 nasopharyngeal endoscopy video frames from 690 patients across three medical centers. With these data, we developed a deep learning-based NPC detection model using the you only look once (YOLO) network. Our model demonstrated high performance, with precision, recall, mean average precision, and F1-score values of 0.977, 0.943, 0.977, and 0.960, respectively, for internal test set and 0.825, 0.743, 0.814, and 0.780 for external test set at 0.5 intersection over union. Remarkably, our model demonstrated a high inference speed (52.9 FPS), surpassing the average frame rate (25.0 FPS) of endoscopy videos, thus making real-time detection in endoscopy feasible.

The Advent of Domain Adaptation into Artificial Intelligence for Gastrointestinal Endoscopy and Medical Imaging

Artificial intelligence (AI) is a subfield of computer science that aims to implement computer systems that perform tasks that generally require human learning, reasoning, and perceptual abilities. AI is widely used in the medical field. The interpretation of medical images requires considerable effort, time, and skill. AI-aided interpretations, such as automated abnormal lesion detection and image classification, are promising areas of AI. However, when images with different characteristics are extracted, depending on the manufacturer and imaging environment, a so-called domain shift problem occurs in which the developed AI has a poor versatility. Domain adaptation is used to address this problem. Domain adaptation is a tool that generates a newly converted image which is suitable for other domains. It has also shown promise in reducing the differences in appearance among the images collected from different devices. Domain adaptation is expected to improve the reading accuracy of AI for heterogeneous image distributions in gastrointestinal (GI) endoscopy and medical image analyses. In this paper, we review the history and basic characteristics of domain shift and domain adaptation. We also address their use in gastrointestinal endoscopy and the medical field more generally through published examples, perspectives, and future directions.

Artificial intelligence and inflammatory bowel disease: Where are we going?

Inflammatory bowel diseases, namely ulcerative colitis and Crohn’s disease, are chronic and relapsing conditions that pose a growing burden on healthcare systems worldwide. Because of their complex and partly unknown etiology and pathogenesis, the management of ulcerative colitis and Crohn’s disease can prove challenging not only from a clinical point of view but also for resource optimization. Artificial intelligence, an umbrella term that encompasses any cognitive function developed by machines for learning or problem solving, and its subsets machine learning and deep learning are becoming ever more essential tools with a plethora of applications in most medical specialties. In this regard gastroenterology is no exception, and due to the importance of endoscopy and imaging numerous clinical studies have been gradually highlighting the relevant role that artificial intelligence has in inflammatory bowel diseases as well. The aim of this review was to summarize the most recent evidence on the use of artificial intelligence in inflammatory bowel diseases in various contexts such as diagnosis, follow-up, treatment, prognosis, cancer surveillance, data collection, and analysis. Moreover, insights into the potential further developments in this field and their effects on future clinical practice were discussed.

Development and Validation of Deep Learning Models for the Multiclassification of Reflux Esophagitis Based on the Los Angeles Classification

This study is to evaluate the feasibility of deep learning (DL) models in the multiclassification of reflux esophagitis (RE) endoscopic images, according to the Los Angeles (LA) classification for the first time. The images were divided into three groups, namely, normal, LA classification A + B, and LA C + D. The images from the HyperKvasir dataset and Suzhou hospital were divided into the training and validation datasets as a ratio of 4 : 1, while the images from Jintan hospital were the independent test set. The CNNs- or Transformer-architectures models (MobileNet, ResNet, Xception, EfficientNet, ViT, and ConvMixer) were transfer learning via Keras. The visualization of the models was proposed using Gradient-weighted Class Activation Mapping (Grad-CAM). Both in the validation set and the test set, the EfficientNet model showed the best performance as follows: accuracy (0.962 and 0.957), recall for LA A + B (0.970 and 0.925) and LA C + D (0.922 and 0.930), Marco-recall (0.946 and 0.928), Matthew's correlation coefficient (0.936 and 0.884), and Cohen's kappa (0.910 and 0.850), which was better than the other models and the endoscopists. According to the EfficientNet model, the Grad-CAM was plotted and highlighted the target lesions on the original images. This study developed a series of DL-based computer vision models with the interpretable Grad-CAM to evaluate the feasibility in the multiclassification of RE endoscopic images. It firstly suggests that DL-based classifiers show promise in the endoscopic diagnosis of esophagitis.

Evaluation of the competence of an artificial intelligence-assisted colonoscopy system in clinical practice: A post hoc analysis

Background

Artificial intelligence-assisted colonoscopy (AIAC) has been proposed and validated in recent years, but the effectiveness of clinic application remains unclear since it was only validated in some clinical trials rather than normal conditions. In addition, previous clinical trials were mostly concerned with colorectal polyp identification, while fewer studies are focusing on adenoma identification and polyps size measurement. In this study, we validated the effectiveness of AIAC in the clinical environment and further investigated its capacity for adenoma identification and polyps size measurement.

Methods

The information of 174 continued patients who went for coloscopy in Chongqing Rongchang District People's hospital with detected colon polyps was retrospectively collected, and their coloscopy images were divided into three validation datasets, polyps dataset, polyps/adenomas dataset (all containing narrow band image, NBI images), and polyp size measurement dataset (images with biopsy forceps and polyps) to assess the competence of the artificial intelligence system, and compare its diagnostic ability with endoscopists with different experiences.

Results

A total of 174 patients were included, and the sensitivity of the colorectal polyp recognition model was 99.40%, the accuracy of the colorectal adenoma diagnostic model was 93.06%, which was higher than that of endoscopists, and the mean absolute error of the polyp size measurement model was 0.62 mm and the mean relative error was 10.89%, which was lower than that of endoscopists.

Conclusion

Artificial intelligence-assisted model demonstrated higher competence compared with endoscopists and stable diagnosis ability in clinical use.

Where do we stand in AI for endoscopic image analysis? Deciphering gaps and future directions

Recent developments in deep learning have enabled data-driven algorithms that can reach human-level performance and beyond. The development and deployment of medical image analysis methods have several challenges, including data heterogeneity due to population diversity and different device manufacturers. In addition, more input from experts is required for a reliable method development process. While the exponential growth in clinical imaging data has enabled deep learning to flourish, data heterogeneity, multi-modality, and rare or inconspicuous disease cases still need to be explored. Endoscopy being highly operator-dependent with grim clinical outcomes in some disease cases, reliable and accurate automated system guidance can improve patient care. Most designed methods must be more generalisable to the unseen target data, patient population variability, and variable disease appearances. The paper reviews recent works on endoscopic image analysis with artificial intelligence (AI) and emphasises the current unmatched needs in this field. Finally, it outlines the future directions for clinically relevant complex AI solutions to improve patient outcomes.

Artificial Endoscopy and Inflammatory Bowel Disease: Welcome to the Future

Artificial intelligence (AI) is assuming an increasingly important and central role in several medical fields. Its application in endoscopy provides a powerful tool supporting human experiences in the detection, characterization, and classification of gastrointestinal lesions. Lately, the potential of AI technology has been emerging in the field of inflammatory bowel disease (IBD), where the current cornerstone is the treat-to-target strategy. A sensible and specific tool able to overcome human limitations, such as AI, could represent a great ally and guide precision medicine decisions. Here we reviewed the available literature on the endoscopic applications of AI in order to properly describe the current state-of-the-art and identify the research gaps in IBD at the dawn of 2022.

Detection Accuracy and Latency of Colorectal Lesions with Computer-Aided Detection System Based on Low-Bias Evaluation

We developed a computer-aided detection (CADe) system to detect and localize colorectal lesions by modifying You-Only-Look-Once version 3 (YOLO v3) and evaluated its performance in two different settings. The test dataset was obtained from 20 randomly selected patients who underwent endoscopic resection for 69 colorectal lesions at the Jikei University Hospital between June 2017 and February 2018. First, we evaluated the diagnostic performances using still images randomly and automatically extracted from video recordings of the entire endoscopic procedure at intervals of 5 s, without eliminating poor quality images. Second, the latency of lesion detection by the CADe system from the initial appearance of lesions was investigated by reviewing the videos. A total of 6531 images, including 662 images with a lesion, were studied in the image-based analysis. The AUC, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 0.983, 94.6%, 95.2%, 68.8%, 99.4%, and 95.1%, respectively. The median time for detecting colorectal lesions measured in the lesion-based analysis was 0.67 s. In conclusion, we proved that the originally developed CADe system based on YOLO v3 could accurately and instantaneously detect colorectal lesions using the test dataset obtained from videos, mitigating operator selection biases.

μRALP and Beyond: Micro-Technologies and Systems for Robot-Assisted Endoscopic Laser Microsurgery

Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project μRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond μRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated μRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.

Automatic colonic polyp detection using integration of modified deep residual convolutional neural network and ensemble learning approaches

BACKGROUND AND OBJECTIVE

The increased incidence of colorectal cancer (CRC) and its mortality rate have attracted interest in the use of artificial intelligence (AI) based computer-aided diagnosis (CAD) tools to detect polyps at an early stage. Although these CAD tools have thus far achieved a good accuracy level to detect polyps, they still have room to improve further (e.g. sensitivity). Therefore, a new CAD tool is developed in this study to detect colonic polyps accurately.

METHODS

In this paper, we propose a novel approach to distinguish colonic polyps by integrating several techniques, including a modified deep residual network, principal component analysis and AdaBoost ensemble learning. A powerful deep residual network architecture, ResNet-50, was investigated to reduce the computational time by altering its architecture. To keep the interference to a minimum, median filter, image thresholding, contrast enhancement, and normalisation techniques were exploited on the endoscopic images to train the classification model. Three publicly available datasets, i.e., Kvasir, ETIS-LaribPolypDB, and CVC-ClinicDB, were merged to train the model, which included images with and without polyps.

RESULTS

The proposed approach trained with a combination of three datasets achieved Matthews Correlation Coefficient (MCC) of 0.9819 with accuracy, sensitivity, precision, and specificity of 99.10%, 98.82%, 99.37%, and 99.38%, respectively.

CONCLUSIONS

These results show that our method could repeatedly classify endoscopic images automatically and could be used to effectively develop computer-aided diagnostic tools for early CRC detection.