Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization

Establishment and validation of an artificial intelligence-based model for real-time detection and classification of colorectal adenoma

Colorectal cancer (CRC) prevention requires early detection and removal of adenomas. We aimed to develop a computational model for real-time detection and classification of colorectal adenoma. Computationally constrained background based on real-time detection, we propose an improved adaptive lightweight ensemble model for real-time detection and classification of adenomas and other polyps. Firstly, we devised an adaptive lightweight network modification and effective training strategy to diminish the computational requirements for real-time detection. Secondly, by integrating the adaptive lightweight YOLOv4 with the single shot multibox detector network, we established the adaptive small object detection ensemble (ASODE) model, which enhances the precision of detecting target polyps without significantly increasing the model's memory footprint. We conducted simulated training using clinical colonoscopy images and videos to validate the method's performance, extracting features from 1148 polyps and employing a confidence threshold of 0.5 to filter out low-confidence sample predictions. Finally, compared to state-of-the-art models, our ASODE model demonstrated superior performance. In the test set, the sensitivity of images and videos reached 87.96% and 92.31%, respectively. Additionally, the ASODE model achieved an accuracy of 92.70% for adenoma detection with a false positive rate of 8.18%. Training results indicate the effectiveness of our method in classifying small polyps. Our model exhibits remarkable performance in real-time detection of colorectal adenomas, serving as a reliable tool for assisting endoscopists.

Artificial intelligence for characterization of diminutive colorectal polyps: A feasibility study comparing two computer-aided diagnosis systems

BACKGROUND

Artificial intelligence (AI) has potential in the optical diagnosis of colorectal polyps.

AIM

To evaluate the feasibility of the real-time use of the computer-aided diagnosis system (CADx) AI for ColoRectal Polyps (AI4CRP) for the optical diagnosis of diminutive colorectal polyps and to compare the performance with CAD EYE^TM (Fujifilm, Tokyo, Japan). CADx influence on the optical diagnosis of an expert endoscopist was also investigated.

METHODS

AI4CRP was developed in-house and CAD EYE was proprietary software provided by Fujifilm. Both CADx-systems exploit convolutional neural networks. Colorectal polyps were characterized as benign or premalignant and histopathology was used as gold standard. AI4CRP provided an objective assessment of its characterization by presenting a calibrated confidence characterization value (range 0.0-1.0). A predefined cut-off value of 0.6 was set with values < 0.6 indicating benign and values ≥ 0.6 indicating premalignant colorectal polyps. Low confidence characterizations were defined as values 40% around the cut-off value of 0.6 (< 0.36 and > 0.76). Self-critical AI4CRP’s diagnostic performances excluded low confidence characterizations.

RESULTS

AI4CRP use was feasible and performed on 30 patients with 51 colorectal polyps. Self-critical AI4CRP, excluding 14 low confidence characterizations [27.5% (14/51)], had a diagnostic accuracy of 89.2%, sensitivity of 89.7%, and specificity of 87.5%, which was higher compared to AI4CRP. CAD EYE had a 83.7% diagnostic accuracy, 74.2% sensitivity, and 100.0% specificity. Diagnostic performances of the endoscopist alone (before AI) increased non-significantly after reviewing the CADx characterizations of both AI4CRP and CAD EYE (AI-assisted endoscopist). Diagnostic performances of the AI-assisted endoscopist were higher compared to both CADx-systems, except for specificity for which CAD EYE performed best.

CONCLUSION

Real-time use of AI4CRP was feasible. Objective confidence values provided by a CADx is novel and self-critical AI4CRP showed higher diagnostic performances compared to AI4CRP.

Application of artificial intelligence in endoscopic gastrointestinal tumors

With an increasing number of patients with gastrointestinal cancer, effective and accurate early diagnostic clinical tools are required provide better health care for patients with gastrointestinal cancer. Recent studies have shown that artificial intelligence (AI) plays an important role in the diagnosis and treatment of patients with gastrointestinal tumors, which not only improves the efficiency of early tumor screening, but also significantly improves the survival rate of patients after treatment. With the aid of efficient learning and judgment abilities of AI, endoscopists can improve the accuracy of diagnosis and treatment through endoscopy and avoid incorrect descriptions or judgments of gastrointestinal lesions. The present article provides an overview of the application status of various artificial intelligence in gastric and colorectal cancers in recent years, and the direction of future research and clinical practice is clarified from a clinical perspective to provide a comprehensive theoretical basis for AI as a promising diagnostic and therapeutic tool for gastrointestinal cancer.

Polyp characterization using deep learning and a publicly accessible polyp video database

OBJECTIVES

Convolutional neural networks (CNN) for computer-aided diagnosis of polyps are often trained using high-quality still images in a single chromoendoscopy imaging modality with sessile serrated lesions (SSLs) often excluded. This study developed a CNN from videos to classify polyps as adenomatous or nonadenomatous using standard narrow-band imaging (NBI) and NBI-near focus (NBI-NF) and created a publicly accessible polyp video database.

METHODS

We trained a CNN with 16,832 high and moderate quality frames from 229 polyp videos (56 SSLs). It was evaluated with 222 polyp videos (36 SSLs) across two test-sets. Test-set I consists of 14,320 frames (157 polyps, 111 diminutive). Test-set II, which is publicly accessible, 3317 video frames (65 polyps, 41 diminutive), which was benchmarked with three expert and three nonexpert endoscopists.

RESULTS

Sensitivity for adenoma characterization was 91.6% in test-set I and 89.7% in test-set II. Specificity was 91.9% and 88.5%. Sensitivity for diminutive polyps was 89.9% and 87.5%; specificity 90.5% and 88.2%. In NBI-NF, sensitivity was 89.4% and 89.5%, with a specificity of 94.7% and 83.3%. In NBI, sensitivity was 85.3% and 91.7%, with a specificity of 87.5% and 90.0%, respectively. The CNN achieved preservation and incorporation of valuable endoscopic innovations (PIVI)-1 and PIVI-2 thresholds for each test-set. In the benchmarking of test-set II, the CNN was significantly more accurate than nonexperts (13.8% difference [95% confidence interval 3.2-23.6], P = 0.01) with no significant difference with experts.

CONCLUSIONS

A single CNN can differentiate adenomas from SSLs and hyperplastic polyps in both NBI and NBI-NF. A publicly accessible NBI polyp video database was created and benchmarked.

An artificial intelligence platform provides an accurate interpretation of esophageal motility from Functional Lumen Imaging Probe Panometry studies

BACKGROUND

Functional lumen imaging probe (FLIP) Panometry is performed at the time of sedated endoscopy and evaluates esophageal motility in response to distension. This study aimed to develop and test an automated artificial intelligence (AI) platform that could interpret FLIP Panometry studies.

METHODS

The study cohort included 678 consecutive patients and 35 asymptomatic controls that completed FLIP Panometry during endoscopy and high-resolution manometry (HRM). "True" study labels for model training and testing were assigned by experienced esophagologists per a hierarchical classification scheme. The supervised, deep learning, AI model generated FLIP Panometry heatmaps from raw FLIP data and based on convolutional neural networks assigned esophageal motility labels using a two-stage prediction model. Model performance was tested on a 15% held-out test set (n = 103); the remainder of the studies were utilized for model training (n = 610).

KEY RESULTS

"True" FLIP labels across the entire cohort included 190 (27%) "normal," 265 (37%) "not normal/not achalasia," and 258 (36%) "achalasia." On the test set, both the Normal/Not normal and the achalasia/not achalasia models achieved an accuracy of 89% (with 89%/88% recall, 90%/89% precision, respectively). Of 28 patients with achalasia (per HRM) in the test set, 0 were predicted as "normal" and 93% as "achalasia" by the AI model.

CONCLUSIONS

An AI platform provided accurate interpretation of FLIP Panometry esophageal motility studies from a single center compared with the impression of experienced FLIP Panometry interpreters. This platform may provide useful clinical decision support for esophageal motility diagnosis from FLIP Panometry studies performed at the time of endoscopy.

A trial deep learning-based model for four-class histologic classification of colonic tumor from narrow band imaging

Narrow band imaging (NBI) has been extensively utilized as a diagnostic tool for colorectal neoplastic lesions. This study aimed to develop a trial deep learning (DL) based four-class classification model for low-grade dysplasia (LGD); high-grade dysplasia or mucosal carcinoma (HGD); superficially invasive submucosal carcinoma (SMs) and deeply invasive submucosal carcinomas (SMd) and evaluate its potential as a diagnostic tool. We collected a total of 1,390 NBI images as the dataset, including 53 LGD, 120 HGD, 20 SMs and 17 SMd. A total of 598,801 patches were trimmed from the lesion and background. A patch-based classification model was built by employing a residual convolutional neural network (CNN) and validated by three-fold cross-validation. The patch-based validation accuracy was 0.876, 0.957, 0.907 and 0.929 in LGD, HGD, SMs and SMd, respectively. The image-level classification algorithm was derived from the patch-based mapping across the entire image domain, attaining accuracies of 0.983, 0.990, 0.964, and 0.992 in LGD, HGD, SMs, and SMd, respectively. Our CNN-based model demonstrated high performance for categorizing the histological grade of dysplasia as well as the depth of invasion in routine colonoscopy, suggesting a potential diagnostic tool with minimal human inputs.

Multifeature Fusion Classification Method for Adaptive Endoscopic Ultrasonography Tumor Image

Endoscopic ultrasonography (EUS) has been found to be of great advantage in the diagnosis of digestive tract submucosal tumors. However, EUS-based diagnosis is limited by variability in subjective interpretation on the part of doctors. Tumor classification of ultrasound images with the computer-aided diagnosis system can significantly improve the diagnostic efficiency and accuracy of doctors. In this study, we proposed a multifeature fusion classification method for adaptive EUS tumor images. First, for different ultrasound tumor images, we selected the region of interest based on prior information to facilitate the estimation in the subsequent works. Second, we proposed a method based on image gray histogram feature extraction with principal component analysis dimensionality reduction, which learns the gray distribution of different tumor images effectively. Third, we fused the reduced grayscale features with the improved local binary pattern features and gray-level co-occurrence matrix features, and then used the multiclassification support vector machine. Finally, in the experiment, we selected the 431 ultrasound images of 109 patients in the hospital and compared the experimental effects of different features and different classifiers. The results revealed that the proposed method performed best, with the highest accuracy of 96.18% and an area under the curve of 99%. It is evident that the method proposed in this study can efficiently contribute to the classification of EUS tumor images.

Improving bowel preparation for colonoscopy with a smartphone application driven by artificial intelligence

Optimal bowel preparation is a prerequisite for a successful colonoscopy; however, the rate of inadequate bowel preparation remains relatively high. In this study, we establish a smartphone app that assesses patient bowel preparation using an artificial intelligence (AI)-based prediction system trained on labeled photographs of feces in the toilet and evaluate its impact on bowel preparation quality in colonoscopy outpatients. We conduct a prospective, single-masked, multicenter randomized clinical trial, enrolling outpatients who own a smartphone and are scheduled for a colonoscopy. We screen 578 eligible patients and randomize 524 in a 1:1 ratio to the control or AI-driven app group for bowel preparation. The study endpoints are the percentage of patients with adequate bowel preparation and the total BBPS score, compliance with dietary restrictions and purgative instructions, polyp detection rate, and adenoma detection rate (secondary). The prediction system has an accuracy of 95.15%, a specificity of 97.25%, and an area under the curve of 0.98 in the test dataset. In the full analysis set (n = 500), adequate preparation is significantly higher in the AI-driven app group (88.54 vs. 65.59%; P < 0.001). The mean BBPS score is 6.74 ± 1.25 in the AI-driven app group and 5.97 ± 1.81 in the control group (P < 0.001). The rates of compliance with dietary restrictions (93.68 vs. 83.81%, P = 0.001) and purgative instructions (96.05 vs. 84.62%, P < 0.001) are significantly higher in the AI-driven app group, as is the rate of additional purgative intake (26.88 vs. 17.41%, P = 0.011). Thus, our AI-driven smartphone app significantly improves the quality of bowel preparation and patient compliance.

Intraoperative Detection of Surgical Gauze Using Deep Convolutional Neural Network

During laparoscopic surgery, surgical gauze is usually inserted into the body cavity to help achieve hemostasis. Retention of surgical gauze in the body cavity may necessitate reoperation and increase surgical risk. Using deep learning technology, this study aimed to propose a neural network model for gauze detection from the surgical video to record the presence of the gauze. The model was trained by the training group using YOLO (You Only Look Once)v5x6, then applied to the testing group. Positive predicted value (PPV), sensitivity, and mean average precision (mAP) were calculated. Furthermore, a timeline of gauze presence in the video was drawn by the model as well as human annotation to evaluate the accuracy. After the model was well-trained, the PPV, sensitivity, and mAP in the testing group were 0.920, 0.828, and 0.881, respectively. The inference time was 11.3 ms per image. The average accuracy of the model adding a marking and filtering process was 0.899. In conclusion, surgical gauze can be successfully detected using deep learning in the surgical video. Our model provided a fast detection of surgical gauze, allowing further real-time gauze tracing in laparoscopic surgery that may help surgeons recall the location of the missing gauze.

Deep-Learning and Device-Assisted Enteroscopy: Automatic Panendoscopic Detection of Ulcers and Erosions

Background and Objectives: Device-assisted enteroscopy (DAE) has a significant role in approaching enteric lesions. Endoscopic observation of ulcers or erosions is frequent and can be associated with many nosological entities, namely Crohn's disease. Although the application of artificial intelligence (AI) is growing exponentially in various imaged-based gastroenterology procedures, there is still a lack of evidence of the AI technical feasibility and clinical applicability of DAE. This study aimed to develop and test a multi-brand convolutional neural network (CNN)-based algorithm for automatically detecting ulcers and erosions in DAE. Materials and Methods: A unicentric retrospective study was conducted for the development of a CNN, based on a total of 250 DAE exams. A total of 6772 images were used, of which 678 were considered ulcers or erosions after double-validation. Data were divided into a training and a validation set, the latter being used for the performance assessment of the model. Our primary outcome measures were sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and an area under the curve precision-recall curve (AUC-PR). Results: Sensitivity, specificity, PPV, and NPV were respectively 88.5%, 99.7%, 96.4%, and 98.9%. The algorithm's accuracy was 98.7%. The AUC-PR was 1.00. The CNN processed 293.6 frames per second, enabling AI live application in a real-life clinical setting in DAE. Conclusion: To the best of our knowledge, this is the first study regarding the automatic multi-brand panendoscopic detection of ulcers and erosions throughout the digestive tract during DAE, overcoming a relevant interoperability challenge. Our results highlight that using a CNN to detect this type of lesion is associated with high overall accuracy. The development of binary CNN for automatically detecting clinically relevant endoscopic findings and assessing endoscopic inflammatory activity are relevant steps toward AI application in digestive endoscopy, particularly for panendoscopic evaluation.

Establishment of clinical predictive model based on the study of influence factors in patients with colorectal polyps

Background

Colorectal cancer (CRC) is the most common gastrointestinal malignancy and is generally thought to be caused by the transformation of colorectal polyps. It has been shown that early detection and removal of colorectal polyps may reduce the mortality and morbidity of colorectal cancer.

Objective

Based on the risk factors associated with colorectal polyps, an individualized clinical prediction model was built to predict and evaluate the possibility of developing colorectal polyp.

Methods

A case-control study was conducted. Clinical data were collected from 475 patients who underwent colonoscopy at the Third Hospital of Hebei Medical University from 2020 to 2021. All clinical data were then divided into training sets and validation sets by using R software (7:3). A multivariate logistic analysis was performed to identify the factors associated with colorectal polyps according to the training set, and a predictive nomogram was created by R software based on the multivariate analysis. The results were internally validated by receiver operating characteristic (ROC) curves, calibration curves, and externally validated by validation sets.

Results

Multivariate logistic regression analysis showed that age (OR = 1.047, 95% CI = 1.029-1.065), history of cystic polyp (OR = 7.596, 95% CI = 0.976-59.129), and history of colorectal diverticulums (OR = 2.548, 95% CI = 1.209-5.366) were independent risk factors for colorectal polyps. History of constipation (OR = 0.457, 95% CI = 0.268-0.799) and fruit consumption (OR = 0.613, 95% CI 0.350-1.037) were protective factors for colorectal polyps. The nomogram demonstrated good accuracy for predicting colorectal polyps, with both C index and AUC being 0.747 (95% CI = 0.692-0.801). The calibration curves showed good agreement between the predicted risk by the nomogram and real outcomes. Both internal and external validation of the model showed good results.

Conclusion

In our study, the nomogram prediction model is reliable and accurate, which can help early clinical screening of patients with high-risk colorectal polyps, improve polyp detection rate, and reduce the incidence of colorectal cancer (CRC).

Mucosal imaging in colon polyps: New advances and what the future may hold

An expanding range of advanced mucosal imaging technologies have been developed with the goal of improving the detection and characterization of lesions in the gastrointestinal tract. Many technologies have targeted colorectal neoplasia given the potential for intervention prior to the development of invasive cancer in the setting of widespread surveillance programs. Improvement in adenoma detection reduces miss rates and prevents interval cancer development. Advanced imaging technologies aim to enhance detection without significantly increasing procedural time. Accurate polyp characterisation guides resection techniques for larger polyps, as well as providing the platform for the “resect and discard” and “do not resect” strategies for small and diminutive polyps. This review aims to collate and summarise the evidence regarding these technologies to guide colonoscopic practice in both interventional and non-interventional endoscopists.

Diagnostic accuracy of artificial intelligence for detecting gastrointestinal luminal pathologies: A systematic review and meta-analysis

Background

Artificial Intelligence (AI) holds considerable promise for diagnostics in the field of gastroenterology. This systematic review and meta-analysis aims to assess the diagnostic accuracy of AI models compared with the gold standard of experts and histopathology for the diagnosis of various gastrointestinal (GI) luminal pathologies including polyps, neoplasms, and inflammatory bowel disease.

Methods

We searched PubMed, CINAHL, Wiley Cochrane Library, and Web of Science electronic databases to identify studies assessing the diagnostic performance of AI models for GI luminal pathologies. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. We performed a meta-analysis and hierarchical summary receiver operating characteristic curves (HSROC). The risk of bias was assessed using Quality Assessment for Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Subgroup analyses were conducted based on the type of GI luminal disease, AI model, reference standard, and type of data used for analysis. This study is registered with PROSPERO (CRD42021288360).

Findings

We included 73 studies, of which 31 were externally validated and provided sufficient information for inclusion in the meta-analysis. The overall sensitivity of AI for detecting GI luminal pathologies was 91.9% (95% CI: 89.0–94.1) and specificity was 91.7% (95% CI: 87.4–94.7). Deep learning models (sensitivity: 89.8%, specificity: 91.9%) and ensemble methods (sensitivity: 95.4%, specificity: 90.9%) were the most commonly used models in the included studies. Majority of studies (n = 56, 76.7%) had a high risk of selection bias while 74% (n = 54) studies were low risk on reference standard and 67% (n = 49) were low risk for flow and timing bias.

Interpretation

The review suggests high sensitivity and specificity of AI models for the detection of GI luminal pathologies. There is a need for large, multi-center trials in both high income countries and low- and middle- income countries to assess the performance of these AI models in real clinical settings and its impact on diagnosis and prognosis.

Systematic review registration

[https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=288360], identifier [CRD42021288360].

Updates in interaction of gastroesophageal reflux disease and extragastroesophageal digestive diseases

INTRODUCTION

Gastroesophageal reflux disease (GERD) is one of the common chronic diseases with prevalence increasing in the last decades. Because of its prevalence and chronicity, GERD affects the quality of life and increases health-care costs. Gastroesophageal diseases leading to GERD have been thoroughly studied, while extragastroesophageal digestive diseases (EGEDDs) may coexist with GERD and affect the occurrence and persistence of GERD symptoms and therapeutic effect.

AREAS COVERED

In this review, we aim to summarize the EGEDDs correlated with GERD and explore the potential mechanisms of this interaction.

EXPERT OPINION

Individuals with troublesome GERD symptoms may have some common gastroesophageal etiologies, but EGEDDs may also overlap and impact on the progression of GERD, which are often ignored in clinic. The lesions in the small intestine, colon, and hepatobiliary tract as well as functional bowel disorders had positive or negative associations with GERD through potential mechanisms. These diseases aggravate GERD symptoms, increase the esophageal acid burden, cause esophageal hypersensitivity, and finally affect the response to therapy in GERD patients. Therefore, it is necessary to clear the interaction between GERD and EGEDDs and their mechanisms.

The role of artificial intelligence based systems for cost optimization in colorectal cancer prevention programs

Colorectal Cancer (CRC) has seen a dramatic increase in incidence globally. In 2019, colorectal cancer accounted for 1.15 million deaths and 24.28 million disability-adjusted life-years (DALYs) worldwide. In India, the annual incidence rates (AARs) for colon cancer was 4.4 per 100,000. There has been a steady rise in the prevalence of CRC in India which may be attributed to urbanization, mass migration of population, westernization of diet and lifestyle practices and a rise of obesity and metabolic risk factors that place the population at a higher risk of CRC. Moreoever, CRC in India differs from that described in the Western countries, with a higher proportion of young patients and more patients presenting with an advanced stage. This may be due to poor access to specialized healthcare and socio-economic factors. Early identification of adenomatous colonic polyps, which are well-recognized pre-cancerous lesions, at the time of screening colonoscopy has been shown to be the most effective measure used for CRC prevention. However, colonic polyps are frequently missed during colonoscopy and moreover, these screening programs necessitate man-power, time and resources for processing resected polyps, that may hamper penetration and efficacy in mid- to low-income countries. In the last decade, there has been significant progress made in the automatic detection of colonic polyps by multiple AI-based systems. With the advent of better AI methodology, the focus has shifted from mere detection to accurate discrimination and diagnosis of colonic polyps. These systems, once validated, could usher in a new era in Colorectal Cancer (CRC) prevention programs which would center around “Leave in-situ” and “Resect and discard” strategies. These new strategies hinge around the specificity and accuracy of AI based systems in correctly identifying the pathological diagnosis of the polyps, thereby providing the endoscopist with real-time information in order to make a clinical decision of either leaving the lesion in-situ (mucosal polyps) or resecting and discarding the polyp (hyperplastic polyps). The major advantage of employing these strategies would be in cost optimization of CRC prevention programs while ensuring good clinical outcomes. The adoption of these AI-based systems in the national cancer prevention program of India in accordance with the mandate to increase technology integration could prove to be cost-effective and enable implementation of CRC prevention programs at the population level. This level of penetration could potentially reduce the incidence of CRC and improve patient survival by enabling early diagnosis and treatment. In this review, we will highlight key advancements made in the field of AI in the identification of polyps during colonoscopy and explore the role of AI based systems in cost optimization during the universal implementation of CRC prevention programs in the context of mid-income countries like India.

In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy

Background and study aims  Artificial intelligence is currently able to accurately predict the histology of colorectal polyps. However, systems developed to date use complex optical technologies and have not been tested in vivo. The objective of this study was to evaluate the efficacy of a new deep learning-based optical diagnosis system, ATENEA, in a real clinical setting using only high-definition white light endoscopy (WLE) and to compare its performance with endoscopists. Methods  ATENEA was prospectively tested in real life on consecutive polyps detected in colorectal cancer screening colonoscopies at Hospital Clínic. No images were discarded, and only WLE was used. The in vivo ATENEA's prediction (adenoma vs non-adenoma) was compared with the prediction of four staff endoscopists without specific training in optical diagnosis for the study purposes. Endoscopists were blind to the ATENEA output. Histology was the gold standard. Results  Ninety polyps (median size: 5 mm, range: 2-25) from 31 patients were included of which 69 (76.7 %) were adenomas. ATENEA correctly predicted the histology in 63 of 69 (91.3 %, 95 % CI: 82 %-97 %) adenomas and 12 of 21 (57.1 %, 95 % CI: 34 %-78 %) non-adenomas while endoscopists made correct predictions in 52 of 69 (75.4 %, 95 % CI: 60 %-85 %) and 20 of 21 (95.2 %, 95 % CI: 76 %-100 %), respectively. The global accuracy was 83.3 % (95 % CI: 74%-90 %) and 80 % (95 % CI: 70 %-88 %) for ATENEA and endoscopists, respectively. Conclusion  ATENEA can accurately be used for in vivo characterization of colorectal polyps, enabling the endoscopist to make direct decisions. ATENEA showed a global accuracy similar to that of endoscopists despite an unsatisfactory performance for non-adenomatous lesions.

A computer-aided diagnosis system using white-light endoscopy for the prediction of conventional adenoma with high grade dysplasia

OBJECTIVES

We developed a computer-aided diagnosis system called ECRCCAD using standard white-light endoscopy (WLE) for predicting conventional adenomas with high-grade dysplasia (HGD) to optimise the patients' management decisions during colonoscopy.

METHODS

Pretraining model was used to fine-tune the model parameters by transfer learning. 2,397 images of HGD and 2,487 low-grade dysplasia (LGD) images were randomly assigned (8:1:1) to the training, optimising, and internal validation dataset. The prospective validation dataset is the frames accessed from colonoscope videoes. One independent rural hospital provided an external validation dataset. Histopathological diagnosis was used as the standard criterion. The capability of the ECRCCAD to distinguish HGD was assessed and compared with two expert endoscopists.

RESULTS

The accuracy, sensitivity and specificity for diagnosis of HGD in the internal validation set were 90.5%, 93.2%, 87.9%, respectively. While 88.2%, 85.4%, 89.8%, respectively, for the external validation set. For the prospective validation set, ECRCCAD achieved an AUC of 93.5% in diagnosing HGD. The performance of ECRCCAD in diagnosing HGD was better than that of the expert endoscopist in the external validation set (88.2% vs. 71.5%, P < 0.0001).

CONCLUSION

ECRCCAD had good diagnostic capability for HGD and enabled a more convenient and accurate diagnosis using WLE.

Artificial intelligence in endoscopy: More than what meets the eye in screening colonoscopy and endosonographic evaluation of pancreatic lesions

Artificial intelligence (AI)-based tools have ushered in a new era of innovation in the field of gastrointestinal (GI) endoscopy. Despite vast improvements in endoscopic techniques and equipment, diagnostic endoscopy remains heavily operator-dependent, in particular, colonoscopy and endoscopic ultrasound (EUS). Recent reports have shown that as much as 25% of colonic adenomas may be missed at colonoscopy. This can result in an increased incidence of interval colon cancer. Similarly, EUS has been shown to have high inter-observer variability, overlap in diagnoses with a relatively low specificity for pancreatic lesions. Our understanding of Machine-learning (ML) techniques in AI have evolved over the last decade and its application in AI–based tools for endoscopic detection and diagnosis is being actively investigated at several centers. ML is an aspect of AI that is based on neural networks, and is widely used for image classification, object detection, and semantic segmentation which are key functional aspects of AI-related computer aided diagnostic systems. In this review, current status and limitations of ML, specifically for adenoma detection and endosonographic diagnosis of pancreatic lesions, will be summarized from existing literature. This will help to better understand its role as viewed through the prism of real world application in the field of GI endoscopy.

Real-Time Artificial Intelligence-Based Histologic Classifications of Colorectal Polyps Using Narrow-Band Imaging

Background and Aims

With the development of artificial intelligence (AI), we have become capable of applying real-time computer-aided detection (CAD) in clinical practice. Our aim is to develop an AI-based CAD-N and optimize its diagnostic performance with narrow-band imaging (NBI) images.

Methods

We developed the CAD-N model with ResNeSt using NBI images for real-time assessment of the histopathology of colorectal polyps (type 1, hyperplastic or inflammatory polyps; type 2, adenomatous polyps, intramucosal cancer, or superficial submucosal invasive cancer; type 3, deep submucosal invasive cancer; and type 4, normal mucosa). We also collected 116 consecutive polyp videos to validate the accuracy of the CAD-N.

Results

A total of 10,573 images (7,032 images from 650 polyps and 3,541 normal mucous membrane images) from 478 patients were finally chosen for analysis. The sensitivity, specificity, PPV, NPV, and accuracy for each type of the CAD-N in the test set were 89.86%, 97.88%, 93.13%, 96.79%, and 95.93% for type 1; 93.91%, 95.49%, 91.80%, 96.69%, and 94.94% for type 2; 90.21%, 99.29%, 90.21%, 99.29%, and 98.68% for type 3; and 94.86%, 97.28%, 94.73%, 97.35%, and 96.45% for type 4, respectively. The overall accuracy was 93%. We also built models for polyps ≤5 mm, and the sensitivity, specificity, PPV, NPV, and accuracy for them were 96.81%, 94.08%, 95%, 95.97%, and 95.59%, respectively. Video validation results showed that the sensitivity, specificity, and accuracy of the CAD-N were 84.62%, 86.27%, and 85.34%, respectively.

Conclusions

We have developed real-time AI-based histologic classifications of colorectal polyps using NBI images with good accuracy, which may help in clinical management and documentation of optical histology results.

Objective Methods of 5-Aminolevulinic Acid-Based Endoscopic Photodynamic Diagnosis Using Artificial Intelligence for Identification of Gastric Tumors

Positive diagnoses of gastric tumors from photodynamic diagnosis (PDD) images after the administration of 5-aminolevulinic acid are subjectively identified by expert endoscopists. Objective methods of tumor identification are needed to reduce potential misidentifications. We developed two methods to identify gastric tumors from PDD images. Method one was applied to segmented regions in the PDD endoscopic image to determine the region in LAB color space to be attributed to tumors using a multi-layer neural network. Method two aimed to diagnose tumors and determine regions in the PDD endoscopic image attributed to tumors using the convoluted neural network method. The efficiencies of diagnosing tumors were 77.8% (7/9) and 93.3% (14/15) for method one and method two, respectively. The efficiencies of determining tumor region defined as the ratio of the area were 35.7% (0.0–78.0) and 48.5% (3.0–89.1) for method one and method two, respectively. False-positive rates defined as the ratio of the area were 0.3% (0.0–2.0) and 3.8% (0.0–17.4) for method one and method two, respectively. Objective methods of determining tumor region in 5-aminolevulinic acid-based endoscopic PDD were developed by identifying regions in LAB color space attributed to tumors or by applying a method of convoluted neural network.

Real-time semantic segmentation of gastric intestinal metaplasia using a deep learning approach

Background/Aims

Previous artificial intelligence (AI) models attempting to segment gastric intestinal metaplasia (GIM) areas have failed to be deployed in real-time endoscopy due to their slow inference speeds. Here, we propose a new GIM segmentation AI model with inference speeds faster than 25 frames per second that maintains a high level of accuracy.

Methods

Investigators from Chulalongkorn University obtained 802 histological-proven GIM images for AI model training. Four strategies were proposed to improve the model accuracy. First, transfer learning was employed to the public colon datasets. Second, an image preprocessing technique contrast-limited adaptive histogram equalization was employed to produce clearer GIM areas. Third, data augmentation was applied for a more robust model. Lastly, the bilateral segmentation network model was applied to segment GIM areas in real time. The results were analyzed using different validity values.

Results

From the internal test, our AI model achieved an inference speed of 31.53 frames per second. GIM detection showed sensitivity, specificity, positive predictive, negative predictive, accuracy, and mean intersection over union in GIM segmentation values of 93%, 80%, 82%, 92%, 87%, and 57%, respectively.

Conclusions

The bilateral segmentation network combined with transfer learning, contrast-limited adaptive histogram equalization, and data augmentation can provide high sensitivity and good accuracy for GIM detection and segmentation.

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Artificial intelligence-assisted colonoscopy: a narrative review of current data and clinical applications

Colonoscopy is the reference standard procedure for the prevention and diagnosis of colorectal cancer, which is a leading cause of cancer-related deaths in Singapore. Artificial intelligence systems are automated, objective and reproducible. Artificial intelligence-assisted colonoscopy has recently been introduced into clinical practice as a clinical decision support tool. This review article provides a summary of the current published data and discusses ongoing research and current clinical applications of artificial intelligence-assisted colonoscopy.

Digital Technology in Visceral Medicine: An Overview in Outpatient Care in Germany

Background

Digital technology has become an integral part of healthcare and will revolutionize the practice of medicine. Although previously the administrative tasks were captured by digital technologies, now the communication channels with the patients are also being focused. For example, taking medical histories is now made possible by this new technology, information and explanatory forms can be sent digitally, and even face-to-face consultations are increasingly made possible by video consultations. Especially in the COVID pandemic, this form of contactless encounter has become a valuable enrichment of medical care. But also telemedical tasks such as teleconsultation or artificial intelligence in the context of adenoma detection are techniques that are conquering outpatient and inpatient visceral medical care.

Summary

This article gives an overview of digital communication and possible uses of digital technologies in medical practices in Germany.

Key Message

Medicine is renewing itself through digital techniques. The pace of change is rapid and unstoppable. Today's medical progress is no longer conceivable without these techniques.

Artificial Intelligence in Endoscopy

Background

Owing to their rapid development, artificial intelligence (AI) technologies offer a great promise for gastroenterology practice and research. At present, AI-guided image interpretation has already been used with success for endoscopic detection of early malignant lesions. Nonetheless, there are complex challenges and possible shortcomings that must be considered before full implementation can be realized.

Summary

In this review, the current status of AI in endoscopy is summarized. Future perspectives and open questions for further studies are stressed.

Key Messages

The usage of AI algorithms for polyp detection in screening colonoscopy results in a significant increase in the adenoma detection rate, mainly attributed to the identification of diminutive polyps. Computer-aided characterization of colorectal polyps accompanies the detection, but further studies are needed to evaluate the clinical benefit. In contrast to colonoscopy, usage of AI in gastroscopy is currently rather limited. Regarding other fields of endoscopic imaging, capsule endoscopy is the ideal imaging platform for AI, due to the potential of saving time in the video analysis.

Development of multi-class computer-aided diagnostic systems using the NICE/JNET classifications for colorectal lesions

BACKGROUND AND AIM

Diagnostic support using artificial intelligence may contribute to the equalization of endoscopic diagnosis of colorectal lesions. We developed computer-aided diagnosis (CADx) support system for diagnosing colorectal lesions using the NBI International Colorectal Endoscopic (NICE) classification and the Japan NBI Expert Team (JNET) classification.

METHODS

Using Residual Network as the classifier and NBI images as training images, we developed a CADx based on the NICE classification (CADx-N) and a CADx based on the JNET classification (CADx-J). For validation, 480 non-magnifying and magnifying NBI images were used for the CADx-N and 320 magnifying NBI images were used for the CADx-J. The diagnostic performance of the CADx-N was evaluated using the magnification rate.

RESULTS

The accuracy of the CADx-N for Types 1, 2, and 3 was 97.5%, 91.2%, and 93.8%, respectively. The diagnostic performance for each magnification level was good (no statistically significant difference). The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the CADx-J were 100%, 96.3%, 82.8%, 100%, and 96.9% for Type 1; 80.3%, 93.7%, 94.1%, 79.2%, and 86.3% for Type 2A; 80.4%, 84.7%, 46.8%, 96.3%, and 84.1% for Type 2B; and 62.5%, 99.6%, 96.8%, 93.8%, and 94.1% for Type 3, respectively.

CONCLUSIONS

The multi-class CADx systems had good diagnostic performance with both the NICE and JNET classifications and may aid in educating non-expert endoscopists and assist in diagnosing colorectal lesions.

Deep Learning and Device-Assisted Enteroscopy: Automatic Detection of Gastrointestinal Angioectasia

Background and Objectives: Device-assisted enteroscopy (DAE) allows deep exploration of the small bowel and combines diagnostic and therapeutic capacities. Suspected mid-gastrointestinal bleeding is the most frequent indication for DAE, and vascular lesions, particularly angioectasia, are the most common etiology. Nevertheless, the diagnostic yield of DAE for the detection of these lesions is suboptimal. Deep learning algorithms have shown great potential for automatic detection of lesions in endoscopy. We aimed to develop an artificial intelligence (AI) model for the automatic detection of angioectasia DAE images. Materials and Methods: A convolutional neural network (CNN) was developed using DAE images. Each frame was labeled as normal/mucosa or angioectasia. The image dataset was split for the constitution of training and validation datasets. The latter was used for assessing the performance of the CNN. Results: A total of 72 DAE exams were included, and 6740 images were extracted (5345 of normal mucosa and 1395 of angioectasia). The model had a sensitivity of 88.5%, a specificity of 97.1% and an AUC of 0.988. The image processing speed was 6.4 ms/frame. Conclusions: The application of AI to DAE may have a significant impact on the management of patients with suspected mid-gastrointestinal bleeding.

Automatic image and text-based description for colorectal polyps using BASIC classification

Colorectal polyps (CRP) are precursor lesions of colorectal cancer (CRC). Correct identification of CRPs during in-vivo colonoscopy is supported by the endoscopist's expertise and medical classification models. A recent developed classification model is the Blue light imaging Adenoma Serrated International Classification (BASIC) which describes the differences between non-neoplastic and neoplastic lesions acquired with blue light imaging (BLI). Computer-aided detection (CADe) and diagnosis (CADx) systems are efficient at visually assisting with medical decisions but fall short at translating decisions into relevant clinical information. The communication between machine and medical expert is of crucial importance to improve diagnosis of CRP during in-vivo procedures. In this work, the combination of a polyp image classification model and a language model is proposed to develop a CADx system that automatically generates text comparable to the human language employed by endoscopists. The developed system generates equivalent sentences as the human-reference and describes CRP images acquired with white light (WL), blue light imaging (BLI) and linked color imaging (LCI). An image feature encoder and a BERT module are employed to build the AI model and an external test set is used to evaluate the results and compute the linguistic metrics. The experimental results show the construction of complete sentences with an established metric scores of BLEU-1 = 0.67, ROUGE-L = 0.83 and METEOR = 0.50. The developed CADx system for automatic CRP image captioning facilitates future advances towards automatic reporting and may help reduce time-consuming histology assessment.

Deep Learning for nasopharyngeal Carcinoma Identification Using Both White Light and Narrow-Band Imaging Endoscopy

OBJECTIVES/HYPOTHESIS

To develop a deep-learning-based automatic diagnosis system for identifying nasopharyngeal carcinoma (NPC) from noncancer (inflammation and hyperplasia), using both white light imaging (WLI) and narrow-band imaging (NBI) nasopharyngoscopy images.

STUDY DESIGN

Retrospective study.

METHODS

A total of 4,783 nasopharyngoscopy images (2,898 WLI and 1,885 NBI) of 671 patients were collected and a novel deep convolutional neural network (DCNN) framework was developed named Siamese deep convolutional neural network (S-DCNN), which can simultaneously utilize WLI and NBI images to improve the classification performance. To verify the effectiveness of combining the above-mentioned two modal images for prediction, we compared the proposed S-DCNN with two baseline models, namely DCNN-1 (only considering WLI images) and DCNN-2 (only considering NBI images).

RESULTS

In the threefold cross-validation, an overall accuracy and area under the curve of the three DCNNs achieved 94.9% (95% confidence interval [CI] 93.3%-96.5%) and 0.986 (95% CI 0.982-0.992), 87.0% (95% CI 84.2%-89.7%) and 0.930 (95% CI 0.906-0.961), and 92.8% (95% CI 90.4%-95.3%) and 0.971 (95% CI 0.953-0.992), respectively. The accuracy of S-DCNN is significantly improved compared with DCNN-1 (P-value <.001) and DCNN-2 (P-value = .008).

CONCLUSION

Using the deep-learning technology to automatically diagnose NPC under nasopharyngoscopy can provide valuable reference for NPC screening. Superior performance can be obtained by simultaneously utilizing the multimodal features of NBI image and WLI image of the same patient.

LEVEL OF EVIDENCE

3 Laryngoscope, 2021.

Optical diagnosis of colorectal polyps using convolutional neural networks

Colonoscopy remains the gold standard investigation for colorectal cancer screening as it offers the opportunity to both detect and resect pre-malignant and neoplastic polyps. Although technologies for image-enhanced endoscopy are widely available, optical diagnosis has not been incorporated into routine clinical practice, mainly due to significant inter-operator variability. In recent years, there has been a growing number of studies demonstrating the potential of convolutional neural networks (CNN) to enhance optical diagnosis of polyps. Data suggest that the use of CNNs might mitigate the inter-operator variability amongst endoscopists, potentially enabling a “resect and discard“ or ”leave in“ strategy to be adopted in real-time. This would have significant financial benefits for healthcare systems, avoid unnecessary polypectomies of non-neoplastic polyps and improve the efficiency of colonoscopy. Here, we review advances in CNN for the optical diagnosis of colorectal polyps, current limitations and future directions.

Artificial intelligence application in diagnostic gastrointestinal endoscopy - Deus ex machina?

The close relationship of medicine with technology and the particular interest in this symbiosis in recent years has led to the development of several computed artificial intelligence (AI) systems aimed at various areas of medicine. A number of studies have demonstrated that those systems allow accurate diagnoses with histological precision, thus facilitating decision-making by clinicians in real time. In the field of gastroenterology, AI has been applied in the diagnosis of pathologies of the entire digestive tract and their attached glands, and are increasingly accepted for the detection of colorectal polyps and confirming their histological classification. Studies have shown high accuracy, sensitivity, and specificity in relation to expert endoscopists, and mainly in relation to those with less experience. Other applications that are increasingly studied and with very promising results are the investigation of dysplasia in patients with Barrett's esophagus and the endoscopic and histological assessment of colon inflammation in patients with ulcerative colitis. In some cases AI is thus better than or at least equal to human abilities. However, additional studies are needed to reinforce the existing data, and mainly to determine the applicability of this technology in other indications. This review summarizes the state of the art of AI in gastroenterological pathology.

Computer-Aided Diagnosis of Diminutive Colorectal Polyps in Endoscopic Images: Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Background

Most colorectal polyps are diminutive and benign, especially those in the rectosigmoid colon, and the resection of these polyps is not cost-effective. Advancements in image-enhanced endoscopy have improved the optical prediction of colorectal polyp histology. However, subjective interpretability and inter- and intraobserver variability prohibits widespread implementation. The number of studies on computer-aided diagnosis (CAD) is increasing; however, their small sample sizes limit statistical significance.

Objective

This review aims to evaluate the diagnostic test accuracy of CAD models in predicting the histology of diminutive colorectal polyps by using endoscopic images.

Methods

Core databases were searched for studies that were based on endoscopic imaging, used CAD models for the histologic diagnosis of diminutive colorectal polyps, and presented data on diagnostic performance. A systematic review and diagnostic test accuracy meta-analysis were performed.

Results

Overall, 13 studies were included. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD models for the diagnosis of diminutive colorectal polyps (adenomatous or neoplastic vs nonadenomatous or nonneoplastic) were 0.96 (95% CI 0.93-0.97), 0.93 (95% CI 0.91-0.95), 0.87 (95% CI 0.76-0.93), and 87 (95% CI 38-201), respectively. The meta-regression analysis showed no heterogeneity, and no publication bias was detected. Subgroup analyses showed robust results. The negative predictive value of CAD models for the diagnosis of adenomatous polyps in the rectosigmoid colon was 0.96 (95% CI 0.95-0.97), and this value exceeded the threshold of the diagnosis and leave strategy.

Conclusions

CAD models show potential for the optical histological diagnosis of diminutive colorectal polyps via the use of endoscopic images.

Trial Registration

PROSPERO CRD42021232189; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=232189

Utility of artificial intelligence in colonoscopy

Colorectal cancer is one of the major causes of death worldwide. Colonoscopy is the most important tool that can identify neoplastic lesion in early stages and resect it in a timely manner which helps in reducing mortality related to colorectal cancer. However, the quality of colonoscopy findings depends on the expertise of the endoscopist and thus the rate of missed adenoma or polyp cannot be controlled. It is desirable to standardize the quality of colonoscopy by reducing the number of missed adenoma/polyps. Introduction of artificial intelligence (AI) in the field of medicine has become popular among physicians nowadays. The application of AI in colonoscopy can help in reducing miss rate and increasing colorectal cancer detection rate as per recent studies. Moreover, AI assistance during colonoscopy has also been utilized in patients with inflammatory bowel disease to improve diagnostic accuracy, assessing disease severity and predicting clinical outcomes. We conducted a literature review on the available evidence on use of AI in colonoscopy. In this review article, we discuss about the principles, application, limitations, and future aspects of AI in colonoscopy.

Application of artificial intelligence to endoscopy on common gastrointestinal benign diseases

Artificial intelligence (AI) has been widely involved in every aspect of healthcare in the preclinical stage. In the digestive system, AI has been trained to assist auxiliary examinations including histopathology, endoscopy, ultrasonography, computerized tomography, and magnetic resonance imaging in detection, diagnosis, classification, differentiation, prognosis, and quality control. In the field of endoscopy, the application of AI, such as automatic detection, diagnosis, classification, and invasion depth, in early gastrointestinal (GI) cancers has received wide attention. There is a paucity of studies of AI application on common GI benign diseases based on endoscopy. In the review, we provide an overview of AI applications to endoscopy on common GI benign diseases including in the esophagus, stomach, intestine, and colon. It indicates that AI will gradually become an indispensable part of normal endoscopic detection and diagnosis of common GI benign diseases as clinical data, algorithms, and other related work are constantly repeated and improved.

Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era

The development of artificial intelligence (AI) algorithms has permeated the medical field with great success. The widespread use of AI technology in diagnosing and treating several types of cancer, especially colorectal cancer (CRC), is now attracting substantial attention. CRC, which represents the third most commonly diagnosed malignancy in both men and women, is considered a leading cause of cancer-related deaths globally. Our review herein aims to provide in-depth knowledge and analysis of the AI applications in CRC screening, diagnosis, and treatment based on current literature. We also explore the role of recent advances in AI systems regarding medical diagnosis and therapy, with several promising results. CRC is a highly preventable disease, and AI-assisted techniques in routine screening represent a pivotal step in declining incidence rates of this malignancy. So far, computer-aided detection and characterization systems have been developed to increase the detection rate of adenomas. Furthermore, CRC treatment enters a new era with robotic surgery and novel computer-assisted drug delivery techniques. At the same time, healthcare is rapidly moving toward precision or personalized medicine. Machine learning models have the potential to contribute to individual-based cancer care and transform the future of medicine.