Efficacy of Real-Time Computer-Aided Detection of Colorectal Neoplasia in a Randomized Trial

Artificial Intelligence in Colorectal Cancer Surgery: Present and Future Perspectives

Artificial intelligence (AI) and computer vision (CV) are beginning to impact medicine. While evidence on the clinical value of AI-based solutions for the screening and staging of colorectal cancer (CRC) is mounting, CV and AI applications to enhance the surgical treatment of CRC are still in their early stage. This manuscript introduces key AI concepts to a surgical audience, illustrates fundamental steps to develop CV for surgical applications, and provides a comprehensive overview on the state-of-the-art of AI applications for the treatment of CRC. Notably, studies show that AI can be trained to automatically recognize surgical phases and actions with high accuracy even in complex colorectal procedures such as transanal total mesorectal excision (TaTME). In addition, AI models were trained to interpret fluorescent signals and recognize correct dissection planes during total mesorectal excision (TME), suggesting CV as a potentially valuable tool for intraoperative decision-making and guidance. Finally, AI could have a role in surgical training, providing automatic surgical skills assessment in the operating room. While promising, these proofs of concept require further development, validation in multi-institutional data, and clinical studies to confirm AI as a valuable tool to enhance CRC treatment.

Artificial Intelligence and Device-Assisted Enteroscopy: Automatic Detection of Enteric Protruding Lesions Using a Convolutional Neural Network

INTRODUCTION

Device-assisted enteroscopy (DAE) plays a major role in the investigation and endoscopic treatment of small bowel diseases. Recently, the implementation of artificial intelligence (AI) algorithms to gastroenterology has been the focus of great interest. Our aim was to develop an AI model for the automatic detection of protruding lesions in DAE images.

METHODS

A deep learning algorithm based on a convolutional neural network was designed. Each frame was evaluated for the presence of enteric protruding lesions. The area under the curve, sensitivity, specificity, and positive and negative predictive values were used to assess the performance of the convolutional neural network.

RESULTS

A total of 7,925 images from 72 patients were included. Our model had a sensitivity and specificity of 97.0% and 97.4%, respectively. The area under the curve was 1.00.

DISCUSSION

Our model was able to efficiently detect enteric protruding lesions. The development of AI tools may enhance the diagnostic capacity of deep enteroscopy techniques.

A coaxial excitation, dual-red-green-blue/near-infrared paired imaging system toward computer-aided detection of parathyroid glands in situ and ex vivo

Early and precise detection of parathyroid glands (PGs) is a challenging problem in thyroidectomy due to their small size and similar appearance to surrounding tissues. Near-infrared autofluorescence (NIRAF) has stimulated interest as a method to localize PGs. However, high incidence of false positives for PGs has been reported with this technique. We introduce a prototype equipped with a coaxial excitation light (785 nm) and a dual-sensor to address the issue of false positives with the NIRAF technique. We test the clinical feasibility of our prototype in situ and ex vivo using sterile drapes on 10 human subjects. Video data (1287 images) of detected PGs were collected to train, validate and compare the performance for PG detection. We achieved a mean average precision of 94.7% and a 19.5-millisecond processing time/detection. This feasibility study supports the effectiveness of the optical design and may open new doors for a deep learning-based PG detection method.

Effect of an artificial intelligence-based quality improvement system on efficacy of a computer-aided detection system in colonoscopy: a four-group parallel study

BACKGROUND

Tandem colonoscopy studies have found that about one in five adenomas are missed at colonoscopy. It remains debatable whether the combination of a computer-aided polyp detection (CADe) system with a computer-aided quality improvement (CAQ) system for real-time monitoring of withdrawal speed results in additional benefits in adenoma detection or if the synergetic effect may be harmed due to excessive visual burden resulting from information overload. This study aimed to evaluate the interaction effect on improving the adenoma detection rate (ADR).

METHODS

This single-center, randomized, four-group, parallel, controlled study was performed at Renmin Hospital of Wuhan University. Between 1 July and 15 October 2020, 1076 patients were randomly allocated into four treatment groups: control 271, CADe 268, CAQ 269, and CADe plus CAQ (COMBO) 268. The primary outcome was ADR.

RESULTS

The ADR in the control, CADe, CAQ, and COMBO groups was 14.76 % (95 % confidence interval [CI] 10.54 to 18.98), 21.27 % (95 %CI 16.37 to 26.17), 24.54 % (95 %CI 19.39 to 29.68), and 30.60 % (95 %CI 25.08 to 36.11), respectively. The ADR was higher in the COMBO group compared with the CADe group (21.27 % vs. 30.6 %, P = 0.024, odds ratio [OR] 1.284, 95 %CI 1.033 to 1.596) but not compared with the CAQ group (24.54 % vs. 30.6 %, P = 0.213, OR 1.309, 95 %CI 0.857 to 2.000, respectively).

CONCLUSIONS

CAQ significantly improved the efficacy of CADe in a four-group, parallel, controlled study. No significant difference in the ADR or polyp detection rate was found between CAQ and COMBO.

Clinical application and diagnostic accuracy of artificial intelligence in colonoscopy for inflammatory bowel disease: systematic review

Background and aims  Artificial intelligence (AI) technology is being evaluated for its potential to improve colonoscopic assessment of inflammatory bowel disease (IBD), particularly with computer-aided image classifiers. This review evaluates the clinical application and diagnostic test accuracy (DTA) of AI algorithms in colonoscopy for IBD. Methods  A systematic review was performed on studies evaluating AI in colonoscopy of adult patients with IBD. MEDLINE, Embase, Emcare, PsycINFO, CINAHL, Cochrane Library and Clinicaltrials.gov databases were searched on 28 ^th April 2021 for English language articles published between January 1, 2000 and April 28, 2021. Risk of bias and applicability were assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool. Diagnostic accuracy was presented as median (interquartile range). Results  Of 1029 records screened, nine studies with 7813 patients were included for review. AI was used to predict endoscopic and histologic disease activity in ulcerative colitis, and differentiation of Crohn's disease from Behcet's disease and intestinal tuberculosis. DTA of AI algorithms ranged between 52-91 %. The sensitivity and specificity for AI algorithms predicting endoscopic severity of disease were 78 % (range 72-83, interquartile range 5.5) and 91 % (range 86-96, interquartile range 5), respectively. Conclusions  AI has been primarily used to assess disease activity in ulcerative colitis. The diagnostic performance is promising and suggests potential for other clinical application of AI in IBD colonoscopy such as dysplasia detection. However, current evidence is limited by retrospective data and models trained on still images only. Future prospective multicenter studies with full-motion videos are needed to replicate the real-world clinical setting.

A new artificial intelligence system successfully detects and localises early neoplasia in Barrett's esophagus by using convolutional neural networks

BACKGROUND AND AIMS

Seattle protocol biopsies for Barrett's Esophagus (BE) surveillance are labour intensive with low compliance. Dysplasia detection rates vary, leading to missed lesions. This can potentially be offset with computer aided detection. We have developed convolutional neural networks (CNNs) to identify areas of dysplasia and where to target biopsy.

METHODS

119 Videos were collected in high-definition white light and optical chromoendoscopy with i-scan (Pentax Hoya, Japan) imaging in patients with dysplastic and non-dysplastic BE (NDBE). We trained an indirectly supervised CNN to classify images as dysplastic/non-dysplastic using whole video annotations to minimise selection bias and maximise accuracy. The CNN was trained using 148,936 video frames (31 dysplastic patients, 31 NDBE, two normal esophagus), validated on 25,161 images from 11 patient videos and tested on 264 iscan-1 images from 28 dysplastic and 16 NDBE patients which included expert delineations. To localise targeted biopsies/delineations, a second directly supervised CNN was generated based on expert delineations of 94 dysplastic images from 30 patients. This was tested on 86 i-scan one images from 28 dysplastic patients.

FINDINGS

The indirectly supervised CNN achieved a per image sensitivity in the test set of 91%, specificity 79%, area under receiver operator curve of 93% to detect dysplasia. Per-lesion sensitivity was 100%. Mean assessment speed was 48 frames per second (fps). 97% of targeted biopsy predictions matched expert and histological assessment at 56 fps. The artificial intelligence system performed better than six endoscopists.

INTERPRETATION

Our CNNs classify and localise dysplastic Barrett's Esophagus potentially supporting endoscopists during surveillance.

An artificial intelligence algorithm is highly accurate for detecting endoscopic features of eosinophilic esophagitis

The endoscopic features associated with eosinophilic esophagitis (EoE) may be missed during routine endoscopy. We aimed to develop and evaluate an Artificial Intelligence (AI) algorithm for detecting and quantifying the endoscopic features of EoE in white light images, supplemented by the EoE Endoscopic Reference Score (EREFS). An AI algorithm (AI-EoE) was constructed and trained to differentiate between EoE and normal esophagus using endoscopic white light images extracted from the database of the University Hospital Augsburg. In addition to binary classification, a second algorithm was trained with specific auxiliary branches for each EREFS feature (AI-EoE-EREFS). The AI algorithms were evaluated on an external data set from the University of North Carolina, Chapel Hill (UNC), and compared with the performance of human endoscopists with varying levels of experience. The overall sensitivity, specificity, and accuracy of AI-EoE were 0.93 for all measures, while the AUC was 0.986. With additional auxiliary branches for the EREFS categories, the AI algorithm (AI-EoE-EREFS) performance improved to 0.96, 0.94, 0.95, and 0.992 for sensitivity, specificity, accuracy, and AUC, respectively. AI-EoE and AI-EoE-EREFS performed significantly better than endoscopy beginners and senior fellows on the same set of images. An AI algorithm can be trained to detect and quantify endoscopic features of EoE with excellent performance scores. The addition of the EREFS criteria improved the performance of the AI algorithm, which performed significantly better than endoscopists with a lower or medium experience level.

Impact of Artificial Intelligence on Miss Rate of Colorectal Neoplasia

BACKGROUND & AIMS

Artificial intelligence (AI) may detect colorectal polyps that have been missed due to perceptual pitfalls. By reducing such miss rate, AI may increase the detection of colorectal neoplasia leading to a higher degree of colorectal cancer (CRC) prevention.

METHODS

Patients undergoing CRC screening or surveillance were enrolled in 8 centers (Italy, UK, US), and randomized (1:1) to undergo 2 same-day, back-to-back colonoscopies with or without AI (deep learning computer aided diagnosis endoscopy) in 2 different arms, namely AI followed by colonoscopy without AI or vice-versa. Adenoma miss rate (AMR) was calculated as the number of histologically verified lesions detected at second colonoscopy divided by the total number of lesions detected at first and second colonoscopy. Mean number of lesions detected in the second colonoscopy and proportion of false negative subjects (no lesion at first colonoscopy and at least 1 at second) were calculated. Odds ratios (ORs) and 95% confidence intervals (CIs) were adjusted by endoscopist, age, sex, and indication for colonoscopy. Adverse events were also measured.

RESULTS

A total of 230 subjects (116 AI first, 114 standard colonoscopy first) were included in the study analysis. AMR was 15.5% (38 of 246) and 32.4% (80 of 247) in the arm with AI and non-AI colonoscopy first, respectively (adjusted OR, 0.38; 95% CI, 0.23-0.62). In detail, AMR was lower for AI first for the ≤5 mm (15.9% vs 35.8%; OR, 0.34; 95% CI, 0.21-0.55) and nonpolypoid lesions (16.8% vs 45.8%; OR, 0.24; 95% CI, 0.13-0.43), and it was lower both in the proximal (18.3% vs 32.5%; OR, 0.46; 95% CI, 0.26-0.78) and distal colon (10.8% vs 32.1%; OR, 0.25; 95% CI, 0.11-0.57). Mean number of adenomas at second colonoscopy was lower in the AI-first group as compared with non-AI colonoscopy first (0.33 ± 0.63 vs 0.70 ± 0.97, P < .001). False negative rates were 6.8% (3 of 44 patients) and 29.6% (13 of 44) in the AI and non-AI first arms, respectively (OR, 0.17; 95% CI, 0.05-0.67). No difference in the rate of adverse events was found between the 2 groups.

CONCLUSIONS

AI resulted in an approximately 2-fold reduction in miss rate of colorectal neoplasia, supporting AI-benefit in reducing perceptual errors for small and subtle lesions at standard colonoscopy.

CLINICALTRIALS

gov, Number: NCT03954548.

Deep Learning Computer-aided Polyp Detection Reduces Adenoma Miss Rate: A United States Multi-center Randomized Tandem Colonoscopy Study (CADeT-CS Trial)

BACKGROUND & AIMS

Artificial intelligence-based computer-aided polyp detection (CADe) systems are intended to address the issue of missed polyps during colonoscopy. The effect of CADe during screening and surveillance colonoscopy has not previously been studied in a United States (U.S.) population.

METHODS

We conducted a prospective, multi-center, single-blind randomized tandem colonoscopy study to evaluate a deep-learning based CADe system (EndoScreener, Shanghai Wision AI, China). Patients were enrolled across 4 U.S. academic medical centers from 2019 through 2020. Patients presenting for colorectal cancer screening or surveillance were randomized to CADe colonoscopy first or high-definition white light (HDWL) colonoscopy first, followed immediately by the other procedure in tandem fashion by the same endoscopist. The primary outcome was adenoma miss rate (AMR), and secondary outcomes included sessile serrated lesion (SSL) miss rate and adenomas per colonoscopy (APC).

RESULTS

A total of 232 patients entered the study, with 116 patients randomized to undergo CADe colonoscopy first and 116 patients randomized to undergo HDWL colonoscopy first. After the exclusion of 9 patients, the study cohort included 223 patients. AMR was lower in the CADe-first group compared with the HDWL-first group (20.12% [34/169] vs 31.25% [45/144]; odds ratio [OR], 1.8048; 95% confidence interval [CI], 1.0780-3.0217; P = .0247). SSL miss rate was lower in the CADe-first group (7.14% [1/14]) vs the HDWL-first group (42.11% [8/19]; P = .0482). First-pass APC was higher in the CADe-first group (1.19 [standard deviation (SD), 2.03] vs 0.90 [SD, 1.55]; P = .0323). First-pass ADR was 50.44% in the CADe-first group and 43.64 % in the HDWL-first group (P = .3091).

CONCLUSION

In this U.S. multicenter tandem colonoscopy randomized controlled trial, we demonstrate a decrease in AMR and SSL miss rate and an increase in first-pass APC with the use of a CADe-system when compared with HDWL colonoscopy alone.

A novel AI device for real-time optical characterization of colorectal polyps

Accurate in-vivo optical characterization of colorectal polyps is key to select the optimal treatment regimen during colonoscopy. However, reported accuracies vary widely among endoscopists. We developed a novel intelligent medical device able to seamlessly operate in real-time using conventional white light (WL) endoscopy video stream without virtual chromoendoscopy (blue light, BL). In this work, we evaluated the standalone performance of this computer-aided diagnosis device (CADx) on a prospectively acquired dataset of unaltered colonoscopy videos. An international group of endoscopists performed optical characterization of each polyp acquired in a prospective study, blinded to both histology and CADx result, by means of an online platform enabling careful video assessment. Colorectal polyps were categorized by reviewers, subdivided into 10 experts and 11 non-experts endoscopists, and by the CADx as either “adenoma” or “non-adenoma”. A total of 513 polyps from 165 patients were assessed. CADx accuracy in WL was found comparable to the accuracy of expert endoscopists (CADx_WL/Exp; OR 1.211 [0.766–1.915]) using histopathology as the reference standard. Moreover, CADx accuracy in WL was found superior to the accuracy of non-expert endoscopists (CADx_WL/NonExp; OR 1.875 [1.191–2.953]), and CADx accuracy in BL was found comparable to it (CADx_BL/CADx_WL; OR 0.886 [0.612–1.282]). The proposed intelligent device shows the potential to support non-expert endoscopists in systematically reaching the performances of expert endoscopists in optical characterization.

MiWEndo: Evaluation of a Microwave Colonoscopy Algorithm for Early Colorectal Cancer Detection in Ex Vivo Human Colon Models

This study assesses the efficacy of detecting colorectal cancer precursors or polyps in an ex vivo human colon model with a microwave colonoscopy algorithm. Nowadays, 22% of polyps go undetected with conventional colonoscopy, and the risk of cancer after a negative colonoscopy can be up to 7.9%. We developed a microwave colonoscopy device that consists of a cylindrical ring-shaped switchable microwave antenna array that can be attached to the tip of a conventional colonoscope as an accessory. The accessory is connected to an external unit that allows successive measurements of the colon and processes the measurements with a microwave imaging algorithm. An acoustic signal is generated when a polyp is detected. Fifteen ex vivo freshly excised human colons with cancer (n = 12) or polyps (n = 3) were examined with the microwave-assisted colonoscopy system simulating a real colonoscopy exploration. After the experiment, the dielectric properties of the specimens were measured with a coaxial probe and the samples underwent a pathology analysis. The results show that all the neoplasms were detected with a sensitivity of 100% and specificity of 87.4%.

Advances and horizons for artificial intelligence of endoscopic screening and surveillance of gastric and esophageal disease

The development of artificial intelligence in endoscopic assessment of the gastrointestinal tract has shown progressive enhancement in diagnostic acuity. This review discusses the expanding applications for gastric and esophageal diseases. The gastric section covers the utility of AI in detecting and characterizing gastric polyps and further explores prevention, detection, and classification of gastric cancer. The esophageal discussion highlights applications for use in screening and surveillance in Barrett's esophagus and in high-risk conditions for esophageal squamous cell carcinoma. Additionally, these discussions highlight applications for use in assessing eosinophilic esophagitis and future potential in assessing esophageal microbiome changes.

Artificial intelligence in endoscopy: More than what meets the eye in screening colonoscopy and endosonographic evaluation of pancreatic lesions

Artificial intelligence (AI)-based tools have ushered in a new era of innovation in the field of gastrointestinal (GI) endoscopy. Despite vast improvements in endoscopic techniques and equipment, diagnostic endoscopy remains heavily operator-dependent, in particular, colonoscopy and endoscopic ultrasound (EUS). Recent reports have shown that as much as 25% of colonic adenomas may be missed at colonoscopy. This can result in an increased incidence of interval colon cancer. Similarly, EUS has been shown to have high inter-observer variability, overlap in diagnoses with a relatively low specificity for pancreatic lesions. Our understanding of Machine-learning (ML) techniques in AI have evolved over the last decade and its application in AI–based tools for endoscopic detection and diagnosis is being actively investigated at several centers. ML is an aspect of AI that is based on neural networks, and is widely used for image classification, object detection, and semantic segmentation which are key functional aspects of AI-related computer aided diagnostic systems. In this review, current status and limitations of ML, specifically for adenoma detection and endosonographic diagnosis of pancreatic lesions, will be summarized from existing literature. This will help to better understand its role as viewed through the prism of real world application in the field of GI endoscopy.

Development of a Deep Learning System to Detect Esophageal Cancer by Barium Esophagram

Background

Implementation of deep learning systems (DLSs) for analysis of barium esophagram, a cost-effective diagnostic test for esophageal cancer detection, is expected to reduce the burden to radiologists while ensuring the accuracy of diagnosis.

Objective

To develop an automated DLS to detect esophageal cancer on barium esophagram.

Methods

This was a retrospective study using deep learning for esophageal cancer detection. A two-stage DLS (including a Selection network and a Classification network) was developed. Five datasets based on barium esophagram were used for stepwise training, validation, and testing of the DLS. Datasets 1 and 2 were used to respectively train and test the Selection network, while Datasets 3, 4, and 5 were respectively used to train, validate, and test the Classification network. Finally, a positioning box with a probability value was outputted by the DLS. A region of interest delineated by experienced radiologists was selected as the ground truth to evaluate the detection and classification efficiency of the DLS. Standard machine learning metrics (accuracy, recall, precision, sensitivity, and specificity) were calculated. A comparison with the conventional visual inspection approach was also conducted.

Results

The accuracy, sensitivity, and specificity of our DLS in detecting esophageal cancer were 90.3%, 92.5%, and 88.7%, respectively. With the aid of DLS, the radiologists’ interpretation time was significantly shortened (Reader1, 45.7 s vs. 72.2 s without DLS aid; Reader2, 54.1 s vs. 108.7 s without DLS aid). Respective diagnostic efficiencies for Reader1 with and without DLS aid were 96.8% vs. 89.3% for accuracy, 97.5% vs. 87.5% for sensitivity, 96.2% vs. 90.6% for specificity, and 0.969 vs. 0.890 for AUC. Respective diagnostic efficiencies for Reader2 with and without DLS aid were 95.7% vs. 88.2% for accuracy, 92.5% vs. 77.5% for sensitivity, 98.1% vs. 96.2% for specificity, and 0.953 vs. 0.869 for AUC. Of note, the positioning boxes outputted by the DLS almost overlapped with those manually labeled by the radiologists on Dataset 5.

Conclusions

The proposed two-stage DLS for detecting esophageal cancer on barium esophagram could effectively shorten the interpretation time with an excellent diagnostic performance. It may well assist radiologists in clinical practice to reduce their burden.

Performance of a Deep Learning System for Automatic Diagnosis of Protruding Lesions in Colon Capsule Endoscopy

Background: Colon capsule endoscopy (CCE) is an alternative for patients unwilling or with contraindications for conventional colonoscopy. Colorectal cancer screening may benefit greatly from widespread acceptance of a non-invasive tool such as CCE. However, reviewing CCE exams is a time-consuming process, with risk of overlooking important lesions. We aimed to develop an artificial intelligence (AI) algorithm using a convolutional neural network (CNN) architecture for automatic detection of colonic protruding lesions in CCE images. An anonymized database of CCE images collected from a total of 124 patients was used. This database included images of patients with colonic protruding lesions or patients with normal colonic mucosa or with other pathologic findings. A total of 5715 images were extracted for CNN development. Two image datasets were created and used for training and validation of the CNN. The AUROC for detection of protruding lesions was 0.99. The sensitivity, specificity, PPV and NPV were 90.0%, 99.1%, 98.6% and 93.2%, respectively. The overall accuracy of the network was 95.3%. The developed deep learning algorithm accurately detected protruding lesions in CCE images. The introduction of AI technology to CCE may increase its diagnostic accuracy and acceptance for screening of colorectal neoplasia.

Polyp detection and false-positive rates by computer-aided analysis of withdrawal-phase videos of colonoscopy of the right-sided colon segment in a randomized controlled trial comparing water exchange and air insufflation

BACKGROUND AND AIMS

Water exchange (WE) improves lesion detection but misses polyps because of human limitations. Computer-aided detection (CADe) identifies additional polyps overlooked by the colonoscopist. Additional polyp detection rate (APDR) is the proportion of patients with at least 1 additional polyp detected by CADe. The number of false positives (because of feces and air bubble) per colonoscopy (FPPC) is a major CADe limitation, which might be reduced by salvage cleaning with WE. We compared the APDR and FPPC by CADe between videos of WE and air insufflation in the right-sided colon.

METHODS

CADe used a convolutional neural network with transfer learning. We edited and coded withdrawal-phase videos in a randomized controlled trial that compared right-sided colon findings between air insufflation and WE. Two experienced blinded endoscopists analyzed the CADe-overlaid videos and identified additional polyps by consensus. An artifact triggered by CADe but not considered a polyp by the reviewers was defined as a false positive. The primary outcome was APDR.

RESULTS

Two hundred forty-five coded videos of colonoscopies inserted with WE (n = 123) and air insufflation (n = 122) methods were analyzed. The APDR in the WE group was significantly higher (37 [30.1%] vs 15 [12.3%], P = .001). The mean [standard deviation] FPPC related to feces (1.78 [1.67] vs 2.09 [2.09], P = .007) and bubbles (.53 [.89] vs 1.25 [2.45], P = .001) in the WE group were significantly lower.

CONCLUSIONS

CADe showed significantly higher APDR and lower number of FPPC related to feces and bubbles in the WE group. The results support the hypothesis that the strengths of CADe and WE complement the weaknesses of each other in optimizing polyp detection.

Human colorectal cancer tissue assessment using optical coherence tomography catheter and deep learning

Optical coherence tomography (OCT) can differentiate normal colonic mucosa from neoplasia, potentially offering a new mechanism of endoscopic tissue assessment and biopsy targeting, with a high optical resolution and an imaging depth of ~1 mm. Recent advances in convolutional neural networks (CNN) have enabled application in ophthalmology, cardiology, and gastroenterology malignancy detection with high sensitivity and specificity. Here, we describe a miniaturized OCT catheter and a residual neural network (ResNet)-based deep learning model manufactured and trained to perform automatic image processing and real-time diagnosis of the OCT images. The OCT catheter has an outer diameter of 3.8 mm, a lateral resolution of ~7 μm, and an axial resolution of ~6 μm. A customized ResNet is utilized to classify OCT catheter colorectal images. An area under the receiver operating characteristic (ROC) curve (AUC) of 0.975 is achieved to distinguish between normal and cancerous colorectal tissue images.

Artificial intelligence implementation in pancreaticobiliary endoscopy

INTRODUCTION

Artificial intelligence has been rapidly deployed in gastroenterology and endoscopy. The acceleration of deep convolutional neural networks along with hardware development has allowed implementation of artificial intelligence algorithms into real-time endoscopy, particularly colonoscopy. However, artificial intelligence implementation in pancreaticobiliary endoscopy is nascent.

AREAS COVERED

Initial studies have been conducted in endoscopic retrograde pancreatography (ERCP), endoscopic ultrasound (EUS), and digital single operator cholangioscopy (DSOC). Machine learning has been implemented in identifying significant landmarks, including the ampulla on ERCP, and the bile duct, pancreas, and portal confluence on EUS. Moreover, artificial intelligence algorithms have been deployed in differentiating pathology including pancreas cancer, autoimmune pancreatitis, pancreatic cystic lesions, and biliary strictures.

EXPERT OPINION

There have been relatively few studies with limited sample sizes in developing these machine learning algorithms. Despite the early successful demonstration of artificial intelligence in pancreaticobiliary endoscopy, additional research needs to be conducted with larger data sets to improve generalizability and assessed in real-time endoscopy before clinical implementation. However, pancreaticobiliary endoscopy remains a promising avenue of artificial intelligence application with the potential to improve clinical practice and outcomes.

Preparation of image databases for artificial intelligence algorithm development in gastrointestinal endoscopy

Over the past decade, technological advances in deep learning have led to the introduction of artificial intelligence (AI) in medical imaging. The most commonly used structure in image recognition is the convolutional neural network, which mimics the action of the human visual cortex. The applications of AI in gastrointestinal endoscopy are diverse. Computer-aided diagnosis has achieved remarkable outcomes with recent improvements in machine-learning techniques and advances in computer performance. Despite some hurdles, the implementation of AI-assisted clinical practice is expected to aid endoscopists in real-time decision-making. In this summary, we reviewed state-of-the-art AI in the field of gastrointestinal endoscopy and offered a practical guide for building a learning image dataset for algorithm development.

Current Status and Future Perspectives of Artificial Intelligence in Colonoscopy

The early endoscopic identification, resection, and treatment of precancerous adenoma and early-stage cancer has been shown to reduce not only the prevalence of colorectal cancer but also its mortality rate. Recent advances in endoscopic devices and imaging technology have dramatically improved our ability to detect colorectal lesions and predict their pathological diagnosis. In addition to this, rapid advances in artificial intelligence (AI) technology mean that AI-related research and development is now progressing in the diagnostic imaging field, particularly colonoscopy, and AIs (i.e., devices that mimic cognitive abilities, such as learning and problem-solving) already approved as medical devices are now being introduced into everyday clinical practice. Today, there is an increasing expectation that sophisticated AIs will be able to provide high-level diagnostic performance irrespective of the level of skill of the endoscopist. In this paper, we review colonoscopy-related AI research and the AIs that have already been approved and discuss the future prospects of this technology.

Scoping out the future: The application of artificial intelligence to gastrointestinal endoscopy

Artificial intelligence (AI) is a quickly expanding field in gastrointestinal endoscopy. Although there are a myriad of applications of AI ranging from identification of bleeding to predicting outcomes in patients with inflammatory bowel disease, a great deal of research has focused on the identification and classification of gastrointestinal malignancies. Several of the initial randomized, prospective trials utilizing AI in clinical medicine have centered on polyp detection during screening colonoscopy. In addition to work focused on colorectal cancer, AI systems have also been applied to gastric, esophageal, pancreatic, and liver cancers. Despite promising results in initial studies, the generalizability of most of these AI systems have not yet been evaluated. In this article we review recent developments in the field of AI applied to gastrointestinal oncology.

Real-time, computer-aided, detection-assisted colonoscopy eliminates differences in adenoma detection rate between trainee and experienced endoscopists

Background and study aims  Adenoma detection rate (ADR) is a well-accepted quality indicator of screening colonoscopy. In recent years, the added value of artificial intelligence (AI) has been demonstrated in terms of ADR and adenoma miss rate (AMR). To date, there are no studies evaluating the impact of AI on the performance of trainee endoscopists (TEs). This study aimed to assess whether AI might eliminate any difference in ADR or AMR between TEs and experienced endoscopists (EEs). Patients and methods  We performed a prospective observational study in 45 subjects referred for screening colonoscopy. A same-day tandem examination was carried out for each patient by a TE with the AI assistance and subsequently by an EE unaware of the lesions detected by the TE. Besides ADR and AMR, we also calculated for each subgroup of endoscopists the adenoma per colonoscopy (APC), polyp detection rate (PDR), polyp per colonoscopy (PPC) and polyp miss rate (PMR). Subgroup analyses according to size, morphology, and site were also performed. Results  ADR, APC, PDR, and PPC of AI-supported TEs were 38 %, 0.93, 62 %, 1.93, respectively. The corresponding parameters for EEs were 40 %, 1.07, 58 %, 2.22. No significant difference was found for each analysis between the two groups ( P  > 0.05). AMR and PMR for AI-assisted TEs were 12.5 % and 13 %, respectively. Sub-analyses did not show any significant difference ( P  > 0.05) between the two categories of operators. Conclusions  In this single-center prospective study, the possible impact of AI on endoscopist quality training was demonstrated. In the future, this could result in better efficacy of screening colonoscopy by reducing the incidence of interval or missed cancers.

Does computer-aided diagnostic endoscopy improve the detection of commonly missed polyps? A meta-analysis

Background/Aims

Colonoscopy is the gold standard diagnostic method for colorectal neoplasia, allowing detection and resection of adenomatous polyps; however, significant proportions of adenomas are missed. Computer-aided detection (CADe) systems in endoscopy are currently available to help identify lesions. Diminutive (≤5 mm) and nonpedunculated polyps are most commonly missed. This meta-analysis aimed to assess whether CADe systems can improve the real-time detection of these commonly missed lesions.

Methods

A comprehensive literature search was performed. Randomized controlled trials evaluating CADe systems categorized by morphology and lesion size were included. The mean number of polyps and adenomas per patient was derived. Independent proportions and their differences were calculated using DerSimonian and Laird random-effects modeling.

Results

Seven studies, including 2,595 CADe-assisted colonoscopies and 2,622 conventional colonoscopies, were analyzed. CADe-assisted colonoscopy demonstrated an 80% increase in the mean number of diminutive adenomas detected per patient compared with conventional colonoscopy (0.31 vs. 0.17; effect size, 0.13; 95% confidence interval [CI], 0.09–0.18); it also demonstrated a 91.7% increase in the mean number of nonpedunculated adenomas detected per patient (0.32 vs. 0.19; effect size, 0.05; 95% CI, 0.02–0.07).

Conclusions

CADe-assisted endoscopy significantly improved the detection of most commonly missed adenomas. Although this method is a potentially exciting technology, limitations still apply to current data, prompting the need for further real-time studies.

Artificial Intelligence in Laryngeal Endoscopy: Systematic Review and Meta-Analysis

Background: Early diagnosis of laryngeal lesions is necessary to begin treatment of patients as soon as possible to preserve optimal organ functions. Imaging examinations are often aided by artificial intelligence (AI) to improve quality and facilitate appropriate diagnosis. The aim of this study is to investigate diagnostic utility of AI in laryngeal endoscopy. Methods: Five databases were searched for studies implementing artificial intelligence (AI) enhanced models assessing images of laryngeal lesions taken during laryngeal endoscopy. Outcomes were analyzed in terms of accuracy, sensitivity, and specificity. Results: All 11 studies included presented an overall low risk of bias. The overall accuracy of AI models was very high (from 0.806 to 0.997). The accuracy was significantly higher in studies using a larger database. The pooled sensitivity and specificity for identification of healthy laryngeal tissue were 0.91 and 0.97, respectively. The same values for differentiation between benign and malignant lesions were 0.91 and 0.94, respectively. The comparison of the effectiveness of AI models assessing narrow band imaging and white light endoscopy images revealed no statistically significant differences (p = 0.409 and 0.914). Conclusion: In assessing images of laryngeal lesions, AI demonstrates extraordinarily high accuracy, sensitivity, and specificity.

Development and evaluation of a deep learning model to improve the usability of polyp detection systems during interventions

Background

The efficiency of artificial intelligence as computer‐aided detection (CADe) systems for colorectal polyps has been demonstrated in several randomized trials. However, CADe systems generate many distracting detections, especially during interventions such as polypectomies. Those distracting CADe detections are often induced by the introduction of snares or biopsy forceps as the systems have not been trained for such situations. In addition, there are a significant number of non‐false but not relevant detections, since the polyp has already been previously detected. All these detections have the potential to disturb the examiner's work.

Objectives

Development and evaluation of a convolutional neuronal network that recognizes instruments in the endoscopic image, suppresses distracting CADe detections, and reliably detects endoscopic interventions.

Methods

A total of 580 different examination videos from 9 different centers using 4 different processor types were screened for instruments and represented the training dataset (519,856 images in total, 144,217 contained a visible instrument). The test dataset included 10 full‐colonoscopy videos that were analyzed for the recognition of visible instruments and detections by a commercially available CADe system (GI Genius, Medtronic).

Results

The test dataset contained 153,623 images, 8.84% of those presented visible instruments (12 interventions, 19 instruments used). The convolutional neuronal network reached an overall accuracy in the detection of visible instruments of 98.59%. Sensitivity and specificity were 98.55% and 98.92%, respectively. A mean of 462.8 frames containing distracting CADe detections per colonoscopy were avoided using the convolutional neuronal network. This accounted for 95.6% of all distracting CADe detections.

Conclusions

Detection of endoscopic instruments in colonoscopy using artificial intelligence technology is reliable and achieves high sensitivity and specificity. Accordingly, the new convolutional neuronal network could be used to reduce distracting CADe detections during endoscopic procedures. Thus, our study demonstrates the great potential of artificial intelligence technology beyond mucosal assessment.

Diagnostic accuracy of a novel artificial intelligence system for adenoma detection in daily practice: a prospective nonrandomized comparative study

BACKGROUND

Adenoma detection rate (ADR) varies significantly between endoscopists, with adenoma miss rates (AMRs) up to 26 %. Artificial intelligence (AI) systems may improve endoscopy quality and reduce the rate of interval cancer. We evaluated the efficacy of an AI system in real-time colonoscopy and its influence on AMR and ADR.

METHODS

This prospective, nonrandomized, comparative study analyzed patients undergoing diagnostic colonoscopy at a single endoscopy center in Germany from June to October 2020. Every patient was examined concurrently by an endoscopist and AI using two opposing screens. The AI system, overseen by a second observer, was not visible to the endoscopist. AMR was the primary outcome. Both methods were compared using McNemar test.

RESULTS

150 patients were included (mean age 65 years [standard deviation 14]; 69 women). There was no significant or clinically relevant difference (P = 0.75) in AMR between the AI system (6/197, 3.0 %; 95 % confidence interval [CI] 1.1-6.5) and routine colonoscopy (4/197, 2.0 %; 95 %CI 0.6-5.1). The polyp miss rate of the AI system (14/311, 4.5 %; 95 %CI 2.5-7.4) was not significantly different (P = 0.72) from routine colonoscopy (17/311, 5.5 %; 95 %CI 3.2-8.6). There was no significant difference (P = 0.50) in ADR between routine colonoscopy (78/150, 52.0 %; 95 %CI 43.7-60.2) and the AI system (76/150, 50.7 %; 95 %CI 42.4-58.9). Routine colonoscopy detected adenomas in two patients that were missed by the AI system.

CONCLUSION

The AI system performance was comparable to that of experienced endoscopists during real-time colonoscopy with similar high ADR (> 50 %).

Comparing the number and relevance of false activations between 2 artificial intelligence computer-aided detection systems: the NOISE study

BACKGROUND AND AIMS

Artificial intelligence has been shown to be effective in polyp detection, and multiple computer-aided detection (CADe) systems have been developed. False-positive (FP) activation emerged as a possible way to benchmark CADe performance in clinical practice. The aim of this study was to validate a previously developed classification of FPs comparing the performances of different brands of approved CADe systems.

METHODS

We compared 2 different consecutive video libraries (40 video per arm) collected at Humanitas Research Hospital with 2 different CADe system brands (CADe A and CADe B). For each video, the number of CADe false activations, cause, and time spent by the endoscopist to examine the area erroneously highlighted were reported. The FP activations were classified according to the previously developed classification of FPs (the NOISE classification) according to their cause and relevance.

RESULTS

In CADe A 1021 FP activations were registered across the 40 videos (25.5 ± 12.2 FPs per colonoscopy), whereas in CADe B 1028 were identified (25.7 ± 13.2 FPs per colonoscopy; P = .53). Among them, 22.9 ± 9.9 (89.8% in CADe A) and 22.1 ± 10.0 (86.0% in CADe B) were because of artifacts from the bowel wall. Conversely, 2.6 ± 1.9 (10.2% in CADe A) and 3.5 ± 2.1 (14% in CADe B) were caused by bowel content (P = .45). Within CADe A each false activation required .2 ± .9 seconds, with 1.6 ± 1.0 FPs (6.3%) requiring additional time for endoscopic assessment. Comparable results were reported within CADe B with .2 ± .8 seconds spent per false activation and 1.8 ± 1.2 FPs per colonoscopy requiring additional inspection.

CONCLUSIONS

The use of a standardized nomenclature provided comparable results with either of the 2 recently approved CADe systems. (Clinical trial registration number: NCT04399590.).

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Advances in screening and detection of gastric cancer

With an estimated one million new cases and 769 000 deaths in 2020, gastric cancer is the fifth most frequent cancer and fourth leading cause of cancer death globally. Incidence rates are highest in Asia and Eastern Europe. This manuscript will review the current modalities of diagnosis, staging, and screening of gastric cancer. We will also highlight development of novel diagnostics and advancements in endoscopic detection of early gastric cancer.

Paper-based genetic assays with bioconjugated gold nanorods and an automated readout pipeline

Paper-based biosensors featuring immunoconjugated gold nanoparticles have gained extraordinary momentum in recent times as the platform of choice in key cases of field applications, including the so-called rapid antigen tests for SARS-CoV-2. Here, we propose a revision of this format, one that may leverage on the most recent advances in materials science and data processing. In particular, we target an amplifiable DNA rather than a protein analyte, and we replace gold nanospheres with anisotropic nanorods, which are intrinsically brighter by a factor of ~ 10, and multiplexable. By comparison with a gold-standard method for dot-blot readout with digoxigenin, we show that gold nanorods entail much faster and easier processing, at the cost of a higher limit of detection (from below 1 to 10 ppm in the case of plasmid DNA containing a target transgene, in our current setup). In addition, we test a complete workflow to acquire and process photographs of dot-blot membranes with custom-made hardware and regression tools, as a strategy to gain more analytical sensitivity and potential for quantification. A leave-one-out approach for training and validation with as few as 36 sample instances already improves the limit of detection reached by the naked eye by a factor around 2. Taken together, we conjecture that the synergistic combination of new materials and innovative tools for data processing may bring the analytical sensitivity of paper-based biosensors to approach the level of lab-grade molecular tests.

Convolutional Neural Networks for Mechanistic Driver Detection in Atrial Fibrillation

The maintaining and initiating mechanisms of atrial fibrillation (AF) remain controversial. Deep learning is emerging as a powerful tool to better understand AF and improve its treatment, which remains suboptimal. This paper aims to provide a solution to automatically identify rotational activity drivers in endocardial electrograms (EGMs) with convolutional recurrent neural networks (CRNNs). The CRNN model was compared with two other state-of-the-art methods (SimpleCNN and attention-based time-incremental convolutional neural network (ATI-CNN)) for different input signals (unipolar EGMs, bipolar EGMs, and unipolar local activation times), sampling frequencies, and signal lengths. The proposed CRNN obtained a detection score based on the Matthews correlation coefficient of 0.680, an ATI-CNN score of 0.401, and a SimpleCNN score of 0.118, with bipolar EGMs as input signals exhibiting better overall performance. In terms of signal length and sampling frequency, no significant differences were found. The proposed architecture opens the way for new ablation strategies and driver detection methods to better understand the AF problem and its treatment.

Artificial Intelligence in Digestive Endoscopy—Where Are We and Where Are We Going?

Artificial intelligence, a computer-based concept that tries to mimic human thinking, is slowly becoming part of the endoscopy lab. It has developed considerably since the first attempt at developing an automated medical diagnostic tool, today being adopted in almost all medical fields, digestive endoscopy included. The detection rate of preneoplastic lesions (i.e., polyps) during colonoscopy may be increased with artificial intelligence assistance. It has also proven useful in detecting signs of ulcerative colitis activity. In upper digestive endoscopy, deep learning models may prove to be useful in the diagnosis and management of upper digestive tract diseases, such as gastroesophageal reflux disease, Barrett’s esophagus, and gastric cancer. As is the case with all new medical devices, there are challenges in the implementation in daily medical practice. The regulatory, economic, organizational culture, and language barriers between humans and machines are a few of them. Even so, many devices have been approved for use by their respective regulators. Future studies are currently striving to develop deep learning models that can replicate a growing amount of human brain activity. In conclusion, artificial intelligence may become an indispensable tool in digestive endoscopy.

Performance of Convolutional Neural Networks for Polyp Localization on Public Colonoscopy Image Datasets

Colorectal cancer is one of the most frequent malignancies. Colonoscopy is the de facto standard for precancerous lesion detection in the colon, i.e., polyps, during screening studies or after facultative recommendation. In recent years, artificial intelligence, and especially deep learning techniques such as convolutional neural networks, have been applied to polyp detection and localization in order to develop real-time CADe systems. However, the performance of machine learning models is very sensitive to changes in the nature of the testing instances, especially when trying to reproduce results for totally different datasets to those used for model development, i.e., inter-dataset testing. Here, we report the results of testing of our previously published polyp detection model using ten public colonoscopy image datasets and analyze them in the context of the results of other 20 state-of-the-art publications using the same datasets. The F1-score of our recently published model was 0.88 when evaluated on a private test partition, i.e., intra-dataset testing, but it decayed, on average, by 13.65% when tested on ten public datasets. In the published research, the average intra-dataset F1-score is 0.91, and we observed that it also decays in the inter-dataset setting to an average F1-score of 0.83.

Quality Indicators for the Detection and Removal of Colorectal Polyps and Interventions to Improve Them

Modifiable risk factors for postcolonoscopy colorectal cancer include suboptimal lesion detection (missed neoplasms) and inadequate lesion removal (incomplete polypectomy) during colonoscopy. Competent detection and removal of colorectal polyps are thus fundamental to ensuring adequate colonoscopy quality. Several well-researched quality metrics for polyp detection have been implemented into clinical practice, chief among these the adenoma detection rate. Less data are available on quality indicators for polyp removal, which currently include complete resection rates and skills assessment tools. This review summarizes the available literature on quality indicators for the detection and removal of colorectal polyps, as well as interventions to improve them.

Artificial Intelligence for Disease Assessment in Inflammatory Bowel Disease: How Will it Change Our Practice?

Artificial intelligence (AI) has arrived and it will directly impact how we assess, monitor, and manage inflammatory bowel disease (IBD). Advances in the machine learning methodologies that power AI have produced astounding results for replicating expert judgment and predicting clinical outcomes, particularly in the analysis of imaging. This review will cover general concepts for AI in IBD, with descriptions of common machine learning methods, including decision trees and neural networks. Applications of AI in IBD will cover recent achievements in endoscopic image interpretation and scoring, new capabilities for cross-sectional image analysis, natural language processing for automated understanding of clinical text, and progress in AI-powered clinical decision support tools. In addition to detailing current evidence supporting the capabilities of AI for replicating expert clinical judgment, speculative commentary on how AI may advance concepts of disease activity assessment, care pathways, and pathophysiologic mechanisms of IBD will be addressed.

Artificial intelligence in colonoscopy: A review on the current status

Artificial intelligence has become an increasingly hot topic in the last several years, and it has also gained its way into the medical field. In recent years, the application of artificial intelligence in the gastroenterology field has been of increasing interest, particularly in the colonoscopy setting. Novel technologies such as deep neural networks have enabled real‐time computer‐aided polyp detection and diagnosis during colonoscopy. This might lead to increased performance of endoscopists as well as potentially reducing the costs of unnecessary polypectomies of hyperplastic polyps. Newly published prospective trials studying computer‐aided detection showed that the assistance of artificial intelligence significantly increased the detection of polyps and non‐advanced adenomas approximately by 10%, while three tandem randomized control trials proved that the adenoma miss rate was significantly reduced (e.g., 13.8% vs. 36.7% in one Japanese multicenter trial). Promising results have also been shown in prospective single‐arm trials on computer‐aided polyp diagnosis, but the evidence is insufficient to reach a conclusion.

Implementation of artificial intelligence in upper gastrointestinal endoscopy

The application of artificial intelligence (AI) using deep learning has significantly expanded in the field of esophagogastric endoscopy. Recent studies have shown promising results in detecting and differentiating early gastric cancer using AI tools built using white light, magnified, or image‐enhanced endoscopic images. Some studies have reported the use of AI tools to predict the depth of early gastric cancer based on endoscopic images. Similarly, studies based on using AI for detecting early esophageal cancer have also been reported, with an accuracy comparable to that of endoscopy specialists. Moreover, an AI system, developed to diagnose pharyngeal cancer, has shown promising performance with high sensitivity. These reports suggest that, if introduced for regular use in clinical settings, AI systems can significantly reduce the burden on physicians. This review summarizes the current status of AI applications in the upper gastrointestinal tract and presents directions for clinical practice implementation and future research.

Artificial intelligence and colonoscopy experience: lessons from two randomised trials

BACKGROUND AND AIMS

Artificial intelligence has been shown to increase adenoma detection rate (ADR) as the main surrogate outcome parameter of colonoscopy quality. To which extent this effect may be related to physician experience is not known. We performed a randomised trial with colonoscopists in their qualification period (AID-2) and compared these data with a previously published randomised trial in expert endoscopists (AID-1).

METHODS

In this prospective, randomised controlled non-inferiority trial (AID-2), 10 non-expert endoscopists (<2000 colonoscopies) performed screening/surveillance/diagnostic colonoscopies in consecutive 40-80 year-old subjects using high-definition colonoscopy with or without a real-time deep-learning computer-aided detection (CADe) (GI Genius, Medtronic). The primary outcome was ADR in both groups with histology of resected lesions as reference. In a post-hoc analysis, data from this randomised controlled trial (RCT) were compared with data from the previous AID-1 RCT involving six experienced endoscopists in an otherwise similar setting.

RESULTS

In 660 patients (62.3±10 years; men/women: 330/330) with equal distribution of study parameters, overall ADR was higher in the CADe than in the control group (53.3% vs 44.5%; relative risk (RR): 1.22; 95% CI: 1.04 to 1.40; p<0.01 for non-inferiority and p=0.02 for superiority). Similar increases were seen in adenoma numbers per colonoscopy and in small and distal lesions. No differences were observed with regards to detection of non-neoplastic lesions. When pooling these data with those from the AID-1 study, use of CADe (RR 1.29; 95% CI: 1.16 to 1.42) and colonoscopy indication, but not the level of examiner experience (RR 1.02; 95% CI: 0.89 to 1.16) were associated with ADR differences in a multivariate analysis.

CONCLUSIONS

In less experienced examiners, CADe assistance during colonoscopy increased ADR and a number of related polyp parameters as compared with the control group. Experience appears to play a minor role as determining factor for ADR.

TRIAL REGISTRATION NUMBER

NCT:04260321.

Detection of colorectal lesions during colonoscopy

Owing to its high mortality rate, the prevention of colorectal cancer is of particular importance. The resection of colorectal polyps is reported to drastically reduce colorectal cancer mortality, and examination by endoscopists who had a high adenoma detection rate was found to lower the risk of colorectal cancer, highlighting the importance of identifying lesions. Various devices, imaging techniques, and diagnostic tools aimed at reducing the rate of missed lesions have therefore been developed to improve detection. The distal attachments and devices for improving the endoscopic view angle are intended to help avoid missing blind spots such as folds and flexures in the colon, whereas the imaging techniques represented by image‐enhanced endoscopy contribute to improving lesion visibility. Recent advances in artificial intelligence‐supported detection systems are expected to supplement an endoscopist's eye through the instant diagnosis of the lesions displayed on the monitor. In this review, we provide an outline of each tool and assess its impact on the reduction in the incidence of missed colorectal polyps by summarizing previous clinical research and meta‐analyses. Although useful, the many devices, image‐enhanced endoscopy, and artificial intelligence tools exhibited various limitations. Integrating these tools can improve their shortcomings. Combining artificial intelligence‐based diagnoses with wide‐angle image‐enhanced endoscopy may be particularly useful. Thus, we hope that such tools will be available in the near future.

Discovering the first US FDA-approved computer-aided polyp detection system

Colorectal cancer is the third most common cancer worldwide. Because of the slow progression of the precancerous precursors, an efficient endoscopic surveillance strategy may be expected. It seems that around one-fourth of colorectal malignancies are still missed during colonoscopy. Several endoscopic technologies have been introduced, without radical changes. Interest in the development of artificial intelligence applications in the medical field has grown in the past decade. Artificial intelligence can help to highlight a specific region of interest that needs closer examination for the identification of polyps. The aim of this review is to report the first clinical experiences with the first US FDA-approved, real-time, deep-learning, computer-aided detection system (GI Genius™, Medtronic).

The influence of computer-aided polyp detection systems on reaction time for polyp detection and eye gaze

BACKGROUND

Multiple computer-aided systems for polyp detection (CADe) have been introduced into clinical practice, with an unclear effect on examiner behavior. This study aimed to measure the influence of a CADe system on reaction time, mucosa misinterpretation, and changes in visual gaze pattern.

METHODS

Participants with variable levels of colonoscopy experience viewed video sequences (n = 29) while eye movement was tracked. Using a crossover design, videos were presented in two assessments, with and without CADe support. Reaction time for polyp detection and eye-tracking metrics were evaluated.

RESULTS

21 participants performed 1218 experiments. CADe was significantly faster in detecting polyps compared with participants (median 1.16 seconds [99 %CI 0.40-3.43] vs. 2.97 seconds [99 %CI 2.53-3.77], respectively). However, the reaction time of participants when using CADe (median 2.90 seconds [99 %CI 2.55-3.38]) was similar to that without CADe. CADe increased misinterpretation of normal mucosa and reduced the eye travel distance.

CONCLUSIONS

Results confirm that CADe systems detect polyps faster than humans. However, use of CADe did not improve human reaction times. It increased misinterpretation of normal mucosa and decreased the eye travel distance. Possible consequences of these findings might be prolonged examination time and deskilling.

Initial Responses to False Positives in AI-Supported Continuous Interactions: A Colonoscopy Case Study

The use of artificial intelligence (AI) in clinical support systems is increasing. In this article, we focus on AI support for continuous interaction scenarios. A thorough understanding of end-user behaviour during these continuous human-AI interactions, in which user input is sustained over time and during which AI suggestions can appear at any time, is still missing. We present a controlled lab study involving 21 endoscopists and an AI colonoscopy support system. Using a custom-developed application and an off-the-shelf videogame controller, we record participants’ navigation behaviour and clinical assessment across 14 endoscopic videos. Each video is manually annotated to mimic an AI recommendation, being either true positive or false positive in nature. We find that time between AI recommendation and clinical assessment is significantly longer for incorrect assessments. Further, the type of medical content displayed significantly affects decision time. Finally, we discover that the participant’s clinical role plays a large part in the perception of clinical AI support systems. Our study presents a realistic assessment of the effects of imperfect and continuous AI support in a clinical scenario.

Artificial Intelligence for Colonoscopy: Past, Present, and Future

During the past decades, many automated image analysis methods have been developed for colonoscopy. Real-time implementation of the most promising methods during colonoscopy has been tested in clinical trials, including several recent multi-center studies. All trials have shown results that may contribute to prevention of colorectal cancer. We summarize the past and present development of colonoscopy video analysis methods, focusing on two categories of artificial intelligence (AI) technologies used in clinical trials. These are (1) analysis and feedback for improving colonoscopy quality and (2) detection of abnormalities. Our survey includes methods that use traditional machine learning algorithms on carefully designed hand-crafted features as well as recent deep-learning methods. Lastly, we present the gap between current state-of-the-art technology and desirable clinical features and conclude with future directions of endoscopic AI technology development that will bridge the current gap.

Software Analysis of Colonoscopy Videos Enhances Teaching and Quality Metrics

Purpose

Machine learning algorithms were hypothesized as being able to predict the quality of colonoscopy luminal images. This is to enhance training and quality indicators in endoscopy.

Methods

A separate study involving a randomized controlled trial of capped vs. un-capped colonoscopies provided the colonoscopy videos for this study. Videos were analyzed with an algorithm devised by the Australian Institute for Machine Learning. The image analysis validated focus measure, steerable filters-based metrics (SFIL), was used to assess luminal visualization quality and was compared with two independent clinician assessments (C1 and C2). Goodman and Kruskal's gamma (G) measure was used to assess rank correlation data using IBM SPSS Statistics for Windows, version 25.0 (IBM Corp., Armonk, NY).

Results

A total of 500 random colonoscopy video clips were extracted and analyzed, 88 being excluded. SFIL scores matched with C1 in 45% and C2 in 42% of cases, respectively. There was a significant correlation between SFIL and C1 (G = 0.644, p < 0.005) and SFIL and C2 (G = 0.734, p < 0.005).

Conclusion

This study demonstrates that machine learning algorithms can recognize the quality of luminal visualization during colonoscopy. We intend to apply this in the future to enhance colonoscopy training and as a metric for quality assessment.