Polyp-Alert: near real-time feedback during colonoscopy

Quality assurance of computer-aided detection and diagnosis in colonoscopy

Recent breakthroughs in artificial intelligence (AI), specifically via its emerging sub-field "deep learning," have direct implications for computer-aided detection and diagnosis (CADe and/or CADx) for colonoscopy. AI is expected to have at least 2 major roles in colonoscopy practice-polyp detection (CADe) and polyp characterization (CADx). CADe has the potential to decrease the polyp miss rate, contributing to improving adenoma detection, whereas CADx can improve the accuracy of colorectal polyp optical diagnosis, leading to reduction of unnecessary polypectomy of non-neoplastic lesions, potential implementation of a resect-and-discard paradigm, and proper application of advanced resection techniques. A growing number of medical-engineering researchers are developing both CADe and CADx systems, some of which allow real-time recognition of polyps or in vivo identification of adenomas, with over 90% accuracy. However, the quality of the developed AI systems as well as that of the study designs vary significantly, hence raising some concerns regarding the generalization of the proposed AI systems. Initial studies were conducted in an exploratory or retrospective fashion by using stored images and likely overestimating the results. These drawbacks potentially hinder smooth implementation of this novel technology into colonoscopy practice. The aim of this article is to review both contributions and limitations in recent machine-learning-based CADe and/or CADx colonoscopy studies and propose some principles that should underlie system development and clinical testing.

Randomized, double-blinded, placebo-controlled trial evaluating simethicone pretreatment with bowel preparation during colonoscopy

BACKGROUND

The presence of small air bubbles and foam are an impediment to a successful colonoscopy. They impair an endoscopist’s view and diminish the diagnostic accuracy of the study. This has been particularly noted to be of concern with the switch to lower volume polyethylene glycol (PEG) and bisacodyl combination preparation.

AIM

To evaluate the effect of oral simethicone addition to bowel preparation on intraluminal bubbles reduction during colonoscopy.

METHODS

Described is a prospective, randomized, multi-center, double-blinded, placebo-controlled study to evaluate the use of premixed simethicone formulation with split-regimen, low-volume PEG-bisacodyl combination bowel preparation for 168 outpatients undergoing screening, surveillance, and diagnostic colonoscopies. Primary outcome includes evaluation of bubbles during colonoscopy graded using the Intraluminal Bubbles Scale. Secondary outcomes include evaluation of the Boston Bowel Preparation Scale (BBPS), total number of polyps, polyp size differentiation, polyp laterality, adenoma detection, mass detection, cecal insertion time, withdrawal time, and patient-reported adverse events.

RESULTS

Higher Intraluminal Bubbles grades III and IV (less than 75% of the mucosa cleared of bubbles/foam requiring intervention with simethicone infused wash) were detected in the placebo group [Simethicone n = 4/84 vs Placebo n = 20/84 (P = 0.007)]. BBPS total score was 7.42 [standard deviation (SD) = ± 1.51] in the simethicone group and 7.28 (SD = ± 1.44) in the placebo group (P = 0.542) from a total of 9. Significantly higher number of adenomas were detected in the simethicone group (P = 0.001).

CONCLUSION

The addition of simethicone to bowel preparation is well advised for its anti-foaming properties. The results of this study suggest that addition of oral simethicone can improve bowel wall visibility.

Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions

Computer-aided diagnosis offers a promising solution to reduce variation in colonoscopy performance. Pooled miss rates for polyps are as high as 22%, and associated interval colorectal cancers after colonoscopy are of concern. Optical biopsy, whereby in-vivo classification of polyps based on enhanced imaging replaces histopathology, has not been incorporated into routine practice because it is limited by interobserver variability and generally only meets accepted standards in expert settings. Real-time decision-support software has been developed to detect and characterise polyps, and also to offer feedback on the technical quality of inspection. Some of the current algorithms, particularly with recent advances in artificial intelligence techniques, match human expert performance for optical biopsy. In this Review, we summarise the evidence for clinical applications of computer-aided diagnosis and artificial intelligence in colonoscopy.

Artificial intelligence in gastrointestinal endoscopy: The future is almost here

Artificial intelligence (AI) enables machines to provide unparalleled value in a myriad of industries and applications. In recent years, researchers have harnessed artificial intelligence to analyze large-volume, unstructured medical data and perform clinical tasks, such as the identification of diabetic retinopathy or the diagnosis of cutaneous malignancies. Applications of artificial intelligence techniques, specifically machine learning and more recently deep learning, are beginning to emerge in gastrointestinal endoscopy. The most promising of these efforts have been in computer-aided detection and computer-aided diagnosis of colorectal polyps, with recent systems demonstrating high sensitivity and accuracy even when compared to expert human endoscopists. AI has also been utilized to identify gastrointestinal bleeding, to detect areas of inflammation, and even to diagnose certain gastrointestinal infections. Future work in the field should concentrate on creating seamless integration of AI systems with current endoscopy platforms and electronic medical records, developing training modules to teach clinicians how to use AI tools, and determining the best means for regulation and approval of new AI technology.

Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy

The detection and removal of precancerous polyps via colonoscopy is the gold standard for the prevention of colon cancer. However, the detection rate of adenomatous polyps can vary significantly among endoscopists. Here, we show that a machine-learning algorithm can detect polyps in clinical colonoscopies, in real time and with high sensitivity and specificity. We developed the deep-learning algorithm by using data from 1,290 patients, and validated it on newly collected 27,113 colonoscopy images from 1,138 patients with at least one detected polyp (per-image-sensitivity, 94.38%; per-image-specificity, 95.92%; area under the receiver operating characteristic curve, 0.984), on a public database of 612 polyp-containing images (per-image-sensitivity, 88.24%), on 138 colonoscopy videos with histologically confirmed polyps (per-image-sensitivity of 91.64%; per-polyp-sensitivity, 100%), and on 54 unaltered full-range colonoscopy videos without polyps (per-image-specificity, 95.40%). By using a multi-threaded processing system, the algorithm can process at least 25 frames per second with a latency of 76.80 ± 5.60 ms in real-time video analysis. The software may aid endoscopists while performing colonoscopies, and help assess differences in polyp and adenoma detection performance among endoscopists.

From Annotation to Computer-Aided Diagnosis

Holistic medical multimedia systems covering end-to-end functionality from data collection to aided diagnosis are highly needed, but rare. In many hospitals, the potential value of multimedia data collected through routine examinations is not recognized. Moreover, the availability of the data is limited, as the health care personnel may not have direct access to stored data. However, medical specialists interact with multimedia content daily through their everyday work and have an increasing interest in finding ways to use it to facilitate their work processes. In this article, we present a novel, holistic multimedia system aiming to tackle automatic analysis of video from gastrointestinal (GI) endoscopy. The proposed system comprises the whole pipeline, including data collection, processing, analysis, and visualization. It combines filters using machine learning, image recognition, and extraction of global and local image features. The novelty is primarily in this holistic approach and its real-time performance, where we automate a complete algorithmic GI screening process. We built the system in a modular way to make it easily extendable to analyze various abnormalities, and we made it efficient in order to run in real time. The conducted experimental evaluation proves that the detection and localization accuracy are comparable or even better than existing systems, but it is by far leading in terms of real-time performance and efficient resource consumption.

IJ-OpenCV: Combining ImageJ and OpenCV for processing images in biomedicine

BACKGROUND AND OBJECTIVE

The effective processing of biomedical images usually requires the interoperability of diverse software tools that have different aims but are complementary. The goal of this work is to develop a bridge to connect two of those tools: ImageJ, a program for image analysis in life sciences, and OpenCV, a computer vision and machine learning library.

METHODS

Based on a thorough analysis of ImageJ and OpenCV, we detected the features of these systems that could be enhanced, and developed a library to combine both tools, taking advantage of the strengths of each system. The library was implemented on top of the SciJava converter framework. We also provide a methodology to use this library.

RESULTS

We have developed the publicly available library IJ-OpenCV that can be employed to create applications combining features from both ImageJ and OpenCV. From the perspective of ImageJ developers, they can use IJ-OpenCV to easily create plugins that use any functionality provided by the OpenCV library and explore different alternatives. From the perspective of OpenCV developers, this library provides a link to the ImageJ graphical user interface and all its features to handle regions of interest.

CONCLUSIONS

The IJ-OpenCV library bridges the gap between ImageJ and OpenCV, allowing the connection and the cooperation of these two systems.

Integrating Online and Offline Three-Dimensional Deep Learning for Automated Polyp Detection in Colonoscopy Videos

Automated polyp detection in colonoscopy videos has been demonstrated to be a promising way for colorectal cancer prevention and diagnosis. Traditional manual screening is time consuming, operator dependent, and error prone; hence, automated detection approach is highly demanded in clinical practice. However, automated polyp detection is very challenging due to high intraclass variations in polyp size, color, shape, and texture, and low interclass variations between polyps and hard mimics. In this paper, we propose a novel offline and online three-dimensional (3-D) deep learning integration framework by leveraging the 3-D fully convolutional network (3D-FCN) to tackle this challenging problem. Compared with the previous methods employing hand-crafted features or 2-D convolutional neural network, the 3D-FCN is capable of learning more representative spatio-temporal features from colonoscopy videos, and hence has more powerful discrimination capability. More importantly, we propose a novel online learning scheme to deal with the problem of limited training data by harnessing the specific information of an input video in the learning process. We integrate offline and online learning to effectively reduce the number of false positives generated by the offline network and further improve the detection performance. Extensive experiments on the dataset of MICCAI 2015 Challenge on Polyp Detection demonstrated the better performance of our method when compared with other competitors.

Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification

Recently, Deep Learning, especially through Convolutional Neural Networks (CNNs) has been widely used to enable the extraction of highly representative features. This is done among the network layers by filtering, selecting, and using these features in the last fully connected layers for pattern classification. However, CNN training for automated endoscopic image classification still provides a challenge due to the lack of large and publicly available annotated databases. In this work we explore Deep Learning for the automated classification of colonic polyps using different configurations for training CNNs from scratch (or full training) and distinct architectures of pretrained CNNs tested on 8-HD-endoscopic image databases acquired using different modalities. We compare our results with some commonly used features for colonic polyp classification and the good results suggest that features learned by CNNs trained from scratch and the “off-the-shelf” CNNs features can be highly relevant for automated classification of colonic polyps. Moreover, we also show that the combination of classical features and “off-the-shelf” CNNs features can be a good approach to further improve the results.