A model of deformable rings for interpretation of wireless capsule endoscopic videos

A Survey on Medical Image Analysis in Capsule Endoscopy

BACKGROUND AND OBJECTIVE

Capsule Endoscopy (CE) is a non-invasive, patient-friendly alternative to conventional endoscopy procedure. However, CE produces 6 to 8 hrs long video posing a tedious challenge to a gastroenterologist for abnormality detection. Major challenges to an expert are lengthy videos, need of constant concentration and subjectivity of the abnormality. To address these challenges along with high diagnostic accuracy, design and development of automated abnormality detection system is a must. Machine learning and computer vision techniques are devised to develop such automated systems.

METHODS

Study presents a review of quality research papers published in IEEE, Scopus, and Science Direct database with search criteria as capsule endoscopy, engineering, and journal papers. The initial search retrieved 144 publications. After evaluating all articles, 62 publications pertaining to image analysis are selected.

RESULTS

This paper presents a rigorous review comprising all the aspects of medical image analysis concerning capsule endoscopy namely video summarization and redundant image elimination, Image enhancement and interpretation, segmentation and region identification, Computer-aided abnormality detection in capsule endoscopy, Image and video compression. The study provides a comparative analysis of various approaches, experimental setup, performance, strengths, and limitations of the aspects stated above.

CONCLUSIONS

The analyzed image analysis techniques for capsule endoscopy have not yet overcome all current challenges mainly due to lack of dataset and complex nature of the gastrointestinal tract.

Clinical Performance of New Software to Automatically Detect Angioectasias in Small Bowel Capsule Endoscopy

Background

Video capsule endoscopy (VCE) revolutionized the diagnosis and management of obscure gastrointestinal bleeding, though the rate of detection of small bowel lesions by the physician is still disappointing. Our group developed a novel algorithm (CMEMS-Uminho) to automatically detect angioectasias which display greater accuracy in VCE static frames than other methods previously published. We aimed to evaluate the algorithm overall performance and assess its diagnostic yield and usability in clinical practice.

Methods

Algorithm overall performance was determined using 54 full-length VCE recordings. To assess its diagnostic yield and usability in clinical practice, 38 VCE examinations with the clinical diagnosis of angioectasias consecutively performed (2017-2018) were evaluated by three physicians with different experiences. The CMEMS-Uminho algorithm was also applied. The performance of the CMEMS-Uminho algorithm was defined by a positive concordance between a frame automatically selected by the software and a study independent capsule endoscopist.

Results

Overall performance in complete VCE recordings was 77.7%, and diagnostic yield was 94.7%. There were significant differences between physicians in regard to global detection rate (p < 0.001), detection rate per capsule (p < 0.001), diagnostic yield (p = 0.007), true positive rate (p < 0.001), time (p < 0.001), and speed viewing (p < 0.001). The application of CMEMS-Uminho algorithm significantly enhanced all readers' global detection rate (p < 0.001) and the differences between them were no longer observed.

Conclusion

The CMEMS-Uminho algorithm detained a good overall performance and was able to enhance physicians' performance, suggesting a potential usability of this tool in clinical practice.

Colored Video Analysis in Wireless Capsule Endoscopy: A Survey of State-of-the-Art

Wireless Capsule Endoscopy (WCE) is a highly promising technology for gastrointestinal (GI) tract abnormality diagnosis. However, low image resolution and low frame rates are challenging issues in WCE. In addition, the relevant frames containing the features of interest for accurate diagnosis only constitute 1% of the complete video information. For these reasons, analyzing the WCE videos is still a time consuming and laborious examination for the gastroenterologists, which reduces WCE system usability. This leads to the emergent need to speed-up and automates the WCE video process for GI tract examinations. Consequently, the present work introduced the concept of WCE technology, including the structure of WCE systems, with a focus on the medical endoscopy video capturing process using image sensors. It discussed also the significant characteristics of the different GI tract for effective feature extraction. Furthermore, video approaches for bleeding and lesion detection in the WCE video were reported with computer-aided diagnosis systems in different applications to support the gastroenterologist in the WCE video analysis. In image enhancement, WCE video review time reduction is also discussed, while reporting the challenges and future perspectives, including the new trend to employ the deep learning models for feature Learning, polyp recognition, and classification, as a new opportunity for researchers to develop future WCE video analysis techniques.

Automatic detection of small bowel tumors in wireless capsule endoscopy images using ensemble learning

PURPOSE

Wireless Capsule Endoscopy (WCE) is a minimally invasive diagnosis tool for lesion detection in the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the significant amount of acquired data leads to difficulties in the diagnosis by the physicians; which can be eased with computer assistance. This paper addresses a method for the automatic detection of tumors in WCE by using a two-step based procedure: region of interest selection and classification.

METHODS

The first step aims to separate abnormal from normal tissue by using automatic segmentation based on a Gaussian Mixture Model (GMM). A modified version of the Anderson method for convergence acceleration of the expectation-maximization (EM) algorithm is proposed. The proposed features for both segmentation and classification are based on the CIELab color space, as a way of bypassing lightness variations, where the L component is discarded. Tissue variability among subjects, light inhomogeneities and even intensity differences among different devices can be overcome by using simultaneously features from both regions. In the second step, an ensemble system with partition of the training data with a new training scheme is proposed. At this stage, the gating network is trained after the experts have been trained decoupling the joint maximization of both modules. The partition module is also used at the test step, leading the incoming data to the most likely expert allowing incremental adaptation by preserving data diversity.

RESULTS

This algorithm outperforms others based on texture features selected from Wavelets and Curvelets transforms, classified by a regular support vector machine (SVM) in more than 5%.

CONCLUSIONS

This work shows that simpler features can outperform more elaborate ones if appropriately designed. In the current case, luminance was discarded to cope with saturated tissue, facilitating the color perception. Ensemble systems remain an open research field. In the current case, changes in both topology and training strategy have led to significant performance improvements. A system with this level of performance can be used in current clinical practice.

2D reconstruction of small intestine's interior wall

Examining and interpreting of a large number of wireless endoscopic images from the gastrointestinal tract is a tiresome task for physicians. A practical solution is to automatically construct a two dimensional representation of the gastrointestinal tract for easy inspection. However, little has been done on wireless endoscopic image stitching, let alone systematic investigation. The proposed new wireless endoscopic image stitching method consists of two main steps to improve the accuracy and efficiency of image registration. First, the keypoints are extracted by Principle Component Analysis and Scale Invariant Feature Transform (PCA-SIFT) algorithm and refined with Maximum Likelihood Estimation SAmple Consensus (MLESAC) outlier removal to find the most reliable keypoints. Second, the optimal transformation parameters obtained from first step are fed to the Normalised Mutual Information (NMI) algorithm as an initial solution. With modified Marquardt-Levenberg search strategy in a multiscale framework, the NMI can find the optimal transformation parameters in the shortest time. The proposed methodology has been tested on two different datasets - one with real wireless endoscopic images and another with images obtained from Micro-Ball (a new wireless cubic endoscopy system with six image sensors). The results have demonstrated the accuracy and robustness of the proposed methodology both visually and quantitatively - registration residual error of 0.93±0.33 pixels on 2500 real endoscopy image pairs and residual error accumulation of 16.59 pixels and without affecting the visual registration quality on stitching 152 images of Micro-Ball.

OdoCapsule: next-generation wireless capsule endoscopy with accurate lesion localization and video stabilization capabilities

In this paper, we propose a platform to achieve accurate localization of small-bowel lesions and endoscopic video stabilization in wireless capsule endoscopy. Current research modules rely on the use of external magnetic fields and triangulation methods to calculate the position vector of the capsule, leading to considerable error margins. Our platform, entitled OdoCapsule (a synthesis of the words Odometer and Capsule), provides real-time distance information from the point of duodenal entry to the point of exit from the small bowel. To achieve this, OdoCapsule is equipped with three miniature legs. Each leg carries a soft rubber wheel, which is made with human-compliant material. These legs are extendable and retractable thanks to a micromotor and three custom-made torsion springs. The wheels are specifically designed to function as microodometers: each rotation they perform is registered. Hence, the covered distance is measured accurately in real time. Furthermore, with its legs fully extended, OdoCapsule can stabilize itself inside the small-bowel lumen thus offering smoother video capture and better image processing. Recent ex vivo testing of this concept, using porcine small bowel and a commercially available (custom-modified) capsule endoscope, has proved its viability.

Automatic detection of small bowel tumors in endoscopic capsule images by ROI selection based on discarded lightness information

This paper addresses the problem of automatic detection of tumoral frames in endoscopic capsule videos by using features directly extracted from the color space. We show that tumor can be appropriately discriminated from normal tissue by using only color information histogram measures from the Lab color space and that light saturated regions are usually classified as tumoral regions when color based discriminative procedures are used. These regions are correctly classified if lightening is discarded becoming the tissue classifier based only on the color differences a and b of the Lab color space. While current state of the art systems for small bowel tumor detection usually rely on the processing of the whole frame regarding features extraction this paper proposes the use of fully automatic segmentation in order to select regions likely to contain tumoral tissue. Classification is performed by using Support Vector Machine (SVM) and Multilayer Perceptron (MLP) by using features from color channels a and b of the Lab color space. The proposed algorithm outperforms in more than 5% a series of other algorithms based on features obtained from the higher frequency components selected from Wavelets and Curvelets transforms while saving important computational resources. In a matter of fact the proposed algorithm is more than 25 times faster than algorithms requiring wavelet/curvelet and co-occurrence computations.

Quantitative measurements in capsule endoscopy

This review summarizes several approaches for quantitative measurement in capsule endoscopy. Video capsule endoscopy (VCE) typically provides wireless imaging of small bowel. Currently, a variety of quantitative measurements are implemented in commercially available hardware/software. The majority is proprietary and hence undisclosed algorithms. Measurement of amount of luminal contamination allows calculating scores from whole VCE studies. Other scores express the severity of small bowel lesions in Crohn׳s disease or the degree of villous atrophy in celiac disease. Image processing with numerous algorithms of textural and color feature extraction is further in the research focuses for automated image analysis. These tools aim to select single images with relevant lesions as blood, ulcers, polyps and tumors or to omit images showing only luminal contamination. Analysis of motility pattern, size measurement and determination of capsule localization are additional topics. Non-visual wireless capsules transmitting data acquired with specific sensors from the gastrointestinal (GI) tract are available for clinical routine. This includes pH measurement in the esophagus for the diagnosis of acid gastro-esophageal reflux. A wireless motility capsule provides GI motility analysis on the basis of pH, pressure, and temperature measurement. Electromagnetically tracking of another motility capsule allows visualization of motility. However, measurement of substances by GI capsules is of great interest but still at an early stage of development.

Comparative assessment of feature extraction methods for visual odometry in wireless capsule endoscopy

Wireless capsule endoscopy (WCE) enables the non-invasive examination of the gastrointestinal (GI) tract by a swallowable device equipped with a miniature camera. Accurate localization of the capsule in the GI tract enables accurate localization of abnormalities for medical interventions such as biopsy and polyp resection; therefore, the optimization of the localization outcome is important. Current approaches to endoscopic capsule localization are mainly based on external sensors and transit time estimations. Recently, we demonstrated the feasibility of capsule localization based-entirely-on visual features, without the use of external sensors. This technique relies on a motion estimation algorithm that enables measurements of the distance and the rotation of the capsule from the acquired video frames. Towards the determination of an optimal visual feature extraction technique for capsule motion estimation, an extensive comparative assessment of several state-of-the-art techniques, using a publicly available dataset, is presented. The results show that the minimization of the localization error is possible at the cost of computational efficiency. A localization error of approximately one order of magnitude higher than the minimal one can be considered as compromise for the use of current computationally efficient feature extraction techniques.

Software for enhanced video capsule endoscopy: challenges for essential progress

Video capsule endoscopy (VCE) has revolutionized the diagnostic work-up in the field of small bowel diseases. Furthermore, VCE has the potential to become the leading screening technique for the entire gastrointestinal tract. Computational methods that can be implemented in software can enhance the diagnostic yield of VCE both in terms of efficiency and diagnostic accuracy. Since the appearance of the first capsule endoscope in clinical practice in 2001, information technology (IT) research groups have proposed a variety of such methods, including algorithms for detecting haemorrhage and lesions, reducing the reviewing time, localizing the capsule or lesion, assessing intestinal motility, enhancing the video quality and managing the data. Even though research is prolific (as measured by publication activity), the progress made during the past 5 years can only be considered as marginal with respect to clinically significant outcomes. One thing is clear-parallel pathways of medical and IT scientists exist, each publishing in their own area, but where do these research pathways meet? Could the proposed IT plans have any clinical effect and do clinicians really understand the limitations of VCE software? In this Review, we present an in-depth critical analysis that aims to inspire and align the agendas of the two scientific groups.

A video-based speed estimation technique for localizing the wireless capsule endoscope inside gastrointestinal tract

Wireless Capsule Endoscopy (WCE) is progressively emerging as one of the most popular non-invasive imaging tools for gastrointestinal (GI) tract inspection. As a critical component of capsule endoscopic examination, physicians need to know the precise position of the endoscopic capsule in order to identify the position of intestinal disease. For the WCE, the position of the capsule is defined as the linear distance it is away from certain fixed anatomical landmarks. In order to measure the distance the capsule has traveled, a precise knowledge of how fast the capsule moves is urgently needed. In this paper, we present a novel computer vision based speed estimation technique that is able to extract the speed of the endoscopic capsule by analyzing the displacements between consecutive frames. The proposed approach is validated using a virtual testbed as well as the real endoscopic images. Results show that the proposed method is able to precisely estimate the speed of the endoscopic capsule with 93% accuracy on average, which enhances the localization accuracy of the WCE to less than 2.49 cm.

Texture and color based image segmentation and pathology detection in capsule endoscopy videos

This paper presents an in-depth study of several approaches to exploratory analysis of wireless capsule endoscopy images (WCE). It is demonstrated that versatile texture and color based descriptors of image regions corresponding to various anomalies of the gastrointestinal tract allows their accurate detection of pathologies in a sequence of WCE frames. Moreover, through classification of single pixels described by texture features of their neighborhood, the images can be segmented into homogeneous areas well matched to the image content. For both, detection and segmentation tasks the same procedure is applied which consists of features calculation, relevant feature subset selection and classification stages. This general three-stage framework is realized using various recognition strategies. In particular, the performance of the developed Vector Supported Convex Hull classification algorithm is compared against Support Vector Machines run in configuration with two different feature selection methods.

Characterization of microcirculation in multiple sclerosis lesions by dynamic texture parameter analysis (DTPA)

Objective

Texture analysis is an alternative method to quantitatively assess MR-images. In this study, we introduce dynamic texture parameter analysis (DTPA), a novel technique to investigate the temporal evolution of texture parameters using dynamic susceptibility contrast enhanced (DSCE) imaging. Here, we aim to introduce the method and its application on enhancing lesions (EL), non-enhancing lesions (NEL) and normal appearing white matter (NAWM) in multiple sclerosis (MS).

Methods

We investigated 18 patients with MS and clinical isolated syndrome (CIS), according to the 2010 McDonald's criteria using DSCE imaging at different field strengths (1.5 and 3 Tesla). Tissues of interest (TOIs) were defined within 27 EL, 29 NEL and 37 NAWM areas after normalization and eight histogram-based texture parameter maps (TPMs) were computed. TPMs quantify the heterogeneity of the TOI. For every TOI, the average, variance, skewness, kurtosis and variance-of-the-variance statistical parameters were calculated. These TOI parameters were further analyzed using one-way ANOVA followed by multiple Wilcoxon sum rank testing corrected for multiple comparisons.

Results

Tissue- and time-dependent differences were observed in the dynamics of computed texture parameters. Sixteen parameters discriminated between EL, NEL and NAWM (pAVG = 0.0005). Significant differences in the DTPA texture maps were found during inflow (52 parameters), outflow (40 parameters) and reperfusion (62 parameters). The strongest discriminators among the TPMs were observed in the variance-related parameters, while skewness and kurtosis TPMs were in general less sensitive to detect differences between the tissues.

Conclusion

DTPA of DSCE image time series revealed characteristic time responses for ELs, NELs and NAWM. This may be further used for a refined quantitative grading of MS lesions during their evolution from acute to chronic state. DTPA discriminates lesions beyond features of enhancement or T2-hypersignal, on a numeric scale allowing for a more subtle grading of MS-lesions.

Categorization and segmentation of intestinal content frames for wireless capsule endoscopy

Wireless capsule endoscopy (WCE) is a device that allows the direct visualization of gastrointestinal tract with minimal discomfort for the patient, but at the price of a large amount of time for screening. In order to reduce this time, several works have proposed to automatically remove all the frames showing intestinal content. These methods label frames as {intestinal content- clear} without discriminating between types of content (with different physiological meaning) or the portion of image covered. In addition, since the presence of intestinal content has been identified as an indicator of intestinal motility, its accurate quantification can show a potential clinical relevance. In this paper, we present a method for the robust detection and segmentation of intestinal content in WCE images, together with its further discrimination between turbid liquid and bubbles. Our proposal is based on a twofold system. First, frames presenting intestinal content are detected by a support vector machine classifier using color and textural information. Second, intestinal content frames are segmented into {turbid, bubbles, and clear} regions. We show a detailed validation using a large dataset. Our system outperforms previous methods and, for the first time, discriminates between turbid from bubbles media.

Adaptable image cuts for motility inspection using WCE

The Wireless Capsule Endoscopy (WCE) technology allows the visualization of the whole small intestine tract. Since the capsule is freely moving, mainly by the means of peristalsis, the data acquired during the study gives a lot of information about the intestinal motility. However, due to: (1) huge amount of frames, (2) complex intestinal scene appearance and (3) intestinal dynamics that make difficult the visualization of the small intestine physiological phenomena, the analysis of the WCE data requires computer-aided systems to speed up the analysis. In this paper, we propose an efficient algorithm for building a novel representation of the WCE video data, optimal for motility analysis and inspection. The algorithm transforms the 3D video data into 2D longitudinal view by choosing the most informative, from the intestinal motility point of view, part of each frame. This step maximizes the lumen visibility in its longitudinal extension. The task of finding "the best longitudinal view" has been defined as a cost function optimization problem which global minimum is obtained by using Dynamic Programming. Validation on both synthetic data and WCE data shows that the adaptive longitudinal view is a good alternative to the traditional motility analysis done by video analysis. The proposed novel data representation a new, holistic insight into the small intestine motility, allowing to easily define and analyze motility events that are difficult to spot by analyzing WCE video. Moreover, the visual inspection of small intestine motility is 4 times faster then by means of video skimming of the WCE.

Advanced technologies for gastrointestinal endoscopy

The gastrointestinal tract is home to some of the most deadly human diseases. Exacerbating the problem is the difficulty of accessing it for diagnosis or intervention and the concomitant patient discomfort. Flexible endoscopy has established itself as the method of choice and its diagnostic accuracy is high, but there remain technical limitations in modern scopes, and the procedure is poorly tolerated by patients, leading to low rates of compliance with screening guidelines. Although advancement in clinical endoscope design has been slow in recent years, a critical mass of enabling technologies is now paving the way for the next generation of gastrointestinal endoscopes. This review describes current endoscopes and provides an overview of innovative flexible scopes and wireless capsules that can enable painless endoscopy and/or enhanced diagnostic and therapeutic capabilities. We provide a perspective on the potential of these new technologies to address the limitations of current endoscopes in mass cancer screening and other contexts and thus to save many lives.

Automatic small bowel tumor diagnosis by using multi-scale wavelet-based analysis in wireless capsule endoscopy images

Background

Wireless capsule endoscopy has been introduced as an innovative, non-invasive diagnostic technique for evaluation of the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the output of this technique is an 8 hours video, whose analysis by the expert physician is very time consuming. Thus, a computer assisted diagnosis tool to help the physicians to evaluate CE exams faster and more accurately is an important technical challenge and an excellent economical opportunity.

Method

The set of features proposed in this paper to code textural information is based on statistical modeling of second order textural measures extracted from co-occurrence matrices. To cope with both joint and marginal non-Gaussianity of second order textural measures, higher order moments are used. These statistical moments are taken from the two-dimensional color-scale feature space, where two different scales are considered. Second and higher order moments of textural measures are computed from the co-occurrence matrices computed from images synthesized by the inverse wavelet transform of the wavelet transform containing only the selected scales for the three color channels. The dimensionality of the data is reduced by using Principal Component Analysis.

Results

The proposed textural features are then used as the input of a classifier based on artificial neural networks. Classification performances of 93.1% specificity and 93.9% sensitivity are achieved on real data. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis systems to assist physicians in their clinical practice.