Deep Endoscopic Visual Measurements

Robotic wireless capsule endoscopy: recent advances and upcoming technologies

Wireless capsule endoscopy (WCE) offers a non-invasive evaluation of the digestive system, eliminating the need for sedation and the risks associated with conventional endoscopic procedures. Its significance lies in diagnosing gastrointestinal tissue irregularities, especially in the small intestine. However, existing commercial WCE devices face limitations, such as the absence of autonomous lesion detection and treatment capabilities. Recent advancements in micro-electromechanical fabrication and computational methods have led to extensive research in sophisticated technology integration into commercial capsule endoscopes, intending to supersede wired endoscopes. This Review discusses the future requirements for intelligent capsule robots, providing a comparative evaluation of various methods' merits and disadvantages, and highlighting recent developments in six technologies relevant to WCE. These include near-field wireless power transmission, magnetic field active drive, ultra-wideband/intrabody communication, hybrid localization, AI-based autonomous lesion detection, and magnetic-controlled diagnosis and treatment. Moreover, we explore the feasibility for future "capsule surgeons".

Computer vision-based solutions to overcome the limitations of wireless capsule endoscopy

Endoscopic investigation plays a critical role in the diagnosis of gastrointestinal (GI) diseases. Since 2001, Wireless Capsule Endoscopy (WCE) has been available for small bowel exploration and is in continuous development. Over the last decade, WCE has achieved impressive improvements in areas such as miniaturisation, image quality and battery life. As a result, WCE is currently a very useful alternative to wired enteroscopy in the investigation of various small bowel abnormalities and has the potential to become the leading screening technique for the entire gastrointestinal tract. However, commercial solutions still have several limitations, namely incomplete examination and limited diagnostic capacity. These deficiencies are related to technical issues, such as image quality, motion estimation and power consumption management. Computational methods, based on image processing and analysis, can help to overcome these challenges and reduce both the time required by reviewers and human interpretation errors. Research groups have proposed a series of methods including algorithms for locating the capsule or lesion, assessing intestinal motility and improving image quality.In this work, we provide a critical review of computational vision-based methods for WCE image analysis aimed at overcoming the technological challenges of capsules. This article also reviews several representative public datasets used to evaluate the performance of WCE techniques and methods. Finally, some promising solutions of computational methods based on the analysis of multiple-camera endoscopic images are presented.

Tracking the Traveled Distance of Capsule Endoscopes along a Gastrointestinal-Tract Model Using Differential Static Magnetic Localization

The traveled distance and orientation of capsule endoscopes for each video frame are not available in commercial systems, but they would be highly relevant for physicians. Furthermore, scientific approaches lack precisely tracking the capsules along curved trajectories within the typical gastrointestinal tract. Recently, we showed that the differential static magnetic localisation method is suitable for the precise absolute localisation of permanent magnets assumed to be integrated into capsule endoscopes. Thus, in the present study, the differential method was employed to track permanent magnets in terms of traveled distance and orientation along a length trajectory of 487.5 mm, representing a model of the winding gastrointestinal tract. Permanent magnets with a diameter of 10 mm and different lengths were used to find a lower boundary for magnet size. Results reveal that the mean relative distance and orientation errors did not exceed 4.3 ± 3.3%, and 2 ± 0.6∘, respectively, when the magnet length was at least 5 mm. Thus, a 5 mm long magnet would be a good compromise between achievable tracking accuracy and magnet volume, which are essential for integration into small commercial capsules. Overall, the proposed tracking accuracy was better than that of the state of the art within a region covering the typical gastrointestinal-tract size.

Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges

Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.

Deep learning for registration of region of interest in consecutive wireless capsule endoscopy frames

BACKGROUND AND OBJECTIVE

Functional gastrointestinal disorders (FGIDs) are reported as worldwide gastrointestinal (GI) diseases. GI motility assessment can assist the diagnosis of patients with intestine motility dysfunction. Wireless capsule endoscopy (WCE) can acquire images in the gastrointestinal (GI) tract including the small intestine where other conventional endoscopes cannot penetrate, and WCE images can reveal GI motility. To generally analyze WCE frames, the high-precision registration of consecutive WCE frames is an absolute necessity. It is difficult and meaningless to register entire WCE frames on a pixel level due to the unpredictable and massive non-rigid deformation between consecutive frames, the low quality of imaging and the complex intestinal environment. Thus, the registration of region of interest (ROI) functioning in a feature level has more significance than entire frame registration.

METHODS

In this paper we present Timecylce-WCE, an end-to-end automatic registration approach of ROIs on WCE images. The clinicians can determine a ROI by drawing a bounding box in any WCE frame to be registered. This proposed approach is based on a deep-learning model of time-consistency in recurrent-registering, skip-registering and self-registering cycle, and it is fully unsupervised without any label. We incorporate the global correlation map with the local correlation map in matching the features, and a novel overall loss function is designed to enable the convergence of the model. As the output, a thin-plate spline (TPS) transformed region in the template frame is highly aligned with the query ROI in a finer-grained level. To the best of our knowledge this is the first time that a deep-learning-based registration method is proposed for WCE imaging motion.

RESULTS

To highlight the effectiveness of the proposed approach, our proposed method is compared with the existing non-deep-learning methods and tested in a validation dataset with labeled matching points. The presented method resulted in the best PCK@10 (Percentage of Correct Key-points, i.e., the predicted and the true joint is within the threshold - 10 pixels) of 66.49%. We also demonstrate that variants of design improved registration accuracy.

CONCLUSIONS

From the experimental analysis, it is clear that our proposed method outperforms the other existing methods. This lays the groundwork for subsequent studies, such as GI motility assessment, and WCE image synthesis.

Deep Ensemble Feature Network for Gastric Section Classification

In this paper, we propose a novel deep ensemble feature (DEF) network to classify gastric sections from endoscopic images. Different from recent deep ensemble learning methods, which need to train deep features and classifiers individually to obtain fused classification results, the proposed method can simultaneously learn the deep ensemble feature from arbitrary number of convolutional neural networks (CNNs) and the decision classifier in an end-to-end trainable manner. It comprises two sub networks, the ensemble feature network and the decision network. The former sub network learns the deep ensemble feature from multiple CNNs to represent endoscopic images. The latter sub network learns to obtain the classification labels by using the deep ensemble feature. Both sub networks are optimized based on the proposed ensemble feature loss and the decision loss which guide the learning of deep features and decisions. As shown in the experimental results, the proposed method outperforms the state-of-the-art deep learning, ensemble learning, and deep ensemble learning methods.

Artificial intelligence and deep learning for small bowel capsule endoscopy

Capsule endoscopy is ideally suited to artificial intelligence-based interpretation given its reliance on pattern recognition in still images. Time saving viewing modes and lesion detection features currently available rely on machine learning algorithms, a form of artificial intelligence. Current software necessitates close human supervision given poor sensitivity relative to an expert reader. However, with the advent of deep learning, artificial intelligence is becoming increasingly reliable and will be increasingly relied upon. We review the major advances in artificial intelligence for capsule endoscopy in recent publications and briefly review artificial intelligence development for historical understanding. Importantly, recent advancements in artificial intelligence have not yet been incorporated into practice and it is immature to judge the potential of this technology based on current platforms. Remaining regulatory and standardization hurdles are being overcome and artificial intelligence-based clinical applications are likely to proliferate rapidly.

Artificial intelligence in small bowel capsule endoscopy - current status, challenges and future promise

Neural network-based solutions are under development to alleviate physicians from the tedious task of small-bowel capsule endoscopy reviewing. Computer-assisted detection is a critical step, aiming to reduce reading times while maintaining accuracy. Weakly supervised solutions have shown promising results; however, video-level evaluations are scarce, and no prospective studies have been conducted yet. Automated characterization (in terms of diagnosis and pertinence) by supervised machine learning solutions is the next step. It relies on large, thoroughly labeled databases, for which preliminary "ground truth" definitions by experts are of tremendous importance. Other developments are under ways, to assist physicians in localizing anatomical landmarks and findings in the small bowel, in measuring lesions, and in rating bowel cleanliness. It is still questioned whether artificial intelligence will enter the market with proprietary, built-in or plug-in software, or with a universal cloud-based service, and how it will be accepted by physicians and patients.

Frontiers of Robotic Colonoscopy: A Comprehensive Review of Robotic Colonoscopes and Technologies

Flexible colonoscopy remains the prime mean of screening for colorectal cancer (CRC) and the gold standard of all population-based screening pathways around the world. Almost 60% of CRC deaths could be prevented with screening. However, colonoscopy attendance rates are affected by discomfort, fear of pain and embarrassment or loss of control during the procedure. Moreover, the emergence and global thread of new communicable diseases might seriously affect the functioning of contemporary centres performing gastrointestinal endoscopy. Innovative solutions are needed: artificial intelligence (AI) and physical robotics will drastically contribute for the future of the healthcare services. The translation of robotic technologies from traditional surgery to minimally invasive endoscopic interventions is an emerging field, mainly challenged by the tough requirements for miniaturization. Pioneering approaches for robotic colonoscopy have been reported in the nineties, with the appearance of inchworm-like devices. Since then, robotic colonoscopes with assistive functionalities have become commercially available. Research prototypes promise enhanced accessibility and flexibility for future therapeutic interventions, even via autonomous or robotic-assisted agents, such as robotic capsules. Furthermore, the pairing of such endoscopic systems with AI-enabled image analysis and recognition methods promises enhanced diagnostic yield. By assembling a multidisciplinary team of engineers and endoscopists, the paper aims to provide a contemporary and highly-pictorial critical review for robotic colonoscopes, hence providing clinicians and researchers with a glimpse of the major changes and challenges that lie ahead.

Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged

Every day, visually challenged people (VCP) face mobility restrictions and accessibility limitations. A short walk to a nearby destination, which for other individuals is taken for granted, becomes a challenge. To tackle this problem, we propose a novel visual perception system for outdoor navigation that can be evolved into an everyday visual aid for VCP. The proposed methodology is integrated in a wearable visual perception system (VPS). The proposed approach efficiently incorporates deep learning, object recognition models, along with an obstacle detection methodology based on human eye fixation prediction using Generative Adversarial Networks. An uncertainty-aware modeling of the obstacle risk assessment and spatial localization has been employed, following a fuzzy logic approach, for robust obstacle detection. The above combination can translate the position and the type of detected obstacles into descriptive linguistic expressions, allowing the users to easily understand their location in the environment and avoid them. The performance and capabilities of the proposed method are investigated in the context of safe navigation of VCP in outdoor environments of cultural interest through obstacle recognition and detection. Additionally, a comparison between the proposed system and relevant state-of-the-art systems for the safe navigation of VCP, focused on design and user-requirements satisfaction, is performed.

Follow-up on: optimizing lesion detection in small bowel capsule endoscopy and beyond: from present problems to future solutions

This review presents noteworthy advances in clinical and experimental Capsule Endoscopy (CE), focusing on the progress that has been reported over the last 5 years since our previous review on the subject. Areas covered: This study presents the commercially available CE platforms, as well as the advances made in optimizing the diagnostic capabilities of CE. The latter includes recent concept and prototype capsule endoscopes, medical approaches to improve diagnostic yield, and progress in software for enhancing visualization, abnormality detection, and lesion localization. Expert commentary: Currently, moving through the second decade of CE evolution, there are still several open issues and remarkable challenges to overcome.

Development of Algorithms for Automated Detection of Cervical Pre-Cancers With a Low-Cost, Point-of-Care, Pocket Colposcope

GOAL

In this paper, we propose methods for (1) automatic feature extraction and classification for acetic acid and Lugol's iodine cervigrams and (2) methods for combining features/diagnosis of different contrasts in cervigrams for improved performance.

METHODS

We developed algorithms to pre-process pathology-labeled cervigrams and extract simple but powerful color and textural-based features. The features were used to train a support vector machine model to classify cervigrams based on corresponding pathology for visual inspection with acetic acid, visual inspection with Lugol's iodine, and a combination of the two contrasts.

RESULTS

The proposed framework achieved a sensitivity, specificity, and accuracy of 81.3%, 78.6%, and 80.0%, respectively, when used to distinguish cervical intraepithelial neoplasia (CIN+) relative to normal and benign tissues. This is superior to the average values achieved by three expert physicians on the same data set for discriminating normal/benign cases from CIN+ (77% sensitivity, 51% specificity, and 63% accuracy).

CONCLUSION

The results suggest that utilizing simple color- and textural-based features from visual inspection with acetic acid and visual inspection with Lugol's iodine images may provide unbiased automation of cervigrams.

SIGNIFICANCE

This would enable automated, expert-level diagnosis of cervical pre-cancer at the point of care.