Improving the accuracy of feature extraction for flexible endoscope calibration by spatial super resolution

EndoL2H: Deep Super-Resolution for Capsule Endoscopy

Although wireless capsule endoscopy is the preferred modality for diagnosis and assessment of small bowel diseases, the poor camera resolution is a substantial limitation for both subjective and automated diagnostics. Enhanced-resolution endoscopy has shown to improve adenoma detection rate for conventional endoscopy and is likely to do the same for capsule endoscopy. In this work, we propose and quantitatively validate a novel framework to learn a mapping from low-to-high-resolution endoscopic images. We combine conditional adversarial networks with a spatial attention block to improve the resolution by up to factors of 8× , 10× , 12× , respectively. Quantitative and qualitative studies demonstrate the superiority of EndoL2H over state-of-the-art deep super-resolution methods Deep Back-Projection Networks (DBPN), Deep Residual Channel Attention Networks (RCAN) and Super Resolution Generative Adversarial Network (SRGAN). Mean Opinion Score (MOS) tests were performed by 30 gastroenterologists qualitatively assess and confirm the clinical relevance of the approach. EndoL2H is generally applicable to any endoscopic capsule system and has the potential to improve diagnosis and better harness computational approaches for polyp detection and characterization. Our code and trained models are available at https://github.com/CapsuleEndoscope/EndoL2H.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Quantitative evaluation of comb-structure correction methods for multispectral fibrescopic imaging

Removing the comb artifact introduced by imaging fibre bundles, or 'fibrescopes', for example in medical endoscopy, is essential to provide high quality images to the observer. Multispectral imaging (MSI) is an emerging method that combines morphological (spatial) and chemical (spectral) information in a single data 'cube'. When a fibrescope is coupled to a spectrally resolved detector array (SRDA) to perform MSI, comb removal is complicated by the demosaicking step required to reconstruct the multispectral data cube. To understand the potential for using SRDAs as multispectral imaging sensors in medical endoscopy, we assessed five comb correction methods with respect to five performance metrics relevant to biomedical imaging applications: processing time, resolution, smoothness, signal and the accuracy of spectral reconstruction. By assigning weights to each metric, which are determined by the particular imaging application, our results can be used to select the correction method to achieve best overall performance. In most cases, interpolation gave the best compromise between the different performance metrics when imaging using an SRDA.

Endomicroscopy for Computer and Robot Assisted Intervention

Endomicroscopy is a new technique that allows human tissue to be characterized in vivo and in situ, circumventing the need for conventional biopsy and histology. Despite increased application and growing research interests in this area, the clinical application of endomicroscopy, however, is limited by difficulties in ergonomic control, consistent probe-tissue contact, large area surveillance, and retargeting. Recently, advances in high-speed imaging, mosaicing, and robotics have aimed to address these difficulties. The development of robot-assisted devices in particular has shown great promises in extending the clinical potential of endomicroscopy. Issues related to miniaturization, adaptation to tissue deformation, control stability, force and position compensation, cost, and sterility are being pursued by both research and commercial communities. In this review, recent clinical and technical developments in different aspects of computer and robotic assisted endomicroscopy interventions including instrumentation, multiscale integration, and high-speed imaging techniques are presented. We further address emerging trends and new research opportunities toward more widespread clinical acceptance of robotically assisted endomicroscopy technologies.

Evaluation of spatial interpolation strategies for the removal of comb-structure in fiber-optic images

Modern techniques for medical diagnosis and therapy in minimal invasive surgery scenarios as well as industrial inspection make considerable use of flexible, fiberoptic endoscopes in order to gain visual access to holes, hollows, antrums and cavities that are difficult to enter and examine. Unfortunately, fiber-optic endoscopes exhibit artifacts in the images that hinder or at worst prevent fundamental image analysis techniques. The dark comb-like artifacts originate from the opaque cladding layer surrounding each single fiber in the image conductor. Although the removal of comb structure is crucial for fiber-optic image analysis, literature covers only a few approaches. Those are based on Fourier analysis and make use of spectral masking or they operate in the spatial domain and rely on interpolation. In this paper, we concentrate on the latter type and introduce interpolation concepts known from related disciplines to the task of comb structure removal. For a quantitative evaluation, we perform experiments with real images as well as with bivariate test functions and rate an algorithm's performance in terms of the normalized root mean square error - a quality metric that it is most commonly used in signal processing for this purpose. Hence, this paper counters the fact that literature lacks an objective performance comparison of the state-of-the-art interpolation based approaches for this type of application.