Endomicroscopy for Computer and Robot Assisted Intervention

A novel contact optimization algorithm for endomicroscopic surface scanning

PURPOSE

Probe-based confocal laser endomicroscopy (pCLE) offers real-time, cell-level imaging and holds promise for early cancer diagnosis. However, a large area surface scanning for image acquisition is needed to overcome the limitation of field-of-view. Obtaining high-quality images during scanning requires maintaining a stable contact distance between the tissue and probe. This work presents a novel contact optimization algorithm to acquire high-quality pCLE images.

METHODS

The contact optimization algorithm, based on swarm intelligence of whale optimization algorithm, is designed to optimize the probe position, according to the quality of the image acquired by probe. An accurate image quality assessment of total co-occurrence entropy is introduced to evaluate the pCLE image quality. The algorithm aims to maintain a consistent probe-tissue contact, resulting in high-quality images acquisition.

RESULTS

Scanning experiments on sponge, ex vivo swine skin tissue and stomach tissue demonstrate the effectiveness of the contact optimization algorithm. Scanning results of the sponge with three different trajectories (spiral trajectory, circle trajectory, and raster trajectory) reveal high-quality mosaics with clear details in every part of the image and no blurred sections.

CONCLUSION

The contact optimization algorithm successfully identifies the optimal distance between probe and tissue, improving the quality of pCLE images. Experimental results confirm the high potential of this method in endomicroscopic surface scanning.

Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation

Image mosaicking has emerged as a universal technique to broaden the field-of-view of the probe-based confocal laser endomicroscopy (pCLE) imaging system. However, due to the influence of probe-tissue contact forces and optical components on imaging quality, existing mosaicking methods remain insufficient to deal with practical challenges. In this paper, we present the texture encoded sum of conditional variance (TESCV) as a novel similarity metric, and effectively incorporate it into a sequential mosaicking scheme to simultaneously correct rigid probe shift and nonrigid tissue deformation. TESCV combines both intensity dependency and texture relevance to quantify the differences between pCLE image frames, where a discriminative binary descriptor named fully cross-detected local derivative pattern (FCLDP) is designed to extract more detailed structural textures. Furthermore, we also analytically derive the closed-form gradient of TESCV with respect to the transformation variables. Experiments on the circular dataset highlighted the advantage of the TESCV metric in improving mosaicking performance compared with the other four recently published metrics. The comparison with the other four state-of-the-art mosaicking methods on the spiral and manual datasets indicated that the proposed TESCV-based method not only worked stably at different contact forces, but was also suitable for both low- and high-resolution imaging systems. With more accurate and delicate mosaics, the proposed method holds promises to meet clinical demands for intraoperative optical biopsy.

Robust Mosaicing of Endomicroscopic Videos via Context-Weighted Correlation Ratio

Probe-based confocal laser endomicroscopy (pCLE) is a promising imaging tool that provides in situ and in vivo optical imaging to perform real-time pathological assessments. However, due to limited field of view, it is difficult for clinicians to get a full understanding of the scanned tissues. In this paper, we develop a novel mosaicing framework to assemble all frame sequences into a full view image. First, a hybrid rigid registration that combines feature matching and template matching is presented to achieve a global alignment of all frames. Then, the parametric free-form deformation (FFD) model with a multiresolution architecture is implemented to accommodate non-rigid tissue distortions. More importantly, we devise a robust similarity metric called context-weighted correlation ratio (CWCR) to promote registration accuracy, where spatial and geometric contexts are incorporated into the estimation of functional intensity dependence. Experiments on both robotic setup and manual manipulation have demonstrated that the proposed scheme significantly precedes some state-of-the-art mosaicing schemes in the presence of intensity fluctuations, insufficient overlap and tissue distortions. Moreover, the comparisons of the proposed CWCR metric and two other metrics have validated the effectiveness of the context-weighted strategy in quantifying the differences between two frames. Benefiting from more rational and delicate mosaics, the proposed scheme is more suitable to instruct diagnosis and treatment during optical biopsies.

Application of Artificial Intelligence in Medicine: An Overview

Artificial intelligence (AI) is a new technical discipline that uses computer technology to research and develop the theory, method, technique, and application system for the simulation, extension, and expansion of human intelligence. With the assistance of new AI technology, the traditional medical environment has changed a lot. For example, a patient's diagnosis based on radiological, pathological, endoscopic, ultrasonographic, and biochemical examinations has been effectively promoted with a higher accuracy and a lower human workload. The medical treatments during the perioperative period, including the preoperative preparation, surgical period, and postoperative recovery period, have been significantly enhanced with better surgical effects. In addition, AI technology has also played a crucial role in medical drug production, medical management, and medical education, taking them into a new direction. The purpose of this review is to introduce the application of AI in medicine and to provide an outlook of future trends.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Modular Robotic Scanning Device for Real-Time Gastric Endomicroscopy

Optical biopsy methods, such as probe-based endomicroscopy, can be used for the identification of early mucosal dysplasia in various gastrointestinal conditions and have potential applications in the screening of early-stage gastric cancer in vivo. However, it is difficult to scan a large area of the gastric mucosa for mosaicing during standard endoscopy. This paper proposes a novel 'snap-on' robotic scanning device that can integrate distally with a commercial endoscope. A customized low-cost endomicroscopy system is used for obtaining micro imaging. The developed device could scan a large area of gastric tissue during standard endoscopy. The device achieves positioning accuracy that is less than 0.23 mm. Experimental results showed that the device could achieve large area mosaicing (15.8-18.6 mm²) and demonstrated the potential clinical value of the device for real-time gastric tissue identification and margin assessment. This approach presents an important alternative to current histology techniques for gastrointestinal tract diagnosis.

Robotic Scanning Device for Intraoperative Thyroid Gland Endomicroscopy

Probe-based confocal laser endomicroscopy has a high potential to be a promising tool that can provide intraoperative high-resolution in vivo morphological imaging at cellular and subcellular levels for thyroidectomy, and allow real-time assessment of tumor margins. However, the typical images acquired with this technique cover a very small area limited by the field of view of the probe, accompanied by tissue deformation and inconsistent probe-tissue contact when operated manually. In this paper, a novel compact robotic device for large area scanning has been developed. The device can scan a large surface in a spiral trajectory by rotating the tip frame along the spiral groove of the base frame. The fiber Bragg grating sensor with a passive linear structure is used to detect and maintain a stable probe-tissue contact force during scanning. An active linear actuation is also integrated for adjusting the probe-tissue contact level prior to each scan. Results demonstrate that the scanning device ensures a suitable probe-tissue contact force and compensates for simulated hand tremor. Mosaicing results of lens tissue paper and porcine belly tissue with both bench and hand-held experiments show the effectiveness and usability of the device, demonstrating the potential clinical value of the system.