Electrophysiology Catheter Detection and Reconstruction From Two Views in Fluoroscopic Images

Design and navigation method of a soft robot for single-port transvesical radical prostatectomy

PURPOSE

Currently, the rigid instruments used for laparoscopic radical resection of prostate cancer not only have the risk of damage to tissues, blood vessels, and nerves, but their limited freedom will also cause surgical blind areas. Soft robots are expected to solve these issues due to inherent flexibility, compliance, and safe interaction with tissues and organs. In addition, to achieve high surgical accuracy and provide precise guidance for surgeons, the navigation method should be studied for the soft robot.

METHODS

A soft robot system for single-port transvesical radical prostatectomy (STRP) is developed, and a navigation method combining fiber Bragg gratings and electromagnetic tracking is proposed for the soft robot. To validate the soft robot design and the effectiveness of the navigation method, different groups of experiments are conducted.

RESULTS

The proposed navigation method can achieve accurate location and shape sensing of the soft manipulator. The experiments show that the maximum tip sensing error is 2.691 mm, which is 5.38 % of the robot length for static configurations, and that the average tip sensing error is 1.966 mm, which corresponds to 3.93 % of the robot length for dynamic scenarios. Additionally, phantom tests demonstrate that the designed soft robot can enter the prostate through navigation guidance in a master-slave control mode and cover the entire prostate space.

CONCLUSIONS

The designed soft robot system, due to its soft structure, good flexibility, and accurate navigation, is expected to improve surgical safety and precision, thereby exhibiting significant potential for STRP.

Topology observing 3D device reconstruction from continuous-sweep limited angle fluoroscopy

BACKGROUND

Minimally invasive procedures usually require navigating a microcatheter and guidewire through endoluminal structures such as blood vessels and airways to sites of the disease. For numerous clinical applications, two-dimensional (2D) fluoroscopy is the primary modality used for real-time image guidance during navigation. However, 2D imaging can pose challenges for navigation in complex structures. Real-time 3D visualization of devices within the anatomic context could provide considerable benefits for these procedures. Continuous-sweep limited angle (CLA) fluoroscopy has recently been proposed to provide a compromise between conventional rotational 3D acquisitions and real-time fluoroscopy.

PURPOSE

The purpose of this work was to develop and evaluate a noniterative 3D device reconstruction approach for CLA fluoroscopy acquisitions, which takes into account endoluminal topology to avoid impossible paths between disconnected branches.

METHODS

The algorithm relies on a static 3D roadmap (RM) of vessels or airways, which may be generated from conventional cone beam CT (CBCT) acquisitions prior to navigation. The RM is converted to a graph representation describing its topology. During catheter navigation, the device is segmented from the live 2D projection images using a deep learning approach from which the centerlines are extracted. Rays from the focal spot to detector pixels representing 2D device points are identified and intersections with the RM are computed. Based on the RM graph, a subset of line segments is selected as candidates to exclude device paths through disconnected branches of the RM. Depth localization for each point along the device is then performed by finding the point closest to the previous 3D reconstruction along the candidate segments. This process is repeated as the projection angle changes for each CLA image frame. The approach was evaluated in a phantom study in which a catheter and guidewire were navigated along five pathways within a complex vessel phantom. The result was compared to static cCBCT acquisitions of the device in the final position.

RESULTS

The average root mean squared 3D distance between CLA reconstruction and reference centerline was1.87 ± 0.30 $1.87 \pm 0.30$ mm. The Euclidean distance at the device tip was2.92 ± 2.35 $2.92 \pm 2.35$ mm. The correct pathway was identified during reconstruction in100 % $100\%$ of frames (n = 1475 $n=1475$ ). The percentage of 3D device points reconstructed inside the 3D roadmap was91.83 ± 2.52 % $91.83 \pm 2.52\%$ with an average distance of0.62 ± 0.30 $0.62 \pm 0.30$ mm between the device points outside the roadmap and the nearest point within the roadmap.

CONCLUSIONS

This study demonstrates the feasibility of reconstructing curvilinear devices such as catheters and guidewires during endoluminal procedures including intravascular and transbronchial interventions using a noniterative reconstruction approach for CLA fluoroscopy. This approach could improve device navigation in cases where the structure of vessels or airways is complex and includes overlapping branches.

Unsharp Structure Guided Filtering for Self-Supervised Low-Dose CT Imaging

Low-dose computed tomography (LDCT) imaging faces great challenges. Although supervised learning has revealed great potential, it requires sufficient and high-quality references for network training. Therefore, existing deep learning methods have been sparingly applied in clinical practice. To this end, this paper presents a novel Unsharp Structure Guided Filtering (USGF) method, which can reconstruct high-quality CT images directly from low-dose projections without clean references. Specifically, we first employ low-pass filters to estimate the structure priors from the input LDCT images. Then, inspired by classical structure transfer techniques, deep convolutional networks are adopted to implement our imaging method which combines guided filtering and structure transfer. Finally, the structure priors serve as the guidance images to alleviate over-smoothing, as they can transfer specific structural characteristics to the generated images. Furthermore, we incorporate traditional FBP algorithms into self-supervised training to enable the transformation of projection domain data to the image domain. Extensive comparisons and analyses on three datasets demonstrate that the proposed USGF has achieved superior performance in terms of noise suppression and edge preservation, and could have a significant impact on LDCT imaging in the future.

An automatic framework for estimating the pose of the catheter distal section using a coarse-to-fine network

BACKGROUND AND OBJECTIVE

During percutaneous coronary intervention procedures, generally only 2D X-ray images are provided. The consequent lack of depth perception makes it difficult for interventionists to visually estimate the pose of medical tools inside the vasculature, especially for novices. Although some automatic methods have been developed to aid interventionists, it is still a challenging task to obtain stable and accurate pose estimation. In this paper, we describe a learning-based framework for estimating the pose of the catheter distal section (CDS). The main innovation of this framework is the proposal of a coarse-to-fine fusion network (CFF-Net) which can achieve the shape and orientation estimation for the CDS.

METHODS

By adopting a two-step fusion, CFF-Net progressively solves the shape and orientation ambiguities. The first step is the early fusion where the 2D projection image fuses with the shape prior before input, which makes the estimated result own a specific catheter distal shape. The second step is the late fusion where CFF-Net fuse feature maps and the orientation data from Electromagnetic (EM) sensors to confirm the overall orientation of the CDS. Finally, the estimated pose in the EM space will be obtained after we combine the estimated shape and orientation from CFF-Net with the position information from the EM sensor.

RESULTS

The effectiveness of CFF-Net has been verified in a simulated environment where RMSE of CFF-Net is 0.706 ± 0.121 mm. This approach was further transferred from simulation to reality using the real-world data, where RMSE of CFF-Net is 1.121 ± 0.124 mm and RMSE of the whole proposed framework is 1.577 ± 0.144 mm.

CONCLUSION

In simulated and real-world experiments, our proposed approach has been proven to achieve high accuracy while ensuring real-time processing for estimating the pose of the CDS.

Masked Joint Bilateral Filtering via Deep Image Prior for Digital X-ray Image Denoising

Medical image denoising faces great challenges. Although deep learning methods have shown great potential, their efficiency is severely affected by millions of trainable parameters. The non-linearity of neural networks also makes them difficult to be understood. Therefore, existing deep learning methods have been sparingly applied to clinical tasks. To this end, we integrate known filtering operators into deep learning and propose a novel Masked Joint Bilateral Filtering (MJBF) via deep image prior for digital X-ray image denoising. Specifically, MJBF consists of a deep image prior generator and an iterative filtering block. The deep image prior generator produces plentiful image priors by a multi-scale fusion network. The generated image priors serve as the guidance for the iterative filtering block, which is utilized for the actual edge-preserving denoising. The iterative filtering block contains three trainable Joint Bilateral Filters (JBFs), each with only 18 trainable parameters. Moreover, a masking strategy is introduced to reduce redundancy and improve the understanding of the proposed network. Experimental results on the ChestX-ray14 dataset and real data show that the proposed MJBF has achieved superior performance in terms of noise suppression and edge preservation. Tests on the portability of the proposed method demonstrate that this denoising modality is simple yet effective, and could have a clinical impact on medical imaging in the future.

Shape Reconstruction Processes for Interventional Application Devices: State of the Art, Progress, and Future Directions

Soft and continuum robots are transforming medical interventions thanks to their flexibility, miniaturization, and multidirectional movement abilities. Although flexibility enables reaching targets in unstructured and dynamic environments, it also creates challenges for control, especially due to interactions with the anatomy. Thus, in recent years lots of efforts have been devoted for the development of shape reconstruction methods, with the advancement of different kinematic models, sensors, and imaging techniques. These methods can increase the performance of the control action as well as provide the tip position of robotic manipulators relative to the anatomy. Each method, however, has its advantages and disadvantages and can be worthwhile in different situations. For example, electromagnetic (EM) and Fiber Bragg Grating (FBG) sensor-based shape reconstruction methods can be used in small-scale robots due to their advantages thanks to miniaturization, fast response, and high sensitivity. Yet, the problem of electromagnetic interference in the case of EM sensors, and poor response to high strains in the case of FBG sensors need to be considered. To help the reader make a suitable choice, this paper presents a review of recent progress on shape reconstruction methods, based on a systematic literature search, excluding pure kinematic models. Methods are classified into two categories. First, sensor-based techniques are presented that discuss the use of various sensors such as FBG, EM, and passive stretchable sensors for reconstructing the shape of the robots. Second, imaging-based methods are discussed that utilize images from different imaging systems such as fluoroscopy, endoscopy cameras, and ultrasound for the shape reconstruction process. The applicability, benefits, and limitations of each method are discussed. Finally, the paper draws some future promising directions for the enhancement of the shape reconstruction methods by discussing open questions and alternative methods.

Haptic Telerobotic Cardiovascular Intervention: A Review of Approaches, Methods, and Future Perspectives

Cardiac diseases are recognized as the leading cause of mortality, hospitalization, and medical prescription globally. The gold standard for the treatment of coronary artery stenosis is the percutaneous cardiac intervention that is performed under live X-ray imaging. Substantial clinical evidence shows that the surgeon and staff are prone to serious health problems due to X-ray exposure and occupational hazards. Telerobotic vascular intervention systems with a master-slave architecture reduced the X-ray exposure and enhanced the clinical outcomes; however, the loss of haptic feedback during surgery has been the main limitation of such systems. This paper is a review of the state of the art for haptic telerobotic cardiovascular interventions. A survey on the literature published between 2000 and 2019 was performed. Results of the survey were screened based on their relevance to this paper. Also, the leading research disciplines were identified based on the results of the survey. Furthermore, different approaches for sensor-based and model-based haptic telerobotic cardiovascular intervention, haptic rendering and actuation, and the pertinent methods were critically reviewed and compared. In the end, the current limitations of the state of the art, unexplored research areas as well as the future perspective of the research on this technology were laid out.

Symmetry, outliers, and geodesics in coronary artery centerline reconstruction from rotational angiography

PURPOSE

The performance of many state-of-the-art coronary artery centerline reconstruction algorithms in rotational angiography heavily depends on accurate two-dimensional centerline information that, in practice, is not available due to segmentation errors. To alleviate the need for correct segmentations, we propose generic extensions to symbolic centerline reconstruction algorithms that target symmetrization, outlier rejection, and topology recovery on asymmetrically reconstructed point clouds.

METHODS

Epipolar geometry- and graph cut-based reconstruction algorithms are used to reconstruct three-dimensional point clouds from centerlines in reference views. These clouds are input to the proposed methods that consist of (a) merging of asymmetric reconstructions, (b) removal of inconsistent three-dimensional points using the reprojection error, and (c) projection domain-informed geodesic computation. We validate our extensions in a numerical phantom study and on two clinical datasets.

RESULTS

In the phantom study, the overlap measure between the reconstructed point clouds and the three-dimensional ground truth increased from 68.4 ± 9.6% to 85.9 ± 3.3% when the proposed extensions were applied. In addition, the averaged mean and maximum reprojection error decreased from 4.32 ± 3.03 mm to 0.189 ± 0.182 mm and from 8.39 ± 6.08 mm to 0.392 ± 0.434 mm. For the clinical data, the mean and maximum reprojection error improved from 1.73 ± 0.97 mm to 0.882 ± 0.428 mm and from 3.83 ± 1.87 mm to 1.48 ± 0.61 mm, respectively.

CONCLUSIONS

The application of the proposed extensions yielded superior reconstruction quality in all cases and effectively removed erroneously reconstructed points. Future work will investigate possibilities to integrate parts of the proposed extensions directly into reconstruction.