Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation

Video domain adaptation for semantic segmentation using perceptual consistency matching

Unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target). Despite the impressive performance, existing approaches have largely focused on image-based UDA only, and video-based UDA has been relatively understudied and received less attention due to the difficulty of adapting diverse modal video features and modeling temporal associations efficiently. To address this, existing studies use optical flow to capture motion cues between in-domain consecutive frames, but is limited by heavy compute requirements and modeling flow patterns across diverse domains is equally challenging. In this work, we propose an adversarial domain adaptation approach for video semantic segmentation that aims to align temporally associated pixels in successive source and target domain frames without relying on optical flow. Specifically, we introduce a Perceptual Consistency Matching (PCM) strategy that leverages perceptual similarity to identify pixels with high correlation across consecutive frames, and infer that such pixels should correspond to the same class. Therefore, we can enhance prediction accuracy for video-UDA by enforcing consistency not only between in-domain frames, but across domains using PCM objectives during model training. Extensive experiments on public datasets show the benefit of our approach over existing state-of-the-art UDA methods. Our approach not only addresses a crucial task in video domain adaptation but also offers notable improvements in performance with faster inference times.

A deep learning based dual encoder-decoder framework for anatomical structure segmentation in chest X-ray images

Automated multi-organ segmentation plays an essential part in the computer-aided diagnostic (CAD) of chest X-ray fluoroscopy. However, developing a CAD system for the anatomical structure segmentation remains challenging due to several indistinct structures, variations in the anatomical structure shape among different individuals, the presence of medical tools, such as pacemakers and catheters, and various artifacts in the chest radiographic images. In this paper, we propose a robust deep learning segmentation framework for the anatomical structure in chest radiographs that utilizes a dual encoder-decoder convolutional neural network (CNN). The first network in the dual encoder-decoder structure effectively utilizes a pre-trained VGG19 as an encoder for the segmentation task. The pre-trained encoder output is fed into the squeeze-and-excitation (SE) to boost the network's representation power, which enables it to perform dynamic channel-wise feature calibrations. The calibrated features are efficiently passed into the first decoder to generate the mask. We integrated the generated mask with the input image and passed it through a second encoder-decoder network with the recurrent residual blocks and an attention the gate module to capture the additional contextual features and improve the segmentation of the smaller regions. Three public chest X-ray datasets are used to evaluate the proposed method for multi-organs segmentation, such as the heart, lungs, and clavicles, and single-organ segmentation, which include only lungs. The results from the experiment show that our proposed technique outperformed the existing multi-class and single-class segmentation methods.

Towards real time guide wire shape extraction in fluoroscopic sequences: A two phase deep learning scheme to extract sparse curvilinear structures

BACKGROUND AND OBJECTIVE

Real time localization and shape extraction of guide wire in fluoroscopic images plays a significant role in the image guided navigation during cerebral and cardiovascular interventions. Given the complexity of the non-rigid and sparse characteristics of guide wire structures, and the low SNR(Signal Noise Ratio) of fluoroscopic images, traditional handcrafted guide wire tracking methods such as Frangi filter, Hessian Matrix, or open active contour usually produce insufficient accuracy with high computational cost, and may require extra human intervention for proper initialization or correction. The application of deep learning techniques to guide wire tracking is reported to produce significant improvement in guide wire localization accuracy, but the heavy calculation cost is still a concern.

METHOD

In this paper we propose a two phase deep learning scheme for accurate and real time guide wire shape extraction in fluoroscopic sequences. In the first phase we train a guide wire localization network to pick image regions containing guide wire structures. From the picked image regions, we train a guide wire shape extraction network in the second phase to mark the guide wire pixels.

RESULTS

We report that our proposed method can accurately distinguish about 99% of the guide wire structure pixels, and the falsely detected pixels in the background are close to 0, the average offset from the ground truth is less than 1 pixel. For extreme cases where traditional handcrafted method may fail, our proposed method can still extract guide wire completely and accurately. The processing time for a 512 × 512 pixels image is 78 ms.

CONCLUSION

Compared with the traditional filtering based method from our previous work, we show that our proposed method can achieve more accurate and stable performance. Compared with other deep learning methods, our proposed method significantly improve calculation efficiency to meet the real time requirement of clinical applications.