FAFS-UNet: Redesigning skip connections in UNet with feature aggregation and feature selection

In recent years, the encoder-decoder U-shaped network architecture has become a mainstream structure for medical image segmentation. Its biggest advantage lies in the incorporation of shallow features into deeper layers of the network through skip connections. However, according to our research, there are still some limitations in the skip connection part of the network: (1) The information from the encoder stage is not completely and effectively supplemented to the decoder stage; (2) The decoder receives the supplemented feature information from the encoder indiscriminately, which sometimes leads to the poor performance of the model. Therefore, to effectively address these limitations, we have redesigned the skip connections in UNet using a feature aggregation and feature selection approach. We firstly design the FA module to aggregate all encoder features and perform local multi-scale information extraction to obtain the complete multi-scale aggregated features. Further, we design the FS module to actively perform specific selection of these aggregated features through the decoder, thus effectively guiding the semantic recovery of the decoder. Finally, we conduct experiments on several medical image datasets, and the results show that our method has higher segmentation accuracy compared with other methods.

Multi-scale context UNet-like network with redesigned skip connections for medical image segmentation

BACKGROUND AND OBJECTIVE

Medical image segmentation has garnered significant research attention in the neural network community as a fundamental requirement for developing intelligent medical assistant systems. A series of UNet-like networks with an encoder-decoder architecture have achieved remarkable success in medical image segmentation. Among these networks, UNet2+ (UNet++) and UNet3+ (UNet+++) have introduced redesigned skip connections, dense skip connections, and full-scale skip connections, respectively, surpassing the performance of the original UNet. However, UNet2+ lacks comprehensive information obtained from the entire scale, which hampers its ability to learn organ placement and boundaries. Similarly, due to the limited number of neurons in its structure, UNet3+ fails to effectively segment small objects when trained with a small number of samples.

METHOD

In this study, we propose UNet_sharp (UNet#), a novel network topology named after the "#" symbol, which combines dense skip connections and full-scale skip connections. In the decoder sub-network, UNet# can effectively integrate feature maps of different scales and capture fine-grained features and coarse-grained semantics from the entire scale. This approach enhances the understanding of organ and lesion positions and enables accurate boundary segmentation. We employ deep supervision for model pruning to accelerate testing and enable mobile device deployment. Additionally, we construct two classification-guided modules to reduce false positives and improve segmentation accuracy.

RESULTS

Compared to current UNet-like networks, our proposed method achieves the highest Intersection over Union (IoU) values ((92.67±0.96)%, (92.38±1.29)%, (95.36±1.22)%, (74.01±2.03)%) and F1 scores ((91.64±1.86)%, (95.70±2.16)%, (97.34±2.76)%, (84.77±2.65)%) on the semantic segmentation tasks of nuclei, brain tumors, liver, and lung nodules, respectively.

CONCLUSIONS

The experimental results demonstrate that the reconstructed skip connections in UNet successfully incorporate multi-scale contextual semantic information. Compared to most state-of-the-art medical image segmentation models, our proposed method more accurately locates organs and lesions and precisely segments boundaries.

Residual dense network for medical magnetic resonance images super-resolution

BACKGROUND AND OBJECTIVE

High-resolution magnetic resonance images (MRI) help experts to localize lesions and diagnose diseases, but it is difficult to obtain high-resolution MRI. Furthermore, image super-resolution technology based on deep learning can effectively improve image resolution.

METHODS

In this work, we propose a medical magnetic resonance (MR) image super-resolution reconstruction method based on residual dense network (MRDN). Firstly, we input the convolutional features of the shallow layer into the residual dense block to obtain global and local features. Secondly, each layer in the residual dense block is directly connected to the previous layer to achieve reuse of features. Finally, we use sub-pixel convolution layer for upsampling and super-resolution reconstruction to get a clear high-resolution image.

RESULTS

For the 2 ×, 3 ×, and 4 × enlargement, we propose the MRDN method shows the superiority over the state-of-the-art methods on the Set5, Set14, and Urban100 benchmark datasets, extensive benchmark experiment and analysis show that the superiority of our MRDN algorithm in terms of the peak signal-to-noise ratio (PSNR) and structural similarity index indicators (SSIM).

CONCLUSION

Quantitative experiments are conducted on three public datasets: Set5, Set14 and Urban10, evaluate with commonly used evaluation metrics, and the experimental results show that the method in this paper is more effective. In addition, we reconstruct the public MR datasets, and the reconstructed high-resolution MR image has a clear structure and rich texture details.

Sharp U-Net: Depthwise convolutional network for biomedical image segmentation

The U-Net architecture, built upon the fully convolutional network, has proven to be effective in biomedical image segmentation. However, U-Net applies skip connections to merge semantically different low- and high-level convolutional features, resulting in not only blurred feature maps, but also over- and under-segmented target regions. To address these limitations, we propose a simple, yet effective end-to-end depthwise encoder-decoder fully convolutional network architecture, called Sharp U-Net, for binary and multi-class biomedical image segmentation. The key rationale of Sharp U-Net is that instead of applying a plain skip connection, a depthwise convolution of the encoder feature map with a sharpening kernel filter is employed prior to merging the encoder and decoder features, thereby producing a sharpened intermediate feature map of the same size as the encoder map. Using this sharpening filter layer, we are able to not only fuse semantically less dissimilar features, but also to smooth out artifacts throughout the network layers during the early stages of training. Our extensive experiments on six datasets show that the proposed Sharp U-Net model consistently outperforms or matches the recent state-of-the-art baselines in both binary and multi-class segmentation tasks, while adding no extra learnable parameters. Furthermore, Sharp U-Net outperforms baselines that have more than three times the number of learnable parameters.