Robust spiking cortical model and total-variational decomposition for multimodal medical image fusion

M4FNet: Multimodal medical image fusion network via multi-receptive-field and multi-scale feature integration

The main purpose of multimodal medical image fusion is to aggregate the significant information from different modalities and obtain an informative image, which provides comprehensive content and may help to boost other image processing tasks. Many existing methods based on deep learning neglect the extraction and retention of multi-scale features of medical images and the construction of long-distance relationships between depth feature blocks. Therefore, a robust multimodal medical image fusion network via the multi-receptive-field and multi-scale feature (M⁴FNet) is proposed to achieve the purpose of preserving detailed textures and highlighting the structural characteristics. Specifically, the dual-branch dense hybrid dilated convolution blocks (DHDCB) is proposed to extract the depth features from multi-modalities by expanding the receptive field of the convolution kernel as well as reusing features, and establish long-range dependencies. In order to make full use of the semantic features of the source images, the depth features are decomposed into multi-scale domain by combining the 2-D scale function and wavelet function. Subsequently, the down-sampling depth features are fused by the proposed attention-aware fusion strategy and inversed to the feature space with equal size of source images. Ultimately, the fusion result is reconstructed by a deconvolution block. To force the fusion network balancing information preservation, a local standard deviation-driven structural similarity is proposed as the loss function. Extensive experiments prove that the performance of the proposed fusion network outperforms six state-of-the-art methods, which SD, MI, Q_AB^F and Q_EP are about 12.8%, 4.1%, 8.5% and 9.7% gains, respectively.

Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain

Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, Q_W, API, SD, EN and time consumption.