A Survey of Deep Learning Approaches to Image Restoration

Single Stage Adaptive Multi-Attention Network for Image Restoration

Recently attention-based networks have been successful for image restoration tasks. However, existing methods are either computationally expensive or have limited receptive fields, adding constraints to the model. They are also less resilient in spatial and contextual aspects and lack pixel-to-pixel correspondence, which may degrade feature representations. In this paper, we propose a novel and computationally efficient architecture Single Stage Adaptive Multi-Attention Network (SSAMAN) for image restoration tasks, particularly for image denoising and image deblurring. SSAMAN efficiently addresses computational challenges and expands receptive fields, enhancing robustness in spatial and contextual feature representation. Its Adaptive Multi-Attention Module (AMAM), which consists of Adaptive Pixel Attention Branch (APAB) and an Adaptive Channel Attention Branch (ACAB), uniquely integrates channel and pixel-wise dimensions, significantly improving sensitivity to edges, shapes, and textures. We perform extensive experiments and ablation studies to validate the performance of SSAMAN. Our model shows state-of-the-art results on various benchmarks, for example, on image denoising tasks, SSAMAN achieves a notable 40.08 dB PSNR on SIDD dataset, outperforming Restormer by 0.06 dB PSNR, with 41.02% less computational cost, and achieves a 40.05 dB PSNR on the DND dataset. For image deblurring, SSAMAN achieves 33.53 dB PSNR on GoPro dataset. Code and models are available at Github.

Multichannel high noise level ECG denoising based on adversarial deep learning

This paper proposes a denoising method based on an adversarial deep learning approach for the post-processing of multi-channel fetal electrocardiogram (ECG) signals. As it's well known, noise leads to misinterpretations of fetal ECG signals and thus limits the use of fetal electrocardiography for healthcare applications. Therefore, denoising algorithms are essential for the exploitation of non-invasive fetal ECG. The proposed method is based on the combination of three end-to-end trained sub-networks to convert noisy fetal ECG signals into clean signals. The first two sub-networks are linked by skip connections and form a deep convolutional network that downsamples the noisy signals into a latent representation and subsequently upsamples this latent representation to recover clean signals. The third sub-network aims to boost the decoder sub-network to generate realistic clean signals. Experiments carried out on synthetic and real data showed that the proposed method improved by the signal-to-noise (SNR) of fetal ECG signals with input SNR ranging from [Formula: see text] to 0 dB by an average of 20 dB, and improve fetal signal quality by significantly increasing the number of true detected QRS complexes and halving QRS complex detection errors.

Improved weed segmentation in UAV imagery of sorghum fields with a combined deblurring segmentation model

BACKGROUND

Efficient and site-specific weed management is a critical step in many agricultural tasks. Image captures from drones and modern machine learning based computer vision methods can be used to assess weed infestation in agricultural fields more efficiently. However, the image quality of the captures can be affected by several factors, including motion blur. Image captures can be blurred because the drone moves during the image capturing process, e.g. due to wind pressure or camera settings. These influences complicate the annotation of training and test samples and can also lead to reduced predictive power in segmentation and classification tasks.

RESULTS

In this study, we propose DeBlurWeedSeg, a combined deblurring and segmentation model for weed and crop segmentation in motion blurred images. For this purpose, we first collected a new dataset of matching sharp and naturally blurred image pairs of real sorghum and weed plants from drone images of the same agricultural field. The data was used to train and evaluate the performance of DeBlurWeedSeg on both sharp and blurred images of a hold-out test-set. We show that DeBlurWeedSeg outperforms a standard segmentation model that does not include an integrated deblurring step, with a relative improvement of [Formula: see text] in terms of the Sørensen-Dice coefficient.

CONCLUSION

Our combined deblurring and segmentation model DeBlurWeedSeg is able to accurately segment weeds from sorghum and background, in both sharp as well as motion blurred drone captures. This has high practical implications, as lower error rates in weed and crop segmentation could lead to better weed control, e.g. when using robots for mechanical weed removal.

Multiscale Convolutional Network for Repairing Coal Slime Foam Images

Visual feature information regarding flotation foam is crucial for the flotation process. Owing to a large amount of noise and blur in the foam images collected in the floatation field, feature extraction and segmentation of foam images pose considerable challenges. Furthermore, the visual properties of foam are strongly correlated with current flotation conditions. Therefore, this study presents a method to repair blurred pixels in foam images. In addition to enhancing the image dataset necessary for network model training, the restored images can provide high-quality images extracting foam-feature information. In addition, this research presents a novel fifth-order residual structure that enlarges the network structure by stacking, enhancing the learning ability of complex networks. Experimental results demonstrate that the suggested method can achieve a satisfactory repair effect for foam images under various blurring conditions, laying a foundation for guiding the intelligent adjustment of flotation field parameters.

Vision Transformers in Image Restoration: A Survey

The Vision Transformer (ViT) architecture has been remarkably successful in image restoration. For a while, Convolutional Neural Networks (CNN) predominated in most computer vision tasks. Now, both CNN and ViT are efficient approaches that demonstrate powerful capabilities to restore a better version of an image given in a low-quality format. In this study, the efficiency of ViT in image restoration is studied extensively. The ViT architectures are classified for every task of image restoration. Seven image restoration tasks are considered: Image Super-Resolution, Image Denoising, General Image Enhancement, JPEG Compression Artifact Reduction, Image Deblurring, Removing Adverse Weather Conditions, and Image Dehazing. The outcomes, the advantages, the limitations, and the possible areas for future research are detailed. Overall, it is noted that incorporating ViT in the new architectures for image restoration is becoming a rule. This is due to some advantages compared to CNN, such as better efficiency, especially when more data are fed to the network, robustness in feature extraction, and a better feature learning approach that sees better the variances and characteristics of the input. Nevertheless, some drawbacks exist, such as the need for more data to show the benefits of ViT over CNN, the increased computational cost due to the complexity of the self-attention block, a more challenging training process, and the lack of interpretability. These drawbacks represent the future research direction that should be targeted to increase the efficiency of ViT in the image restoration domain.

A Survey of Deep Learning-Based Image Restoration Methods for Enhancing Situational Awareness at Disaster Sites: The Cases of Rain, Snow and Haze

This survey article is concerned with the emergence of vision augmentation AI tools for enhancing the situational awareness of first responders (FRs) in rescue operations. More specifically, the article surveys three families of image restoration methods serving the purpose of vision augmentation under adverse weather conditions. These image restoration methods are: (a) deraining; (b) desnowing; (c) dehazing ones. The contribution of this article is a survey of the recent literature on these three problem families, focusing on the utilization of deep learning (DL) models and meeting the requirements of their application in rescue operations. A faceted taxonomy is introduced in past and recent literature including various DL architectures, loss functions and datasets. Although there are multiple surveys on recovering images degraded by natural phenomena, the literature lacks a comprehensive survey focused explicitly on assisting FRs. This paper aims to fill this gap by presenting existing methods in the literature, assessing their suitability for FR applications, and providing insights for future research directions.