Image super-resolution with an enhanced group convolutional neural network

Multi-level feature interaction image super-resolution network based on convolutional nonlinear spiking neural model

Image super-resolution (ISR) is designed to recover lost detail information from low-resolution images, resulting in high-quality and high-definition high-resolution images. In the existing single ISR (SISR) methods based on convolutional neural networks (CNN), however, most of the models cannot effectively combine global and local information and are also easy to ignore the correlation between different hierarchical feature information. To address these problems, this study proposes a multi-level feature interactive image super-resolution network, which is constructed by the convolutional units inspired by nonlinear spiking mechanism in nonlinear spiking neural P systems, including shallow feature processing, deep feature extraction and fusion, and reconstruction modules. The different omni domain self-attention blocks are introduced to extract global information in the deep feature extraction and fusion stage and formed a feature enhancement module having a Transformer structure using a novel convolutional unit for extracting local information. Furthermore, to adaptively fuse features between different hierarchies, we design a multi-level feature fusion module, which not only can adaptively fuse features between different hierarchies, but also can better interact with contextual information. The proposed model is compared with 16 state-of-the-art or baseline models on five benchmark datasets. The experimental results show that the proposed model not only achieves good reconstruction performance, but also strikes a good balance between model parameters and performance.

Dual-stage feedback network for lightweight color image compression artifact reduction

Lossy image coding techniques usually result in various undesirable compression artifacts. Recently, deep convolutional neural networks have seen encouraging advances in compression artifact reduction. However, most of them focus on the restoration of the luma channel without considering the chroma components. Besides, most deep convolutional neural networks are hard to deploy in practical applications because of their high model complexity. In this article, we propose a dual-stage feedback network (DSFN) for lightweight color image compression artifact reduction. Specifically, we propose a novel curriculum learning strategy to drive a DSFN to reduce color image compression artifacts in a luma-to-RGB manner. In the first stage, the DSFN is dedicated to reconstructing the luma channel, whose high-level features containing rich structural information are then rerouted to the second stage by a feedback connection to guide the RGB image restoration. Furthermore, we present a novel enhanced feedback block for efficient high-level feature extraction, in which an adaptive iterative self-refinement module is carefully designed to refine the low-level features progressively, and an enhanced separable convolution is advanced to exploit multiscale image information fully. Extensive experiments show the notable advantage of our DSFN over several state-of-the-art methods in both quantitative indices and visual effects with lower model complexity.

REMA: A Rich Elastic Mixed Attention Module for Single Image Super-Resolution

Detail preservation is a major challenge for single image super-resolution (SISR). Many deep learning-based SISR methods focus on lightweight network design, but these may fall short in real-world scenarios where performance is prioritized over network size. To address these problems, we propose a novel plug-and-play attention module, rich elastic mixed attention (REMA), for SISR. REMA comprises the rich spatial attention module (RSAM) and the rich channel attention module (RCAM), both built on Rich Structure. Based on the results of our research on the module's structure, size, performance, and compatibility, Rich Structure is proposed to enhance REMA's adaptability to varying input complexities and task requirements. RSAM learns the mutual dependencies of multiple LR-HR pairs and multi-scale features, while RCAM accentuates key features through interactive learning, effectively addressing detail loss. Extensive experiments demonstrate that REMA significantly improves performance and compatibility in SR networks compared to other attention modules. The REMA-based SR network (REMA-SRNet) outperforms comparative algorithms in both visual effects and objective evaluation quality. Additionally, we find that module compatibility correlates with cardinality and in-branch feature bandwidth, and that networks with high effective parameter counts exhibit enhanced robustness across various datasets and scale factors in SISR.

Omnidirectional image super-resolution via position attention network

For convenient transmission, omnidirectional images (ODIs) usually follow the equirectangular projection (ERP) format and are low-resolution. To provide better immersive experience, omnidirectional image super resolution (ODISR) is essential. However, ERP ODIs suffer from serious geometric distortion and pixel stretching across latitudes, generating massive redundant information at high latitudes. This characteristic poses a huge challenge for the traditional SR methods, which can only obtain the suboptimal ODISR performance. To address this issue, we propose a novel position attention network (PAN) for ODISR in this paper. Specifically, a two-branch structure is introduced, in which the basic enhancement branch (BE) serves to achieve coarse deep feature enhancement for extracted shallow features. Meanwhile, the position attention enhancement branch (PAE) builds a positional attention mechanism to dynamically adjust the contribution of features at different latitudes in the ERP representation according to their positions and stretching degrees, which achieves the enhancement for the differentiated information, suppresses the redundant information, and modulate the deep features with spatial distortion. Subsequently, the features of two branches are fused effectively to achieve the further refinement and adapt the distortion characteristic of ODIs. After that, we exploit a long-term memory module (LM), promoting information interactions and fusions between the branches to enhance the perception of the distortion, aggregating the prior hierarchical features to keep the long-term memory and boosting the ODISR performance. Extensive results demonstrate the state-of-the-art performance and the high efficiency of our PAN in ODISR.

Double Transformer Super-Resolution for Breast Cancer ADC Images

Diffusion-weighted imaging (DWI) has been extensively explored in guiding the clinic management of patients with breast cancer. However, due to the limited resolution, accurately characterizing tumors using DWI and the corresponding apparent diffusion coefficient (ADC) is still a challenging problem. In this paper, we aim to address the issue of super-resolution (SR) of ADC images and evaluate the clinical utility of SR-ADC images through radiomics analysis. To this end, we propose a novel double transformer-based network (DTformer) to enhance the resolution of ADC images. More specifically, we propose a symmetric U-shaped encoder-decoder network with two different types of transformer blocks, named as UTNet, to extract deep features for super-resolution. The basic backbone of UTNet is composed of a locally-enhanced Swin transformer block (LeSwin-T) and a convolutional transformer block (Conv-T), which are responsible for capturing long-range dependencies and local spatial information, respectively. Additionally, we introduce a residual upsampling network (RUpNet) to expand image resolution by leveraging initial residual information from the original low-resolution (LR) images. Extensive experiments show that DTformer achieves superior SR performance. Moreover, radiomics analysis reveals that improving the resolution of ADC images is beneficial for tumor characteristic prediction, such as histological grade and human epidermal growth factor receptor 2 (HER2) status.

CVANet: Cascaded visual attention network for single image super-resolution

Deep convolutional neural networks (DCNNs) have exhibited excellent feature extraction and detail reconstruction capabilities for single image super-resolution (SISR). Nevertheless, most previous DCNN-based methods do not fully utilize the complementary strengths between feature maps, channels, and pixels. Therefore, it hinders the ability of DCNNs to represent abundant features. To tackle the aforementioned issues, we present a Cascaded Visual Attention Network for SISR called CVANet, which simulates the visual attention mechanism of the human eyes to focus on the reconstruction process of details. Specifically, we first designed a trainable feature attention module (FAM) for feature-level attention learning. Afterward, we introduce a channel attention module (CAM) to reinforce feature maps under channel-level attention learning. Meanwhile, we propose a pixel attention module (PAM) that adaptively selects representative features from the previous layers, which are utilized to generate a high-resolution image. Satisfactory, our CVANet can effectively improve the resolution of images by exploring the feature representation capabilities of different modules and the visual perception properties of the human eyes. Extensive experiments with different methods on four benchmarks demonstrate that our CVANet outperforms the state-of-the-art (SOTA) methods in subjective visual perception, PSNR, and SSIM.The code will be made available https://github.com/WilyZhao8/CVANet.

An efficient multi-scale learning method for image super-resolution networks

The image super-resolution (SR) operation holds multiple solutions with the one-to-many mapping from low-resolution (LR) to high-resolution (HR) space. However, the SR of different scales for the same image is usually regarded as independent tasks in the existing SR networks. Therefore, these networks are inflexible to effectively utilize feature learning experience and require much more computing time to recover HR images in higher resolutions. Recent arbitrary scale SR methods still cannot solve these problems. To efficiently and effectively recover HR images, this paper presents an efficient multi-scale learning method for image SR networks based on a novel self-generating (SG) mechanism. This method (briefly named SG-SR) utilizes the feature learning results of SR networks to generate upscale filters by using the novel SG upscale module, which is proposed to replace the traditional upscale module. For each scale factor, the SG upscale module provides the corresponding amount of the spatial weights to filter the LR tensor and then converts filtered tensors with the original tensor to corresponding HR images. The proposed method is evaluated through extensive experiments and compared with state-of-the-art (SOTA) methods on widely used benchmark datasets. The experimental results show that our method has superior performance compared with SOTA methods, and the SG upscale module can improve the performance of existing SR networks effectively. What is more, our module has a much less calculation cost than the other upscale modules.

Multi-scale feature selection network for lightweight image super-resolution

Recently, many super-resolution (SR) methods based on convolutional neural networks (CNNs) have achieved superior performance by utilizing deep and heavy models, which may not be suitable for real-world low-budget devices. To address this issue, we propose a novel lightweight SR network called a multi-scale feature selection network (MFSN). As the basic building block of MFSN, the multi-scale feature selection block (MFSB) is presented to extract the rich multi-scale features from a coarse-to-fine receptive field level. For a better representation ability, a wide-activated residual unit is adopted in each branch of MFSB except the last one. In MFSB, the scale selection module (SSM) is designed to effectively fuse the features from two adjacent branches by adjusting receptive field sizes adaptively. Further, a comprehensive channel attention mechanism (CCAM) is integrated into SSM to learn the dynamic selection weight by considering the local and global inter-channel dependencies. Extensive experimental results illustrate that the proposed MFSN is superior to other lightweight methods.

A Lightweight Recurrent Grouping Attention Network for Video Super-Resolution

Effective aggregation of temporal information of consecutive frames is the core of achieving video super-resolution. Many scholars have utilized structures such as sliding windows and recurrences to gather the spatio-temporal information of frames. However, although the performances of constructed video super-resolution models are improving, the sizes of the models are also increasing, exacerbating the demand on the equipment. Thus, to reduce the stress on the device, we propose a novel lightweight recurrent grouping attention network. The parameters of this model are only 0.878 M, which is much lower than the current mainstream model for studying video super-resolution. We have designed a forward feature extraction module and a backward feature extraction module to collect temporal information between consecutive frames from two directions. Moreover, a new grouping mechanism is proposed to efficiently collect spatio-temporal information of the reference frame and its neighboring frames. The attention supplementation module is presented to further enhance the information gathering range of the model. The feature reconstruction module aims to aggregate information from different directions to reconstruct high-resolution features. Experiments demonstrate that our model achieves state-of-the-art performance on multiple datasets.

A super-resolution network for medical imaging via transformation analysis of wavelet multi-resolution

In recent years, deep learning super-resolution models for progressive reconstruction have achieved great success. However, these models which refer to multi-resolution analysis basically ignore the information contained in the lower subspaces and do not explore the correlation between features in the wavelet and spatial domain, resulting in not fully utilizing the auxiliary information brought by multi-resolution analysis with multiple domains. Therefore, we propose a super-resolution network based on the wavelet multi-resolution framework (WMRSR) to capture the auxiliary information contained in multiple subspaces and to be aware of the interdependencies between spatial domain and wavelet domain features. Initially, the wavelet multi-resolution input (WMRI) is generated by combining wavelet sub-bands obtained from each subspace through wavelet multi-resolution analysis and the corresponding spatial domain image content, which serves as input to the network. Then, the WMRSR captures the corresponding features from the WMRI in the wavelet domain and spatial domain, respectively, and fuses them adaptively, thus learning fully explored features in multi-resolution and multi-domain. Finally, the high-resolution images are gradually reconstructed in the wavelet multi-resolution framework by our convolution-based wavelet transform module which is suitable for deep neural networks. Extensive experiments conducted on two public datasets demonstrate that our method outperforms other state-of-the-art methods in terms of objective and visual qualities.

Lightweight image super-resolution based multi-order gated aggregation network

Recently, Transformer-based models are taken much focus on solving the task of image super-resolution (SR) due to their ability to achieve better performance. However, these models combined huge computational cost during the computing self-attention mechanism. To solve this problem, we proposed a multi-order gated aggregation super-resolution network (MogaSRN) for low-level vision based on the concept of the MogaNet that is developed for high-level vision. The concept of the MogaSRN model is based on spatial multi-order context aggregation and adaptive channel-wise reallocation with the aid of the multi-layer perceptron (MLP). In contrast to the MogaNet model, in which the resolution of each stage decreased by a factor of 2, the resolution of the MogaSRN is stayed fixed during the deep features extraction. Moreover, the structure of the MogaSRN model is built based on balancing the performance and the model complexity. We evaluated our model based on five benchmark datasets concluding that the MogaSRN model can achieve significant improvements compared to the state-of-the-art. Moreover, our model shows the good visual quality and accuracy of the reconstruction. Finally, our model has 3.7 × faster runtime at the scale of × 4 compared to LWSwinIR with better performance.

A Network with Composite Loss and Parameter-free Chunking Fusion Block for Super-Resolution MR Image

MRI is often influenced by many factors, and single image super-resolution (SISR) based on a neural network is an effective and cost-effective alternative technique for the high-resolution restoration of low-resolution images. However, deep neural networks can easily lead to overfitting and make the test results worse. The network with a shallow training network is difficult to fit quickly and cannot completely learn training samples. To solve the above problems, a new end-to-end super-resolution (SR) method is proposed for magnetic resonance (MR) images. Firstly, in order to better fuse features, a parameter-free chunking fusion block (PCFB) is proposed, which can divide the feature map into n branches by splitting channels to obtain parameter-free attention. Secondly, the proposed training strategy including perceptual loss, gradient loss, and L1 loss has significantly improved the accuracy of model fitting and prediction. Finally, the proposed model and training strategy take the super-resolution IXISR dataset (PD, T1, and T2) as an example to compare with the existing excellent methods and obtain advanced performance. A large number of experiments have proved that the proposed method performs better than the advanced methods in highly reliable measurement.

Restoration and enhancement on low exposure raw images by joint demosaicing and denoising

Restoring high quality images from raw data in low light is challenging due to various noises caused by limited photon count and complicated Image Signal Process (ISP). Although several restoration and enhancement approaches are proposed, they may fail in extreme conditions, such as imaging short exposure raw data. The first path-breaking attempt is to utilize the connection between a pair of short and long exposure raw data and outputs RGB images as the final results. However, the whole pipeline still suffers from some blurs and color distortion. To overcome those difficulties, we propose an end-to-end network that contains two effective subnets to joint demosaic and denoise low exposure raw images. While traditional ISP are difficult to image them in acceptable conditions, the short exposure raw images can be better restored and enhanced by our model. For denoising, the proposed Short2Long raw restoration subnet outputs pseudo long exposure raw data with little noisy points. Then for demosaicing, the proposed Color consistent RGB enhancement subnet generates corresponding RGB images with the desired attributes: sharpness, color vividness, good contrast and little noise. By training the network in an end-to-end manner, our method avoids additional tuning by experts. We conduct experiments to reveal good results on three raw data datasets. We also illustrate the effectiveness of each module and the well generalization ability of this model.

Self-attention learning network for face super-resolution

Existing face super-resolution methods depend on deep convolutional networks (DCN) to recover high-quality reconstructed images. They either acquire information in a single space by designing complex models for direct reconstruction, or employ additional networks to extract multiple prior information to enhance the representation of features. However, existing methods are still challenging to perform well due to the inability to learn complete and uniform representations. To this end, we propose a self-attention learning network (SLNet) for three-stage face super-resolution, which fully explores the interdependence of low- and high-level spaces to achieve compensation of the information used for reconstruction. Firstly, SLNet uses a hierarchical feature learning framework to obtain shallow information in the low-level space. Then, the shallow information with cumulative errors due to DCN is improved under high-resolution (HR) supervision, while bringing an intermediate reconstruction result and a powerful intermediate benchmark. Finally, the improved feature representation is further enhanced in high-level space by a multi-scale context-aware encoder-decoder for facial reconstruction. The features in both spaces are explored progressively from coarse to fine reconstruction information. The experimental results show that SLNet has a competitive performance compared to the state-of-the-art methods.

GCS-YOLOV4-Tiny: A lightweight group convolution network for multi-stage fruit detection

Fruits require different planting techniques at different growth stages. Traditionally, the maturity stage of fruit is judged visually, which is time-consuming and labor-intensive. Fruits differ in size and color, and sometimes leaves or branches occult some of fruits, limiting automatic detection of growth stages in a real environment. Based on YOLOV4-Tiny, this study proposes a GCS-YOLOV4-Tiny model by (1) adding squeeze and excitation (SE) and the spatial pyramid pooling (SPP) modules to improve the accuracy of the model and (2) using the group convolution to reduce the size of the model and finally achieve faster detection speed. The proposed GCS-YOLOV4-Tiny model was executed on three public fruit datasets. Results have shown that GCS-YOLOV4-Tiny has favorable performance on mAP, Recall, F1-Score and Average IoU on Mango YOLO and Rpi-Tomato datasets. In addition, with the smallest model size of 20.70 MB, the mAP, Recall, F1-score, Precision and Average IoU of GCS-YOLOV4-Tiny achieve 93.42 ± 0.44, 91.00 ± 1.87, 90.80 ± 2.59, 90.80 ± 2.77 and 76.94 ± 1.35%, respectively, on F. margarita dataset. The detection results outperform the state-of-the-art YOLOV4-Tiny model with a 17.45% increase in mAP and a 13.80% increase in F1-score. The proposed model provides an effective and efficient performance to detect different growth stages of fruits and can be extended for different fruits and crops for object or disease detections.

Interaction of Secure Cloud Network and Crowd Computing for Smart City Data Obfuscation

There can be many inherent issues in the process of managing cloud infrastructure and the platform of the cloud. The platform of the cloud manages cloud software and legality issues in making contracts. The platform also handles the process of managing cloud software services and legal contract-based segmentation. In this paper, we tackle these issues directly with some feasible solutions. For these constraints, the Averaged One-Dependence Estimators (AODE) classifier and the SELECT Applicable Only to Parallel Server (SELECT-APSL ASA) method are proposed to separate the data related to the place. ASA is made up of the AODE and SELECT Applicable Only to Parallel Server. The AODE classifier is used to separate the data from smart city data based on the hybrid data obfuscation technique. The data from the hybrid data obfuscation technique manages 50% of the raw data, and 50% of hospital data is masked using the proposed transmission. The analysis of energy consumption before the cryptosystem shows the total packet delivered by about 71.66% compared with existing algorithms. The analysis of energy consumption after cryptosystem assumption shows 47.34% consumption, compared to existing state-of-the-art algorithms. The average energy consumption before data obfuscation decreased by 2.47%, and the average energy consumption after data obfuscation was reduced by 9.90%. The analysis of the makespan time before data obfuscation decreased by 33.71%. Compared to existing state-of-the-art algorithms, the study of makespan time after data obfuscation decreased by 1.3%. These impressive results show the strength of our methodology.