Noniterative interpolation-based super-resolution minimizing aliasing in the reconstructed image

Super-Resolution Reconstruction of Cytoskeleton Image Based on A-Net Deep Learning Network

To date, live-cell imaging at the nanometer scale remains challenging. Even though super-resolution microscopy methods have enabled visualization of sub-cellular structures below the optical resolution limit, the spatial resolution is still far from enough for the structural reconstruction of biomolecules in vivo (i.e., ~24 nm thickness of microtubule fiber). In this study, a deep learning network named A-net was developed and shows that the resolution of cytoskeleton images captured by a confocal microscope can be significantly improved by combining the A-net deep learning network with the DWDC algorithm based on a degradation model. Utilizing the DWDC algorithm to construct new datasets and taking advantage of A-net neural network's features (i.e., considerably fewer layers and relatively small dataset), the noise and flocculent structures which originally interfere with the cellular structure in the raw image are significantly removed, with the spatial resolution improved by a factor of 10. The investigation shows a universal approach for exacting structural details of biomolecules, cells and organs from low-resolution images.

TDPN: Texture and Detail-Preserving Network for Single Image Super-Resolution

Single image super-resolution (SISR) using deep convolutional neural networks (CNNs) achieves the state-of-the-art performance. Most existing SISR models mainly focus on pursuing high peak signal-to-noise ratio (PSNR) and neglect textures and details. As a result, the recovered images are often perceptually unpleasant. To address this issue, in this paper, we propose a texture and detail-preserving network (TDPN), which focuses not only on local region feature recovery but also on preserving textures and details. Specifically, the high-resolution image is recovered from its corresponding low-resolution input in two branches. First, a multi-reception field based branch is designed to let the network fully learn local region features by adaptively selecting local region features in different reception fields. Then, a texture and detail-learning branch supervised by the textures and details decomposed from the ground-truth high resolution image is proposed to provide additional textures and details for the super-resolution process to improve the perceptual quality. Finally, we introduce a gradient loss into the SISR field and define a novel hybrid loss to strengthen boundary information recovery and to avoid overly smooth boundary in the final recovered high-resolution image caused by using only the MAE loss. More importantly, the proposed method is model-agnostic, which can be applied to most off-the-shelf SISR networks. The experimental results on public datasets demonstrate the superiority of our TDPN on most state-of-the-art SISR methods in PSNR, SSIM and perceptual quality. We will share our code on https://github.com/tocaiqing/TDPN.

RSPCN: Super-Resolution of Digital Elevation Model Based on Recursive Sub-Pixel Convolutional Neural Networks

The digital elevation model (DEM) is known as one kind of the most significant fundamental geographical data models. The theory, method and application of DEM are hot research issues in geography, especially in geomorphology, hydrology, soil and other related fields. In this paper, we improve the efficient sub-pixel convolutional neural networks (ESPCN) and propose recursive sub-pixel convolutional neural networks (RSPCN) to generate higher-resolution DEMs (HRDEMs) from low-resolution DEMs (LRDEMs). Firstly, the structure of RSPCN is described in detail based on recursion theory. This paper explores the effects of different training datasets, with the self-adaptive learning rate Adam algorithm optimizing the model. Furthermore, the adding-“zero” boundary method is introduced into the RSPCN algorithm as a data preprocessing method, which improves the RSPCN method’s accuracy and convergence. Extensive experiments are conducted to train the method till optimality. Finally, comparisons are made with other traditional interpolation methods, such as bicubic, nearest-neighbor and bilinear methods. The results show that our method has obvious improvements in both accuracy and robustness and further illustrate the feasibility of deep learning methods in the DEM data processing area.

Multimodal Deep Unfolding for Guided Image Super-Resolution

The reconstruction of a high resolution image given a low resolution observation is an ill-posed inverse problem in imaging. Deep learning methods rely on training data to learn an end-to-end mapping from a low-resolution input to a highresolution output. Unlike existing deep multimodal models that do not incorporate domain knowledge about the problem, we propose a multimodal deep learning design that incorporates sparse priors and allows the effective integration of information from another image modality into the network architecture. Our solution relies on a novel deep unfolding operator, performing steps similar to an iterative algorithm for convolutional sparse coding with side information; therefore, the proposed neural network is interpretable by design. The deep unfolding architecture is used as a core component of a multimodal framework for guided image super-resolution. An alternative multimodal design is investigated by employing residual learning to improve the training efficiency. The presented multimodal approach is applied to super-resolution of near-infrared and multi-spectral images as well as depth upsampling using RGB images as side information. Experimental results show that our model outperforms state-ofthe-art methods.

A Comparable Study of CNN-Based Single Image Super-Resolution for Space-Based Imaging Sensors

In the case of space-based space surveillance (SBSS), images of the target space objects captured by space-based imaging sensors usually suffer from low spatial resolution due to the extremely long distance between the target and the imaging sensor. Image super-resolution is an effective data processing operation to get informative high resolution images. In this paper, we comparably study four recent popular models for single image super-resolution based on convolutional neural networks (CNNs) with the purpose of space applications. We specially fine-tune the super-resolution models designed for natural images using simulated images of space objects, and test the performance of different CNN-based models in different conditions that are mainly considered for SBSS. Experimental results show the advantages and drawbacks of these models, which could be helpful for the choice of proper CNN-based super-resolution method to deal with image data of space objects.

Interpolation-based image super-resolution using multisurface fitting

In this paper, we propose a new interpolation-based method of image super-resolution reconstruction. The idea is using multisurface fitting to take full advantage of spatial structure information. Each site of low-resolution pixels is fitted with one surface, and the final estimation is made by fusing the multisampling values on these surfaces in the maximum a posteriori fashion. With this method, the reconstructed high-resolution images preserve image details effectively without any hypothesis on image prior. Furthermore, we extend our method to a more general noise model. Experimental results on the simulated and real-world data show the superiority of the proposed method in both quantitative and visual comparisons.

Super-resolution without dense flow

Super-resolution is a widely applied technique that improves the resolution of input images by software methods. Most conventional reconstruction-based super-resolution algorithms assume accurate dense optical flow fields between the input frames, and their performance degrades rapidly when the motion estimation result is not accurate enough. However, optical flow estimation is usually difficult, particularly when complicated motion is presented in real-world videos. In this paper, we explore a new way to solve this problem by using sparse feature point correspondences between the input images. The feature point correspondences, which are obtained by matching a set of feature points, are usually precise and much more robust than dense optical flow fields. This is because the feature points represent well-selected significant locations in the image, and performing matching on the feature point set is usually very accurate. In order to utilize the sparse correspondences in conventional super-resolution, we extract an adaptive support region with a reliable local flow field from each corresponding feature point pair. The normalized prior is also proposed to increase the visual consistency of the reconstructed result. Extensive experiments on real data were carried out, and results show that the proposed algorithm produces high-resolution images with better quality, particularly in the presence of large-scale or complicated motion fields.

Spatially adaptive block-based super-resolution

Super-resolution technology provides an effective way to increase image resolution by incorporating additional information from successive input images or training samples. Various super-resolution algorithms have been proposed based on different assumptions, and their relative performances can differ in regions of different characteristics within a single image. Based on this observation, an adaptive algorithm is proposed in this paper to integrate a higher level image classification task and a lower level super-resolution process, in which we incorporate reconstruction-based super-resolution algorithms, single-image enhancement, and image/video classification into a single comprehensive framework. The target high-resolution image plane is divided into adaptive-sized blocks, and different suitable super-resolution algorithms are automatically selected for the blocks. Then, a deblocking process is applied to reduce block edge artifacts. A new benchmark is also utilized to measure the performance of super-resolution algorithms. Experimental results with real-life videos indicate encouraging improvements with our method.