Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants

Super-resolution biomedical imaging via reference-free statistical implicit neural representation

Objective.Supervised deep learning for image super-resolution (SR) has limitations in biomedical imaging due to the lack of large amounts of low- and high-resolution image pairs for model training. In this work, we propose a reference-free statistical implicit neural representation (INR) framework, which needs only a single or a few observed low-resolution (LR) image(s), to generate high-quality SR images.Approach.The framework models the statistics of the observed LR images via maximum likelihood estimation and trains the INR network to represent the latent high-resolution (HR) image as a continuous function in the spatial domain. The INR network is constructed as a coordinate-based multi-layer perceptron, whose inputs are image spatial coordinates and outputs are corresponding pixel intensities. The trained INR not only constrains functional smoothness but also allows an arbitrary scale in SR imaging.Main results.We demonstrate the efficacy of the proposed framework on various biomedical images, including computed tomography (CT), magnetic resonance imaging (MRI), fluorescence microscopy, and ultrasound images, across different SR magnification scales of 2×, 4×, and 8×. A limited number of LR images were used for each of the SR imaging tasks to show the potential of the proposed statistical INR framework.Significance.The proposed method provides an urgently needed unsupervised deep learning framework for numerous biomedical SR applications that lack HR reference images.

Super-resolution reconstruction based on Gaussian transform and attention mechanism

Image super-resolution reconstruction can reconstruct low resolution blurred images in the same scene into high-resolution images. Combined with multi-scale Gaussian difference transform, attention mechanism and feedback mechanism are introduced to construct a new super-resolution reconstruction network. Three improvements are made. Firstly, its multi-scale Gaussian difference transform can strengthen the details of low resolution blurred images. Secondly, it introduces the attention mechanism and increases the network depth to better express the high-frequency features. Finally, pixel loss function and texture loss function are used together, focusing on the learning of structure and texture respectively. The experimental results show that this method is superior to the existing methods in quantitative and qualitative indexes, and promotes the recovery of high-frequency detail information.

URNet: A U-Shaped Residual Network for Lightweight Image Super-Resolution

It is extremely important and necessary for low computing power or portable devices to design more lightweight algorithms for image super-resolution (SR). Recently, most SR methods have achieved outstanding performance by sacrificing computational cost and memory storage, or vice versa. To address this problem, we introduce a lightweight U-shaped residual network (URNet) for fast and accurate image SR. Specifically, we propose a more effective feature distillation pyramid residual group (FDPRG) to extract features from low-resolution images. The FDPRG can effectively reuse the learned features with dense shortcuts and capture multi-scale information with a cascaded feature pyramid block. Based on the U-shaped structure, we utilize a step-by-step fusion strategy to improve the performance of feature fusion of different blocks. This strategy is different from the general SR methods which only use a single Concat operation to fuse the features of all basic blocks. Moreover, a lightweight asymmetric residual non-local block is proposed to model the global context information and further improve the performance of SR. Finally, a high-frequency loss function is designed to alleviate smoothing image details caused by pixel-wise loss. Simultaneously, the proposed modules and high-frequency loss function can be easily plugged into multiple mature architectures to improve the performance of SR. Extensive experiments on multiple natural image datasets and remote sensing image datasets show the URNet achieves a better trade-off between image SR performance and model complexity against other state-of-the-art SR methods.

Multiframe Super-Resolution of Color Images Based on Cross Channel Prior

Color images have a wider range of applications than gray images. There are two ways to extend the traditional super-resolution reconstruction method to color images: Super resolution reconstructs each channel of the color image individually; Change the RGB color bands into YCrCb color bands, then super-resolution reconstructs the luminance component and interpolates the chrominance components.These algorithms cannot effectively utilize the property that the edges and textures are similar in the RGB channels, and the results of those methods may lead to color artifacts. Aiming to solve these problems, we propose a new super-resolution method based on cross channel prior. First, a cross channel prior is proposed to describe the similarity of gradient in RGB channels. Then, a new super-resolution method is proposed for color images via combination of the cross channel prior and the traditional super-resolution methods. Finally, the proposed method reconstructs the color channels alternately. The experimental results show that the proposed method could effectively suppress the generation of color artifacts and improve the quality of the reconstructed images.

Super-Resolution of Magnetic Resonance Images via Convex Optimization with Local and Global Prior Regularization and Spectrum Fitting

Given a low-resolution image, there are many challenges to obtain a super-resolved, high-resolution image. Many of those approaches try to simultaneously upsample and deblur an image in signal domain. However, the nature of the super-resolution is to restore high-frequency components in frequency domain rather than upsampling in signal domain. In that sense, there is a close relationship between super-resolution of an image and extrapolation of the spectrum. In this study, we propose a novel framework for super-resolution, where the high-frequency components are theoretically restored with respect to the frequency fidelities. This framework helps to introduce multiple simultaneous regularizers in both signal and frequency domains. Furthermore, we propose a new super-resolution model where frequency fidelity, low-rank (LR) prior, low total variation (TV) prior, and boundary prior are considered at once. The proposed method is formulated as a convex optimization problem which can be solved by the alternating direction method of multipliers. The proposed method is the generalized form of the multiple super-resolution methods such as TV super-resolution, LR and TV super-resolution, and the Gerchberg method. Experimental results show the utility of the proposed method comparing with some existing methods using both simulational and practical images.

Super resolution image reconstruction through Bregman iteration using morphologic regularization

Multiscale morphological operators are studied extensively in the literature for image processing and feature extraction purposes. In this paper, we model a nonlinear regularization method based on multiscale morphology for edge-preserving super resolution (SR) image reconstruction. We formulate SR image reconstruction as a deblurring problem and then solve the inverse problem using Bregman iterations. The proposed algorithm can suppress inherent noise generated during low-resolution image formation as well as during SR image estimation efficiently. Experimental results show the effectiveness of the proposed regularization and reconstruction method for SR image.

Statistical performance modeling for superresolution: a discrete data-continuous reconstruction framework

We address performance modeling of superresolution (SR) techniques. Superresolution consists in combining several images of the same scene to produce an image with better resolution and contrast. We propose a discrete data-continuous reconstruction framework to conduct SR performance analysis and derive a theoretical expression of the reconstruction mean squared error (MSE) as a compact, computationally tractable function of signal-to-noise ratio (SNR), scene model, sensor transfer function, number of frames, interframe translation motion, and SR reconstruction filter. A formal expression for the MSE is obtained that allows a qualitative study of SR behavior. In particular we provide an original outlook on the balance between noise and aliasing reduction in linear SR. Explicit account for the SR reconstruction filter is an original feature of our model. It allows for the first time to study not only optimal filters but also suboptimal ones, which are often used in practice.

An improved observation model for super-resolution under affine motion

Super-resolution (SR) techniques make use of subpixel shifts between frames in an image sequence to yield higher resolution images. We propose an original observation model devoted to the case of nonisometric inter-frame motion as required, for instance, in the context of airborne imaging sensors. First, we describe how the main observation models used in the SR literature deal with motion, and we explain why they are not suited for nonisometric motion. Then, we propose an extension of the observation model by Elad and Feuer adapted to affine motion. This model is based on a decomposition of affine transforms into successive shear transforms, each one efficiently implemented by row-by-row or column-by-column one-dimensional affine transforms. We demonstrate on synthetic and real sequences that our observation model incorporated in a SR reconstruction technique leads to better results in the case of variable scale motions and it provides equivalent results in the case of isometric motions.

Super-resolution reconstruction of compressed video using transform-domain statistics

Considerable attention has been directed to the problem of producing high-resolution video and still images from multiple low-resolution images. This multiframe reconstruction, also known as super-resolution reconstruction, is beginning to be applied to compressed video. Super-resolution techniques that have been designed for raw (i.e., uncompressed) video may not be effective when applied to compressed video because they do not incorporate the compression process into their models. The compression process introduces quantization error, which is the dominant source of error in some cases. In this paper, we propose a stochastic framework where quantization information as well as other statistical information about additive noise and image prior can be utilized effectively.

Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration

In this paper, we propose a high-resolution image reconstruction algorithm considering inaccurate subpixel registration. A regularized iterative reconstruction algorithm is adopted to overcome the ill-posedness problem resulting from inaccurate subpixel registration. In particular, we use multichannel image reconstruction algorithms suitable for applications with multiframe environments. Since the registration error in each low-resolution image has a different pattern, the regularization parameters are determined adaptively for each channel. We propose two methods for estimating the regularization parameter automatically. The proposed algorithms are robust against registration error noise, and they do not require any prior information about the original image or the registration error process. Information needed to determine the regularization parameter and to reconstruct the image is updated at each iteration step based on the available partially reconstructed image. Experimental results indicate that the proposed algorithms outperform conventional approaches in terms of both objective measurements and visual evaluation.