Super-resolution without dense flow

Coupled Deep Autoencoder for Single Image Super-Resolution

Sparse coding has been widely applied to learning-based single image super-resolution (SR) and has obtained promising performance by jointly learning effective representations for low-resolution (LR) and high-resolution (HR) image patch pairs. However, the resulting HR images often suffer from ringing, jaggy, and blurring artifacts due to the strong yet ad hoc assumptions that the LR image patch representation is equal to, is linear with, lies on a manifold similar to, or has the same support set as the corresponding HR image patch representation. Motivated by the success of deep learning, we develop a data-driven model coupled deep autoencoder (CDA) for single image SR. CDA is based on a new deep architecture and has high representational capability. CDA simultaneously learns the intrinsic representations of LR and HR image patches and a big-data-driven function that precisely maps these LR representations to their corresponding HR representations. Extensive experimentation demonstrates the superior effectiveness and efficiency of CDA for single image SR compared to other state-of-the-art methods on Set5 and Set14 datasets.

On Bayesian adaptive video super resolution

Although multiframe super resolution has been extensively studied in past decades, super resolving real-world video sequences still remains challenging. In existing systems, either the motion models are oversimplified or important factors such as blur kernel and noise level are assumed to be known. Such models cannot capture the intrinsic characteristics that may differ from one sequence to another. In this paper, we propose a Bayesian approach to adaptive video super resolution via simultaneously estimating underlying motion, blur kernel, and noise level while reconstructing the original high-resolution frames. As a result, our system not only produces very promising super resolution results outperforming the state of the art, but also adapts to a variety of noise levels and blur kernels. To further analyze the effect of noise and blur kernel, we perform a two-step analysis using the Cramer-Rao bounds. We study how blur kernel and noise influence motion estimation with aliasing signals, how noise affects super resolution with perfect motion, and finally how blur kernel and noise influence super resolution with unknown motion. Our analysis results confirm empirical observations, in particular that an intermediate size blur kernel achieves the optimal image reconstruction results.

Unified multiframe super-resolution of matte, foreground, and background

Reconstruction of a super-resolved image from multiple frames and extraction of matte are two popular topics that have been solved independently. In this paper, we advocate a unified framework that assimilates matting within the super-resolution model. We show that joint estimation is advantageous, as super-resolved edge information helps in obtaining a sharp matte, while the matte in turn aids in resolving fine details. We propose a multiframe approach to increase the spatial resolution of the matte, foreground, and background. This is validated extensively on examples from standard matting datasets.