Reconstruction of distorted underwater images using robust registration

Inverted pyramid frame forward and backward prediction for distorted video by water waves

There has been much research on how to restore a single image from distorted video. Random water surface variation, an inability to model the surface, and multiple factors in the imaging processing leading to different geometric distortions in each frame are among the challenges. This paper proposes an inverted pyramid structure based on the cross optical flow registration approach and a multi-scale weight fusion method based on wavelet decomposition. The inverted pyramid based on the registration method is used to estimate the original pixel positions. A multi-scale image fusion method is applied to fuse the two inputs processed by optical flow and backward mapping, and two iterations are proposed to improve the accuracy and stability of the output video. The method is tested on several reference distorted videos and our videos, which were obtained through our experimental equipment. The obtained results exhibit significant improvements over other reference methods. The corrected videos obtained with our approach have a higher degree of sharpness, and the time required to restore the videos is significantly reduced.

Water-Air Interface Imaging: Recovering the Images Distorted by Surface Waves via an Efficient Registration Algorithm

Imaging through the wavy water-air interface is challenging since the random fluctuations of water will cause complex geometric distortion and motion blur in the images, seriously affecting the effective identification of the monitored object. Considering the problems of image recovery accuracy and computational efficiency, an efficient reconstruction scheme that combines lucky-patch search and image registration technologies was proposed in this paper. Firstly, a high-quality reference frame is rebuilt using a lucky-patch search strategy. Then an iterative registration algorithm is employed to remove severe geometric distortions by registering warped frames to the reference frame. During the registration process, we integrate JADE and LBFGS algorithms as an optimization strategy to expedite the control parameter optimization process. Finally, the registered frames are refined using PCA and the lucky-patch search algorithm to remove residual distortions and random noise. Experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of sharpness and contrast.

Optical-image communication through random-phase propagation channels using phase-shifting coding

We propose an optical-image communication system robust to random-phase propagation using phase-shifting (PS) image coding. That is, this optical-image communication system is based on digital PS interferometry principles. Each pixel of the parallel transmitted image is coded as the phase of a sequence of N phase-shifted fringe patterns. The temporal fringe patterns may be displayed on a TV screen (or a multimedia projector) for transmission through the random-phase channel. At the receiver, the PS fringe patterns are digitized with a telescopic digital camera. The received fringes are phase-demodulated using an N-steps least-squares PS algorithm (LS-PSA). We show that the received, phase-demodulated images are less blurred and have better contrast than any received image without PS coding. We propose and analyze a mathematical model for the received PS fringes degraded by random-phase propagation. This PS communication system can also be used for robust optical communications through random refractive media such as underwater, air-water, or random-thickness textured glass. In particular, we show experiments for LS-PSA imaging through textured-glass, obtaining sharper images. As far as we know, this is the first time that PS interferometry has been used for parallel optical-image communication through random-phase channels.

Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network

Imaging through the wavy air-water surface suffers from severe geometric distortions, which are caused by the light refraction effect that affects the normal operations of underwater exploration equipment such as the autonomous underwater vehicle (AUV). In this paper, we propose a deep learning-based framework, namely the self-attention generative adversarial network (SAGAN), to remove the geometric distortions and restore the distorted image captured through the water-air surface. First, a K-means-based image pre-selection method is employed to acquire a less distorted image that preserves much useful information from an image sequence. Second, an improved generative adversarial network (GAN) is trained to translate the distorted image into the non-distorted image. During this process, the attention mechanism and the weighted training objective are adopted in our GAN framework to get the high-quality restored results of distorted underwater images. The network is able to restore the colors and fine details in the distorted images by combining the three objective losses, i.e., the content loss, the adversarial loss, and the perceptual loss. Experimental results show that our proposed method outperforms other state-of-the-art methods on the validation set and our sea trial set.