Synthetic imaging through wavy water surface with centroid evolution

Inverted pyramid frame forward and backward prediction for distorted video by water waves

There has been much research on how to restore a single image from distorted video. Random water surface variation, an inability to model the surface, and multiple factors in the imaging processing leading to different geometric distortions in each frame are among the challenges. This paper proposes an inverted pyramid structure based on the cross optical flow registration approach and a multi-scale weight fusion method based on wavelet decomposition. The inverted pyramid based on the registration method is used to estimate the original pixel positions. A multi-scale image fusion method is applied to fuse the two inputs processed by optical flow and backward mapping, and two iterations are proposed to improve the accuracy and stability of the output video. The method is tested on several reference distorted videos and our videos, which were obtained through our experimental equipment. The obtained results exhibit significant improvements over other reference methods. The corrected videos obtained with our approach have a higher degree of sharpness, and the time required to restore the videos is significantly reduced.

Water-Air Interface Imaging: Recovering the Images Distorted by Surface Waves via an Efficient Registration Algorithm

Imaging through the wavy water-air interface is challenging since the random fluctuations of water will cause complex geometric distortion and motion blur in the images, seriously affecting the effective identification of the monitored object. Considering the problems of image recovery accuracy and computational efficiency, an efficient reconstruction scheme that combines lucky-patch search and image registration technologies was proposed in this paper. Firstly, a high-quality reference frame is rebuilt using a lucky-patch search strategy. Then an iterative registration algorithm is employed to remove severe geometric distortions by registering warped frames to the reference frame. During the registration process, we integrate JADE and LBFGS algorithms as an optimization strategy to expedite the control parameter optimization process. Finally, the registered frames are refined using PCA and the lucky-patch search algorithm to remove residual distortions and random noise. Experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of sharpness and contrast.

Seeing through Wavy Water–Air Interface: A Restoration Model for Instantaneous Images Distorted by Surface Waves

Imaging through a wavy water–air interface is challenging since light rays are bent by unknown amounts, leading to complex geometric distortions. Considering the restoration of instantaneous distorted images, this paper proposes an image recovery model via structured light projection. The algorithm is composed of two separate parts. In the first part, an algorithm for the determination of the instantaneous shape of the water surface via structured light projection is developed. Then, we synchronously recover the distorted airborne scene image through reverse ray tracing in the second part. The experimental results show that, compared with the state-of-the-art methods, the proposed method not only can overcome the influence of changes in natural illumination conditions for WAI reconstruction, but also can significantly reduce the distortion and achieve better performance.

Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network

Imaging through the wavy air-water surface suffers from severe geometric distortions, which are caused by the light refraction effect that affects the normal operations of underwater exploration equipment such as the autonomous underwater vehicle (AUV). In this paper, we propose a deep learning-based framework, namely the self-attention generative adversarial network (SAGAN), to remove the geometric distortions and restore the distorted image captured through the water-air surface. First, a K-means-based image pre-selection method is employed to acquire a less distorted image that preserves much useful information from an image sequence. Second, an improved generative adversarial network (GAN) is trained to translate the distorted image into the non-distorted image. During this process, the attention mechanism and the weighted training objective are adopted in our GAN framework to get the high-quality restored results of distorted underwater images. The network is able to restore the colors and fine details in the distorted images by combining the three objective losses, i.e., the content loss, the adversarial loss, and the perceptual loss. Experimental results show that our proposed method outperforms other state-of-the-art methods on the validation set and our sea trial set.

Reconstruction of distorted underwater images using robust registration

Imaging through a fluctuating air-water surface is a challenging task since light rays bent by unknown amounts lead to complex geometric distortions. This paper presents a new algorithm to undistort dynamic refractive effects. An iterative robust registration algorithm is employed to overcome the structural turbulence of the waves of the frames by registering each frame to a reference frame. To get the high-quality reference frame, the image is reconstructed by the patches selected from the sequence frames. A blind deconvolution algorithm is also performed to improve the reference frame. Experiments show our method exhibits significant improvement over other methods.