Correction of geometrically distorted underwater images using shift map analysis

Inverted pyramid frame forward and backward prediction for distorted video by water waves

There has been much research on how to restore a single image from distorted video. Random water surface variation, an inability to model the surface, and multiple factors in the imaging processing leading to different geometric distortions in each frame are among the challenges. This paper proposes an inverted pyramid structure based on the cross optical flow registration approach and a multi-scale weight fusion method based on wavelet decomposition. The inverted pyramid based on the registration method is used to estimate the original pixel positions. A multi-scale image fusion method is applied to fuse the two inputs processed by optical flow and backward mapping, and two iterations are proposed to improve the accuracy and stability of the output video. The method is tested on several reference distorted videos and our videos, which were obtained through our experimental equipment. The obtained results exhibit significant improvements over other reference methods. The corrected videos obtained with our approach have a higher degree of sharpness, and the time required to restore the videos is significantly reduced.

Water-Air Interface Imaging: Recovering the Images Distorted by Surface Waves via an Efficient Registration Algorithm

Imaging through the wavy water-air interface is challenging since the random fluctuations of water will cause complex geometric distortion and motion blur in the images, seriously affecting the effective identification of the monitored object. Considering the problems of image recovery accuracy and computational efficiency, an efficient reconstruction scheme that combines lucky-patch search and image registration technologies was proposed in this paper. Firstly, a high-quality reference frame is rebuilt using a lucky-patch search strategy. Then an iterative registration algorithm is employed to remove severe geometric distortions by registering warped frames to the reference frame. During the registration process, we integrate JADE and LBFGS algorithms as an optimization strategy to expedite the control parameter optimization process. Finally, the registered frames are refined using PCA and the lucky-patch search algorithm to remove residual distortions and random noise. Experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of sharpness and contrast.

Straightness metric for measurement of turbulence-induced distortion in long-range imaging

Algorithms used for mitigation of the effects of atmospheric turbulence on video sequences often rely on a process for creating a reference image to register all of the frames. Because such a pristine image is generally not available, no-reference image quality metrics can be used to identify frames in a sequence that have minimum distortion. Here, we propose a metric that quantifies image warping by measuring image straightness based on line detection. The average length of straight lines in a frame is used to select best frames in a sequence and to generate a reference frame for a subsequent dewarping algorithm. We perform tests with this metric on simulated data that exhibits varying degrees of distortion and blur and spans normalized turbulence strengths between 0.75 and 4.5. We show, through these simulations, that the metric can differentiate between weak and moderate turbulence effects. We also show in simulations that the optical flow that uses a reference frame generated by this metric produces consistently improved image quality. This improvement is even higher when we employ the metric to guide optical flow that is applied to three real video sequences taken over a 7 km path.

Optical-image communication through random-phase propagation channels using phase-shifting coding

We propose an optical-image communication system robust to random-phase propagation using phase-shifting (PS) image coding. That is, this optical-image communication system is based on digital PS interferometry principles. Each pixel of the parallel transmitted image is coded as the phase of a sequence of N phase-shifted fringe patterns. The temporal fringe patterns may be displayed on a TV screen (or a multimedia projector) for transmission through the random-phase channel. At the receiver, the PS fringe patterns are digitized with a telescopic digital camera. The received fringes are phase-demodulated using an N-steps least-squares PS algorithm (LS-PSA). We show that the received, phase-demodulated images are less blurred and have better contrast than any received image without PS coding. We propose and analyze a mathematical model for the received PS fringes degraded by random-phase propagation. This PS communication system can also be used for robust optical communications through random refractive media such as underwater, air-water, or random-thickness textured glass. In particular, we show experiments for LS-PSA imaging through textured-glass, obtaining sharper images. As far as we know, this is the first time that PS interferometry has been used for parallel optical-image communication through random-phase channels.

Robust underwater image enhancement method based on natural light and reflectivity

The poor visibility of underwater images is caused not only by scattering and absorption effects but is also related to light conditions. To improve robustness, a novel underwater image enhancement method based on natural light and reflectivity is proposed. Aiming at the scattering effects of reflectivity, a dehazing process based on the non-correlation of a foreground scene and background light is first conducted. Then, a more precise reflectivity can be estimated by substituting the captured image with the dehazed image. Moreover, classical methods often regard the dehazed image as the final result, but ignore the fact that attenuated natural light and nonuniform artificial light, which lead to insufficient brightness and halo effects, are included in the dehazed image, and are not robust to all scenes. This phenomenon enables us to remove the artificial light disturbance by introducing the dehazed image in the Lambertian model, and compensate for the loss of natural light energy by exploiting the light attenuation ratio map. Thus, the least-attenuated natural light can be further derived. Experimental results demonstrate that our method is satisfactory in producing more pleasing results under various circumstances.

Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network

Imaging through the wavy air-water surface suffers from severe geometric distortions, which are caused by the light refraction effect that affects the normal operations of underwater exploration equipment such as the autonomous underwater vehicle (AUV). In this paper, we propose a deep learning-based framework, namely the self-attention generative adversarial network (SAGAN), to remove the geometric distortions and restore the distorted image captured through the water-air surface. First, a K-means-based image pre-selection method is employed to acquire a less distorted image that preserves much useful information from an image sequence. Second, an improved generative adversarial network (GAN) is trained to translate the distorted image into the non-distorted image. During this process, the attention mechanism and the weighted training objective are adopted in our GAN framework to get the high-quality restored results of distorted underwater images. The network is able to restore the colors and fine details in the distorted images by combining the three objective losses, i.e., the content loss, the adversarial loss, and the perceptual loss. Experimental results show that our proposed method outperforms other state-of-the-art methods on the validation set and our sea trial set.

A Fast Image Deformity Correction Algorithm for Underwater Turbulent Image Distortion

An algorithm correcting distortion based on estimating the pixel shift is proposed for the degradation caused by underwater turbulence. The distorted image is restored and reconstructed by reference frame selection and two-dimensional pixel registration. A support vector machine-based kernel correlation filtering algorithm is proposed and applied to improve the speed and efficiency of the correction algorithm. In order to validate the algorithm, laboratory experiments on a controlled simulation system of turbulent water and field experiments in rivers and oceans are carried out, and the experimental results are compared with traditional, theoretical model-based and particle image velocimetry-based restoration and reconstruction algorithms. Using subjective visual evaluation, image distortion has been effectively suppressed; based on an objective performance statistical analysis, the measured values are better than the traditional and formerly studied restoration and reconstruction algorithms. The method proposed in this paper is also much faster than the other algorithms. It can be concluded that the proposed algorithm can effectively improve the de-distortion effect of the underwater turbulence degraded image, and provide potential techniques for the accurate operation of underwater target detection in real time.

Reconstruction of distorted underwater images using robust registration

Imaging through a fluctuating air-water surface is a challenging task since light rays bent by unknown amounts lead to complex geometric distortions. This paper presents a new algorithm to undistort dynamic refractive effects. An iterative robust registration algorithm is employed to overcome the structural turbulence of the waves of the frames by registering each frame to a reference frame. To get the high-quality reference frame, the image is reconstructed by the patches selected from the sequence frames. A blind deconvolution algorithm is also performed to improve the reference frame. Experiments show our method exhibits significant improvement over other methods.

Synthetic imaging through wavy water surface with centroid evolution

Imaging through a wavy water surface is a challenging task, as the wavy water surface introduces anisoplanatism effects difficult to model and track. A typical recovery method is usually involving multiple-stage processing on a pre-acquired image sequence. A new progressive restoration scheme is demonstrated, it can run simultaneously with image acquisition and mitigate both distortion and blur progressively. This method extends the anisotropic evolution in lucky region fusion with a novel progressive optical flow based de-warping scheme, centroid evolution. A comparison has been made with other state-of-art techniques, the proposed method can create comparable results, even with much less frames acquired. Experiments with real through-water scenes have also proved the effectiveness of the method.