Active polarization descattering

Polarization-Enhanced Underwater Laser Range-Gated Imaging for Subaquatic Applications

Laser range-gated underwater imaging technology, by removing most of the backscattering noise, can effectively increase image contrast and extend the detection range. The optical signal captured by a range-gated imaging system primarily comprises reflected light from the object and backscattered light from the surrounding water. Consequently, surfaces with low reflectivity or highly turbid water environments substantially constrain the applicability of the range-gated imaging system. To enhance the detection capability of underwater laser range-gated imaging, this paper proposes the incorporation of underwater polarized light imaging technology as an enhancement method. Based on polarization differences, backscattered light and reflected light from an object can be distinguished. Experimental results indicate that, compared to images obtained using a conventional range-gated laser imaging system, those captured with a polarization-enhanced system exhibit an increase of up to 47% for the corresponding Enhancement Measure Evaluation (EME) index. The proposed approach, which integrates polarization imaging with range-gated laser imaging, has the potential to broaden the applicability of underwater laser imaging scenarios, such as deep-sea exploration and military applications.

Computational Polarization Imaging In Vivo through Surgical Smoke Using Refined Polarization Difference

In surgery, the surgical smoke generated during tissue dissection and hemostasis can degrade the image quality, affecting tissue visibility and interfering with the further image processing. Developing reliable and interpretable computational imaging methods for restoring smoke-affected surgical images is crucial, as typical image restoration methods relying on color-texture information are insufficient. Here a computational polarization imaging method through surgical smoke is demonstrated, including a refined polarization difference estimation based on the discrete electric field direction, and a corresponding prior-based estimation method, for better parameter estimation and image restoration performance. Results and analyses for ex vivo, the first in vivo animal experiments, and human oral cavity tests show that the proposed method achieves visibility restoration and color recovery of higher quality, and exhibits good generalization across diverse imaging scenarios with interpretability. The method is expected to enhance the precision, safety, and efficiency of advanced image-guided and robotic surgery.

Irradiance-tailoring integral-illumination polarization homogenizer based on anamorphic aspheric microlens arrays

In the realm of active polarization detection systems, the imperative for polarization illumination systems with high-uniformity and predefined-shape irradiance distribution is evident. This paper introduces a novel anamorphic aspheric (AAS) microlens array (MLA) integral polarization homogenizer, incorporating projection MLA (PMLA), condenser MLA (CMLA), polarization film (PF), and a sub-image array (SIA) mask based on Kohler illumination principles. Firstly, the optimal design of an AAS-based projection sub-lens is proposed to facilitate the creation of a short-working-distance, predefined-geometric and sharp polarization irradiance tailoring. The SIA mask is constituted by plenty of predistortion SIs, which are generated through a combination of chief ray tracing and the radial basis function (RBF) image warping method. In addition, accompanied with tolerance sensitivity analysis, detailed analysis of stray light generation factors and proposed elimination or suppression methods further ensure the engineering reliability and stability of the proposed system. A compact integral-illumination polarization homogenizer design example is realized with an overall irradiance uniformity exceeding 90% and a volume of 25 mm × 25 mm × 18.25 mm. Different predefined-geometrical-profile and high-uniformity polarization irradiance distribution can be achieved by substituting different SIA masks and PFs, without replacing MLA optical elements, which greatly saves cost. Substantial simulations and experiments corroborate the efficacy of our polarization homogenizer.

Target recognition and segmentation in turbid water using data from non-turbid conditions: a unified approach and experimental validation

Semantic segmentation of targets in underwater images within turbid water environments presents significant challenges, hindered by factors such as environmental variability, difficulties in acquiring datasets, imprecise data annotation, and the poor robustness of conventional methods. This paper addresses this issue by proposing a novel joint method using deep learning to effectively perform semantic segmentation tasks in turbid environments, with the practical case of efficiently collecting polymetallic nodules in deep-sea while minimizing damage to the seabed environment. Our approach includes a novel data expansion technique and a modified U-net based model. Drawing on the underwater image formation model, we introduce noise to clear water images to simulate images captured under varying degrees of turbidity, thus providing an alternative to the required data. Furthermore, traditional U-net-based modified models have shown limitations in enhancing performance in such tasks. Based on the primary factors underlying image degradation, we propose a new model which incorporates an improved dual-channel encoder. Our method significantly advances the fine segmentation of underwater images in turbid media, and experimental validation demonstrates its effectiveness and superiority under different turbidity conditions. The study provides new technical means for deep-sea resource development, holding broad application prospects and scientific value.

Image recovery method for underwater targets with complex polarization characteristics

Polarization imaging techniques have been effective in improving the clarity of turbid underwater images affected by water scattering. These techniques offer valuable additional information compared to traditional methods. However, previous descattering methods have mostly been designed for targets with uniform distribution of polarimetric characteristics. Therefore, targets with complex polarization characteristics have non-uniform distribution of polarization characteristics when dealing with such problems, additional parameter estimation errors can limit the results of image recovery. This paper proposes what we believe is a novel approach to address this issue. The method involves obtaining a new set of images using the polarization images vector space transformation method. The angle of polarization (AOP) of the target reflected light is estimated using the additivity law of the Stokes vector. This information is then combined with the physical model of underwater imaging to recover the underwater images affected by scattering. Experimental results conducted under varying levels of water turbidity validate the effectiveness of the proposed method. Moreover, the proposed method significantly broadens the range of application scenarios.

Unsupervised underwater imaging based on polarization and binocular depth estimation

Scattering caused by suspended particles in the water severely reduces the radiance of the scene. This paper proposes an unsupervised underwater restoration method based on binocular estimation and polarization. Based on the correlation between the underwater transmission process and depth, this method combines the depth information and polarization information in the scene, uses the neural network to perform global optimization and the depth information is recalculated and updated in the network during the optimization process, and reduces the error generated by using the polarization image to calculate parameters, so that detailed parts of the image are restored. Furthermore, the method reduces the requirement for rigorous pairing of data compared to previous approaches for underwater imaging using neural networks. Experimental results show that this method can effectively reduce the noise in the original image and effectively preserve the detailed information in the scene.

Super-resolution reconstruction of underwater polarized images with a fused attention mechanism

The polarization imaging technique leverages the disparity between target and background polarization information to mitigate the impact of backward scattered light, thereby enhancing image quality. However, the imaging model of this method exhibits limitations in extracting inter-image features, resulting in less-than-optimal outcomes in turbid underwater environments. In recent years, machine learning methodologies, particularly neural networks, have gained traction. These networks, renowned for their superior fitting capabilities, can effectively extract information from multiple images. The incorporation of an attention mechanism significantly augments the capacity of neural networks to extract inter-image correlation attributes, thereby mitigating the constraints of polarization imaging methods to a certain degree. To enhance the efficacy of polarization imaging in complex underwater environments, this paper introduces a super-resolution network with an integrated attention mechanism, termed as SRGAN-DP. This network is a fusion of an enhanced SRGAN network and the high-performance deep pyramidal split attention (DPSA) module, also proposed in this paper. SRGAN-DP is employed to perform high-resolution reconstruction of the underwater polarimetric image dataset, constructed specifically for this study. A comparative analysis with existing algorithms demonstrates that our proposed algorithm not only produces superior images but also exhibits robust performance in real-world environments.

Underwater dynamic polarization imaging without dependence on the background region

Active-polarization imaging holds significant promise for achieving clear underwater vision. However, only static targets were considered in previous studies, and a background region was required for image restoration. To address these issues, this study proposes an underwater dynamic polarization imaging method based on image pyramid decomposition and reconstruction. During the decomposition process, the polarized image is downsampled to generate an image pyramid. Subsequently, the spatial distribution of the polarization characteristics of the backscattered light is reconstructed by upsampling, which recovered the clear scene. The proposed method avoids dependence on the background region and is suitable for moving targets with varying polarization properties. The experimental results demonstrate effective elimination of backscattered light while sufficiently preserving the target details. In particular, for dynamic targets, processing times that fulfill practical requirements and yield superior recovery effects are simultaneously obtained.

Dynamic polarization fusion network (DPFN) for imaging in different scattering systems

Deep learning has broad applications in imaging through scattering media. Polarization, as a distinctive characteristic of light, exhibits superior stability compared to light intensity within scattering media. Consequently, the de-scattering network trained using polarization is expected to achieve enhanced performance and generalization. For getting optimal outcomes in diverse scattering conditions, it makes sense to train expert networks tailored for each corresponding condition. Nonetheless, it is often unfeasible to acquire the corresponding data for every possible condition. And, due to the uniqueness of polarization, different polarization information representation methods have different sensitivity to different environments. As another of the most direct approaches, a generalist network can be trained with a range of polarization data from various scattering situations, however, it requires a larger network to capture the diversity of the data and a larger training set to prevent overfitting. Here, in order to achieve flexible adaptation to diverse environmental conditions and facilitate the selection of optimal polarization characteristics, we introduce a dynamic learning framework. This framework dynamically adjusts the weights assigned to different polarization components, thus effectively accommodating a wide range of scattering conditions. The proposed architecture incorporates a Gating Network (GTN) that efficiently integrates multiple polarization features and dynamically determines the suitable polarization information for various scenarios. Experimental result demonstrates that the network exhibits robust generalization capabilities across continuous scattering conditions.

Robust underwater image enhancement with cascaded multi-level sub-networks and triple attention mechanism

With the growing exploration of marine resources, underwater image enhancement has gained significant attention. Recent advances in convolutional neural networks (CNN) have greatly impacted underwater image enhancement techniques. However, conventional CNN-based methods typically employ a single network structure, which may compromise robustness in challenging conditions. Additionally, commonly used UNet networks generally force fusion from low to high resolution for each layer, leading to inaccurate contextual information encoding. To address these issues, we propose a novel network called Cascaded Network with Multi-level Sub-networks (CNMS), which encompasses the following key components: (a) a cascade mechanism based on local modules and global networks for extracting feature representations with richer semantics and enhanced spatial precision, (b) information exchange between different resolution streams, and (c) a triple attention module for extracting attention-based features. CNMS selectively cascades multiple sub-networks through triple attention modules to extract distinct features from underwater images, bolstering the network's robustness and improving generalization capabilities. Within the sub-network, we introduce a Multi-level Sub-network (MSN) that spans multiple resolution streams, combining contextual information from various scales while preserving the original underwater images' high-resolution spatial details. Comprehensive experiments on multiple underwater datasets demonstrate that CNMS outperforms state-of-the-art methods in image enhancement tasks.

Underwater motion scene image restoration based on an improved U-Net network

Active underwater polarization imaging is a common underwater imaging method, which uses the polarization difference between the reflected light and the scattered light in the underwater scene to suppress the scattered light, so as to improve the imaging quality of the underwater scene. However, the implementation often requires the acquisition of multiple polarization images, which is not suitable for the restoration of images of underwater motion scenes. To address the problem, a U-AD-Net deep learning network model based on a single polarized image is proposed, taking the polarization information of the single polarized image as the feature input, based on the classic U-Net network model, and introducing Dense-Net and spatial attention module. The learning ability and generalization ability of the proposed model for deep features are enhanced, and the polarization information that is most helpful to the image restoration is extracted, so as to restore the scene image more comprehensively. IE, AG, UCIQE, and SSIM are selected as evaluation metrics to assess the quality of the restored images. Experimental results show that the images restored through this proposed method contain richer detail information, having an obvious advantage to the existing network models. Since only a single polarized image is needed for restoration, this method has dynamic adaptability to underwater moving scene restoration.

Underwater ghost imaging with detection distance up to 9.3 attenuation lengths

Underwater ghost imaging LiDAR is an effective method of underwater detection. In this research, theoretical and experimental investigations were conducted on underwater ghost imaging, combining the underwater optical field transmission model with the inherent optical parameters of a water body. In addition, the Wells model and the approximate Sahu-Shanmugam scattering phase function were used to create a model for underwater optical transmission. The second-order Glauber function of the optical field was then employed to analyze the scattering field degradation during the transmission process. The simulation and experimental results verified that the proposed underwater model could better reveal the degrading effect of a water body on ghost imaging. A further series of experiments comparing underwater ghost imaging at different detection distances was also conducted. In the experimental system, gated photomultiplier tube (PMT) was used to filter out the peak of backscattering, allowing a larger gain to be set for longer-range detection of the target. The laser with a central wavelength of 532 nm was operated at a frequency of 2 KHz, with a single pulse energy of 2 mJ, a pulse width of 10 ns. High-reflective targets were imaged up to 65.2 m (9.3 attenuation lengths (ALs), attenuation coefficient c = 0.1426 m-1, and scattering coefficient b = 0.052 m-1) and diffuse-reflection targets up to 41.2 m (6.4 ALs, c = 0.1569 m-1, and b = 0.081 m-1). For the Jerlov-I (c = 0.048 m-1 and b = 0.002 m-1) water body, the experimentally obtained maximum detection distance of 9.3 ALs can be equivalent to 193.7 m under the same optical system conditions.

Experimental Characterization of Polarized Light Backscattering in Fog Environments

This paper focuses on the experimental characterization of the polarization behavior of light backscattered through fog. A polarimetric orthogonal state contrast imager and an active, purely polarized white illuminator system are used to evaluate both linear and circular polarization signals. The experiments are carried out in a macro-scale fog chamber under controlled artificial fog conditions. We explore the effect of backscattering in each imaging channel, and the persistence of both polarization signals as a function of meteorological visibility. We confirm the presence of the polarization memory effect with circularly polarized light, and, as a consequence, the maintenance of helicity in backscattering. Moreover, the circular cross-polarized channel is found to be the imaging channel less affected by fog backscattering. These results are useful and should be taken into account when considering active polarimetric imaging techniques for outdoor applications under foggy conditions.

Automatic Suppression Method for Water Surface Glints Using a Division of Focal Plane Visible Polarimeter

To address the problem of water surface detection imaging equipment being susceptible to water surface glints, this study demonstrates a method called De-Glints for suppressing glints and obtaining clear underwater images using a division of focal plane (DoFP) polarimeter. Based on the principle of polarization imaging, the best polarization angle and the image corresponding to the minimal average gray level of each pixel are calculated. To evaluate the improvement in image quality, the index E was designed. The results of indoor and outdoor experiments show that the error of the angle calculation of this method is within 10%, and the minimum error is only 3%. The E index is positively improved and can be relatively improved by 8.00 under the interference of strong outdoor glints, and the method proposed in this paper shows a good adaptive ability to the dynamic scene.

Single-shot decoherence polarization gated imaging through turbid media

We propose a method for imaging through a turbid medium by using a single-shot decoherence polarization gate (DPG). The DPG is made up of a polarizer, an analyzer, and a weakly scattering medium. Contrary to intuition, we discover that the preferential utilization of sparsely scattered photons by introducing weakly scattering mediums can lead to better image quality. The experimental results show that the visibilities of the images acquired from the DPG imaging method are obviously improved. The contrast of the bar can be increased by 50% by the DPG imaging technique. Furthermore, we study the effect of the volume concentration of the weakly scattering medium on the speckle suppression and the enhancement of the visibilities of the images. The variances of the contrasts of the image show that there exists an optimum optical depth (∼0.8) of the weakly scattering medium for DPG imaging through a specific turbid medium.

Underwater active polarization descattering based on a single polarized image

Active polarization imaging techniques have tremendous potential for a variety of underwater applications. However, multiple polarization images as input are necessary for almost all methods, thereby limiting the range of applicable scenarios. In this paper, via taking full advantage of the polarization feature of target reflective light, the cross-polarized backscatter image is reconstructed via introducing an exponential function for the first time, only based on mapping relations of co-polarized image. Compared with rotating the polarizer, the result performs a more uniform and continuous distribution of grayscale. Furthermore, the relationship of degree of polarization (DOP) between the whole scene and backscattered light is established. This leads to an accurate estimation of backscattered noise and high-contrast restored images. Besides, single-input greatly simplifies the experimental process and upgrades efficiency. Experimental results demonstrate the advancement of the proposed method for objects with high polarization under various turbidities.

Underwater Target Detection Utilizing Polarization Image Fusion Algorithm Based on Unsupervised Learning and Attention Mechanism

Since light propagation in water bodies is subject to absorption and scattering effects, underwater images using only conventional intensity cameras will suffer from low brightness, blurred images, and loss of details. In this paper, a deep fusion network is applied to underwater polarization images; that is, the underwater polarization images are fused with intensity images using the deep learning method. To construct a training dataset, we establish an experimental setup to obtain underwater polarization images and perform appropriate transformations to expand the dataset. Next, an end-to-end learning framework based on unsupervised learning and guided by an attention mechanism is constructed for fusing polarization and light intensity images. The loss function and weight parameters are elaborated. The produced dataset is used to train the network under different loss weight parameters, and the fused images are evaluated based on different image evaluation metrics. The results show that the fused underwater images are more detailed. Compared with light intensity images, the information entropy and standard deviation of the proposed method increase by 24.48% and 139%. The image processing results are better than other fusion-based methods. In addition, the improved U-net network structure is used to extract features for image segmentation. The results show that the target segmentation based on the proposed method is feasible under turbid water. The proposed method does not require manual adjustment of weight parameters, has faster operation speed, and has strong robustness and self-adaptability, which is important for research in vision fields, such as ocean detection and underwater target recognition.

Optimized method for polarization-based image dehazing

Image dehazing is desired under the foggy, rainy weather, or the underwater condition. Since the polarization-based image dehazing utilizes additional polarization information of light to de-scatter, image detail can be recovered well, but how to segment the polarization information of the background radiance and the object radiance becomes the key problem. For solving this problem, a method which combing polarization and contrast enhancement is demonstrated. This method contains two main steps, (a) by seeking the region of large mean intensity, low contrast and large mean degree of polarization, the no-object region can be found, and (b) through defining a weight function and judging whether the dehazed image can achieve high contrast and low information loss, the degree of polarization for object radiance can be estimated. Based on the estimated parameters, the scatter of light by the mediums can be diminished considerably. The theoretical derivation shows that this method can achieve advantages complementation, such as being able to obtain more details like the polarization-based method and high image contrast like the contrast enhancement based method. Besides, it is physically sound and can achieve good dehazing performance under different conditions, which has been verified by different hazing polarization images.

Fast image visibility enhancement based on active polarization and color constancy for operation in turbid water

Vehicles operating in a water medium sometimes encounter harsh conditions with high turbidity and low scene illumination, making it challenging to obtain reliable target information through optical devices. Although many post-processing solutions were proposed, they are not applicable to continuous vehicle operations. Inspired by the advanced polarimetric hardware technology, a joint fast algorithm was developed in this study to address the above problems. Backscatter attenuation and direct signal attenuation were solved separately by utilizing the revised underwater polarimetric image formation model. A fast local adaptive Wiener filtering method was used to improve the backscatter estimation by reducing the additive noise. Further, the image was recovered using the fast local space average color method. By using a low-pass filter guided by the color constancy theory, the problems of nonuniform illumination caused by artificial light and direct signal attenuation were both addressed. The results of testing on images from laboratory experiments showed improved visibility and realistic chromatic rendition.

Underwater image enhancement based on zero-shot learning and level adjustment

Light is scattered and partially absorbed while traveling through water, hence, underwater captured images often exhibit issues such as low contrast, detail blurring, color attenuation, and low illumination. To improve the visual performance of underwater imaging, herein, we propose a two-step method of zero-shot dehazing and level adjustment. In the newly developed approach, the original image is fed into a "zero-shot" dehazing network and further enhanced by an improved level adjustment methodology combined with auto-contrast. By conducting experiments, we then compare the performance of the proposed method with six classical state-of-the-art methods. The qualitative results confirm that the proposed method is capable of effectively removing haze, correcting color deviations, and maintaining the naturalness of images. We further perform a quantitative evaluation, revealing that the proposed method outperforms the comparison methods in terms of peak signal-to-noise ratio and structural similarity. The enhancement results are also measured by employing the underwater color image quality evaluation index (UCIQE), indicating that the proposed approach exhibits the highest mean values of 0.58 and 0.53 on the two data sets. The experimental results collectively validate the efficiency of the proposed methodology in enhancing underwater blurred images.

Influence mechanism of the particle size on underwater active polarization imaging of reflective targets

Underwater active polarization imaging is a promising imaging method, however, it is ineffective in some scenarios. In this work, the influence of the particle size from isotropic (Rayleigh regime) to forward-scattering on polarization imaging is investigated by both Monte Carlo simulation and quantitative experiments. The results show the non-monotonic law of imaging contrast with the particle size of scatterers. Furthermore, through polarization-tracking program, the polarization evolution of backscattered light and target diffuse light are detailed quantitatively with Poincaré sphere. The findings indicate that the noise light's polarization and intensity scattering field change significantly with the particle size. Based on this, the influence mechanism of the particle size on underwater active polarization imaging of reflective targets is revealed for the first time. Moreover, the adapted principle of scatterer particle scale is also provided for different polarization imaging methods.

INAM-Based Image-Adaptive 3D LUTs for Underwater Image Enhancement

To the best of our knowledge, applying adaptive three-dimensional lookup tables (3D LUTs) to underwater image enhancement is an unprecedented attempt. It can achieve excellent enhancement results compared to some other methods. However, in the image weight prediction process, the model uses the normalization method of Instance Normalization, which will significantly reduce the standard deviation of the features, thus degrading the performance of the network. To address this issue, we propose an Instance Normalization Adaptive Modulator (INAM) that amplifies the pixel bias by adaptively predicting modulation factors and introduce the INAM into the learning image-adaptive 3D LUTs for underwater image enhancement. The bias amplification strategy in INAM makes the edge information in the features more distinguishable. Therefore, the adaptive 3D LUTs with INAM can substantially improve the performance on underwater image enhancement. Extensive experiments are undertaken to demonstrate the effectiveness of the proposed method.

Underwater 3D reconstruction using a photometric stereo with illuminance estimation

Underwater image sensing is directly affected by the change in illuminance towards a camera caused by the refraction of light in different media. In this study, the convergence of a near-field point light source in water is analyzed using a light propagation model. The photometric stereo (PS) formula is determined based on an accurate estimation of the illuminance entering the camera. An underwater PS system is designed to verify the proposed method's feasibility. The experimental results demonstrate improved accuracy in normal calculation. This helps achieve accurate underwater 3D reconstruction of objects that is suitable for underwater surface microdefect detection.

Self-attention module in a multi-scale improved U-net (SAM-MIU-net) motivating high-performance polarization scattering imaging

Polarization imaging has outstanding advantages in the field of scattering imaging, which still encounters great challenges in heavy scattering media systems even though there are helps from deep learning technology. In this paper, we propose a self-attention module (SAM) in multi-scale improved U-net (SAM-MIU-net) for the polarization scattering imaging, which can extract a new combination of multidimensional information from targets effectively. The proposed SAM-MIU-net can focus on the stable feature carried by polarization characteristics of the target, so as to enhance the expression of the available features, and make it easier to extract polarization features which help to recover the detail of targets for the polarization scattering imaging. Meanwhile, the SAM's effectiveness has been verified in a series of experiments. Based on proposed SAM-MIU-net, we have investigated the generalization abilities for the targets' structures and materials, and the imaging distances between the targets and the ground glass. Experimental results demonstrate that our proposed SAM-MIU-net can achieve high-precision reconstruction of target information under incoherent light conditions for the polarization scattering imaging.

Underwater polarization imaging for visibility enhancement of moving targets in turbid environments

Polarization imaging techniques have more prominent advantages for imaging in strongly scattered media. Previous de-scattering methods of polarization imaging usually require the priori information of the background region, and rarely consider the effect of non-uniformity of the optical field on image recovery, which not only reduces the processing speed of imaging but also introduces errors in image recovery, especially for moving targets in complex scattering environments. In this paper, we propose a turbid underwater moving image recovery method based on the global estimation of the intensity and the degree of polarization (DOP) of the backscattered light, combined with polarization-relation histogram processing techniques. The full spatial distribution of the intensity and the DOP of the backscattered light are obtained by using frequency domain analysis and filtering. Besides, a threshold factor is set in the frequency domain low-pass filter, which is used to adjust the execution region of the filter, which effectively reduces the error in image recovery caused by estimating the DOP of the backscattered light as a constant in traditional methods with non-uniform illumination. Meanwhile, our method requires no human-computer interaction, which effectively solves the drawbacks that the moving target is difficult to be recovered by traditional methods. Experimental studies were conducted on static and moving targets under turbid water, and satisfactory image recovery quality is achieved.

Active non-uniform illumination-based underwater polarization imaging method for objects with complex polarization properties

Active polarization imaging is one of the most effective underwater optical imaging methods that can eliminate the degradation of image contrast and clarity caused by macro-molecule scattering. However, the non-uniformity of active illumination and the diversity of object polarization properties may decrease the quality of underwater imaging. This paper proposes a non-uniform illumination-based active polarization imaging method for underwater objects with complex optical properties. Firstly, illumination homogenization in the frequency domain is proposed to extract and homogenize the natural incident light from the total receiving light. Then, the weight values of the polarized and non-polarized images are computed according to each pixel's degree of linear polarization (DoLP) in the original underwater image. By this means, the two images can be fused to overcome the problem of reflected light loss generated by the complex polarization properties of underwater objects. Finally, the fusion image is normalized as the final result of the proposed underwater polarization imaging method. Both qualitative and quantitative experimental results show that the presented method can effectively eliminate the uneven brightness of the whole image and obtain the underwater fusion image with significantly improved contrast and clarity. In addition, the ablation experiment of different operation combinations shows that each component of the proposed method has noticeable enhancement effects on underwater polarization imaging. Our codes are available in Code 1.

Visibility enhancement of underwater images based on polarization common-mode rejection of a highly polarized target signal

Underwater active polarization imaging is promising due to its effect of significantly descattering. Polarization-difference is commonly used to filter out backscattered noise. However, the polarization common-mode rejection of target signal has rarely been utilized. In this paper, via taking full advantage of this feature of Stokes vectors S₂ which ably avoids interference from target light, the spatial variation of the degree of polarization of backscattered light is accurately estimated, and the whole scene intensity distribution of background is reconstructed by Gaussian surface fitting based on least square. Meanwhile, the underwater image quality measure is applied as optimization feedback, through iterative computations, not only sufficiently suppresses backscattered noise but also better highlights the details of the target. Experimental results demonstrate the effectiveness of the proposed method for highly polarized target in strongly scattering water.

Estimation and removal of backscattered light with nonuniform polarization information in underwater environments

The nonuniform of polarization information of backscattered light has always been a neglected characteristic in polarization underwater imaging, but its accurate estimation plays an important role in the quality of imaging results. Traditional polarization imaging methods assume that the degree of polarization and angle of polarization of backscattered light are constant. In fact, the polarization information of backscattering light is gradual, this assumption makes traditional methods work only in a small area of the camera's field of view, in which the change of the polarization information of backscattered light can be ignored. In this paper, by analyzing the distribution of backscattered light, it is concluded that its polarization information has the characteristics of low-rank. Then, the degree of polarization and angle of polarization of backscattered light were estimated by low-rank and sparse matrix decomposition, and the clear scene was reconstructed. Experimental results show that the proposed method breaks through the limitation of the assumption of backscattered light in traditional polarization imaging method, and expands the detection field under the same conditions, which makes it possible to develop polarization underwater imaging method to the direction of large field of view detection.

Descattering for transmissive inspection in production line using slanted linear image sensors

We propose a descattering method that can be easily applied to food production lines. The system consists of several sets of linear image sensors and linear light sources slanted at different angles. The images captured by these sensors are partially clear along the direction perpendicular to the sensors. We computationally integrate these images on the frequency domain into a single clear image. The effectiveness of the proposed method is assessed by simulation and real-world experiments. The results show that our method recovers clear images. We demonstrate the applicability of the proposed method to a real production line by a prototype system.

Fusion-based underwater image enhancement with category-specific color correction and dehazing

Underwater imaging is usually affected by water scattering and absorption, resulting in image blur and color distortion. In order to achieve color correction and dehazing for different underwater scenes, in this paper we report a fusion-based underwater image enhancement technique. First, statistics of the hue channel of underwater images are used to divide the underwater images into two categories: color-distorted images and non-distorted images. Then, category-specific combinations of color compensation and color constancy algorithms are used to remove the color shift. Second, a ground-dehazing algorithm using haze-line prior is employed to remove the haze in the underwater image. Finally, a channel-wise fusion method based on the CIE L* a* b* color space is used to fuse the color-corrected image and dehazed image. For experimental validation, we built a setup to acquire underwater images. The experimental results validate that the category-specific color correction strategy is robust to different categories of underwater images and the fusion strategy simultaneously removes haze and corrects color casts. The quantitative metrics on the UIEBD and EUVP datasets validate its state-of-the-art performance.

Bispectral phasor imaging using continuous-wave time-of-flight camera for scattering-scene depth recovery

In scattering scenes, depth measurements are greatly distorted due to light scattering for Time-of-flight imaging. We propose a bispectral Time-of-flight system and phasor-based depth-recovery method to improve the quality of depth maps in scattering scenes. We reveal that the amplitude of scattered light is wavelength dependent while the phase measured is wavelength independent. The method uses bispectral measurements to nullify the effects of scattering components by calculating the amplitude ratio of scattering phasors. Experimental results demonstrate that the proposed method has a significant improvement in depth recovery with robustness and low computational cost.

Physics-informed neural network for polarimetric underwater imaging

Utilizing the polarization analysis in underwater imaging can effectively suppress the scattered light and help to restore target signals in turbid water. Neural network-based solutions can also boost the performance of polarimetric underwater imaging, while most of the existing networks are pure data driven which suffer from ignoring the physical mode. In this paper, we proposed an effective solution that informed the polarimetric physical model and constrains into the well-designed deep neural network. Especially compared with the conventional underwater imaging model, we mathematically transformed the two polarization-dependent parameters to a single parameter, making it easier for the network to converge to a better level. In addition, a polarization perceptual loss is designed and applied to the network to make full use of polarization information on the feature level rather than on the pixel level. Accordingly, the network was able to learn the polarization modulated parameter and to obtain clear de-scattered images. The experimental results verified that the combination of polarization model and neural network was beneficial to improve the image quality and outperformed other existing methods, even in a high turbidity condition.

Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement

Underwater images typically suffer from color deviations and low visibility due to the wavelength-dependent light absorption and scattering. To deal with these degradation issues, we propose an efficient and robust underwater image enhancement method, called MLLE. Specifically, we first locally adjust the color and details of an input image according to a minimum color loss principle and a maximum attenuation map-guided fusion strategy. Afterward, we employ the integral and squared integral maps to compute the mean and variance of local image blocks, which are used to adaptively adjust the contrast of the input image. Meanwhile, a color balance strategy is introduced to balance the color differences between channel a and channel b in the CIELAB color space. Our enhanced results are characterized by vivid color, improved contrast, and enhanced details. Extensive experiments on three underwater image enhancement datasets demonstrate that our method outperforms the state-of-the-art methods. Our method is also appealing in its fast processing speed within 1s for processing an image of size 1024×1024×3 on a single CPU. Experiments further suggest that our method can effectively improve the performance of underwater image segmentation, keypoint detection, and saliency detection. The project page is available at https://li-chongyi.github.io/proj_MMLE.html.

Image-restoration algorithm based on an underwater polarization imaging visualization model

The polarization bidirectional reflection distribution function theory of a target is combined with microfacet theory, and the Monte Carlo method is used to establish an underwater laser active-polarization imaging model based on Mie scattering theory. The model presented herein can simulate imaging of an underwater target with a high degree of polarization, and the effects of optical thickness and target surface roughness on active underwater laser imaging results are demonstrated by the simulation image. Combined with histogram equalization and the traditional polarization differential imaging algorithm, an algorithm is presented herein that globally estimates the mutual information value between the target polarization degree and the correction factor of backscattered light polarization degree. The images received from the simulation test can be restored, and results show that the algorithm can restore the target image with a high degree of polarization to some extent. Finally, the correctness of the active underwater laser polarization imaging model and the feasibility of global estimation based on the polarization differential restoration algorithm are verified experimentally.

Underwater Multispectral Computational Imaging Based on a Broadband Water-Resistant Sb2Se3 Heterojunction Photodetector

Exploration, utilization, and protection of marine resources are of great significance to the survival and development of mankind. However, currently classical optical cameras suffer information loss, low contrast, and color distortion due to the absorption and scattering nature for the underwater environment. Here, we demonstrate an underwater multispectral computational imaging system combined with single-photodetector imaging algorithm technology and a CdS/Sb₂Se₃ heterojunction photodetector. The computational imaging technology coupled with an advanced Fourier algorithm can capture a scene by a single photodetector without spatial resolution that avoids the need to rely on high-density detectors array and bulky optical components in traditional imaging systems. This convenient computational imaging method provides more flexible possibilities for underwater imaging and promises to give more imaging capabilities (such as multispectral imaging, antiscattering imaging capability) to meet ever-changing demand of underwater imaging. In addition, the water-resistant CdS/Sb₂Se₃ heterojunction photodetector fabricated by the close spaced sublimation (Sb₂Se₃) and chemical bath deposition (CdS) shows excellent self-powered photodetection performance at zero bias with high LDR of 128 dB, broadband response spectrum range of 300-1050 nm, high responsivity up to 0.47 A/W, and high specific detectivity over 5 × 10¹² jones. Compared with the traditional optical imaging system, our designed computational imaging system that combines the advanced Fourier algorithm and a high-performance CdS/Sb₂Se₃ heterojunction photodetector exhibits outstanding antiscattering imaging capability (shielded by frosted glass), weak light imaging capability (∼0.2 μW/cm², corresponding to moonlight intensity), and multispectral imaging capability. Therefore, we believe that this work will boost the progress of marine science.

Time-of-Flight Imaging in Fog Using Polarization Phasor Imaging

Due to the light scattered by atmospheric aerosols, the amplitude image contrast is degraded and the depth measurement is greatly distorted for time-of-flight (ToF) imaging in fog. The problem limits ToF imaging to be applied in outdoor settings, such as autonomous driving. To improve the quality of the images captured by ToF cameras, we propose a polarization phasor imaging method for image recovery in foggy scenes. In this paper, optical polarimetric defogging is introduced into ToF phasor imaging, and the degree of polarization phasor is proposed to estimate the scattering component. A polarization phasor imaging model is established, aiming at separating the target component from the signal received by ToF cameras to recover the amplitude and depth information. The effectiveness of this method is confirmed by several experiments with artificial fog, and the experimental results demonstrate that the proposed method significantly improves the image quality, with robustness in different thicknesses of fog.

Insect-inspired nanofibrous polyaniline multi-scale films for hybrid polarimetric imaging with scattered light

We demonstrate a bio-inspired coating for novel imaging and sensing designs: the coating sorts different colors and linear polarizations. This coating, composed of conducting, nanofibrous polyaniline in an inverse opal film (PANI-IOF), is inexpensive and can feasibly be deposited over large areas on a range of flexible and non-flat substrates. With PANI IOFs, light is scattered into azimuthally polarized Debye rings. Subsequently, the diffracted speckle patterns carry compressed representations of the polarized illumination, which we reconstruct using shallow neural networks.

Calibration of a polarization image sensor and investigation of influencing factors

Polarization measurements conducted with a polarization camera using the Sony IMX 250 MZR polarization image sensor are assessed with the super-pixel calibration technique and a simple test setup. We define an error that quantifies the quality of the polarization measurements. Multiple factors influencing the measurement quality of the polarization camera are investigated and discussed. We demonstrate that polarization measurements are generally consistent throughout the sensor if not corrupted by large chief ray angles or large angles of incidence. The central 600×400pixels were analyzed, and it is shown that sufficiently large f-numbers no longer influence measurement quality. We also argue that lens design and focal length have little influence on these central pixels. The findings of this study provide useful guidance for researchers using such a polarization image sensor.

A polarization-based image restoration method for both haze and underwater scattering environment

Existing polarization-based defogging algorithms rely on the polarization degree or polarization angle and are not effective enough in scenes with little polarized light. In this article, a method of image restoration for both haze and underwater scattering environment is proposed. It bases on the general assumption that gray variance and average gradient of a clear image are larger than those of an image in a scattering medium. Firstly, based on the assumption, polarimetric images with the maximum variance (Ibest) and minimum variance (Iworst) are calculated from the captured four polarization images. Secondly, the transmittance is estimated and used to remove the scattering light from background medium of Ibest and Iworst. Thirdly, two images are fused to form a clear image and the color is also restored. Experimental results show that the proposed method obtains clear restored images both in haze and underwater scattering media. Because it does not rely on the polarization degree or polarization angle, it is more universal and suitable for scenes with little polarized light.

Underwater image restoration by structured light and flood light imaging

An underwater optical imaging system is indispensable for many oceanic engineering tasks, yet still plagued by poor visibility conditions. The serious degradation of underwater image results from light scattering and absorption. Removal of the backscattered light is the focus issue of underwater imaging technology to improve the image visibility, particularly in turbid water. In this paper, we present an approach for underwater image recovery using structured light imaging and flood light imaging to compose a combined imaging model with which the backscatter component is completely offset. The convolutional image is obtained using the structured light scanning imaging mode where the backscatter intensity is proportional to that of the flood light image with an unknown scale parameter. An algorithm to refine the matching of the backscatter components of both the convolutional image and the flood light image is proposed. Thus, subtraction of both images gives rise the combined imaging model without the backscatter component. Consequently, image restoration is completed by employing the deconvolution process. Results of underwater experiments are given.

Enhancement of underwater vision by fully exploiting the polarization information from the Stokes vector

Underwater imaging method based on polarization information is extremely popular due to its ability to effectively remove the backscattered light. The Stokes vector contains the information of both the degree and angle of polarization of the light wave. However, this aspect has been rarely utilized in image reconstruction. In this study, an underwater polarimetric imaging model is established by fully exploiting this feature of Stokes vectors. The transmission of light wave is described in terms of the polarization information derived from the Stokes vector. Then, an optimization function is designed based on the independent characteristics of target light and backscattered light to estimate the target and backscattered field information. The real-world experiments and mean squared error analysis verify that the proposed method can remove the backscattered light and recover the target information accurately.

Time-Resolved Far Infrared Light Transport Decomposition for Thermal Photometric Stereo

We present a novel time-resolved light transport decomposition method using thermal imaging. Because the speed of heat propagation is much slower than the speed of light propagation, the transient transport of far infrared light can be observed at a video frame rate. A key observation is that the thermal image looks similar to the visible light image in an appropriately controlled environment. This implies that conventional computer vision techniques can be straightforwardly applied to the thermal image. We show that the diffuse component in the thermal image can be separated, and therefore, the surface normals of objects can be estimated by the Lambertian photometric stereo. The effectiveness of our method is evaluated by conducting real-world experiments, and its applicability to black body, transparent, and translucent objects is shown.

Fast processing of underwater polarization imaging based on optical correlation

Underwater polarization differential imaging requires the estimation of different parameters, and the parameters can be accurately obtained by using optical correlation. However, optical correlation as a criterion function to estimate parameters takes a lot of time. To expedite the parameters' estimation process, we propose two operations to process underwater polarization images. One operation is to update the analyzer angle range to reduce the number of processed images. The other is image downsampling, which reduces the amount of calculation for the corresponding images. In experiments, we confirmed the feasibility of our method. We have obtained an average of 42 times the calculation speed increase under the conditions of updating the analyzer angle range 3 times and reducing the image scale by 16 times. The results of our method are consistent with those of traditional methods. This established method is conducive to the practical application of underwater polarization differential imaging.

Time-of-flight imaging in fog using multiple time-gated exposures

We propose a time-of-flight measurement algorithm for depth and intensity that is robust to fog. The key idea of the algorithm is to compensate for the scattering effects of fog by using multiple time-gating and assigning one time-gated exposure for scattering property estimation. Once the property is estimated, the depth and intensity can be reconstructed from the rest of the exposures via a physics-based model. Several experiments with artificial fog show that our method can measure depth and intensity irrespective of the traits of the fog. We also confirm the effectiveness of our method in real fog through an outdoor experiment.

Evaluation of Several Feature Detectors/Extractors on Underwater Images towards vSLAM

Modern visual SLAM (vSLAM) algorithms take advantage of computer vision developments in image processing and in interest point detectors to create maps and trajectories from camera images. Different feature detectors and extractors have been evaluated for this purpose in air and ground environments, but not extensively for underwater scenarios. In this paper (I) we characterize underwater images where light and suspended particles alter considerably the images captured, (II) evaluate the performance of common interest points detectors and descriptors in a variety of underwater scenes and conditions towards vSLAM in terms of the number of features matched in subsequent video frames, the precision of the descriptors and the processing time. This research justifies the usage of feature detectors in vSLAM for underwater scenarios and present its challenges and limitations.

Compressive ghost imaging through scattering media with deep learning

Imaging through scattering media is challenging since the signal to noise ratio (SNR) of the reflection can be heavily reduced by scatterers. Single-pixel detectors (SPD) with high sensitivities offer compelling advantages for sensing such weak signals. In this paper, we focus on the use of ghost imaging to resolve 2D spatial information using just an SPD. We prototype a polarimetric ghost imaging system that suppresses backscattering from volumetric media and leverages deep learning for fast reconstructions. In this work, we implement ghost imaging by projecting Hadamard patterns that are optimized for imaging through scattering media. We demonstrate good quality reconstructions in highly scattering conditions using a 1.6% sampling rate.

Optical 4D signal detection in turbid water by multi-dimensional integral imaging using spatially distributed and temporally encoded multiple light sources

We propose an underwater optical signal detection system based on multi-dimensional integral imaging with spatially distributed multiple light sources and four-dimensional (4D) spatial-temporal correlation. We demonstrate our system for the detection of optical signals in turbid water. A 4D optical signal is generated from a three-dimensional (3D) spatial distribution of underwater light sources, which are temporally encoded using spread spectrum techniques. The optical signals are captured by an array of cameras, and 3D integral imaging reconstruction is performed, followed by multi-dimensional correlation to detect the optical signal. Inclusion of multiple light sources located at different depths allows for successful signal detection at turbidity levels not feasible using only a single light source. We consider the proposed system under varied turbidity levels using both Pseudorandom and Gold Codes for temporal signal coding. We also compare the effectiveness of the proposed underwater optical signal detection system to a similar system using only a single light source and compare between conventional and integral imaging-based signal detection. The underwater signal detection capabilities are measured through performance-based metrics such as receiver operating characteristic (ROC) curves, the area under the curve (AUC), and the number of detection errors. Furthermore, statistical analysis, including Kullback-Leibler divergence and Bhattacharya distance, shows improved performance of the proposed multi-source integral imaging underwater system. The proposed integral-imaging based approach is shown to significantly outperform conventional imaging-based methods.