IPLNet: a neural network for intensity-polarization imaging in low light

TSMPN-PSI: high-performance polarization scattering imaging based on three-stage multi-pipeline networks

Polarization imaging, which provides multidimensional information beyond traditional intensity imaging, has prominent advantages for complex imaging tasks, particularly in scattering environments. By introducing deep learning (DL) into computational imaging and sensing, polarization scattering imaging (PSI) has obtained impressive progresses, however, it remains a challenging but long-standing puzzle due to the fact that scattering medium can result in significant degradation of the object information. Herein, we explore the relationship between multiple polarization feature learning strategy and the PSI performances, and propose a new multi-polarization driven multi-pipeline (MPDMP) framework to extract rich hierarchical representations from multiple independent polarization feature maps. Based on the MPDMP framework, we introduce a well-designed three-stage multi-pipeline networks (TSMPN) architecture to achieve the PSI, named TSMPN-PSI. The proposed TSMPN-PSI comprises three stages: pre-processing polarization image for de-speckling, multiple polarization feature learning, and target information reconstruction. Furthermore, we establish a real-world polarization scattering imaging system under active light illumination to acquire a dataset of real-life scenarios for training the model. Both qualitative and quantitative experimental results show that the proposed TSMPN-PSI achieves higher generalization performance than other methods on three testing data sets refer to imaging distances, target structures, and target materials and their background materials. We believe that our work presents a new framework for the PSI and paves the way to its pragmatic applications.

Pol2Pol: self-supervised polarimetric image denoising

In this Letter, we present a self-supervised method, polarization to polarization (Pol2Pol), for polarimetric image denoising with only one-shot noisy images. First, a polarization generator is proposed to generate training image pairs, which are synthesized from one-shot noisy images by exploiting polarization relationships. Second, the Pol2Pol method is extensible and compatible, and any network that performs well in supervised image denoising tasks can be deployed to Pol2Pol after proper modifications. Experimental results show Pol2Pol outperforms other self-supervised methods and achieves comparable performance to supervised methods.

Polarized image super-resolution via a deep convolutional neural network

Reduced resolution of polarized images makes it difficult to distinguish detailed polarization information and limits the ability to identify small targets and weak signals. A possible way to handle this problem is the polarization super-resolution (SR), which aims to obtain a high-resolution polarized image from a low-resolution one. However, compared with the traditional intensity-mode image SR, the polarization SR is more challenging because more channels and their nonlinear cross-links need to be considered as well as the polarization and intensity information need to be reconstructed simultaneously. This paper analyzes the polarized image degradation and proposes a deep convolutional neural network for polarization SR reconstruction based on two degradation models. The network structure and the well-designed loss function have been verified to effectively balance the restoration of intensity and polarization information, and can realize the SR with a maximum scaling factor of four. Experimental results show that the proposed method outperforms other SR methods in terms of both quantitative evaluation and visual effect evaluation for two degradation models with different scaling factors.

Physics-informed neural network for polarimetric underwater imaging

Utilizing the polarization analysis in underwater imaging can effectively suppress the scattered light and help to restore target signals in turbid water. Neural network-based solutions can also boost the performance of polarimetric underwater imaging, while most of the existing networks are pure data driven which suffer from ignoring the physical mode. In this paper, we proposed an effective solution that informed the polarimetric physical model and constrains into the well-designed deep neural network. Especially compared with the conventional underwater imaging model, we mathematically transformed the two polarization-dependent parameters to a single parameter, making it easier for the network to converge to a better level. In addition, a polarization perceptual loss is designed and applied to the network to make full use of polarization information on the feature level rather than on the pixel level. Accordingly, the network was able to learn the polarization modulated parameter and to obtain clear de-scattered images. The experimental results verified that the combination of polarization model and neural network was beneficial to improve the image quality and outperformed other existing methods, even in a high turbidity condition.

Attention-based neural network for polarimetric image denoising

In this Letter, we propose an attention-based neural network specially designed for the challenging task of polarimetric image denoising. In particular, the channel attention mechanism is used to effectively extract the features underlying the polarimetric images by rescaling the contributions of channels in the network. In addition, we also design the adaptive polarization loss to make the network focus on the polarization information. Experiments show that our method can well restore the details flooded by serious noise and outperforms previous methods. Moreover, the underlying mechanism of channel attention is revealed visually.

Enhanced polarization demosaicking network via a precise angle of polarization loss calculation method

Many existing polarization networks reconstruct polarization information based on calculating the angle of polarization (AoP) loss. Yet, the conventional loss calculation method, which is based on a linear difference approach, compromises the reconstruction accuracy and causes additional training time when combined with learning-based methods. In this Letter, we present a new, to the best of our knowledge, method to calculate the AoP loss and apply it in an enhanced color polarization demosaicking network with a "multi-branch" structure, i.e., ePDNet. Experiments are performed to demonstrate the efficacy and superiority of the method, which improves the network convergence speed by three times as well as the output image quality. The new method may find important applications in the field of polarimetric imaging.

Unsupervised anomaly detection of MEMS in low illumination based on polarimetric Support Vector Data Description

Low illuminated images make it challenging to conduct anomaly detection on material surface. Adding polarimetric information helps expand pixel range and recover background structure of network inputs. In this letter, an anomaly detection method in low illumination is proposed which utilizes polarization imaging and patch-wise Support Vector Data Description (SVDD) model. Polarimetric information of Micro Electromechanical System (MEMS) surface is captured by a division-of-focal- plane (DoFP) polarization camera and used to enhance low illuminated images. The enhanced images without defects serve as training sets of model to make it available for anomaly detection. The proposed method can generate heatmaps to locate defects correctly. It reaches 0.996 anomaly scores, which is 22.4% higher than that of low illuminated images and even higher than normal illuminated images.

Research on Polarized Multi-Spectral System and Fusion Algorithm for Remote Sensing of Vegetation Status at Night

The monitoring of vegetation via remote sensing has been widely applied in various fields, such as crop diseases and pests, forest coverage and vegetation growth status, but such monitoring activities were mainly carried out in the daytime, resulting in limitations in sensing the status of vegetation at night. In this article, with the aim of monitoring the health status of outdoor plants at night by remote sensing, a polarized multispectral low-illumination-level imaging system (PMSIS) was established, and a fusion algorithm was proposed to detect vegetation by sensing the spectrum and polarization characteristics of the diffuse and specular reflection of vegetation. The normalized vegetation index (NDVI), degree of linear polarization (DoLP) and angle of polarization (AOP) are all calculated in the fusion algorithm to better detect the health status of plants in the night environment. Based on NDVI, DoLP and AOP fusion images (NDAI), a new index of night plant state detection (NPSDI) was proposed. A correlation analysis was made for the chlorophyll content (SPAD), nitrogen content (NC), NDVI and NPSDI to understand their capabilities to detect plants under stress. The scatter plot of NPSDI shows a good distinction between vegetation with different health levels, which can be seen from the high specificity and sensitivity values. It can be seen that NPSDI has a good correlation with NDVI (coefficient of determination R2 = 0.968), PSAD (R2 = 0.882) and NC (R2 = 0.916), which highlights the potential of NPSDI in the identification of plant health status. The results clearly show that the proposed fusion algorithm can enhance the contrast effect and the generated fusion image will carry richer vegetation information, thereby monitoring the health status of plants at night more effectively. This algorithm has a great potential in using remote sensing platform to monitor the health of vegetation and crops.

Integration time optimization and starting angle autocalibration of full Stokes imagers based on a rotating retarder

Full Stokes imaging can be performed with a continuously rotating retarder in front of a fixed polarizer and a standard camera (RRFP) or a division of a focal plane polarization camera (RRDOFP). We determine the optimal number and duration of intensity measurements through a cycle of the retarder for these two types of setups as a function of instrument and noise parameters. We show that this number mainly depends on the type of noise that corrupts the measurements. We also show that with these setups, the starting angle of the retarder need not be known precisely and can be autocalibrated, which facilitates synchronization of the rotating retarder with the camera. We investigate the precision and feasibility domain of this autocalibration and show the RRDOFP setup has more attractive properties compared with RRFP setup. These results are important to optimize and facilitate the operation of polarization imagers based on a rotating retarder.