Real-time polarization imaging algorithm for camera-based polarization navigation sensors

Skylight Polarization Pattern Simulator Based on a Virtual-Real-Fusion Framework for Urban Bionic Polarization Navigation

In a data-driven context, bionic polarization navigation requires a mass of skylight polarization pattern data with diversity, complete ground truth, and scene information. However, acquiring such data in urban environments, where bionic polarization navigation is widely utilized, remains challenging. In this paper, we proposed a virtual-real-fusion framework of the skylight polarization pattern simulator and provided a data preparation method complementing the existing pure simulation or measurement method. The framework consists of a virtual part simulating the ground truth of skylight polarization pattern, a real part measuring scene information, and a fusion part fusing information of the first two parts according to the imaging projection relationship. To illustrate the framework, we constructed a simulator instance adapted to the urban environment and clear weather and verified it in 174 urban scenes. The results showed that the simulator can provide a mass of diverse urban skylight polarization pattern data with scene information and complete ground truth based on a few practical measurements. Moreover, we released a dataset based on the results and opened our code to facilitate researchers preparing and adapting their datasets to their research targets.

Short-wave infrared polarimetric image reconstruction using a deep convolutional neural network based on a high-frequency correlation

Imaging in visible and short-wave infrared (SWIR) wavebands is essential in most remote sensing applications. However, compared to visible imaging cameras, SWIR cameras typically have lower spatial resolution, which limits the detailed information shown in SWIR images. We propose a method to reconstruct high-resolution polarization SWIR images with the help of color images using the deep learning method. The training dataset is constructed from color images, and the trained model is well suited for SWIR image reconstruction. The experimental results show the effectiveness of the proposed method in enhancing the quality of the polarized SWIR images with much better spatial resolution. Some buried spatial and polarized information may be recovered in the reconstructed SWIR images.

Biomimetic navigation system using a polarization sensor and a binocular camera

With the vigorous development of vision techniques, simultaneous localization and mapping (SLAM) has shown the capability of navigating autonomous robots in global-navigation-satellite-system-denied environments. However, the long-term robust navigation of lightweight autonomous robots in outdoor environments with complex interferences, such as illumination change, dynamic objects, and electromagnetic interference, is still a great challenge. In this paper, a polarization sensor-aided SLAM (POL-SLAM) that can provide absolute heading constraints for pure SLAM is proposed. POL-SLAM is a lightweight, tightly coupled system consisting of a polarization sensor and binocular camera. By means of an initialization that uses a polarization sensor, an absolute heading angle for the entire map is designed. Additionally, an algorithm to eliminate mismatching points using the matching point vector is proposed. The objective function of bundle adjustment is then deduced according to the re-projection error and polarization sensor. The vehicle test shows that the yaw and trajectory accuracies of POL-SLAM are significantly improved compared to pure SLAM. The yaw and trajectory accuracies are increased by 43.1% and 36.6%, respectively. These results indicate that the proposed POL-SLAM can improve the reliability and robustness of pure SLAM and can be used in lightweight autonomous robots in outdoor environments.

Optical Design of a Common-Aperture Camera for Infrared Guided Polarization Imaging

Polarization and infrared imaging technology have unique advantages for various applications ranging from biology to ocean remote sensing. However, conventional combined polarization camera and infrared camera have limitations because they are constrained to single-band imaging systems with rotating polarizers and cascaded optics. Therefore, we propose a common-aperture mode based on multi-band infrared guided polarization imaging system (IGPIS) in this paper, which consists of infrared wide-area sensing and polarization features acquisition for accurate detection of ship targets. The IGPIS can provide images in visible polarization (0.45–0.76 μm), near-infrared polarization (0.76–0.9 μm), and long-wave infrared (8–12 μm) bands. Satellite attitude parameters and camera optical parameters are accurately calculated by establishing a dynamic imaging model for guidance imaging. We illustrate the imaging principle, sensors specifications and imaging performance analysis and the experimental results show that the MTF is 0.24 for visible and near-infrared, and 0.13 for long-wave infrared. The obtained multi-band images have an average gradient of 12.77 after accurate fusion. These results provide theoretical guidance for the design of common-aperture cameras in remote sensing imaging field.

Sensor Modeling and Calibration Method Based on Extinction Ratio Error for Camera-Based Polarization Navigation Sensor

The performance of camera-based polarization sensors largely depends on the estimated model parameters obtained through calibration. Limited by manufacturing processes, the low extinction ratio and inconsistency of the polarizer can reduce the measurement accuracy of the sensor. To account for the challenges, one extinction ratio coefficient was introduced into the calibration model to unify the light intensity of two orthogonal channels. Since the introduced extinction ratio coefficient is associated with degree of polarization (DOP), a new calibration method considering both azimuth of polarization (AOP) error and DOP error for the bionic camera-based polarization sensor was proposed to improve the accuracy of the calibration model parameter estimation. To evaluate the performance of the proposed camera-based polarization calibration model using the new calibration method, both indoor and outdoor calibration experiments were carried out. It was found that the new calibration method for the proposed calibration model could achieve desirable performance in terms of stability and robustness of the calculated AOP and DOP values.

A Bio-Inspired Polarization Sensor with High Outdoor Accuracy and Central-Symmetry Calibration Method with Integrating Sphere

A bio-inspired polarization sensor with lenses for navigation was evaluated in this study. Two new calibration methods are introduced, referred to as "central-symmetry calibration" (with an integrating sphere) and "noncontinuous calibration". A comparison between the indoor calibration results obtained from different calibration methods shows that the two proposed calibration methods are more effective. The central-symmetry calibration method optimized the nonconstant calibration voltage deviations, caused by the off-axis feature of the integrating sphere, to be constant values which can be calibrated easily. The section algorithm proposed previously showed no experimental advantages until the central-symmetry calibration method was proposed. The outdoor experimental results indicated that the indoor calibration parameters did not perform very well in practice outdoor conditions. To establish the reason, four types of calibration parameters were analyzed using the replacement method. It can be concluded that three types can be easily calibrated or affect the sensor accuracy slightly. However, before the sensor is used outdoors every time, the last type must be replaced with the corresponding outdoor parameter, and the calculation needs a precise rotary table. This parameter, which is mainly affected by the spectrum of incident light, is the main factor determining the sensor accuracy. After calibration, the sensor reaches an indoor accuracy of ±0.009° and a static outdoor accuracy of ±0.05° under clear sky conditions. The dynamic outdoor experiment shows a ±0.5° heading deviation between the polarization sensor and the inertial navigation system with a ±0.06° angular accuracy.

Orthogonal vector algorithm to obtain the solar vector using the single-scattering Rayleigh model

Information obtained from a polarization pattern in the sky provides many animals like insects and birds with vital long-distance navigation cues. The solar vector can be derived from the polarization pattern using the single-scattering Rayleigh model. In this paper, an orthogonal vector algorithm, which utilizes the redundancy of the single-scattering Rayleigh model, is proposed. We use the intersection angles between the polarization vectors as the main criteria in our algorithm. The assumption that all polarization vectors can be considered coplanar is used to simplify the three-dimensional (3D) problem with respect to the polarization vectors in our simulation. The surface-normal vector of the plane, which is determined by the polarization vectors after translation, represents the solar vector. Unfortunately, the two-directionality of the polarization vectors makes the resulting solar vector ambiguous. One important result of this study is, however, that this apparent disadvantage has no effect on the complexity of the algorithm. Furthermore, two other universal least-squares algorithms were investigated and compared. A device was then constructed, which consists of five polarized-light sensors as well as a 3D attitude sensor. Both the simulation and experimental data indicate that the orthogonal vector algorithms, if used with a suitable threshold, perform equally well or better than the other two algorithms. Our experimental data reveal that if the intersection angles between the polarization vectors are close to 90°, the solar-vector angle deviations are small. The data also support the assumption of coplanarity. During the 51 min experiment, the mean of the measured solar-vector angle deviations was about 0.242°, as predicted by our theoretical model.