Polarization patterns under different sky conditions and a navigation method based on the symmetry of the AOP map of skylight

Passive Polarized Vision for Autonomous Vehicles: A Review

This review article aims to address common research questions in passive polarized vision for robotics. What kind of polarization sensing can we embed into robots? Can we find our geolocation and true north heading by detecting light scattering from the sky as animals do? How should polarization images be related to the physical properties of reflecting surfaces in the context of scene understanding? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying future directions in passive polarized vision for robotics. After an introduction, three key interconnected areas will be covered in the following sections: embedded polarization imaging; polarized vision for robotics navigation; and polarized vision for scene understanding. We will then discuss how polarized vision, a type of vision commonly used in the animal kingdom, should be implemented in robotics; this type of vision has not yet been exploited in robotics service. Passive polarized vision could be a supplemental perceptive modality of localization techniques to complement and reinforce more conventional ones.

Underwater biomimetic orientation method using imaging polarization sensor based on direct sunlight compensation

This paper addresses the challenge of significant interference caused by direct sunlight, which adversely affects the orientation accuracy of underwater imaging polarization sensors (IPS). A novel underwater polarization orientation method is proposed based on direct sunlight compensation. Firstly, based on the polarization transmission model at the water-air interface, the interference mechanism of the underwater direct sunlight polarization detection model was analyzed. The underwater IPS detection model based on direct sunlight compensation is constructed, which uses the weight coefficient of underwater direct sunlight to compensate for the interference on the polarization channel and improve the accuracy of underwater polarization detection models. Furthermore, the analytical solution method for the polarization state information of underwater IPS is proposed, employing the augmented Stokes vectors to construct a linear equation for solving the weight coefficients of direct sunlight and improving the computational efficiency. Finally, an underwater polarization orientation experimental platform is established, and both simulation and actual underwater experiments are conducted. Compared with the traditional methods, the proposed method reduces heading error by an average of 92.53% at different solar altitudes.

Attitude and heading measurement based on adaptive complementary Kalman filter for PS/MIMU integrated system

The bionic polarization sensor (PS)/MEMS inertial measurement unit (MIMU) integrated system can provide reliable attitude and heading information for unmanned vehicles in the case of GNSS rejection. However, the existing measurement methods have poor adaptability to inclining, sheltering, and other harsh environments, and do not make full use of the complementary characteristics of the gyroscopes, accelerometers, and PS, which seriously affects the system performance. Therefore, this paper proposes an attitude and heading measurement method based on an adaptive complementary Kalman filter (ACKF), which corrects the gyroscopes according to the gravity measured by the accelerometers to improve the attitude accuracy and fuses the IMU heading and tilt-compensated polarization heading by Kalman optimal estimation. On this basis, the maximum correlation entropy of the measured gravity and the theoretical gravity is used to construct an adaptive factor to realize the adaptive complementary of the gyroscopes and the accelerometers. Finally, the effectiveness of the method is verified by the outdoor rotation test without occlusion and the vehicle test with occlusion. Compared with the traditional Kalman filter, the pitch, roll, and heading RMSE of the vehicle test are reduced by 89.3%, 93.2% and, 9.6% respectively, which verifies the great advantages.

Image-registration-based solar meridian detection for accurate and robust polarization navigation

Skylight polarization, inspired by the foraging behavior of insects, has been widely used for navigation for various platforms, such as robots, unmanned aerial vehicles, and others, owing to its stability and non-error-accumulation. Among the characteristics of skylight-polarized patterns, the angle of polarization (AOP) and the degree of polarization (DOP) are two of the most significant characteristics that provide abundant information regarding the position of the sun. In this study, we propose an accurate method for detecting the solar meridian for real-time bioinspired navigation through image registration. This method uses the AOP pattern to detect the solar meridian and eliminates the ambiguity between anti-solar meridian and solar meridian using the DOP pattern, resulting in an accurate heading of the observer. Simulation experiments demonstrated the superior performance of the proposed method compared to the alternative approaches. Field experiments demonstrate that the proposed method achieves real-time, robust, and accurate performance under different weather conditions with a root mean square error of 0.1° under a clear sky, 0.18° under an overcast sky with a thin layer of clouds, and 0.32° under an isolated thick cloud cover. Our findings suggest that the proposed method can be potentially used in skylight polarization for real-time and accurate navigation in GPS-denied environments.

Measurement Modeling and Performance Analysis of a Bionic Polarimetric Imaging Navigation Sensor Using Rayleigh Scattering to Generate Scattered Sunlight

The bionic polarimetric imaging navigation sensor (BPINS) is a navigation sensor that provides absolute heading, and it is of practical engineering significance to model the measurement error of BPINS. The existing BPINSs are still modeled using photodiode-based measurements rather than imaging measurements and are not modeled systematically enough. This paper proposes a measurement performance analysis method of BPINS that takes into account the geometric and polarization errors of the optical system. Firstly, the key error factors affecting the overall measurement performance of BPINS are investigated, and the Stokes vector-based measurement error model of BPINS is introduced. Secondly, based on its measurement error model, the effect of the error source on the measurement performance of BPINS is quantitatively analyzed using Rayleigh scattering to generate scattered sunlight as a known incident light source. The numerical results show that in angle of E-vector (AoE) measurement, the coordinate deviation of the principal point has a greater impact, followed by grayscale response inconsistency of CMOS and integration angle error of micro-polarization array, and finally lens attenuation; in degree of linear polarization (DoLP) measurement, the grayscale response inconsistency of CMOS has a more significant impact. This finding can accurately guide the subsequent calibration of BPINS, and the quantitative results provide an important theoretical reference for its optimal design.

Speedy bearings to slacked steering: Mapping the navigation patterns and motions of Viking voyages

Viking sailors ruled the North Atlantic Ocean for about three hundred years. Their main sailing route was the 60° 21' 55'' latitude between Norway and Greenland. Although they did not have a magnetic compass, in sunshine they used a sun-compass to determine the geographical north (solar Viking navigation: SVN). It has been hypothesized that when the Sun was invisible, Viking navigators determined the direction of polarization of skylight with sunstones (dichroic/birefringent crystals), and then estimated the geographical north using the sun-compass (sky-polarimetric Viking navigation: SPVN). Many details of the hypothetical SPVN have been thoroughly revealed in psychophysical laboratory and planetarium experiments. Combining these results with measured celestial polarization patterns, the success of SPVN was obtained as functions of sailing, meteorological and navigation parameters (sunstone type, sailing date, navigation periodicity, night sailing, cloudiness conditions). What was so far lacking in this experimental and computational archeological approach is the study of the success of SVN and a combined navigation using solar cues in sunshine (SVN) and sky polarization at invisible Sun (SPVN), the latter being the most realistic method. In this work we determine the success of the sole SVN and the combined SVN-SPVN relative to the mere SPVN for three navigator types (determining the intended sailing direction with large, medium or small frequencies) at spring equinox and summer solstice, with and without night sailing. We found that to maximize the sailing success, navigators had to choose different navigation methods depending on the navigation frequency. Using sky polarization with very frequent navigation, resulted in the highest chance to survive a three-week voyage from Norway to Greenland.

Modeling the celestial distribution of skylight polarization patterns by incorporating the influence of both the sun and the moon through an analytical model

The orientation of many polarization-sensitive animals and the hypothetical sky-polarimetric Viking navigation both rely on the polarization pattern of skylight. For 40 years, scientists have attempted to construct various models to simulate this pattern. However, existing theoretical models have only analyzed the polarization pattern of skylight that is influenced separately by the sun or the moon and have built their modeling frameworks based on the position of one light source. This approach fails to account for the combined influence of the sun and the moon on the distribution of skylight polarization patterns at certain times. In fact, ignoring the influence of the moon during the dawn and dusk periods in clear weather conditions may lead to significant errors in the simulation results compared to the measured data. In this paper, we present an analytical model that considers various factors, including skylight intensity, horizon correction factor, atmospheric turbidity condition, and combined influence of both the sun and moon on the distribution of polarized skylight. We believe our model demonstrates enhanced agreement with measured data and will further our understanding of how animals use the celestial polarization pattern for navigation, particularly when both the sun and the moon appear in the sky. Moreover, the findings of this study may facilitate the advancement of bio-inspired navigation systems.

Impact of aerosols on the polarization patterns of full-sky background radiation

Regarding aerosol particle-laded turbid atmospheres, full-sky background radiation polarization patterns can be adversely affected, an important factor limiting their effective near-ground observation and acquisition. We established a multiple-scattering polarization computational model and measurement system and conducted the following three tasks. (a) We thoroughly analyzed the impact of aerosol scattering characteristics on polarization distributions, calculating the degree of polarization (DOP) and angle of polarization (AOP) patterns for a more comprehensive set of atmospheric aerosol compositions and aerosol optical depth (AOD) values than calculated in previous studies. (b) We assessed the uniqueness of the DOP and AOP patterns as a function of AOD. (c) By employing a new polarized radiation acquisition system for measurements, we demonstrated that our computational models are more representative of the DOP and AOP patterns under actual atmospheric conditions. We found that under a clear sky without clouds, the impact of the AOD on the DOP was detectable. With increasing AOD, the DOP decreased, and the decreasing trend became increasingly obvious. When the AOD was above 0.3, the maximum DOP did not exceed 0.5. The AOP pattern did not change notably and remained stable, except for the contraction point at the sun position under an AOD of 2.

High-similarity analytical model of skylight polarization pattern based on position variations of neutral points

The skylight polarization pattern contains rich information for navigation, meteorological monitoring, and remote sensing. In this paper, we propose a high-similarity analytical model by considering the influence of the solar altitude angle on the neutral point position variations for the distribution pattern of the polarized skylight. A novel function is built to determine the relationship between the neutral point position and solar elevation angle based on a large number of measured data. The experimental results show that the proposed analytical model achieves a higher similarity to measured data compared with existing models. Furthermore, data from several consecutive months verifies the universality, effectiveness, and accuracy of this model.

Polarized light compass decoding

The brains of some insects can encode and decode polarization information and obtain heading angle information. Referring to the encoding ability of insects, exponential function encoding is designed to improve the stability of the polarized light compass artificial neural network. However, in the decoding process, only neurons with the largest activation degree are used for decoding (maximum value decoding), so the heading information contained in other neurons is not used. Therefore, average value decoding (AVD) and weighted AVD are proposed to use the heading information contained in multiple neurons to determine the heading. In addition, concerning the phenomenon of threshold activation of insect neurons, threshold value decoding (TVD) and weighted TVD are proposed, which can effectively eliminate the interference of neurons with low activation. Moreover, this paper proposes to improve the heading determination accuracy of the artificial neural network through pre-training. The simulation and experimental results show that the new, to the best of our knowledge, decoding methods and pre-training can effectively improve the heading determination accuracy of the artificial neural network.

Ultraviolet-visible light compass method based on local atmospheric polarization characteristics in adverse weather conditions

Bionic polarization navigation has attracted extensive attention because of its strong anti-interference performance and no accumulation of errors over time. However, very few studies have fully considered the influence of adverse weather conditions such as cloudy and overcast weather, which play a key role in navigation accuracy. Therefore, we propose an adaptive ultraviolet-visible light compass method based on local atmospheric polarization characteristics applicable to various weather conditions. The proposed method transforms the heading determination problem into a multiclassification problem by using a weather recognition technique. Ultraviolet detection is used to weaken the depolarization effect of cloud particles and to obtain more accurate skylight polarization patterns. Then, on the basis of screening effective data, the sun direction vector is calculated by using the electric vector direction and is finally combined with the astronomical calendar to achieve navigation. The experimental results confirm that, compared to the other methods, the designed algorithm can suppress the interference of clouds better and adapt to complex weather conditions. Under cloudy and overcast conditions, the heading angle error is reduced to less than 2°.

Polarized light sun position determination artificial neural network

Our previous work has constructed a polarized light orientation determination (PLOD) artificial neural network. Although a PLOD network can determine the solar azimuth angle, it cannot determine the solar elevation angle. Therefore, this paper proposes an artificial neural network for polarized light solar position determination (PLSPD), which has two branches: the solar azimuth angle determination branch and the solar elevation angle determination branch. Since the solar elevation angle has no cyclic characteristics, and the angle range of the solar elevation angle is different from that of the solar azimuth angle, the solar elevation angle exponential function encoding is redesigned. In addition, compared with the PLOD, the PLSPD deletes a local full connection layer to simplify the network structure. The experimental results show that the PLSPD can determine not only the solar azimuth angle but also the solar elevation angle, and the solar azimuth angle determination accuracy of the PLSPD is higher than that of the PLOD.

Measurement error model of the bio-inspired polarization imaging orientation sensor

This article studies the measurement error model and calibration method of the bio-inspired polarization imaging orientation sensor (BPIOS), which has important engineering significance for promoting bio-inspired polarization navigation. Firstly, we systematically analyzed the measurement errors in the imaging process of polarized skylight and accurately established an error model of BPIOS based on Stokes vector. Secondly, using the simulated Rayleigh skylight as the incident surface light source, the influence of multi-source factors on the measurement accuracy of BPIOS is quantitatively given for the first time. These simulation results can guide the later calibration of BPIOS. We then proposed a calibration method of BPIOS based on geometric parameters and the Mueller matrix of the optical system and conducted an indoor calibration experiment. Experimental results show that the measurement accuracy of the calibrated BPIOS can reach 0.136°. Finally, the outdoor performance of BPIOS is studied. Outdoor dynamic performance test and field compensation were performed. Outdoor results show that the heading accuracy of BPIOS is 0.667°.

Neutral point detection using the AOP of polarized skylight patterns

The neutral points are one of the most significant characteristics of the polarized skylight pattern in the whole sky. At present, detection of the neutral points mostly utilizes ellipse fitting of the degree of polarization. However, because the degree of polarization distribution characteristics of a polarized skylight pattern is easily affected by the environment, the robustness of the detection is unstable. Aiming at the problem, we analyzed the angle of polarization distribution characteristics of polarized skylight patterns in the region around the neutral point by measurement experiments. Based on this, we proposed an automatic detection method of neutral points using the angle of polarization of the polarized skylight pattern. The experimental results of different times in a continuous period of time show that compared with ellipse fitting of the degree of polarization, the detection accuracy of the proposed method is almost the same, but the robustness is better. It provides a novel method for the position detecting of the neutral point, which is in favor of the measurement applications of polarization technology.

Foraging trip duration of honeybee increases during a poor air quality episode and the increase persists thereafter

Increased concentration of airborne particulate matter (PM) in the atmosphere alters the degree of polarization of skylight which is used by honeybees for navigation during their foraging trips. However, little has empirically shown whether poor air quality indeed affects foraging performance (foraging trip duration) of honeybee. Here, we show apparent increases in the average duration of honeybee foraging during and after a heavy air pollution event compared with that of the pre-event period. The average foraging duration of honeybees during the event increased by 32 min compared with the pre-event conditions, indicating that 71% more time was spent on foraging. Moreover, the average foraging duration measured after the event did not recover to its pre-event level. We further investigated whether an optical property (Depolarization Ratio, DR) of dominant PM in the atmosphere and level of air pollution (fine PM mass concentration) affect foraging trip duration. The result demonstrates the DR and fine PM mass concentration have significant effects on honeybee foraging trip duration. Foraging trip duration increases with decreasing DR while it increases with increasing fine PM mass concentration. In addition, the effects of fine PM mass concentration are synergistic with overcast skies. Our study implies that poor air quality could pose a new threat to bee foraging.

Digital all-sky polarization imaging of the total solar eclipse on 21 August 2017 in Rexburg, Idaho, USA

All-sky polarization images were measured from sunrise to sunset and during a cloud-free totality on 21 August 2017 in Rexburg, Idaho using two digital three-camera all-sky polarimeters and a time-sequential liquid-crystal-based all-sky polarimeter. Twenty-five polarimetric images were recorded during totality, revealing a highly dynamic evolution of the distribution of skylight polarization, with the degree of linear polarization becoming nearly zenith-symmetric by the end of totality. The surrounding environment was characterized with an infrared cloud imager that confirmed the complete absence of clouds during totality, an AERONET solar radiometer that measured aerosol properties, a portable weather station, and a hand-held spectrometer with satellite images that measured surface reflectance at and near the observation site. These observations confirm that previously observed totality patterns are general and not unique to those specific eclipses. The high temporal image resolution revealed a transition of a neutral point from the zenith in totality to the normal Babinet point just above the Sun after third contact, providing the first indication that the transition between totality and normal daytime polarization patterns occurs over of a time period of approximately 13 s.

Limitation of Rayleigh sky model for bioinspired polarized skylight navigation in three-dimensional attitude determination

Insects such as desert ants and drosophilae can sense polarized skylight for navigation. Inspired by insects, many researchers have begun to study how to use skylight polarization patterns for attitude determination. The Rayleigh sky model has become the most widely used skylight polarization model for bioinspired polarized skylight navigation due to its simplicity and practicality. However, this is an ideal model considering only single Rayleigh scatter events, and the limitation of this model in bio-inspired attitude determination has not been paid much attention and lacks strict inference proof. To address this problem, the rotational and plane symmetry of the Rayleigh sky model are analyzed in detail, and it is theoretically proved that this model contains only single solar vector information, which contains only two independent scalar pieces of attitude information, so it is impossible to determine three Euler angles simultaneously in real-time. To further verify this conclusion, based on a designed hypothetical polarization camera, we discuss what conditions different three-dimensional attitudes must satisfy so that the polarization images taken at different 3D attitudes are the same; this indicates that multiple solutions will appear when only using the Rayleigh sky model to determine 3D attitude. In conclusion, due to its single solar vector information and the existence of multiple solutions, it is fully proved that 3D attitude cannot be determined in real time based only upon the Rayleigh sky model. Code is available at: https://github.com/HuajuLiang/HypotheticalPolarizationCamera.

Bio-inspired attitude measurement method using a polarization skylight and a gravitational field

High precision and reliability attitude measurement play an important role in autonomous unmanned navigation. Finding inspiration from desert ants, known as highly efficient navigators who can find their way after foraging for hundreds of meters from their home in hostile environments, we propose an attitude measurement method using polarization skylight and gravitational field. Contrary to the previous method, we utilize three-dimensional polarization vectors and any one-dimensional output of the accelerometers to calculate attitudes. In addition, we designed an accelerometer component selection algorithm, which is to select the one-dimensional component with the minimum motion acceleration from the output of the three-dimensional accelerometer. With this method, even if the carriers remain in a maneuvering state, the motion acceleration of the vehicle will have less impact on the accuracy of attitude measurement. To evaluate the performance of our method, the outdoor experiment was carried out to compare our method with existing traditional methods. Comparison results show that our method has higher measurement accuracy than others and is still applicable in the case of carriers maneuvering in practice under a clear sky.

Polarized skylight compass based on a soft-margin support vector machine working in cloudy conditions

The skylight polarization pattern, which is a result of the scattering of unpolarized sunlight by particles in the atmosphere, can be used by many insects for navigation. Inspired by insects, several polarization navigation sensors have been designed and combined with various heading determination methods in recent years. However, up until now, few of these studies have fully considered the influences of different meteorological conditions, which play key roles in navigation accuracy, especially in cloudy weather. Therefore, this study makes a major contribution to the study on bio-inspired heading determination by designing a skylight compass method to suppress cloud disturbances. The proposed method transforms the heading determination problem into a binary classification problem by segmentation, connected component detection, and inversion. Considering the influences of noise and meteorological conditions, the binary classification problem is solved by the soft-margin support vector machine. In addition, to verify this method, a pixelated polarization compass platform is constructed that can take polarization images at four different orientations simultaneously in real time. Finally, field experimental results show that the designed method can more effectively suppress the interference of clouds compared with other methods.

Improved Models of Imaging of Skylight Polarization Through a Fisheye Lens

Researchers have found that some animals can use the skylight polarization pattern for navigation. It is also expected to use the skylight polarization pattern for human navigating in the near future. However, the challenge is that the need for a more accurate and efficient model of the imaging of skylight polarization is always felt. In this paper, three improved models of imaging of skylight polarization are proposed. The proposed models utilize the analysis of the distribution of the skylight polarization pattern after the polarization imaging system. Given that the skylight polarization pattern after the polarization imaging system is distorted, the focus of this paper is on the degree of distortion of the skylight polarization pattern in these imaging models. Experiments in clear weather conditions demonstrate that the proposed model operates close to the actual acquired skylight polarization pattern.

Bio-inspired orientation using the polarization pattern in the sky based on artificial neural networks

Many insects use the pattern of polarized light in the sky as a navigational cue. In this study, we use this sensory ability as a source of inspiration to create a computational orientation model based on an artificial neural network (POL-ANN). After a training phase using numerically generated sky polarization patterns, stable and convergent networks are obtained. We undertook a series of verification tests using four typical but different sky conditions and showed that the post-trained networks were able to make an accurate prediction of the direction of the sun. Comparisons between the proposed models and models based on the convolutional neural network (CNN) structure revealed the merits of the bio-inspired architecture. We further investigated the accuracy of the models based on two different (locust-like, broader; Drosophila-like, narrower) visual fields of the sky. We find that the accuracy of the computations depends on the overhead visual scene, specifically that wider fields of view perform better when information about the overhead polarization pattern is missing.

Brain-Like Navigation Scheme based on MEMS-INS and Place Recognition

Animals have certain cognitive competence about the environment so they can correct their navigation errors. Inspired by the excellent navigational behavior of animals, this paper proposes a brain-like navigation scheme to improve the accuracy and intelligence of Micro-Electro-Mechanical System based Inertial Navigation Systems (MEMS-INS). The proposed scheme employs vision to acquire external perception information as an absolute reference to correct the position errors of INS, which is established by analyzing the navigation and error correction mechanism of rat brains. In addition, to improve the place matching speed and precision of the system for visual scene recognition, this paper presents a novel place recognition algorithm that combines image scanline intensity (SI) and grid-based motion statistics (GMS) together which is named the SI-GMS algorithm. The proposed SI-GMS algorithm can effectively reduce the influence of uncertain environment factors on the recognition results, such as pedestrians and vehicles. It solves the problem that the matching result will occasionally go wrong when simply using the scanline intensity (SI) algorithm, or the slow matching speed when simply using grid-based motion statistics (GMS) algorithm. Finally, an outdoor Unmanned Aerial Vehicle (UAV) flight test is carried out. Based on the reference information from the high-precision GPS device, the results illustrate the effectiveness of the scheme in error correction of INS and the algorithm in place recognition.

An Angle of Polarization (AoP) Visualization Method for DoFP Polarization Image Sensors Based on Three Dimensional HSI Color Space

A demand for division of focal plane (DoFP) polarization image sensors grows rapidly as nanofabrication technologies become mature. The DoFP sensor can output real time data of polarization information. In this paper, a novel visualization method for angle of polarization (AoP) is proposed for DoFP polarization image sensors. The data characteristics of AoP are analyzed, and strategies for a visualization method are proposed which conforms to the characteristics of AoP data. According to these strategies, we propose a visualization method for AoP data based on three dimensional HSI color space. This method uses intensity and saturation to characterize the magnitude of the angle between the polarization direction and the horizontal direction wherein the hue indicates the deflection direction. It is shown by the numerical simulation that the noise in the AoP image can be suppressed by our visualization method. In addition, the real-world experiment results are consistent with the numerical simulation and verify that the AoP image obtained by our method can suppress the influence of characterization noise, and the image is simple and intuitive, which is advantageous to human vision. The proposed method can be directly used for the commercialized DoFP polarization image sensor to display real-time AoP data.