Insect-inspired vision for autonomous vehicles

Finding the gap: neuromorphic motion-vision in dense environments

Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.

A virtuous cycle between invertebrate and robotics research: perspective on a decade of Living Machines research

Many invertebrates are ideal model systems on which to base robot design principles due to their success in solving seemingly complex tasks across domains while possessing smaller nervous systems than vertebrates. Three areas are particularly relevant for robot designers: Research on flying and crawling invertebrates has inspired new materials and geometries from which robot bodies (their morphologies) can be constructed, enabling a new generation of softer, smaller, and lighter robots. Research on walking insects has informed the design of new systems for controlling robot bodies (their motion control) and adapting their motion to their environment without costly computational methods. And research combining wet and computational neuroscience with robotic validation methods has revealed the structure and function of core circuits in the insect brain responsible for the navigation and swarming capabilities (their mental faculties) displayed by foraging insects. The last decade has seen significant progress in the application of principles extracted from invertebrates, as well as the application of biomimetic robots to model and better understand how animals function. This Perspectives paper on the past 10 years of the Living Machines conference outlines some of the most exciting recent advances in each of these fields before outlining lessons gleaned and the outlook for the next decade of invertebrate robotic research.

The Understanding of ON-Edge Motion Detection Through the Simulation Based on the Connectome of ’s Optic Lobe

The optic lobe of the fly is one of the prominent model systems for the neural mechanism of the motion detection. How a fly who lives under various visual situations of the nature processes the information from at most a few thousands of ommatidia in their neural circuit for the detection of moving objects is not exactly clear though many computational models of the fly optic lobe as a moving objects detector were suggested. Here we attempted to elucidate the mechanisms of ON-edge motion detection by a simulation approach based on the TEM connectome of Drosophila. Our simulation model of the optic lobe with the NEURON simulator that covers the full scale of ommatidia, reproduced the characteristics of the receptor neurons, lamina monopolar neurons, and T4 cells in the lobula. The contribution of each neuron can be estimated by changing synaptic connection strengths in the simulation and measuring the response to the motion stimulus. Those show the paradelle pathway provide motion detection in the fly optic lobe has more robustness and is more sophisticated than a simple combination of HR and BL systems.

Hybrid visual control in fly flight: insights into gaze shift via saccades

Flies fly by alternating between periods of fixation and body saccades, analogous to how our own eyes move. Gaze fixation via smooth movement in fly flight has been studied extensively, but comparatively less is known about the mechanism by which flies trigger and control body saccades to shift their gaze. Why do flies implement a hybrid fixate-and-saccade locomotion strategy? Here we review recent developments that provide new insights into this question. We focus on the interplay between smooth movement and saccades, the trigger classes of saccades, and the timeline of saccade execution. We emphasize recent mechanistic advances in Drosophila, where genetic tools have enabled cellular circuit analysis at an unprecedented level in a flying insect. In addition, we review trade-offs in behavioral paradigms used to study saccades. Throughout we highlight exciting avenues for future research in the control of fly flight.

Limitation of Rayleigh sky model for bioinspired polarized skylight navigation in three-dimensional attitude determination

Insects such as desert ants and drosophilae can sense polarized skylight for navigation. Inspired by insects, many researchers have begun to study how to use skylight polarization patterns for attitude determination. The Rayleigh sky model has become the most widely used skylight polarization model for bioinspired polarized skylight navigation due to its simplicity and practicality. However, this is an ideal model considering only single Rayleigh scatter events, and the limitation of this model in bio-inspired attitude determination has not been paid much attention and lacks strict inference proof. To address this problem, the rotational and plane symmetry of the Rayleigh sky model are analyzed in detail, and it is theoretically proved that this model contains only single solar vector information, which contains only two independent scalar pieces of attitude information, so it is impossible to determine three Euler angles simultaneously in real-time. To further verify this conclusion, based on a designed hypothetical polarization camera, we discuss what conditions different three-dimensional attitudes must satisfy so that the polarization images taken at different 3D attitudes are the same; this indicates that multiple solutions will appear when only using the Rayleigh sky model to determine 3D attitude. In conclusion, due to its single solar vector information and the existence of multiple solutions, it is fully proved that 3D attitude cannot be determined in real time based only upon the Rayleigh sky model. Code is available at: https://github.com/HuajuLiang/HypotheticalPolarizationCamera.

A Bio-Inspired Polarization Sensor with High Outdoor Accuracy and Central-Symmetry Calibration Method with Integrating Sphere

A bio-inspired polarization sensor with lenses for navigation was evaluated in this study. Two new calibration methods are introduced, referred to as "central-symmetry calibration" (with an integrating sphere) and "noncontinuous calibration". A comparison between the indoor calibration results obtained from different calibration methods shows that the two proposed calibration methods are more effective. The central-symmetry calibration method optimized the nonconstant calibration voltage deviations, caused by the off-axis feature of the integrating sphere, to be constant values which can be calibrated easily. The section algorithm proposed previously showed no experimental advantages until the central-symmetry calibration method was proposed. The outdoor experimental results indicated that the indoor calibration parameters did not perform very well in practice outdoor conditions. To establish the reason, four types of calibration parameters were analyzed using the replacement method. It can be concluded that three types can be easily calibrated or affect the sensor accuracy slightly. However, before the sensor is used outdoors every time, the last type must be replaced with the corresponding outdoor parameter, and the calculation needs a precise rotary table. This parameter, which is mainly affected by the spectrum of incident light, is the main factor determining the sensor accuracy. After calibration, the sensor reaches an indoor accuracy of ±0.009° and a static outdoor accuracy of ±0.05° under clear sky conditions. The dynamic outdoor experiment shows a ±0.5° heading deviation between the polarization sensor and the inertial navigation system with a ±0.06° angular accuracy.