Reactive searching and infotaxis in odor source localization

Active smelling in the American cockroach

Motion plays an essential role in sensory acquisition. From changing the position in which information can be acquired to fine-scale probing and active sensing, animals actively control the way they interact with the environment. In olfaction, movement impacts the time and location of odour sampling as well as the flow of odour molecules around the olfactory organs. Employing a detailed spatiotemporal analysis, we investigated how insect antennae interact with the olfactory environment in a species with a well-studied olfactory system - the American cockroach. Cockroaches were tested in a wind-tunnel setup during the presentation of odours with different attractivity levels: colony extract, butanol and linalool. Our analysis revealed significant changes in antennal kinematics when odours were presented, including a shift towards the stream position, an increase in vertical movement and high-frequency local oscillations. Nevertheless, the antennal shifting occurred predominantly in a single antenna while the overall range covered by both antennae was maintained throughout. These findings hold true for both static and moving stimuli and were more pronounced for attractive odours. Furthermore, we found that upon odour encounter, there was an increase in the occurrence of high-frequency antennal sweeps and vertical strokes, which were shown to impact the olfactory environment's statistics directly. Our study lays out a tractable system for exploring the tight coupling between sensing and movement, in which antennal sweeps, in parallel to mammalian sniffing, are actively involved in facilitating odour capture and transport, generating odour intermittency in environments with low air movement where cockroaches dwell.

Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark

The olfactory search POMDP (partially observable Markov decision process) is a sequential decision-making problem designed to mimic the task faced by insects searching for a source of odor in turbulence, and its solutions have applications to sniffer robots. As exact solutions are out of reach, the challenge consists in finding the best possible approximate solutions while keeping the computational cost reasonable. We provide a quantitative benchmarking of a solver based on deep reinforcement learning against traditional POMDP approximate solvers. We show that deep reinforcement learning is a competitive alternative to standard methods, in particular to generate lightweight policies suitable for robots.

Evidence for a consistent use of external cues by marine fish larvae for orientation

The larval stage is the main dispersive process of most marine teleost species. The degree to which larval behavior controls dispersal has been a subject of debate. Here, we apply a cross-species meta-analysis, focusing on the fundamental question of whether larval fish use external cues for directional movement (i.e., directed movement). Under the assumption that directed movement results in straighter paths (i.e., higher mean vector lengths) compared to undirected, we compare observed patterns to those expected under undirected pattern of Correlated Random Walk (CRW). We find that the bulk of larvae exhibit higher mean vector lengths than those expected under CRW, suggesting the use of external cues for directional movement. We discuss special cases which diverge from our assumptions. Our results highlight the potential contribution of orientation to larval dispersal outcomes. This finding can improve the accuracy of larval dispersal models, and promote a sustainable management of marine resources.

Autonomous search of an airborne release in urban environments using informed tree planning

The use of autonomous vehicles for source localisation is a key enabling tool for disaster response teams to safely and efficiently deal with chemical emergencies. Whilst much work has been performed on source localisation using autonomous systems, most previous works have assumed an open environment or employed simplistic obstacle avoidance, separate from the estimation procedure. In this paper, we explore the coupling of the path planning task for both source term estimation and obstacle avoidance in an adaptive framework. The proposed system intelligently produces potential gas sampling locations that will reliably inform the estimation engine by not sampling in the wake of buildings as frequently. Then a tree search is performed to generate paths toward the estimated source location that traverse around any obstacles and still allow for exploration of potentially superior sampling locations.The proposed informed tree planning algorithm is then tested against the standard Entrotaxis and Entrotaxis-Jump techniques in a series of high fidelity simulations. The proposed system is found to reduce source estimation error far more efficiently than its competitors in a feature rich environment, whilst also exhibiting vastly more consistent and robust results.

Alternation emerges as a multi-modal strategy for turbulent odor navigation

Foraging mammals exhibit a familiar yet poorly characterized phenomenon, 'alternation', a pause to sniff in the air preceded by the animal rearing on its hind legs or raising its head. Rodents spontaneously alternate in the presence of airflow, suggesting that alternation serves an important role during plume-tracking. To test this hypothesis, we combine fully resolved simulations of turbulent odor transport and Bellman optimization methods for decision-making under partial observability. We show that an agent trained to minimize search time in a realistic odor plume exhibits extensive alternation together with the characteristic cast-and-surge behavior observed in insects. Alternation is linked with casting and occurs more frequently far downwind of the source, where the likelihood of detecting airborne cues is higher relative to ground cues. Casting and alternation emerge as complementary tools for effective exploration with sparse cues. A model based on marginal value theory captures the interplay between casting, surging, and alternation.

Searching for a source without gradients: how good is infotaxis and how to beat it

Infotaxis is a popular search algorithm designed to track a source of odour in a turbulent environment using information provided by odour detections. To exemplify its capabilities, the source-tracking task was framed as a partially observable Markov decision process consisting in finding, as fast as possible, a stationary target hidden in a two-dimensional grid using stochastic partial observations of the target location. Here, we provide an extended review of infotaxis, together with a toolkit for devising better strategies. We first characterize the performance of infotaxis in domains from one dimension to four dimensions. Our results show that, while being suboptimal, infotaxis is reliable (the probability of not reaching the source approaches zero), efficient (the mean search time scales as expected for the optimal strategy) and safe (the tail of the distribution of search times decays faster than any power law, though subexponentially). We then present three possible ways of beating infotaxis, all inspired by methods used in artificial intelligence: tree search, heuristic approximation of the value function, and deep reinforcement learning. The latter is able to find, without any prior human knowledge, the (near) optimal strategy. Altogether, our results provide evidence that the margin of improvement of infotaxis towards the optimal strategy gets smaller as the dimensionality increases.

Identification of Exploration and Exploitation Balance in the Silkmoth Olfactory Search Behavior by Information-Theoretic Modeling

Insects search for and find odor sources as their basic behaviors, such as when looking for food or a mate. This has motivated research to describe how they achieve such behavior under turbulent odor plumes with a small number of neurons. Among different insects, the silk moth has been studied owing to its clear motor response to olfactory input. In past studies, the "programmed behavior" of the silk moth has been modeled as the average duration of a sequence of maneuvers based on the duration of periods without odor hits. However, this model does not fully represent the fine variations in their behavior. In this study, we used silk moth olfactory search trajectories from an experimental virtual reality device. We achieved an accurate input by using optogenetic silk moths that react to blue light. We then modeled such trajectories as a probabilistic learning agent with a belief of possible source locations. We found that maneuvers mismatching the programmed behavior are related to larger entropy decrease, that is, they are more likely to increase the certainty of the belief. This implies that silkmoths include some stochasticity in their search policy to balance the exploration and exploitation of olfactory information by matching or mismatching the programmed behavior model. We believe that this information-theoretic representation of insect behavior is important for the future implementation of olfactory searches in artificial agents such as robots.

A bio-hybrid odor-guided autonomous palm-sized air vehicle

Biohybrid systems integrate living materials with synthetic devices, exploiting their respective advantages to solve challenging engineering problems. One challenge of critical importance to society is detecting and localizing airborne volatile chemicals. Many flying animals depend their ability to detect and locate the source of aerial chemical plumes for finding mates and food sources. A robot with comparable capability could reduce human hazard and drastically improve performance on tasks such as locating disaster survivors, hazardous gas leaks, incipient fires, or explosives. Three advances are needed before they can rival their biological counterparts: (1) a chemical sensor with a much faster response time that nevertheless satisfies the size, weight, and power constraints of flight, (2) a design, sensor suite, and control system that allows it to move toward the source of a plume fully autonomously while navigating obstacles, and (3) the ability to detect the plume with high specificity and sensitivity among the assortment of chemicals that invariably exist in the air. Here we address the first two, introducing a human-safe palm-sized air vehicle equipped with the odor-sensing antenna of an insect, the first odor-sensing biohybrid robot system to fly. Using this sensor along with a suite of additional navigational sensors, as well as passive wind fins, our robot orients upwind and navigates autonomously toward the source of airborne plumes. Our robot is the first flying biohybrid system to successfully perform odor localization in a confined space, and it is able to do so while detecting and avoiding obstacles in its flight path. We show that insect antennae respond more quickly than metal oxide gas sensors, enabling odor localization at an improved speed over previous flying robots. By using the insect antennae, we anticipate a feasible path toward improved chemical specificity and sensitivity by leveraging recent advances in gene editing.

Interpreting the Spatial-Temporal Structure of Turbulent Chemical Plumes Utilized in Odor Tracking by Lobsters

Olfactory systems in animals play a major role in finding food and mates, avoiding predators, and communication. Chemical tracking in odorant plumes has typically been considered a spatial information problem where individuals navigate towards higher concentration. Recent research involving chemosensory neurons in the spiny lobster, Panulirus argus, show they possess rhythmically active or ‘bursting’ olfactory receptor neurons that respond to the intermittency in the odor signal. This suggests a possible, previously unexplored olfactory search strategy that enables lobsters to utilize the temporal variability within a turbulent plume to track the source. This study utilized computational fluid dynamics to simulate the turbulent dispersal of odorants and assess a number of search strategies thought to aid lobsters. These strategies include quantification of concentration magnitude using chemosensory antennules and leg chemosensors, simultaneous sampling of water velocities using antennule mechanosensors, and utilization of antennules to quantify intermittency of the odorant plume. Results show that lobsters can utilize intermittency in the odorant signal to track an odorant plume faster and with greater success in finding the source than utilizing concentration alone. However, the additional use of lobster leg chemosensors reduced search time compared to both antennule intermittency and concentration strategies alone by providing spatially separated odorant sensors along the body.

Odor tracking in aquatic organisms: the importance of temporal and spatial intermittency of the turbulent plume

In aquatic and terrestrial environments, odorants are dispersed by currents that create concentration distributions that are spatially and temporally complex. Animals navigating in a plume must therefore rely upon intermittent, and time-varying information to find the source. Navigation has typically been studied as a spatial information problem, with the aim of movement towards higher mean concentrations. However, this spatial information alone, without information of the temporal dynamics of the plume, is insufficient to explain the accuracy and speed of many animals tracking odors. Recent studies have identified a subpopulation of olfactory receptor neurons (ORNs) that consist of intrinsically rhythmically active 'bursting' ORNs (bORNs) in the lobster, Panulirus argus. As a population, bORNs provide a neural mechanism dedicated to encoding the time between odor encounters. Using a numerical simulation of a large-scale plume, the lobster is used as a framework to construct a computer model to examine the utility of intermittency for orienting within a plume. Results show that plume intermittency is reliably detectable when sampling simulated odorants on the order of seconds, and provides the most information when animals search along the plume edge. Both the temporal and spatial variation in intermittency is predictably structured on scales relevant for a searching animal that encodes olfactory information utilizing bORNs, and therefore is suitable and useful as a navigational cue.

Analysis of the role of wind information for efficient chemical plume tracing based on optogenetic silkworm moth behavior

Many animals use olfactory information to search for feeding areas and other individuals in real time and with high efficiency. We focus on the chemical plume tracing (CPT) ability of male silkworm moths and investigate an efficient CPT strategy for an autonomous robot. In the case of flying insects, the wind direction is an important factor in CPT, because the wind carries odors amongst other environmental information. However, whether the same phenomenon occurs in the walking silkworm moth has not been investigated. Therefore, we examine how the silkworm moth uses wind information during CPT. To accurately investigate the response to the wind direction, we introduce an optogenetic approach that replaces the odor stimulation with light stimulation, allowing us to separate the 'wind stimulus' from the 'odor stimulus'. We examine how the moth uses wind direction information in a biological experiment, and find that the movement speed is significantly reduced when the wind speed is relatively fast (1.0 m s^-1). By implementing this phenomenon in an autonomous robot, we can improve the successful search rate over that of the conventional moth-inspired algorithm. Regarding the search time, the proposed algorithm finds the odor source faster in a low-frequency odorant emission environment, whereas the search is slower than the conventional method when the odor frequency is higher. Therefore, switching from the use of wind direction information to odor information according to the frequency with which the odor is encountered leads to efficient CPT performance.

An olfactory virtual reality system for mice

All motile organisms use spatially distributed chemical features of their surroundings to guide their behaviors, but the neural mechanisms underlying such behaviors in mammals have been difficult to study, largely due to the technical challenges of controlling chemical concentrations in space and time during behavioral experiments. To overcome these challenges, we introduce a system to control and maintain an olfactory virtual landscape. This system uses rapid flow controllers and an online predictive algorithm to deliver precise odorant distributions to head-fixed mice as they explore a virtual environment. We establish an odor-guided virtual navigation behavior that engages hippocampal CA1 "place cells" that exhibit similar properties to those previously reported for real and visual virtual environments, demonstrating that navigation based on different sensory modalities recruits a similar cognitive map. This method opens new possibilities for studying the neural mechanisms of olfactory-driven behaviors, multisensory integration, innate valence, and low-dimensional sensory-spatial processing.

History dependence in insect flight decisions during odor tracking

Natural decision-making often involves extended decision sequences in response to variable stimuli with complex structure. As an example, many animals follow odor plumes to locate food sources or mates, but turbulence breaks up the advected odor signal into intermittent filaments and puffs. This scenario provides an opportunity to ask how animals use sparse, instantaneous, and stochastic signal encounters to generate goal-oriented behavioral sequences. Here we examined the trajectories of flying fruit flies (Drosophila melanogaster) and mosquitoes (Aedes aegypti) navigating in controlled plumes of attractive odorants. While it is known that mean odor-triggered flight responses are dominated by upwind turns, individual responses are highly variable. We asked whether deviations from mean responses depended on specific features of odor encounters, and found that odor-triggered turns were slightly but significantly modulated by two features of odor encounters. First, encounters with higher concentrations triggered stronger upwind turns. Second, encounters occurring later in a sequence triggered weaker upwind turns. To contextualize the latter history dependence theoretically, we examined trajectories simulated from three normative tracking strategies. We found that neither a purely reactive strategy nor a strategy in which the tracker learned the plume centerline over time captured the observed history dependence. In contrast, "infotaxis", in which flight decisions maximized expected information gain about source location, exhibited a history dependence aligned in sign with the data, though much larger in magnitude. These findings suggest that while true plume tracking is dominated by a reactive odor response it might also involve a history-dependent modulation of responses consistent with the accumulation of information about a source over multi-encounter timescales. This suggests that short-term memory processes modulating decision sequences may play a role in natural plume tracking.

Autonomous Multi-Robot Search for a Hazardous Source in a Turbulent Environment

Finding the source of an accidental or deliberate release of a toxic substance into the atmosphere is of great importance for national security. The paper presents a search algorithm for turbulent environments which falls into the class of cognitive (infotaxi) algorithms. Bayesian estimation of the source parameter vector is carried out using the Rao-Blackwell dimension-reduction method, while the robots are controlled autonomously to move in a scalable formation. Estimation and control are carried out in a centralised replicated fusion architecture assuming all-to-all communication. The paper presents a comprehensive numerical analysis of the proposed algorithm, including the search-time and displacement statistics.

Environmental monitoring using autonomous vehicles: a survey of recent searching techniques

Autonomous vehicles are becoming an essential tool in a wide range of environmental applications that include ambient data acquisition, remote sensing, and mapping of the spatial extent of pollutant spills. Among these applications, pollution source localization has drawn increasing interest due to its scientific and commercial interest and the emergence of a new breed of robotic vehicles capable of operating in harsh environments without human supervision. The aim is to find the location of a region that is the source of a given substance of interest (e.g. a chemical pollutant at sea or a gas leakage in air) using a group of cooperative autonomous vehicles. Motivated by fast paced advances in this challenging area, this paper surveys recent advances in searching techniques that are at the core of environmental monitoring strategies using autonomous vehicles.

The neurobiological basis of orientation in insects: insights from the silkmoth mating dance

Counterturning is a common movement pattern during orientation behavior in insects. Once male moths sense sex pheromones and then lose the input, they demonstrate zigzag movements, alternating between left and right turns, to increase the probability to contact with the pheromone plume. We summarize the anatomy and function of the neural circuit involved in pheromone orientation in the silkmoth. A neural circuit, the lateral accessory lobe (LAL), serves a role as the circuit module for zigzag movements and controls this operation using a flip-flop neural switch. Circuit design of the LAL is well conserved across species. We hypothesize that this zigzag module is utilized in a wide range of insect behavior. We introduce two examples of the potential use: orientation flight and the waggle dance in bees.