Many Paths to the Same Goal: Balancing Exploration and Exploitation during Probabilistic Route Planning

Dynamics of odor-source localization: Insights from real-time odor plume recordings and head-motion tracking in freely moving mice

Animals navigating turbulent odor plumes exhibit a rich variety of behaviors, and employ efficient strategies to locate odor sources. A growing body of literature has started to probe this complex task of localizing airborne odor sources in walking mammals to further our understanding of neural encoding and decoding of naturalistic sensory stimuli. However, correlating the intermittent olfactory information with behavior has remained a long-standing challenge due to the stochastic nature of the odor stimulus. We recently reported a method to record real-time olfactory information available to freely moving mice during odor-guided navigation, hence overcoming that challenge. Here we combine our odor-recording method with head-motion tracking to establish correlations between plume encounters and head movements. We show that mice exhibit robust head-pitch motions in the 5-14Hz range during an odor-guided navigation task, and that these head motions are modulated by plume encounters. Furthermore, mice reduce their angles with respect to the source upon plume contact. Head motions may thus be an important part of the sensorimotor behavioral repertoire during naturalistic odor-source localization.

The spiking output of the mouse olfactory bulb encodes large-scale temporal features of natural odor environments

In natural odor environments, odor travels in plumes. Odor concentration dynamics change in characteristic ways across the width and length of a plume. Thus, spatiotemporal dynamics of plumes have informative features for animals navigating to an odor source. Population activity in the olfactory bulb (OB) has been shown to follow odor concentration across plumes to a moderate degree (Lewis et al., 2021). However, it is unknown whether the ability to follow plume dynamics is driven by individual cells or whether it emerges at the population level. Previous research has explored the responses of individual OB cells to isolated features of plumes, but it is difficult to adequately sample the full feature space of plumes as it is still undetermined which features navigating mice employ during olfactory guided search. Here we released odor from an upwind odor source and simultaneously recorded both odor concentration dynamics and cellular response dynamics in awake, head-fixed mice. We found that longer timescale features of odor concentration dynamics were encoded at both the cellular and population level. At the cellular level, responses were elicited at the beginning of the plume for each trial, signaling plume onset. Plumes with high odor concentration elicited responses at the end of the plume, signaling plume offset. Although cellular level tracking of plume dynamics was observed to be weak, we found that at the population level, OB activity distinguished whiffs and blanks (accurately detected odor presence versus absence) throughout the duration of a plume. Even ~20 OB cells were enough to accurately discern odor presence throughout a plume. Our findings indicate that the full range of odor concentration dynamics and high frequency fluctuations are not encoded by OB spiking activity. Instead, relatively lower-frequency temporal features of plumes, such as plume onset, plume offset, whiffs, and blanks, are represented in the OB.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Network models to enhance the translational impact of cross-species studies

Neuroscience studies are often carried out in animal models for the purpose of understanding specific aspects of the human condition. However, the translation of findings across species remains a substantial challenge. Network science approaches can enhance the translational impact of cross-species studies by providing a means of mapping small-scale cellular processes identified in animal model studies to larger-scale inter-regional circuits observed in humans. In this Review, we highlight the contributions of network science approaches to the development of cross-species translational research in neuroscience. We lay the foundation for our discussion by exploring the objectives of cross-species translational models. We then discuss how the development of new tools that enable the acquisition of whole-brain data in animal models with cellular resolution provides unprecedented opportunity for cross-species applications of network science approaches for understanding large-scale brain networks. We describe how these tools may support the translation of findings across species and imaging modalities and highlight future opportunities. Our overarching goal is to illustrate how the application of network science tools across human and animal model studies could deepen insight into the neurobiology that underlies phenomena observed with non-invasive neuroimaging methods and could simultaneously further our ability to translate findings across species.

Learning-Induced Shifts in Mice Navigational Strategies Are Unveiled by a Minimal Behavioral Model of Spatial Exploration

Shifts in spatial patterns produced during the execution of a navigational task can be used to track the effects of the accumulation of knowledge and the acquisition of structured information about the environment. Here, we provide a quantitative analysis of mice behavior while performing a novel goal localization task in a large, modular arena, the HexMaze. To demonstrate the effects of different forms of previous knowledge we first obtain a precise statistical characterization of animals' paths with sub-trial resolution and over different phases of learning. The emergence of a flexible representation of the task is accompanied by a progressive improvement of performance, mediated by multiple, multiplexed time scales. We then use a generative mathematical model of the animal behavior to isolate the specific contributions to the final navigational strategy. We find that animal behavior can be accurately reproduced by the combined effect of a goal-oriented component, becoming stronger with the progression of learning, and of a random walk component, producing choices unrelated to the task and only partially weakened in time.

Balancing exploration and exploitation with information and randomization

Explore-exploit decisions require us to trade off the benefits of exploring unknown options to learn more about them, with exploiting known options, for immediate reward. Such decisions are ubiquitous in nature, but from a computational perspective, they are notoriously hard. There is therefore much interest in how humans and animals make these decisions and recently there has been an explosion of research in this area. Here we provide a biased and incomplete snapshot of this field focusing on the major finding that many organisms use two distinct strategies to solve the explore-exploit dilemma: a bias for information ('directed exploration') and the randomization of choice ('random exploration'). We review evidence for the existence of these strategies, their computational properties, their neural implementations, as well as how directed and random exploration vary over the lifespan. We conclude by highlighting open questions in this field that are ripe to both explore and exploit.

Systems Neuroscience of Natural Behaviors in Rodents

Animals evolved in complex environments, producing a wide range of behaviors, including navigation, foraging, prey capture, and conspecific interactions, which vary over timescales ranging from milliseconds to days. Historically, these behaviors have been the focus of study for ecology and ethology, while systems neuroscience has largely focused on short timescale behaviors that can be repeated thousands of times and occur in highly artificial environments. Thanks to recent advances in machine learning, miniaturization, and computation, it is newly possible to study freely moving animals in more natural conditions while applying systems techniques: performing temporally specific perturbations, modeling behavioral strategies, and recording from large numbers of neurons while animals are freely moving. The authors of this review are a group of scientists with deep appreciation for the common aims of systems neuroscience, ecology, and ethology. We believe it is an extremely exciting time to be a neuroscientist, as we have an opportunity to grow as a field, to embrace interdisciplinary, open, collaborative research to provide new insights and allow researchers to link knowledge across disciplines, species, and scales. Here we discuss the origins of ethology, ecology, and systems neuroscience in the context of our own work and highlight how combining approaches across these fields has provided fresh insights into our research. We hope this review facilitates some of these interactions and alliances and helps us all do even better science, together.

Using Head-Mounted Ethanol Sensors to Monitor Olfactory Information and Determine Behavioral Changes Associated with Ethanol-Plume Contact during Mouse Odor-Guided Navigation

Olfaction guides navigation and decision-making in organisms from multiple animal phyla. Understanding how animals use olfactory cues to guide navigation is a complicated problem for two main reasons. First, the sensory cues used to guide animals to the source of an odor consist of volatile molecules, which form plumes. These plumes are governed by turbulent air currents, resulting in an intermittent and spatiotemporally varying olfactory signal. A second problem is that the technologies for chemical quantification are cumbersome and cannot be used to detect what the freely moving animal senses in real time. Understanding how the olfactory system guides this behavior requires knowing the sensory cues and the accompanying brain signals during navigation. Here, we present a method for real-time monitoring of olfactory information using low-cost, lightweight sensors that robustly detect common solvent molecules, like alcohols, and can be easily mounted on the heads of freely behaving mice engaged in odor-guided navigation. To establish the accuracy and temporal response properties of these sensors we compared their responses with those of a photoionization detector (PID) to precisely controlled ethanol stimuli. Ethanol-sensor recordings, deconvolved using a difference-of-exponentials kernel, showed robust correlations with the PID signal at behaviorally relevant time, frequency, and spatial scales. Additionally, calcium imaging of odor responses from the olfactory bulbs (OBs) of awake, head-fixed mice showed strong correlations with ethanol plume contacts detected by these sensors. Finally, freely behaving mice engaged in odor-guided navigation showed robust behavioral changes such as speed reduction that corresponded to ethanol plume contacts.

Spatial information from the odour environment in mammalian olfaction

The sense of smell is an essential modality for many species, in particular nocturnal and crepuscular mammals, to gather information about their environment. Olfactory cues provide information over a large range of distances, allowing behaviours ranging from simple detection and recognition of objects, to tracking trails and navigating using odour plumes from afar. In this review, we discuss the features of the natural olfactory environment and provide a brief overview of how odour information can be sampled and might be represented and processed by the mammalian olfactory system. Finally, we discuss recent behavioural approaches that address how mammals extract spatial information from the environment in three different contexts: odour trail tracking, odour plume tracking and, more general, olfactory-guided navigation. Recent technological developments have seen the spatiotemporal aspect of mammalian olfaction gain significant attention, and we discuss both the promising aspects of rapidly developing paradigms and stimulus control technologies as well as their limitations. We conclude that, while still in its beginnings, research on the odour environment offers an entry point into understanding the mechanisms how mammals extract information about space.