Which coordinate system for modelling path integration?

Human navigation strategies and their errors result from dynamic interactions of spatial uncertainties

Goal-directed navigation requires continuously integrating uncertain self-motion and landmark cues into an internal sense of location and direction, concurrently planning future paths, and sequentially executing motor actions. Here, we provide a unified account of these processes with a computational model of probabilistic path planning in the framework of optimal feedback control under uncertainty. This model gives rise to diverse human navigational strategies previously believed to be distinct behaviors and predicts quantitatively both the errors and the variability of navigation across numerous experiments. This furthermore explains how sequential egocentric landmark observations form an uncertain allocentric cognitive map, how this internal map is used both in route planning and during execution of movements, and reconciles seemingly contradictory results about cue-integration behavior in navigation. Taken together, the present work provides a parsimonious explanation of how patterns of human goal-directed navigation behavior arise from the continuous and dynamic interactions of spatial uncertainties in perception, cognition, and action.

Neural Networks for Navigation: From Connections to Computations

Many animals can navigate toward a goal they cannot see based on an internal representation of that goal in the brain's spatial maps. These maps are organized around networks with stable fixed-point dynamics (attractors), anchored to landmarks, and reciprocally connected to motor control. This review summarizes recent progress in understanding these networks, focusing on studies in arthropods. One factor driving recent progress is the availability of the Drosophila connectome; however, it is increasingly clear that navigation depends on ongoing synaptic plasticity in these networks. Functional synapses appear to be continually reselected from the set of anatomical potential synapses based on the interaction of Hebbian learning rules, sensory feedback, attractor dynamics, and neuromodulation. This can explain how the brain's maps of space are rapidly updated; it may also explain how the brain can initialize goals as stable fixed points for navigation.

The potential underlying mechanisms during learning flights

Hymenopterans, such as bees and wasps, have long fascinated researchers with their sinuous movements at novel locations. These movements, such as loops, arcs, or zigzags, serve to help insects learn their surroundings at important locations. They also allow the insects to explore and orient themselves in their environment. After they gained experience with their environment, the insects fly along optimized paths guided by several guidance strategies, such as path integration, local homing, and route-following, forming a navigational toolkit. Whereas the experienced insects combine these strategies efficiently, the naive insects need to learn about their surroundings and tune the navigational toolkit. We will see that the structure of the movements performed during the learning flights leverages the robustness of certain strategies within a given scale to tune other strategies which are more efficient at a larger scale. Thus, an insect can explore its environment incrementally without risking not finding back essential locations.

The chicken and egg problem of grid cells and place cells

Place cells and grid cells are major building blocks of the hippocampal cognitive map. The prominent forward model postulates that grid-cell modules are generated by a continuous attractor network; that a velocity signal evoked during locomotion moves entorhinal activity bumps; and that place-cell activity constitutes summation of entorhinal grid-cell modules. Experimental data support the first postulate, but not the latter two. Several families of solutions that depart from these postulates have been put forward. We suggest a modified model (spatial modulation continuous attractor network; SCAN), whereby place cells are generated from spatially selective nongrid cells. Locomotion causes these cells to move the hippocampal activity bump, leading to movement of the entorhinal manifolds. Such inversion accords with the shift of hippocampal thought from navigation to more abstract functions.

Anesthesia disrupts distance, but not direction, of path integration memory

Solitary foraging insects, such as ants, maintain an estimate of the direction and distance to their starting location as they move away from it, in a process known as path integration. This estimate, commonly known as the "home vector," is updated continuously as the ant moves^1-4 and is reset as soon as it enters its nest,⁵ yet ants prevented from returning to their nest can still use their home vector when released several hours later.^6,7 This conjunction of fast update and long persistence of the home vector memory does not directly map to existing accounts of short-, mid-, and long-term memory;^2,8-12 hence, the substrate of this memory remains unknown. Chill-coma anesthesia^13-15 has previously been shown to affect associative memory retention in fruit flies^14,16 and honeybees.^9,17,18 We investigate the nature of path integration memory by anesthetizing ants after they have accumulated home vector information and testing if the memory persists on recovery. We show that after anesthesia the memory of the distance ants have traveled is degraded, but the memory of the direction is retained. We also show that this is consistent with models of path integration that maintain the memory in a redundant Cartesian coordinate system and with the hypothesis that chill-coma produces a proportional reduction of the memory, rather than a subtractive reduction or increase of noise. The observed effect is not compatible with a memory based on recurrent circuit activity and points toward an activity-dependent molecular process as the basis of path integration memory.

A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection

Flexible behaviors over long timescales are thought to engage recurrent neural networks in deep brain regions, which are experimentally challenging to study. In insects, recurrent circuit dynamics in a brain region called the central complex (CX) enable directed locomotion, sleep, and context- and experience-dependent spatial navigation. We describe the first complete electron microscopy-based connectome of the Drosophila CX, including all its neurons and circuits at synaptic resolution. We identified new CX neuron types, novel sensory and motor pathways, and network motifs that likely enable the CX to extract the fly's head direction, maintain it with attractor dynamics, and combine it with other sensorimotor information to perform vector-based navigational computations. We also identified numerous pathways that may facilitate the selection of CX-driven behavioral patterns by context and internal state. The CX connectome provides a comprehensive blueprint necessary for a detailed understanding of network dynamics underlying sleep, flexible navigation, and state-dependent action selection.

The Neuroscience of Spatial Navigation and the Relationship to Artificial Intelligence

Recent advances in artificial intelligence (AI) and neuroscience are impressive. In AI, this includes the development of computer programs that can beat a grandmaster at GO or outperform human radiologists at cancer detection. A great deal of these technological developments are directly related to progress in artificial neural networks-initially inspired by our knowledge about how the brain carries out computation. In parallel, neuroscience has also experienced significant advances in understanding the brain. For example, in the field of spatial navigation, knowledge about the mechanisms and brain regions involved in neural computations of cognitive maps-an internal representation of space-recently received the Nobel Prize in medicine. Much of the recent progress in neuroscience has partly been due to the development of technology used to record from very large populations of neurons in multiple regions of the brain with exquisite temporal and spatial resolution in behaving animals. With the advent of the vast quantities of data that these techniques allow us to collect there has been an increased interest in the intersection between AI and neuroscience, many of these intersections involve using AI as a novel tool to explore and analyze these large data sets. However, given the common initial motivation point-to understand the brain-these disciplines could be more strongly linked. Currently much of this potential synergy is not being realized. We propose that spatial navigation is an excellent area in which these two disciplines can converge to help advance what we know about the brain. In this review, we first summarize progress in the neuroscience of spatial navigation and reinforcement learning. We then turn our attention to discuss how spatial navigation has been modeled using descriptive, mechanistic, and normative approaches and the use of AI in such models. Next, we discuss how AI can advance neuroscience, how neuroscience can advance AI, and the limitations of these approaches. We finally conclude by highlighting promising lines of research in which spatial navigation can be the point of intersection between neuroscience and AI and how this can contribute to the advancement of the understanding of intelligent behavior.

Multimodal interactions in insect navigation

Animals travelling through the world receive input from multiple sensory modalities that could be important for the guidance of their journeys. Given the availability of a rich array of cues, from idiothetic information to input from sky compasses and visual information through to olfactory and other cues (e.g. gustatory, magnetic, anemotactic or thermal) it is no surprise to see multimodality in most aspects of navigation. In this review, we present the current knowledge of multimodal cue use during orientation and navigation in insects. Multimodal cue use is adapted to a species’ sensory ecology and shapes navigation behaviour both during the learning of environmental cues and when performing complex foraging journeys. The simultaneous use of multiple cues is beneficial because it provides redundant navigational information, and in general, multimodality increases robustness, accuracy and overall foraging success. We use examples from sensorimotor behaviours in mosquitoes and flies as well as from large scale navigation in ants, bees and insects that migrate seasonally over large distances, asking at each stage how multiple cues are combined behaviourally and what insects gain from using different modalities.

Principles of Insect Path Integration

Continuously monitoring its position in space relative to a goal is one of the most essential tasks for an animal that moves through its environment. Species as diverse as rats, bees, and crabs achieve this by integrating all changes of direction with the distance covered during their foraging trips, a process called path integration. They generate an estimate of their current position relative to a starting point, enabling a straight-line return, following what is known as a home vector. While in theory path integration always leads the animal precisely back home, in the real world noise limits the usefulness of this strategy when operating in isolation. Noise results from stochastic processes in the nervous system and from unreliable sensory information, particularly when obtaining heading estimates. Path integration, during which angular self-motion provides the sole input for encoding heading (idiothetic path integration), results in accumulating errors that render this strategy useless over long distances. In contrast, when using an external compass this limitation is avoided (allothetic path integration). Many navigating insects indeed rely on external compass cues for estimating body orientation, whereas they obtain distance information by integration of steps or optic-flow-based speed signals. In the insect brain, a region called the central complex plays a key role for path integration. Not only does the central complex house a ring-attractor network that encodes head directions, neurons responding to optic flow also converge with this circuit. A neural substrate for integrating direction and distance into a memorized home vector has therefore been proposed in the central complex. We discuss how behavioral data and the theoretical framework of path integration can be aligned with these neural data.

Analysis of aligning active local searchers orbiting around their common home position

We discuss effects of pairwise aligning interactions in an ensemble of central place foragers or of searchers that are connected to a common home. In a wider sense, we also consider self-moving entities that are attracted to a central place such as, for instance, the zooplankton Daphnia being attracted to a beam of light. Single foragers move with constant speed due to some propulsive mechanism. They explore at random loops the space around and return rhytmically to their home. In the ensemble, the direction of the velocity of a searcher is aligned to the motion of its neighbors. At first, we perform simulations of this ensemble and find a cooperative behavior of the entities. Above an overcritical interaction strength the trajectories of the searcher qualitatively changes and searchers start to move along circles around the home position. Thereby, all searchers rotate either clockwise or anticlockwise around the central home position as it was reported for the zooplankton Daphnia. At second, the computational findings are analytically explained by the formulation of transport equations outgoing from the nonlinear mean field Fokker-Planck equation of the considered situation. In the asymptotic stationary limit, we find expressions for the critical interaction strength, the mean radial and orbital velocities of the searchers and their velocity variances. We also obtain the marginal spatial and angular densities in the undercritical regime where the foragers behave like individuals as well as in the overcritical regime where they rotate collectively around the considered home. We additionally elaborate the overdamped Smoluchowski-limit for the ensemble.

The Central Complex as a Potential Substrate for Vector Based Navigation

Insects use path integration (PI) to maintain a home vector, but can also store and recall vector-memories that take them from home to a food location, and even allow them to take novel shortcuts between food locations. The neural circuit of the Central Complex (a brain area that receives compass and optic flow information) forms a plausible substrate for these behaviors. A recent model, grounded in neurophysiological and neuroanatomical data, can account for PI during outbound exploratory routes and the control of steering to return home. Here, we show that minor, hypothetical but neurally plausible, extensions of this model can additionally explain how insects could store and recall PI vectors to follow food-ward paths, take shortcuts, search at the feeder and re-calibrate their vector-memories with experience. In addition, a simple assumption about how one of multiple vector-memories might be chosen at any point in time can produce the development and maintenance of efficient routes between multiple locations, as observed in bees. The central complex circuitry is therefore well-suited to allow for a rich vector-based navigational repertoire.

The internal maps of insects

Insect navigation is strikingly geometric. Many species use path integration to maintain an accurate estimate of their distance and direction (a vector) to their nest and can store the vector information for multiple salient locations in the world, such as food sources, in a common coordinate system. Insects can also use remembered views of the terrain around salient locations or along travelled routes to guide return, which is a fundamentally geometric process. Recent modelling of these abilities shows convergence on a small set of algorithms and assumptions that appear sufficient to account for a wide range of behavioural data. Notably, this 'base model' does not include any significant topological knowledge: the insect does not need to recover the information (implicit in their vector memory) about the relationships between salient places; nor to maintain any connectedness or ordering information between view memories; nor to form any associations between views and vectors. However, there remains some experimental evidence not fully explained by this base model that may point towards the existence of a more complex or integrated mental map in insects.

A simple mathematical model using centred loops and random perturbations accurately reconstructs search patterns observed in desert ants

This paper describes a new mathematical model that is based on centred loops to reconstruct the "Systematic Search" behaviour of Cataglyphis desert ants. The notable advantage of this model is the combination of simplicity, efficiency and performance. All model input is kept to a minimum, using only parameters that previous research has shown to be available to the animals at all times: distance from the origin, direction of the last step and home vector. Outbound and inbound search paths are being combined into loops that return to the origin, sampling this area more intensely. A stochastic element is added by random perturbations during the next step, mimicking unsystematic errors during the process of path integration and yielding the typical search patterns observed in Cataglyphis desert ants. The model output is compared to runs observed in the field.

Optimal multiguidance integration in insect navigation

In the last decades, desert ants have become model organisms for the study of insect navigation. In finding their way, they use two major navigational routines: path integration using a celestial compass and landmark guidance based on sets of panoramic views of the terrestrial environment. It has been claimed that this information would enable the insect to acquire and use a centralized cognitive map of its foraging terrain. Here, we present a decentralized architecture, in which the concurrently operating path integration and landmark guidance routines contribute optimally to the directions to be steered, with "optimal" meaning maximizing the certainty (reliability) of the combined information. At any one time during its journey, the animal computes a path integration (global) vector and landmark guidance (local) vector, in which the length of each vector is proportional to the certainty of the individual estimates. Hence, these vectors represent the limited knowledge that the navigator has at any one place about the direction of the goal. The sum of the global and local vectors indicates the navigator's optimal directional estimate. Wherever applied, this decentralized model architecture is sufficient to simulate the results of quite a number of diverse cue-conflict experiments, which have recently been performed in various behavioral contexts by different authors in both desert ants and honeybees. They include even those experiments that have deliberately been designed by former authors to strengthen the evidence for a metric cognitive map in bees.

The interaction of path integration and terrestrial visual cues in navigating desert ants: what can we learn from path characteristics?

Ant foragers make use of multiple navigational cues to navigate through the world and the combination of innate navigational strategies and the learning of environmental information is the secret to their navigational success. We present here detailed information about the paths of Cataglyphis fortis desert ants navigating by an innate strategy, namely path integration. Firstly, we observed that the ants' walking speed decreases significantly along their homing paths, such that they slow down just before reaching the goal, and maintain a slower speed during subsequent search paths. Interestingly, this drop in walking speed is independent of absolute home-vector length and depends on the proportion of the home vector that has been completed. Secondly, we found that ants are influenced more strongly by novel or altered visual cues the further along the homing path they are. These results suggest that path integration modulates speed along the homing path in a way that might help ants search for, utilise or learn environmental information at important locations. Ants walk more slowly and sinuously when encountering novel or altered visual cues and occasionally stop and scan the world; this might indicate the re-learning of visual information.

Path Integration: Combining Optic Flow with Compass Orientation

The discovery of translational optic flow detectors in the central complex of a bee has inspired a new model of path integration.

An Anatomically Constrained Model for Path Integration in the Bee Brain

Path integration is a widespread navigational strategy in which directional changes and distance covered are continuously integrated on an outward journey, enabling a straight-line return to home. Bees use vision for this task-a celestial-cue-based visual compass and an optic-flow-based visual odometer-but the underlying neural integration mechanisms are unknown. Using intracellular electrophysiology, we show that polarized-light-based compass neurons and optic-flow-based speed-encoding neurons converge in the central complex of the bee brain, and through block-face electron microscopy, we identify potential integrator cells. Based on plausible output targets for these cells, we propose a complete circuit for path integration and steering in the central complex, with anatomically identified neurons suggested for each processing step. The resulting model circuit is thus fully constrained biologically and provides a functional interpretation for many previously unexplained architectural features of the central complex. Moreover, we show that the receptive fields of the newly discovered speed neurons can support path integration for the holonomic motion (i.e., a ground velocity that is not precisely aligned with body orientation) typical of bee flight, a feature not captured in any previously proposed model of path integration. In a broader context, the model circuit presented provides a general mechanism for producing steering signals by comparing current and desired headings-suggesting a more basic function for central complex connectivity, from which path integration may have evolved.

A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

Despite their small size, insect brains are able to produce robust and efficient navigation in complex environments. Specifically in social insects, such as ants and bees, these navigational capabilities are guided by orientation directing vectors generated by a process called path integration. During this process, they integrate compass and odometric cues to estimate their current location as a vector, called the home vector for guiding them back home on a straight path. They further acquire and retrieve path integration-based vector memories globally to the nest or based on visual landmarks. Although existing computational models reproduced similar behaviors, a neurocomputational model of vector navigation including the acquisition of vector representations has not been described before. Here we present a model of neural mechanisms in a modular closed-loop control—enabling vector navigation in artificial agents. The model consists of a path integration mechanism, reward-modulated global learning, random search, and action selection. The path integration mechanism integrates compass and odometric cues to compute a vectorial representation of the agent's current location as neural activity patterns in circular arrays. A reward-modulated learning rule enables the acquisition of vector memories by associating the local food reward with the path integration state. A motor output is computed based on the combination of vector memories and random exploration. In simulation, we show that the neural mechanisms enable robust homing and localization, even in the presence of external sensory noise. The proposed learning rules lead to goal-directed navigation and route formation performed under realistic conditions. Consequently, we provide a novel approach for vector learning and navigation in a simulated, situated agent linking behavioral observations to their possible underlying neural substrates.

Visual Navigation in Nocturnal Insects

Despite their tiny eyes and brains, nocturnal insects have evolved a remarkable capacity to visually navigate at night. Whereas some use moonlight or the stars as celestial compass cues to maintain a straight-line course, others use visual landmarks to navigate to and from their nest. These impressive abilities rely on highly sensitive compound eyes and specialized visual processing strategies in the brain.

Grid-cell representations in mental simulation

Anticipating the future is a key motif of the brain, possibly supported by mental simulation of upcoming events. Rodent single-cell recordings suggest the ability of spatially tuned cells to represent subsequent locations. Grid-like representations have been observed in the human entorhinal cortex during virtual and imagined navigation. However, hitherto it remains unknown if grid-like representations contribute to mental simulation in the absence of imagined movement. Participants imagined directions between building locations in a large-scale virtual-reality city while undergoing fMRI without re-exposure to the environment. Using multi-voxel pattern analysis, we provide evidence for representations of absolute imagined direction at a resolution of 30° in the parahippocampal gyrus, consistent with the head-direction system. Furthermore, we capitalize on the six-fold rotational symmetry of grid-cell firing to demonstrate a 60° periodic pattern-similarity structure in the entorhinal cortex. Our findings imply a role of the entorhinal grid-system in mental simulation and future thinking beyond spatial navigation.

DOI:http://dx.doi.org/10.7554/eLife.17089.001

Recordings of brain activity in moving rats have found neurons that fire when the rat is at specific locations. These neurons are known as grid cells because their activity produces a grid-like pattern. A separate group of neurons, called head direction cells, represents the rat’s facing direction. Functional magnetic resonance imaging (fMRI) studies that have tracked brain activity in humans as they navigate virtual environments have found similar grid-like and direction-related responses. A recent study showed grid-like responses even if the people being studied just imagined moving around an arena while lying still. Theoretical work suggests that spatially tuned cells might generally be important for our ability to imagine and simulate future events. However, it is not clear whether these location- and direction-responsive cells are active when people do not visualize themselves moving.

Bellmund et al. used fMRI to track brain activity in volunteers as they imagined different views in a virtual reality city. Before the fMRI experiment, the volunteers completed extensive training where they learned the layout of the city and the names of its buildings. Then, during the fMRI experiment, the volunteers had to imagine themselves standing in front of certain buildings and facing different directions. Crucially, they did not imagine themselves moving between these buildings.

By using representational similarity analysis, which compares patterns of brain activity, Bellmund et al. could distinguish between the directions the volunteers were imagining. Activity patterns in the parahippocampal gyrus (a brain region known to be important for navigation) were more similar when participants were imagining similar directions.

The fMRI results also show grid-like responses in a brain area called entorhinal cortex, which is known to contain grid cells. While participants were imagining, this region exhibited activity patterns with a six-fold symmetry, as Bellmund et al. predicted from the characteristic firing patterns of grid cells.

The findings presented by Bellmund et al. provide evidence that suggests that grid cells are involved in planning how to navigate, and so support previous theoretical assumptions. The computations of these cells might contribute to other kinds of thinking too, such as remembering the past or imagining future events.

DOI:http://dx.doi.org/10.7554/eLife.17089.002

Optimal cue integration in ants

In situations with redundant or competing sensory information, humans have been shown to perform cue integration, weighting different cues according to their certainty in a quantifiably optimal manner. Ants have been shown to merge the directional information available from their path integration (PI) and visual memory, but as yet it is not clear that they do so in a way that reflects the relative certainty of the cues. In this study, we manipulate the variance of the PI home vector by allowing ants (Cataglyphis velox) to run different distances and testing their directional choice when the PI vector direction is put in competition with visual memory. Ants show progressively stronger weighting of their PI direction as PI length increases. The weighting is quantitatively predicted by modelling the expected directional variance of home vectors of different lengths and assuming optimal cue integration. However, a subsequent experiment suggests ants may not actually compute an internal estimate of the PI certainty, but are using the PI home vector length as a proxy.

Early ant trajectories: spatial behaviour before behaviourism

In the beginning of the twentieth century, when Jacques Loeb's and John Watson's mechanistic view of life started to dominate animal physiology and behavioural biology, several scientists with different academic backgrounds got engaged in studying the wayfinding behaviour of ants. Largely unaffected by the scientific spirit of the time, they worked independently of each other in different countries: in Algeria, Tunisia, Spain, Switzerland and the United States of America. In the current literature on spatial cognition these early ant researchers--Victor Cornetz, Felix Santschi, Charles Turner and Rudolf Brun--are barely mentioned. Moreover, it is virtually unknown that the great neuroanatomist Santiago Ramón y Cajal had also worked on spatial orientation in ants. This general neglect is certainly due to the fact that nearly all these ant researchers were scientific loners, who did their idiosyncratic investigations outside the realm of comparative physiology, neurobiology and the behavioural sciences of the time, and published their results in French, German, and Spanish at rather inaccessible places. Even though one might argue that much of their work resulted in mainly anecdotal evidence, the conceptual approaches of these early ant researchers preempt much of the present-day discussions on spatial representation in animals.

Neural mechanisms of self-location

The ability to self-localise and to navigate to remembered goals in complex and changeable environments is crucial to the survival of many mobile species. Electrophysiological investigations of the mammalian hippocampus and associated brain structures have identified several classes of neurons which represent information about an organism's position and orientation. These include place cells, grid cells, head direction cells, and boundary vector cells, as well as cells representing aspects of self-motion. Understanding how these neural representations are formed and updated from environmental sensory information and from information relating to self-motion is an important topic attracting considerable current interest. Here we review the computational mechanisms thought to underlie the formation of these different spatial representations, the interactions between them, and their use in guiding behaviour. These include some of the clearest examples of computational mechanisms of general interest to neuroscience, such as attractor dynamics, temporal coding and multi-modal integration. We also discuss the close relationships between computational modelling and experimental research which are driving progress in this area.

How cumulative error in grid cell firing is literally bounded by the environment

In this issue of Neuron, Hardcastle et al. (2015) show that the spatial firing patterns of grid cells accumulate error, drifting coherently, until reset by encounters with environmental boundaries. These results reveal important aspects of the neural dynamics of self-localization from self-motion and environmental information.

Animal path integration: a model of positional uncertainty along tortuous paths

Exact closed form mathematical solutions are reported which quantify the dynamic uncertainty resulting from path integration (PI) along tortuous paths. Based on a correlated random walk model, the derived results quantify positional estimation error moments with and without a compass, in discrete and continuous time. Consistent with earlier studies on attempted straight-line navigation, using a compass significantly reduces the uncertainty during PI, making purely idiothetic PI biologically implausible except over short distances. Examples are used to illustrate the contributions of angular noise, linear noise and path tortuosity, under different conditions. Linear noise is shown to be relatively more important with a compass while angular noise is more important without. It is shown that increasing path tortuosity decreases positional uncertainty, true for long and short journeys, irrespective of whether a compass is used, or the level of noise. In contrast, reducing angular noise also reduces uncertainty, but only below some critical level of noise. Using canonical equations of PI, it is shown that polar PI using a compass accumulates uncertainty in a manner similar to Cartesian PI without a compass. Issues of data sampling bias and intermittent use of a compass are also considered for PI along tortuous paths.

Ant navigation: fractional use of the home vector

Home is a special location for many animals, offering shelter from the elements, protection from predation, and a common place for gathering of the same species. Not surprisingly, many species have evolved efficient, robust homing strategies, which are used as part of each and every foraging journey. A basic strategy used by most animals is to take the shortest possible route home by accruing the net distances and directions travelled during foraging, a strategy well known as path integration. This strategy is part of the navigation toolbox of ants occupying different landscapes. However, when there is a visual discrepancy between test and training conditions, the distance travelled by animals relying on the path integrator varies dramatically between species: from 90% of the home vector to an absolute distance of only 50 cm. We here ask what the theoretically optimal balance between PI-driven and landmark-driven navigation should be. In combination with well-established results from optimal search theory, we show analytically that this fractional use of the home vector is an optimal homing strategy under a variety of circumstances. Assuming there is a familiar route that an ant recognizes, theoretically optimal search should always begin at some fraction of the home vector, depending on the region of familiarity. These results are shown to be largely independent of the search algorithm used. Ant species from different habitats appear to have optimized their navigation strategy based on the availability and nature of navigational information content in their environment.

Maintaining a cognitive map in darkness: the need to fuse boundary knowledge with path integration

Spatial navigation requires the processing of complex, disparate and often ambiguous sensory data. The neurocomputations underpinning this vital ability remain poorly understood. Controversy remains as to whether multimodal sensory information must be combined into a unified representation, consistent with Tolman's “cognitive map”, or whether differential activation of independent navigation modules suffice to explain observed navigation behaviour. Here we demonstrate that key neural correlates of spatial navigation in darkness cannot be explained if the path integration system acted independently of boundary (landmark) information. In vivo recordings demonstrate that the rodent head direction (HD) system becomes unstable within three minutes without vision. In contrast, rodents maintain stable place fields and grid fields for over half an hour without vision. Using a simple HD error model, we show analytically that idiothetic path integration (iPI) alone cannot be used to maintain any stable place representation beyond two to three minutes. We then use a measure of place stability based on information theoretic principles to prove that featureless boundaries alone cannot be used to improve localization above chance level. Having shown that neither iPI nor boundaries alone are sufficient, we then address the question of whether their combination is sufficient and – we conjecture – necessary to maintain place stability for prolonged periods without vision. We addressed this question in simulations and robot experiments using a navigation model comprising of a particle filter and boundary map. The model replicates published experimental results on place field and grid field stability without vision, and makes testable predictions including place field splitting and grid field rescaling if the true arena geometry differs from the acquired boundary map. We discuss our findings in light of current theories of animal navigation and neuronal computation, and elaborate on their implications and significance for the design, analysis and interpretation of experiments.

Do animals need “cognitive maps“? One of the main difficulties in answering this question is finding a definitive scenario where having and not having a “cognitive map“ result in measurably different outcomes. Many key predictions made by models involving some sort of “cognitive map“ can also be replicated by models without a “cognitive map“. Here we consider published data on rodents navigating in darkness inside homogeneous arenas. The head direction system becomes unstable within three minutes in darkness, yet place and grid cells have been reported to fire in the same locations for thirty minutes or longer. We show firstly that it is theoretically implausible for path integration alone to maintain a stable positional representation beyond three minutes, given a drifting head direction system in darkness. Secondly, we prove that even assuming perfect boundary knowledge is insufficient to maintain a stable positional representation. Finally, we show in simulated and real arenas that a nearoptimal combination of path integration and boundary representation is sufficient to produce stable positional representations in darkness consistent with published data. The necessity for fusing path integration and landmark information for accurate localization in darkness is both consistent with, and motivates the existence of, “cognitive maps.“

The neuroscience of learning: beyond the Hebbian synapse

From the traditional perspective of associative learning theory, the hypothesis linking modifications of synaptic transmission to learning and memory is plausible. It is less so from an information-processing perspective, in which learning is mediated by computations that make implicit commitments to physical and mathematical principles governing the domains where domain-specific cognitive mechanisms operate. We compare the properties of associative learning and memory to the properties of long-term potentiation, concluding that the properties of the latter do not explain the fundamental properties of the former. We briefly review the neuroscience of reinforcement learning, emphasizing the representational implications of the neuroscientific findings. We then review more extensively findings that confirm the existence of complex computations in three information-processing domains: probabilistic inference, the representation of uncertainty, and the representation of space. We argue for a change in the conceptual framework within which neuroscientists approach the study of learning mechanisms in the brain.

Path integration mediated systematic search: a Bayesian model

The systematic search behaviour is a backup system that increases the chances of desert ants finding their nest entrance after foraging when the path integrator has failed to guide them home accurately enough. Here we present a mathematical model of the systematic search that is based on extensive behavioural studies in North African desert ants Cataglyphis fortis. First, a simple search heuristic utilising Bayesian inference and a probability density function is developed. This model, which optimises the short-term nest detection probability, is then compared to three simpler search heuristics and to recorded search patterns of Cataglyphis ants. To compare the different searches a method to quantify search efficiency is established as well as an estimate of the error rate in the ants' path integrator. We demonstrate that the Bayesian search heuristic is able to automatically adapt to increasing levels of positional uncertainty to produce broader search patterns, just as desert ants do, and that it outperforms the three other search heuristics tested. The searches produced by it are also arguably the most similar in appearance to the ant's searches.

No need for a cognitive map: decentralized memory for insect navigation

In many animals the ability to navigate over long distances is an important prerequisite for foraging. For example, it is widely accepted that desert ants and honey bees, but also mammals, use path integration for finding the way back to their home site. It is however a matter of a long standing debate whether animals in addition are able to acquire and use so called cognitive maps. Such a ‘map’, a global spatial representation of the foraging area, is generally assumed to allow the animal to find shortcuts between two sites although the direct connection has never been travelled before. Using the artificial neural network approach, here we develop an artificial memory system which is based on path integration and various landmark guidance mechanisms (a bank of individual and independent landmark-defined memory elements). Activation of the individual memory elements depends on a separate motivation network and an, in part, asymmetrical lateral inhibition network. The information concerning the absolute position of the agent is present, but resides in a separate memory that can only be used by the path integration subsystem to control the behaviour, but cannot be used for computational purposes with other memory elements of the system. Thus, in this simulation there is no neural basis of a cognitive map. Nevertheless, an agent controlled by this network is able to accomplish various navigational tasks known from ants and bees and often discussed as being dependent on a cognitive map. For example, map-like behaviour as observed in honey bees arises as an emergent property from a decentralized system. This behaviour thus can be explained without referring to the assumption that a cognitive map, a coherent representation of foraging space, must exist. We hypothesize that the proposed network essentially resides in the mushroom bodies of the insect brain.

When desert ants search for food, they often have to travel over long distances, more then ten thousand times their body lengths and then turn back to find the nest entrance. It is known from many experiments that these animals employ a skylight compass including the sun, a pedometer, and a mechanism called path integration. This means that during walking they continuously update the vector pointing from their actual position back to the nest site. In addition they use landmarks. However, based on observations of the behaviour of ants and honey bees several authors have argued that these animals finally employ a neural system that is able to represent frequently visited locations in the form of a map (a “cognitive map”). Having a map-like system available would allow the animal to find a shortcut between two separately learned locations without having learned this direct path between both locations beforehand. As such shortcuts have been observed, cognitive maps have been assumed to exist. Here we show in a simulation study based on artificial neural networks that shortcuts as observed in the experiments are also possible with a memory system using a completely decentralized architecture not including an explicit cognitive map.

Finding the way with a noisy brain

Successful navigation is fundamental to the survival of nearly every animal on earth, and achieved by nervous systems of vastly different sizes and characteristics. Yet surprisingly little is known of the detailed neural circuitry from any species which can accurately represent space for navigation. Path integration is one of the oldest and most ubiquitous navigation strategies in the animal kingdom. Despite a plethora of computational models, from equational to neural network form, there is currently no consensus, even in principle, of how this important phenomenon occurs neurally. Recently, all path integration models were examined according to a novel, unifying classification system. Here we combine this theoretical framework with recent insights from directed walk theory, and develop an intuitive yet mathematically rigorous proof that only one class of neural representation of space can tolerate noise during path integration. This result suggests many existing models of path integration are not biologically plausible due to their intolerance to noise. This surprising result imposes significant computational limitations on the neurobiological spatial representation of all successfully navigating animals, irrespective of species. Indeed, noise-tolerance may be an important functional constraint on the evolution of neuroarchitectural plans in the animal kingdom.

The ability to navigate allows animals to vastly increase the action space for finding resources, mates, and to avoid predators. The benefits are many and it is commonly believed that modern brain functions have emerged from ancestral forms evolved for effective navigation. Since the time of Charles Darwin, it has been recognized that path integration is a navigation strategy innate to many species. Path integration involves adding the stepwise displacements during a circuitous journey to compute a net homeward direction. Over the past century, this phenomenon has been described for birds to mammals to arthropods, and a long list of mathematical, algorithmic, and neural network models have been proposed to explain the necessary computations. This work shows how the different types of models behave in the presence of noise. It turns out that only one class of models can function properly in the presence of noise. Since noise appears to be present at all levels of brain physiology, we arrive at the surprising conclusion that the general computational principles for path integration must be the same across all species. Two subtypes of path integration models share the same critical computational principles, and are compared to known neuroanatomy and physiology.