The hippocampo-cortical loop: spatio-temporal learning and goal-oriented planning in navigation

Elevated Gamma Connectivity in Nidopallium Caudolaterale of Pigeons during Spatial Path Adjustment

Simple Summary

Imagine that you need to reach a designated destination, but the familiar path you most often choose suddenly becomes impassable. Then, what will you do? Of course, you will try to adjust the path according to your cognition of the current environment and the goal. During this, how will be the spatial environment, especially the path adjustment process, be represented in your brain? That is a very interesting research topic. In this study, we attempted to explore the internal neural patterns within the brain, especially within the higher-order cognitive brain areas, by taking pigeons, a species with excellent navigation ability, as a model animal. The most classical detour paradigm was used to train pigeons in a task of spatial path adjustment, and the neural signals in pigeons’ nidopallium caudolaterale ((NCL) functionally similar to mammalian “prefrontal cortex”) were recorded. We found that the spatial path adjustment process is accompanied by modifications of the changes in spectral and connectivity properties of the neural activities in the NCL. The elevated gamma connectivity in the NCL found in this study supports the role of the NCL in spatial cognition and contributes to explaining the potential neural mechanism of path adjustment.

Abstract

Previous studies showed that spatial navigation depends on a local network including multiple brain regions with strong interactions. However, it is still not fully understood whether and how the neural patterns in avian nidopallium caudolaterale (NCL), which is suggested to play a key role in navigation as a higher cognitive structure, are modulated by the behaviors during spatial navigation, especially involved path adjustment needs. Hence, we examined neural activity in the NCL of pigeons and explored the local field potentials’ (LFPs) spectral and functional connectivity patterns in a goal-directed spatial cognitive task with the detour paradigm. We found the pigeons progressively learned to solve the path adjustment task when the learned path was blocked suddenly. Importantly, the behavioral changes during the adjustment were accompanied by the modifications in neural patterns in the NCL. Specifically, the spectral power in lower bands (1–4 Hz and 5–12 Hz) decreased as the pigeons were tested during the adjustment. Meanwhile, an elevated gamma (31–45 Hz and 55–80 Hz) connectivity in the NCL was also detected. These results and the partial least square discriminant analysis (PLS-DA) modeling analysis provide insights into the neural activities in the avian NCL during the spatial path adjustment, contributing to understanding the potential mechanism of avian spatial encoding. This study suggests the important role of the NCL in spatial learning, especially path adjustment in avian navigation.

Time as the fourth dimension in the hippocampus

Experiences of animal and human beings are structured by the continuity of space and time coupled with the uni-directionality of time. In addition to its pivotal position in spatial processing and navigation, the hippocampal system also plays a central, multiform role in several types of temporal processing. These include timing and sequence learning, at scales ranging from meso-scales of seconds to macro-scales of minutes, hours, days and beyond, encompassing the classical functions of short term memory, working memory, long term memory, and episodic memories (comprised of information about when, what, and where). This review article highlights the principal findings and behavioral contexts of experiments in rats showing: 1) timing: tracking time during delays by hippocampal 'time cells' and during free behavior by hippocampal-afferent lateral entorhinal cortex ramping cells; 2) 'online' sequence processing: activity coding sequences of events during active behavior; 3) 'off-line' sequence replay: during quiescence or sleep, orderly reactivation of neuronal assemblies coding awake sequences. Studies in humans show neurophysiological correlates of episodic memory comparable to awake replay. Neural mechanisms are discussed, including ion channel properties, plateau and ramping potentials, oscillations of excitation and inhibition of population activity, bursts of high amplitude discharges (sharp wave ripples), as well as short and long term synaptic modifications among and within cell assemblies. Specifically conceived neural network models will suggest processes supporting the emergence of scalar properties (Weber's law), and include different classes of feedforward and recurrent network models, with intrinsic hippocampal coding for 'transitions' (sequencing of events or places).

The Role of Hp-NCL Network in Goal-Directed Routing Information Encoding of Bird: A Review

Goal-directed navigation is a crucial behavior for the survival of animals, especially for the birds having extraordinary spatial navigation ability. In the studies of the neural mechanism of the goal-directed behavior, especially involving the information encoding mechanism of the route, the hippocampus (Hp) and nidopallium caudalle (NCL) of the avian brain are the famous regions that play important roles. Therefore, they have been widely concerned and a series of studies surrounding them have increased our understandings of the navigation mechanism of birds in recent years. In this paper, we focus on the studies of the information encoding mechanism of the route in the avian goal-directed behavior. We first summarize and introduce the related studies on the role of the Hp and NCL for goal-directed behavior comprehensively. Furthermore, we review the related cooperative interaction studies about the Hp-NCL local network and other relevant brain regions supporting the goal-directed routing information encoding. Finally, we summarize the current situation and prospect the existing important questions in this field. We hope this paper can spark fresh thinking for the following research on routing information encoding mechanism of birds.

Merging information in the entorhinal cortex: what can we learn from robotics experiments and modeling?

Place recognition is a complex process involving idiothetic and allothetic information. In mammals, evidence suggests that visual information stemming from the temporal and parietal cortical areas (‘what’ and ‘where’ information) is merged at the level of the entorhinal cortex (EC) to build a compact code of a place. Local views extracted from specific feature points can provide information important for view cells (in primates) and place cells (in rodents) even when the environment changes dramatically. Robotics experiments using conjunctive cells merging ‘what’ and ‘where’ information related to different local views show their important role for obtaining place cells with strong generalization capabilities. This convergence of information may also explain the formation of grid cells in the medial EC if we suppose that: (1) path integration information is computed outside the EC, (2) this information is compressed at the level of the EC owing to projection (which follows a modulo principle) of cortical activities associated with discretized vector fields representing angles and/or path integration, and (3) conjunctive cells merge the projections of different modalities to build grid cell activities. Applying modulo projection to visual information allows an interesting compression of information and could explain more recent results on grid cells related to visual exploration. In conclusion, the EC could be dedicated to the build-up of a robust yet compact code of cortical activity whereas the hippocampus proper recognizes these complex codes and learns to predict the transition from one state to another.

Hippocampus-Dependent Goal Localization by Head-Fixed Mice in Virtual Reality

The demonstration of the ability of rodents to navigate in virtual reality (VR) has made it an important behavioral paradigm for studying spatially modulated neuronal activity in these animals. However, their behavior in such simulated environments remains poorly understood. Here, we show that encoding and retrieval of goal location memory in mice head-fixed in VR depends on the postsynaptic scaffolding protein Shank2 and the dorsal hippocampus. In our newly developed virtual cued goal location task, a head-fixed mouse moves from one end of a virtual linear track to seek rewards given at a target location along the track. The mouse needs to visually recognize the target location and stay there for a short period of time to receive the reward. Transient pharmacological blockade of fast glutamatergic synaptic transmission in the dorsal hippocampus dramatically and reversibly impaired performance of this task. Encoding and updating of virtual cued goal location memory was impaired in mice deficient in the postsynaptic scaffolding protein Shank2, a mouse model of autism that exhibits impaired spatial learning in a real environment. These results highlight the crucial roles of the dorsal hippocampus and postsynaptic protein complexes in spatial learning and navigation in VR.

Iterative free-energy optimization for recurrent neural networks (INFERNO)

The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes' synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic) evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle.

Scale-Free Navigational Planning by Neuronal Traveling Waves

Spatial navigation and planning is assumed to involve a cognitive map for evaluating trajectories towards a goal. How such a map is realized in neuronal terms, however, remains elusive. Here we describe a simple and noise-robust neuronal implementation of a path finding algorithm in complex environments. We consider a neuronal map of the environment that supports a traveling wave spreading out from the goal location opposite to direction of the physical movement. At each position of the map, the smallest firing phase between adjacent neurons indicate the shortest direction towards the goal. In contrast to diffusion or single-wave-fronts, local phase differences build up in time at arbitrary distances from the goal, providing a minimal and robust directional information throughout the map. The time needed to reach the steady state represents an estimate of an agent’s waiting time before it heads off to the goal. Given typical waiting times we estimate the minimal number of neurons involved in the cognitive map. In the context of the planning model, forward and backward spread of neuronal activity, oscillatory waves, and phase precession get a functional interpretation, allowing for speculations about the biological counterpart.

Solving the detour problem in navigation: a model of prefrontal and hippocampal interactions

Adapting behavior to accommodate changes in the environment is an important function of the nervous system. A universal problem for motile animals is the discovery that a learned route is blocked and a detour is required. Given the substantial neuroscience research on spatial navigation and decision-making it is surprising that so little is known about how the brain solves the detour problem. Here we review the limited number of relevant functional neuroimaging, single unit recording and lesion studies. We find that while the prefrontal cortex (PFC) consistently responds to detours, the hippocampus does not. Recent evidence suggests the hippocampus tracks information about the future path distance to the goal. Based on this evidence we postulate a conceptual model in which: Lateral PFC provides a prediction error signal about the change in the path, frontopolar and superior PFC support the re-formulation of the route plan as a novel subgoal and the hippocampus simulates the new path. More data will be required to validate this model and understand (1) how the system processes the different options; and (2) deals with situations where a new path becomes available (i.e., shortcuts).

Comparison of interval timing behaviour in mice following dorsal or ventral hippocampal lesions with mice having δ-opioid receptor gene deletion

Mice with cytotoxic lesions of the dorsal hippocampus (DH) underestimated 15 s and 45 s target durations in a bi-peak procedure as evidenced by proportional leftward shifts of the peak functions that emerged during training as a result of decreases in both 'start' and 'stop' times. In contrast, mice with lesions of the ventral hippocampus (VH) displayed rightward shifts that were immediately present and were largely limited to increases in the 'stop' time for the 45 s target duration. Moreover, the effects of the DH lesions were congruent with the scalar property of interval timing in that the 15 s and 45 s functions superimposed when plotted on a relative timescale, whereas the effects of the VH lesions violated the scalar property. Mice with DH lesions also showed enhanced reversal learning in comparison to control and VH lesioned mice. These results are compared with the timing distortions observed in mice lacking δ-opioid receptors (Oprd1(-/-)) which were similar to mice with DH lesions. Taken together, these results suggest a balance between hippocampal-striatal interactions for interval timing and demonstrate possible functional dissociations along the septotemporal axis of the hippocampus in terms of motivation, timed response thresholds and encoding in temporal memory.

From self-assessment to frustration, a small step toward autonomy in robotic navigation

Autonomy and self-improvement capabilities are still challenging in the fields of robotics and machine learning. Allowing a robot to autonomously navigate in wide and unknown environments not only requires a repertoire of robust strategies to cope with miscellaneous situations, but also needs mechanisms of self-assessment for guiding learning and for monitoring strategies. Monitoring strategies requires feedbacks on the behavior's quality, from a given fitness system in order to take correct decisions. In this work, we focus on how a second-order controller can be used to (1) manage behaviors according to the situation and (2) seek for human interactions to improve skills. Following an incremental and constructivist approach, we present a generic neural architecture, based on an on-line novelty detection algorithm that may be able to self-evaluate any sensory-motor strategies. This architecture learns contingencies between sensations and actions, giving the expected sensation from the previous perception. Prediction error, coming from surprising events, provides a measure of the quality of the underlying sensory-motor contingencies. We show how a simple second-order controller (emotional system) based on the prediction progress allows the system to regulate its behavior to solve complex navigation tasks and also succeeds in asking for help if it detects dead-lock situations. We propose that this model could be a key structure toward self-assessment and autonomy. We made several experiments that can account for such properties for two different strategies (road following and place cells based navigation) in different situations.