Grid cell co-activity patterns during sleep reflect spatial overlap of grid fields during active behaviors

Grid Cells in Cognition: Mechanisms and Function

The activity patterns of grid cells form distinctively regular triangular lattices over the explored spatial environment and are largely invariant to visual stimuli, animal movement, and environment geometry. These neurons present numerous fascinating challenges to the curious (neuro)scientist: What are the circuit mechanisms responsible for creating spatially periodic activity patterns from the monotonic input-output responses of single neurons? How and why does the brain encode a local, nonperiodic variable-the allocentric position of the animal-with a periodic, nonlocal code? And, are grid cells truly specialized for spatial computations? Otherwise, what is their role in general cognition more broadly? We review efforts in uncovering the mechanisms and functional properties of grid cells, highlighting recent progress in the experimental validation of mechanistic grid cell models, and discuss the coding properties and functional advantages of the grid code as suggested by continuous attractor network models of grid cells.

Robust variability of grid cell properties within individual grid modules enhances encoding of local space

Although grid cells are one of the most well studied functional classes of neurons in the mammalian brain, the assumption that there is a single grid orientation and spacing per grid module has not been carefully tested. We investigate and analyze a recent large-scale recording of medial entorhinal cortex to characterize the presence and degree of heterogeneity of grid properties within individual modules. We find evidence for small, but robust, variability and hypothesize that this property of the grid code could enhance the ability of encoding local spatial information. Performing analysis on synthetic populations of grid cells, where we have complete control over the amount heterogeneity in grid properties, we demonstrate that variability, of a similar magnitude to the analyzed data, leads to significantly decreased decoding error, even when restricted to activity from a single module. Our results highlight how the heterogeneity of the neural response properties may benefit coding and opens new directions for theoretical and experimental analysis of grid cells.

Mental navigation in the primate entorhinal cortex

A cognitive map is a suitably structured representation that enables novel computations using previous experience; for example, planning a new route in a familiar space1. Work in mammals has found direct evidence for such representations in the presence of exogenous sensory inputs in both spatial2,3 and non-spatial domains4-10. Here we tested a foundational postulate of the original cognitive map theory1,11: that cognitive maps support endogenous computations without external input. We recorded from the entorhinal cortex of monkeys in a mental navigation task that required the monkeys to use a joystick to produce one-dimensional vectors between pairs of visual landmarks without seeing the intermediate landmarks. The ability of the monkeys to perform the task and generalize to new pairs indicated that they relied on a structured representation of the landmarks. Task-modulated neurons exhibited periodicity and ramping that matched the temporal structure of the landmarks and showed signatures of continuous attractor networks12,13. A continuous attractor network model of path integration14 augmented with a Hebbian-like learning mechanism provided an explanation of how the system could endogenously recall landmarks. The model also made an unexpected prediction that endogenous landmarks transiently slow path integration, reset the dynamics and thereby reduce variability. This prediction was borne out in a reanalysis of firing rate variability and behaviour. Our findings link the structured patterns of activity in the entorhinal cortex to the endogenous recruitment of a cognitive map during mental navigation.

Remapping revisited: how the hippocampus represents different spaces

The representation of distinct spaces by hippocampal place cells has been linked to changes in their place fields (the locations in the environment where the place cells discharge strongly), a phenomenon that has been termed 'remapping'. Remapping has been assumed to be accompanied by the reorganization of subsecond cofiring relationships among the place cells, potentially maximizing hippocampal information coding capacity. However, several observations challenge this standard view. For example, place cells exhibit mixed selectivity, encode non-positional variables, can have multiple place fields and exhibit unreliable discharge in fixed environments. Furthermore, recent evidence suggests that, when measured at subsecond timescales, the moment-to-moment cofiring of a pair of cells in one environment is remarkably similar in another environment, despite remapping. Here, I propose that remapping is a misnomer for the changes in place fields across environments and suggest instead that internally organized manifold representations of hippocampal activity are actively registered to different environments to enable navigation, promote memory and organize knowledge.

Visual boundary cues suffice to anchor place and grid cells in virtual reality

The hippocampal formation contains neurons responsive to an animal's current location and orientation, which together provide the organism with a neural map of space.1,2,3 Spatially tuned neurons rely on external landmark cues and internally generated movement information to estimate position.4,5 An important class of landmark cue are the boundaries delimiting an environment, which can define place cell field position6,7 and stabilize grid cell firing.8 However, the precise nature of the sensory information used to detect boundaries remains unknown. We used 2-dimensional virtual reality (VR)9 to show that visual cues from elevated walls surrounding the environment are both sufficient and necessary to stabilize place and grid cell responses in VR, when only visual and self-motion cues are available. By contrast, flat boundaries formed by the edges of a textured floor did not stabilize place and grid cells, indicating only specific forms of visual boundary stabilize hippocampal spatial firing. Unstable grid cells retain internally coherent, hexagonally arranged firing fields, but these fields "drift" with respect to the virtual environment over periods >5 s. Optic flow from a virtual floor does not slow drift dynamics, emphasizing the importance of boundary-related visual information. Surprisingly, place fields are more stable close to boundaries even with floor and wall cues removed, suggesting invisible boundaries are inferred using the motion of a discrete, separate cue (a beacon signaling reward location). Subsets of place cells show allocentric directional tuning toward the beacon, with strength of tuning correlating with place field stability when boundaries are removed.

A consistent map in the medial entorhinal cortex supports spatial memory

The medial entorhinal cortex (MEC) is hypothesized to function as a cognitive map for memory-guided navigation. How this map develops during learning and influences memory remains unclear. By imaging MEC calcium dynamics while mice successfully learned a novel virtual environment over ten days, we discovered that the dynamics gradually became more spatially consistent and then stabilized. Additionally, grid cells in the MEC not only exhibited improved spatial tuning consistency, but also maintained stable phase relationships, suggesting a network mechanism involving synaptic plasticity and rigid recurrent connectivity to shape grid cell activity during learning. Increased c-Fos expression in the MEC in novel environments further supports the induction of synaptic plasticity. Unsuccessful learning lacked these activity features, indicating that a consistent map is specific for effective spatial memory. Finally, optogenetically disrupting spatial consistency of the map impaired memory-guided navigation in a well-learned environment. Thus, we demonstrate that the establishment of a spatially consistent MEC map across learning both correlates with, and is necessary for, successful spatial memory.

Grid codes underlie multiple cognitive maps in the human brain

Grid cells fire at multiple positions that organize the vertices of equilateral triangles tiling a 2D space and are well studied in rodents. The last decade witnessed rapid progress in two other research lines on grid codes-empirical studies on distributed human grid-like representations in physical and multiple non-physical spaces, and cognitive computational models addressing the function of grid cells based on principles of efficient and predictive coding. Here, we review the progress in these fields and integrate these lines into a systematic organization. We also discuss the coordinate mechanisms of grid codes in the human entorhinal cortex and medial prefrontal cortex and their role in neurological and psychiatric diseases.

Abnormal patterns of sleep and waking behaviors are accompanied by neocortical oscillation disturbances in an Ank3 mouse model of epilepsy-bipolar disorder comorbidity

ANK3 is a leading bipolar disorder (BD) candidate gene in humans and provides a unique opportunity for studying epilepsy-BD comorbidity. Previous studies showed that deletion of Ank3-1b, a BD-associated variant of Ank3 in mice leads to increased firing threshold and diminished action potential dynamic range of parvalbumin (PV) interneurons and absence epilepsy, thus providing a biological mechanism linking epilepsy and BD. To explore the behavioral overlap of these disorders, we characterized behavioral patterns of Ank3-1b KO mice during overnight home-cage activity and examined network activity during these behaviors using paired video and EEG recordings. Since PV interneurons contribute to the generation of high-frequency gamma oscillations, we anticipated changes in the power of neocortical EEG signals in the gamma frequency range (> 25 Hz) during behavioral states related to human BD symptoms, including abnormal sleep, hyperactivity, and repetitive behaviors. Ank3-1b KO mice exhibited an overall increase in slow gamma (~25-45 Hz) power compared to controls, and slow gamma power correlated with seizure phenotype severity across behaviors. During sleep, increased slow gamma power correlated with decreased time spent in the rapid eye movement (REM) stage of sleep. Seizures were more common during REM sleep compared to non-REM (NREM) sleep. We also found that Ank3-1b KO mice were hyperactive and exhibited a repetitive behavior phenotype that co-occurred with increased slow gamma power. Our results identify a novel EEG biomarker associating Ank3 genetic variation with BD and epilepsy and suggest modulation of gamma oscillations as a potential therapeutic target.

Ketamine evoked disruption of entorhinal and hippocampal spatial maps

Ketamine, a rapid-acting anesthetic and acute antidepressant, carries undesirable spatial cognition side effects including out-of-body experiences and spatial memory impairments. The neural substrates that underlie these alterations in spatial cognition however, remain incompletely understood. Here, we used electrophysiology and calcium imaging to examine ketamine's impacts on the medial entorhinal cortex and hippocampus, which contain neurons that encode an animal's spatial position, as mice navigated virtual reality and real world environments. Ketamine acutely increased firing rates, degraded cell-pair temporal firing-rate relationships, and altered oscillations, leading to longer-term remapping of spatial representations. In the reciprocally connected hippocampus, the activity of neurons that encode the position of the animal was suppressed after ketamine administration. Together, these findings demonstrate ketamine-induced dysfunction of the MEC-hippocampal circuit at the single cell, local-circuit population, and network levels, connecting previously demonstrated physiological effects of ketamine on spatial cognition to alterations in the spatial navigation circuit.

A consistent map in the medial entorhinal cortex supports spatial memory

The medial entorhinal cortex (MEC) is hypothesized to function as a cognitive map for memory-guided navigation. How this map develops during learning and influences memory remains unclear. By imaging MEC calcium dynamics while mice successfully learned a novel virtual environment over ten days, we discovered that the dynamics gradually became more spatially consistent and then stabilized. Additionally, grid cells in the MEC not only exhibited improved spatial tuning consistency, but also maintained stable phase relationships, suggesting a network mechanism involving synaptic plasticity and rigid recurrent connectivity to shape grid cell activity during learning. Increased c-Fos expression in the MEC in novel environments further supports the induction of synaptic plasticity. Unsuccessful learning lacked these activity features, indicating that a consistent map is specific for effective spatial memory. Finally, optogenetically disrupting spatial consistency of the map impaired memory-guided navigation in a well-learned environment. Thus, we demonstrate that the establishment of a spatially consistent MEC map across learning both correlates with, and is necessary for, successful spatial memory.

How Do You Build a Cognitive Map? The Development of Circuits and Computations for the Representation of Space in the Brain

In mammals, the activity of neurons in the entorhinal-hippocampal network is modulated by the animal's position and its movement through space. At multiple stages of this distributed circuit, distinct populations of neurons can represent a rich repertoire of navigation-related variables like the animal's location, the speed and direction of its movements, or the presence of borders and objects. Working together, spatially tuned neurons give rise to an internal representation of space, a cognitive map that supports an animal's ability to navigate the world and to encode and consolidate memories from experience. The mechanisms by which, during development, the brain acquires the ability to create an internal representation of space are just beginning to be elucidated. In this review, we examine recent work that has begun to investigate the ontogeny of circuitry, firing patterns, and computations underpinning the representation of space in the mammalian brain.

Experience alters hippocampal and cortical network communication via a KIBRA-dependent mechanism

Synaptic plasticity is hypothesized to underlie "replay" of salient experience during hippocampal sharp-wave/ripple (SWR)-based ensemble activity and to facilitate systems-level memory consolidation coordinated by SWRs and cortical sleep spindles. It remains unclear how molecular changes at synapses contribute to experience-induced modification of network function. The synaptic protein KIBRA regulates plasticity and memory. To determine the impact of KIBRA-regulated plasticity on circuit dynamics, we recorded in vivo neural activity from wild-type (WT) mice and littermates lacking KIBRA and examined circuit function before, during, and after novel experience. In WT mice, experience altered population activity and oscillatory dynamics in a manner consistent with incorporation of new information content in replay and enhanced hippocampal-cortical communication. While baseline SWR features were normal in KIBRA conditional knockout (cKO) mice, experience-dependent alterations in SWRs were absent. Furthermore, intra-hippocampal and hippocampal-cortical communication during SWRs was disrupted following KIBRA deletion. These results indicate molecular mechanisms that underlie network-level adaptations to experience.

Are grid cells used for navigation? On local metrics, subjective spaces, and black holes

The symmetric, lattice-like spatial pattern of grid-cell activity is thought to provide a neuronal global metric for space. This view is compatible with grid cells recorded in empty boxes but inconsistent with data from more naturalistic settings. We review evidence arguing against the global-metric notion, including the distortion and disintegration of the grid pattern in complex and three-dimensional environments. We argue that deviations from lattice symmetry are key for understanding grid-cell function. We propose three possible functions for grid cells, which treat real-world grid distortions as a feature rather than a bug. First, grid cells may constitute a local metric for proximal space rather than a global metric for all space. Second, grid cells could form a metric for subjective action-relevant space rather than physical space. Third, distortions may represent salient locations. Finally, we discuss mechanisms that can underlie these functions. These ideas may transform our thinking about grid cells.

Grid cell disruption in a mouse model of early Alzheimer's disease reflects reduced integration of self-motion cues

Converging evidence from human and rodent studies suggests that disrupted grid cell coding in the medial entorhinal cortex (MEC) underlies path integration behavioral deficits during early Alzheimer's disease (AD). However, grid cell firing relies on both self-motion cues and environmental features, and it remains unclear whether disrupted grid coding can account for specific path integration deficits reported during early AD. Here, we report in the J20 transgenic amyloid beta (Aβ) mouse model of early AD that grid cells were spatially unstable toward the center of the arena, had qualitatively different spatial components that aligned parallel to the borders of the environment, and exhibited impaired integration of distance traveled via reduced theta phase precession. Our results suggest that disrupted early AD grid coding reflects reduced integration of self-motion cues but not environmental information via geometric boundaries, providing evidence that grid cell impairments underlie path integration deficits during early AD.

Sleep-A brain-state serving systems memory consolidation

Although long-term memory consolidation is supported by sleep, it is unclear how it differs from that during wakefulness. Our review, focusing on recent advances in the field, identifies the repeated replay of neuronal firing patterns as a basic mechanism triggering consolidation during sleep and wakefulness. During sleep, memory replay occurs during slow-wave sleep (SWS) in hippocampal assemblies together with ripples, thalamic spindles, neocortical slow oscillations, and noradrenergic activity. Here, hippocampal replay likely favors the transformation of hippocampus-dependent episodic memory into schema-like neocortical memory. REM sleep following SWS might balance local synaptic rescaling accompanying memory transformation with a sleep-dependent homeostatic process of global synaptic renormalization. Sleep-dependent memory transformation is intensified during early development despite the immaturity of the hippocampus. Overall, beyond its greater efficacy, sleep consolidation differs from wake consolidation mainly in that it is supported, rather than impaired, by spontaneous hippocampal replay activity possibly gating memory formation in neocortex.

Ketamine evoked disruption of entorhinal and hippocampal spatial maps

Ketamine, a rapid-acting anesthetic and acute antidepressant, carries undesirable spatial cognition side effects including out-of-body experiences and spatial memory impairments. The neural substrates that underlie these alterations in spatial cognition however, remain incompletely understood. Here, we used electrophysiology and calcium imaging to examine ketamine's impacts on the medial entorhinal cortex and hippocampus, which contain neurons that encode an animal's spatial position, as mice navigated virtual reality and real world environments. Ketamine induced an acute disruption and long-term re-organization of entorhinal spatial representations. This acute ketamine-induced disruption reflected increased excitatory neuron firing rates and degradation of cell-pair temporal firing rate relationships. In the reciprocally connected hippocampus, the activity of neurons that encode the position of the animal was suppressed after ketamine administration. Together, these findings point to disruption in the spatial coding properties of the entorhinal-hippocampal circuit as a potential neural substrate for ketamine-induced changes in spatial cognition.

The chicken and egg problem of grid cells and place cells

Place cells and grid cells are major building blocks of the hippocampal cognitive map. The prominent forward model postulates that grid-cell modules are generated by a continuous attractor network; that a velocity signal evoked during locomotion moves entorhinal activity bumps; and that place-cell activity constitutes summation of entorhinal grid-cell modules. Experimental data support the first postulate, but not the latter two. Several families of solutions that depart from these postulates have been put forward. We suggest a modified model (spatial modulation continuous attractor network; SCAN), whereby place cells are generated from spatially selective nongrid cells. Locomotion causes these cells to move the hippocampal activity bump, leading to movement of the entorhinal manifolds. Such inversion accords with the shift of hippocampal thought from navigation to more abstract functions.

Rethinking retrosplenial cortex: Perspectives and predictions

The last decade has produced exciting new ideas about retrosplenial cortex (RSC) and its role in integrating diverse inputs. Here, we review the diversity in forms of spatial and directional tuning of RSC activity, temporal organization of RSC activity, and features of RSC interconnectivity with other brain structures. We find that RSC anatomy and dynamics are more consistent with roles in multiple sensorimotor and cognitive processes than with any isolated function. However, two more generalized categories of function may best characterize roles for RSC in complex cognitive processes: (1) shifting and relating perspectives for spatial cognition and (2) prediction and error correction for current sensory states with internal representations of the environment. Both functions likely take advantage of RSC's capacity to encode conjunctions among sensory, motor, and spatial mapping information streams. Together, these functions provide the scaffold for intelligent actions, such as navigation, perspective taking, interaction with others, and error detection.

Querying hippocampal replay with subcortical inputs

During sleep, the hippocampus recapitulates neuronal patterns corresponding to behavioral trajectories during previous experiences. This hippocampal replay supports the formation of long-term memories. Yet, whether replay originates within the hippocampal circuitry or is initiated by extrahippocampal inputs is unknown. Here, I review recent findings regarding the organization of neuronal activity upstream to the hippocampus, in the head-direction (HD) and grid cell networks, and its relationship to replay. I argue that hippocampal activity at the onset of replay is under the influence of signals from primary spatial areas. In turn, hippocampal replay resets the HD network activity to select a new direction for the next replay event. This reciprocal interaction between the HD network and the hippocampus may be essential in grounding meaning to hippocampal activity, specifically by training decoders of hippocampal sequences. Neuronal dynamics in thalamo-hippocampal loops may thus be instrumental for memory processes during sleep.

Microcircuits for spatial coding in the medial entorhinal cortex

The hippocampal formation is critically involved in learning and memory and contains a large proportion of neurons encoding aspects of the organism's spatial surroundings. In the medial entorhinal cortex (MEC), this includes grid cells with their distinctive hexagonal firing fields as well as a host of other functionally defined cell types including head direction cells, speed cells, border cells, and object-vector cells. Such spatial coding emerges from the processing of external inputs by local microcircuits. However, it remains unclear exactly how local microcircuits and their dynamics within the MEC contribute to spatial discharge patterns. In this review we focus on recent investigations of intrinsic MEC connectivity, which have started to describe and quantify both excitatory and inhibitory wiring in the superficial layers of the MEC. Although the picture is far from complete, it appears that these layers contain robust recurrent connectivity that could sustain the attractor dynamics posited to underlie grid pattern formation. These findings pave the way to a deeper understanding of the mechanisms underlying spatial navigation and memory.

Grid-cell modules remain coordinated when neural activity is dissociated from external sensory cues

The representation of an animal’s position in the medial entorhinal cortex (MEC) is distributed across several modules of grid cells, each characterized by a distinct spatial scale. The population activity within each module is tightly coordinated and preserved across environments and behavioral states. Little is known, however, about the coordination of activity patterns across modules. We analyzed the joint activity patterns of hundreds of grid cells simultaneously recorded in animals that were foraging either in the light, when sensory cues could stabilize the representation, or in darkness, when such stabilization was disrupted. We found that the states of different modules are tightly coordinated, even in darkness, when the internal representation of position within the MEC deviates substantially from the true position of the animal. These findings suggest that internal brain mechanisms dynamically coordinate the representation of position in different modules, ensuring that they jointly encode a coherent and smooth trajectory.

•

Hundreds of grid cells were recorded simultaneously from multiple grid modules

•

Coordination between grid modules was assessed in rats that foraged in darkness

•

Coordination persists despite relative drift of the represented versus true position

•

This suggests that internal network mechanisms maintain inter-module coordination

Toroidal topology of population activity in grid cells

The medial entorhinal cortex is part of a neural system for mapping the position of an individual within a physical environment1. Grid cells, a key component of this system, fire in a characteristic hexagonal pattern of locations2, and are organized in modules3 that collectively form a population code for the animal's allocentric position1. The invariance of the correlation structure of this population code across environments4,5 and behavioural states6,7, independent of specific sensory inputs, has pointed to intrinsic, recurrently connected continuous attractor networks (CANs) as a possible substrate of the grid pattern1,8-11. However, whether grid cell networks show continuous attractor dynamics, and how they interface with inputs from the environment, has remained unclear owing to the small samples of cells obtained so far. Here, using simultaneous recordings from many hundreds of grid cells and subsequent topological data analysis, we show that the joint activity of grid cells from an individual module resides on a toroidal manifold, as expected in a two-dimensional CAN. Positions on the torus correspond to positions of the moving animal in the environment. Individual cells are preferentially active at singular positions on the torus. Their positions are maintained between environments and from wakefulness to sleep, as predicted by CAN models for grid cells but not by alternative feedforward models12. This demonstration of network dynamics on a toroidal manifold provides a population-level visualization of CAN dynamics in grid cells.

Sleep replay reveals premotor circuit structure for a skilled behavior

Neural circuits often exhibit sequences of activity, but the contribution of local networks to their generation remains unclear. In the zebra finch, song-related premotor sequences within HVC may result from some combination of local connectivity and long-range thalamic inputs from nucleus uvaeformis (Uva). Because lesions to either structure abolish song, we examine "sleep replay" using high-density recording methods to reconstruct precise song-related events. Replay activity persists after the upstream nucleus interfacialis of the nidopallium is lesioned and slows when HVC is cooled, demonstrating that HVC provides temporal structure for these events. To further gauge the importance of intra-HVC connectivity for shaping network dynamics, we lesion Uva during sleep and find that residual replay sequences could span syllable boundaries, supporting a model in which HVC can propagate sequences throughout the duration of the song. Our results highlight the power of studying offline activity to investigate behaviorally relevant circuit organization.

A Plane-Dependent Model of 3D Grid Cells for Representing Both 2D and 3D Spaces Under Various Navigation Modes

Grid cells are crucial in path integration and representation of the external world. The spikes of grid cells spatially form clusters called grid fields, which encode important information about allocentric positions. To decode the information, studying the spatial structures of grid fields is a key task for both experimenters and theorists. Experiments reveal that grid fields form hexagonal lattice during planar navigation, and are anisotropic beyond planar navigation. During volumetric navigation, they lose global order but possess local order. How grid cells form different field structures behind these different navigation modes remains an open theoretical question. However, to date, few models connect to the latest discoveries and explain the formation of various grid field structures. To fill in this gap, we propose an interpretive plane-dependent model of three-dimensional (3D) grid cells for representing both two-dimensional (2D) and 3D space. The model first evaluates motion with respect to planes, such as the planes animals stand on and the tangent planes of the motion manifold. Projection of the motion onto the planes leads to anisotropy, and error in the perception of planes degrades grid field regularity. A training-free recurrent neural network (RNN) then maps the processed motion information to grid fields. We verify that our model can generate regular and anisotropic grid fields, as well as grid fields with merely local order; our model is also compatible with mode switching. Furthermore, simulations predict that the degradation of grid field regularity is inversely proportional to the interval between two consecutive perceptions of planes. In conclusion, our model is one of the few pioneers that address grid field structures in a general case. Compared to the other pioneer models, our theory argues that the anisotropy and loss of global order result from the uncertain perception of planes rather than insufficient training.

The grid code for ordered experience

Entorhinal cortical grid cells fire in a periodic pattern that tiles space, which is suggestive of a spatial coordinate system. However, irregularities in the grid pattern as well as responses of grid cells in contexts other than spatial navigation have presented a challenge to existing models of entorhinal function. In this Perspective, we propose that hippocampal input provides a key informative drive to the grid network in both spatial and non-spatial circumstances, particularly around salient events. We build on previous models in which neural activity propagates through the entorhinal-hippocampal network in time. This temporal contiguity in network activity points to temporal order as a necessary characteristic of representations generated by the hippocampal formation. We advocate that interactions in the entorhinal-hippocampal loop build a topological representation that is rooted in the temporal order of experience. In this way, the structure of grid cell firing supports a learned topology rather than a rigid coordinate frame that is bound to measurements of the physical world.

Replay in minds and machines

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Dynamics of fMRI patterns reflect sub-second activation sequences and reveal replay in human visual cortex

Neural computations are often fast and anatomically localized. Yet, investigating such computations in humans is challenging because non-invasive methods have either high temporal or spatial resolution, but not both. Of particular relevance, fast neural replay is known to occur throughout the brain in a coordinated fashion about which little is known. We develop a multivariate analysis method for functional magnetic resonance imaging that makes it possible to study sequentially activated neural patterns separated by less than 100 ms with precise spatial resolution. Human participants viewed five images individually and sequentially with speeds up to 32 ms between items. Probabilistic pattern classifiers were trained on activation patterns in visual and ventrotemporal cortex during individual image trials. Applied to sequence trials, probabilistic classifier time courses allow the detection of neural representations and their order. Order detection remains possible at speeds up to 32 ms between items (plus 100 ms per item). The frequency spectrum of the sequentiality metric distinguishes between sub- versus supra-second sequences. Importantly, applied to resting-state data our method reveals fast replay of task-related stimuli in visual cortex. This indicates that non-hippocampal replay occurs even after tasks without memory requirements and shows that our method can be used to detect such spontaneously occurring replay.

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure.

Perineuronal nets stabilize the grid cell network

Grid cells are part of a widespread network which supports navigation and spatial memory. Stable grid patterns appear late in development, in concert with extracellular matrix aggregates termed perineuronal nets (PNNs) that condense around inhibitory neurons. It has been suggested that PNNs stabilize synaptic connections and long-term memories, but their role in the grid cell network remains elusive. We show that removal of PNNs leads to lower inhibitory spiking activity, and reduces grid cells' ability to create stable representations of a novel environment. Furthermore, in animals with disrupted PNNs, exposure to a novel arena corrupted the spatiotemporal relationships within grid cell modules, and the stored representations of a familiar arena. Finally, we show that PNN removal in entorhinal cortex distorted spatial representations in downstream hippocampal neurons. Together this work suggests that PNNs provide a key stabilizing element for the grid cell network.

Elemental and Configural Associative Learning in Spatial Tasks: Could Zebrafish be Used to Advance Our Knowledge?

Spatial learning and memory have been studied for several decades. Analyses of these processes pose fundamental scientific questions but are also relevant from a biomedical perspective. The cellular, synaptic and molecular mechanisms underlying spatial learning have been intensively investigated, yet the behavioral mechanisms/strategies in a spatial task still pose unanswered questions. Spatial learning relies upon configural information about cues in the environment. However, each of these cues can also independently form part of an elemental association with the specific spatial position, and thus spatial tasks may be solved using elemental (single CS and US association) learning. Here, we first briefly review what we know about configural learning from studies with rodents. Subsequently, we discuss the pros and cons of employing a relatively novel laboratory organism, the zebrafish in such studies, providing some examples of methods with which both elemental and configural learning may be explored with this species. Last, we speculate about future research directions focusing on how zebrafish may advance our knowledge. We argue that zebrafish strikes a reasonable compromise between system complexity and practical simplicity and that adding this species to the studies with laboratory rodents will allow us to gain a better understanding of both the evolution of and the mechanisms underlying spatial learning. We conclude that zebrafish research will enhance the translational relevance of our findings.

Recurrent amplification of grid-cell activity

High-level cognitive abilities such as navigation and spatial memory are thought to rely on the activity of grid cells in the medial entorhinal cortex (MEC), which encode the animal's position in space with periodic triangular patterns. Yet the neural mechanisms that underlie grid-cell activity are still unknown. Recent in vitro and in vivo experiments indicate that grid cells are embedded in highly structured recurrent networks. But how could recurrent connectivity become structured during development? And what is the functional role of these connections? With mathematical modeling and simulations, we show that recurrent circuits in the MEC could emerge under the supervision of weakly grid-tuned feedforward inputs. We demonstrate that a learned excitatory connectivity could amplify grid patterns when the feedforward sensory inputs are available and sustain attractor states when the sensory cues are lost. Finally, we propose a Fourier-based measure to quantify the spatial periodicity of grid patterns: the grid-tuning index.

Transferring structural knowledge across cognitive maps in humans and models

Relations between task elements often follow hidden underlying structural forms such as periodicities or hierarchies, whose inferences fosters performance. However, transferring structural knowledge to novel environments requires flexible representations that are generalizable over particularities of the current environment, such as its stimuli and size. We suggest that humans represent structural forms as abstract basis sets and that in novel tasks, the structural form is inferred and the relevant basis set is transferred. Using a computational model, we show that such representation allows inference of the underlying structural form, important task states, effective behavioural policies and the existence of unobserved state-trajectories. In two experiments, participants learned three abstract graphs during two successive days. We tested how structural knowledge acquired on Day-1 affected Day-2 performance. In line with our model, participants who had a correct structural prior were able to infer the existence of unobserved state-trajectories and appropriate behavioural policies.

A theory of joint attractor dynamics in the hippocampus and the entorhinal cortex accounts for artificial remapping and grid cell field-to-field variability

The representation of position in the mammalian brain is distributed across multiple neural populations. Grid cell modules in the medial entorhinal cortex (MEC) express activity patterns that span a low-dimensional manifold which remains stable across different environments. In contrast, the activity patterns of hippocampal place cells span distinct low-dimensional manifolds in different environments. It is unknown how these multiple representations of position are coordinated. Here, we develop a theory of joint attractor dynamics in the hippocampus and the MEC. We show that the system exhibits a coordinated, joint representation of position across multiple environments, consistent with global remapping in place cells and grid cells. In addition, our model accounts for recent experimental observations that lack a mechanistic explanation: variability in the firing rate of single grid cells across firing fields, and artificial remapping of place cells under depolarization, but not under hyperpolarization, of layer II stellate cells of the MEC.

Precise coupling of the thalamic head-direction system to hippocampal ripples

The anterior thalamus is a key relay of neuronal signals within the limbic system. During sleep, the occurrence of hippocampal sharp wave-ripples (SWRs), believed to mediate consolidation of explicit memories, is modulated by thalamocortical network activity, yet how information is routed around SWRs and how this communication depends on neuronal dynamics remains unclear. Here, by simultaneously recording ensembles of neurons in the anterior thalamus and local field potentials in the CA1 area of the hippocampus, we show that the head-direction (HD) cells of the anterodorsal nucleus are set in stable directions immediately before SWRs. This response contrasts with other thalamic cells that exhibit diverse couplings to the hippocampus related to their intrinsic dynamics but independent of their anatomical location. Thus, our data suggest a specific and homogeneous contribution of the HD signal to hippocampal activity and a diverse and cell-specific coupling of non-HD neurons.

Efficient and flexible representation of higher-dimensional cognitive variables with grid cells

We shed light on the potential of entorhinal grid cells to efficiently encode variables of dimension greater than two, while remaining faithful to empirical data on their low-dimensional structure. Our model constructs representations of high-dimensional inputs through a combination of low-dimensional random projections and "classical" low-dimensional hexagonal grid cell responses. Without reconfiguration of the recurrent circuit, the same system can flexibly encode multiple variables of different dimensions while maximizing the coding range (per dimension) by automatically trading-off dimension with an exponentially large coding range. It achieves high efficiency and flexibility by combining two powerful concepts, modularity and mixed selectivity, in what we call "mixed modular coding". In contrast to previously proposed schemes, the model does not require the formation of higher-dimensional grid responses, a cell-inefficient and rigid mechanism. The firing fields observed in flying bats or climbing rats can be generated by neurons that combine activity from multiple grid modules, each representing higher-dimensional spaces according to our model. The idea expands our understanding of grid cells, suggesting that they could implement a general circuit that generates on-demand coding and memory states for variables in high-dimensional vector spaces.

Coordinated activities of retrosplenial ensembles during resting-state encode spatial landmarks

The brain likely uses offline periods to consolidate recent memories. One hypothesis holds that the hippocampal output provides a unique, global linking or 'index' code for each memory, and that this code is stored in the cortex in association with locally encoded attributes of each memory. Activation of the index code is hypothesized to evoke coordinated memory trace reactivation thus facilitating consolidation. Retrosplenial cortex (RSC) is a major recipient of hippocampal outflow and we have described populations of neurons there with sparse and orthogonal coding characteristics that resemble hippocampal 'place' cells, and whose expression depends on an intact hippocampus. Using two-photon Ca²⁺ imaging, we recorded ensembles of neurons in the RSC during periods of immobility before and after active running on a familiar linear treadmill track. Synchronous bursting of distinct groups of neurons occurred during rest both prior to and after running. In the second rest epoch, these patterns were associated with the locations of tactile landmarks and reward. Complementing established views on the functions of the RSC, our findings indicate that the structure is involved with processing landmark information during rest. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

On the methods for reactivation and replay analysis

A major task in the history of neurophysiology has been to relate patterns of neural activity to ongoing external stimuli. More recently, this approach has branched out to relating current neural activity patterns to external stimuli or experiences that occurred in the past or future. Here, we aim to review the large body of methodological approaches used towards this goal, and to assess the assumptions each makes with reference to the statistics of neural data that are commonly observed. These methods primarily fall into two categories, those that quantify zero-lag relationships without examining temporal evolution, termed reactivation, and those that quantify the temporal structure of changing activity patterns, termed replay. However, no two studies use the exact same approach, which prevents an unbiased comparison between findings. These observations should instead be validated by multiple and, if possible, previously established tests. This will help the community to speak a common language and will eventually provide tools to study, more generally, the organization of neuronal patterns in the brain. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Spatial Navigation

The hippocampus is critical for spatial navigation. In this review, we focus on the role of the hippocampus in three basic strategies used for spatial navigation: path integration, stimulus-response association, and map-based navigation. First, the hippocampus is not required for path integration unless the path of path integration is too long and complex. The hippocampus provides mnemonic support when involved in the process of path integration. Second, the hippocampus's involvement in stimulus-response association is dependent on how the strategy is conducted. The hippocampus is not required for the habit form of stimulus-response association. Third, while the hippocampus is fully engaged in map-based navigation, the shared characteristics of place cells, grid cells, head direction cells, and other spatial encoding cells, which are detected in the hippocampus and associated areas, offer a possibility that there is a stand-alone allocentric space perception (or mental representation) of the environment outside and independent of the hippocampus, and the spatially specific firing patterns of these spatial encoding cells are the unfolding of the intermediate stages of the processing of this allocentric spatial information when conveyed into the hippocampus for information storage or retrieval. Furthermore, the presence of all the spatially specific firing patterns in the hippocampus and the related neural circuits during the path integration and map-based navigation support such a notion that in essence, path integration is the same allocentric space perception provided with only idiothetic inputs. Taken together, the hippocampus plays a general mnemonic role in spatial navigation.

During hippocampal inactivation, grid cells maintain synchrony, even when the grid pattern is lost

The grid cell network in the medial entorhinal cortex (MEC) has been subject to thorough testing and analysis, and many theories for their formation have been suggested. To test some of these theories, we re-analyzed data from Bonnevie et al., 2013, in which the hippocampus was inactivated and grid cells were recorded in the rat MEC. We investigated whether the firing associations of grid cells depend on hippocampal inputs. Specifically, we examined temporal and spatial correlations in the firing times of simultaneously recorded grid cells before and during hippocampal inactivation. Our analysis revealed evidence of network coherence in grid cells even in the absence of hippocampal input to the MEC, both in regular grid cells and in those that became head-direction cells after hippocampal inactivation. This favors models, which suggest that phase relations between grid cells in the MEC are dependent on intrinsic connectivity within the MEC.

Velocity coupling of grid cell modules enables stable embedding of a low dimensional variable in a high dimensional neural attractor

Grid cells in the medial entorhinal cortex (MEC) encode position using a distributed representation across multiple neural populations (modules), each possessing a distinct spatial scale. The modular structure of the representation confers the grid cell neural code with large capacity. Yet, the modularity poses significant challenges for the neural circuitry that maintains the representation, and updates it based on self motion. Small incompatible drifts in different modules, driven by noise, can rapidly lead to large, abrupt shifts in the represented position, resulting in catastrophic readout errors. Here, we propose a theoretical model of coupled modules. The coupling suppresses incompatible drifts, allowing for a stable embedding of a two-dimensional variable (position) in a higher dimensional neural attractor, while preserving the large capacity. We propose that coupling of this type may be implemented by recurrent synaptic connectivity within the MEC with a relatively simple and biologically plausible structure.

Validating the theoretical bases of sleep reactivation during sharp-wave ripples and their association with emotional valence

Sleep is important for memory consolidation, and an abundant literature suggests that reactivation in the hippocampus during sleep is instrumental to this process. Yet, the current interpretation of activity during sharp-waves ripples (SWRs), as replay of wake experiences, relies on hypotheses that, while widely accepted, have only recently begun to be tested directly. Moreover, this theory has been mainly studied in the context of pure spatial learning, and it is still not clear how emotional valence can fit into this conceptual framework when considering reward- or punishment-based learning. In this review, we will present recent experimental arguments validating the interpretation of sleep replay as reactivation of awake experiences and examine the evidence showing that the emotional valence is also replayed during sleep in a coordinated fashion with hippocampal SWRs. Finally, we will detail recent experiments showing that brain-computer interfaces can be used to modify the emotional valence associated with sleep replay.

CA3 place cells that represent a novel waking experience are preferentially reactivated during sharp wave-ripples in subsequent sleep

A popular model of memory consolidation posits that recent memories stored in the hippocampus are reactivated during sleep and thereby transferred to neocortex for long-term storage. This process is thought to occur during sharp wave-ripples (SWRs) in nonrapid eye movement sleep (NREM). However, whether the hippocampus consolidates all recent memories in the same manner remains unclear. An efficient memory system may extract novel information from recent experiences for preferential consolidation. In the hippocampus, memories are thought to be stored initially in CA3. Therefore, CA3 place cells that encode novel experiences may be preferentially reactivated during SWRs in subsequent sleep. To test this hypothesis, we recorded CA3 place cells in rats during exposure to a familiar and a novel environment and during subsequent overnight sleep. We found that CA3 place cells that preferentially coded a novel environment showed larger firing rate increases during SWRs in NREM than place cells that preferentially coded a familiar environment. Moreover, CA3 place cell ensembles replayed trajectories from a novel environment during NREM with higher fidelity than trajectories from a familiar environment. Together, these results suggest that CA3 representations of novel experiences are preferentially processed during subsequent sleep.