Alternating sequences of future and past behavior encoded within hippocampal theta oscillations

Mental Time Travel: A Retrospective

Because imagination activates the same neural circuits used in understanding the present, one can access that imagination even in non-linguistic animals through decoding techniques applied to large neural ensembles. This personal retrospective traces the history of the initial discovery that hippocampal theta sequences sweep forward to goals during moments of deliberation and discusses the history that was necessary to put ourselves in the position to recognize this signal. It also discusses how that discovery fits into the larger picture of hippocampal function and the concept of cognition as computation.

Idiothetic representations are modulated by availability of sensory inputs and task demands in the hippocampal-septal circuit

The hippocampus is a higher-order brain structure responsible for encoding new episodic memories and predicting future outcomes. In the absence of external stimuli, neurons in the hippocampus track elapsed time, distance traveled, and other idiothetic variables. To this day, the exact determinants of idiothetic representations during free navigation remain unclear. Here, we developed unsupervised approaches to extract population and single-cell properties of more than 30,000 CA1 pyramidal neurons in freely moving mice. We find that spatiotemporal representations are composed of a mixture of idiothetic and allocentric information, the balance of which is dictated by task demand and environmental conditions. Additionally, a subset of CA1 pyramidal neurons encodes the spatiotemporal distance to rewards. Finally, distance and time information is integrated postsynaptically in the lateral septum, indicating that these high-level representations are effectively integrated in downstream neurons.

Analysis methods for large-scale neuronal recordings

Simultaneous recordings from hundreds or thousands of neurons are becoming routine because of innovations in instrumentation, molecular tools, and data processing software. Such recordings can be analyzed with data science methods, but it is not immediately clear what methods to use or how to adapt them for neuroscience applications. We review, categorize, and illustrate diverse analysis methods for neural population recordings and describe how these methods have been used to make progress on longstanding questions in neuroscience. We review a variety of approaches, ranging from the mathematically simple to the complex, from exploratory to hypothesis-driven, and from recently developed to more established methods. We also illustrate some of the common statistical pitfalls in analyzing large-scale neural data.

Integration of rate and phase codes by hippocampal cell-assemblies supports flexible encoding of spatiotemporal context

Spatial information is encoded by location-dependent hippocampal place cell firing rates and sub-second, rhythmic entrainment of spike times. These rate and temporal codes have primarily been characterized in low-dimensional environments under limited cognitive demands; but how is coding configured in complex environments when individual place cells signal several locations, individual locations contribute to multiple routes and functional demands vary? Quantifying CA1 population dynamics of male rats during a decision-making task, here we show that the phase of individual place cells' spikes relative to the local theta rhythm shifts to differentiate activity in different place fields. Theta phase coding also disambiguates repeated visits to the same location during different routes, particularly preceding spatial decisions. Using unsupervised detection of cell assemblies alongside theoretical simulation, we show that integrating rate and phase coding mechanisms dynamically recruits units to different assemblies, generating spiking sequences that disambiguate episodes of experience and multiplexing spatial information with cognitive context.

Hippocampal representations of alternative possibilities are flexibly generated to meet cognitive demands

The cognitive ability to go beyond the present to consider alternative possibilities, including potential futures and counterfactual pasts, can support adaptive decision making. Complex and changing real-world environments, however, have many possible alternatives. Whether and how the brain can select among them to represent alternatives that meet current cognitive needs remains unknown. We therefore examined neural representations of alternative spatial locations in the rat hippocampus during navigation in a complex patch foraging environment with changing reward probabilities. We found representations of multiple alternatives along paths ahead and behind the animal, including in distant alternative patches. Critically, these representations were modulated in distinct patterns across successive trials: alternative paths were represented proportionate to their evolving relative value and predicted subsequent decisions, whereas distant alternatives were prevalent during value updating. These results demonstrate that the brain modulates the generation of alternative possibilities in patterns that meet changing cognitive needs for adaptive behavior.

Humans adaptively deploy forward and backward prediction

The formation of predictions is essential to our ability to build models of the world and use them for intelligent decision-making. Here we challenge the dominant assumption that humans form only forward predictions, which specify what future events are likely to follow a given present event. We demonstrate that in some environments, it is more efficient to use backward prediction, which specifies what present events are likely to precede a given future event. This is particularly the case in diverging environments, where possible future events outnumber possible present events. Correspondingly, in six preregistered experiments (n = 1,299) involving both simple decision-making and more challenging planning tasks, we find that humans engage in backward prediction in divergent environments and use forward prediction in convergent environments. We thus establish that humans adaptively deploy forward and backward prediction in the service of efficient decision-making.

Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus

The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.

Firing rate adaptation affords place cell theta sweeps, phase precession, and procession

Hippocampal place cells in freely moving rodents display both theta phase precession and procession, which is thought to play important roles in cognition, but the neural mechanism for producing theta phase shift remains largely unknown. Here, we show that firing rate adaptation within a continuous attractor neural network causes the neural activity bump to oscillate around the external input, resembling theta sweeps of decoded position during locomotion. These forward and backward sweeps naturally account for theta phase precession and procession of individual neurons, respectively. By tuning the adaptation strength, our model explains the difference between 'bimodal cells' showing interleaved phase precession and procession, and 'unimodal cells' in which phase precession predominates. Our model also explains the constant cycling of theta sweeps along different arms in a T-maze environment, the speed modulation of place cells' firing frequency, and the continued phase shift after transient silencing of the hippocampus. We hope that this study will aid an understanding of the neural mechanism supporting theta phase coding in the brain.

Self-avoidance dominates the selection of hippocampal replay

Spontaneous neural activity sequences are generated by the brain in the absence of external input 1-12 , yet how they are produced remains unknown. During immobility, hippocampal replay sequences depict spatial paths related to the animal's past experience or predicted future 13 . By recording from large ensembles of hippocampal place cells 14 in combination with optogenetic manipulation of cortical input in freely behaving rats, we show here that the selection of hippocampal replay is governed by a novel self-avoidance principle. Following movement cessation, replay of the animal's past path is strongly avoided, while replay of the future path predominates. Moreover, when the past and future paths overlap, early replays avoid both and depict entirely different trajectories. Further, replays avoid self-repetition, on a shorter timescale compared to the avoidance of previous behavioral trajectories. Eventually, several seconds into the stopping period, replay of the past trajectory dominates. This temporal organization contrasts with established and recent predictions 9,10,15,16 but is well-recapitulated by a symmetry-breaking attractor model of sequence generation in which individual neurons adapt their firing rates over time 26-35 . However, while the model is sufficient to produce avoidance of recently traversed or reactivated paths, it requires an additional excitatory input into recently activated cells to produce the later window of past-dominance. We performed optogenetic perturbations to demonstrate that this input is provided by medial entorhinal cortex, revealing its role in maintaining a memory of past experience that biases hippocampal replay. Together, these data provide specific evidence for how hippocampal replays are generated.

Hippocampal cholecystokinin-expressing interneurons regulate temporal coding and contextual learning

Cholecystokinin-expressing interneurons (CCKIs) are hypothesized to shape pyramidal cell-firing patterns and regulate network oscillations and related network state transitions. To directly probe their role in the CA1 region, we silenced their activity using optogenetic and chemogenetic tools in mice. Opto-tagged CCKIs revealed a heterogeneous population, and their optogenetic silencing triggered wide disinhibitory network changes affecting both pyramidal cells and other interneurons. CCKI silencing enhanced pyramidal cell burst firing and altered the temporal coding of place cells: theta phase precession was disrupted, whereas sequence reactivation was enhanced. Chemogenetic CCKI silencing did not alter the acquisition of spatial reference memories on the Morris water maze but enhanced the recall of contextual fear memories and enabled selective recall when similar environments were tested. This work suggests the key involvement of CCKIs in the control of place-cell temporal coding and the formation of contextual memories.

An Improved Dyna-Q Algorithm Inspired by the Forward Prediction Mechanism in the Rat Brain for Mobile Robot Path Planning

The traditional Model-Based Reinforcement Learning (MBRL) algorithm has high computational cost, poor convergence, and poor performance in robot spatial cognition and navigation tasks, and it cannot fully explain the ability of animals to quickly adapt to environmental changes and learn a variety of complex tasks. Studies have shown that vicarious trial and error (VTE) and the hippocampus forward prediction mechanism in rats and other mammals can be used as key components of action selection in MBRL to support "goal-oriented" behavior. Therefore, we propose an improved Dyna-Q algorithm inspired by the forward prediction mechanism of the hippocampus to solve the above problems and tackle the exploration-exploitation dilemma of Reinforcement Learning (RL). This algorithm alternately presents the potential path in the future for mobile robots and dynamically adjusts the sweep length according to the decision certainty, so as to determine action selection. We test the performance of the algorithm in a two-dimensional maze environment with static and dynamic obstacles, respectively. Compared with classic RL algorithms like State-Action-Reward-State-Action (SARSA) and Dyna-Q, the algorithm can speed up spatial cognition and improve the global search ability of path planning. In addition, our method reflects key features of how the brain organizes MBRL to effectively solve difficult tasks such as navigation, and it provides a new idea for spatial cognitive tasks from a biological perspective.

Dynamic prediction of goal location by coordinated representation of prefrontal-hippocampal theta sequences

Prefrontal (PFC) and hippocampal (HPC) sequences of neuronal firing modulated by theta rhythms could represent upcoming choices during spatial memory-guided decision-making. How the PFC-HPC network dynamically coordinates theta sequences to predict specific goal locations and how it is interrupted in memory impairments induced by amyloid beta (Aβ) remain unclear. Here, we detected theta sequences of firing activities of PFC neurons and HPC place cells during goal-directed spatial memory tasks. We found that PFC ensembles exhibited predictive representation of the specific goal location since the starting phase of memory retrieval, earlier than the hippocampus. High predictive accuracy of PFC theta sequences existed during successful memory retrieval and positively correlated with memory performance. Coordinated PFC-HPC sequences showed PFC-dominant prediction of goal locations during successful memory retrieval. Furthermore, we found that theta sequences of both regions still existed under Aβ accumulation, whereas their predictive representation of goal locations was weakened with disrupted spatial representation of HPC place cells and PFC neurons. These findings highlight the essential role of coordinated PFC-HPC sequences in successful memory retrieval of a precise goal location.

The cognitive reality monitoring network and theories of consciousness

Theories of consciousness abound. However, it is difficult to arbitrate reliably among competing theories because they target different levels of neural and cognitive processing or anatomical loci, and only some were developed with computational models in mind. In particular, theories of consciousness need to fully address the three levels of understanding of the brain proposed by David Marr: computational theory, algorithms and hardware. Most major theories refer to only one or two levels, often indirectly. The cognitive reality monitoring network (CRMN) model is derived from computational theories of mixture-of-experts architecture, hierarchical reinforcement learning and generative/inference computing modules, addressing all three levels of understanding. A central feature of the CRMN is the mapping of a gating network onto the prefrontal cortex, making it a prime coding circuit involved in monitoring the accuracy of one's mental states and distinguishing them from external reality. Because the CRMN builds on the hierarchical and layer structure of the cerebral cortex, it may connect research and findings across species, further enabling concrete computational models of consciousness with new, explicitly testable hypotheses. In sum, we discuss how the CRMN model can help further our understanding of the nature and function of consciousness.

The generative neural microdynamics of cognitive processing

The entorhinal cortex and hippocampus form a recurrent network that informs many cognitive processes, including memory, planning, navigation, and imagination. Neural recordings from these regions reveal spatially organized population codes corresponding to external environments and abstract spaces. Aligning the former cognitive functionalities with the latter neural phenomena is a central challenge in understanding the entorhinal-hippocampal circuit (EHC). Disparate experiments demonstrate a surprising level of complexity and apparent disorder in the intricate spatiotemporal dynamics of sequential non-local hippocampal reactivations, which occur particularly, though not exclusively, during immobile pauses and rest. We review these phenomena with a particular focus on their apparent lack of physical simulative realism. These observations are then integrated within a theoretical framework and proposed neural circuit mechanisms that normatively characterize this neural complexity by conceiving different regimes of hippocampal microdynamics as neuromarkers of diverse cognitive computations.

Disruption of awake sharp-wave ripples does not affect memorization of locations in repeated-acquisition spatial memory tasks

Storing and accessing memories is required to successfully perform day-to-day tasks, for example for engaging in a meaningful conversation. Previous studies in both rodents and primates have correlated hippocampal cellular activity with behavioral expression of memory. A key role has been attributed to awake hippocampal replay - a sequential reactivation of neurons representing a trajectory through space. However, it is unclear if awake replay impacts immediate future behavior, gradually creates and stabilizes long-term memories over a long period of time (hours and longer), or enables the temporary memorization of relevant events at an intermediate time scale (seconds to minutes). In this study, we aimed to address the uncertainty around the timeframe of impact of awake replay by collecting causal evidence from behaving rats. We detected and disrupted sharp wave ripples (SWRs) - signatures of putative replay events - using electrical stimulation of the ventral hippocampal commissure in rats that were trained on three different spatial memory tasks. In each task, rats were required to memorize a new set of locations in each trial or each daily session. Interestingly, the rats performed equally well with or without SWR disruptions. These data suggest that awake SWRs - and potentially replay - does not affect the immediate behavior nor the temporary memorization of relevant events at a short timescale that are required to successfully perform the spatial tasks. Based on these results, we hypothesize that the impact of awake replay on memory and behavior is long-term and cumulative over time.

Dorsal hippocampus represents locations to avoid as well as locations to approach during approach-avoidance conflict

Worrying about perceived threats is a hallmark of multiple psychological disorders including anxiety. This concern about future events is particularly important when an individual is faced with an approach-avoidance conflict. Potential goals to approach are known to be represented in the dorsal hippocampus during theta sweeps. Similarly, important non-local information is represented during hippocampal high synchrony events (HSEs), which are correlated with sharp-wave ripples (SWRs). It is likely that potential future threats may be similarly represented. We examined how threats and rewards were represented within the hippocampus during approach-avoidance conflicts in rats faced with a predator-like robot guarding a food reward. We found representations of the pseudo-predator during HSEs when hesitating in the nest, and during theta prior to retreating as the rats approached the pseudo-predator. After the first attack, we observed new place fields appearing at the location of the robot (not the location the rat was when attacked). The anxiolytic diazepam reduced anxiety-like behavior and altered hippocampal local field potentials, including reducing SWRs, suggesting that one potential mechanism of diazepam's actions may be through altered representations of imagined threat. These results suggest that hippocampal representation of potential threats could be an important mechanism that underlies worry and a potential target for anxiolytics.

Fantastic beasts and how to study them: rethinking experimental animal behavior

Humans have been trying to understand animal behavior at least since recorded history. Recent rapid development of new technologies has allowed us to make significant progress in understanding the physiological and molecular mechanisms underlying behavior, a key goal of neuroethology. However, there is a tradeoff when studying animal behavior and its underlying biological mechanisms: common behavior protocols in the laboratory are designed to be replicable and controlled, but they often fail to encompass the variability and breadth of natural behavior. This Commentary proposes a framework of 10 key questions that aim to guide researchers in incorporating a rich natural context into their experimental design or in choosing a new animal study system. The 10 questions cover overarching experimental considerations that can provide a template for interspecies comparisons, enable us to develop studies in new model organisms and unlock new experiments in our quest to understand behavior.

Local activation of CA1 pyramidal cells induces theta-phase precession

Hippocampal theta-phase precession is involved in spatiotemporal coding and in generating multineural spike sequences, but how precession originates remains unresolved. To determine whether precession can be generated directly in hippocampal area CA1 and disambiguate multiple competing mechanisms, we used closed-loop optogenetics to impose artificial place fields in pyramidal cells of mice running on a linear track. More than one-third of the CA1 artificial fields exhibited synthetic precession that persisted for a full theta cycle. By contrast, artificial fields in the parietal cortex did not exhibit synthetic precession. These findings are incompatible with precession models based on inheritance, dual-input, spreading activation, inhibition-excitation summation, or somato-dendritic competition. Thus, a precession generator resides locally within CA1.

Special Issue-Biosensors and Neuroscience: Is Biosensors Engineering Ready to Embrace Design Principles from Neuroscience?

In partnership with the Air Force Office of Scientific Research (AFOSR), the National Science Foundation's (NSF) Emerging Frontiers and Multidisciplinary Activities (EFMA) office of the Directorate for Engineering (ENG) launched an Emerging Frontiers in Research and Innovation (EFRI) topic for the fiscal years FY22 and FY23 entitled "Brain-inspired Dynamics for Engineering Energy-Efficient Circuits and Artificial Intelligence" (BRAID) [...].

Wired together, change together: Spike timing modifies transmission in converging assemblies

The precise timing of neuronal spikes may lead to changes in synaptic connectivity and is thought to be crucial for learning and memory. However, the effect of spike timing on neuronal connectivity in the intact brain remains unknown. Using closed-loop optogenetic stimulation in CA1 of freely moving mice, we generated unique spike patterns between presynaptic pyramidal cells (PYRs) and postsynaptic parvalbumin (PV)-immunoreactive cells. The stimulation led to spike transmission changes that occurred together across all presynaptic PYRs connected to the same postsynaptic PV cell. The precise timing of all presynaptic and postsynaptic cell spikes affected transmission changes. These findings reveal an unexpected plasticity mechanism, in which the spike timing of an entire cell assembly has a more substantial impact on effective connectivity than that of individual cell pairs.

Neural Sequences and the Encoding of Time

Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.

New information triggers prospective codes to adapt for flexible navigation

Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a novel task that precisely introduces new information using virtual reality, and we recorded neural activity as mice flexibly adapted their planned destinations. We found that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not flexibly adapt, prefrontal choice codes failed to switch, despite relatively intact hippocampal goal representations. Prospective code updating depended on the commitment to the initial choice and degree of adaptation needed. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.

A goal pointer for a cognitive map in the orbitofrontal cortex

Knowing where you are and where you go is a prerequisite for planning a goal-directed journey. The discovery of spatially tuned neurons in the hippocampus and parahippocampal cortices provides a mechanism by which the brain pinpoints an animal's own position in an environment. By contrast, how the brain encodes a remote navigational goal remained largely obscure until recently. In this review, we discuss algorithmic challenges and requirements for the brain to form a representation of a remote navigational goal at which an animal is not present. We then highlight a line of evidence that neurons in the orbitofrontal cortex (OFC) represent a goal location persistently while an animal navigates to this destination. Finally, we propose a new perspective of navigation research opened by this recently reported brain's goal map.

Bidirectional synaptic changes in deep and superficial hippocampal neurons following in vivo activity

Neuronal activity during experience is thought to induce plastic changes within the hippocampal network that underlie memory formation, although the extent and details of such changes in vivo remain unclear. Here, we employed a temporally precise marker of neuronal activity, CaMPARI2, to label active CA1 hippocampal neurons in vivo, followed by immediate acute slice preparation and electrophysiological quantification of synaptic properties. Recently active neurons in the superficial sublayer of stratum pyramidale displayed larger post-synaptic responses at excitatory synapses from area CA3, with no change in pre-synaptic release probability. In contrast, in vivo activity correlated with weaker pre- and post-synaptic excitatory weights onto pyramidal cells in the deep sublayer. In vivo activity of deep and superficial neurons within sharp-wave/ripples was bidirectionally changed across experience, consistent with the observed changes in synaptic weights. These findings reveal novel, fundamental mechanisms through which the hippocampal network is modified by experience to store information.

Hippocampo-cortical circuits for selective memory encoding, routing, and replay

Traditionally considered a homogeneous cell type, hippocampal pyramidal cells have been recently shown to be highly diverse. However, how this cellular diversity relates to the different hippocampal network computations that support memory-guided behavior is not yet known. We show that the anatomical identity of pyramidal cells is a major organizing principle of CA1 assembly dynamics, the emergence of memory replay, and cortical projection patterns in rats. Segregated pyramidal cell subpopulations encoded trajectory and choice-specific information or tracked changes in reward configuration respectively, and their activity was selectively read out by different cortical targets. Furthermore, distinct hippocampo-cortical assemblies coordinated the reactivation of complementary memory representations. These findings reveal the existence of specialized hippocampo-cortical subcircuits and provide a cellular mechanism that supports the computational flexibility and memory capacities of these structures.

Theta-phase-specific modulation of dentate gyrus memory neurons

The theta rhythm, a quasi-periodic 4-10 Hz oscillation, is observed during memory processing in the hippocampus, with different phases of theta hypothesized to separate independent streams of information related to the encoding and recall of memories. At the cellular level, the discovery of hippocampal memory cells (engram neurons), as well as the modulation of memory recall through optogenetic activation of these cells, has provided evidence that certain memories are stored, in part, in a sparse ensemble of neurons in the hippocampus. In previous research, however, engram reactivation has been carried out using open-loop stimulation at fixed frequencies; the relationship between engram neuron reactivation and ongoing network oscillations has not been taken into consideration. To address this concern, we implemented a closed-loop reactivation of engram neurons that enabled phase-specific stimulation relative to theta oscillations in the local field potential in CA1. Using this real-time approach, we tested the impact of activating dentate gyrus engram neurons during the peak (encoding phase) and trough (recall phase) of theta oscillations. Consistent with previously hypothesized functions of theta oscillations in memory function, we show that stimulating dentate gyrus engram neurons at the trough of theta is more effective in eliciting behavioral recall than either fixed-frequency stimulation or stimulation at the peak of theta. Moreover, phase-specific trough stimulation is accompanied by an increase in the coupling between gamma and theta oscillations in CA1 hippocampus. Our results provide a causal link between phase-specific activation of engram cells and the behavioral expression of memory.

Experience alters hippocampal and cortical network communication via a KIBRA-dependent mechanism

Synaptic plasticity is hypothesized to underlie "replay" of salient experience during hippocampal sharp-wave/ripple (SWR)-based ensemble activity and to facilitate systems-level memory consolidation coordinated by SWRs and cortical sleep spindles. It remains unclear how molecular changes at synapses contribute to experience-induced modification of network function. The synaptic protein KIBRA regulates plasticity and memory. To determine the impact of KIBRA-regulated plasticity on circuit dynamics, we recorded in vivo neural activity from wild-type (WT) mice and littermates lacking KIBRA and examined circuit function before, during, and after novel experience. In WT mice, experience altered population activity and oscillatory dynamics in a manner consistent with incorporation of new information content in replay and enhanced hippocampal-cortical communication. While baseline SWR features were normal in KIBRA conditional knockout (cKO) mice, experience-dependent alterations in SWRs were absent. Furthermore, intra-hippocampal and hippocampal-cortical communication during SWRs was disrupted following KIBRA deletion. These results indicate molecular mechanisms that underlie network-level adaptations to experience.

Dynamic synchronization between hippocampal representations and stepping

The hippocampus is a mammalian brain structure that expresses spatial representations1 and is crucial for navigation2,3. Navigation, in turn, intricately depends on locomotion; however, current accounts suggest a dissociation between hippocampal spatial representations and the details of locomotor processes. Specifically, the hippocampus is thought to represent mainly higher-order cognitive and locomotor variables such as position, speed and direction of movement4-7, whereas the limb movements that propel the animal can be computed and represented primarily in subcortical circuits, including the spinal cord, brainstem and cerebellum8-11. Whether hippocampal representations are actually decoupled from the detailed structure of locomotor processes remains unknown. To address this question, here we simultaneously monitored hippocampal spatial representations and ongoing limb movements underlying locomotion at fast timescales. We found that the forelimb stepping cycle in freely behaving rats is rhythmic and peaks at around 8 Hz during movement, matching the approximately 8 Hz modulation of hippocampal activity and spatial representations during locomotion12. We also discovered precisely timed coordination between the time at which the forelimbs touch the ground ('plant' times of the stepping cycle) and the hippocampal representation of space. Notably, plant times coincide with hippocampal representations that are closest to the actual position of the nose of the rat, whereas between these plant times, the hippocampal representation progresses towards possible future locations. This synchronization was specifically detectable when rats approached spatial decisions. Together, our results reveal a profound and dynamic coordination on a timescale of tens of milliseconds between central cognitive representations and peripheral motor processes. This coordination engages and disengages rapidly in association with cognitive demands and is well suited to support rapid information exchange between cognitive and sensory-motor circuits.

Rapid learning of predictive maps with STDP and theta phase precession

The predictive map hypothesis is a promising candidate principle for hippocampal function. A favoured formalisation of this hypothesis, called the successor representation, proposes that each place cell encodes the expected state occupancy of its target location in the near future. This predictive framework is supported by behavioural as well as electrophysiological evidence and has desirable consequences for both the generalisability and efficiency of reinforcement learning algorithms. However, it is unclear how the successor representation might be learnt in the brain. Error-driven temporal difference learning, commonly used to learn successor representations in artificial agents, is not known to be implemented in hippocampal networks. Instead, we demonstrate that spike-timing dependent plasticity (STDP), a form of Hebbian learning, acting on temporally compressed trajectories known as 'theta sweeps', is sufficient to rapidly learn a close approximation to the successor representation. The model is biologically plausible - it uses spiking neurons modulated by theta-band oscillations, diffuse and overlapping place cell-like state representations, and experimentally matched parameters. We show how this model maps onto known aspects of hippocampal circuitry and explains substantial variance in the temporal difference successor matrix, consequently giving rise to place cells that demonstrate experimentally observed successor representation-related phenomena including backwards expansion on a 1D track and elongation near walls in 2D. Finally, our model provides insight into the observed topographical ordering of place field sizes along the dorsal-ventral axis by showing this is necessary to prevent the detrimental mixing of larger place fields, which encode longer timescale successor representations, with more fine-grained predictions of spatial location.

Heterogeneity of network and coding states in mouse CA1 place cells

Theta sequences and phase precession shape hippocampal activity and are considered key underpinnings of memory formation. Theta sequences are sweeps of spikes from multiple cells, tracing trajectories from past to future. Phase precession is the correlation between theta firing phase and animal position. Here, we reconsider these temporal processes in CA1 and the computational principles that they are thought to obey. We find stronger heterogeneity than previously described: we identify cells that do not phase precess but reliably express theta sequences. Other cells phase precess only when medium gamma (linked to entorhinal inputs) is strongest. The same cells express more sequences, but not precession, when slow gamma (linked to CA3 inputs) dominates. Moreover, sequences occur independently in distinct cell groups. Our results challenge the view that phase precession is the mechanism underlying the emergence of theta sequences, suggesting a role for CA1 cells in multiplexing diverse computational processes.

Frequency matters: how changes in hippocampal theta frequency can influence temporal coding, anxiety-reduction, and memory

Hippocampal theta frequency is a somewhat neglected topic relative to theta power, phase, coherence, and cross-frequency coupling. Accordingly, here we review and present new data on variation in hippocampal theta frequency, focusing on functional associations (temporal coding, anxiety reduction, learning, and memory). Taking the rodent hippocampal theta frequency to running-speed relationship as a model, we identify two doubly-dissociable frequency components: (a) the slope component of the theta frequency-to-stimulus-rate relationship ("theta slope"); and (b) its y-intercept frequency ("theta intercept"). We identify three tonic determinants of hippocampal theta frequency. (1) Hotter temperatures increase theta frequency, potentially consistent with time intervals being judged as shorter when hot. Initial evidence suggests this occurs via the "theta slope" component. (2) Anxiolytic drugs with widely-different post-synaptic and pre-synaptic primary targets share the effect of reducing the "theta intercept" component, supporting notions of a final common pathway in anxiety reduction involving the hippocampus. (3) Novelty reliably decreases, and familiarity increases, theta frequency, acting upon the "theta slope" component. The reliability of this latter finding, and the special status of novelty for learning, prompts us to propose a Novelty Elicits Slowing of Theta frequency (NEST) hypothesis, involving the following elements: (1) Theta frequency slowing in the hippocampal formation is a generalised response to novelty of different types and modalities; (2) Novelty-elicited theta slowing is a hippocampal-formation-wide adaptive response functioning to accommodate the additional need for learning entailed by novelty; (3) Lengthening the theta cycle enhances associativity; (4) Even part-cycle lengthening may boost associativity; and (5) Artificial theta stimulation aimed at enhancing learning should employ low-end theta frequencies.

Temporal context and latent state inference in the hippocampal splitter signal

The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by 'splitter cells', hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function.

Neurodynamical Computing at the Information Boundaries of Intelligent Systems

Artificial intelligence has not achieved defining features of biological intelligence despite models boasting more parameters than neurons in the human brain. In this perspective article, we synthesize historical approaches to understanding intelligent systems and argue that methodological and epistemic biases in these fields can be resolved by shifting away from cognitivist brain-as-computer theories and recognizing that brains exist within large, interdependent living systems. Integrating the dynamical systems view of cognition with the massive distributed feedback of perceptual control theory highlights a theoretical gap in our understanding of nonreductive neural mechanisms. Cell assemblies-properly conceived as reentrant dynamical flows and not merely as identified groups of neurons-may fill that gap by providing a minimal supraneuronal level of organization that establishes a neurodynamical base layer for computation. By considering information streams from physical embodiment and situational embedding, we discuss this computational base layer in terms of conserved oscillatory and structural properties of cortical-hippocampal networks. Our synthesis of embodied cognition, based in dynamical systems and perceptual control, aims to bypass the neurosymbolic stalemates that have arisen in artificial intelligence, cognitive science, and computational neuroscience.

Imagination as a fundamental function of the hippocampus

Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Sampling motion trajectories during hippocampal theta sequences

Efficient planning in complex environments requires that uncertainty associated with current inferences and possible consequences of forthcoming actions is represented. Representation of uncertainty has been established in sensory systems during simple perceptual decision making tasks but it remains unclear if complex cognitive computations such as planning and navigation are also supported by probabilistic neural representations. Here, we capitalized on gradually changing uncertainty along planned motion trajectories during hippocampal theta sequences to capture signatures of uncertainty representation in population responses. In contrast with prominent theories, we found no evidence of encoding parameters of probability distributions in the momentary population activity recorded in an open-field navigation task in rats. Instead, uncertainty was encoded sequentially by sampling motion trajectories randomly and efficiently in subsequent theta cycles from the distribution of potential trajectories. Our analysis is the first to demonstrate that the hippocampus is well equipped to contribute to optimal planning by representing uncertainty.

Do sparse brain activity patterns underlie human cognition?

Accumulating multivariate pattern analysis (MVPA) results from fMRI studies suggest that information is represented in fingerprint patterns of activations and deactivations during perception, emotions, and cognition. We postulate that these fingerprint patterns might reflect neuronal-population level sparse code documented in two-photon calcium imaging studies in animal models, i.e., information represented in specific and reproducible ensembles of a few percent of active neurons amidst widespread inhibition in neural populations. We suggest that such representations constitute a fundamental organizational principle via interacting across multiple levels of brain hierarchy, thus giving rise to perception, emotions, and cognition.

Internally Triggered Experiences of Hedonic Valence in Nonhuman Animals: Cognitive and Welfare Considerations

Do any nonhuman animals have hedonically valenced experiences not directly caused by stimuli in their current environment? Do they, like us humans, experience anticipated or previously experienced pains and pleasures as respectively painful and pleasurable? We review evidence from comparative neuroscience about hippocampus-dependent simulation in relation to this question. Hippocampal sharp-wave ripples and theta oscillations have been found to instantiate previous and anticipated experiences. These hippocampal activations coordinate with neural reward and fear centers as well as sensory and cortical areas in ways that are associated with conscious episodic mental imagery in humans. Moreover, such hippocampal “re- and preplay” has been found to contribute to instrumental decision making, the learning of value representations, and the delay of rewards in rats. The functional and structural features of hippocampal simulation are highly conserved across mammals. This evidence makes it reasonable to assume that internally triggered experiences of hedonic valence (IHVs) are pervasive across (at least) all mammals. This conclusion has important welfare implications. Most prominently, IHVs act as a kind of “welfare multiplier” through which the welfare impacts of any given experience of pain or pleasure are increased through each future retrieval. However, IHVs also have practical implications for welfare assessment and cause prioritization.

Multiple-Timescale Representations of Space: Linking Memory to Navigation

When navigating through space, we must maintain a representation of our position in real time; when recalling a past episode, a memory can come back in a flash. Interestingly, the brain's spatial representation system, including the hippocampus, supports these two distinct timescale functions. How are neural representations of space used in the service of both real-world navigation and internal mnemonic processes? Recent progress has identified sequences of hippocampal place cells, evolving at multiple timescales in accordance with either navigational behaviors or internal oscillations, that underlie these functions. We review experimental findings on experience-dependent modulation of these sequential representations and consider how they link real-world navigation to time-compressed memories. We further discuss recent work suggesting the prevalence of these sequences beyond hippocampus and propose that these multiple-timescale mechanisms may represent a general algorithm for organizing cell assemblies, potentially unifying the dual roles of the spatial representation system in memory and navigation.

Impact of optogenetic pulse design on CA3 learning and replay: A neural model

Optogenetic manipulation of hippocampal circuitry is an important tool for investigating learning in vivo. Numerous approaches to pulse design have been employed to elicit desirable circuit and behavioral outcomes. Here, we systematically test the outcome of different single-pulse waveforms in a rate-based model of hippocampal memory function at the level of mnemonic replay extension and de novo synaptic weight formation in CA3 and CA1. Lower-power waveforms with long forward or forward and backward ramps yield more natural sequence replay dynamics and induce synaptic plasticity that allows for more natural memory replay timing, in contrast to square or backward ramps. These differences between waveform shape and amplitude are preserved with the addition of noise in membrane potential, light scattering, and protein expression, improving the potential validity of predictions for in vivo work. These results inform future optogenetic experimental design choices in the field of learning and memory.

Spatial Learning Drives Rapid Goal Representation in Hippocampal Ripples without Place Field Accumulation or Goal-Oriented Theta Sequences

The hippocampus is critical for rapid acquisition of many forms of memory, although the circuit-level mechanisms through which the hippocampus rapidly consolidates novel information are unknown. Here, the activity of large ensembles of hippocampal neurons in adult male Long-Evans rats was monitored across a period of rapid spatial learning to assess how the network changes during the initial phases of memory formation and retrieval. In contrast to several reports, the hippocampal network did not display enhanced representation of the goal location via accumulation of place fields or elevated firing rates at the goal. Rather, population activity rates increased globally as a function of experience. These alterations in activity were mirrored in the power of the theta oscillation and in the quality of theta sequences, without preferential encoding of paths to the learned goal location. In contrast, during brief "offline" pauses in movement, representation of a novel goal location emerged rapidly in ripples, preceding other changes in network activity. These data demonstrate that the hippocampal network can facilitate active navigation without enhanced goal representation during periods of active movement, and further indicate that goal representation in hippocampal ripples before movement onset supports subsequent navigation, possibly through activation of downstream cortical networks.SIGNIFICANCE STATEMENT Understanding the mechanisms through which the networks of the brain rapidly assimilate information and use previously learned knowledge are fundamental areas of focus in neuroscience. In particular, the hippocampal circuit is a critical region for rapid formation and use of spatial memory. In this study, several circuit-level features of hippocampal function were quantified while rats performed a spatial navigation task requiring rapid memory formation and use. During periods of active navigation, a general increase in overall network activity is observed during memory acquisition, which plateaus during memory retrieval periods, without specific enhanced representation of the goal location. During pauses in navigation, rapid representation of the distant goal well emerges before either behavioral improvement or changes in online activity.

Discriminating Sleep From Freezing With Cortical Spindle Oscillations

In-vivo longitudinal recordings require reliable means to automatically discriminate between distinct behavioral states, in particular between awake and sleep epochs. The typical approach is to use some measure of motor activity together with extracellular electrophysiological signals, namely the relative contribution of theta and delta frequency bands to the Local Field Potential (LFP). However, these bands can partially overlap with oscillations characterizing other behaviors such as the 4 Hz accompanying rodent freezing. Here, we first demonstrate how standard methods fail to discriminate between sleep and freezing in protocols where both behaviors are observed. Then, as an alternative, we propose to use the smoothed cortical spindle power to detect sleep epochs. Finally, we show the effectiveness of this method in discriminating between sleep and freezing in our recordings.

Goal discrimination in hippocampal nonplace cells when place information is ambiguous

SignificanceGoal-directed spatial navigation has been found to rely on hippocampal neurons that are spatially modulated. We show that "nonplace" cells without significant spatial modulation play a role in discriminating goals when environmental cues for goals are ambiguous. This nonplace cell activity is performance-dependent and is modulated by gamma oscillations. Finally, nonplace cell goal discrimination coding fails in a mouse model of Alzheimer's disease (AD). Together, these results show that nonplace cell firing can signal unique task-relevant information when spatial information is ambiguous; these signals depend on performance and are absent in a mouse model of AD.

Decoding cognition from spontaneous neural activity

In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

Modulation of lateral septal and dorsomedial striatal neurons by hippocampal sharp-wave ripples, theta rhythm, and running speed

Single units were recorded in hippocampus, lateral septum (LS), and dorsomedial striatum (DMS) while freely behaving rats (n = 3) ran trials in a T‐maze task and rested in a holding bucket between trials. In LS, 28% (64/226) of recorded neurons were excited and 14% (31/226) were inhibited during sharp wave ripples (SWRs). LS neurons that were excited during SWRs fired preferentially on the downslope of hippocampal theta rhythm and had firing rates that were positively correlated with running speed; LS neurons that were inhibited during SWRs fired preferentially on the upslope of hippocampal theta rhythm and had firing rates that were negatively correlated with running speed. In DMS, only 3.3% (12/366) of recorded neurons were excited and 5.7% (21/366) were inhibited during SWRs. As in LS, DMS neurons that were excited by SWRs tended to have firing rates that were positively modulated by running speed, whereas DMS neurons that were inhibited by SWRs tended to have firing rates that were negatively modulated by running speed. But in contrast with LS, these two DMS subpopulations did not clearly segregate their spikes to different phases of the theta cycle. Based on these results and a review of prior findings, we discuss how concurrent activation of spatial trajectories in hippocampus and motor representations in LS and DMS may contribute to neural computations that support reinforcement learning and value‐based decision making.

The learning of prospective and retrospective cognitive maps within neural circuits

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.

CA2 inhibition reduces the precision of hippocampal assembly reactivation

The structured reactivation of hippocampal neuronal ensembles during fast synchronous oscillatory events, termed sharp-wave ripples (SWRs), has been suggested to play a crucial role in the storage and use of memory. Activity in both the CA2 and CA3 subregions can precede this population activity in CA1, and chronic inhibition of either region alters SWR oscillations. However, the precise contribution of CA2 to the oscillation, as well as to the reactivation of CA1 neurons within it, remains unclear. Here, we employ chemogenetics to transiently silence CA2 pyramidal cells in mice, and we observe that although SWRs still occur, the reactivation of CA1 pyramidal cell ensembles within the events lose both temporal and informational precision. These observations suggest that CA2 activity contributes to the fidelity of experience-dependent hippocampal replay.

Ripple band phase precession of place cell firing during replay

Neuronal “replay,” in which place cell firing during rest recapitulates recently experienced trajectories, is thought to mediate the transmission of information from hippocampus to neocortex, but the mechanism for this transmission is unknown. Here, we show that replay uses a phase code to represent spatial trajectories by the phase of firing relative to the 150- to 250-Hz “ripple” oscillations that accompany replay events. This phase code is analogous to the theta phase precession of place cell firing during navigation, in which place cells fire at progressively earlier phases of the 6- to 12-Hz theta oscillation as their place field is traversed, providing information about self-location that is additional to the rate code and a necessary precursor of replay. Thus, during replay, each ripple cycle contains a “forward sweep” of decoded locations along the recapitulated trajectory. Our results indicate a novel encoding of trajectory information during replay and implicates phase coding as a general mechanism by which the hippocampus transmits experienced and replayed sequential information to downstream targets.

•

Place cells fire at successively earlier ripple band phases during replay

•

Ripple band firing phase during replay encodes location within the place field

•

This produces forward sweeps of place cell activity during each ripple cycle

Skipping ahead: A circuit for representing the past, present, and future

Envisioning the future is intuitively linked to our ability to remember the past. Within the memory system, substantial work has demonstrated the involvement of the prefrontal cortex and the hippocampus in representing the past and present. Recent data shows that both the prefrontal cortex and the hippocampus encode future trajectories, which are segregated in time by alternating cycles of the theta rhythm. Here, we discuss how information is temporally organized by these brain regions supported by the medial septum, nucleus reuniens, and parahippocampal regions. Finally, we highlight a brain circuit that we predict is essential for the temporal segregation of future scenarios.

Hippocampal replay of experience at real-world speeds

Representations related to past experiences play a critical role in memory and decision-making processes. The rat hippocampus expresses these types of representations during sharp-wave ripple (SWR) events, and previous work identified a minority of SWRs that contain ‘replay’ of spatial trajectories at ∼20x the movement speed of the animal. Efforts to understand replay typically make multiple assumptions about which events to examine and what sorts of representations constitute replay. We therefore lack a clear understanding of both the prevalence and the range of representational dynamics associated with replay. Here, we develop a state space model that uses a combination of movement dynamics of different speeds to capture the spatial content and time evolution of replay during SWRs. Using this model, we find that the large majority of replay events contain spatially coherent, interpretable content. Furthermore, many events progress at real-world, rather than accelerated, movement speeds, consistent with actual experiences.