Decoding the cognitive map: ensemble hippocampal sequences and decision making

A recurrent network model of planning explains hippocampal replay and human behavior

When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.

Working memory features are embedded in hippocampal place fields

Hippocampal principal neurons display both spatial tuning properties and memory features. Whether this distinction corresponds to separate neuron types or a context-dependent continuum has been debated. We report here that the task-context ("splitter") feature is highly variable along both trial and spatial position axes. Neurons acquire or lose splitter features across trials even when place field features remain unaltered. Multiple place fields of the same neuron can individually encode both past or future run trajectories, implying that splitter fields are under the control of assembly activity. Place fields can be differentiated into subfields by the behavioral choice of the animal, and splitting within subfields evolves across trials. Interneurons also differentiate choices by integrating inputs from pyramidal cells. Finally, bilateral optogenetic inactivation of the medial entorhinal cortex reversibly decreases the fraction of splitter fields. Our findings suggest that place or splitter features are different manifestations of the same hippocampal computation.

Significance of the anterior cingulate cortex in neurogenesis plasticity: Connections, functions, and disorders across postnatal and adult stages

The anterior cingulate cortex (ACC) is a complex and continually evolving brain region that remains a primary focus of research due to its multifaceted functions. Various studies and analyses have significantly advanced our understanding of how the ACC participates in a wide spectrum of memory and cognitive processes. However, despite its strong connections to brain areas associated with hippocampal and olfactory neurogenesis, the functions of the ACC in regulating postnatal and adult neurogenesis in these regions are still insufficiently explored. Investigating the intricate involvement of the ACC in neurogenesis could enhance our comprehension of essential aspects of brain plasticity. This involvement stems from its complex circuitry with other relevant brain regions, thereby exerting both direct and indirect impacts on the neurogenesis process. This review sheds light on the promising significance of the ACC in orchestrating postnatal and adult neurogenesis in conditions related to memory, cognitive behavior, and associated disorders.

New information triggers prospective codes to adapt for flexible navigation

Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a novel task that precisely introduces new information using virtual reality, and we recorded neural activity as mice flexibly adapted their planned destinations. We found that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not flexibly adapt, prefrontal choice codes failed to switch, despite relatively intact hippocampal goal representations. Prospective code updating depended on the commitment to the initial choice and degree of adaptation needed. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.

Associative and predictive hippocampal codes support memory-guided behaviors

Episodic memory involves learning and recalling associations between items and their spatiotemporal context. Those memories can be further used to generate internal models of the world that enable predictions to be made. The mechanisms that support these associative and predictive aspects of memory are not yet understood. In this study, we used an optogenetic manipulation to perturb the sequential structure, but not global network dynamics, of place cells as rats traversed specific spatial trajectories. This perturbation abolished replay of those trajectories and the development of predictive representations, leading to impaired learning of new optimal trajectories during memory-guided navigation. However, place cell assembly reactivation and reward-context associative learning were unaffected. Our results show a mechanistic dissociation between two complementary hippocampal codes: an associative code (through coactivity) and a predictive code (through sequences).

Linking temporal coordination of hippocampal activity to memory function

Oscillations in neural activity are widespread throughout the brain and can be observed at the population level through the local field potential. These rhythmic patterns are associated with cycles of excitability and are thought to coordinate networks of neurons, in turn facilitating effective communication both within local circuits and across brain regions. In the hippocampus, theta rhythms (4-12 Hz) could contribute to several key physiological mechanisms including long-range synchrony, plasticity, and at the behavioral scale, support memory encoding and retrieval. While neurons in the hippocampus appear to be temporally coordinated by theta oscillations, they also tend to fire in sequences that are developmentally preconfigured. Although loss of theta rhythmicity impairs memory, these sequences of spatiotemporal representations persist in conditions of altered hippocampal oscillations. The focus of this review is to disentangle the relative contribution of hippocampal oscillations from single-neuron activity in learning and memory. We first review cellular, anatomical, and physiological mechanisms underlying the generation and maintenance of hippocampal rhythms and how they contribute to memory function. We propose candidate hypotheses for how septohippocampal oscillations could support memory function while not contributing directly to hippocampal sequences. In particular, we explore how theta rhythms could coordinate the integration of upstream signals in the hippocampus to form future decisions, the relevance of such integration to downstream regions, as well as setting the stage for behavioral timescale synaptic plasticity. Finally, we leverage stimulation-based treatment in Alzheimer's disease conditions as an opportunity to assess the sufficiency of hippocampal oscillations for memory function.

Enhanced Reactivation of Remapping Place Cells during Aversive Learning

Study of the hippocampal place cell system has greatly enhanced our understanding of memory encoding for distinct places, but how episodic memories for distinct experiences occurring within familiar environments are encoded is less clear. We developed a spatial decision-making task in which male rats learned to navigate a multiarm maze to a goal location for food reward while avoiding maze arms in which aversive stimuli were delivered. Task learning induced partial remapping in CA1 place cells, allowing us to identify both remapping and stable cell populations. Remapping cells were recruited into sharp-wave ripples and associated replay events to a greater extent than stable cells, despite having similar firing rates during navigation of the maze. Our results suggest that recruitment into replay events may be a mechanism to incorporate new contextual information into a previously formed and stabilized spatial representation.SIGNIFICANCE STATEMENT Hippocampal place cells provide a map of space that animals use to navigate. This map can change to reflect changes in the physical properties of the environment in which the animal finds itself, and also in response to nonphysical contextual changes, such as changes in the valence of specific locations within that environment. We show here that cells which change their spatial tuning after a change in context are preferentially recruited into sharp-wave ripple-associated replay events compared with stable nonremapping cells. Thus, our data lend strong support to the hypothesis that replay is a mechanism for the storage of new spatial maps.

Place cells are nonrandomly clustered by field location in CA1 hippocampus

A challenge in both modern and historic neuroscience has been achieving an understanding of neuron circuits, and determining the computational and organizational principles that underlie these circuits. Deeper understanding of the organization of brain circuits and cell types, including in the hippocampus, is required for advances in behavioral and cognitive neuroscience, as well as for understanding principles governing brain development and evolution. In this manuscript, we pioneer a new method to analyze the spatial clustering of active neurons in the hippocampus. We use calcium imaging and a rewarded navigation task to record from 100 s of place cells in the CA1 of freely moving rats. We then use statistical techniques developed for and in widespread use in geographic mapping studies, global Moran's I, and local Moran's I to demonstrate that cells that code for similar spatial locations tend to form small spatial clusters. We present evidence that this clustering is not the result of artifacts from calcium imaging, and show that these clusters are primarily formed by cells that have place fields around previously rewarded locations. We go on to show that, although cells with similar place fields tend to form clusters, there is no obvious topographic mapping of environmental location onto the hippocampus, such as seen in the visual cortex. Insights into hippocampal organization, as in this study, can elucidate mechanisms underlying motivational behaviors, spatial navigation, and memory formation.

Imagination as a fundamental function of the hippocampus

Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Artificial intelligence insights into hippocampal processing

Advances in artificial intelligence, machine learning, and deep neural networks have led to new discoveries in human and animal learning and intelligence. A recent artificial intelligence agent in the DeepMind family, muZero, can complete a variety of tasks with limited information about the world in which it is operating and with high uncertainty about features of current and future space. To perform, muZero uses only three functions that are general yet specific enough to allow learning across a variety of tasks without overgeneralization across different contexts. Similarly, humans and animals are able to learn and improve in complex environments while transferring learning from other contexts and without overgeneralizing. In particular, the mammalian extrahippocampal system (eHPCS) can guide spatial decision making while simultaneously encoding and processing spatial and contextual information. Like muZero, the eHPCS is also able to adjust contextual representations depending on the degree and significance of environmental changes and environmental cues. In this opinion, we will argue that the muZero functions parallel those of the hippocampal system. We will show that the different components of the muZero model provide a framework for thinking about generalizable learning in the eHPCS, and that the evaluation of how transitions in cell representations occur between similar and distinct contexts can be informed by advances in artificial intelligence agents such as muZero. We additionally explain how advances in AI agents will provide frameworks and predictions by which to investigate the expected link between state changes and neuronal firing. Specifically, we will discuss testable predictions about the eHPCS, including the functions of replay and remapping, informed by the mechanisms behind muZero learning. We conclude with additional ways in which agents such as muZero can aid in illuminating prospective questions about neural functioning, as well as how these agents may shed light on potential expected answers.

Reversal learning: It's just a phase

Being able to let go of behaviors that are no longer valuable and adopt actions that achieve the same outcome is fundamental for animal survival. A new study offers clues on the neural mechanisms that allow animals to reverse their behavior as needed.

Adult hippocampal neurogenesis shapes adaptation and improves stress response: a mechanistic and integrative perspective

Adult hippocampal neurogenesis (AHN) represents a remarkable form of neuroplasticity that has increasingly been linked to the stress response in recent years. However, the hippocampus does not itself support the expression of the different dimensions of the stress response. Moreover, the main hippocampal functions are essentially preserved under AHN depletion and adult-born immature neurons (abGNs) have no extrahippocampal projections, which questions the mechanisms by which abGNs influence functions supported by brain areas far from the hippocampus. Within this framework, we propose that through its computational influences AHN is pivotal in shaping adaption to environmental demands, underlying its role in stress response. The hippocampus with its high input convergence and output divergence represents a computational hub, ideally positioned in the brain (1) to detect cues and contexts linked to past, current and predicted stressful experiences, and (2) to supervise the expression of the stress response at the cognitive, affective, behavioral, and physiological levels. AHN appears to bias hippocampal computations toward enhanced conjunctive encoding and pattern separation, promoting contextual discrimination and cognitive flexibility, reducing proactive interference and generalization of stressful experiences to safe contexts. These effects result in gating downstream brain areas with more accurate and contextualized information, enabling the different dimensions of the stress response to be more appropriately set with specific contexts. Here, we first provide an integrative perspective of the functional involvement of AHN in the hippocampus and a phenomenological overview of the stress response. We then examine the mechanistic underpinning of the role of AHN in the stress response and describe its potential implications in the different dimensions accompanying this response.

Entrainment of Astrocytic and Neuronal Ca2+ Population Dynamics During Information Processing of Working Memory in Mice

Astrocytes are increasingly recognized to play an active role in learning and memory, but whether neural inputs can trigger event-specific astrocytic Ca²⁺ dynamics in real time to participate in working memory remains unclear due to the difficulties in directly monitoring astrocytic Ca²⁺ dynamics in animals performing tasks. Here, using fiber photometry, we showed that population astrocytic Ca²⁺ dynamics in the hippocampus were gated by sensory inputs (centered at the turning point of the T-maze) and modified by the reward delivery during the encoding and retrieval phases. Notably, there was a strong inter-locked and antagonistic relationship between the astrocytic and neuronal Ca²⁺ dynamics with a 3-s phase difference. Furthermore, there was a robust synchronization of astrocytic Ca²⁺ at the population level among the hippocampus, medial prefrontal cortex, and striatum. The inter-locked, bidirectional communication between astrocytes and neurons at the population level may contribute to the modulation of information processing in working memory.

Replay in minds and machines

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Parcellation of the Hippocampus According to Its Connection Probability with Prefrontal Cortex Subdivisions in a Malaysian Malay Population: Preliminary Findings

Background

Lesion studies have shown distinct roles for the hippocampus, with the dorsal subregion being involved in processing of spatial information and memory, and the ventral aspect coding for emotion and motivational behaviour. However, its structural connectivity with the subdivisions of the prefrontal cortex (PFC), the executive area of the brain that also has various distinct functions, has not been fully explored, especially in the Malaysian population.

Methods

We performed diffusion magnetic resonance imaging with probabilistic tractography on four Malay males to parcellate the hippocampus according to its relative connection probability to the six subdivisions of the PFC.

Results

Our findings revealed that each hippocampus showed putative connectivity to all the subdivisions of PFC, with the highest connectivity to the orbitofrontal cortex (OFC). Parcellation of the hippocampus according to its connection probability to the six PFC subdivisions showed variability in the pattern of the connection distribution and no clear distinction between the hippocampal subregions.

Conclusion

Hippocampus displayed highest connectivity to the OFC as compared to other PFC subdivisions. We did not find a unifying pattern of distribution based on the connectivity-based parcellation of the hippocampus.

Flexible modulation of sequence generation in the entorhinal-hippocampal system

Exploration, consolidation and planning depend on the generation of sequential state representations. However, these algorithms require disparate forms of sampling dynamics for optimal performance. We theorize how the brain should adapt internally generated sequences for particular cognitive functions and propose a neural mechanism by which this may be accomplished within the entorhinal-hippocampal circuit. Specifically, we demonstrate that the systematic modulation along the medial entorhinal cortex dorsoventral axis of grid population input into the hippocampus facilitates a flexible generative process that can interpolate between qualitatively distinct regimes of sequential hippocampal reactivations. By relating the emergent hippocampal activity patterns drawn from our model to empirical data, we explain and reconcile a diversity of recently observed, but apparently unrelated, phenomena such as generative cycling, diffusive hippocampal reactivations and jumping trajectory events.

The overfitted brain: Dreams evolved to assist generalization

Understanding of the evolved biological function of sleep has advanced considerably in the past decade. However, no equivalent understanding of dreams has emerged. Contemporary neuroscientific theories often view dreams as epiphenomena, and many of the proposals for their biological function are contradicted by the phenomenology of dreams themselves. Now, the recent advent of deep neural networks (DNNs) has finally provided the novel conceptual framework within which to understand the evolved function of dreams. Notably, all DNNs face the issue of overfitting as they learn, which is when performance on one dataset increases but the network's performance fails to generalize (often measured by the divergence of performance on training versus testing datasets). This ubiquitous problem in DNNs is often solved by modelers via "noise injections" in the form of noisy or corrupted inputs. The goal of this paper is to argue that the brain faces a similar challenge of overfitting and that nightly dreams evolved to combat the brain's overfitting during its daily learning. That is, dreams are a biological mechanism for increasing generalizability via the creation of corrupted sensory inputs from stochastic activity across the hierarchy of neural structures. Sleep loss, specifically dream loss, leads to an overfitted brain that can still memorize and learn but fails to generalize appropriately. Herein this "overfitted brain hypothesis" is explicitly developed and then compared and contrasted with existing contemporary neuroscientific theories of dreams. Existing evidence for the hypothesis is surveyed within both neuroscience and deep learning, and a set of testable predictions is put forward that can be pursued both in vivo and in silico.

When will's wont wants wanting

We use neural reinforcement learning concepts including Pavlovian versus instrumental control, liking versus wanting, model-based versus model-free control, online versus offline learning and planning, and internal versus external actions and control to reflect on putative conflicts between short-term temptations and long-term goals.

Neuromodulation of the mind-wandering brain state: the interaction between neuromodulatory tone, sharp wave-ripples and spontaneous thought

Mind-wandering has become a captivating topic for cognitive neuroscientists. By now, it is reasonably well described in terms of its phenomenology and the large-scale neural networks that support it. However, we know very little about what neurobiological mechanisms trigger a mind-wandering episode and sustain the mind-wandering brain state. Here, we focus on the role of ascending neuromodulatory systems (i.e. acetylcholine, noradrenaline, serotonin and dopamine) in shaping mind-wandering. We advance the hypothesis that the hippocampal sharp wave-ripple (SWR) is a compelling candidate for a brain state that can trigger mind-wandering episodes. This hippocampal rhythm, which occurs spontaneously in quiescent behavioural states, is capable of propagating widespread activity in the default network and is functionally associated with recollective, associative, imagination and simulation processes. The occurrence of the SWR is heavily dependent on hippocampal neuromodulatory tone. We describe how the interplay of neuromodulators may promote the hippocampal SWR and trigger mind-wandering episodes. We then identify the global neuromodulatory signatures that shape the evolution of the mind-wandering brain state. Under our proposed framework, mind-wandering emerges due to the interplay between neuromodulatory systems that influence the transitions between brain states, which either facilitate, or impede, a wandering mind. This article is part of the theme issue 'Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation'.

An Integrated Index: Engrams, Place Cells, and Hippocampal Memory

The hippocampus and its extended network contribute to encoding and recall of episodic experiences. Drawing from recent anatomical, physiological, and behavioral studies, we propose that hippocampal engrams function as indices to mediate memory recall. We broaden this idea to discuss potential relationships between engrams and hippocampal place cells, as well as the molecular, cellular, physiological, and circuit determinants of engrams that permit flexible routing of information to intra- and extrahippocampal circuits for reinstatement, a feature critical to memory indexing. Incorporating indexing into frameworks of memory function opens new avenues of study and even therapies for hippocampal dysfunction.

Mental imagery in animals: Learning, memory, and decision-making in the face of missing information

When we open our eyes, we see a world filled with objects and events. Yet, due to occlusion of some objects by others, we only have partial perceptual access to the events that transpire around us. I discuss the body of research on mental imagery in animals. I first cover prior studies of mental rotation in pigeons and imagery using working memory procedures first developed for human studies. Next, I discuss the seminal work on a type of learning called mediated conditioning in rats. I then provide more in-depth coverage of work from my lab suggesting that rats can use imagery to fill in missing details of the world that are expected but hidden from perception. We have found that rats make use of an active expectation (i.e., an image) of a hidden visual event. I describe the behavioral and neurobiological studies investigating the use of a mental image, its theoretical basis, and its connections to current human cognitive neuroscience research on episodic memory, imagination, and mental simulations. Collectively, the reviewed literature provides insight into the mechanisms that mediate the flexible use of an image during ambiguous situations. I position this work in the broader scientific and philosophical context surrounding the concept of mental imagery in human and nonhuman animals.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

Replay as wavefronts and theta sequences as bump oscillations in a grid cell attractor network

Grid cells fire in sequences that represent rapid trajectories in space. During locomotion, theta sequences encode sweeps in position starting slightly behind the animal and ending ahead of it. During quiescence and slow wave sleep, bouts of synchronized activity represent long trajectories called replays, which are well-established in place cells and have been recently reported in grid cells. Theta sequences and replay are hypothesized to facilitate many cognitive functions, but their underlying mechanisms are unknown. One mechanism proposed for grid cell formation is the continuous attractor network. We demonstrate that this established architecture naturally produces theta sequences and replay as distinct consequences of modulating external input. Driving inhibitory interneurons at the theta frequency causes attractor bumps to oscillate in speed and size, which gives rise to theta sequences and phase precession, respectively. Decreasing input drive to all neurons produces traveling wavefronts of activity that are decoded as replays.

Dopamine neuron ensembles signal the content of sensory prediction errors

Dopamine neurons respond to errors in predicting value-neutral sensory information. These data, combined with causal evidence that dopamine transients support sensory-based associative learning, suggest that the dopamine system signals a multidimensional prediction error. Yet such complexity is not evident in the activity of individual neurons or population averages. How then do downstream areas know what to learn in response to these signals? One possibility is that information about content is contained in the pattern of firing across many dopamine neurons. Consistent with this, here we show that the pattern of firing across a small group of dopamine neurons recorded in rats signals the identity of a mis-predicted sensory event. Further, this same information is reflected in the BOLD response elicited by sensory prediction errors in human midbrain. These data provide evidence that ensembles of dopamine neurons provide highly specific teaching signals, opening new possibilities for how this system might contribute to learning.

Hippocampal BDNF regulates a shift from flexible, goal-directed to habit memory system function following cocaine abstinence

The transition from recreational drug use to addiction involves pathological learning processes that support a persistent shift from flexible, goal-directed to habit behavioral control. Here, we examined the molecular mechanisms supporting altered function in hippocampal (HPC) and dorsolateral striatum (DLS) memory systems following abstinence from repeated cocaine. After 3 weeks of cocaine abstinence (experimenter- or self-administered), we tested new behavioral learning in male rats using a dual-solution maze task, which provides an unbiased approach to assess HPC- versus DLS-dependent learning strategies. Dorsal hippocampus (dHPC) and DLS brain tissues were collected after memory testing to identify transcriptional adaptations associated with cocaine-induced shifts in behavioral learning. Our results demonstrate that following prolonged cocaine abstinence rats show a bias toward the use of an inflexible, habit memory system (DLS) in lieu of a more flexible, easily updated memory system involving the HPC. This memory system bias was associated with upregulation and downregulation of brain-derived neurotrophic factor (BDNF) gene expression and transcriptionally permissive histone acetylation (acetylated histone H3, AcH3) in the DLS and dHPC, respectively. Using viral-mediated gene transfer, we overexpressed BDNF in the dHPC during cocaine abstinence and new maze learning. This manipulation restored HPC-dependent behavioral control. These findings provide a system-level understanding of altered plasticity and behavioral learning following cocaine abstinence and inform mechanisms mediating the organization of learning and memory more broadly.

Awake Reactivation of Prior Experiences Consolidates Memories and Biases Cognition

After experiences are encoded into memory, post-encoding reactivation mechanisms have been proposed to mediate long-term memory stabilization and transformation. Spontaneous reactivation of hippocampal representations, together with hippocampal-cortical interactions, are leading candidate mechanisms for promoting systems-level memory strengthening and reorganization. While the replay of spatial representations has been extensively studied in rodents, here we review recent fMRI work that provides evidence for spontaneous reactivation of nonspatial, episodic event representations in the human hippocampus and cortex, as well as for experience-dependent alterations in systems-level hippocampal connectivity. We focus on reactivation during awake post-encoding periods, relationships between reactivation and subsequent behavior, how reactivation is modulated by factors that influence consolidation, and the implications of persistent reactivation for biasing ongoing perception and cognition.

Are We There Yet? Identification of Reward-Selective Cells in the Hippocampus

Navigation to a previously visited reward site requires a reliable and accurate spatial memory. In this issue of Neuron, Gauthier and Tank (2018) use two-photon calcium imaging to uncover a discrete hippocampal subpopulation specialized for encoding reward location.

Hippocampal Sequences During Exploration: Mechanisms and Functions

Although the hippocampus plays a critical role in spatial and episodic memories, the mechanisms underlying memory formation, stabilization, and recall for adaptive behavior remain relatively unknown. During exploration, within single cycles of the ongoing theta rhythm that dominates hippocampal local field potentials, place cells form precisely ordered sequences of activity. These neural sequences result from the integration of both external inputs conveying sensory-motor information, and intrinsic network dynamics possibly related to memory processes. Their endogenous replay during subsequent sleep is critical for memory consolidation. The present review discusses possible mechanisms and functions of hippocampal theta sequences during exploration. We present several lines of evidence suggesting that these neural sequences play a key role in information processing and support the formation of initial memory traces, and discuss potential functional distinctions between neural sequences emerging during theta vs. awake sharp-wave ripples.

Pattern separation of spiketrains in hippocampal neurons

Pattern separation is a process that minimizes overlap between patterns of neuronal activity representing similar experiences. Theoretical work suggests that the dentate gyrus (DG) performs this role for memory processing but a direct demonstration is lacking. One limitation is the difficulty to measure DG inputs and outputs simultaneously. To rigorously assess pattern separation by DG circuitry, we used mouse brain slices to stimulate DG afferents and simultaneously record DG granule cells (GCs) and interneurons. Output spiketrains of GCs are more dissimilar than their input spiketrains, demonstrating for the first time temporal pattern separation at the level of single neurons in the DG. Pattern separation is larger in GCs than in fast-spiking interneurons and hilar mossy cells, and is amplified in CA3 pyramidal cells. Analysis of the neural noise and computational modelling suggest that this form of pattern separation is not explained by simple randomness and arises from specific presynaptic dynamics. Overall, by reframing the concept of pattern separation in dynamic terms and by connecting it to the physiology of different types of neurons, our study offers a new window of understanding in how hippocampal networks might support episodic memory.

Age-associated changes in waking hippocampal sharp-wave ripples

Hippocampal sharp-wave ripples are brief high-frequency (120-250 Hz) oscillatory events that support mnemonic processes during sleep and awake behavior. Although ripples occurring during sleep are believed to facilitate memory consolidation, waking ripples may also be involved in planning and memory retrieval. Recent work from our group determined that normal aging results in a significant reduction in the peak oscillatory frequency and rate-of-occurrence of ripples during sleep that may contribute to age-associated memory decline. It is unknown, however, how aging alters waking ripples. We investigated whether characteristics of waking ripples undergo age-dependent changes. Sharp-wave ripple events were recorded from the CA1 region of the hippocampus in old (n = 5) and young (n = 6) F344 male rats as they performed a place-dependent eyeblink conditioning task. Several novel observations emerged from this analysis. First, although aged rats expressed more waking ripples than young rats during track running and reward consumption, this effect was eliminated, and, in the case of track-running, reversed when time spent in each location was accounted for. Thus, aged rats emit more ripples, but young rats express a higher ripple rate. This likely results from reduced locomotor activity in aged animals. Furthermore, although ripple rates increased as young rats approached rewards, rates did not increase in aged rats, and rates in aged and young animals were not affected by eyeblink conditioning. Finally, although the oscillatory frequency of ripples was lower in aged animals during rest, frequencies in aged rats increased during behavior to levels indistinguishable from young rats. Given the involvement of waking ripples in memory retrieval, a possible consequence of slower movement speeds of aged animals is to provide more opportunity to replay task-relevant information and compensate for age-related declines in ripple rate during task performance.

On How the Dentate Gyrus Contributes to Memory Discrimination

The dentate gyrus (DG) is crucial for behaviorally discriminating similar spatial memories, predicting that DG place cells change ("remap") their relative spatial tuning ("place fields") for memory discrimination. This prediction was never tested, although DG place cells remap across similar environments without memory tasks. We confirm this prior finding but find that DG place fields do not remap across spatial tasks that require DG-dependent memory discrimination. Instead of remapping, place-discriminating discharge is observed transiently among DG place cells, particularly when memory discrimination is most necessary. The DG network may signal memory discrimination by expressing distinctive sub-second network patterns of co-firing at memory discrimination sites. This involves increased coupling of discharge from place cells and interneurons, as was observed during successful, but not failed, behavioral expression of memory discrimination. Instead of remapping, these findings indicate that memory discrimination is signaled by sub-second patterns of correlated discharge within the dentate network.

What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated

We update complementary learning systems (CLS) theory, which holds that intelligent agents must possess two learning systems, instantiated in mammalians in neocortex and hippocampus. The first gradually acquires structured knowledge representations while the second quickly learns the specifics of individual experiences. We broaden the role of replay of hippocampal memories in the theory, noting that replay allows goal-dependent weighting of experience statistics. We also address recent challenges to the theory and extend it by showing that recurrent activation of hippocampal traces can support some forms of generalization and that neocortical learning can be rapid for information that is consistent with known structure. Finally, we note the relevance of the theory to the design of artificial intelligent agents, highlighting connections between neuroscience and machine learning.

The transformation of multi-sensory experiences into memories during sleep

Our everyday lives present us with a continuous stream of multi-modal sensory inputs. While most of this information is soon forgotten, sensory information associated with salient experiences can leave long-lasting memories in our minds. Extensive human and animal research has established that the hippocampus is critically involved in this process of memory formation and consolidation. However, the underlying mechanistic details are still only partially understood. Specifically, the hippocampus has often been suggested to encode information during experience, temporarily store it, and gradually transfer this information to the cortex during sleep. In rodents, ample evidence has supported this notion in the context of spatial memory, yet whether this process adequately describes the consolidation of multi-sensory experiences into memories is unclear. Here, focusing on rodent studies, I examine how multi-sensory experiences are consolidated into long term memories by hippocampal and cortical circuits during sleep. I propose that in contrast to the classical model of memory consolidation, the cortex is a "fast learner" that has a rapid and instructive role in shaping hippocampal-dependent memory consolidation. The proposed model may offer mechanistic insight into memory biasing using sensory cues during sleep.

Cognitive Maps to Visualise Clinical Cases in Handovers

Background: Clinical handovers at changes of shifts are typical scenarios of time restricted and information intensive communication, which are highly cognitively demanding. The currently available applications supporting handovers typically present complex information in a textual checklist-like manner. This presentation style has been criticised for not meeting the specific user requirements.

Objectives: We, therefore, aimed at developing a concept for visualising the overview of a clinical case that serves as an alternative way to checklist-like presentations in clinical handovers. We also aimed at implementing this concept in a handoverEHR in order to support the pre-handover phase, the actual handover, and the post-handover phase as well as at evaluating its usability and attractiveness.

Results: We developed and implemented a concept that draws on Tolman’s pioneering work on cognitive maps that we designed in accordance with Gestalt principles. These maps provide a pictorial overview of a clinical case. The application to build, manipulate, and store the cognitive maps was integrated into an openEHR based handover record that extends conventional records with handover specific information. Usability (n = 28) and attractiveness (n = 26) testing with experienced clinicians resulted in good ratings for suitability for the task as well as for attractiveness and pragmatism.

Conclusion: We propose cognitive maps to represent and visualise the clinical case in situations where there is limited time to present complex information.

The role of replay and theta sequences in mediating hippocampal-prefrontal interactions for memory and cognition

Sequential activity is seen in the hippocampus during multiple network patterns, prominently as replay activity during both awake and sleep sharp-wave ripples (SWRs), and as theta sequences during active exploration. Although various mnemonic and cognitive functions have been ascribed to these hippocampal sequences, evidence for these proposed functions remains primarily phenomenological. Here, we briefly review current knowledge about replay events and theta sequences in spatial memory tasks. We reason that in order to gain a mechanistic and causal understanding of how these patterns influence memory and cognitive processing, it is important to consider how these sequences influence activity in other regions, and in particular, the prefrontal cortex, which is crucial for memory-guided behavior. For spatial memory tasks, we posit that hippocampal-prefrontal interactions mediated by replay and theta sequences play complementary and overlapping roles at different stages in learning, supporting memory encoding and retrieval, deliberative decision making, planning, and guiding future actions. This framework offers testable predictions for future physiology and closed-loop feedback inactivation experiments for specifically targeting hippocampal sequences as well as coordinated prefrontal activity in different network states, with the potential to reveal their causal roles in memory-guided behavior.

Dendrites, deep learning, and sequences in the hippocampus

The hippocampus places us both in time and space. It does so over remarkably large spans: milliseconds to years, and centimeters to kilometers. This works for sensory representations, for memory, and for behavioral context. How does it fit in such wide ranges of time and space scales, and keep order among the many dimensions of stimulus context? A key organizing principle for a wide sweep of scales and stimulus dimensions is that of order in time, or sequences. Sequences of neuronal activity are ubiquitous in sensory processing, in motor control, in planning actions, and in memory. Against this strong evidence for the phenomenon, there are currently more models than definite experiments about how the brain generates ordered activity. The flip side of sequence generation is discrimination. Discrimination of sequences has been extensively studied at the behavioral, systems, and modeling level, but again physiological mechanisms are fewer. It is against this backdrop that I discuss two recent developments in neural sequence computation, that at face value share little beyond the label "neural." These are dendritic sequence discrimination, and deep learning. One derives from channel physiology and molecular signaling, the other from applied neural network theory - apparently extreme ends of the spectrum of neural circuit detail. I suggest that each of these topics has deep lessons about the possible mechanisms, scales, and capabilities of hippocampal sequence computation.

On the Integration of Space, Time, and Memory

The hippocampus is famous for mapping locations in spatially organized environments, and several recent studies have shown that hippocampal networks also map moments in temporally organized experiences. Here I consider how space and time are integrated in the representation of memories. The brain pathways for spatial and temporal cognition involve overlapping and interacting systems that converge on the hippocampal region. There is evidence that spatial and temporal aspects of memory are processed somewhat differently in the circuitry of hippocampal subregions but become fully integrated within CA1 neuronal networks as independent, multiplexed representations of space and time. Hippocampal networks also map memories across a broad range of abstract relations among events, suggesting that the findings on spatial and temporal organization reflect a generalized mechanism for organizing memories.

Suppression of Ventral Hippocampal Output Impairs Integrated Orbitofrontal Encoding of Task Structure

The hippocampus and orbitofrontal cortex (OFC) both make important contributions to decision making and other cognitive processes. However, despite anatomical links between the two, few studies have tested the importance of hippocampal-OFC interactions. Here, we recorded OFC neurons in rats performing a decision making task while suppressing activity in a key hippocampal output region, the ventral subiculum. OFC neurons encoded information about expected outcomes and rats' responses. With hippocampal output suppressed, rats were slower to adapt to changes in reward contingency, and OFC encoding of response information was strongly attenuated. In addition, ventral subiculum inactivation prevented OFC neurons from integrating information about features of outcomes to form holistic representations of the outcomes available in specific trial blocks. These data suggest that the hippocampus contributes to OFC encoding of both concrete, low-level features of expected outcomes, and abstract, inferred properties of the structure of the world, such as task state.

Neural Representation. A Survey-Based Analysis of the Notion

The word representation (as in “neural representation”), and many of its related terms, such as to represent, representational and the like, play a central explanatory role in neuroscience literature. For instance, in “place cell” literature, place cells are extensively associated with their role in “the representation of space.” In spite of its extended use, we still lack a clear, universal and widely accepted view on what it means for a nervous system to represent something, on what makes a neural activity a representation, and on what is re-presented. The lack of a theoretical foundation and definition of the notion has not hindered actual research. My aim here is to identify how active scientists use the notion of neural representation, and eventually to list a set of criteria, based on actual use, that can help in distinguishing between genuine or non-genuine neural-representation candidates. In order to attain this objective, I present first the results of a survey of authors within two domains, place-cell and multivariate pattern analysis (MVPA) research. Based on the authors’ replies, and on a review of neuroscientific research, I outline a set of common properties that an account of neural representation seems to require. I then apply these properties to assess the use of the notion in two domains of the survey, place-cell and MVPA studies. I conclude by exploring a shift in the notion of representation suggested by recent literature.

Theta-paced flickering between place-cell maps in the hippocampus: A model based on short-term synaptic plasticity

Hippocampal place cells represent different environments with distinct neural activity patterns. Following an abrupt switch between two familiar configurations of visual cues defining two environments, the hippocampal neural activity pattern switches almost immediately to the corresponding representation. Surprisingly, during a transient period following the switch to the new environment, occasional fast transitions between the two activity patterns (flickering) were observed (Jezek, Henriksen, Treves, Moser, & Moser, 2011). Here we show that an attractor neural network model of place cells with connections endowed with short‐term synaptic plasticity can account for this phenomenon. A memory trace of the recent history of network activity is maintained in the state of the synapses, allowing the network to temporarily reactivate the representation of the previous environment in the absence of the corresponding sensory cues. The model predicts that the number of flickering events depends on the amplitude of the ongoing theta rhythm and the distance between the current position of the animal and its position at the time of cue switching. We test these predictions with new analysis of experimental data. These results suggest a potential role of short‐term synaptic plasticity in recruiting the activity of different cell assemblies and in shaping hippocampal activity of behaving animals.

Phase precession: a neural code underlying episodic memory?

In the hippocampal formation, the sequential activation of place-specific cells represents a conceptual model for the spatio-temporal events that assemble episodic memories. The imprinting of behavioral sequences in hippocampal networks might be achieved via spike-timing-dependent plasticity and phase precession of the spiking activity of neurons. It is unclear, however, whether phase precession plays an active role by enabling sequence learning via synaptic plasticity or whether phase precession passively reflects retrieval dynamics. Here we examine these possibilities in the context of potential mechanisms generating phase precession. Knowledge of these mechanisms would allow to selectively alter phase precession and test its role in episodic memory. We finally review the few successful approaches to degrade phase precession and the resulting impact on behavior.

Flexible theta sequence compression mediated via phase precessing interneurons

Encoding of behavioral episodes as spike sequences during hippocampal theta oscillations provides a neural substrate for computations on events extended across time and space. However, the mechanisms underlying the numerous and diverse experimentally observed properties of theta sequences remain poorly understood. Here we account for theta sequences using a novel model constrained by the septo-hippocampal circuitry. We show that when spontaneously active interneurons integrate spatial signals and theta frequency pacemaker inputs, they generate phase precessing action potentials that can coordinate theta sequences in place cell populations. We reveal novel constraints on sequence generation, predict cellular properties and neural dynamics that characterize sequence compression, identify circuit organization principles for high capacity sequential representation, and show that theta sequences can be used as substrates for association of conditioned stimuli with recent and upcoming events. Our results suggest mechanisms for flexible sequence compression that are suited to associative learning across an animal’s lifespan.

DOI:http://dx.doi.org/10.7554/eLife.20349.001

Nerve cells in the brain exchange information via electrical impulses. In a given brain area, the electrical impulses at any particular moment can be thought of as forming a code that represents an aspect of the outside world. For example, groups of nerve cells in the hippocampus generate a type of code called a theta sequence, which represents a series of recent and upcoming events. The specific timing of electrical impulses within a theta sequence is crucial in creating certain types of memory.

There are two major classes of nerve cell in the brain: excitatory cells activate impulses in neighbouring cells, while inhibitory cells act to temporarily block impulses from other nerve cells. Many groups, or “circuits”, of nerve cells contain combinations of both cell types to control how and when they communicate. Previous studies show that both types of cell are active within theta sequences, but it is not known precisely how they contribute to creating the sequences.

Chadwick et al. developed a new mathematical model that simulates how theta sequences can emerge from circuits of both excitatory and inhibitory nerve cells. The connections between these simulated cells are based on experimental data from real nerve cells in the hippocampus. The model predicts that inhibitory cells play an important role in generating theta sequences by interacting with groups of excitatory cells to coordinate the timing of electrical impulses. Furthermore, the model shows how memory capacity depends on these connections.

The next step following on from this work is to carry out experiments to test the model’s predictions. This will include monitoring the same group of nerve cells in multiple different situations to find out how their theta sequences change, and recording electrical events in individual nerve cells during theta sequences. If the theory’s predictions are confirmed this would lead to a deeper understanding of how our brains remember sequences of events.

DOI:http://dx.doi.org/10.7554/eLife.20349.002

Interplay between Hippocampal Sharp-Wave-Ripple Events and Vicarious Trial and Error Behaviors in Decision Making

Current theories posit that memories encoded during experiences are subsequently consolidated into longer-term storage. Hippocampal sharp-wave-ripple (SWR) events have been linked to this consolidation process during sleep, but SWRs also occur during awake immobility, where their role remains unclear. We report that awake SWR rates at the reward site are inversely related to the prevalence of vicarious trial and error (VTE) behaviors, thought to be involved in deliberation processes. SWR rates were diminished immediately after VTE behaviors and an increase in the rate of SWR events at the reward site predicted a decrease in subsequent VTE behaviors at the choice point. Furthermore, SWR disruptions increased VTE behaviors. These results suggest an inverse relationship between SWRs and VTE behaviors and suggest that awake SWRs and associated planning and memory consolidation mechanisms are engaged specifically in the context of higher levels of behavioral certainty.

Hippocampus at 25

The journal Hippocampus has passed the milestone of 25 years of publications on the topic of a highly studied brain structure, and its closely associated brain areas. In a recent celebration of this event, a Boston memory group invited 16 speakers to address the question of progress in understanding the hippocampus that has been achieved. Here we present a summary of these talks organized as progress on four main themes: (1) Understanding the hippocampus in terms of its interactions with multiple cortical areas within the medial temporal lobe memory system, (2) understanding the relationship between memory and spatial information processing functions of the hippocampal region, (3) understanding the role of temporal organization in spatial and memory processing by the hippocampus, and (4) understanding how the hippocampus integrates related events into networks of memories. © 2016 Wiley Periodicals, Inc.

Cognitive Offloading Does Not Prevent but Rather Promotes Cognitive Development

We investigate the relation between the development of reactive and cognitive capabilities. In particular we investigate whether the development of reactive capabilities prevents or promotes the development of cognitive capabilities in a population of evolving robots that have to solve a time-delay navigation task in a double T-Maze environment. Analysis of the experiments reveals that the evolving robots always select reactive strategies that rely on cognitive offloading, i.e., the possibility of acting so as to encode onto the relation between the agent and the environment the states that can be used later to regulate the agent's behavior. The discovery of these strategies does not prevent, but rather facilitates, the development of cognitive strategies that also rely on the extraction and use of internal states. Detailed analysis of the results obtained in the different experimental conditions provides evidence that helps clarify why, contrary to expectations, reactive and cognitive strategies tend to have synergetic relationships.

Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex

The hippocampus and the orbitofrontal cortex (OFC) both have important roles in cognitive processes such as learning, memory and decision making. Nevertheless, research on the OFC and hippocampus has proceeded largely independently, and little consideration has been given to the importance of interactions between these structures. Here, evidence is reviewed that the hippocampus and OFC encode parallel, but interactive, cognitive 'maps' that capture complex relationships between cues, actions, outcomes and other features of the environment. A better understanding of the interactions between the OFC and hippocampus is important for understanding the neural bases of flexible, goal-directed decision making.

Ripple-triggered stimulation of the locus coeruleus during post-learning sleep disrupts ripple/spindle coupling and impairs memory consolidation

Experience-induced replay of neuronal ensembles occurs during hippocampal high-frequency oscillations, or ripples. Post-learning increase in ripple rate is predictive of memory recall, while ripple disruption impairs learning. Ripples may thus present a fundamental component of a neurophysiological mechanism of memory consolidation. In addition to system-level local and cross-regional interactions, a consolidation mechanism involves stabilization of memory representations at the synaptic level. Synaptic plasticity within experience-activated neuronal networks is facilitated by noradrenaline release from the axon terminals of the locus coeruleus (LC). Here, to better understand interactions between the system and synaptic mechanisms underlying “off-line” consolidation, we examined the effects of ripple-associated LC activation on hippocampal and cortical activity and on spatial memory. Rats were trained on a radial maze; after each daily learning session neural activity was monitored for 1 h via implanted electrode arrays. Immediately following “on-line” detection of ripple, a brief train of electrical pulses (0.05 mA) was applied to LC. Low-frequency (20 Hz) stimulation had no effect on spatial learning, while higher-frequency (100 Hz) trains transiently blocked generation of ripple-associated cortical spindles and caused a reference memory deficit. Suppression of synchronous ripple/spindle events appears to interfere with hippocampal-cortical communication, thereby reducing the efficiency of “off-line” memory consolidation.

Anterior hippocampus: the anatomy of perception, imagination and episodic memory

The brain creates a model of the world around us. We can use this representation to perceive and comprehend what we see at any given moment, but also to vividly re-experience scenes from our past and imagine future (or even fanciful) scenarios. Recent work has shown that these cognitive functions--perception, imagination and recall of scenes and events--all engage the anterior hippocampus. In this Opinion article, we capitalize on new findings from functional neuroimaging to propose a model that links high-level cognitive functions to specific structures within the anterior hippocampus.

Spontaneous Neural Dynamics and Multi-scale Network Organization

Spontaneous neural activity has historically been viewed as task-irrelevant noise that should be controlled for via experimental design, and removed through data analysis. However, electrophysiology and functional MRI studies of spontaneous activity patterns, which have greatly increased in number over the past decade, have revealed a close correspondence between these intrinsic patterns and the structural network architecture of functional brain circuits. In particular, by analyzing the large-scale covariation of spontaneous hemodynamics, researchers are able to reliably identify functional networks in the human brain. Subsequent work has sought to identify the corresponding neural signatures via electrophysiological measurements, as this would elucidate the neural origin of spontaneous hemodynamics and would reveal the temporal dynamics of these processes across slower and faster timescales. Here we survey common approaches to quantifying spontaneous neural activity, reviewing their empirical success, and their correspondence with the findings of neuroimaging. We emphasize invasive electrophysiological measurements, which are amenable to amplitude- and phase-based analyses, and which can report variations in connectivity with high spatiotemporal precision. After summarizing key findings from the human brain, we survey work in animal models that display similar multi-scale properties. We highlight that, across many spatiotemporal scales, the covariance structure of spontaneous neural activity reflects structural properties of neural networks and dynamically tracks their functional repertoire.