The hippocampus as a predictive map

Formation and retrieval of cell assemblies in a biologically realistic spiking neural network model of area CA3 in the mouse hippocampus

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

Interactions between neural representations of the social and spatial environment

Even in our highly interconnected modern world, geographic factors play an important role in human social connections. Similarly, social relationships influence how and where we travel, and how we think about our spatial world. Here, we review the growing body of neuroscience research that is revealing multiple interactions between social and spatial processes in both humans and non-human animals. We review research on the cognitive and neural representation of spatial and social information, and highlight recent findings suggesting that underlying mechanisms might be common to both. We discuss how spatial factors can influence social behaviour, and how social concepts modify representations of space. In so doing, this review elucidates not only how neural representations of social and spatial information interact but also similarities in how the brain represents and operates on analogous information about its social and spatial surroundings.This article is part of the theme issue 'The spatial-social interface: a theoretical and empirical integration'.

Predictive Representations: Building Blocks of Intelligence

Adaptive behavior often requires predicting future events. The theory of reinforcement learning prescribes what kinds of predictive representations are useful and how to compute them. This review integrates these theoretical ideas with work on cognition and neuroscience. We pay special attention to the successor representation and its generalizations, which have been widely applied as both engineering tools and models of brain function. This convergence suggests that particular kinds of predictive representations may function as versatile building blocks of intelligence.

Brain-machine convergent evolution: Why finding parallels between brain and artificial systems is informative

Central nervous system neurons manifest a rich diversity of selectivity profiles-whose precise role is still poorly understood. Following the striking success of artificial networks, a major debate has emerged concerning their usefulness in explaining neuronal properties. Here we propose that finding parallels between artificial and neuronal networks is informative precisely because these systems are so different from each other. Our argument is based on an extension of the concept of convergent evolution-well established in biology-to the domain of artificial systems. Applying this concept to different areas and levels of the cortical hierarchy can be a powerful tool for elucidating the functional role of well-known cortical selectivities. Importantly, we further demonstrate that such parallels can uncover novel functionalities by showing that grid cells in the entorhinal cortex can be modeled to function as a set of basis functions in a lossy representation such as the well-known JPEG compression. Thus, contrary to common intuition, here we illustrate that finding parallels with artificial systems provides novel and informative insights, particularly in those cases that are far removed from realistic brain biology.

Consolidation Enhances Sequential Multistep Anticipation but Diminishes Access to Perceptual Features

Many experiences unfold predictably over time. Memory for these temporal regularities enables anticipation of events multiple steps into the future. Because temporally predictable events repeat over days, weeks, and years, we must maintain-and potentially transform-memories of temporal structure to support adaptive behavior. We explored how individuals build durable models of temporal regularities to guide multistep anticipation. Healthy young adults (Experiment 1: N = 99, age range = 18-40 years; Experiment 2: N = 204, age range = 19-40 years) learned sequences of scene images that were predictable at the category level and contained incidental perceptual details. Individuals then anticipated upcoming scene categories multiple steps into the future, immediately and at a delay. Consolidation increased the efficiency of anticipation, particularly for events further in the future, but diminished access to perceptual features. Further, maintaining a link-based model of the sequence after consolidation improved anticipation accuracy. Consolidation may therefore promote efficient and durable models of temporal structure, thus facilitating anticipation of future events.

Human hippocampus and dorsomedial prefrontal cortex infer and update latent causes during social interaction

Latent-cause inference is the process of identifying features of the environment that have caused an outcome. This problem is especially important in social settings where individuals may not make equal contributions to the outcomes they achieve together. Here, we designed a novel task in which participants inferred which of two characters was more likely to have been responsible for outcomes achieved by working together. Using computational modeling, univariate and multivariate analysis of human fMRI, and continuous theta-burst stimulation, we identified two brain regions that solved the task. Notably, as each outcome occurred, it was possible to decode the inference of its cause (the responsible character) from hippocampal activity. Activity in dorsomedial prefrontal cortex (dmPFC) updated estimates of association between cause-responsible character-and the outcome. Disruption of dmPFC activity impaired participants' ability to update their estimate as a function of inferred responsibility but spared their ability to infer responsibility.

Human hippocampal and entorhinal neurons encode the temporal structure of experience

Extracting the underlying temporal structure of experience is a fundamental aspect of learning and memory that allows us to predict what is likely to happen next. Current knowledge about the neural underpinnings of this cognitive process in humans stems from functional neuroimaging research1-5. As these methods lack direct access to the neuronal level, it remains unknown how this process is computed by neurons in the human brain. Here we record from single neurons in individuals who have been implanted with intracranial electrodes for clinical reasons, and show that human hippocampal and entorhinal neurons gradually modify their activity to encode the temporal structure of a complex image presentation sequence. This representation was formed rapidly, without providing specific instructions to the participants, and persisted when the prescribed experience was no longer present. Furthermore, the structure recovered from the population activity of hippocampal-entorhinal neurons closely resembled the structural graph defining the sequence, but at the same time, also reflected the probability of upcoming stimuli. Finally, learning of the sequence graph was related to spontaneous, time-compressed replay of individual neurons' activity corresponding to previously experienced graph trajectories. These findings demonstrate that neurons in the hippocampus and entorhinal cortex integrate the 'what' and 'when' information to extract durable and predictive representations of the temporal structure of human experience.

Replay shapes abstract cognitive maps for efficient social navigation

To make adaptive social decisions, people must anticipate how information flows through their social network. While this requires knowledge of how people are connected, networks are too large to have first-hand experience with every possible route between individuals. How, then, are people able to accurately track information flow through social networks? Here we find that people immediately cache abstract knowledge about social network structure as they learn who is friends with whom, which enables the identification of efficient routes between remotely connected individuals. These cognitive maps of social networks, which are built while learning, are then reshaped through overnight rest. During these extended periods of rest, a replay-like mechanism helps to make these maps increasingly abstract, which privileges improvements in social navigation accuracy for the longest communication paths that span distinct communities within the network. Together, these findings provide mechanistic insight into the sophisticated mental representations humans use for social navigation.

Linking cognitive strategy, neural mechanism, and movement statistics in group foraging behaviors

Foraging for food is a rich and ubiquitous animal behavior that involves complex cognitive decisions, and interactions between different individuals and species. There has been exciting recent progress in understanding multi-agent foraging behavior from cognitive, neuroscience, and statistical perspectives, but integrating these perspectives can be elusive. This paper seeks to unify these perspectives, allowing statistical analysis of observational animal movement data to shed light on the viability of cognitive models of foraging strategies. We start with cognitive agents with internal preferences expressed as value functions, and implement this in a biologically plausible neural network, and an equivalent statistical model, where statistical predictors of agents' movements correspond to the components of the value functions. We test this framework by simulating foraging agents and using Bayesian statistical modeling to correctly identify the factors that best predict the agents' behavior. As further validation, we use this framework to analyze an open-source locust foraging dataset. Finally, we collect new multi-agent real-world bird foraging data, and apply this method to analyze the preferences of different species. Together, this work provides an initial roadmap to integrate cognitive, neuroscience, and statistical approaches for reasoning about animal foraging in complex multi-agent environments.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Explaining dopamine through prediction errors and beyond

The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic dopamine observed in behaving animals. This Perspective helps to resolve some of the conflicting interpretations of dopamine that currently exist in the literature. We focus on the following three empirical challenges to the RPE theory of dopamine: why does dopamine (1) ramp up as animals approach rewards, (2) respond to sensory and motor features and (3) influence action selection? We argue that the prediction error concept, once it has been suitably modified and generalized based on an analysis of each computational problem, answers each challenge. Nonetheless, there are a number of additional empirical findings that appear to demand fundamentally different theoretical explanations beyond encoding RPE. Therefore, looking forward, we discuss the prospects for a unifying theory that respects the diversity of dopamine signaling and function as well as the complex circuitry that both underlies and responds to dopaminergic transmission.

Same Same, But Different: Brain Areas Underlying the Learning from Repetitive Episodic Prediction Errors

Prediction errors (PEs) function as learning signals. It is yet unclear how varying compared to repetitive PEs affect episodic memory in brain and behavior. The current study investigated cerebral and behavioral effects of experiencing either multiple alternative versions ("varying") or one single alternative version ("repetitive") of a previously encoded episode. Participants encoded a set of episodes ("originals") by watching videos showing toy stories. During scanning, participants either experienced originals, one single, or multiple alternative versions of the previously encoded episodes. Participants' memory performance was tested through recall of original objects. Varying and repetitive PEs revealed typical brain responses to the detection of mismatching information including inferior frontal and posterior parietal regions, as well as hippocampus, which is further linked to memory reactivation, and the amygdala, known for modulating memory consolidation. Furthermore, experiencing varying and repetitive PEs triggered distinct brain areas as revealed by direct contrast. Among others, experiencing varying versions triggered activity in the caudate, a region that has been associated with PEs. In contrast, repetitive PEs activated brain areas that resembled more those for retrieval of originally encoded episodes. Thus, ACC and posterior cingulate cortex activation seemed to serve both reactivating old and integrating new but similar information in episodic memory. Consistent with neural findings, participants recalled original objects less accurately when only presented with the same, but not varying, PE during fMRI. The current findings suggest that repeated PEs interact more strongly with a recalled original episodic memory than varying PEs.

Age-related neural changes underlying long-term recognition of musical sequences

Aging is often associated with decline in brain processing power and neural predictive capabilities. To challenge this notion, we used magnetoencephalography (MEG) and magnetic resonance imaging (MRI) to record the whole-brain activity of 39 older adults (over 60 years old) and 37 young adults (aged 18-25 years) during recognition of previously memorised and varied musical sequences. Results reveal that when recognising memorised sequences, the brain of older compared to young adults reshapes its functional organisation. In fact, it shows increased early activity in sensory regions such as the left auditory cortex (100 ms and 250 ms after each note), and only moderate decreased activity (350 ms) in medial temporal lobe and prefrontal regions. When processing the varied sequences, older adults show a marked reduction of the fast-scale functionality (250 ms after each note) of higher-order brain regions including hippocampus, ventromedial prefrontal and inferior temporal cortices, while no differences are observed in the auditory cortex. Accordingly, young outperform older adults in the recognition of novel sequences, while no behavioural differences are observed with regards to memorised ones. Our findings show age-related neural changes in predictive and memory processes, integrating existing theories on compensatory neural mechanisms in non-pathological aging.

Predictive grid coding in the medial entorhinal cortex

The entorhinal cortex represents allocentric spatial geometry and egocentric speed and heading information required for spatial navigation. However, it remains unclear whether it contributes to the prediction of an animal's future location. We discovered grid cells in the medial entorhinal cortex (MEC) that have grid fields representing future locations during goal-directed behavior. These predictive grid cells represented prospective spatial information by shifting their grid fields against the direction of travel. Predictive grid cells discharged at the trough phases of the hippocampal CA1 theta oscillation and, together with other types of grid cells, organized sequences of the trajectory from the current to future positions across each theta cycle. Our results suggest that the MEC provides a predictive map that supports forward planning in spatial navigation.

Predictive sequence learning in the hippocampal formation

The hippocampus receives sequences of sensory inputs from the cortex during exploration and encodes the sequences with millisecond precision. We developed a predictive autoencoder model of the hippocampus including the trisynaptic and monosynaptic circuits from the entorhinal cortex (EC). CA3 was trained as a self-supervised recurrent neural network to predict its next input. We confirmed that CA3 is predicting ahead by analyzing the spike coupling between simultaneously recorded neurons in the dentate gyrus, CA3, and CA1 of the mouse hippocampus. In the model, CA1 neurons signal prediction errors by comparing CA3 predictions to the next direct EC input. The model exhibits the rapid appearance and slow fading of CA1 place cells and displays replay and phase precession from CA3. The model could be learned in a biologically plausible way with error-encoding neurons. Similarities between the hippocampal and thalamocortical circuits suggest that such computation motif could also underlie self-supervised sequence learning in the cortex.

Space is a latent sequence: A theory of the hippocampus

Fascinating phenomena such as landmark vector cells and splitter cells are frequently discovered in the hippocampus. Without a unifying principle, each experiment seemingly uncovers new anomalies or coding types. Here, we provide a unifying principle that the mental representation of space is an emergent property of latent higher-order sequence learning. Treating space as a sequence resolves numerous phenomena and suggests that the place field mapping methodology that interprets sequential neuronal responses in Euclidean terms might itself be a source of anomalies. Our model, clone-structured causal graph (CSCG), employs higher-order graph scaffolding to learn latent representations by mapping aliased egocentric sensory inputs to unique contexts. Learning to compress sequential and episodic experiences using CSCGs yields allocentric cognitive maps that are suitable for planning, introspection, consolidation, and abstraction. By explicating the role of Euclidean place field mapping and demonstrating how latent sequential representations unify myriad observed phenomena, our work positions the hippocampus in a sequence-centric paradigm, challenging the prevailing space-centric view.

Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus

The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Grid Cells in Cognition: Mechanisms and Function

The activity patterns of grid cells form distinctively regular triangular lattices over the explored spatial environment and are largely invariant to visual stimuli, animal movement, and environment geometry. These neurons present numerous fascinating challenges to the curious (neuro)scientist: What are the circuit mechanisms responsible for creating spatially periodic activity patterns from the monotonic input-output responses of single neurons? How and why does the brain encode a local, nonperiodic variable-the allocentric position of the animal-with a periodic, nonlocal code? And, are grid cells truly specialized for spatial computations? Otherwise, what is their role in general cognition more broadly? We review efforts in uncovering the mechanisms and functional properties of grid cells, highlighting recent progress in the experimental validation of mechanistic grid cell models, and discuss the coding properties and functional advantages of the grid code as suggested by continuous attractor network models of grid cells.

Determinantal point process attention over grid cell code supports out of distribution generalization

Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization - successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) - a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.

Inductive biases of neural network modularity in spatial navigation

The brain may have evolved a modular architecture for daily tasks, with circuits featuring functionally specialized modules that match the task structure. We hypothesize that this architecture enables better learning and generalization than architectures with less specialized modules. To test this, we trained reinforcement learning agents with various neural architectures on a naturalistic navigation task. We found that the modular agent, with an architecture that segregates computations of state representation, value, and action into specialized modules, achieved better learning and generalization. Its learned state representation combines prediction and observation, weighted by their relative uncertainty, akin to recursive Bayesian estimation. This agent's behavior also resembles macaques' behavior more closely. Our results shed light on the possible rationale for the brain's modularity and suggest that artificial systems can use this insight from neuroscience to improve learning and generalization in natural tasks.

Transformation of spatial representations along hippocampal circuits

The hippocampus is thought to provide the brain with a cognitive map of the external world by processing various types of spatial information. To understand how essential spatial variables such as direction, position, and distance are transformed along its circuits to construct this global map, we perform single-photon widefield microendoscope calcium imaging in the dentate gyrus and CA3 of mice freely navigating along a narrow corridor. We find that spatial activity maps in the dentate gyrus, but not in CA3, are correlated after aligning them to the running directions, suggesting that they represent the distance traveled along the track in egocentric coordinates. Together with population activity decoding, our data suggest that while spatial representations in the dentate gyrus and CA3 are anchored in both egocentric and allocentric coordinates, egocentric distance coding is more prevalent in the dentate gyrus than in CA3, providing insights into the assembly of the cognitive map.

Perirhinal cortex learns a predictive map of the task environment

Goal-directed tasks involve acquiring an internal model, known as a predictive map, of relevant stimuli and associated outcomes to guide behavior. Here, we identified neural signatures of a predictive map of task behavior in perirhinal cortex (Prh). Mice learned to perform a tactile working memory task by classifying sequential whisker stimuli over multiple training stages. Chronic two-photon calcium imaging, population analysis, and computational modeling revealed that Prh encodes stimulus features as sensory prediction errors. Prh forms stable stimulus-outcome associations that can progressively be decoded earlier in the trial as training advances and that generalize as animals learn new contingencies. Stimulus-outcome associations are linked to prospective network activity encoding possible expected outcomes. This link is mediated by cholinergic signaling to guide task performance, demonstrated by acetylcholine imaging and systemic pharmacological perturbation. We propose that Prh combines error-driven and map-like properties to acquire a predictive map of learned task behavior.

The homogenous hippocampus: How hippocampal cells process available and potential goals

We present here a view of the firing patterns of hippocampal cells that is contrary, both functionally and anatomically, to conventional wisdom. We argue that the hippocampus responds to efference copies of goals encoded elsewhere; and that it uses these to detect and resolve conflict or interference between goals in general. While goals can involve space, hippocampal cells do not encode spatial (or other special types of) memory, as such. We also argue that the transverse circuits of the hippocampus operate in an essentially homogeneous way along its length. The apparently different functions of different parts (e.g. memory retrieval versus anxiety) result from the different (situational/motivational) inputs on which those parts perform the same fundamental computational operations. On this view, the key role of the hippocampus is the iterative adjustment, via Papez-like circuits, of synaptic weights in cell assemblies elsewhere.

In vivo structural connectivity of the reward system along the hippocampal long axis

Recent work has identified a critical role for the hippocampus in reward-sensitive behaviors, including motivated memory, reinforcement learning, and decision-making. Animal histology and human functional neuroimaging have shown that brain regions involved in reward processing and motivation are more interconnected with the ventral/anterior hippocampus. However, direct evidence examining gradients of structural connectivity between reward regions and the hippocampus in humans is lacking. The present study used diffusion MRI (dMRI) and probabilistic tractography to quantify the structural connectivity of the hippocampus with key reward processing regions in vivo. Using a large sample of subjects (N = 628) from the human connectome dMRI data release, we found that connectivity profiles with the hippocampus varied widely between different regions of the reward circuit. While the dopaminergic midbrain (ventral tegmental area) showed stronger connectivity with the anterior versus posterior hippocampus, the ventromedial prefrontal cortex showed stronger connectivity with the posterior hippocampus. The limbic (ventral) striatum demonstrated a more homogeneous connectivity profile along the hippocampal long axis. This is the first study to generate a probabilistic atlas of the hippocampal structural connectivity with reward-related networks, which is essential to investigating how these circuits contribute to normative adaptive behavior and maladaptive behaviors in psychiatric illness. These findings describe nuanced structural connectivity that sets the foundation to better understand how the hippocampus influences reward-guided behavior in humans.

Hippocampal and orbitofrontal neurons contribute to complementary aspects of associative structure

The ability to establish associations between environmental stimuli is fundamental for higher-order brain functions like state inference and generalization. Both the hippocampus and orbitofrontal cortex (OFC) play pivotal roles in this, demonstrating complex neural activity changes after associative learning. However, how precisely they contribute to representing learned associations remains unclear. Here, we train head-restrained mice to learn four 'odor-outcome' sequence pairs composed of several task variables-the past and current odor cues, sequence structure of 'cue-outcome' arrangement, and the expected outcome; and perform calcium imaging from these mice throughout learning. Sequence-splitting signals that distinguish between paired sequences are detected in both brain regions, reflecting associative memory formation. Critically, we uncover differential contents in represented associations by examining, in each area, how these task variables affect splitting signal generalization between sequence pairs. Specifically, the hippocampal splitting signals are influenced by the combination of past and current cues that define a particular sensory experience. In contrast, the OFC splitting signals are similar between sequence pairs that share the same sequence structure and expected outcome. These findings suggest that the hippocampus and OFC uniquely and complementarily organize the acquired associative structure.

Associative Learning of an Unnormalized Successor Representation

The successor representation is known to relate to temporal associations learned in the temporal context model (Gershman et al., 2012), and subsequent work suggests a wide relevance of the successor representation across spatial, visual, and abstract relational tasks. I demonstrate that the successor representation and purely associative learning have an even deeper relationship than initially indicated: Hebbian temporal associations are an unnormalized form of the successor representation, such that the two converge on an identical representation whenever all states are equally frequent and can correlate highly in practice even when the state distribution is nonuniform.

Expanding horizons in reinforcement learning for curious exploration and creative planning

Curiosity and creativity are expressions of the trade-off between leveraging that with which we are familiar or seeking out novelty. Through the computational lens of reinforcement learning, we describe how formulating the value of information seeking and generation via their complementary effects on planning horizons formally captures a range of solutions to striking this balance.

Multiple mechanisms of visual prediction as revealed by the timecourse of scene-object facilitation

Not only semantic, but also recently learned arbitrary associations have the potential to facilitate visual processing in everyday life-for example, knowledge of a (moveable) object's location at a specific time may facilitate visual processing of that object. In our prior work, we showed that previewing a scene can facilitate processing of recently associated objects at the level of visual analysis (Smith and Federmeier in Journal of Cognitive Neuroscience, 32(5):783-803, 2020). In the current study, we assess how rapidly this facilitation unfolds by manipulating scene preview duration. We then compare our results to studies using well-learned object-scene associations in a first-pass assessment of whether systems consolidation might speed up high-level visual prediction. In two ERP experiments (N = 60), we had participants study categorically organized novel object-scene pairs in an explicit paired associate learning task. At test, we varied contextual pre-exposure duration, both between (200 vs. 2500 ms) and within subjects (0-2500 ms). We examined the N300, an event-related potential component linked to high-level visual processing of objects and scenes and found that N300 effects of scene congruity increase with longer scene previews, up to approximately 1-2 s. Similar results were obtained for response times and in a separate component-neutral ERP analysis of visual template matching. Our findings contrast with prior evidence that scenes can rapidly facilitate visual processing of commonly associated objects. This raises the possibility that systems consolidation might mediate different kinds of predictive processing with different temporal profiles.

In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Similar to associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work, we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.

Neuromorphic one-shot learning utilizing a phase-transition material

Design of hardware based on biological principles of neuronal computation and plasticity in the brain is a leading approach to realizing energy- and sample-efficient AI and learning machines. An important factor in selection of the hardware building blocks is the identification of candidate materials with physical properties suitable to emulate the large dynamic ranges and varied timescales of neuronal signaling. Previous work has shown that the all-or-none spiking behavior of neurons can be mimicked by threshold switches utilizing material phase transitions. Here, we demonstrate that devices based on a prototypical metal-insulator-transition material, vanadium dioxide (VO2), can be dynamically controlled to access a continuum of intermediate resistance states. Furthermore, the timescale of their intrinsic relaxation can be configured to match a range of biologically relevant timescales from milliseconds to seconds. We exploit these device properties to emulate three aspects of neuronal analog computation: fast (~1 ms) spiking in a neuronal soma compartment, slow (~100 ms) spiking in a dendritic compartment, and ultraslow (~1 s) biochemical signaling involved in temporal credit assignment for a recently discovered biological mechanism of one-shot learning. Simulations show that an artificial neural network using properties of VO2 devices to control an agent navigating a spatial environment can learn an efficient path to a reward in up to fourfold fewer trials than standard methods. The phase relaxations described in our study may be engineered in a variety of materials and can be controlled by thermal, electrical, or optical stimuli, suggesting further opportunities to emulate biological learning in neuromorphic hardware.

Prediction error processing and sharpening of expected information across the face-processing hierarchy

The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.

Conjunctive encoding of exploratory intentions and spatial information in the hippocampus

The hippocampus creates a cognitive map of the external environment by encoding spatial and self-motion-related information. However, it is unclear whether hippocampal neurons could also incorporate internal cognitive states reflecting an animal's exploratory intention, which is not driven by rewards or unexpected sensory stimuli. In this study, a subgroup of CA1 neurons was found to encode both spatial information and animals' investigatory intentions in male mice. These neurons became active before the initiation of exploration behaviors at specific locations and were nearly silent when the same fields were traversed without exploration. Interestingly, this neuronal activity could not be explained by object features, rewards, or mismatches in environmental cues. Inhibition of the lateral entorhinal cortex decreased the activity of these cells during exploration. Our findings demonstrate that hippocampal neurons may bridge external and internal signals, indicating a potential connection between spatial representation and intentional states in the construction of internal navigation systems.

Beyond hippocampus: Thalamic and prefrontal contributions to an evolving memory

The hippocampus has long been at the center of memory research, and rightfully so. However, with emerging technological capabilities, we can increasingly appreciate memory as a more dynamic and brain-wide process. In this perspective, our goal is to begin developing models to understand the gradual evolution, reorganization, and stabilization of memories across the brain after their initial formation in the hippocampus. By synthesizing studies across the rodent and human literature, we suggest that as memory representations initially form in hippocampus, parallel traces emerge in frontal cortex that cue memory recall, and as they mature, with sustained support initially from limbic then diencephalic then cortical circuits, they become progressively independent of hippocampus and dependent on a mature cortical representation. A key feature of this model is that, as time progresses, memory representations are passed on to distinct circuits with progressively longer time constants, providing the opportunity to filter, forget, update, or reorganize memories in the process of committing to long-term storage.

Functional brain networks in Developmental Topographical Disorientation

Despite a decade-long study on Developmental Topographical Disorientation, the underlying mechanism behind this neurological condition remains unknown. This lifelong selective inability in orientation, which causes these individuals to get lost even in familiar surroundings, is present in the absence of any other neurological disorder or acquired brain damage. Herein, we report an analysis of the functional brain network of individuals with Developmental Topographical Disorientation ($n = 19$) compared against that of healthy controls ($n = 21$), all of whom underwent resting-state functional magnetic resonance imaging, to identify if and how their underlying functional brain network is altered. While the established resting-state networks (RSNs) are confirmed in both groups, there is, on average, a greater connectivity and connectivity strength, in addition to increased global and local efficiency in the overall functional network of the Developmental Topographical Disorientation group. In particular, there is an enhanced connectivity between some RSNs facilitated through indirect functional paths. We identify a handful of nodes that encode part of these differences. Overall, our findings provide strong evidence that the brain networks of individuals suffering from Developmental Topographical Disorientation are modified by compensatory mechanisms, which might open the door for new diagnostic tools.

Emergent neural dynamics and geometry for generalization in a transitive inference task

Relational cognition-the ability to infer relationships that generalize to novel combinations of objects-is fundamental to human and animal intelligence. Despite this importance, it remains unclear how relational cognition is implemented in the brain due in part to a lack of hypotheses and predictions at the levels of collective neural activity and behavior. Here we discovered, analyzed, and experimentally tested neural networks (NNs) that perform transitive inference (TI), a classic relational task (if A > B and B > C, then A > C). We found NNs that (i) generalized perfectly, despite lacking overt transitive structure prior to training, (ii) generalized when the task required working memory (WM), a capacity thought to be essential to inference in the brain, (iii) emergently expressed behaviors long observed in living subjects, in addition to a novel order-dependent behavior, and (iv) expressed different task solutions yielding alternative behavioral and neural predictions. Further, in a large-scale experiment, we found that human subjects performing WM-based TI showed behavior inconsistent with a class of NNs that characteristically expressed an intuitive task solution. These findings provide neural insights into a classical relational ability, with wider implications for how the brain realizes relational cognition.

The generative neural microdynamics of cognitive processing

The entorhinal cortex and hippocampus form a recurrent network that informs many cognitive processes, including memory, planning, navigation, and imagination. Neural recordings from these regions reveal spatially organized population codes corresponding to external environments and abstract spaces. Aligning the former cognitive functionalities with the latter neural phenomena is a central challenge in understanding the entorhinal-hippocampal circuit (EHC). Disparate experiments demonstrate a surprising level of complexity and apparent disorder in the intricate spatiotemporal dynamics of sequential non-local hippocampal reactivations, which occur particularly, though not exclusively, during immobile pauses and rest. We review these phenomena with a particular focus on their apparent lack of physical simulative realism. These observations are then integrated within a theoretical framework and proposed neural circuit mechanisms that normatively characterize this neural complexity by conceiving different regimes of hippocampal microdynamics as neuromarkers of diverse cognitive computations.

The hippocampus and implicit memory (by any other name)

Is the hippocampus involved in implicit memory? I argue that contemporary views on hippocampal function, going beyond the classic dichotomy of explicit versus implicit, predict involvement of the hippocampus whenever flexible, predictive associations are rapidly encoded. This involvement is independent of conscious awareness. A paradigm case is statistical learning: the unconscious extraction of statistical regularities from the environment. In line with this, a substantial body of literature on contextual cueing in visual search has established hippocampal involvement in this form of implicit learning. To conclude, implicit memory (as such or by any other name) is associated with the hippocampus.

Formation and Retrieval of Cell Assemblies in a Biologically Realistic Spiking Neural Network Model of Area CA3 in the Mouse Hippocampus

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

Local prediction-learning in high-dimensional spaces enables neural networks to plan

Planning and problem solving are cornerstones of higher brain function. But we do not know how the brain does that. We show that learning of a suitable cognitive map of the problem space suffices. Furthermore, this can be reduced to learning to predict the next observation through local synaptic plasticity. Importantly, the resulting cognitive map encodes relations between actions and observations, and its emergent high-dimensional geometry provides a sense of direction for reaching distant goals. This quasi-Euclidean sense of direction provides a simple heuristic for online planning that works almost as well as the best offline planning algorithms from AI. If the problem space is a physical space, this method automatically extracts structural regularities from the sequence of observations that it receives so that it can generalize to unseen parts. This speeds up learning of navigation in 2D mazes and the locomotion with complex actuator systems, such as legged bodies. The cognitive map learner that we propose does not require a teacher, similar to self-attention networks (Transformers). But in contrast to Transformers, it does not require backpropagation of errors or very large datasets for learning. Hence it provides a blue-print for future energy-efficient neuromorphic hardware that acquires advanced cognitive capabilities through autonomous on-chip learning.

Transcranial direct current stimulation suggests a causal role of the medial prefrontal cortex in learning social hierarchy

Social hierarchies can be inferred through observational learning of social relationships between individuals. Yet, little is known about the causal role of specific brain regions in learning hierarchies. Here, using transcranial direct current stimulation, we show a causal role of the medial prefrontal cortex (mPFC) in learning social versus non-social hierarchies. In a Training phase, participants acquired knowledge about social and non-social hierarchies by trial and error. During a Test phase, they were presented with two items from hierarchies that were never encountered together, requiring them to make transitive inferences. Anodal stimulation over mPFC impaired social compared with non-social hierarchy learning, and this modulation was influenced by the relative social rank of the members (higher or lower status). Anodal stimulation also impaired transitive inference making, but only during early blocks before learning was established. Together, these findings demonstrate a causal role of the mPFC in learning social ranks by observation.

Lateral entorhinal cortex subpopulations represent experiential epochs surrounding reward

During goal-directed navigation, 'what' information, describing the experiences occurring in periods surrounding a reward, can be combined with spatial 'where' information to guide behavior and form episodic memories. This integrative process likely occurs in the hippocampus, which receives spatial information from the medial entorhinal cortex; however, the source of the 'what' information is largely unknown. Here, we show that mouse lateral entorhinal cortex (LEC) represents key experiential epochs during reward-based navigation tasks. We discover separate populations of neurons that signal goal approach and goal departure and a third population signaling reward consumption. When reward location is moved, these populations immediately shift their respective representations of each experiential epoch relative to reward, while optogenetic inhibition of LEC disrupts learning the new reward location. Therefore, the LEC contains a stable code of experiential epochs surrounding and including reward consumption, providing reward-centric information to contextualize the spatial information carried by the medial entorhinal cortex.

A generative model of memory construction and consolidation

Episodic memories are (re)constructed, share neural substrates with imagination, combine unique features with schema-based predictions and show schema-based distortions that increase with consolidation. Here we present a computational model in which hippocampal replay (from an autoassociative network) trains generative models (variational autoencoders) to (re)create sensory experiences from latent variable representations in entorhinal, medial prefrontal and anterolateral temporal cortices via the hippocampal formation. Simulations show effects of memory age and hippocampal lesions in agreement with previous models, but also provide mechanisms for semantic memory, imagination, episodic future thinking, relational inference and schema-based distortions including boundary extension. The model explains how unique sensory and predictable conceptual elements of memories are stored and reconstructed by efficiently combining both hippocampal and neocortical systems, optimizing the use of limited hippocampal storage for new and unusual information. Overall, we believe hippocampal replay training generative models provides a comprehensive account of memory construction, imagination and consolidation.

Endotaxis: A neuromorphic algorithm for mapping, goal-learning, navigation, and patrolling

An animal entering a new environment typically faces three challenges: explore the space for resources, memorize their locations, and navigate towards those targets as needed. Here we propose a neural algorithm that can solve all these problems and operates reliably in diverse and complex environments. At its core, the mechanism makes use of a behavioral module common to all motile animals, namely the ability to follow an odor to its source. We show how the brain can learn to generate internal "virtual odors" that guide the animal to any location of interest. This endotaxis algorithm can be implemented with a simple 3-layer neural circuit using only biologically realistic structures and learning rules. Several neural components of this scheme are found in brains from insects to humans. Nature may have evolved a general mechanism for search and navigation on the ancient backbone of chemotaxis.

Neural Trajectories of Conceptually Related Events

In a series of conceptually related episodes, meaning arises from the link between these events rather than from each event individually. How does the brain keep track of conceptually related sequences of events (i.e., conceptual trajectories)? In a particular kind of conceptual trajectory-a social relationship-meaning arises from a specific sequence of interactions. To test whether such abstract sequences are neurally tracked, we had participants complete a naturalistic narrative-based social interaction game, during functional magnetic resonance imaging. We modeled the simulated relationships as trajectories through an abstract affiliation and power space. In two independent samples, we found evidence of individual social relationships being tracked with unique sequences of hippocampal states. The neural states corresponded to the accumulated trial-to-trial affiliation and power relations between the participant and each character, such that each relationship's history was captured by its own neural trajectory. Each relationship had its own sequence of states, and all relationships were embedded within the same manifold. As such, we show that the hippocampus represents social relationships with ordered sequences of low-dimensional neural patterns. The number of distinct clusters of states on this manifold is also related to social function, as measured by the size of real-world social networks. These results suggest that our evolving relationships with others are represented in trajectory-like neural patterns.

Sequential activity of CA1 hippocampal cells constitutes a temporal memory map for associative learning in mice

Sequential neural dynamics encoded by time cells play a crucial role in hippocampal function. However, the role of hippocampal sequential neural dynamics in associative learning is an open question. We used two-photon Ca2+ imaging of dorsal CA1 (dCA1) neurons in the stratum pyramidale (SP) in head-fixed mice performing a go-no go associative learning task to investigate how odor valence is temporally encoded in this area of the brain. We found that SP cells responded differentially to the rewarded or unrewarded odor. The stimuli were decoded accurately from the activity of the neuronal ensemble, and accuracy increased substantially as the animal learned to differentiate the stimuli. Decoding the stimulus from individual SP cells responding differentially revealed that decision-making took place at discrete times after stimulus presentation. Lick prediction decoded from the ensemble activity of cells in dCA1 correlated linearly with lick behavior. Our findings indicate that sequential activity of SP cells in dCA1 constitutes a temporal memory map used for decision-making in associative learning. VIDEO ABSTRACT.

RatInABox, a toolkit for modelling locomotion and neuronal activity in continuous environments

Generating synthetic locomotory and neural data is a useful yet cumbersome step commonly required to study theoretical models of the brain's role in spatial navigation. This process can be time consuming and, without a common framework, makes it difficult to reproduce or compare studies which each generate test data in different ways. In response, we present RatInABox, an open-source Python toolkit designed to model realistic rodent locomotion and generate synthetic neural data from spatially modulated cell types. This software provides users with (i) the ability to construct one- or two-dimensional environments with configurable barriers and visual cues, (ii) a physically realistic random motion model fitted to experimental data, (iii) rapid online calculation of neural data for many of the known self-location or velocity selective cell types in the hippocampal formation (including place cells, grid cells, boundary vector cells, head direction cells) and (iv) a framework for constructing custom cell types, multi-layer network models and data- or policy-controlled motion trajectories. The motion and neural models are spatially and temporally continuous as well as topographically sensitive to boundary conditions and walls. We demonstrate that out-of-the-box parameter settings replicate many aspects of rodent foraging behaviour such as velocity statistics and the tendency of rodents to over-explore walls. Numerous tutorial scripts are provided, including examples where RatInABox is used for decoding position from neural data or to solve a navigational reinforcement learning task. We hope this tool will significantly streamline computational research into the brain's role in navigation.

Grid-like entorhinal representation of an abstract value space during prospective decision making

How valuable a choice option is often changes over time, making the prediction of value changes an important challenge for decision making. Prior studies identified a cognitive map in the hippocampal-entorhinal system that encodes relationships between states and enables prediction of future states, but does not inherently convey value during prospective decision making. In this fMRI study, participants predicted changing values of choice options in a sequence, forming a trajectory through an abstract two-dimensional value space. During this task, the entorhinal cortex exhibited a grid-like representation with an orientation aligned to the axis through the value space most informative for choices. A network of brain regions, including ventromedial prefrontal cortex, tracked the prospective value difference between options. These findings suggest that the entorhinal grid system supports the prediction of future values by representing a cognitive map, which might be used to generate lower-dimensional value signals to guide prospective decision making.

Hippocampal sequences span experience relative to rewards

Hippocampal place cells fire in sequences that span spatial environments and non-spatial modalities, suggesting that hippocampal activity can anchor to the most behaviorally salient aspects of experience. As reward is a highly salient event, we hypothesized that sequences of hippocampal activity can anchor to rewards. To test this, we performed two-photon imaging of hippocampal CA1 neurons as mice navigated virtual environments with changing hidden reward locations. When the reward moved, the firing fields of a subpopulation of cells moved to the same relative position with respect to reward, constructing a sequence of reward-relative cells that spanned the entire task structure. The density of these reward-relative sequences increased with task experience as additional neurons were recruited to the reward-relative population. Conversely, a largely separate subpopulation maintained a spatially-based place code. These findings thus reveal separate hippocampal ensembles can flexibly encode multiple behaviorally salient reference frames, reflecting the structure of the experience.

The successor representation subserves hierarchical abstraction for goal-directed behavior

Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior.