Remembering rewarding futures: A simulation-selection model of the hippocampus

A bird's eye view of the hippocampus beyond space: Behavioral, neuroanatomical, and neuroendocrine perspectives

Although the hippocampus is one of the most-studied brain regions in mammals, research on the avian hippocampus has been more limited in scope. It is generally agreed that the hippocampus is an ancient feature of the amniote brain, and therefore homologous between the two lineages. Because birds and mammals are evolutionarily not very closely related, any shared anatomy is likely to be crucial for shared functions of their hippocampi. These functions, in turn, are likely to be essential if they have been conserved for over 300 million years. Therefore, research on the avian hippocampus can help us understand how this brain region evolved and how it has changed over evolutionary time. Further, there is a strong research foundation in birds on hippocampal-supported behaviors such as spatial navigation, food caching, and brood parasitism that scientists can build upon to better understand how hippocampal anatomy, network circuitry, endocrinology, and physiology can help control these behaviors. In this review, we summarize our current understanding of the avian hippocampus in spatial cognition as well as in regulating anxiety, approach-avoidance behavior, and stress responses. Although there are still some questions about the exact number of subdivisions in the avian hippocampus and how that might vary in different avian families, there is intriguing evidence that the avian hippocampus might have complementary functional profiles along the rostral-caudal axis similar to the dorsal-ventral axis of the rodent hippocampus, where the rostral/dorsal hippocampus is more involved in cognitive processes like spatial learning and the caudal/ventral hippocampus regulates emotional states, anxiety, and the stress response. Future research should focus on elucidating the cellular and molecular mechanisms - including endocrinological - in the avian hippocampus that underlie behaviors such as spatial navigation, spatial memory, and anxiety-related behaviors, and in so doing, resolve outstanding questions about avian hippocampal function and organization.

Sensory cortical ensembles exhibit differential coupling to ripples in distinct hippocampal subregions

Cortical neurons activated during recent experiences often reactivate with dorsal hippocampal CA1 ripples during subsequent rest. Less is known about cortical interactions with intermediate hippocampal CA1, whose connectivity, functions, and ripple events differ from dorsal CA1. We identified three clusters of putative excitatory neurons in mouse visual cortex that are preferentially excited together with either dorsal or intermediate CA1 ripples or suppressed before both ripples. Neurons in each cluster were evenly distributed across primary and higher visual cortices and co-active even in the absence of ripples. These ensembles exhibited similar visual responses but different coupling to thalamus and pupil-indexed arousal. We observed a consistent activity sequence preceding and predicting ripples: (1) suppression of ripple-suppressed cortical neurons, (2) thalamic silence, and (3) activation of intermediate CA1-ripple-activated cortical neurons. We propose that coordinated dynamics of these ensembles relay visual experiences to distinct hippocampal subregions for incorporation into different cognitive maps.

Sensory cortical ensembles exhibit differential coupling to ripples in distinct hippocampal subregions

Cortical neurons activated during recent experiences often reactivate with dorsal hippocampal CA1 sharp-wave ripples (SWRs) during subsequent rest. Less is known about cortical interactions with intermediate hippocampal CA1, whose connectivity, functions, and SWRs differ from those of dorsal CA1. We identified three clusters of visual cortical excitatory neurons that are excited together with either dorsal or intermediate CA1 SWRs, or suppressed before both SWRs. Neurons in each cluster were distributed across primary and higher visual cortices and co-active even in the absence of SWRs. These ensembles exhibited similar visual responses but different coupling to thalamus and pupil-indexed arousal. We observed a consistent activity sequence: (i) suppression of SWR-suppressed cortical neurons, (ii) thalamic silence, and (iii) activation of the cortical ensemble preceding and predicting intermediate CA1 SWRs. We propose that the coordinated dynamics of these ensembles relay visual experiences to distinct hippocampal subregions for incorporation into different cognitive maps.

Septotemporal variations in hippocampal value and outcome processing

A large body of evidence indicates functional variations along the hippocampal longitudinal axis. To investigate whether and how value and outcome processing vary between the dorsal (DH) and the ventral hippocampus (VH), we examined neuronal activity and inactivation effects of the DH and VH in mice performing probabilistic classical conditioning tasks. Inactivation of either structure disrupts value-dependent anticipatory licking, and value-coding neurons are found in both structures, indicating their involvement in value processing. However, the DH neuronal population increases activity as a function of value, while the VH neuronal population is preferentially responsive to the highest-value sensory cue. Also, signals related to outcome-dependent value learning are stronger in the DH. VH neurons instead show rapid responses to punishment and strongly biased responses to negative prediction error. These findings suggest that the DH faithfully represents the external value landscape, whereas the VH preferentially represents behaviorally relevant, salient features of experienced events.

Correcting the hebbian mistake: Toward a fully error-driven hippocampus

The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., "cells that fire together, wire together"). However, Hebbian learning is computationally suboptimal as it does not learn in a way that is driven toward, and limited by, the objective of achieving effective retrieval. Thus, Hebbian learning results in more interference and a lower overall capacity. Our previous computational models have utilized a powerful, biologically plausible form of error-driven learning in hippocampal CA1 and entorhinal cortex (EC) (functioning as a sparse autoencoder) by contrasting local activity states at different phases in the theta cycle. Based on specific neural data and a recent abstract computational model, we propose a new model called Theremin (Total Hippocampal ERror MINimization) that extends error-driven learning to area CA3-the mnemonic heart of the hippocampal system. In the model, CA3 responds to the EC monosynaptic input prior to the EC disynaptic input through dentate gyrus (DG), giving rise to a temporal difference between these two activation states, which drives error-driven learning in the EC→CA3 and CA3↔CA3 projections. In effect, DG serves as a teacher to CA3, correcting its patterns into more pattern-separated ones, thereby reducing interference. Results showed that Theremin, compared with our original Hebbian-based model, has significantly increased capacity and learning speed. The model makes several novel predictions that can be tested in future studies.

Blue-Light-Blocking Lenses Ameliorate Structural Alterations in the Rodent Hippocampus

Evidence suggests that prolonged blue-light exposure can impact vision; however, less is known about its impact on non-visual higher-order functions in the brain, such as learning and memory. Blue-light-blocking lenses (BBLs) claim to reduce these potential impacts. Hence, we assessed structural and functional hippocampal alterations following blue-light exposure and the protective efficacy of BBLs. Male Wistar rats were divided into (n = 6 in each group) normal control (NC), blue-light exposure (LE), and blue-light with BBLs (Crizal Prevencia, CP and DuraVision Blue, DB) groups. After 28 days of light exposure (12:12 light: dark cycle), rats were trained for the Morris water maze memory retention test, and brain tissues were sectioned for hippocampal neuronal analysis using Golgi and Cresyl violet stains. The memory retention test was significantly delayed (p < 0.05) in LE compared with DB groups on day 1 of training. Comparison of Golgi-stained neurons showed significant structural alterations, particularly in the basal dendrites of hippocampal neurons in the LE group, with BBLs significantly mitigating these structural changes (p < 0.05). Comparison of Cresyl-violet-stained neurons revealed significantly (p < 0.001) increased degenerated hippocampal neurons in LE rats, with fewer degenerated neurons in the CP lens group for CA1 neurons (p < 0.05), and for both CP and DB groups (p < 0.05) for CA3 neurons. Thus, in addition to documented effects on visual centers, high-level blue-light exposure also results in degeneration in hippocampal neurons with associated behavioral deficits. These changes can be partially ameliorated with blue-light-blocking lenses.

Space, feature, and risk sensitivity in homing pigeons (Columba livia): Broadening the conversation on the role of the avian hippocampus in memory

David Sherry has been a pioneer in investigating the avian hippocampal formation (HF) and spatial memory. Following on his work and observations that HF is sensitive to the occurrence of reward (food), we were interested in carrying out an exploratory study to investigate possible HF involvement in the representation goal value and risk. Control sham-lesioned and hippocampal-lesioned pigeons were trained in an open field to locate one food bowl containing a constant two food pellets on all trials, and two variable bowls with one containing five pellets on 75% (High Variable) and another on 25% (Low Variable) of their respective trials (High-Variable and Low-Variable bowls were never presented together). One pairing of pigeons learned bowl locations (space); another bowl colors (feature). Trained to color, hippocampal-lesioned pigeons performed as rational agents in their bowl choices and were indistinguishable from the control pigeons, a result consistent with HF regarded as unimportant for non-spatial memory. By contrast, when trained to location, hippocampal-lesioned pigeons differed from the control pigeons. They made more first-choice errors to bowls that never contained food, consistent with a role of HF in spatial memory. Intriguingly, the hippocampal-lesioned pigeons also made fewer first choices to both variable bowls, suggesting that hippocampal lesions resulted in the pigeons becoming more risk averse. Acknowledging that the results are preliminary and further research is needed, the data nonetheless support the general hypothesis that HF-dependent memory representations of space capture properties of reward value and risk, properties that contribute to decision making when confronted with a choice.

Navigating for reward

An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.

Cortical reactivations of recent sensory experiences predict bidirectional network changes during learning

Salient experiences are often relived in the mind. Human neuroimaging studies suggest that such experiences drive activity patterns in visual association cortex that are subsequently reactivated during quiet waking. Nevertheless, the circuit-level consequences of such reactivations remain unclear. Here, we imaged hundreds of neurons in visual association cortex across days as mice learned a visual discrimination task. Distinct patterns of neurons were activated by different visual cues. These same patterns were subsequently reactivated during quiet waking in darkness, with higher reactivation rates during early learning and for food-predicting versus neutral cues. Reactivations involving ensembles of neurons encoding both the food cue and the reward predicted strengthening of next-day functional connectivity of participating neurons, while the converse was observed for reactivations involving ensembles encoding only the food cue. We propose that task-relevant neurons strengthen while task-irrelevant neurons weaken their dialog with the network via participation in distinct flavors of reactivation.

Flexible use of allocentric and egocentric spatial memories activates differential neural networks in mice

Goal-directed navigation can be based on world-centered (allocentric) or body-centered (egocentric) representations of the environment, mediated by a wide network of interconnected brain regions, including hippocampus, striatum and prefrontal cortex. The relative contribution of these regions to navigation from novel or familiar routes, that demand a different degree of flexibility in the use of the stored spatial representations, has not been completely explored. To address this issue, we trained mice to find a reward relying on allocentric or egocentric information, in a modified version of the cross-maze task. Then we used Zif268 expression to map brain activation when well-trained mice were required to find the goal from a novel or familiar location. Successful navigation was correlated with the activation of CA1, posterior-dorsomedial striatum, nucleus accumbens core and infralimbic cortex when allocentric-trained mice needed to use a novel route. Allocentric navigation from a familiar route activated dorsomedial striatum, nucleus accumbens, prelimbic and infralimbic cortex. None of the structures analyzed was significantly activated in egocentric-trained mice, irrespective of the starting position. These data suggest that a flexible use of stored allocentric information, that allows goal finding even from a location never explored during training, induces a shift from fronto-striatal to hippocampal circuits.

Distinct effects of reward and navigation history on hippocampal forward and reverse replays

To better understand the functional roles of hippocampal forward and reverse replays, we trained rats in a spatial sequence memory task and examined how these replays are modulated by reward and navigation history. We found that reward enhances both forward and reverse replays during the awake state, but in different ways. Reward enhances the rate of reverse replays, but it increases the fidelity of forward replays for recently traveled as well as other alternative trajectories heading toward a rewarding location. This suggests roles for forward and reverse replays in reinforcing representations for all potential rewarding trajectories. We also found more faithful reactivation of upcoming than already rewarded trajectories in forward replays. This suggests a role for forward replays in preferentially reinforcing representations for high-value trajectories. We propose that hippocampal forward and reverse replays might contribute to constructing a map of potential navigation trajectories and their associated values (a "value map") via distinct mechanisms.

Remembering rewarding futures: A simulation-selection model of the hippocampus

Despite tremendous progress, the neural circuit dynamics underlying hippocampal mnemonic processing remain poorly understood. We propose a new model for hippocampal function-the simulation-selection model-based on recent experimental findings and neuroecological considerations. Under this model, the mammalian hippocampus evolved to simulate and evaluate arbitrary navigation sequences. Specifically, we suggest that CA3 simulates unexperienced navigation sequences in addition to remembering experienced ones, and CA1 selects from among these CA3-generated sequences, reinforcing those that are likely to maximize reward during offline idling states. High-value sequences reinforced in CA1 may allow flexible navigation toward a potential rewarding location during subsequent navigation. We argue that the simulation-selection functions of the hippocampus have evolved in mammals mostly because of the unique navigational needs of land mammals. Our model may account for why the mammalian hippocampus has evolved not only to remember, but also to imagine episodes, and how this might be implemented in its neural circuits.