Context prediction analysis and episodic memory

Contextual memory engrams, and the neuromodulatory influence of the locus coeruleus

Here, we review the basis of contextual memory at a conceptual and cellular level. We begin with an overview of the philosophical foundations of traversing space, followed by theories covering the material bases of contextual representations in the hippocampus (engrams), exploring functional characteristics of the cells and subfields within. Next, we explore various methodological approaches for investigating contextual memory engrams, emphasizing plasticity mechanisms. This leads us to discuss the role of neuromodulatory inputs in governing these dynamic changes. We then outline a recent hypothesis involving noradrenergic and dopaminergic projections from the locus coeruleus (LC) to different subregions of the hippocampus, in sculpting contextual representations, giving a brief description of the neuroanatomical and physiological properties of the LC. Finally, we examine how activity in the LC influences contextual memory processes through synaptic plasticity mechanisms to alter hippocampal engrams. Overall, we find that phasic activation of the LC plays an important role in promoting new learning and altering mnemonic processes at the behavioral and cellular level through the neuromodulatory influence of NE/DA in the hippocampus. These findings may provide insight into mechanisms of hippocampal remapping and memory updating, memory processes that are potentially dysregulated in certain psychiatric and neurodegenerative disorders.

Correcting the hebbian mistake: Toward a fully error-driven hippocampus

The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., "cells that fire together, wire together"). However, Hebbian learning is computationally suboptimal as it does not learn in a way that is driven toward, and limited by, the objective of achieving effective retrieval. Thus, Hebbian learning results in more interference and a lower overall capacity. Our previous computational models have utilized a powerful, biologically plausible form of error-driven learning in hippocampal CA1 and entorhinal cortex (EC) (functioning as a sparse autoencoder) by contrasting local activity states at different phases in the theta cycle. Based on specific neural data and a recent abstract computational model, we propose a new model called Theremin (Total Hippocampal ERror MINimization) that extends error-driven learning to area CA3-the mnemonic heart of the hippocampal system. In the model, CA3 responds to the EC monosynaptic input prior to the EC disynaptic input through dentate gyrus (DG), giving rise to a temporal difference between these two activation states, which drives error-driven learning in the EC→CA3 and CA3↔CA3 projections. In effect, DG serves as a teacher to CA3, correcting its patterns into more pattern-separated ones, thereby reducing interference. Results showed that Theremin, compared with our original Hebbian-based model, has significantly increased capacity and learning speed. The model makes several novel predictions that can be tested in future studies.

Response Flexibility: The Role of the Lateral Habenula

The ability to make appropriate decisions that result in an optimal outcome is critical for survival. This process involves assessing the environment as well as integrating prior knowledge about the environment with information about one’s current internal state. There are many neural structures that play critical roles in mediating these processes, but it is not yet known how such information coalesces to influence behavioral output. The lateral habenula (LHb) has often been cited as a structure critical for adaptive and flexible responding when environmental contexts and internal state changes. A challenge, however, has been understanding how LHb promotes response flexibility. In this review, we hypothesize that the LHb enables flexible responding following the integration of context memory and internal state information by signaling downstream brainstem structures known to drive hippocampal theta. In this way, animals respond more flexibly in a task situation not because the LHb selects a particular action, but rather because LHb enhances a hippocampal neural state that is often associated with greater attention, arousal, and exploration. In freely navigating animals, these are essential conditions that are needed to discover and implement appropriate alternative choices and behaviors. As a corollary to our hypothesis, we describe short- and intermediate-term functions of the LHb. Finally, we discuss the effects on the behavior of LHb dysfunction in short- and intermediate-timescales, and then suggest that new therapies may act on the LHb to alleviate the behavioral impairments following long-term LHb disruption.

Prediction Errors for Aversive Events Shape Long-Term Memory Formation through a Distinct Neural Mechanism

Prediction errors (PEs) have been known for decades to guide associative learning, but their role in episodic memory formation has been discovered only recently. To identify the neural mechanisms underlying the impact of aversive PEs on long-term memory formation, we used functional magnetic resonance imaging, while participants saw a series of unique stimuli and estimated the probability that an aversive shock would follow. Our behavioral data showed that negative PEs (i.e., omission of an expected outcome) were associated with superior recognition of the predictive stimuli, whereas positive PEs (i.e., presentation of an unexpected outcome) impaired subsequent memory. While medial temporal lobe (MTL) activity during stimulus encoding was overall associated with enhanced memory, memory-enhancing effects of negative PEs were linked to even decreased MTL activation. Additional large-scale network analyses showed PE-related increases in crosstalk between the "salience network" and a frontoparietal network commonly implicated in memory formation for expectancy-congruent events. These effects could not be explained by mere changes in physiological arousal or the prediction itself. Our results suggest that the superior memory for events associated with negative aversive PEs is driven by a potentially distinct neural mechanism that might serve to set these memories apart from those with expected outcomes.

The anterior thalamic nuclei and cognition: A role beyond space?

•

Anterior thalamic nuclei important for specific classes of temporal discriminations.

•

Anterior thalamic nuclei required for hippocampal-dependent contextual processes.

•

Critical role for anterior thalamic nuclei in selective attention.

•

Significance of anterior thalamic – anterior cingulate interactions.

The anterior thalamic nuclei are a vital node within hippocampal-diencephalic-cingulate circuits that support spatial learning and memory. Reflecting this interconnectivity, the overwhelming focus of research into the cognitive functions of the anterior thalamic nuclei has been spatial processing. However, there is increasing evidence that the functions of the anterior thalamic nuclei extend beyond the spatial realm. This work has highlighted how these nuclei are required for certain classes of temporal discrimination as well as their importance for processing other contextual information; revealing parallels with the non-spatial functions of the hippocampal formation. Yet further work has shown how the anterior thalamic nuclei may be important for other forms of non-spatial learning, including a critical role for these nuclei in attentional mechanisms. This evidence signals the need to reconsider the functions of the anterior thalamic within the framework of their wider connections with sites including the anterior cingulate cortex that subserve non-spatial functions.

What's in a context? Cautions, limitations, and potential paths forward

The purpose of memory is to guide current and future behavior based on previous experiences. Part of this process involves either discriminating between or generalizing across similar experiences that contain overlapping conditions (such as space, time, or internal state), which we often conceptualize as "contexts". In this review, we highlight major challenges facing the field as we attempt a neuroscience-based approach to the study of context and its impact on learning and memory. Here, we review some of the methodologies and approaches used to investigate context in both animals and humans, including the neurobiological mechanisms involved. Finally, we propose three tenets for operationalizing context in the experimental setting: 1) contexts must be stable over time along an experiential dimension; 2) contexts must be at least moderately complex in nature and their representations must be modifiable or adaptable, and 3) contexts must have some behavioral relevance (be it overt or incidental) so that its role can be measured.

Reward-Based Spatial Learning in Teens With Bulimia Nervosa

OBJECTIVE

To assess the functioning of mesolimbic and fronto-striatal areas involved in reward-based spatial learning in teenaged girls with bulimia nervosa (BN) that might be involved in the development and maintenance of maladaptive behaviors characteristic of the disorder.

METHOD

We compared functional magnetic resonance imaging blood oxygen level-dependent response in 27 adolescent girls with BN to that of 27 healthy, age-matched control participants during a reward-based learning task that required learning to use extra-maze cues to navigate a virtual 8-arm radial maze to find hidden rewards. We compared groups in their patterns of brain activation associated with reward-based spatial learning versus a control condition in which rewards were unexpected because they were allotted pseudo-randomly to experimentally prevent learning.

RESULTS

Both groups learned to navigate the maze to find hidden rewards, but group differences in brain activity associated with maze navigation and reward processing were detected in the fronto-striatal regions and right anterior hippocampus. Unlike healthy adolescents, those with BN did not engage the right inferior frontal gyrus during maze navigation, activated the right anterior hippocampus during the receipt of unexpected rewards (control condition), and deactivated the left superior frontal gyrus and right anterior hippocampus during expected reward receipt (learning condition). These patterns of hippocampal activation in the control condition were significantly associated with the frequency of binge-eating episodes.

CONCLUSION

Adolescents with BN displayed abnormal functioning of the anterior hippocampus and fronto-striatal regions during reward-based spatial learning. These findings suggest that an imbalance in control and reward circuits may arise early in the course of BN. Clinical trial registration information-An fMRI Study of Self-Regulation in Adolescents With Bulimia Nervosa; https://clinicaltrials.gov/; NCT00345943.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Integrative hippocampal and decision-making neurocircuitry during goal-relevant predictions and encoding

It has become clear that the hippocampus plays a critical role in the identification of new contexts and for the detection of changes in familiar contexts. The hippocampus accomplishes these goals through a continual process of comparing predicted features of a context or situation to those actually experienced. A mismatch between expected and experienced context expectations is thought to lead to the generation of a context prediction error (Mizumori, 2013) that functionally alerts connected brain areas to alter subsequent decision making and response selection. Little is understood about how hippocampal context analyses impact downstream decision processes. This issue is evaluated here first by comparing the nature of the information represented in hippocampus and decision-related midbrain-striatal structures, while rats perform a hippocampal-dependent spatial memory task in which rewards of different value are found at different locations. In contrast to place-specific and egocentric neural representations, neural representations of goal information are broadly distributed in hippocampal and decision neural circuitry, but they appear in different forms for different brain structures. It is suggested that further researching on how goal information processing occurs in hippocampus and decision neural circuitry may reveal insights into the nature of the interaction between memory and decision systems. The second part of this review describes neural pathways by which hippocampal context information might arrive within the decision circuit. The third section presents a hypothesis that the nature of the interactions between hippocampal and midbrain-striatal circuitry is regulated by the prefrontal cortex.

Modulating the map: dopaminergic tuning of hippocampal spatial coding and interactions

Salient events activate the midbrain dopaminergic system and have important impacts on various aspects of mnemonic function, including the stability of hippocampus-dependent memories. Dopamine is also central to modulation of neocortical memory processing, particularly during prefrontal cortex-dependent working memory. Here, we review the current state of the circuitry and physiology underlying dopamine's actions, suggesting that--alongside local effects within hippocampus and prefrontal cortex--dopamine released from the midbrain ventral tegmental area is well positioned to dynamically tune interactions between limbic-cortical circuits through modulation of rhythmic network activity.

Functional heterogeneity of the limbic thalamus: From hippocampal to cortical functions

Today, the idea that the integrity of the limbic thalamus is necessary for normal memory functions is well established. However, if the study of thalamic patients emphasized the anterior and the mediodorsal thalamus as the critical thalamic loci supporting cognitive functions, clinical studies have so far failed to attribute a specific role to each of these regions. In view of these difficulties, we review here the experimental data conducted in rodents harboring specific lesions of each thalamic region. These data clearly indicate a major functional dissociation within the limbic thalamus. The anterior thalamus provides critical support for hippocampal functions due to its cardinal location in the Papez circuit, while the mediodorsal thalamus may signal relevant information in a circuit encompassing the basolateral amygdala and the prefrontal cortex. Interestingly, while clinical studies have suggested that diencephalic pathologies may disconnect the medial temporal lobe from the cortex, experimental studies conducted in rodent show how this may differently affect distinct temporo-thalamo-cortical circuits, sharing the same general organization but supporting dissociable functions.

A role for the lateral dorsal tegmentum in memory and decision neural circuitry

A role for the hippocampus in memory is clear, although the mechanism for its contribution remains a matter of debate. Converging evidence suggests that hippocampus evaluates the extent to which context-defining features of events occur as expected. The consequence of mismatches, or prediction error, signals from hippocampus is discussed in terms of its impact on neural circuitry that evaluates the significance of prediction errors: Ventral tegmental area (VTA) dopamine cells burst fire to rewards or cues that predict rewards (Schultz, Dayan, & Montague, 1997). Although the lateral dorsal tegmentum (LDTg) importantly controls dopamine cell burst firing (Lodge & Grace, 2006) the behavioral significance of the LDTg control is not known. Therefore, we evaluated LDTg functional activity as rats performed a spatial memory task that generates task-dependent reward codes in VTA (Jo, Lee, & Mizumori, 2013; Puryear, Kim, & Mizumori, 2010) and another VTA afferent, the pedunculopontine nucleus (PPTg, Norton, Jo, Clark, Taylor, & Mizumori, 2011). Reversible inactivation of the LDTg significantly impaired choice accuracy. LDTg neurons coded primarily egocentric information in the form of movement velocity, turning behaviors, and behaviors leading up to expected reward locations. A subset of the velocity-tuned LDTg cells also showed high frequency bursts shortly before or after reward encounters, after which they showed tonic elevated firing during consumption of small, but not large, rewards. Cells that fired before reward encounters showed stronger correlations with velocity as rats moved toward, rather than away from, rewarded sites. LDTg neural activity was more strongly regulated by egocentric behaviors than that observed for PPTg or VTA cells that were recorded by Puryear et al. and Norton et al. While PPTg activity was uniquely sensitive to ongoing sensory input, all three regions encoded reward magnitude (although in different ways), reward expectation, and reward encounters. Only VTA encoded reward prediction errors. LDTg may inform VTA about learned goal-directed movement that reflects the current motivational state, and this in turn may guide VTA determination of expected subjective goal values. When combined it is clear the LDTg and PPTg provide only a portion of the information that dopamine cells need to assess the value of prediction errors, a process that is essential to future adaptive decisions and switches of cognitive (i.e. memorial) strategies and behavioral responses.

The form and function of hippocampal context representations

Context is an essential component of learning and memory processes, and the hippocampus is critical for encoding contextual information. However, connecting hippocampal physiology with its role in context and memory has only recently become possible. It is now clear that contexts are represented by coherent ensembles of hippocampal neurons and new optogenetic stimulation studies indicate that activity in these ensembles can trigger the retrieval of context appropriate memories. We interpret these findings in the light of recent evidence that the hippocampus is critically involved in using contextual information to prevent interference, and propose a theoretical framework for understanding contextual influence on memory retrieval. When a new context is encountered, a unique hippocampal ensemble is recruited to represent it. Memories for events that occur in the context become associated with the hippocampal representation. Revisiting the context causes the hippocampal context code to be re-expressed and the relevant memories are primed. As a result, retrieval of appropriate memories is enhanced and interference from memories belonging to other contexts is minimized.