Predictions transform memories: How expected versus unexpected events are integrated or separated in memory

Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.

Lack of source memory as a potential marker of early assimilation of novel items into current knowledge

In daily life, humans process a plethora of new information that can be either consistent (familiar) or inconsistent (novel) with prior knowledge. Over time, both types of information can integrate into our accumulated knowledge base via distinct pathways. However, the mnemonic processes supporting the integration of information that is inconsistent with prior knowledge remain under-characterized. In the current study, we used functional magnetic resonance imaging (fMRI) to examine the initial assimilation of novel items into the semantic network. Participants saw three repetitions of adjective-noun word pairs that were either consistent or inconsistent with prior knowledge. Twenty-four hours later, they were presented with the same stimuli again while undergoing fMRI scans. Outside the scanner, participants completed a surprise recognition test. We found that when the episodic context associated with initially inconsistent items was irretrievable, the neural signature of these items was indistinguishable from that of consistent items. In contrast, initially inconsistent items with accessible episodic contexts showed neural signatures that differed from those associated with consistent items. We suggest that, at least one day post encoding, items inconsistent with prior knowledge can show early assimilation into the semantic network only when their episodic contexts become inaccessible during retrieval, thus evoking a sense of familiarity.

Developmental differences in memory reactivation relate to encoding and inference in the human brain

Despite the fact that children can draw on their memories to make novel inferences, it is unknown whether they do so through the same neural mechanisms as adults. We measured memory reinstatement as participants aged 7-30 years learned new, related information. While adults brought memories to mind throughout learning, adolescents did so only transiently, and children not at all. Analysis of trial-wise variability in reactivation showed that discrepant neural mechanisms-and in particular, what we interpret as suppression of interfering memories during learning in early adolescence-are nevertheless beneficial for later inference at each developmental stage. These results suggest that while adults build integrated memories well-suited to informing inference directly, children and adolescents instead must rely on separate memories to be individually referenced at the time of inference decisions.

The hippocampus shows an own-age bias during unfamiliar face viewing

The present study investigated the neural correlates of the own-age bias for face recognition in a repetition suppression paradigm. Healthy young and older adults viewed upright and inverted unfamiliar faces. Some of the upright faces were repeated following one of two delays (lag 0 or lag 11). Repetition suppression effects were observed in bilateral fusiform cortex. However, there were no significant effects indicating an own-age bias in repetition suppression. The absence of these effects is arguably inconsistent with perceptual expertise accounts of own-age biases in face processing. By contrast, the right anterior hippocampus showed an own-age bias (greater activity for own-age compared to other-age faces) when viewing an unfamiliar face for the first time. Given the importance of the hippocampus for episodic memory encoding, we conjecture that the increased hippocampal activity for own-age relative to other-age faces reflects differential engagement of neural processes supporting the episodic encoding of faces and might provide insight into the neural underpinnings of own-age biases in face recognition memory.

Building memories on prior knowledge: behavioral and fMRI evidence of impairment in early Alzheimer's disease

Impaired memory is a hallmark of prodromal Alzheimer's disease (AD). Prior knowledge associated with the memoranda improves memory in healthy individuals, but we ignore whether the same occurs in early AD. We used functional MRI to investigate whether prior knowledge enhances memory encoding in early AD, and whether the nature of this prior knowledge matters. Patients with early AD and Controls underwent a task-based fMRI experiment where they learned face-scene associations. Famous faces carried pre-experimental knowledge (PEK), while unknown faces with which participants were familiarized prior to learning carried experimental knowledge (EK). Surprisingly, PEK strongly enhanced subsequent memory in healthy controls, but importantly not in patients. Partly nonoverlapping brain networks supported PEK vs. EK associative encoding in healthy controls. No such networks were identified in patients. In addition, patients displayed impaired activation in a right sub hippocampal region where activity predicted successful associative memory formation for PEK stimuli. Despite the limited sample sizes of this study, these findings suggest that the role prior knowledge in new learning might have been so far overlooked and underestimated in AD patients. Prior knowledge may drive critical differences in the way healthy elderly and early AD patients learn novel associations.

Examining the transition of novel information toward familiarity

Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.

Stimulus valence, episodic memory, and the priming of brain activation profiles in borderline personality disorder

BACKGROUND

Borderline personality disorder (BPD) is characterized by instability in affective regulation that can result in a loss of cognitive control. Triggers may be neuronal responses to emotionally valenced context and/or stimuli. 'Neuronal priming' indexes the familiarity of stimuli, and may capture the obligatory effects of affective valence on the brain's processing system, and how such valence mediates responses to the repeated presentation of stimuli. We investigated the effects of affective valence of stimuli on neuronal priming (i.e. changes in activation to repeated presentation of stimuli), and if these effects distinguished BPD patients from controls.

METHODS

Forty BPD subjects and 25 control subjects (age range: 18-44) participated in an episodic memory task during fMRI. Stimuli were presented in alternating epochs of encoding (six images of positive, negative, and neutral valence) and recognition (six images for 'old' v. 'new' recognition). Analyses focused on inter-group differences in the change in activation to repeated stimuli (presented during Encoding and Recognition).

RESULTS

Relative to controls, BPD showed greater priming (generally greater decrease from encoding to recognition) for negatively valenced stimuli. Conversely, BPD showed less priming for positively valenced stimuli (generally greater increase from encoding to recognition).

CONCLUSION

Plausibly, the relative familiarity of negative valence to patients with BPD exerts an influence on obligatory responses to repeated stimuli leading to repetition priming of neuronal profiles. The specific effects of valence on memory and/or attention, and consequently on priming can inform the understanding of mechanisms of altered salience for affective stimuli in BPD.

Distinct patterns of repetition suppression in Fragile X syndrome, down syndrome, tuberous sclerosis complex and mutations in SYNGAP1

Sensory processing is the gateway to information processing and more complex processes such as learning. Alterations in sensory processing is a common phenotype of many genetic syndromes associated with intellectual disability (ID). It is currently unknown whether sensory processing alterations converge or diverge on brain responses between syndromes. Here, we compare for the first time four genetic conditions with ID using the same basic sensory learning paradigm. One hundred and five participants, aged between 3 and 30 years old, composing four clinical ID groups and one control group, were recruited: Fragile X syndrome (FXS; n = 14), tuberous sclerosis complex (TSC; n = 9), Down syndrome (DS; n = 19), SYNGAP1 mutations (n = 8) and Neurotypical controls (NT; n = 55)). All groups included female and male participants. Brain responses were recorded using electroencephalography (EEG) during an audio-visual task that involved three repetitions of the pronunciation of the phoneme /a/. Event Related Potentials (ERP) were used to: 1) compare peak-to-peak amplitudes between groups, 2) evaluate the presence of repetition suppression within each group and 3) compare the relative repetition suppression between groups. Our results revealed larger overall amplitudes in FXS. A repetition suppression (RS) pattern was found in the NT group, FXS and DS, suggesting spared repetition suppression in a multimodal task in these two ID syndromes. Interestingly, FXS presented a stronger RS on one peak-to-peak value in comparison with the NT. The results of our study reveal the distinctiveness of ERP and RS brain responses in ID syndromes. Further studies should be conducted to understand the molecular mechanisms involved in these patterns of responses.

Prior knowledge promotes hippocampal separation but cortical assimilation in the left inferior frontal gyrus

An adaptive memory system rarely learns information tabula rasa, but rather builds on prior knowledge to facilitate learning. How prior knowledge influences the neural representation of novel associations remains unknown. Here, participants associated pairs of faces in two conditions: a famous, highly familiar face with a novel face or two novel faces while undergoing fMRI. We examine multivoxel activity patterns corresponding to individual faces before and after learning. The activity patterns representing members of famous-novel pairs becomes separated in the hippocampus, that is, more distinct from one another through learning, in striking contrast to paired novel faces that become similar. In the left inferior frontal gyrus, however, prior knowledge leads to integration, and in a specific direction: the representation of the novel face becomes similar to that of the famous face after learning, suggesting assimilation of new into old memories. We propose that hippocampal separation might resolve interference between existing and newly learned information, allowing cortical assimilation. Thus, associative learning with versus without prior knowledge relies on radically different computations.

Prior knowledge strongly impacts new learning, but its influence on the neural representation of novel information is unknown. Here, the authors show multiple neural codes for learning: prior knowledge leads to integrated cortical representations, while promoting hippocampal separation.

Perceived Duration: The Interplay of Top-Down Attention and Task-Relevant Information

Perception of time is susceptible to distortions; among other factors, it has been suggested that the perceived duration of a stimulus is affected by the observer’s expectations. It has been hypothesized that the duration of an oddball stimulus is overestimated because it is unexpected, whereas repeated stimuli have a shorter perceived duration because they are expected. However, recent findings suggest instead that fulfilled expectations about a stimulus elicit an increase in perceived duration, and that the oddball effect occurs because the oddball is a target stimulus, not because it is unexpected. Therefore, it has been suggested that top-down attention is sometimes sufficient to explain this effect, and sometimes only necessary, with an additional contribution from saliency. However, how the expectedness of a target stimulus and its salient features affect its perceived duration is still an open question. In the present study, participants’ expectations about and the saliency of target stimuli were orthogonally manipulated with stimuli presented on a short (Experiment 1) or long (Experiment 2) temporal scale. Four repetitive standard stimuli preceded each target stimulus in a task in which participants judged whether the target was longer or shorter in duration than the standards. Engagement of top-down attention to target stimuli increased their perceived duration to the same extent irrespective of their expectedness. A small but significant additional contribution to this effect from the saliency of target stimuli was dependent on the temporal scale of stimulus presentation. In Experiment 1, saliency only significantly increased perceived duration in the case of expected target stimuli. In contrast, in Experiment 2, saliency exerted a significant effect on the overestimation elicited by unexpected target stimuli, but the contribution of this variable was eliminated in the case of expected target stimuli. These findings point to top-down attention as the primary cognitive mechanism underlying the perceptual extraction and processing of task-relevant information, which may be strongly correlated with perceived duration. Furthermore, the scalar properties of timing were observed, favoring the pacemaker-accumulator model of timing as the underlying timing mechanism.

Knowledge supports memory retrieval through familiarity, not recollection

Semantic memory, or general knowledge of the world, guides learning and supports the formation and retrieval of new episodic memories. Behavioral evidence suggests that this knowledge effect is supported by recollection-a more controlled form of memory retrieval generally accompanied by contextual details-to a greater degree than familiarity-a more automatic form of memory retrieval generally absent of contextual details. In the current study, we used functional magnetic resonance imaging (fMRI) to investigate the role that regions associated with recollection and familiarity play in retrieving recent instances of known (e.g., The Summer Olympic Games are held four years apart) and unknown (e.g., A flaky deposit found in port bottles is beeswing) statements. Our results revealed a surprising pattern: Episodic retrieval of known statements recruited regions associated with familiarity, but not recollection. Instead, retrieval of unknown statements recruited regions associated with recollection. These data, in combination with quicker reaction times for the retrieval of known than unknown statements, suggest that known statements can be successfully retrieved on the basis of familiarity, whereas unknown statements were retrieved on the basis of recollection. Our results provide insight into how knowledge influences episodic retrieval and demonstrate the role of neuroimaging in providing insights into cognitive processes in the absence of explicit behavioral responses.

Specifying the role of the ventromedial prefrontal cortex in memory formation

Recent neuroimaging research suggests that the ventromedial prefrontal cortex (vmPFC) plays an important role for successful memory formation that takes place in the context of activated prior knowledge. These findings led to the notion that the vmPFC integrates new information into existing knowledge structures. However, a considerable number of neuroimaging studies that have investigated memory formation in the context of prior knowledge have not found vmPFC involvement. To resolve this inconsistency, we propose a distinction between knowledge-relevance (the degree to which new information can be linked to prior knowledge) and knowledge-congruency (the perceived match between prior knowledge and the to-be-encoded information). We hypothesized that the vmPFC contributes to successful memory formation only when perceived knowledge-congruency is high, independent of knowledge-relevance. We tested this hypothesis in a design that varied both congruency and relevance during memory encoding, which was performed in the MR scanner. As predicted, the results showed that vmPFC contributions to memory formation vary as a function of knowledge-congruency, but not as a function of knowledge-relevance. Our finding contributes to elucidating the seemingly inconsistent findings in the literature and helps to specify the role of the vmPFC in memory formation.

Memory integration constructs maps of space, time, and concepts

Recent evidence demonstrates that new events are learned in the context of their relationships to existing memories. Within the hippocampus and medial prefrontal cortex, related memories are represented by integrated codes that connect events experienced at different times and places. Integrated codes form the basis of spatial, temporal, and conceptual maps of experience. These maps represent information that goes beyond direct experience and support generalization behaviors that require knowledge be used in new ways. The degree to which an individual memory is integrated into a coherent map is determined by its spatial, temporal, and conceptual proximity to existing knowledge. Integration is observed over a wide range of scales, suggesting that memories contain information about both broad and fine-grained contexts.

A multisensory perspective on object memory

Traditional studies of memory and object recognition involved objects presented within a single sensory modality (i.e., purely visual or purely auditory objects). However, in naturalistic settings, objects are often evaluated and processed in a multisensory manner. This begets the question of how object representations that combine information from the different senses are created and utilised by memory functions. Here we review research that has demonstrated that a single multisensory exposure can influence memory for both visual and auditory objects. In an old/new object discrimination task, objects that were presented initially with a task-irrelevant stimulus in another sense were better remembered compared to stimuli presented alone, most notably when the two stimuli were semantically congruent. The brain discriminates between these two types of object representations within the first 100ms post-stimulus onset, indicating early "tagging" of objects/events by the brain based on the nature of their initial presentation context. Interestingly, the specific brain networks supporting the improved object recognition vary based on a variety of factors, including the effectiveness of the initial multisensory presentation and the sense that is task-relevant. We specify the requisite conditions for multisensory contexts to improve object discrimination following single exposures, and the individual differences that exist with respect to these improvements. Our results shed light onto how memory operates on the multisensory nature of object representations as well as how the brain stores and retrieves memories of objects.

Building concepts one episode at a time: The hippocampus and concept formation

Concepts organize our experiences and allow for meaningful inferences in novel situations. Acquiring new concepts requires extracting regularities across multiple learning experiences, a process formalized in mathematical models of learning. These models posit a computational framework that has increasingly aligned with the expanding repertoire of functions associated with the hippocampus. Here, we propose the Episodes-to-Concepts (EpCon) theoretical model of hippocampal function in concept learning and review evidence for the hippocampal computations that support concept formation including memory integration, attentional biasing, and memory-based prediction error. We focus on recent studies that have directly assessed the hippocampal role in concept learning with an innovative approach that combines computational modeling and sophisticated neuroimaging measures. Collectively, this work suggests that the hippocampus does much more than encode individual episodes; rather, it adaptively transforms initially-encoded episodic memories into organized conceptual knowledge that drives novel behavior.

Neurobiology of Schemas and Schema-Mediated Memory

Schemas are superordinate knowledge structures that reflect abstracted commonalities across multiple experiences, exerting powerful influences over how events are perceived, interpreted, and remembered. Activated schema templates modulate early perceptual processing, as they get populated with specific informational instances (schema instantiation). Instantiated schemas, in turn, can enhance or distort mnemonic processing from the outset (at encoding), impact offline memory transformation and accelerate neocortical integration. Recent studies demonstrate distinctive neurobiological processes underlying schema-related learning. Interactions between the ventromedial prefrontal cortex (vmPFC), hippocampus, angular gyrus (AG), and unimodal associative cortices support context-relevant schema instantiation and schema mnemonic effects. The vmPFC and hippocampus may compete (as suggested by some models) or synchronize (as suggested by others) to optimize schema-related learning depending on the specific operationalization of schema memory. This highlights the need for more precise definitions of memory schemas.

Distinctiveness Benefits Novelty (and Not Familiarity), but Only Up to a Limit: The Prior Knowledge Perspective

Novelty is a pivotal player in cognition, and its contribution to superior memory performance is a widely accepted convention. On the other hand, mnemonic advantages for familiar information are also well documented. Here, we examine the role of experimental distinctiveness as a potential explanation for these apparently conflicting findings. Across two experiments, we demonstrate that conceptual novelty, an unfamiliar combination of familiar constituents, is sensitive to its experimental proportions: Improved memory for novelty was observed when novel stimuli were relatively rare. Memory levels for familiar items, in contrast, were completely unaffected by experimental proportions, highlighting their insensitivity to list-based distinctiveness. Finally, no mnemonic advantage for conceptual novelty over familiarity was observed even when novel stimuli were extremely rare at study. Together, these results imply that novel and familiar items are processed via partially distinct mechanisms, with (at least some facets of) novelty not providing a mnemonic advantage over familiarity.

Semantic Congruency Improves Recognition Memory Performance for Both Audiovisual and Visual Stimuli

Audiovisual semantic congruency during memory encoding has been shown to facilitate later recognition memory performance. However, it is still unclear whether this improvement is due to multisensory semantic congruency or just semantic congruencyper se. We investigated whether dual visual encoding facilitates recognition memory in the same way as audiovisual encoding. The participants memorized auditory or visual stimuli paired with a semantically congruent, incongruent or non-semantic stimulus in the same modality or in the other modality during encoding. Subsequent recognition memory performance was better when the stimulus was initially paired with a semantically congruent stimulus than when it was paired with a non-semantic stimulus. This congruency effect was observed with both audiovisual and dual visual stimuli. The present results indicate that not only multisensory but also unisensory semantically congruent stimuli can improve memory performance. Thus, the semantic congruency effect is not solely a multisensory phenomenon, as has been suggested previously.