Neural Global Pattern Similarity Underlies True and False Memories

Temporal asymmetry of neural representations predicts memory decisions

A stimulus can be familiar for multiple reasons. It might have been recently encountered, or is similar to recent experience, or is similar to 'typical' experience. Understanding how the brain translates these sources of similarity into memory decisions is a fundamental, but challenging goal. Here, using fMRI, we computed neural similarity between a current stimulus and events from different temporal windows in the past and future (from seconds to days). We show that trial-by-trial memory decisions (is this stimulus 'old'?) were predicted by the difference in similarity to past vs. future events (temporal asymmetry). This relationship was (i) evident in lateral parietal and occipitotemporal cortices, (ii) strongest when considering events from the recent past (minutes ago), and (iii) most pronounced when veridical (true) memories were weak. These findings suggest a new perspective in which the brain supports memory decisions by comparing what actually occurred to what is likely to occur.

False memories in forensic psychology: do cognition and brain activity tell the same story?

One of the most important problems in forensic psychology is the impossibility of reliably discriminating between true and false memories when the only prosecution evidence comes from the memory of a witness or a victim. Unfortunately, both children and adults can be persuaded that they have been victims of past criminal acts, usually of a sexual nature. In adults, suggestion often occurs in the context of suggestive therapies based on the belief that traumatic events are repressed, while children come to believe and report events that never occurred as a result of repeated suggestive questioning. Cognitive Researchers have designed false memory paradigms (i.e., misinformation effect, Deese-Roediger-McDermott paradigm, event implantation paradigm) to first form false memories and then determine whether it is possible to reliably differentiate between false and true memories. In the present study, we review the contribution of cognitive research to the formation of false memories and the neuropsychological approaches aimed to discriminate between true and false memories. Based on these results, we analyze the applicability of the cognitive and neuropsychological evidence to the forensic setting.

The memory trace of an intrusive trauma-analog episode

Intrusive memories are a core symptom of posttraumatic stress disorder. Compared with memories of everyday events, they are characterized by several seemingly contradictory features: intrusive memories contain distinct sensory and emotional details of the traumatic event and can be triggered by various perceptually similar cues, but they are poorly integrated into conceptual memory. Here, we conduct exploratory whole-brain analyses to investigate the neural representations of trauma-analog experiences and how they are reactivated during memory intrusions. We show that trauma-analog movies induce excessive processing and generalized representations in sensory areas but decreased blood-oxygen-level-dependent (BOLD) responses and highly distinct representations in conceptual/semantic areas. Intrusive memories activate generalized representations in sensory areas and reactivate memory traces specific to trauma-analog events in the anterior cingulate cortex. These findings provide the first evidence of how traumatic events could distort memory representations in the human brain, which may form the basis for future confirmatory research on the neural representations of traumatic experiences.

Intrinsic functional connectivity in medial temporal lobe networks is associated with susceptibility to misinformation

Memory is notoriously fallible and susceptible to misinformation. Yet little is known about the underlying cognitive and neural mechanisms that render individuals vulnerable to this type of false memory. The current experiments take an individual differences approach to examine whether susceptibility to misinformation reflects stable underlying factors related to memory retrieval. In Study 1, we report for the first time the existence of substantial individual variability in susceptibility to misinformation in the context of repeated memory retrieval, when the misinformation effect is most pronounced. This variability was not related to an individual's tendency to adopt an episodic retrieval style during remembering (trait mnemonics). In Study 2, we next examined whether susceptibility to misinformation is related to intrinsic functional connectivity in medial temporal lobe (MTL) networks known to coordinate memory reactivation during event retrieval. Stronger resting-state functional connectivity between the MTL and cortical areas associated with visual memory reactivation (occipital cortex) was associated with better protection from misinformation. Together, these results reveal that while memory distortion is a universal property of our reconstructive memory system, susceptibility to misinformation varies at the individual level and may depend on one's ability to reactivate visual details during memory retrieval.

Predictions transform memories: How expected versus unexpected events are integrated or separated in memory

Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Cross-stage neural pattern similarity in the hippocampus predicts false memory derived from post-event inaccurate information

The misinformation effect occurs when people's memory of an event is altered by subsequent inaccurate information. No study has systematically tested theories about the dynamics of human hippocampal representations during the three stages of misinformation-induced false memory. This study replicates behavioral results of the misinformation effect, and investigates the cross-stage pattern similarity in the hippocampus and cortex using functional magnetic resonance imaging. Results show item-specific hippocampal pattern similarity between original-event and post-event stages. During the memory-test stage, hippocampal representations of original information are weakened for true memory, whereas hippocampal representations of misinformation compete with original information to create false memory. When false memory occurs, this conflict is resolved by the lateral prefrontal cortex. Individuals' memory traces of post-event information in the hippocampus predict false memory, whereas original information in the lateral parietal cortex predicts true memory. These findings support the multiple-trace model, and emphasize the reconstructive nature of human memory.

The depth of semantic processing modulates cross-language pattern similarity in Chinese-English bilinguals

Previous studies have investigated factors related to the degree of cross-language overlap in brain activations in bilinguals/multilinguals. However, it is still unclear whether and how the depth of semantic processing (a critical task-related factor) affects the neural pattern similarity between native and second languages. To address this question, 26 Chinese-English bilinguals were scanned with fMRI while performing a word naming task (i.e., a task with shallow semantic processing) and a semantic judgment task (i.e., a task with deep semantic processing) in both native and second languages. Based on three sets of representational similarity analysis (whole brain, ROI-based, and within-language vs. cross-language semantic representation), we found that select regions in the reading brain network showed higher cross-language pattern similarity and higher cross-language semantic representations during deep semantic processing than during shallow semantic processing. These results suggest that compared to shallow semantic processing, deep semantic processing may lead to greater language-independent processing (i.e., cross-language semantic representation) and cross-language pattern similarity, and provide direct quantitative neuroimaging evidence for cognitive models of bilingual lexical memory.

False memory confidence depends on the prefrontal reinstatement of true memory

For confidence of memory, a neural basis such as traces of stored memories should be required. However, because false memories have never been stored, the neural basis for false memory confidence remains unclear. Here we monitored the brain activity in participants while they viewed learned or novel objects, subsequently decided whether each presented object was learned and assessed their confidence levels. We found that when novel objects are presented, false memory confidence significantly depends on the shared representations with learned objects in the prefrontal cortex. However, such a tendency was not found in posterior regions including the visual cortex, which may be involved in the processing of perceptual gist. Furthermore, the confidence-dependent shared representations were not observed when participants correctly answered novel objects as non-learned objects. These results demonstrate that false memory confidence is critically based on the reinstatement of high-level semantic gist of stored memories in the prefrontal cortex.

Age Effects on Neural Discriminability and Monitoring Process During Memory Retrieval for Auditory Words

After hearing a list of words (e.g., dream, awake, and bed), older adults tended to have more difficulty than younger adults in distinguishing targets (e.g., dream) from lures (e.g., sleep) and foils (e.g., pen) in a visual recognition test. Age-related reduction in neural discriminability in the visual cortex has been linked to deficits in memory discriminability of pictures. However, no study has examined age differences in auditory discrimination and prefrontal monitoring during true and false memory retrieval after hearing words. The current study used a visual recognition test following an auditory study of words and showed that older adults had lower true recognition and higher propensity for high-confidence false recognition compared to young adults. Using classification-based multivariate pattern analysis for functional neuroimaging data during memory retrieval, we found that neural activation patterns in the primary auditory cortex could be used to distinguish between auditorily-studied targets and unstudied lures in young adults, but not in older adults. Moreover, prefrontal monitoring for lures was weaker in older adults as compared to young adults. Individual differences analysis showed that neural discriminability in the primary auditory cortex was positively related to true recognition, whereas prefrontal activation for lures was negatively related to the propensity for high-confidence false recognition in young adults but not in older adults. Together, age differences in true and false memories following auditory study are associated with reduced neural discriminability in the primary auditory cortex and reduced prefrontal monitoring during retrieval.

Dynamic changes in neural representations underlie the repetition effect on false memory

Restudying word lists (e.g., dream, awake, and bed) strengthens true memory of the studied words and reduces false memory for unstudied but semantically related lures (e.g., sleep). Yet, the neural mechanisms involved in this repetition effect on false memory remain unclear. Possible mechanisms involve item-specific and semantic neural representations at encoding, and the memory strength between encoding and retrieval. This study first replicated the behavioral results (Exp. 1) and then investigated various neural mechanisms by using slow event-related functional magnetic resonance imaging (fMRI) and representational similarity analysis (Exp. 2). Behavioral results confirmed that restudy improved true memory and reduced false memory. The fMRI results showed that restudy induced item-specific neural representations at encoding in the left occipital pole, but reduced neural overlap between semantic representations at encoding in the left temporal pole. Individual differences in these two encoding neural mechanisms were correlated with the behavioral measure of false memory, with greater restudy-induced representational changes at encoding (item-specific neural representations and reduced neural overlap between semantic representations) being associated with lower false memory. Moreover, restudy enhanced the memory strength between encoding and retrieval in the visuoparietal cortex but reduced it in the frontal cortex. These findings suggest that dynamic changes in neural representations underlie the repetition effect on false memory, supporting a dual-coding neural framework.

Contributions of representational distinctiveness and stability to memory performance and age differences

Long-standing theories of cognitive aging suggest that memory decline is associated with age-related differences in the way information is neurally represented. Multivariate pattern similarity analyses enabled researchers to take a representational perspective on brain and cognition, and allowed them to study the properties of neural representations that support successful episodic memory. Two representational properties have been identified as crucial for memory performance, namely the distinctiveness and the stability of neural representations. Here, we review studies that used multivariate analysis tools for different neuroimaging techniques to clarify how these representational properties relate to memory performance across adulthood. While most evidence on age differences in neural representations involved stimulus category information , recent studies demonstrated that particularly item-level stability and specificity of activity patterns are linked to memory success and decline during aging. Overall, multivariate methods offer a versatile tool for our understanding of age differences in the neural representations underlying memory.

Transformative neural representations support long-term episodic memory

Memory is often conceived as a dynamic process that involves substantial transformations of mental representations. However, the neural mechanisms underlying these transformations and their role in memory formation and retrieval have only started to be elucidated. Combining intracranial EEG recordings with deep neural network models, we provide a detailed picture of the representational transformations from encoding to short-term memory maintenance and long-term memory retrieval that underlie successful episodic memory. We observed substantial representational transformations during encoding. Critically, more pronounced semantic representational formats predicted better subsequent long-term memory, and this effect was mediated by more consistent item-specific representations across encoding events. The representations were further transformed right after stimulus offset, and the representations during long-term memory retrieval were more similar to those during short-term maintenance than during encoding. Our results suggest that memory representations pass through multiple stages of transformations to achieve successful long-term memory formation and recall.

Individual-specific and shared representations during episodic memory encoding and retrieval

Although human memories seem unique to each individual, they are shared to a great extent across individuals. Previous studies have examined, separately, subject-specific and cross-subject shared representations during memory encoding and retrieval, but how shared memories are formed from individually encoded representations is not clearly understood. Using a unique fMRI design involving memory encoding and retrieval, and representational similarity analysis to link representations from different individuals, brain regions, and processing stages, the current study revealed that distributed brain regions showed both subject-specific and shared neural representations during both memory encoding and retrieval. Furthermore, different brain regions showed stage-specific representational strength, with the visual cortex showing greater unique and shared representations during encoding, whereas the left angular gyrus showing greater unique and shared representations during retrieval. The neural representations during encoding were transformed during retrieval, as shown by smaller cross-subject encoding-retrieval similarity (ERS) than cross-subject similarity either during encoding or during retrieval. This cross-subject and cross-stage similarity was found both within and across regions, with strong pattern similarity between the encoded representation in VVC and the retrieved representation in the angular gyrus. Simulation analysis further suggested that these patterns could be achieved by incorporating stage-specific representational strength, and cross-region reinstatement from encoding to retrieval, but not by a common transformation from encoding to retrieval across subjects. Together, our results shed light on how memory representations are encoded and transformed to maintain individual characteristics and at the same time to create shared representations to facilitate interpersonal communication.

Eye movements support behavioral pattern completion

The ability to recall a detailed event from a simple reminder is supported by pattern completion, a cognitive operation performed by the hippocampus wherein existing mnemonic representations are retrieved from incomplete input. In behavioral studies, pattern completion is often inferred through the false endorsement of lure (i.e., similar) items as old. However, evidence that such a response is due to the specific retrieval of a similar, previously encoded item is severely lacking. We used eye movement (EM) monitoring during a partial-cue recognition memory task to index reinstatement of lure images behaviorally via the recapitulation of encoding-related EMs or gaze reinstatement. Participants reinstated encoding-related EMs following degraded retrieval cues and this reinstatement was negatively correlated with accuracy for lure images, suggesting that retrieval of existing representations (i.e., pattern completion) underlies lure false alarms. Our findings provide evidence linking gaze reinstatement and pattern completion and advance a functional role for EMs in memory retrieval.

Cortical Overlap and Cortical-Hippocampal Interactions Predict Subsequent True and False Memory

The declarative memory system allows us to accurately recognize a countless number of items and events, particularly those strengthened by repeated exposure. However, increased familiarity due to repetition can also lead to false recognition of related but new items, particularly when mechanisms supporting fine-grain mnemonic discrimination fail. The hippocampus is thought to be particularly important in separating overlapping cortical inputs during encoding so that similar experiences can be differentiated. In the current study of male and female human subjects, we examine how neural pattern similarity between repeated exemplars of a given concept (e.g., apple) influences true and false memory for target or lure images. Consistent with past work, we found that subsequent true recognition was related to pattern similarity between concept exemplars and the entire encoding set (global encoding similarity), particularly in ventral visual stream. In addition, memory for an individual target exemplar (a specific apple) could be predicted solely by the degree of pattern overlap between the other exemplars (different apple pictures) of that concept (concept-specific encoding similarity). Critically, subsequent false memory for lures was mitigated when high concept-specific similarity in cortical areas was accompanied by differentiated hippocampal representations of the corresponding exemplars. Furthermore, both true and false memory entailed the reinstatement of concept-related information at varying levels of specificity. These results link both true and false memory to a measure of concept strength expressed in the overlap of cortical representations, and importantly, illustrate how the hippocampus serves to separate concurrent cortical overlap in the service of detailed memory.SIGNIFICANCE STATEMENT In some instances, the same processes that help promote memory for a general idea or concept can also hinder more detailed memory judgments, which may involve differentiating between closely related items. The current study shows that increased overlap in cortical representations for conceptually-related pictures is associated with increased recognition of repeated concept pictures. Whether similar lure items were falsely remembered as old further depended on the hippocampus, where the presence of more distinct representations protected against later false memory. This work suggests that the differentiability of brain patterns during perception is related to the differentiability of items in memory, but that fine-grain discrimination depends on the interaction between cortex and hippocampus.

Neural Pattern Similarity Differentially Relates to Memory Performance in Younger and Older Adults

Age-related memory decline is associated with changes in neural functioning, but little is known about how aging affects the quality of information representation in the brain. Whereas a long-standing hypothesis of the aging literature links cognitive impairments to less distinct neural representations in old age ("neural dedifferentiation"), memory studies have shown that overlapping neural representations of different studied items are beneficial for memory performance. In an electroencephalography (EEG) study, we addressed the question whether distinctiveness or similarity between patterns of neural activity supports memory differentially in younger and older adults. We analyzed between-item neural pattern similarity in 50 younger (19-27 years old) and 63 older (63-75 years old) male and female human adults who repeatedly studied and recalled scene-word associations using a mnemonic imagery strategy. We compared the similarity of spatiotemporal EEG frequency patterns during initial encoding in relation to subsequent recall performance. The within-person association between memory success and pattern similarity differed between age groups: For older adults, better memory performance was linked to higher similarity early in the encoding trials, whereas young adults benefited from lower similarity between earlier and later periods during encoding, which might reflect their better success in forming unique memorable mental images of the joint picture-word pairs. Our results advance the understanding of the representational properties that give rise to subsequent memory, as well as how these properties may change in the course of aging.SIGNIFICANCE STATEMENT Declining memory abilities are one of the most evident limitations for humans when growing older. Despite recent advances of our understanding of how the brain represents and stores information in distributed activation patterns, little is known about how the quality of information representation changes during aging and thus affects memory performance. We investigated how the similarity between neural representations relates to subsequent memory in younger and older adults. We present novel evidence that the interaction of pattern similarity and memory performance differs between age groups: Older adults benefited from higher similarity during early encoding, whereas young adults benefited from lower similarity between early and later encoding. These results provide insights into the nature of memory and age-related memory deficits.

Spaced Learning Enhances Episodic Memory by Increasing Neural Pattern Similarity Across Repetitions

Spaced learning has been shown consistently to benefit memory compared with massed learning, yet the neural representations and processes underlying the spacing effect are still poorly understood. In particular, two influential models (i.e., the encoding variability hypothesis and the study-phase retrieval hypothesis) could both model behavioral performance very well, but they make opposite hypotheses regarding the spacing effect's neural mechanisms. The present study attempted to provide empirical neural evidence to adjudicate these competing hypotheses. Using spatiotemporal pattern similarity (STPS) analysis of EEG data, this study investigated whether and how repetition lags (massed/short-spaced/long-spaced) modulated the STPS's contribution to episodic memory encoding in male and female human participants. The results revealed that greater item-specific STPS in the right frontal electrodes at 543-727 ms after stimulus onset was associated with better memory performance. More importantly, this STPS was larger under the spaced-learning condition than the massed-learning condition and partially mediated the spacing effect on memory performance. In addition, we found that massed learning was associated with stronger repetition suppression in the N400 component that reflected momentary retrieval strength, but reduced activity in the late positive component that was associated with memory retrieval. These results suggest that spaced learning improves long-term memory by increasing retrieval effort and enhancing the pattern reinstatement of prior neural representations, which may be achieved by reducing the momentary retrieval strength as the extended repetition lags might help to eliminate the residual representation in working memory.SIGNIFICANCE STATEMENT As one of the most ubiquitous and fundamental phenomena in the history of memory research, the spacing effect provides an important window into understanding how enduring memory is formed in the brain and how different practice strategies could modulate these mechanisms to affect memory performance. By leveraging the neural representational analysis on scalp EEG data, the current study provides the first empirical data to show that spaced learning enhances memory by improving the spatiotemporal similarity that occurs at a late time window. Our results support the study-phase retrieval hypothesis but not the encoding variability hypothesis and emphasize the role of neural pattern reinstatement in strengthening memory via repeated study.

Neural specificity of scene representations is related to memory performance in childhood

Successful memory encoding is supported by medial temporal, retrosplenial, and occipital regions, which show developmental differences in recruitment from childhood to adulthood. However, little is known about the extent to which neural specificity in these brain regions, or the distinctiveness with which sensory information is represented, continues to develop during middle childhood and how it contributes to memory performance. The present study used multivariate pattern analysis to examine the distinctiveness of different scene representations in 169 children and 31 adults, and its relation to memory performance. Most children provided data over up to three measurement occasions between 8 and 15 years (267 total scans), allowing us to examine changes in memory and neural specificity over time. Memory performance was lower in children than in adults, and increased in children over time. Different scenes presented during memory encoding could be reliably decoded from parahippocampal, lateral occipital, and retrosplenial regions in children and adults. Neural specificity in children was similar to adults, and did not change reliably over time. Among children, higher neural specificity in scene-processing regions was associated with better memory concurrently. These results suggest that the distinctiveness with which incoming information is represented is important for memory performance in childhood, but other processes operating on these representations support developmental improvements in memory performance over time.

Multiple interactive memory representations underlie the induction of false memory

Theoretical and computational models such as transfer-appropriate processing (TAP) and global matching models have emphasized the encoding-retrieval interaction of memory representations in generating false memories, but relevant neural mechanisms are still poorly understood. By manipulating the sensory modalities (visual and auditory) at different processing stages (learning and test) in the Deese-Roediger-McDermott task, we found that the auditory-learning visual-test (AV) group produced more false memories (59%) than the other three groups (42∼44%) [i.e., visual learning visual test (VV), auditory learning auditory test (AA), and visual learning auditory test (VA)]. Functional imaging results showed that the AV group's proneness to false memories was associated with (i) reduced representational match between the tested item and all studied items in the visual cortex, (ii) weakened prefrontal monitoring process due to the reliance on frontal memory signal for both targets and lures, and (iii) enhanced neural similarity for semantically related words in the temporal pole as a result of auditory learning. These results are consistent with the predictions based on the TAP and global matching models and highlight the complex interactions of representations during encoding and retrieval in distributed brain regions that contribute to false memories.

Transformed Neural Pattern Reinstatement during Episodic Memory Retrieval

Contemporary models of episodic memory posit that remembering involves the reenactment of encoding processes. Although encoding-retrieval similarity has been consistently reported and linked to memory success, the nature of neural pattern reinstatement is poorly understood. Using high-resolution fMRI on human subjects, our results obtained clear evidence for item-specific pattern reinstatement in the frontoparietal cortex, even when the encoding-retrieval pairs shared no perceptual similarity. No item-specific pattern reinstatement was found in the ventral visual cortex. Importantly, the brain regions and voxels carrying item-specific representation differed significantly between encoding and retrieval, and the item specificity for encoding-retrieval similarity was smaller than that for encoding or retrieval, suggesting different nature of representations between encoding and retrieval. Moreover, cross-region representational similarity analysis suggests that the encoded representation in the ventral visual cortex was reinstated in the frontoparietal cortex during retrieval. Together, these results suggest that, in addition to reinstatement of the originally encoded pattern in the brain regions that perform encoding processes, retrieval may also involve the reinstatement of a transformed representation of the encoded information. These results emphasize the constructive nature of memory retrieval that helps to serve important adaptive functions.SIGNIFICANCE STATEMENT Episodic memory enables humans to vividly reexperience past events, yet how this is achieved at the neural level is barely understood. A long-standing hypothesis posits that memory retrieval involves the faithful reinstatement of encoding-related activity. We tested this hypothesis by comparing the neural representations during encoding and retrieval. We found strong pattern reinstatement in the frontoparietal cortex, but not in the ventral visual cortex, that represents visual details. Critically, even within the same brain regions, the nature of representation during retrieval was qualitatively different from that during encoding. These results suggest that memory retrieval is not a faithful replay of past event but rather involves additional constructive processes to serve adaptive functions.