Differential Representations of Perceived and Retrieved Visual Information in Hippocampus and Cortex

Perception and memory have distinct spatial tuning properties in human visual cortex

Reactivation of earlier perceptual activity is thought to underlie long-term memory recall. Despite evidence for this view, it is unclear whether mnemonic activity exhibits the same tuning properties as feedforward perceptual activity. Here, we leverage population receptive field models to parameterize fMRI activity in human visual cortex during spatial memory retrieval. Though retinotopic organization is present during both perception and memory, large systematic differences in tuning are also evident. Whereas there is a three-fold decline in spatial precision from early to late visual areas during perception, this pattern is not observed during memory retrieval. This difference cannot be explained by reduced signal-to-noise or poor performance on memory trials. Instead, by simulating top-down activity in a network model of cortex, we demonstrate that this property is well explained by the hierarchical structure of the visual system. Together, modeling and empirical results suggest that computational constraints imposed by visual system architecture limit the fidelity of memory reactivation in sensory cortex.

Neural activity from perception is thought to be reactivated during memory recall. Here, the authors show that memory reactivation in visual cortex exhibits different tuning properties from those observed during perception.

Schemas provide a scaffold for neocortical integration of new memories over time

Memory transformation is increasingly acknowledged in theoretical accounts of systems consolidation, yet how memory quality and neural representation change over time and how schemas influence this process remains unclear. We examined the behavioral quality and neural representation of schema-congruent and incongruent object-scene pairs retrieved across 10-minutes and 72-hours using fMRI. When a congruent schema was available, memory became coarser over time, aided by post-encoding coupling between the anterior hippocampus and medial prefrontal cortex (mPFC). Only schema-congruent representations were integrated in the mPFC over time, and were organized according to schematic context. In the hippocampus, pattern similarity changed across 72-hours such that the posterior hippocampus represented specific details and the anterior hippocampus represented the general context of specific memories, irrespective of congruency. Our findings suggest schemas are used as a scaffold to facilitate neocortical integration of congruent information, and illustrate evolution in hippocampal organization of detailed contextual memory over time.

Stronger memory representation after memory reinstatement during retrieval in the human hippocampus

Memory retrieval allows us to reinstate previously encoded information but is also considered to contribute to memory enhancement. Retrieval-induced enhancement may involve processing to strengthen memory traces, but neural processing beyond reinstatement during retrieval remains elusive. Here, we show that hippocampal processing, different from memory reinstatement, exists during retrieval in the human brain. By tracking changes in the response patterns in the selected hippocampal and cortical regions over time during retrieval based on functional MRI, we found that the representation of associative memory in CA3/DG became stronger even after cortical memory reinstatement, while CA1 showed significant memory representation at retrieval onset with the cortical reinstatement, but not afterwards. This tendency was not observed in the condition without active retrieval. Moreover, subsequent long-term memory performance depended on the delayed CA3/DG representation during retrieval. These findings suggest that CA3/DG contributes to neural processing beyond memory reinstatement during retrieval, which may lead to memory enhancement.

What do neuroanatomical networks reveal about the ontology of human cognitive abilities?

Over the last decades, cognitive psychology has come to a fair consensus about the human intelligence ontological structure. However, it remains an open question whether anatomical properties of the brain support the same ontology. The present study explored the ontological structure derived from neuroanatomical networks associated with performance on 15 cognitive tasks indicating various abilities. Results suggest that the brain-derived (neurometric) ontology partly agrees with the cognitive performance-derived (psychometric) ontology complemented with interpretable differences. Moreover, the cortical areas associated with different inferred abilities are segregated, with little or no overlap. Nevertheless, these spatially segregated cortical areas are integrated via denser white matter structural connections as compared with the general brain connectome. The integration of ability-related cortical networks constitutes a neural counterpart to the psychometric construct of general intelligence, while the consistency and difference between psychometric and neurometric ontologies represent crucial pieces of knowledge for theory building, clinical diagnostics, and treatment.

•

Psychometric and neurometric cognitive ontologies are partly equivalent

•

Ability-related brain areas are ontologically segregated with little to no overlap

•

However, ability-related brain areas are densely interconnected by fiber tracts

Hippocampal Asymmetry of Regional Development and Structural Covariance in Preterm Neonates

Premature birth is associated with a high prevalence of neurodevelopmental impairments in surviving infants. The hippocampus is known to be critical for learning and memory, yet the putative effects of hippocampal dysfunction remain poorly understood in preterm neonates. In particular, while asymmetry of the hippocampus has been well noted both structurally and functionally, how preterm birth impairs hippocampal development and to what extent the hippocampus is asymmetrically impaired by preterm birth have not been well delineated. In this study, we compared volumetric growth and shape development in the hippocampal hemispheres and structural covariance (SC) between hippocampal vertices and cortical thickness in cerebral cortex regions between two groups. We found that premature infants had smaller volumes of the right hippocampi only. Lower thickness was observed in the hippocampal head in both hemispheres for preterm neonates compared with full-term peers, though preterm neonates exhibited an accelerated age-related change of hippocampal thickness in the left hippocampi. The SC between the left hippocampi and the limbic lobe of the premature infants was severely impaired compared with the term-born neonates. These findings suggested that the development of the hippocampus during the third trimester may be altered following early extrauterine exposure with a high degree of asymmetry.

The Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2): A Psychometric Alternative to Measure and Explain Supernatural Experiences

This paper presents the English adaptation of the Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2), a questionnaire developed specifically for psychological assessment and prediction of anomalous phenomena. The sample consisted of 613 respondents from England (47.6% were women and 52.4% men). All of them were of legal age (mean = 34.5; standard deviation = 8.15). An exploratory factor analysis was applied, and three confirmatory factor models were adjusted. Omega coefficients and test-retest designs were used for reliability analysis. The MMSI-2 has a valid internal structure consisting of five macrofactors: Clinical Personality Tendencies (CPT), Anomalous Perceived Phenomena (APP), Incoherent Manipulations (IMA), Altered States of Consciousness (ASC), and Openness (OP). Omega coefficients for CPT and OP factors were low but acceptable. Furthermore, test-retest trials were excellent for all scales and factors. The psychological factors CPT, IMA, and ASC predicted 18.3% of the variance of anomalous experiences (APP). The authors concluded the English MMSI-2 was a valid and reliable test for the evaluation of anomalous phenomena but recommend that subsequent research reviews the predictive quality of the underlying model.

Primate ventral striatum maintains neural representations of the value of previously rewarded objects for habitual seeking

The ventral striatum (VS) is considered a key region that flexibly updates recent changes in reward values for habit learning. However, this update process may not serve to maintain learned habitual behaviors, which are insensitive to value changes. Here, using fMRI in humans and single-unit electrophysiology in macaque monkeys we report another role of the primate VS: that the value memory subserving habitual seeking is stably maintained in the VS. Days after object-value associative learning, human and monkey VS continue to show increased responses to previously rewarded objects, even when no immediate reward outcomes are expected. The similarity of neural response patterns to each rewarded object increases after learning among participants who display habitual seeking. Our data show that long-term memory of high-valued objects is retained as a single representation in the VS and may be utilized to evaluate visual stimuli automatically to guide habitual behavior.

Reinstatement of item-specific contextual details during retrieval supports recombination-related false memories

Flexible retrieval mechanisms that allow us to infer relationships across events may also lead to memory errors or distortion when details of one event are misattributed to the related event. Here, we tested how making successful inferences alters representation of overlapping events, leading to false memories. Participants encoded overlapping associations ('AB' and 'BC'), each of which was superimposed on different indoor and outdoor scenes that were pre-exposed prior to associative learning. Participants were subsequently tested on both the directly learned pairs ('AB' and 'BC') and inferred relationships across pairs ('AC'). We predicted that when people make a correct inference, features associated with overlapping events may become integrated in memory. To test this hypothesis, participants completed a final detailed retrieval test, in which they had to recall the scene associated with initially learned 'AB' pairs (or 'BC' pairs). We found that the outcome of inference decisions impacted the degree to which neural patterns elicited during detailed 'AB' retrieval reflected reinstatement of the scene associated with the overlapping 'BC' event. After successful inference, neural patterns in the anterior hippocampus, posterior medial prefrontal cortex, and our content-reinstatement region (left inferior temporal gyrus) were more similar to the overlapping, yet incorrect 'BC' context relative to after unsuccessful inference. Further, greater hippocampal activity during inference was associated with greater reinstatement of the incorrect, overlapping context in our content-reinstatement region, which in turn tracked contextual misattributions during detailed retrieval. These results suggest recombining memories during successful inference can lead to misattribution of contextual details across related events, resulting in false memories.

Is Reconsolidation a General Property of Memory?

Memory reconsolidation holds great hope for memory modification approaches and clinical treatments of mental disorders associated with maladaptive memories. However, it remains controversial as to whether reconsolidation is a general property of all types of memory. Especially, discrepancies have been reported in research focusing on whether declarative memory undergoes reconsolidation, and whether old memories can be reorganized after retrieval. Here, we discuss how these inconsistent results can be reconciled and what information we need to uncover for the general use of reconsolidation.

Cortical Representations of Visual Stimuli Shift Locations with Changes in Memory States

Episodic memory retrieval is thought to rely on reactivation of the same content-sensitive neural activity patterns initially expressed during memory encoding.^1-6 Yet there are emerging examples of content representations expressed in different brain regions during encoding versus retrieval.^7-14 Although these differences have been observed by comparing encoding and retrieval tasks that differ in terms of perceptual experience and cognitive demands, there are many real-world contexts-e.g., meeting a new colleague who reminds you of an old acquaintance-where the memory system might be intrinsically biased either toward encoding (the new colleague) or retrieval (the old acquaintance).¹⁵¹⁶ Here, we test whether intrinsic memory states, independent of task demands, determine the cortical location of content representations. In a human fMRI study, subjects (n = 33) viewed object images and were instructed to either encode the current object or retrieve a similar object from memory. Using pattern classifiers, we show that biases toward encoding versus retrieval were reflected in large-scale attentional networks.^17-19 Critically, memory states decoded from these networks-even when entirely independent from task instructions-predicted shifts of object representations from visual cortex (encoding) to ventral parietal cortex (retrieval). Finally, visual versus ventral parietal cortices exhibited differential connectivity with the hippocampus during memory encoding versus retrieval, consistent with the idea that the hippocampus mediates cortical shifts in content representations. Collectively, these findings demonstrate that intrinsic biases toward memory encoding versus retrieval determine the specific cortical locations that express content information.

Strong correspondence between prefrontal and visual representations during emotional perception

Emotion is thought to cause focal enhancement or distortion of certain components of memory, indicating a complex property of emotional modulation on memory rather than simple enhancement. However, the neural basis for detailed modulation of emotional memory contents has remained unclear. Here has been shown that the information processing of the prefrontal cortex differentially affects sensory representations during experience of emotional information compared with neutral information, using functional magnetic resonance imaging (fMRI). It was found that during perception of emotional pictures, information representation in primary visual cortex (V1) significantly corresponded with the representations in dorsolateral prefrontal cortex (dlPFC). This correspondence was not observed for neutral pictures. Furthermore, participants with greater correspondence between visual and prefrontal representations showed better memory for high-level semantic components but not for low-level visual components of emotional stimuli. These results suggest that sensory representation during experience of emotional stimuli, compared with neutral stimuli, is more directly influenced by internally generated higher-order information from the prefrontal cortex.

Distinct Representational Structure and Localization for Visual Encoding and Recall during Visual Imagery

During memory recall and visual imagery, reinstatement is thought to occur as an echoing of the neural patterns during encoding. However, the precise information in these recall traces is relatively unknown, with previous work primarily investigating either broad distinctions or specific images, rarely bridging these levels of information. Using ultra-high-field (7T) functional magnetic resonance imaging with an item-based visual recall task, we conducted an in-depth comparison of encoding and recall along a spectrum of granularity, from coarse (scenes, objects) to mid (e.g., natural, manmade scenes) to fine (e.g., living room, cupcake) levels. In the scanner, participants viewed a trial-unique item, and after a distractor task, visually imagined the initial item. During encoding, we observed decodable information at all levels of granularity in category-selective visual cortex. In contrast, information during recall was primarily at the coarse level with fine-level information in some areas; there was no evidence of mid-level information. A closer look revealed segregation between voxels showing the strongest effects during encoding and those during recall, and peaks of encoding-recall similarity extended anterior to category-selective cortex. Collectively, these results suggest visual recall is not merely a reactivation of encoding patterns, displaying a different representational structure and localization from encoding, despite some overlap.

Transforming the Concept of Memory Reactivation

Reactivation refers to the phenomenon wherein patterns of neural activity expressed during perceptual experience are re-expressed at a later time, a putative neural marker of memory. Reactivation of perceptual content has been observed across many cortical areas and correlates with objective and subjective expressions of memory in humans. However, because reactivation emphasizes similarities between perceptual and memory-based representations, it obscures differences in how perceptual events and memories are represented. Here, we highlight recent evidence of systematic differences in how (and where) perceptual events and memories are represented in the brain. We argue that neural representations of memories are best thought of as spatially transformed versions of perceptual representations. We consider why spatial transformations occur and identify critical questions for future research.

Probing the neural dynamics of mnemonic representations after the initial consolidation

Memories are not stored as static engrams, but as dynamic representations affected by processes occurring after initial encoding. Previous studies revealed changes in activity and mnemonic representations in visual processing areas, parietal lobe, and hippocampus underlying repeated retrieval and suppression. However, these neural changes are usually induced by memory modulation immediately after memory formation. Here, we investigated 27 healthy participants with a two-day functional Magnetic Resonance Imaging study design to probe how established memories are dynamically modulated by retrieval and suppression 24 h after learning. Behaviorally, we demonstrated that established memories can still be strengthened by repeated retrieval. By contrast, repeated suppression had a modest negative effect, and suppression-induced forgetting was associated with individual suppression efficacy. Neurally, we demonstrated item-specific pattern reinstatements in visual processing areas, parietal lobe, and hippocampus. Then, we showed that repeated retrieval reduced activity amplitude in the ventral visual cortex and hippocampus, but enhanced the distinctiveness of activity patterns in the ventral visual cortex and parietal lobe. Critically, reduced activity was associated with enhanced representation of idiosyncratic memory traces in the ventral visual cortex and precuneus. In contrast, repeated memory suppression was associated with reduced lateral prefrontal activity, but relative intact mnemonic representations. Our results replicated most of the neural changes induced by memory retrieval and suppression immediately after learning and extended those findings to established memories after initial consolidation. Active retrieval seems to promote episode-unique mnemonic representations in the neocortex after initial encoding but also consolidation.

When the Memory System Gets Ahead of Itself

Humans are adept at learning and exploiting statistical regularities to predict future events from current experience. A recent paper by Sherman and Turk-Browne demonstrates that statistical regularities bias the hippocampus toward representing future states over current experience and reduce the degree to which current experience is encoded into memory.

Statistical prediction of the future impairs episodic encoding of the present

Memory is typically thought of as enabling reminiscence about past experiences. However, memory also informs and guides processing of future experiences. These two functions of memory are often at odds: Remembering specific experiences from the past requires storing idiosyncratic properties that define particular moments in space and time, but by definition such properties will not be shared with similar situations in the future and thus may not be applicable to future situations. We discovered that, when faced with this conflict, the brain prioritizes prediction over encoding. Behavioral tests of recognition and source recall showed that items allowing for prediction of what will appear next based on learned regularities were less likely to be encoded into memory. Brain imaging revealed that the hippocampus was responsible for this interference between statistical learning and episodic memory. The more that the hippocampus predicted the category of an upcoming item, the worse the current item was encoded. This competition may serve an adaptive purpose, focusing encoding on experiences for which we do not yet have a predictive model.

Holding On to the Past: Older Adults Show Lingering Neural Activation of No-Longer-Relevant Items in Working Memory

Goal-relevant information can be maintained in working memory over a brief delay interval to guide an upcoming decision. There is also evidence suggesting the existence of a complementary process: namely, the ability to suppress information that is no longer relevant to ongoing task goals. Moreover, this ability to suppress or inhibit irrelevant information appears to decline with age. In this study, we compared younger and older adults undergoing fMRI on a working memory task designed to address whether the modulation of neural representations of relevant and no-longer-relevant items during a delay interval is related to age and overall task performance. Following from the theoretical predictions of the inhibitory deficit hypothesis of aging, we hypothesized that older adults would show higher activation of no-longer-relevant items during a retention delay compared to young adults and that higher activation of these no-longer-relevant items would predict worse recognition memory accuracy for relevant items. Our results support this prediction and more generally demonstrate the importance of goal-driven modulation of neural activity in successful working memory maintenance. Furthermore, we showed that the largest age differences in the regulation of category-specific pattern activity during working memory maintenance were seen throughout the medial temporal lobe and prominently in the hippocampus, further establishing the importance of "long-term memory" retrieval mechanisms in the context of high-load working memory tasks that place large demands on attentional selection mechanisms.

How Multiple Retrievals Affect Neural Reactivation in Young and Older Adults

OBJECTIVES

Aging can reduce the specificity with which memory episodes are represented as distributed patterns of brain activity. It remains unclear, however, whether repeated encoding and retrieval of stimuli modulate this decline. Memory repetition is thought to promote semanticization, a transformative process during which episodic memory becomes gradually decontextualized and abstracted. Because semantic memory is considered more resilient to aging than context-rich episodic memory, we hypothesized that repeated retrieval would affect cortical reinstatement differently in young versus older adults.

METHODS

We reanalyzed data from young and older adults undergoing functional magnetic resonance imaging while repeatedly viewing and recalling short videos. We derived trial-unique multivariate measures of similarity between video-specific brain activity patterns elicited at perception and at recall, which we compared between age groups at each repetition.

RESULTS

With repetition, memory representation became gradually more distinct from perception in young adults, as reinstatement specificity converged downward toward levels observed in the older group. In older adults, alternative representations that were item-specific but orthogonal to patterns elicited at perception became more salient with repetition.

DISCUSSION

Repetition transformed dominant patterns of memory representation away and orthogonally from perception in young and older adults, respectively. Although distinct, both changes are consistent with repetition-induced semanticization.