Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach

Pattern separation and pattern completion: Behaviorally separable processes?

Episodic memory capacity requires several processes, including mnemonic discrimination of similar experiences, termed pattern separation, and holistic retrieval of multidimensional experiences given a cue, termed pattern completion. Both computations seem to rely on the hippocampus proper, but they also seem to be instantiated by distinct hippocampal subfields. Thus, we investigated whether individual differences in behavioral expressions of pattern separation and pattern completion were correlated after accounting for general mnemonic ability. Young adult participants learned events comprised of a scene-animal-object triad. In the pattern separation task, we estimated mnemonic discrimination using lure classification for events that contained a similar lure element. In the pattern completion task, we estimated holistic recollection using dependency in retrieval success for different associations from the same event. Although overall accuracies for the two tasks correlated as expected, specific measures of individual variation in holistic retrieval and mnemonic discrimination did not correlate, suggesting that these two processes involve distinguishable properties of episodic memory.

Neural Development of Memory and Metamemory in Childhood and Adolescence: Toward an Integrative Model of the Development of Episodic Recollection

Memory and metamemory processes are essential to retrieve detailed memories and appreciate the phenomenological experience of recollection. Developmental cognitive neuroscience has made strides in revealing the neural changes associated with improvements in memory and metamemory during childhood and adolescence. We argue that hippocampal changes, in concert with surrounding cortical regions, support developmental improvements in the precision, complexity, and flexibility of memory representations. In contrast, changes in frontoparietal regions promote efficient encoding and retrieval strategies. A smaller body of literature on the neural substrates of metamemory development suggests that error monitoring processes implemented in the anterior insula and dorsal anterior cingulate cortex trigger, and perhaps support the development of, metacognitive evaluationsin the prefrontal cortex, while developmental changes in the parietal cortex support changes in the phenomenological experience of episodic retrieval. Our conclusions highlight the necessity of integrating these lines of research into a comprehensive model on the neurocognitive development of episodic recollection.

EPS mid-career prize 2018: Inference within episodic memory reflects pattern completion

Recollection of episodic memories is a process of reconstruction where coherent events are inferred from subsets of remembered associations. Here, we investigated the formation of multielement events from sequential presentation of overlapping pairs of elements (people, places, and objects/animals), interleaved with pairs from other events. Retrievals of paired associations from a fully observed event (e.g., AB, BC, AC) were statistically dependent, indicating a process of pattern completion, but retrievals from a partially observed event (e.g., AB, BC, CD) were not. However, inference for unseen “indirect” associations (i.e., AC, BD or AD) from a partially observed event showed strong dependency with each other and with linking direct associations from that event. In addition, inference of indirect associations correlated with the product of performance on the linking direct associations across events (e.g., AC with ABxBC) but not on the non-linking association (e.g., AC with CD). These results were seen across three experiments, with greater differences in dependency between indirect and direct associations when they were separately tested, but similar results following single and repeated presentations of the direct associations. The results could be accounted for by a simple auto-associative network model of hippocampal memory function. Our findings suggest that pattern completion supports recollection of fully observed multielement events and the inference of indirect associations in partly observed multielement events, mediated via the directly observed linking associations (although the direct associations themselves were retrieved independently). Together with previous work, our results suggest that associative inference plays a key role in reconstructive episodic memory and does so through hippocampal pattern completion.

Object recognition is enabled by an experience-dependent appraisal of visual features in the brain's value system

This paper addresses perceptual synthesis by comparing responses evoked by visual stimuli before and after they are recognized, depending on prior exposure. Using magnetoencephalography, we analyzed distributed patterns of neuronal activity - evoked by Mooney figures - before and after they were recognized as meaningful objects. Recognition induced changes were first seen at 100-120 ​ms, for both faces and tools. These early effects - in right inferior and middle occipital regions - were characterized by an increase in power in the absence of any changes in spatial patterns of activity. Within a later 210-230 ​ms window, a quite different type of recognition effect appeared. Regions of the brain's value system (insula, entorhinal cortex and cingulate of the right hemisphere for faces and right orbitofrontal cortex for tools) evinced a reorganization of their neuronal activity without an overall power increase in the region. Finally, we found that during the perception of disambiguated face stimuli, a face-specific response in the right fusiform gyrus emerged at 240-290 ​ms, with a much greater latency than the well-known N170m component, and, crucially, followed the recognition effect in the value system regions. These results can clarify one of the most intriguing issues of perceptual synthesis, namely, how a limited set of high-level predictions, which is required to reduce the uncertainty when resolving the ill-posed inverse problem of perception, can be available before category-specific processing in visual cortex. We suggest that a subset of local spatial features serves as partial cues for a fast re-activation of object-specific appraisal by the value system. The ensuing top-down feedback from value system to visual cortex, in particular, the fusiform gyrus enables high levels of processing to form category-specific predictions. This descending influence of the value system was more prominent for faces than for tools, the fact that reflects different dependence of these categories on value-related information.

Processing of Different Temporal Scales in the Human Brain

While recalling life events, we reexperience events of different durations, ranging across varying temporal scales, from several minutes to years. However, the brain mechanisms underlying temporal cognition are usually investigated only in small-scale periods—milliseconds to minutes. Are the same neurocognitive systems used to organize memory at different temporal scales? Here, we asked participants to compare temporal distances (time elapsed) to personal events at four different temporal scales (hour, day, week, and month) under fMRI. Cortical activity showed temporal scale sensitivity at the medial and lateral parts of the parietal lobe, bilaterally. Activity at the medial parietal cortex also showed a gradual progression from large- to small-scale processing, along a posterior–anterior axis. Interestingly, no sensitivity was found along the hippocampal long axis. In the medial scale-sensitive region, most of the voxels were preferentially active for the larger scale (month), and in the lateral region, scale selectivity was higher for the smallest scale (hour). These results demonstrate how scale-selective activity characterizes autobiographical memory processing and may provide a basis for understanding how the human brain processes and integrates experiences across timescales in a hierarchical manner.

Hippocampal spatial memory representations in mice are heterogeneously stable

The population of hippocampal neurons actively coding space continually changes across days as mice repeatedly perform tasks. Many hippocampal place cells become inactive while other previously silent neurons become active, challenging the idea that stable behaviors and memory representations are supported by stable patterns of neural activity. Active cell replacement may disambiguate unique episodes that contain overlapping memory cues, and could contribute to reorganization of memory representations. How active cell replacement affects the evolution of representations of different behaviors within a single task is unknown. We trained mice to perform a delayed nonmatching to place task over multiple weeks, and performed calcium imaging in area CA1 of the dorsal hippocampus using head-mounted miniature microscopes. Cells active on the central stem of the maze "split" their calcium activity according to the animal's upcoming turn direction (left or right), the current task phase (study or test), or both task dimensions, even while spatial cues remained unchanged. We found that, among reliably active cells, different splitter neuron populations were replaced at unequal rates, resulting in an increasing number of cells modulated by turn direction and a decreasing number of cells with combined modulation by both turn direction and task phase. Despite continual reorganization, the ensemble code stably segregated these task dimensions. These results show that hippocampal memories can heterogeneously reorganize even while behavior is unchanging.

Flexible Working Memory Through Selective Gating and Attentional Tagging

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.

Usefulness of familiarity signals during recognition depends on test format: Neurocognitive evidence for a core assumption of the CLS framework

Familiarity-based discrimination between studied items and similar foils in yes/no recognition memory tests is relatively poor. The complementary learning systems (CLS) framework explains this with the small difference in familiarity strength between targets and foils. The framework, however, also predicts that familiarity values of targets and corresponding similar foils are directly comparable - as long as they are presented side by side in a forced-choice corresponding (FCC) test. This is because in each trial, targets tend to be more familiar than their corresponding foils. In contrast, when forced-choice displays contain non-corresponding foils (FCNC) which are similar to other studied items, familiarity values are not directly comparable (as in yes/no-tasks). In a recognition memory task with pictures of objects, we found that the putative ERP correlate of familiarity, the mid-frontal old/new effect for targets vs. foils, was significantly larger in FCC compared to FCNC displays. Moreover, single-trial target-foil amplitude differences predicted the accuracy of the recognition judgment. This study supports the assumption of the CLS framework that the test format can influence the diagnostic reliability of familiarity. Moreover, it implies that the mid-frontal old/new effect does not reflect the difference in the familiarity signal between studied and non-studied items but the task-adequate assessment of this signal.

Differential Representations of Perceived and Retrieved Visual Information in Hippocampus and Cortex

Memory retrieval is thought to depend on interactions between hippocampus and cortex, but the nature of representation in these regions and their relationship remains unclear. Here, we performed an ultra-high field fMRI (7T) experiment, comprising perception, learning and retrieval sessions. We observed a fundamental difference between representations in hippocampus and high-level visual cortex during perception and retrieval. First, while object-selective posterior fusiform cortex showed consistent responses that allowed us to decode object identity across both perception and retrieval one day after learning, object decoding in hippocampus was much stronger during retrieval than perception. Second, in visual cortex but not hippocampus, there was consistency in response patterns between perception and retrieval, suggesting that substantial neural populations are shared for both perception and retrieval. Finally, the decoding in hippocampus during retrieval was not observed when retrieval was tested on the same day as learning suggesting that the retrieval process itself is not sufficient to elicit decodable object representations. Collectively, these findings suggest that while cortical representations are stable between perception and retrieval, hippocampal representations are much stronger during retrieval, implying some form of reorganization of the representations between perception and retrieval.

Pupil size changes signal hippocampus-related memory functions

A major task of episodic memory is to create unique, distinguishable representations of highly overlapping perceptual inputs. Several studies on this basic function have shown that it is based on the intact functioning of certain subregions of the hippocampus and is among the most sensitive behavioral indicators of mild cognitive impairment (MCI) and dementia. Here we assessed pupil dilation associated with performance in a widely used recognition paradigm that aims to uncover the intactness of fine-graded mnemonic discrimination. A sample of healthy undergraduate students was used. First, we showed that the correct discrimination between highly similar lure items and target items elicit larger pupil response than correct target identification. Second, we found that mnemonic discrimination is associated with larger pupil response in general as compared to target identification, regardless of whether the response was correct or not. These results suggest the pupil changes differentiate mnemonic discrimination and memory identification processes in recognition performance.

Something old, something new: A review of the literature on sleep-related lexicalization of novel words in adults

Word learning is a crucial aspect of human development that depends on the formation and consolidation of novel memory traces. In this paper, we critically review the behavioural research on sleep-related lexicalization of novel words in healthy young adult speakers. We first describe human memory systems, the processes underlying memory consolidation, then we describe the complementary learning systems account of memory consolidation. We then review behavioural studies focusing on novel word learning and sleep-related lexicalization in monolingual samples, while highlighting their relevance to three main theoretical questions. Finally, we review the few studies that have investigated sleep-related lexicalization in L2 speakers. Overall, while several studies suggest that sleep promotes the gradual transformation of initially labile traces into more stable representations, a growing body of work suggests a rich variety of time courses for novel word lexicalization. Moreover, there is a need for more work on sleep-related lexicalization patterns in varied populations, such as L2 speakers and bilingual speakers, and more work on individual differences, to fully understand the boundary conditions of this phenomenon.

Protecting memory from misinformation: Warnings modulate cortical reinstatement during memory retrieval

Exposure to misleading information can distort memory for past events (misinformation effect). Here, we show that providing individuals with a simple warning about the threat of misinformation significantly reduces the misinformation effect, regardless of whether warnings are provided proactively (before exposure to misinformation) or retroactively (after exposure to misinformation). In the brain, this protective effect of warning is associated with increased reactivation of sensory regions associated with the original event and decreased reactivation of sensory regions associated with the misleading information. These findings reveal that warnings can protect memory from misinformation by modulating reconstructive processes at the time of memory retrieval and have important practical implications for improving the accuracy of eyewitness testimony as well as everyday memory reports.

Exposure to even subtle forms of misleading information can significantly alter memory for past events. Memory distortion due to misinformation has been linked to faulty reconstructive processes during memory retrieval and the reactivation of brain regions involved in the initial encoding of misleading details (cortical reinstatement). The current study investigated whether warning participants about the threat of misinformation can modulate cortical reinstatement during memory retrieval and reduce misinformation errors. Participants watched a silent video depicting a crime (original event) and were given an initial test of memory for the crime details. Then, participants listened to an auditory narrative describing the crime in which some original details were altered (misinformation). Importantly, participants who received a warning about the reliability of the auditory narrative either before or after exposure to misinformation demonstrated less susceptibility to misinformation on a final test of memory compared to unwarned participants. Warned and unwarned participants also demonstrated striking differences in neural activity during the final memory test. Compared to participants who did not receive a warning, participants who received a warning (regardless of its timing) demonstrated increased activity in visual regions associated with the original source of information as well as decreased activity in auditory regions associated with the misleading source of information. Stronger visual reactivation was associated with reduced susceptibility to misinformation, whereas stronger auditory reactivation was associated with increased susceptibility to misinformation. Together, these results suggest that a simple warning can modulate reconstructive processes during memory retrieval and reduce memory errors due to misinformation.

The Effects of High-intensity Functional Training (HIFT) on Spatial Learning, Visual Pattern Separation and Attention Span in Adolescents

Aerobic, anaerobic, and strength exercises are known to improve various cognitive functions, such as executive functions, pattern separation, and working memory. High-intensity functional training (HIFT) is a form of physical activity that can be modified to any fitness level and elicits greater muscle recruitment than repetitive aerobic exercises, thereby improving cardiovascular endurance, strength, and flexibility. HIFT emphasizes functional, multi-joint movements via high-intensity interval training (HIIT) and muscle-strengthening exercises. It is yet unknown, however, whether HIFT affects cognitive functions in adolescents. To address this question, we subjected adolescents to 3 × 20 min training sessions/week of HIFT for 3 months. The effects of HIFT were tested on performance in: (1) virtual reality (VR)-based spatial learning task; (2) computerized visual pattern separation; and (3) attention span. The control group performed a typical physical class three times per week. The effects on cognition were tested at baseline and following 3 months of HIFT. Three months into the intervention, the HIFT group achieved higher scores in the spatial learning task, pattern separation task, and in the attention span test, compared with controls. These data suggest that HIFT can potentially translate into improving school performance in adolescents.

Statistical prediction of the future impairs episodic encoding of the present

Memory is typically thought of as enabling reminiscence about past experiences. However, memory also informs and guides processing of future experiences. These two functions of memory are often at odds: Remembering specific experiences from the past requires storing idiosyncratic properties that define particular moments in space and time, but by definition such properties will not be shared with similar situations in the future and thus may not be applicable to future situations. We discovered that, when faced with this conflict, the brain prioritizes prediction over encoding. Behavioral tests of recognition and source recall showed that items allowing for prediction of what will appear next based on learned regularities were less likely to be encoded into memory. Brain imaging revealed that the hippocampus was responsible for this interference between statistical learning and episodic memory. The more that the hippocampus predicted the category of an upcoming item, the worse the current item was encoded. This competition may serve an adaptive purpose, focusing encoding on experiences for which we do not yet have a predictive model.

Age Differences In Retrieval-Related Reinstatement Reflect Age-Related Dedifferentiation At Encoding

Age-related reductions in neural selectivity have been linked to cognitive decline. We examined whether age differences in the strength of retrieval-related cortical reinstatement could be explained by analogous differences in neural selectivity at encoding, and whether reinstatement was associated with memory performance in an age-dependent or an age-independent manner. Young and older adults underwent fMRI as they encoded words paired with images of faces or scenes. During a subsequent scanned memory test participants judged whether test words were studied or unstudied and, for words judged studied, also made a source memory judgment about the associated image category. Using multi-voxel pattern similarity analyses, we identified robust evidence for reduced scene reinstatement in older relative to younger adults. This decline was however largely explained by age differences in neural differentiation at encoding; moreover, a similar relationship between neural selectivity at encoding and retrieval was evident in young participants. The results suggest that, regardless of age, the selectivity with which events are neurally processed at the time of encoding can determine the strength of retrieval-related cortical reinstatement.

Dissecting the Fornix in Basic Memory Processes and Neuropsychiatric Disease: A Review

Background: The fornix is the primary axonal tract of the hippocampus, connecting it to modulatory subcortical structures. This review reveals that fornix damage causes cognitive deficits that closely mirror those resulting from hippocampal lesions. Methods: We reviewed the literature on the fornix, spanning non-human animal lesion research, clinical case studies of human patients with fornix damage, as well as diffusion-weighted imaging (DWI) work that evaluates fornix microstructure in vivo. Results: The fornix is essential for memory formation because it serves as the conduit for theta rhythms and acetylcholine, as well as providing mnemonic representations to deep brain structures that guide motivated behavior, such as when and where to eat. In rodents and non-human primates, fornix lesions lead to deficits in conditioning, reversal learning, and navigation. In humans, damage to the fornix manifests as anterograde amnesia. DWI research reveals that the fornix plays a key role in mild cognitive impairment and Alzheimer's Disease, and can potentially predict conversion from the former to the latter. Emerging DWI findings link perturbations in this structure to schizophrenia, mood disorders, and eating disorders. Cutting-edge research has investigated how deep brain stimulation of the fornix can potentially attenuate memory loss, control epileptic seizures, and even improve mood. Conclusions: The fornix is essential to a fully functioning memory system and is implicated in nearly all neurological functions that rely on the hippocampus. Future research needs to use optimized DWI methods to study the fornix in vivo, which we discuss, given the difficult nature of fornix reconstruction. Impact Statement The fornix is a white matter tract that connects the hippocampus to several subcortical brain regions and is pivotal for episodic memory functioning. Functionally, the fornix transmits essential neurotransmitters, as well as theta rhythms, to the hippocampus. In addition, it is the conduit by which memories guide decisions. The fornix is biomedically important because lesions to this tract result in irreversible anterograde amnesia. Research using in vivo imaging methods has linked fornix pathology to cognitive aging, mild cognitive impairment, psychosis, epilepsy, and, importantly, Alzheimer's Disease.

Behavioral evidence for pattern separation in human episodic memory

An essential feature of episodic memory is the ability to recall the multiple elements relating to one event from the multitude of elements relating to other, potentially similar events. Hippocampal pattern separation is thought to play a fundamental role in this process, by orthogonalizing the representations of overlapping events during encoding, to reduce interference between them during the process of pattern completion by which one or other is recalled. We introduce a new paradigm to test the hypothesis that similar memories, but not unrelated memories, are actively separated at encoding. Participants memorized events which were either unique or shared a common element with another event (paired “overlapping” events). We used a measure of dependency, originally devised to measure pattern completion, to quantify how much the probability of successfully retrieving associations from one event depends on successful retrieval of associations from the same event, an unrelated event or the overlapping event. In two experiments, we saw that within event retrievals were highly dependent, indicating pattern completion; retrievals from unrelated events were independent; and retrievals from overlapping events were antidependent (i.e., less than independent), indicating pattern separation. This suggests that representations of similar (overlapping) memories are actively separated, resulting in lowered dependency of retrieval performance between them, as would be predicted by the pattern separation account.

Mnemonic prediction errors bias hippocampal states

When our experience violates our predictions, it is adaptive to upregulate encoding of novel information, while down-weighting retrieval of erroneous memory predictions to promote an updated representation of the world. We asked whether mnemonic prediction errors promote hippocampal encoding versus retrieval states, as marked by distinct network connectivity between hippocampal subfields. During fMRI scanning, participants were cued to internally retrieve well-learned complex room-images and were then presented with either an identical or a modified image (0-4 changes). In the left hemisphere, we find that CA1-entorhinal connectivity increases, and CA1-CA3 connectivity decreases, with the number of changes. Further, in the left CA1, the similarity between activity patterns during cued-retrieval of the learned room and during the image is lower when the image includes changes, consistent with a prediction error signal in CA1. Our findings provide a mechanism by which mnemonic prediction errors may drive memory updating—by biasing hippocampal states.

When our expectations are violated, it is adaptive to update our internal models to improve predictions in the future. Here, the authors show that during mnemonic violations, hippocampal networks are biased towards an encoding state and away from a retrieval state to potentially update these predictions.

Examining the Neural Basis of Congruent and Incongruent Configural Contexts during Associative Retrieval

Disrupting the configural context, or relative organization and orientation of paired stimuli, between encoding and retrieval negatively impacts memory. Using univariate and multivariate fMRI analyses, we examined the effect of retaining and manipulating the configural context on neural mechanisms supporting associative retrieval. Behavioral results showed participants had significantly higher hit rates for recollecting pairs in a contextually congruent, versus incongruent, configuration. In addition, contextual congruency between memory phases was a critical determinant to characterizing both the magnitude and patterns of neural activation within visual and parietal cortices. Regions within visual cortices also exhibited higher correlations between patterns of activity at encoding and retrieval when configural context was congruent across memory phases than incongruent. Collectively, these findings shed light on how manipulating configural context between encoding and retrieval affects associative recognition, with changes in the configural context leading to reductions in information transfer and increases in task difficulty.

Neurophotonics Approaches for the Study of Pattern Separation

Successful memory involves not only remembering over time but also keeping memories distinct. Computational models suggest that pattern separation appears as a highly efficient process to discriminate between overlapping memories. Furthermore, lesion studies have shown that the dentate gyrus (DG) participates in pattern separation. However, these manipulations did not allow identifying the neuronal mechanism underlying pattern separation. The development of different neurophotonics techniques, together with other genetic tools, has been useful for the study of the microcircuit involved in this process. It has been shown that less-overlapped information would generate distinct neuronal representations within the granule cells (GCs). However, because glutamatergic or GABAergic cells in the DG are not functionally or structurally homogeneous, identifying the specific role of the different subpopulations remains elusive. Then, understanding pattern separation requires the ability to manipulate a temporal and spatially specific subset of cells in the DG and ideally to analyze DG cells activity in individuals performing a pattern separation dependent behavioral task. Thus, neurophotonics and calcium imaging techniques in conjunction with activity-dependent promoters and high-resolution microscopy appear as important tools for this endeavor. In this work, we review how different neurophotonics techniques have been implemented in the elucidation of a neuronal network that supports pattern separation alone or in combination with traditional techniques. We discuss the limitation of these techniques and how other neurophotonic techniques could be used to complement the advances presented up to this date.

How Proactive Interference during New Associative Learning Impacts General and Specific Memory in Young and Old

Some prior research has found that older adults are more susceptible to proactive interference than young adults. The current study investigated whether age-related deficits in pFC-mediated cognitive control processes that act to detect and resolve interference underlie increased susceptibility to proactive interference in an associative memory task. Young and older adults were scanned while tasked with remembering which associate (face or scene) objects were paired with most recently during study, under conditions of high, low, or no proactive interference. After scanning, participants' memory was tested for varying levels of episodic detail about the pairings (i.e., target category vs. specific target category vs. specific target associate). Young and older adults were similarly susceptible to proactive interference. Memory for both the general target category and the specific target associate worsened as the level of proactive interference increased, with no robust age differences. For both young and older adults, the left ventrolateral pFC, which has been indicated in controlled retrieval of goal-relevant conceptual representations, was sensitive to increasing levels of interference during encoding but was insensitive to associative memory accuracy. Consistent with the Compensation-Related Utilization of Neural Circuits Hypothesis model of cognitive aging, the ventromedial pFC, which is involved in the monitoring of internally generated information, was recruited more by older than young adults to support the successful retrieval of target-object pairs at lower levels of proactive interference. Collectively, these results suggest that some older adults are able to engage in the cognitive control processes necessary to resolve proactive interference to the same extent as young adults.

Functions of ventral visual cortex after bilateral medial temporal lobe damage

Repeated stimuli elicit attenuated responses in visual cortex relative to novel stimuli. This adaptation can be considered as a form of rapid learning and a signature of perceptual memory. Adaptation occurs not only when a stimulus is repeated immediately, but also when there is a lag in terms of time and other intervening stimuli before the repetition. But how does the visual system keep track of which stimuli are repeated, especially after long delays and many intervening stimuli? We hypothesized that the hippocampus and medial temporal lobe (MTL) support long-lag adaptation, given that this memory system can learn from single experiences, maintain information over delays, and send feedback to visual cortex. We tested this hypothesis with fMRI in an amnesic patient, LSJ, who has encephalitic damage to the MTL resulting in extensive bilateral lesions including complete hippocampal loss. We measured adaptation at varying time lags between repetitions in functionally localized visual areas that were intact in LSJ. We observed that these areas track information over a few minutes even when the hippocampus and extended parts of the MTL are unavailable. LSJ and controls were identical when attention was directed away from the repeating stimuli: adaptation occurred for lags up to three minutes, but not six minutes. However, when attention was directed toward stimuli, controls now showed an adaptation effect at six minutes but LSJ did not. These findings suggest that visual cortex can support one-shot perceptual memories lasting for several minutes but that the hippocampus and surrounding MTL structures are necessary for adaptation in visual cortex after longer delays when stimuli are task-relevant.

An update on memory formation and retrieval: An engram-centric approach

OBJECTIVE

We explore here that memory loss observed in the early stage of Alzheimer's disease (AD) is a disorder of memory retrieval, instead of a storage impairment. This engram-centric explanation aims to enlarge the conceptual frame of memory as an emergent behavior of the brain and to propose a new treatment strategy for memory retrieval in dementia-AD.

BACKGROUND

The conventional memory hypothesis suggests that memory is stored as multiple traces in hippocampal neurons but recent evidence indicates that there are specialized memory engrams responsible for the storage and the retrieval of different memory types.

UPDATED MEMORY HYPOTHESIS

There are specialized memory engram neurons for each memory type and when information will be stored as a memory arrives in the hippocampus through afferent neurons finds its neuron according to the excitability states of engram neurons. The excitability level in engram neurons seems like a code canalizing the interactions between engrams and information. Therefore, to enhance the excitability of memory engram neurons improves memory loss observed in AD. In addition, we suggest that the hippocampus creates an index for information stored in memory engram cells in specialized regions for different types of memory, instead of storing all information; and different anatomic locations of engram cells and their roles in memory retrieval point out that memory could be an emergent behavior of the brain, and the interaction between serotonin fluctuation and engram neurons could be neural underpinnings of terminal lucidity.

MAJOR CHALLENGES FOR THE MODEL

The major challenge for this engram-centric memory retrieval model is the translation from bench to patient, specifically the delivery of optogenetic tools in patients. Engram neurons can be specifically activated by optogenetic tools, but optogenetics is an invasive technique which requires optic fiber implantation into the brain. In addition, light can overheat the tissue and thus induce damage in tissue. Furthermore, light is a foreign object and its direct implantation into the brain may cause neuroinflammation, the main trigger of neurodegenerative diseases. Therefore, to test the engram hypothesis in human, new tools to allow specific engram activation should be discovered.

Memory engrams: Recalling the past and imagining the future

In 1904, Richard Semon introduced the term "engram" to describe the neural substrate for storing memories. An experience, Semon proposed, activates a subset of cells that undergo off-line, persistent chemical and/or physical changes to become an engram. Subsequent reactivation of this engram induces memory retrieval. Although Semon's contributions were largely ignored in his lifetime, new technologies that allow researchers to image and manipulate the brain at the level of individual neurons has reinvigorated engram research. We review recent progress in studying engrams, including an evaluation of evidence for the existence of engrams, the importance of intrinsic excitability and synaptic plasticity in engrams, and the lifetime of an engram. Together, these findings are beginning to define an engram as the basic unit of memory.

Electrophysiological signatures of memory reactivation in humans

The reactivation of neural activity that was present during the encoding of an event is assumed to be essential for human episodic memory retrieval and the consolidation of memories during sleep. Pioneering animal work has already established a crucial role of memory reactivation to prepare and guide behaviour. Research in humans is now delineating the neural processes involved in memory reactivation during both wakefulness and sleep as well as their functional significance. Focusing on the electrophysiological signatures of memory reactivation in humans during both memory retrieval and sleep-related consolidation, this review provides an overview of the state of the art in the field. We outline recent advances, methodological developments and open questions and specifically highlight commonalities and differences in the neuronal signatures of memory reactivation during the states of wakefulness and sleep.

This article is part of the Theo Murphy meeting issue ‘Memory reactivation: replaying events past, present and future’.

Mechanisms of memory: An intermediate level of analysis and organization

Research in the last five years has made great strides toward mechanistic explanations of how the brain enables memory. This progress builds upon decades of research from two complementary strands: a Levels of Analysis approach and a Levels of Organization approach. We review how research in cognitive psychology and cognitive neuroscience under these two approaches has recently converged on mechanistic, brain-based theories, couched at the optimal level for explaining cognitive phenomena - the intermediate level. Furthermore, novel empirical and data analysis techniques are now providing ways to test these theories' predictions, a crucial step in unraveling the mechanisms of memory.

Pattern Separation Underpins Expectation-Modulated Memory

Pattern separation and completion are fundamental hippocampal computations supporting memory encoding and retrieval. However, despite extensive exploration of these processes, it remains unclear whether and how top-down processes adaptively modulate the dynamics between these computations. Here we examine the role of expectation in shifting the hippocampus to perform pattern separation. In a behavioral task, 29 participants (7 males) learned a cue-object category contingency. Then, at encoding, one-third of the cues preceding the to-be-memorized objects, violated the studied rule. At test, participants performed a recognition task with old objects (targets) and a set of parametrically manipulated (very similar to dissimilar) foils for each object. Accuracy was found to be better for foils of high similarity to targets that were contextually unexpected at encoding compared with expected ones. Critically, there were no expectation-driven differences for targets and low similarity foils. To further explore these effects, we implemented a computational model of the hippocampus, performing the same task as the human participants. We used representational similarity analysis to examine how top-down expectation interacts with bottom-up perceptual input, in each layer. All subfields showed more dissimilar representations for unexpected items, with dentate gyrus (DG) and CA3 being more sensitive to expectation violation than CA1. Again, representational differences between expected and unexpected inputs were prominent for moderate to high levels of input similarity. This effect diminished when inputs from DG and CA3 into CA1 were lesioned. Overall, these novel findings strongly suggest that pattern separation in DG/CA3 underlies the effect that violation of expectation exerts on memory.SIGNIFICANCE STATEMENT What makes some events more memorable than others is a key question in cognitive neuroscience. Violation of expectation often leads to better memory performance, but the neural mechanism underlying this benefit remains elusive. In a behavioral study, we found that memory accuracy is enhanced selectively for unexpected highly similar foils, suggesting expectation violation does not enhance memory indiscriminately, but specifically aids the disambiguation of overlapping inputs. This is further supported by our subsequent investigation using a hippocampal computational model, revealing increased representational dissimilarity for unexpected highly similar foils in DG and CA3. These convergent results provide the first evidence that pattern separation plays an explicit role in supporting memory for unexpected information.

Modeling the influence of the hippocampal memory system on the oculomotor system

Visual exploration is related to activity in the hippocampus (HC) and/or extended medial temporal lobe system (MTL), is influenced by stored memories, and is altered in amnesic cases. An extensive set of polysynaptic connections exists both within and between the HC and oculomotor systems such that investigating how HC responses ultimately influence neural activity in the oculomotor system, and the timing by which such neural modulation could occur, is not trivial. We leveraged TheVirtualBrain, a software platform for large-scale network simulations, to model the functional dynamics that govern the interactions between the two systems in the macaque cortex. Evoked responses following the stimulation of the MTL and some, but not all, subfields of the HC resulted in observable responses in oculomotor regions, including the frontal eye fields, within the time of a gaze fixation. Modeled lesions to some MTL regions slowed the dissipation of HC signal to oculomotor regions, whereas HC lesions generally did not affect the rapid MTL activity propagation to oculomotor regions. These findings provide a framework for investigating how information represented by the HC/MTL may influence the oculomotor system during a fixation and predict how HC lesions may affect visual exploration.

The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications

Decades of cognitive neuroscience research has shown that where we look is intimately connected to what we remember. In this article, we review findings from human and nonhuman animals, using behavioral, neuropsychological, neuroimaging, and computational modeling methods, to show that the oculomotor and hippocampal memory systems interact in a reciprocal manner, on a moment-to-moment basis, mediated by a vast structural and functional network. Visual exploration serves to efficiently gather information from the environment for the purpose of creating new memories, updating existing memories, and reconstructing the rich, vivid details from memory. Conversely, memory increases the efficiency of visual exploration. We call for models of oculomotor control to consider the influence of the hippocampal memory system on the cognitive control of eye movements, and for models of hippocampal and broader medial temporal lobe function to consider the influence of the oculomotor system on the development and expression of memory. We describe eye movement-based applications for the detection of neurodegeneration and delivery of therapeutic interventions for mental health disorders for which the hippocampus is implicated and memory dysfunctions are at the forefront.

Varieties of recollective experience

Four variants on Tulving's "Remember/Know" paradigm supported a tripartite classification of recollective experience in recognition memory into Remembering (as in conscious recollection of a past episode), Knowing (similar to retrieval from semantic memory), and Feeling (a priming-based judgment of familiarity). Recognition-by-knowing and recognition-by-feeling are differentiated by level of processing at the time of encoding (Experiments 1-3), shifts in the criterion for item recognition (Experiment 2), response latencies (Experiments 1-3), and changes in the response window (Experiment 3). False recognition is often accompanied by "feeling", but rarely by "knowing"; d' is higher for knowing than for feeling (Experiments 1-4). Recognition-by-knowing increases with additional study trials, while recognition-by-feeling falls to zero (Experiment 4). In these ways, recognition-by-knowing is distinguished from recognition-by-feeling in much the same way as, in the traditional Remember/Know paradigm, recognition-by-remembering can be distinguished from recognition-without-remembering. Implications are discussed for dual-process theories of memory, and the search for the neural substrates of memory retrieval.

Human hippocampal CA3 damage disrupts both recent and remote episodic memories

Neocortical-hippocampal interactions support new episodic (event) memories, but there is conflicting evidence about the dependence of remote episodic memories on the hippocampus. In line with systems consolidation and computational theories of episodic memory, evidence from model organisms suggests that the cornu ammonis 3 (CA3) hippocampal subfield supports recent, but not remote, episodic retrieval. In this study, we demonstrated that recent and remote memories were susceptible to a loss of episodic detail in human participants with focal bilateral damage to CA3. Graph theoretic analyses of 7.0-Tesla resting-state fMRI data revealed that CA3 damage disrupted functional integration across the medial temporal lobe (MTL) subsystem of the default network. The loss of functional integration in MTL subsystem regions was predictive of autobiographical episodic retrieval performance. We conclude that human CA3 is necessary for the retrieval of episodic memories long after their initial acquisition and functional integration of the default network is important for autobiographical episodic memory performance.

Cortical Overlap and Cortical-Hippocampal Interactions Predict Subsequent True and False Memory

The declarative memory system allows us to accurately recognize a countless number of items and events, particularly those strengthened by repeated exposure. However, increased familiarity due to repetition can also lead to false recognition of related but new items, particularly when mechanisms supporting fine-grain mnemonic discrimination fail. The hippocampus is thought to be particularly important in separating overlapping cortical inputs during encoding so that similar experiences can be differentiated. In the current study of male and female human subjects, we examine how neural pattern similarity between repeated exemplars of a given concept (e.g., apple) influences true and false memory for target or lure images. Consistent with past work, we found that subsequent true recognition was related to pattern similarity between concept exemplars and the entire encoding set (global encoding similarity), particularly in ventral visual stream. In addition, memory for an individual target exemplar (a specific apple) could be predicted solely by the degree of pattern overlap between the other exemplars (different apple pictures) of that concept (concept-specific encoding similarity). Critically, subsequent false memory for lures was mitigated when high concept-specific similarity in cortical areas was accompanied by differentiated hippocampal representations of the corresponding exemplars. Furthermore, both true and false memory entailed the reinstatement of concept-related information at varying levels of specificity. These results link both true and false memory to a measure of concept strength expressed in the overlap of cortical representations, and importantly, illustrate how the hippocampus serves to separate concurrent cortical overlap in the service of detailed memory.SIGNIFICANCE STATEMENT In some instances, the same processes that help promote memory for a general idea or concept can also hinder more detailed memory judgments, which may involve differentiating between closely related items. The current study shows that increased overlap in cortical representations for conceptually-related pictures is associated with increased recognition of repeated concept pictures. Whether similar lure items were falsely remembered as old further depended on the hippocampus, where the presence of more distinct representations protected against later false memory. This work suggests that the differentiability of brain patterns during perception is related to the differentiability of items in memory, but that fine-grain discrimination depends on the interaction between cortex and hippocampus.

Cross-Situational Statistical Learning of New Words Despite Bilateral Hippocampal Damage and Severe Amnesia

Word learning requires learners to bind together arbitrarily-related phonological, visual, and conceptual information. Prior work suggests that this binding can be robustly achieved via incidental cross-situational statistical exposure to words and referents. When cross-situational statistical learning (CSSL) is tested in the laboratory, there is no information on any given trial to identify the referent of a novel word. However, by tracking which objects co-occur with each word across trials, learners may acquire mappings through statistical association. While CSSL behavior is well-characterized, its brain correlates are not. The arbitrary nature of CSSL mappings suggests hippocampal involvement, but the incremental, statistical nature of the learning raises the possibility of neocortical or procedural learning systems. Prior studies have shown that neurological patients with hippocampal pathology have word-learning impairments, but this has not been tested in a statistical learning paradigm. Here, we used a neuropsychological approach to test whether patients with bilateral hippocampal pathology (N = 3) could learn new words in a CSSL paradigm. In the task, patients and healthy comparison participants completed a CSSL word-learning task in which they acquired eight word/object mappings. During each trial of the CSSL task, participants saw two objects on a computer display, heard one novel word, and selected the most likely referent. Across trials, words were 100% likely to co-occur with their referent, but only 14.3% likely with non-referents. Two of three amnesic patients learned the associations between objects and word forms, although performance was impaired relative to healthy comparison participants. Our findings show that the hippocampus is not strictly necessary for CSSL for words, although it may facilitate such learning. This is consistent with a hybrid account of CSSL supported by implicit and explicit memory systems, and may have translational applications for remediation of (word-) learning deficits in neurological populations with hippocampal pathology.

Modeling list-strength and spacing effects using version 3 of the retrieving effectively from memory (REM.3) model and its superimposition-of-similar-images assumption

Shiffrin and Steyvers (1997) introduced a model of recognition memory called retrieving effectively from memory (REM) and successfully applied it to a number of basic memory phenomena. REM incorporates differentiation, wherein item repetitions are accumulated in a single mnemonic trace rather than separate traces. This allows REM to account for several benchmark findings, including the null list-strength effect in recognition (Ratcliff, Clark, & Shiffrin, 1990). The original REM treated massed and spaced repetitions identically, which prevents it from predicting a mnemonic advantage for spaced over massed repetitions (i.e., the spacing effect). However, Shiffrin and Steyvers discussed the possibility that repetitions might be represented in a single trace only if the subject identifies that the repeated item was previously studied. It is quite plausible that subjects would notice repetitions more for massed than for spaced items. Here we show that incorporating this idea allows REM to predict three important findings in the recognition memory literature: (1) the spacing effect, (2) the finding of slightly positive list-strength effects with spaced repetitions, as opposed to massed repetitions or increased study time, and (3) list-strength effects that have been observed using very large strong-to-weak ratios (see Norman, 2002).

How do memory modules differentially contribute to familiarity and recollection?

We fully support dissociating the subjective experience from the memory contents in recognition memory, as Bastin et al. posit in the target article. However, having two generic memory modules with qualitatively different functions is not mandatory and is in fact inconsistent with experimental evidence. We propose that quantitative differences in the properties of the memory modules can account for the apparent dissociation of recollection and familiarity along anatomical lines.

A Fast Visual Recognition Memory System in Humans Identified Using Intracerebral ERP

One key item of information retrieved when surveying our visual world is whether or not objects are familiar. However, there is no consensus on the respective roles of medial temporal lobe structures, particularly the perirhinal cortex (PRC) and hippocampus. We considered whether the PRC could support a fast recognition memory system independently from the hippocampus. We recorded the intracerebral electroencephalograph activity of epileptic patients while they were performing a fast visual recognition memory task, constraining them to use their quickest strategy. We performed event-related potential (ERP) and classification analyses. The PRC was, by far, the earliest region involved in recognition memory. This activity occurred before the first behavioral responses and was found to be related to reaction times, unlike the hippocampus. Single-trial analyses showed that decoding power was equivalent in the PRC and hippocampus but occurred much earlier in the PRC. A critical finding was that recognition memory-related activity occurred in different frontal and parietal regions, including the supplementary motor area, before the hippocampus. These results, based on ERP analyses, suggest that the human brain is equipped with a fast recognition memory system, which may bypass the hippocampus and in which the PRC plays a critical role.

The hippocampus as a visual area organized by space and time: A spatiotemporal similarity hypothesis

The hippocampus is the canonical memory system in the brain and is not typically considered part of the visual system. Yet, it sits atop the ventral visual stream and has been implicated in certain aspects of vision. Here I review the place of the hippocampal memory system in vision science. After a brief primer on the local circuity, external connectivity, and computational functions of the hippocampus, I explore what can be learned from each field about the other. I first present four areas of vision science (scene perception, imagery, eye movements, attention) that challenge our current understanding of the hippocampus in terms of its role in episodic memory. In the reverse direction, I leverage this understanding to inform vision science in other ways, presenting a working hypothesis about a unique form of visual representation. This spatiotemporal similarity hypothesis states that the hippocampus represents objects according to whether they co-occur in space and/or time, and not whether they look alike, as elsewhere in the visual system. This tuning may reflect hippocampal mechanisms of pattern separation, relational binding, and statistical learning, allowing the hippocampus to generate visual expectations to facilitate search and recognition.

Development of Holistic Episodic Recollection

Episodic memory binds the diverse elements of an event into a coherent representation. This coherence allows for the reconstruction of different aspects of an experience when triggered by a cue related to a past event-a process of pattern completion. Previous work has shown that such holistic recollection is evident in young adults, as revealed by dependency in retrieval success for various associations from the same event. In addition, episodic memory shows clear quantitative increases during early childhood. However, the ontogeny of holistic recollection is uncharted. Using dependency analyses, we found here that 4-year-olds (n = 32), 6-year-olds (n = 30), and young adults (n = 31) all retrieved complex events in a holistic manner; specifically, retrieval accuracy for one aspect of an event predicted accuracy for other aspects of the same event. However, the degree of holistic retrieval increased from the age 4 to adulthood. Thus, extended refinement of multiway binding may be one aspect of episodic memory development.

Unitization modulates recognition of within-domain and cross-domain associations: Evidence from event-related potentials

Although it is often assumed that memory of episodic associations requires recollection, it has been suggested that, when stimuli are experienced as a unit, earlier memory processes might contribute to their subsequent associative recognition. We investigated the effects of associative relations and perceptual domain during episodic encoding on the ability to utilize early memory processes to retrieve associative information. During the study phase, participants encoded compound and noncompound words pairs, presented either to the same sensory modality (visual presentation) or to different sensory modalities (audiovisual presentation). At the test phase, they discriminated between old, rearranged, and new pairs while ERPs were recorded. In an early ERP component, differences related to associative memory emerged only for compounds, regardless of their encoding modality. These findings indicate that episodic retrieval of compound words can be supported by early-onset recognition processes regardless of whether both words were presented to the same or different sensory modalities, and suggests that unitization can operate at an abstract level, across a broad range of materials.

Rapid Memory Reactivation at Movie Event Boundaries Promotes Episodic Encoding

Segmentation of continuous experience into discrete events is driven by rapid fluctuations in encoding stability at context shifts (i.e., event boundaries), yet the mechanisms underlying the online formation of event memories are poorly understood. We investigated the neural per-time point spatial similarity patterns of the scalp electrophysiological (EEG) activity of 30 human participants (male and female) watching a 50 min movie and found that event boundaries triggered the rapid reinstatement of the just-encoded movie event EEG patterns. We also found that the onset of memory reinstatement at boundary onset was accompanied by a left-lateralized anterior negative ERP effect, which likely reflects the detection of a shift in the narrative structure of the movie. A data-driven approach based on Hidden Markov modeling allowed us to detect event boundaries as shifts between stable patterns of brain EEG activity during encoding, and to identify their reactivation during a free recall task. These results provide the first neurophysiological underpinnings for how the memory systems segment a continuous long stream of experience into episodic events.SIGNIFICANCE OF STATEMENT Memory for specific episodic events are the building blocks of our autobiographical memory. However, it is still unclear how the memory systems structure the unfolding experience into discrete event units that can be understood and remembered at the long-term. Here, we show that the detection of context shifts, or event boundaries, during a 50 min movie viewing triggers the rapid memory reactivation of the just-encoded event to promote its successful encoding into long-term memory. By finding that memory reactivation, a neural mechanism critical for episodic memory formation and consolidation, takes place under these ecologically valid experimental circumstances, our results provide valuable insights into how the brain shapes the ongoing experience into episodic memories in the real-life.

Activation of stimulus-specific processing regions at retrieval tracks the strength of relational memory

Many theories of episodic memory posit that the subjective experience of recollection may be driven by the activation of stimulus-specific cortical regions during memory retrieval. This study examined cortical activation during associative memory retrieval to identify brain regions that support confidence judgments of source memory in stimulus-specific ways. Adjectives were encoded with either a picture of a face or a scene. During a source memory test, the word was presented alone and the participant was asked if the word had been previously paired with a face or a scene. We identified brain regions that were selectively active when viewing pictures of scenes or faces with a separate localizer scan. We then identified brain regions that were differentially activated to words during the source memory test that had been previously paired with faces or scenes, masked by the localizer activations, and examined how those regions were modulated by the strength of the source memory. Bilateral amygdala activation tracked source memory confidence for faces, while parahippocampal cortex tracked source memory confidence for scenes. The magnitude of the activation of these domain-specific perceptual-processing brain regions during memory retrieval may contribute to the subjective strength of episodic recollection.

Visual working memory impairments for single items following medial temporal lobe damage

A growing body of research indicates that the medial temporal lobe (MTL) is essential not only for long-term episodic memory but also for visual working memory (VWM). In particular, recent work has shown that the MTL is especially important for VWM when complex, high-resolution binding is required. However, all of these studies tested VWM for multiple items which invites the possibility that working memory capacity was exceeded and patient impairments instead reflected deficits in long-term memory. Thus, the precise conditions under which the MTL is critical for VWM and the type of working memory processes that are affected by MTL damage are not yet clear. To address these issues, we examined the effects of MTL damage on VWM for a single item (i.e., a square that contained color, location, and orientation information) using confidence-based receiver operating characteristic methods to assess VWM discriminability and to separate perceiving- and sensing-based memory judgments. This approach was motivated by dual-process theories of cognition that posit distinct subprocesses underlie performance across perception, working memory, and long-term memory. The results indicated that MTL patients were significantly impaired in VWM for a single item. Interestingly, the patients were not impaired at making accurate high-confidence judgments that a change had occurred (i.e., perceiving), rather they were impaired at making low-confidence judgments that they sensed whether or not there had been a change in the absence of identifying the exact change. These results demonstrate that the MTL is critical in supporting working memory even for a single item, and that it contributes selectively to sensing-based discriminations.

Hippocampal Engagement during Recall Depends on Memory Content

The hippocampus is considered pivotal to recall, allowing retrieval of information not available in the immediate environment. In contrast, neocortex is thought to signal familiarity, contributing to recall only when called upon by the hippocampus. However, this view is not compatible with representational accounts of memory, which reject the mapping of cognitive processes onto brain regions. According to representational accounts, the hippocampus is not engaged by recall per se, rather it is engaged whenever hippocampal representations are required. To test whether hippocampus is engaged by recall when hippocampal representations are not required, we used functional imaging and a non-associative recall task, with images (objects, scenes) studied in isolation, and image patches as cues. As predicted by a representational account, hippocampal activation was modulated by the content of the recalled memory, increasing during recall of scenes-which are known to be processed by hippocampus-but not during recall of objects. Object recall instead engaged neocortical regions known to be involved in object-processing. Further supporting the representational account, effective connectivity analyses revealed that changes in functional activation during recall were driven by increased information flow from neocortical sites, rather than by the spreading of recall-related activation from hippocampus back to neocortex.