Methods for reducing interference in the Complementary Learning Systems model: oscillating inhibition and autonomous memory rehearsal

Recurring memory reactivation: The offline component of learning

One can be aware of the effort needed to memorize a new fact or to recall the name of a new acquaintance. Because of experiences like this, learning can seem to have only two components, encoding information and, after some delay, retrieving information. To the contrary, learning entails additional, intervening steps that sometimes are hidden from the learner. For firmly acquiring fact and event knowledge in particular, learners are generally not cognizant of the necessity of offline consolidation. The memories that persist to be available reliably at a later time, according to the present conceptualization, are the ones we repeatedly rehearse and integrate with other knowledge, whether we do this intentionally or unknowingly, awake or asleep. This article examines the notion that learning is not a function of waking brain activity alone. What happens in the brain while we sleep also impacts memory storage, and consequently is a critical component of learning. The idea that memories can change over time and become enduring has long been present in memory research and is foundational for the concept of memory consolidation. Nevertheless, the notion that memory consolidation happens during sleep faced much resistance before eventually being firmly established. Research is still needed to elucidate the operation and repercussions of repeated reactivation during sleep. Comprehensively understanding how offline memory reactivation contributes to learning is vital for both theoretical and practical considerations.

A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation

How do we build up our knowledge of the world over time? Many theories of memory formation and consolidation have posited that the hippocampus stores new information, then "teaches" this information to the neocortex over time, especially during sleep. But it is unclear, mechanistically, how this actually works-How are these systems able to interact during periods with virtually no environmental input to accomplish useful learning and shifts in representation? We provide a framework for thinking about this question, with neural network model simulations serving as demonstrations. The model is composed of hippocampus and neocortical areas, which replay memories and interact with one another completely autonomously during simulated sleep. Oscillations are leveraged to support error-driven learning that leads to useful changes in memory representation and behavior. The model has a non-rapid eye movement (NREM) sleep stage, where dynamics between the hippocampus and neocortex are tightly coupled, with the hippocampus helping neocortex to reinstate high-fidelity versions of new attractors, and a REM sleep stage, where neocortex is able to more freely explore existing attractors. We find that alternating between NREM and REM sleep stages, which alternately focuses the model's replay on recent and remote information, facilitates graceful continual learning. We thus provide an account of how the hippocampus and neocortex can interact without any external input during sleep to drive useful new cortical learning and to protect old knowledge as new information is integrated.

Learning cortical representations through perturbed and adversarial dreaming

Humans and other animals learn to extract general concepts from sensory experience without extensive teaching. This ability is thought to be facilitated by offline states like sleep where previous experiences are systemically replayed. However, the characteristic creative nature of dreams suggests that learning semantic representations may go beyond merely replaying previous experiences. We support this hypothesis by implementing a cortical architecture inspired by generative adversarial networks (GANs). Learning in our model is organized across three different global brain states mimicking wakefulness, non-rapid eye movement (NREM), and REM sleep, optimizing different, but complementary, objective functions. We train the model on standard datasets of natural images and evaluate the quality of the learned representations. Our results suggest that generating new, virtual sensory inputs via adversarial dreaming during REM sleep is essential for extracting semantic concepts, while replaying episodic memories via perturbed dreaming during NREM sleep improves the robustness of latent representations. The model provides a new computational perspective on sleep states, memory replay, and dreams, and suggests a cortical implementation of GANs.

Theta Neurofeedback Training Supports Motor Performance and Flow Experience

Flow is defined as a cognitive state that is associated with a feeling of automatic and effortless control, enabling peak performance in highly challenging situations. In sports, flow can be enhanced by mindfulness training, which has been associated with frontal theta activity (4-8 Hz). Moreover, frontal-midline theta oscillations were shown to subserve control processes in a large variety of cognitive tasks. Based on previous theta neurofeedback training studies, which revealed that one training session is sufficient to enhance motor performance, the present study investigated whether one 30-minute session of frontal-midline theta neurofeedback training (1) enhances flow experience additionally to motor performance in a finger tapping task, and (2) transfers to cognitive control processes in an n-back task. Participants, who were able to successfully upregulate their theta activity during neurofeedback training (responders), showed better motor performance and flow experience after training than participants, who did not enhance their theta activity (non-responders). Across all participants, increase of theta activity during training was associated with motor performance enhancement from pretest to posttest irrespective of pre-training performance. Interestingly, theta training gains were also linked to the increase of flow experience, even when corresponding increases in motor performance were controlled for. Results for the n-back task were not significant. Even though these findings are mainly correlational in nature and additional flow-promoting influences need to be investigated, the present findings suggest that frontal-midline theta neurofeedback training is a promising tool to support flow experience with additional relevance for performance enhancement.

The Modulation of Working-Memory Performance Using Gamma-Electroacupuncture and Theta-Electroacupuncture in Healthy Adults

Working memory (WM), a central component of general cognition, plays an essential role in human beings' daily lives. WM impairments often occur in psychiatric, neurodegenerative, and neurodevelopmental disorders, mainly presenting as loss of high-load WM. In previous research, electroacupuncture (EA) has been shown to be an effective treatment for cognitive impairments. Frequency parameters are an important factor in therapeutic results, but the optimal frequency parameters of EA have not yet been identified. In this study, we chose theta-EA (θ-EA; 6 Hz) and gamma-EA (γ-EA; 40 Hz), corresponding to the transcranial alternating-current stimulation (tACS) frequency parameters at the Baihui (DU20) and Shenting (DU24) acupoints, in order to compare the effects of different EA frequencies on WM. We evaluated WM performance using visual 1-back, 2-back, and 3-back WM tasks involving digits. Each participant (N = 30) attended three different sessions in accordance with a within-subject crossover design. We performed θ-EA, γ-EA, and sham-EA in a counterbalanced order, conducting the WM task both before and after intervention. The results showed that d-prime (d′) under all three stimulation conditions had no significance in the 1-back and 2-back tasks. However, in the 3-back task, there was a significant improvement in d′ after intervention compared to d′ before intervention under θ-EA (F [1, 29] = 22.64; P < 0.001), while we saw no significant difference in the γ-EA and sham-EA groups. Reaction times for hits (RT-hit) under all three stimulation conditions showed decreasing trends in 1-, 2-, and 3-back tasks but without statistically significant differences. These findings suggest that the application of θ-EA might facilitate high-load WM performance.

Neurofeedback training improves episodic and semantic long-term memory performance

Understanding and improving memory are vital to enhance human life. Theta rhythm is associated with memory consolidation and coding, but the trainability and effects on long-term memory of theta rhythm are unknown. This study investigated the ability to improve long-term memory using a neurofeedback (NFB) technique reflecting the theta/low-beta power ratio on an electroencephalogram (EEG). Our study consisted of three stages. First, the long-term memory of participants was measured. In the second stage, the participants in the NFB group received 3 days of theta/low-beta NFB training. In the third stage, the long-term memory was measured again. The NFB group had better episodic and semantic long-term memory than the control group and significant differences in brain activity between episodic and semantic memory during the recall tests were revealed. These findings suggest that it is possible to improve episodic and semantic long-term memory abilities through theta/low-beta NFB training.

Does memory reactivation during sleep support generalization at the cost of memory specifics?

Sleep is important for memory, but does it favor consolidation of specific details or extraction of generalized information? Both may occur together when memories are reactivated during sleep, or a loss of certain memory details may facilitate generalization. To examine these issues, we tested memory in participants who viewed landscape paintings by six artists. Paintings were cropped to show only a section of the scene. During a learning phase, each painting section was presented with the artist's name and with a nonverbal sound that had been uniquely associated with that artist. In a test of memory for specifics, participants were shown arrays of six painting sections, all by the same artist. Participants attempted to select the one that was seen in the learning phase. Generalization was tested by asking participants to view new paintings and, for each one, decide which of the six artists created it. After this testing, participants had a 90-minute sleep opportunity with polysomnographic monitoring. When slow-wave sleep was detected, three of the sound cues associated with the artists were repeatedly presented without waking the participants. After sleep, participants were again tested for memory specifics and generalization. Memory reactivation during sleep due to the sound cues led to a relative decline in accuracy on the specifics test, which could indicate the transition to a loss of detail that facilitates generalization, particularly details such as the borders. Generalization performance showed very little change after sleep and was unaffected by the sound cues. Although results tentatively implicate sleep in memory transformation, further research is needed to examine memory change across longer time periods.

Memory and Sleep: How Sleep Cognition Can Change the Waking Mind for the Better

The memories that we retain can serve many functions. They guide our future actions, form a scaffold for constructing the self, and continue to shape both the self and the way we perceive the world. Although most memories we acquire each day are forgotten, those integrated within the structure of multiple prior memories tend to endure. A rapidly growing body of research is steadily elucidating how the consolidation of memories depends on their reactivation during sleep. Processing memories during sleep not only helps counteract their weakening but also supports problem solving, creativity, and emotional regulation. Yet, sleep-based processing might become maladaptive, such as when worries are excessively revisited. Advances in research on memory and sleep can thus shed light on how this processing influences our waking life, which can further inspire the development of novel strategies for decreasing detrimental rumination-like activity during sleep and for promoting beneficial sleep cognition.

Stimulating the sleeping brain: Current approaches to modulating memory-related sleep physiology

BACKGROUND

One of the most audacious proposals throughout the history of psychology was the potential ability to learn while we sleep. The idea penetrated culture via sci-fi movies and inspired the invention of devices that claimed to teach foreign languages, facts, and even quit smoking by simply listening to audiocassettes or other devices during sleep. However, the promises from this endeavor didn't stand up to experimental scrutiny, and the dream was shunned from the scientific community. Despite the historic evidence that the sleeping brain cannot learn new complex information (i.e., words, images, facts), a new wave of current interventions are demonstrating that sleep can be manipulated to strengthen recent memories.

NEW METHOD

Several recent approaches have been developed that play with the sleeping brain in order to modify ongoing memory processing. Here, we provide an overview of the available techniques to non-invasively modulate memory-related sleep physiology, including sensory, vestibular and electrical stimulation, as well as pharmacological approaches.

RESULTS

N/A.

COMPARISON WITH EXISTING METHODS

N/A.

CONCLUSIONS

Although the results are encouraging, suggesting that in general the sleeping brain may be optimized for better memory performance, the road to bring these techniques in free-living conditions is paved with unanswered questions and technical challenges that need to be carefully addressed.

Computational models of memory consolidation and long-term synaptic plasticity during sleep

The brain stores memories by persistently changing the connectivity between neurons. Sleep is known to be critical for these changes to endure. Research on the neurobiology of sleep and the mechanisms of long-term synaptic plasticity has provided data in support of various theories of how brain activity during sleep affects long-term synaptic plasticity. The experimental findings - and therefore the theories - are apparently quite contradictory, with some evidence pointing to a role of sleep in the forgetting of irrelevant memories, whereas other results indicate that sleep supports the reinforcement of the most valuable recollections. A unified theoretical framework is in need. Computational modeling and simulation provide grounds for the quantitative testing and comparison of theoretical predictions and observed data, and might serve as a strategy to organize the rather complicated and diverse pool of data and methodologies used in sleep research. This review article outlines the emerging progress in the computational modeling and simulation of the main theories on the role of sleep in memory consolidation.

Memory corticalization triggered by REM sleep: mechanisms of cellular and systems consolidation

Once viewed as a passive physiological state, sleep is a heterogeneous and complex sequence of brain states with essential effects on synaptic plasticity and neuronal functioning. Rapid-eye-movement (REM) sleep has been shown to promote calcium-dependent plasticity in principal neurons of the cerebral cortex, both during memory consolidation in adults and during post-natal development. This article reviews the plasticity mechanisms triggered by REM sleep, with a focus on the emerging role of kinases and immediate-early genes for the progressive corticalization of hippocampus-dependent memories. The body of evidence suggests that memory corticalization triggered by REM sleep is a systemic phenomenon with cellular and molecular causes.

Retrieval and sleep both counteract the forgetting of spatial information

Repeatedly studying information is a good way to strengthen memory storage. Nevertheless, testing recall often produces superior long-term retention. Demonstrations of this testing effect, typically with verbal stimuli, have shown that repeated retrieval through testing reduces forgetting. Sleep also benefits memory storage, perhaps through repeated retrieval as well. That is, memories may generally be subject to forgetting that can be counteracted when memories become reactivated, and there are several types of reactivation: (i) via intentional restudying, (ii) via testing, (iii) without provocation during wake, or (iv) during sleep. We thus measured forgetting for spatial material subjected to repeated study or repeated testing followed by retention intervals with sleep versus wake. Four groups of subjects learned a set of visual object-location associations and either restudied the associations or recalled locations given the objects as cues. We found the advantage for restudied over retested information was greater in the PM than AM group. Additional groups tested at 5-min and 1-wk retention intervals confirmed previous findings of greater relative benefits for restudying in the short-term and for retesting in the long-term. Results overall support the conclusion that repeated reactivation through testing or sleeping stabilizes information against forgetting.

What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated

We update complementary learning systems (CLS) theory, which holds that intelligent agents must possess two learning systems, instantiated in mammalians in neocortex and hippocampus. The first gradually acquires structured knowledge representations while the second quickly learns the specifics of individual experiences. We broaden the role of replay of hippocampal memories in the theory, noting that replay allows goal-dependent weighting of experience statistics. We also address recent challenges to the theory and extend it by showing that recurrent activation of hippocampal traces can support some forms of generalization and that neocortical learning can be rapid for information that is consistent with known structure. Finally, we note the relevance of the theory to the design of artificial intelligent agents, highlighting connections between neuroscience and machine learning.

The Sync/deSync Model: How a Synchronized Hippocampus and a Desynchronized Neocortex Code Memories

Neural oscillations are important for memory formation in the brain. The desynchronization of alpha (10 Hz) oscillations in the neocortex has been shown to predict successful memory encoding and retrieval. However, when engaging in learning, it has been found that the hippocampus synchronizes in theta (4 Hz) oscillations, and that learning is dependent on the phase of theta. This inconsistency as to whether synchronization is "good" for memory formation leads to confusion over which oscillations we should expect to see and where during learning paradigm experiments. This paper seeks to respond to this inconsistency by presenting a neural network model of how a well functioning learning system could exhibit both of these phenomena, i.e., desynchronization of alpha and synchronization of theta during successful memory encoding.We present a spiking neural network (the Sync/deSync model) of the neocortical and hippocampal system. The simulated hippocampus learns through an adapted spike-time dependent plasticity rule, in which weight change is modulated by the phase of an extrinsically generated theta oscillation. Additionally, a global passive weight decay is incorporated, which is also modulated by theta phase. In this way, the Sync/deSync model exhibits theta phase-dependent long-term potentiation and long-term depression. We simulated a learning paradigm experiment and compared the oscillatory dynamics of our model with those observed in single-cell and scalp-EEG studies of the medial temporal lobe. Our Sync/deSync model suggests that both the desynchronization of neocortical alpha and the synchronization of hippocampal theta are necessary for successful memory encoding and retrieval.SIGNIFICANCE STATEMENT A fundamental question is the role of rhythmic activation of neurons, i.e., how and why their firing oscillates between high and low rates. A particularly important question is how oscillatory dynamics between the neocortex and hippocampus support memory formation. We present a spiking neural-network model of such memory formation, with the central ideas that (1) in neocortex, neurons need to break out of an alpha oscillation to represent a stimulus (i.e., alpha desynchronizes), whereas (2) in hippocampus, the firing of neurons at theta facilitates formation of memories (i.e., theta synchronizes). Accordingly, successful memory formation is marked by reduced neocortical alpha and increased hippocampal theta. This pattern has been observed experimentally and gives our model its name-the Sync/deSync model.

Distinct Neurophysiological Mechanisms Support the Online Formation of Individual and Across-Episode Memory Representations

Individual experiences often overlap in their content, presenting opportunities for rapid generalization across them. In this study, we show in 2 independent experiments that integrative encoding-the ability to form individual and across memory representations during online encoding-is supported by 2 distinct neurophysiological responses. Brain potential is increased gradually during encoding and fit to a trial level memory measure for individual episodes, whereas neural oscillations in the theta range (4-6 Hz) emerge later during learning and predict participants' generalization performance in a subsequent test. These results suggest that integrative encoding requires the recruitment of 2 separate neural mechanisms that, despite their co-occurrence in time, differ in their underlying neural dynamics, reflect different brain learning rates and are supportive of the formation of opposed memory representations, individual versus across-event episodes.

Neural Differentiation of Incorrectly Predicted Memories

When an item is predicted in a particular context but the prediction is violated, memory for that item is weakened (Kim et al., 2014). Here, we explore what happens when such previously mispredicted items are later reencountered. According to prior neural network simulations, this sequence of events-misprediction and subsequent restudy-should lead to differentiation of the item's neural representation from the previous context (on which the misprediction was based). Specifically, misprediction weakens connections in the representation to features shared with the previous context and restudy allows new features to be incorporated into the representation that are not shared with the previous context. This cycle of misprediction and restudy should have the net effect of moving the item's neural representation away from the neural representation of the previous context. We tested this hypothesis using human fMRI by tracking changes in item-specific BOLD activity patterns in the hippocampus, a key structure for representing memories and generating predictions. In left CA2/3/DG, we found greater neural differentiation for items that were repeatedly mispredicted and restudied compared with items from a control condition that was identical except without misprediction. We also measured prediction strength in a trial-by-trial fashion and found that greater misprediction for an item led to more differentiation, further supporting our hypothesis. Therefore, the consequences of prediction error go beyond memory weakening. If the mispredicted item is restudied, the brain adaptively differentiates its memory representation to improve the accuracy of subsequent predictions and to shield it from further weakening.SIGNIFICANCE STATEMENT Competition between overlapping memories leads to weakening of nontarget memories over time, making it easier to access target memories. However, a nontarget memory in one context might become a target memory in another context. How do such memories get restrengthened without increasing competition again? Computational models suggest that the brain handles this by reducing neural connections to the previous context and adding connections to new features that were not part of the previous context. The result is neural differentiation away from the previous context. Here, we provide support for this theory, using fMRI to track neural representations of individual memories in the hippocampus and how they change based on learning.

tACS Phase Locking of Frontal Midline Theta Oscillations Disrupts Working Memory Performance

Background: Frontal midline theta (FMT) oscillations (4–8 Hz) are strongly related to cognitive and executive control during mental tasks such as memory processing, arithmetic problem solving or sustained attention. While maintenance of temporal order information during a working memory (WM) task was recently linked to FMT phase, a positive correlation between FMT power, WM demand and WM performance was shown. However, the relationship between these measures is not well understood, and it is unknown whether purposeful FMT phase manipulation during a WM task impacts FMT power and WM performance. Here we present evidence that FMT phase manipulation mediated by transcranial alternating current stimulation (tACS) can block WM demand-related FMT power increase (FMTΔpower) and disrupt normal WM performance.

Methods: Twenty healthy volunteers were assigned to one of two groups (group A, group B) and performed a 2-back task across a baseline block (block 1) and an intervention block (block 2) while 275-sensor magnetoencephalography (MEG) was recorded. After no stimulation was applied during block 1, participants in group A received tACS oscillating at their individual FMT frequency over the prefrontal cortex (PFC) while group B received sham stimulation during block 2. After assessing and mapping phase locking values (PLV) between the tACS signal and brain oscillatory activity across the whole brain, FMT power and WM performance were assessed and compared between blocks and groups.

Results: During block 2 of group A but not B, FMT oscillations showed increased PLV across task-related cortical areas underneath the frontal tACS electrode. While WM task-related FMTΔpower and WM performance were comparable across groups in block 1, tACS resulted in lower FMTΔpower and WM performance compared to sham stimulation in block 2.

Conclusion: tACS-related manipulation of FMT phase can disrupt WM performance and influence WM task-related FMTΔpower. This finding may have important implications for the treatment of brain disorders such as depression and attention deficit disorder associated with abnormal regulation of FMT activity or disorders characterized by dysfunctional coupling of brain activity, e.g., epilepsy, Alzheimer’s or Parkinson’s disease (AD/PD).

Laminar profile of spontaneous and evoked theta: Rhythmic modulation of cortical processing during word integration

Theta may play a central role during language understanding and other extended cognitive processing, providing an envelope for widespread integration of participating cortical areas. We used linear microelectrode arrays in epileptics to define the circuits generating theta in inferotemporal, perirhinal, entorhinal, prefrontal and anterior cingulate cortices. In all locations, theta was generated by excitatory current sinks in middle layers which receive predominantly feedforward inputs, alternating with sinks in superficial layers which receive mainly feedback/associative inputs. Baseline and event-related theta were generated by indistinguishable laminar profiles of transmembrane currents and unit-firing. Word presentation could reset theta phase, permitting theta to contribute to late event-related potentials, even when theta power decreases relative to baseline. Limited recordings during sentence reading are consistent with rhythmic theta activity entrained by a given word modulating the neural background for the following word. These findings show that theta occurs spontaneously, and can be momentarily suppressed, reset and synchronized by words. Theta represents an alternation between feedforward/divergent and associative/convergent processing modes that may temporally organize sustained processing and optimize the timing of memory formation. We suggest that words are initially encoded via a ventral feedforward stream which is lexicosemantic in the anteroventral temporal lobe; its arrival may trigger a widespread theta rhythm which integrates the word within a larger context.

Targeted Memory Reactivation during Sleep Depends on Prior Learning

STUDY OBJECTIVES

When sounds associated with learning are presented again during slow-wave sleep, targeted memory reactivation (TMR) can produce improvements in subsequent location recall. Here we used TMR to investigate memory consolidation during an afternoon nap as a function of prior learning.

PARTICIPANTS

Twenty healthy individuals (8 male, 19-23 y old).

MEASUREMENTS AND RESULTS

Participants learned to associate each of 50 common objects with a unique screen location. When each object appeared, its characteristic sound was played. After electroencephalography (EEG) electrodes were applied, location recall was assessed for each object, followed by a 90-min interval for sleep. During EEG-verified slow-wave sleep, half of the sounds were quietly presented over white noise. Recall was assessed 3 h after initial learning. A beneficial effect of TMR was found in the form of higher recall accuracy for cued objects compared to uncued objects when pre-sleep accuracy was used as an explanatory variable. An analysis of individual differences revealed that this benefit was greater for participants with higher pre-sleep recall accuracy. In an analysis for individual objects, cueing benefits were apparent as long as initial recall was not highly accurate. Sleep physiology analyses revealed that the cueing benefit correlated with delta power and fast spindle density.

CONCLUSIONS

These findings substantiate the use of targeted memory reactivation (TMR) methods for manipulating consolidation during sleep. TMR can selectively strengthen memory storage for object-location associations learned prior to sleep, except for those near-perfectly memorized. Neural measures found in conjunction with TMR-induced strengthening provide additional evidence about mechanisms of sleep consolidation.

REM sleep rescues learning from interference

Classical human memory studies investigating the acquisition of temporally-linked events have found that the memories for two events will interfere with each other and cause forgetting (i.e., interference; Wixted, 2004). Importantly, sleep helps consolidate memories and protect them from subsequent interference (Ellenbogen, Hulbert, Stickgold, Dinges, & Thompson-Schill, 2006). We asked whether sleep can also repair memories that have already been damaged by interference. Using a perceptual learning paradigm, we induced interference either before or after a consolidation period. We varied brain states during consolidation by comparing active wake, quiet wake, and naps with either non-rapid eye movement sleep (NREM), or both NREM and REM sleep. When interference occurred after consolidation, sleep and wake both produced learning. However, interference prior to consolidation impaired memory, with retroactive interference showing more disruption than proactive interference. Sleep rescued learning damaged by interference. Critically, only naps that contained REM sleep were able to rescue learning that was highly disrupted by retroactive interference. Furthermore, the magnitude of rescued learning was correlated with the amount of REM sleep. We demonstrate the first evidence of a process by which the brain can rescue and consolidate memories damaged by interference, and that this process requires REM sleep. We explain these results within a theoretical model that considers how interference during encoding interacts with consolidation processes to predict which memories are retained or lost.

Neural Differentiation Tracks Improved Recall of Competing Memories Following Interleaved Study and Retrieval Practice

Selective retrieval of overlapping memories can generate competition. How does the brain adaptively resolve this competition? One possibility is that competing memories are inhibited; in support of this view, numerous studies have found that selective retrieval leads to forgetting of memories that are related to the just-retrieved memory. However, this retrieval-induced forgetting (RIF) effect can be eliminated or even reversed if participants are given opportunities to restudy the materials between retrieval attempts. Here, we outline an explanation for such a reversal, rooted in a neural network model of RIF that predicts representational differentiation when restudy is interleaved with selective retrieval. To test this hypothesis, we measured changes in pattern similarity of the BOLD fMRI signal elicited by related memories after undergoing interleaved competitive retrieval and restudy. Reduced pattern similarity within the hippocampus positively correlated with retrieval-induced facilitation of competing memories. This result is consistent with an adaptive differentiation process that allows individuals to learn to distinguish between once-confusable memories.

Memory consolidation from seconds to weeks: a three-stage neural network model with autonomous reinstatement dynamics

Declarative long-term memories are not created in an instant. Gradual stabilization and temporally shifting dependence of acquired declarative memories in different brain regions-called systems consolidation-can be tracked in time by lesion experiments. The observation of temporally graded retrograde amnesia (RA) following hippocampal lesions points to a gradual transfer of memory from hippocampus to neocortical long-term memory. Spontaneous reactivations of hippocampal memories, as observed in place cell reactivations during slow-wave-sleep, are supposed to drive neocortical reinstatements and facilitate this process. We propose a functional neural network implementation of these ideas and furthermore suggest an extended three-state framework that includes the prefrontal cortex (PFC). It bridges the temporal chasm between working memory percepts on the scale of seconds and consolidated long-term memory on the scale of weeks or months. We show that our three-stage model can autonomously produce the necessary stochastic reactivation dynamics for successful episodic memory consolidation. The resulting learning system is shown to exhibit classical memory effects seen in experimental studies, such as retrograde and anterograde amnesia (AA) after simulated hippocampal lesioning; furthermore the model reproduces peculiar biological findings on memory modulation, such as retrograde facilitation of memory after suppressed acquisition of new long-term memories-similar to the effects of benzodiazepines on memory.

Multilayered neural network with structural lateral inhibition for incremental learning and conceptualization

Distributed connectionist networks have difficulty learning incrementally because the representations in the network overlap. Therefore, it is necessary to reduce the overlaps of representations for incremental learning. At the same time, the representational overlaps give these networks the ability to generalize. In this study, we use a modified multilayered neural network to numerically examine the trade-off between incremental learning and generalization abilities, and then we propose a novel network model with structural lateral inhibitions to reconcile the two abilities. We also analyze the behavior of the proposed model using Formal Concept Analysis, which reveals that the network implements "conceptualization": differentiation and meditation between intensional and extensional representations. This study suggests a new paradigm for the traditional question, whether representations in the brain are distributed or not.

A matched filter hypothesis for cognitive control

The prefrontal cortex exerts top-down influences on several aspects of higher-order cognition by functioning as a filtering mechanism that biases bottom-up sensory information toward a response that is optimal in context. However, research also indicates that not all aspects of complex cognition benefit from prefrontal regulation. Here we review and synthesize this research with an emphasis on the domains of learning and creative cognition, and outline how the appropriate level of cognitive control in a given situation can vary depending on the organism's goals and the characteristics of the given task. We offer a matched filter hypothesis for cognitive control, which proposes that the optimal level of cognitive control is task-dependent, with high levels of cognitive control best suited to tasks that are explicit, rule-based, verbal or abstract, and can be accomplished given the capacity limits of working memory and with low levels of cognitive control best suited to tasks that are implicit, reward-based, non-verbal or intuitive, and which can be accomplished irrespective of working memory limitations. Our approach promotes a view of cognitive control as a tool adapted to a subset of common challenges, rather than an all-purpose optimization system suited to every problem the organism might encounter.

Frontal midline theta oscillations during working memory maintenance and episodic encoding and retrieval

Neural oscillations in the theta band (4-8 Hz) are prominent in the human electroencephalogram (EEG), and many recent electrophysiological studies in animals and humans have implicated scalp-recorded frontal midline theta (FMT) in working memory and episodic memory encoding and retrieval processes. However, the functional significance of theta oscillations in human memory processes remains largely unknown. Here, we review studies in human and animals examining how scalp-recorded FMT relates to memory behaviors and also their possible neural generators. We also discuss models of the functional relevance of theta oscillations to memory processes and suggest promising directions for future research.

Phase coding by grid cells in unconstrained environments: two-dimensional phase precession

Action potential timing is thought to play a critical role in neural representation. For example, theta phase precession is a robust phenomenon exhibited by spatial cells of the rat entorhinal-hippocampal circuit. In phase precession, the time a neuron fires relative to the phase of theta rhythm (6-10 Hz) oscillations in the local field potential reduces uncertainty about the position of the animal. This relationship between neural firing and behavior has made precession an important constraint for hypothetical mechanisms of temporal coding. However, challenges exist in identifying what regulates the spike timing of these cells. We have developed novel analytical techniques for mapping between behavior and neural firing that provide sufficient sensitivity to examine features of grid cell phase coding in open environments. Here, we show robust, omnidirectional phase precession by entorhinal grid cells in openfield enclosures. We present evidence that full phase precession persists regardless of how close the animal comes to the center of a firing field. Many conjunctive grid cells, previously thought to be phase locked, also exhibited phase coding. However, we were unable to detect directional- or field-specific phase coding predicted by some variants of models. Finally, we present data that suggest bursting of layer II grid cells contributes to the bimodality of phase precession. We discuss implications of these observations for models of temporal coding and propose the utility of these techniques in other domains where behavior is aligned to neural spiking.

The effect of attention on repetition suppression and multivoxel pattern similarity

Fundamental to our understanding of learning is the role of attention. We investigated how attention affects two fMRI measures of stimulus-specific memory: repetition suppression (RS) and pattern similarity (PS). RS refers to the decreased fMRI signal when a stimulus is repeated, and it is sensitive to manipulations of attention and task demands. In PS, region-wide voxel-level patterns of responses are evaluated for their similarity across repeated presentations of a stimulus. More similarity across presentations is related to better learning, but the role of attention on PS is not known. Here, we directly compared these measures during the visual repetition of scenes while manipulating attention. Consistent with previous findings, we observed RS in the scene-sensitive parahippocampal place area only when a scene was attended both at initial presentation and upon repetition in subsequent trials, indicating that attention is important for RS. Likewise, we observed greater PS in response to repeated pairs of scenes when both instances of the scene were attended than when either or both were ignored. However, RS and PS did not correlate on either a scene-by-scene or subject-by-subject basis, and PS measures revealed above-chance similarity even when stimuli were ignored. Thus, attention has different effects on RS and PS measures of perceptual repetition.

Why trace and delay conditioning are sometimes (but not always) hippocampal dependent: a computational model

A recurrent-network model provides a unified account of the hippocampal region in mediating the representation of temporal information in classical eyeblink conditioning. Much empirical research is consistent with a general conclusion that delay conditioning (in which the conditioned stimulus CS and unconditioned stimulus US overlap and co-terminate) is independent of the hippocampal system, while trace conditioning (in which the CS terminates before US onset) depends on the hippocampus. However, recent studies show that, under some circumstances, delay conditioning can be hippocampal-dependent and trace conditioning can be spared following hippocampal lesion. Here, we present an extension of our prior trial-level models of hippocampal function and stimulus representation that can explain these findings within a unified framework. Specifically, the current model includes adaptive recurrent collateral connections that aid in the representation of intra-trial temporal information. With this model, as in our prior models, we argue that the hippocampus is not specialized for conditioned response timing, but rather is a general-purpose system that learns to predict the next state of all stimuli given the current state of variables encoded by activity in recurrent collaterals. As such, the model correctly predicts that hippocampal involvement in classical conditioning should be critical not only when there is an intervening trace interval, but also when there is a long delay between CS onset and US onset. Our model simulates empirical data from many variants of classical conditioning, including delay and trace paradigms in which the length of the CS, the inter-stimulus interval, or the trace interval is varied. Finally, we discuss model limitations, future directions, and several novel empirical predictions of this temporal processing model of hippocampal function and learning.

Inhibition in the dynamics of selective attention: an integrative model for negative priming

We introduce a computational model of the negative priming (NP) effect that includes perception, memory, attention, decision making, and action. The model is designed to provide a coherent picture across competing theories of NP. The model is formulated in terms of abstract dynamics for the activations of features, their binding into object entities, their semantic categorization as well as related memories and appropriate reactions. The dynamic variables interact in a connectionist network which is shown to be adaptable to a variety of experimental paradigms. We find that selective attention can be modeled by means of inhibitory processes and by a threshold dynamics. From the necessity of quantifying the experimental paradigms, we conclude that the specificity of the experimental paradigm must be taken into account when predicting the nature of the NP effect.

Malignant synaptic growth and Alzheimer's disease

In this article, we will describe the malignant synaptic growth hypothesis of Alzheimer's disease. Originally presented in 1994, the hypothesis remains a viable model of the functional and biophysical mechanisms underlying the development and progression of Alzheimer's disease. In this article, we will refresh the model with references to relevant empirical support that has been generated in the intervening two decades since it's original presentation. We will include discussion of its relationship, in terms of points of alignment and points of contention, to other models of Alzheimer's disease, including the cholinergic hypothesis and the tau and β-amyloid models of Alzheimer's disease. Finally, we propose several falsifiable predictions made by the malignant synaptic growth hypothesis and describe the avenues of treatment that hold the greatest promise under this hypothesis.

Cholinergic modulation of cognitive processing: insights drawn from computational models

Acetylcholine plays an important role in cognitive function, as shown by pharmacological manipulations that impact working memory, attention, episodic memory, and spatial memory function. Acetylcholine also shows striking modulatory influences on the cellular physiology of hippocampal and cortical neurons. Modeling of neural circuits provides a framework for understanding how the cognitive functions may arise from the influence of acetylcholine on neural and network dynamics. We review the influences of cholinergic manipulations on behavioral performance in working memory, attention, episodic memory, and spatial memory tasks, the physiological effects of acetylcholine on neural and circuit dynamics, and the computational models that provide insight into the functional relationships between the physiology and behavior. Specifically, we discuss the important role of acetylcholine in governing mechanisms of active maintenance in working memory tasks and in regulating network dynamics important for effective processing of stimuli in attention and episodic memory tasks. We also propose that theta rhythm plays a crucial role as an intermediary between the physiological influences of acetylcholine and behavior in episodic and spatial memory tasks. We conclude with a synthesis of the existing modeling work and highlight future directions that are likely to be rewarding given the existing state of the literature for both empiricists and modelers.

An opportunistic theory of cellular and systems consolidation

Memories are often classified as hippocampus dependent or independent, and sleep has been found to facilitate both, but in different ways. In this Opinion, we explore the optimal neural state for cellular and systems consolidation of hippocampus-dependent memories that benefit from sleep. We suggest that these two kinds of consolidation, which are ordinarily treated separately, overlap in time and jointly benefit from a period of reduced interference (during which no new memories are formed). Conditions that result in reduced interference include slow wave sleep (SWS), NMDA receptor antagonists, benzodiazepines, alcohol and acetylcholine antagonists. We hypothesize that the consolidation of hippocampal-dependent memories might not depend on SWS per se. Instead, the brain opportunistically consolidates previously encoded memories whenever the hippocampus is not otherwise occupied by the task of encoding new memories.

Overlapping memory replay during sleep builds cognitive schemata

Sleep enhances integration across multiple stimuli, abstraction of general rules, insight into hidden solutions and false memory formation. Newly learned information is better assimilated if compatible with an existing cognitive framework or schema. This article proposes a mechanism by which the reactivation of newly learned memories during sleep could actively underpin both schema formation and the addition of new knowledge to existing schemata. Under this model, the overlapping replay of related memories selectively strengthens shared elements. Repeated reactivation of memories in different combinations progressively builds schematic representations of the relationships between stimuli. We argue that this selective strengthening forms the basis of cognitive abstraction, and explain how it facilitates insight and false memory formation.

An Infomax Algorithm Can Perform Both Familiarity Discrimination and Feature Extraction in a Single Network

Psychological experiments have shown that the capacity of the brain for discriminating visual stimuli as novel or familiar is almost limitless. Neurobiological studies have established that the perirhinal cortex is critically involved in both familiarity discrimination and feature extraction. However, opinion is divided as to whether these two processes are performed by the same neurons. Previously proposed models have been unable to simultaneously extract features and discriminate familiarity for large numbers of stimuli. We show that a well-known model of visual feature extraction, Infomax, can simultaneously perform familiarity discrimination and feature extraction efficiently. This model has a significantly larger capacity than previously proposed models combining these two processes, particularly when correlation exists between inputs, as is the case in the perirhinal cortex. Furthermore, we show that once the model fully extracts features, its ability to perform familiarity discrimination increases markedly.

How hippocampus and cortex contribute to recognition memory: revisiting the complementary learning systems model

We describe how the Complementary Learning Systems neural network model of recognition memory (Norman and O'Reilly (2003) Psychol Rev 104:611-646) can shed light on current debates regarding hippocampal and cortical contributions to recognition memory. We review simulation results illustrating three critical differences in how (according to the model) hippocampus and cortex contribute to recognition memory, all of which derive from the hippocampus' use of pattern separated representations. Pattern separation makes the hippocampus especially well-suited for discriminating between studied items and related lures; it makes the hippocampus especially poorly suited for computing global match; and it imbues the hippocampal ROC curve with a Y-intercept > 0. We also describe a key boundary condition on these differences: When the average level of similarity between items in an experiment is very high, hippocampal pattern separation can fail, at which point the hippocampal model will start to behave like the cortical model. We describe the implications of these simulation results for extant debates over how to describe hippocampal versus cortical contributions and how to measure these contributions.

Recognition memory: material, processes, and substrates

The proposal that a system centering on the perirhinal cortex is responsible for familiarity discrimination, particularly for single items, whereas a system centering on the hippocampus is responsible for recollective and more complex associational aspects of recognition memory is reviewed in the light of recent findings. In particular, the proposal is reviewed in relation to recent animal work with rats and results from human clinical studies. Notably, progress has been made in determining potential neural memory substrate mechanisms within the perirhinal cortex in rats. Recent findings have emphasized the importance of specifying the type of material, the type of test, and the strategy used by subjects to solve recognition memory tests if substrates are to be accurately inferred. It is to be expected that the default condition is that both the hippocampal and perirhinal systems will contribute to recognition memory performance. Indeed, rat lesion experiments provide examples of where cooperation between both systems is essential. Nevertheless, there remain examples of the independent operation of the hippocampal and perirhinal systems. Overall, it is concluded that most, though not all, of the recent findings are in support of the proposal. However, there is also evidence that the systems involved in recognition memory need to include structures outside the medial temporal lobe: there are significant but as yet only partially defined roles for the prefrontal cortex and sensory association cortices in recognition memory processes.

REM-dependent repair of competitive memory suppression

Memories are enhanced during sleep through memory consolidation processes. A recent study reported that sleep increases competitive forgetting in the absence of sleep-dependent consolidation of the target memory (Racsmany et al. in Psychol Sci 21:80-85, 2010). Here, using a modified retrieval-induced forgetting task, we examined whether sleep-dependent modulation of forgetting occurs concurrently with the consolidation of related target memories. Participants encoded a word-pair list and then practiced retrieving a portion of these pairs. Following a break with sleep or wake, recall of all pairs was tested. As expected, recall for practiced pairs was greater following sleep relative to wake. Contrary to Racsmany et al. (Psychol Sci 21:80-85, 2010), competitive forgetting decreased following sleep. Moreover, recall for practiced pairs correlated with slow wave sleep (SWS) while forgetting of competing targets correlated negatively with REM, suggesting a novel function of these sequential brain states on memory processing.

A single-trace dual-process model of episodic memory: a novel computational account of familiarity and recollection

Dual-process theories of episodic memory state that retrieval is contingent on two independent processes: familiarity (providing a sense of oldness) and recollection (recovering events and their context). A variety of studies have reported distinct neural signatures for familiarity and recollection, supporting dual-process theory. One outstanding question is whether these signatures reflect the activation of distinct memory traces or the operation of different retrieval mechanisms on a single memory trace. We present a computational model that uses a single neuronal network to store memory traces, but two distinct and independent retrieval processes access the memory. The model is capable of performing familiarity and recollection-based discrimination between old and new patterns, demonstrating that dual-process models need not to rely on multiple independent memory traces, but can use a single trace. Importantly, our putative familiarity and recollection processes exhibit distinct characteristics analogous to those found in empirical data; they diverge in capacity and sensitivity to sparse and correlated patterns, exhibit distinct ROC curves, and account for performance on both item and associative recognition tests. The demonstration that a single-trace, dual-process model can account for a range of empirical findings highlights the importance of distinguishing between neuronal processes and the neuronal representations on which they operate.

A complementary systems account of word learning: neural and behavioural evidence

In this paper we present a novel theory of the cognitive and neural processes by which adults learn new spoken words. This proposal builds on neurocomputational accounts of lexical processing and spoken word recognition and complementary learning systems (CLS) models of memory. We review evidence from behavioural studies of word learning that, consistent with the CLS account, show two stages of lexical acquisition: rapid initial familiarization followed by slow lexical consolidation. These stages map broadly onto two systems involved in different aspects of word learning: (i) rapid, initial acquisition supported by medial temporal and hippocampal learning, (ii) slower neocortical learning achieved by offline consolidation of previously acquired information. We review behavioural and neuroscientific evidence consistent with this account, including a meta-analysis of PET and functional Magnetic Resonance Imaging (fMRI) studies that contrast responses to spoken words and pseudowords. From this meta-analysis we derive predictions for the location and direction of cortical response changes following familiarization with pseudowords. This allows us to assess evidence for learning-induced changes that convert pseudoword responses into real word responses. Results provide unique support for the CLS account since hippocampal responses change during initial learning, whereas cortical responses to pseudowords only become word-like if overnight consolidation follows initial learning.

The sleeping brain's influence on verbal memory: boosting resistance to interference

Memories evolve. After learning something new, the brain initiates a complex set of post-learning processing that facilitates recall (i.e., consolidation). Evidence points to sleep as one of the determinants of that change. But whenever a behavioral study of episodic memory shows a benefit of sleep, critics assert that sleep only leads to a temporary shelter from the damaging effects of interference that would otherwise accrue during wakefulness. To evaluate the potentially active role of sleep for verbal memory, we compared memory recall after sleep, with and without interference before testing. We demonstrated that recall performance for verbal memory was greater after sleep than after wakefulness. And when using interference testing, that difference was even more pronounced. By introducing interference after sleep, this study confirms an experimental paradigm that demonstrates the active role of sleep in consolidating memory, and unmasks the large magnitude of that benefit.

Computational models can replicate the capacity of human recognition memory

The capacity of human recognition memory was investigated by Standing, who presented several groups of participants with different numbers of pictures (from 20 to 10 000), and subsequently tested their ability to distinguish between previously presented and novel pictures. The estimated number of pictures retained in recognition memory by different groups when plotted as a logarithmic function of the number of pictures presented formed a straight line, representing a power-law relationship. Here, we investigate if published models of familiarity discrimination can replicate Standing's results. We first consider a simplified assumption that visual stimuli are represented by uncorrelated patterns of firing of visual neurons providing input to the familiarity discrimination network. We show that for this case three models (Familiarity discrimination based on Energy (FamE), Anti-Hebbian and Info-max) can reproduce the observed power-law relationship when their synaptic weights are appropriately initialized. For more realistic assumptions on neural representation of stimuli, the FamE model is no longer able to reproduce the power-law relationship in simulations, while the Anti-Hebbian and Info-max can reproduce it. Nevertheless, the slopes of the power-law relationships produced by the models in all simulations differ from that observed by Standing. We discuss possible reasons for this difference, including separate contributions of familiarity and recollection processes, and describe experimentally testable predictions based on our analysis.