The Role of Hippocampal Replay in Memory and Planning

Is episodic-like memory like episodic memory?

Episodic memory involves the conscious recollection of personally experienced events and when absent, results in profound losses to the typical human conscious experience. Over the last 2.5 decades, the debate surrounding whether episodic memory is unique to humans has seen a lot of controversy and accordingly has received significant research attention. Various behavioural paradigms have been developed to test episodic-like memory; a term designed to reflect the behavioural characteristics of episodic memory in the absence of evidence for consciously experienced recall. In this review, we first outline the most influential paradigms that have been developed to assess episodic-like memory across a variety of non-human taxa (including mammals, birds and cephalopods), namely the what-where-when memory, incidental encoding and unexpected question, and source memory paradigms. Then, we examine whether various key features of human episodic memory are conceptually represented in episodic-like memory across phylogenetically and neurologically diverse taxa, identifying similarities, differences and gaps in the literature. We conclude that the evidence is mixed, and as episodic memory encompasses a variety of cognitive structures and processes, research on episodic-like memory in non-humans should follow this multifaceted approach and assess evidence across various behavioural paradigms that each target different aspects of human episodic memory.This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Memory engram stability and flexibility

Many studies have shown that memories are encoded in sparse neural ensembles distributed across the brain. During the post-encoding period, often during sleep, many of the cells that were active during encoding are reactivated, supporting consolidation of this memory. During memory recall, many of the same cells that were active during encoding and reactivated during consolidation are reactivated during recall. These ensembles of cells have been referred to as the memory engram cells, stably representing a specific memory. However, recent studies question the rigidity of the "stable memory engram." Here we review the past literature of how episodic-like memories are encoded, consolidated, and recalled. We also highlight more recent studies (as well as some older literature) that suggest that these stable memories and their representations are much more dynamic and flexible than previously thought. We highlight some of these processes, including memory updating, reconsolidation, forgetting, schema learning, memory-linking, and representational drift.

Cost-benefit Tradeoff Mediates the Rule- to Memory-based Processing Transition during Practice

Practice not only improves task performance but also changes task execution from rule-to memory-based processing by incorporating experiences from practice. However, how and when this change occurs is unclear. We test the hypothesis that strategy transitions in task learning can result from decision-making guided by cost-benefit analysis. Participants learn two task sequences and are then queried about the task type at a cued sequence and position. Behavioral improvement with practice can be accounted for by a computational model implementing cost-benefit analysis, and the model-predicted strategy transition points align with the observed behavioral slowing. Model comparisons using behavioral data show that strategy transitions are better explained by a cost-benefit analysis across alternative strategies rather than solely on memory strength. Model-guided fMRI findings suggest that the brain encodes a decision variable reflecting the cost-benefit analysis and that different strategy representations are double-dissociated. Further analyses reveal that strategy transitions are associated with activation patterns in the dorsolateral prefrontal cortex and increased pattern separation in the ventromedial prefrontal cortex. Together, these findings support cost-benefit analysis as a mechanism of practice-induced strategy shift.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Impaired hippocampal functions underlying memory encoding and consolidation precede robust Aβ deposition in a mouse model of Alzheimer's disease

Current therapeutic strategies for Alzheimer's disease (AD) target amyloid-beta (Aβ) fibrils and high molecular weight protofibrils associated with plaques, but molecular cascades associated with AD may drive neural systems failure before Aβ plaque deposition in AD. Employing hippocampal electrophysiological recordings and dynamic calcium imaging across the sleep-wake cycle in the APP/PS1 mouse model of AD before Aβ plaques accumulated, we detected marked impairments of hippocampal systems function: In a spatial behavioral task, but not REM sleep, phase-amplitude coupling (PAC) of the hippocampal theta and gamma oscillations was impaired and place cell calcium fluctuations were hyper-synchronized with the theta oscillation. In subsequent slow wave sleep (SWS), place cell reactivation was reduced. These degraded neural functions underlying memory encoding and consolidation support targeting pathological processes of the pre-plaque phase of AD to treat and prevent hippocampal impairments.

Expectancy-related changes in firing of dopamine neurons depend on hippocampus

The orbitofrontal cortex (OFC) and hippocampus (HC) both contribute to the cognitive maps that support flexible behavior. Previously, we used the dopamine neurons to measure the functional role of OFC. We recorded midbrain dopamine neurons as rats performed an odor-based choice task, in which expected rewards were manipulated across blocks. We found that ipsilateral OFC lesions degraded dopaminergic prediction errors, consistent with reduced resolution of the task states. Here we have repeated this experiment in male rats with ipsilateral HC lesions. The results show HC also shapes the task states, however unlike OFC, which provides information local to the trial, the HC is necessary for estimating upper-level hidden states that distinguish blocks. The results contrast the roles of the OFC and HC in cognitive mapping and suggest that the dopamine neurons access rich information from distributed regions regarding the environment's structure, potentially enabling this teaching signal to support complex behaviors.

A tradeoff between efficiency and robustness in the hippocampal-neocortical memory network during human and rodent sleep

Sleep constitutes a brain state of disengagement from the external world that supports memory consolidation and restores cognitive resources. The precise mechanisms how sleep and its varied stages support information processing remain largely unknown. Synaptic scaling models imply that daytime learning accumulates neural information, which is then consolidated and downregulated during sleep. Currently, there is a lack of in-vivo data from humans and rodents that elucidate if, and how, sleep renormalizes information processing capacities. From an information-theoretical perspective, a consolidation process should entail a reduction in neural pattern variability over the course of a night. Here, in a cross-species intracranial study, we identify a tradeoff in the neural population code during sleep where information coding efficiency is higher in the neocortex than in hippocampal archicortex in humans than in rodents as well as during wakefulness compared to sleep. Critically, non-REM sleep selectively reduces information coding efficiency through pattern repetition in the neocortex in both species, indicating a transition to a more robust information coding regime. Conversely, the coding regime in the hippocampus remained consistent from wakefulness to non-REM sleep. These findings suggest that new information could be imprinted to the long-term mnemonic storage in the neocortex through pattern repetition during sleep. Lastly, our results show that task engagement increased coding efficiency, while medically-induced unconsciousness disrupted the population code. In sum, these findings suggest that neural pattern variability could constitute a fundamental principle underlying cognitive engagement and memory formation, while pattern repetition reflects robust coding, possibly underlying the consolidation process.

A circuit from lateral hypothalamic to dorsal hippocampal dentate gyrus modulates behavioral despair in mice

Behavioral despair is one of the clinical manifestations of major depressive disorder and an important cause of disability and death. However, the neural circuit mechanisms underlying behavioral despair are poorly understood. In a well-established chronic behavioral despair (CBD) mouse model, using a combination of viral tracing, in vivo fiber photometry, chemogenetic and optogenetic manipulations, in vitro electrophysiology, pharmacological profiling techniques, and behavioral tests, we investigated the neural circuit mechanisms in regulating behavioral despair. Here, we found that CBD enhanced CaMKIIα neuronal excitability in the dorsal dentate gyrus (dDG) and dDGCaMKIIα neurons involved in regulating behavioral despair in CBD mice. Besides, dDGCaMKIIα neurons received 5-HT inputs from median raphe nucleus (MRN) and were mediated by 5-HT1A receptors, whereas MRN5-HT neurons received CaMKIIα inputs from lateral hypothalamic (LH) and were mediated by AMPA receptors to regulate behavioral despair. Furthermore, fluvoxamine exerted its role in resisting behavioral despair through the LH-MRN-dDG circuit. These findings suggest that a previously unidentified circuit of LHCaMKIIα-MRN5-HT-dDGCaMKIIα mediates behavioral despair induced by CBD. Furthermore, these support the important role of AMPA receptors in MRN and 5-HT1A receptors in dDG that might be the potential targets for treatment of behavioral despair, and explain the neural circuit mechanism of fluvoxamine-resistant behavioral despair.

Inhibitory plasticity supports replay generalization in the hippocampus

Memory consolidation assimilates recent experiences into long-term memory. This process requires the replay of learned sequences, although the content of these sequences remains controversial. Recent work has shown that the statistics of replay deviate from those of experience: stimuli that are experientially salient may be either recruited or suppressed from sharp-wave ripples. In this study, we found that this phenomenon can be explained parsimoniously and biologically plausibly by a Hebbian spike-time-dependent plasticity rule at inhibitory synapses. Using models at three levels of abstraction-leaky integrate-and-fire, biophysically detailed and abstract binary-we show that this rule enables efficient generalization, and we make specific predictions about the consequences of intact and perturbed inhibitory dynamics for network dynamics and cognition. Finally, we use optogenetics to artificially implant non-generalizable representations into the network in awake behaving mice, and we find that these representations also accumulate inhibition during sharp-wave ripples, experimentally validating a major prediction of our model. Our work outlines a potential direct link between the synaptic and cognitive levels of memory consolidation, with implications for both normal learning and neurological disease.

Repetition dynamically and rapidly increases cortical, but not hippocampal, offline reactivation

No sooner is an experience over than its neural representation begins to be transformed through memory reactivation during offline periods. The lion's share of prior research has focused on understanding offline reactivation within the hippocampus. However, it is hypothesized that consolidation processes involve offline reactivation in cortical regions as well as coordinated reactivation in the hippocampus and cortex. Using fMRI, we presented novel and repeated paired associates to participants during encoding and measured offline memory reactivation for those events during an immediate post-encoding rest period. post-encoding reactivation frequency of repeated and once-presented events did not differ in the hippocampus. However, offline reactivation in widespread cortical regions and hippocampal-cortical coordinated reactivation were significantly enhanced for repeated events. These results provide evidence that repetition might facilitate the distribution of memory representations across cortical networks, a hallmark of systems-level consolidation. Interestingly, we found that offline reactivation frequency in both hippocampus and cortex explained variance in behavioral success on an immediate associative recognition test for the once-presented information, potentially indicating a role of offline reactivation in maintaining these novel, weaker, memories. Together, our findings highlight that endogenous offline reactivation can be robustly and significantly modulated by study repetition.

Human hippocampal and entorhinal neurons encode the temporal structure of experience

Extracting the underlying temporal structure of experience is a fundamental aspect of learning and memory that allows us to predict what is likely to happen next. Current knowledge about the neural underpinnings of this cognitive process in humans stems from functional neuroimaging research1-5. As these methods lack direct access to the neuronal level, it remains unknown how this process is computed by neurons in the human brain. Here we record from single neurons in individuals who have been implanted with intracranial electrodes for clinical reasons, and show that human hippocampal and entorhinal neurons gradually modify their activity to encode the temporal structure of a complex image presentation sequence. This representation was formed rapidly, without providing specific instructions to the participants, and persisted when the prescribed experience was no longer present. Furthermore, the structure recovered from the population activity of hippocampal-entorhinal neurons closely resembled the structural graph defining the sequence, but at the same time, also reflected the probability of upcoming stimuli. Finally, learning of the sequence graph was related to spontaneous, time-compressed replay of individual neurons' activity corresponding to previously experienced graph trajectories. These findings demonstrate that neurons in the hippocampus and entorhinal cortex integrate the 'what' and 'when' information to extract durable and predictive representations of the temporal structure of human experience.

Why might animals remember? A functional framework for episodic memory research in comparative psychology

One of Clayton's major contributions to our understanding of animal minds has been her work on episodic-like memory. A central reason for the success of this work was its focus on ecological validity: rather than looking for episodic memory for arbitrary stimuli in artificial contexts, focussing on contexts in which episodic memory would serve a biological function such as food caching. This review aims to deepen this insight by surveying the numerous functions that have been proposed for episodic memory, articulating a philosophically grounded framework for understanding what exactly functions are, and drawing on these to make suggestions for future directions in the comparative cognitive psychology of episodic memory. Our review suggests four key insights. First, episodic memory may have more than one function and may have different functions in different species. Second, cross-disciplinary work is key to developing a functional account of episodic memory. Third, there is scope for further theoretical elaboration of proposals relating episodic memory to food caching and, in particular, future-oriented cognition. Finally, learning-related functions suggested by AI (artificial intelligence)-based models are a fruitful avenue for future behavioural research.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Replay-triggered brain-wide activation in humans

The consolidation of discrete experiences into a coherent narrative shapes the cognitive map, providing structured mental representations of our experiences. In this process, past memories are reactivated and replayed in sequence, fostering hippocampal-cortical dialogue. However, brain-wide engagement coinciding with sequential reactivation (or replay) of memories remains largely unexplored. In this study, employing simultaneous EEG-fMRI, we capture both the spatial and temporal dynamics of memory replay. We find that during mental simulation, past memories are replayed in fast sequences as detected via EEG. These transient replay events are associated with heightened fMRI activity in the hippocampus and medial prefrontal cortex. Replay occurrence strengthens functional connectivity between the hippocampus and the default mode network, a set of brain regions key to representing the cognitive map. On the other hand, when subjects are at rest following learning, memory reactivation of task-related items is stronger than that of pre-learning rest, and is also associated with heightened hippocampal activation and augmented hippocampal connectivity to the entorhinal cortex. Together, our findings highlight a distributed, brain-wide engagement associated with transient memory reactivation and its sequential replay.

Space is a latent sequence: A theory of the hippocampus

Fascinating phenomena such as landmark vector cells and splitter cells are frequently discovered in the hippocampus. Without a unifying principle, each experiment seemingly uncovers new anomalies or coding types. Here, we provide a unifying principle that the mental representation of space is an emergent property of latent higher-order sequence learning. Treating space as a sequence resolves numerous phenomena and suggests that the place field mapping methodology that interprets sequential neuronal responses in Euclidean terms might itself be a source of anomalies. Our model, clone-structured causal graph (CSCG), employs higher-order graph scaffolding to learn latent representations by mapping aliased egocentric sensory inputs to unique contexts. Learning to compress sequential and episodic experiences using CSCGs yields allocentric cognitive maps that are suitable for planning, introspection, consolidation, and abstraction. By explicating the role of Euclidean place field mapping and demonstrating how latent sequential representations unify myriad observed phenomena, our work positions the hippocampus in a sequence-centric paradigm, challenging the prevailing space-centric view.

Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus

The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.

Shared structure facilitates working memory of multiple sequences

Daily experiences often involve the processing of multiple sequences, yet storing them challenges the limited capacity of working memory (WM). To achieve efficient memory storage, relational structures shared by sequences would be leveraged to reorganize and compress information. Here, participants memorized a sequence of items with different colors and spatial locations and later reproduced the full color and location sequences one after another. Crucially, we manipulated the consistency between location and color sequence trajectories. First, sequences with consistent trajectories demonstrate improved memory performance and a trajectory correlation between reproduced color and location sequences. Second, sequences with consistent trajectories show neural reactivation of common trajectories, and display spontaneous replay of color sequences when recalling locations. Finally, neural reactivation correlates with WM behavior. Our findings suggest that a shared common structure is leveraged for the storage of multiple sequences through compressed encoding and neural replay, together facilitating efficient information organization in WM.

Self-avoidance dominates the selection of hippocampal replay

Spontaneous neural activity sequences are generated by the brain in the absence of external input 1-12 , yet how they are produced remains unknown. During immobility, hippocampal replay sequences depict spatial paths related to the animal's past experience or predicted future 13 . By recording from large ensembles of hippocampal place cells 14 in combination with optogenetic manipulation of cortical input in freely behaving rats, we show here that the selection of hippocampal replay is governed by a novel self-avoidance principle. Following movement cessation, replay of the animal's past path is strongly avoided, while replay of the future path predominates. Moreover, when the past and future paths overlap, early replays avoid both and depict entirely different trajectories. Further, replays avoid self-repetition, on a shorter timescale compared to the avoidance of previous behavioral trajectories. Eventually, several seconds into the stopping period, replay of the past trajectory dominates. This temporal organization contrasts with established and recent predictions 9,10,15,16 but is well-recapitulated by a symmetry-breaking attractor model of sequence generation in which individual neurons adapt their firing rates over time 26-35 . However, while the model is sufficient to produce avoidance of recently traversed or reactivated paths, it requires an additional excitatory input into recently activated cells to produce the later window of past-dominance. We performed optogenetic perturbations to demonstrate that this input is provided by medial entorhinal cortex, revealing its role in maintaining a memory of past experience that biases hippocampal replay. Together, these data provide specific evidence for how hippocampal replays are generated.

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

Prefrontal coding of learned and inferred knowledge during REM and NREM sleep

Idling brain activity has been proposed to facilitate inference, insight, and innovative problem-solving. However, it remains unclear how and when the idling brain can create novel ideas. Here, we show that cortical offline activity is both necessary and sufficient for building unlearned inferential knowledge from previously acquired information. In a transitive inference paradigm, male C57BL/6J mice gained the inference 1 day after, but not shortly after, complete training. Inhibiting the neuronal computations in the anterior cingulate cortex (ACC) during post-learning either non-rapid eye movement (NREM) or rapid eye movement (REM) sleep, but not wakefulness, disrupted the inference without affecting the learned knowledge. In vivo Ca2+ imaging suggests that NREM sleep organizes the scattered learned knowledge in a complete hierarchy, while REM sleep computes the inferential information from the organized hierarchy. Furthermore, after insufficient learning, artificial activation of medial entorhinal cortex-ACC dialog during only REM sleep created inferential knowledge. Collectively, our study provides a mechanistic insight on NREM and REM coordination in weaving inferential knowledge, thus highlighting the power of idling brain in cognitive flexibility.

Reactivation strength during cued recall is modulated by graph distance within cognitive maps

Declarative memory retrieval is thought to involve reinstatement of neuronal activity patterns elicited and encoded during a prior learning episode. Furthermore, it is suggested that two mechanisms operate during reinstatement, dependent on task demands: individual memory items can be reactivated simultaneously as a clustered occurrence or, alternatively, replayed sequentially as temporally separate instances. In the current study, participants learned associations between images that were embedded in a directed graph network and retained this information over a brief 8 min consolidation period. During a subsequent cued recall session, participants retrieved the learned information while undergoing magnetoencephalographic recording. Using a trained stimulus decoder, we found evidence for clustered reactivation of learned material. Reactivation strength of individual items during clustered reactivation decreased as a function of increasing graph distance, an ordering present solely for successful retrieval but not for retrieval failure. In line with previous research, we found evidence that sequential replay was dependent on retrieval performance and was most evident in low performers. The results provide evidence for distinct performance-dependent retrieval mechanisms, with graded clustered reactivation emerging as a plausible mechanism to search within abstract cognitive maps.

Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Experience replay is a powerful mechanism to learn efficiently from limited experience. Despite several decades of compelling experimental results, the factors that determine which experiences are selected for replay remain unclear. A particular challenge for current theories is that on tasks that feature unbalanced experience, rats paradoxically replay the less-experienced trajectory. To understand why, we simulated a feedforward neural network with two regimes: rich learning (structured representations tailored to task demands) and lazy learning (unstructured, task-agnostic representations). Rich, but not lazy, representations degraded following unbalanced experience, an effect that could be reversed with paradoxical replay. To test if this computational principle can account for the experimental data, we examined the relationship between paradoxical replay and learned task representations in the rat hippocampus. Strikingly, we found a strong association between the richness of learned task representations and the paradoxicality of replay. Taken together, these results suggest that paradoxical replay specifically serves to protect rich representations from the destructive effects of unbalanced experience, and more generally demonstrate a novel interaction between the nature of task representations and the function of replay in artificial and biological systems.

In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Similar to associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work, we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.

Altered patterning of neural activity in a tauopathy mouse model

Alzheimer's disease (AD) is a complex neurodegenerative condition that manifests at multiple levels and involves a spectrum of abnormalities ranging from the cellular to cognitive. Here, we investigate the impact of AD-related tau-pathology on hippocampal circuits in mice engaged in spatial navigation, and study changes of neuronal firing and dynamics of extracellular fields. While most studies are based on analyzing instantaneous or time-averaged characteristics of neuronal activity, we focus on intermediate timescales-spike trains and waveforms of oscillatory potentials, which we consider as single entities. We find that, in healthy mice, spike arrangements and wave patterns (series of crests or troughs) are coupled to the animal's location, speed, and acceleration. In contrast, in tau-mice, neural activity is structurally disarrayed: brainwave cadence is detached from locomotion, spatial selectivity is lost, the spike flow is scrambled. Importantly, these alterations start early and accumulate with age, which exposes progressive disinvolvement the hippocampus circuit in spatial navigation. These features highlight qualitatively different neurodynamics than the ones provided by conventional analyses, and are more salient, thus revealing a new level of the hippocampal circuit disruptions.

The Effectiveness of Mindfulness in the Treatment of Methamphetamine Addiction Symptoms: Does Neuroplasticity Play a Role?

INTRODUCTION

Methamphetamine is a highly stimulating psychoactive drug that causes life-threatening addictions and affects millions of people around the world. Its effects on the brain are complex and include disturbances in the neurotransmitter systems and neurotoxicity. There are several known treatment methods, but their effectiveness is moderate. It must be emphasised that no drugs have been approved for treatment. For this reason, there is an urgent need to develop new, effective, and safe treatments for methamphetamine. One of the potential treatments is mindfulness meditation. In recent years, this technique has been researched extensively in the context of many neurological and psychiatric disorders.

METHODS

This review explores the use of mindfulness in the treatment of methamphetamine addiction. Searches were conducted in the PubMed/Medline, Research Gate, and Cochrane databases.

RESULTS

Ten studies were identified that used mindfulness-based interventions in the treatment of methamphetamine addiction. The results show that mindfulness is an effective form of reducing hunger, risk of relapses, stress indicators, depression, and aggression, alone or in combination with transcranial direct current stimulation (tDCS). Mindfulness also improved the cognitive function in addicts. The included studies used only behavioural measures. The potential mechanisms of mindfulness in addiction were explained, and it was proposed that it can induce neuroplasticity, alleviating the symptoms of addiction.

CONCLUSIONS

Evidence from the studies suggest that mindfulness may be an effective treatment option for methamphetamine addiction, used alone or in combination with tDCS. However, further high-quality research is required to establish the role of this treatment option in this field. The use of neuroimaging and neurophysiological measures is fundamental to understand the mechanisms of mindfulness.

No Benefit in Memory Performance after Nocturnal Memory Reactivation Coupled with Theta-tACS

Targeted memory reactivation (TMR) is an effective technique to enhance sleep-associated memory consolidation. The successful reactivation of memories by external reminder cues is typically accompanied by an event-related increase in theta oscillations, preceding better memory recall after sleep. However, it remains unclear whether the increase in theta oscillations is a causal factor or an epiphenomenon of successful TMR. Here, we used transcranial alternating current stimulation (tACS) to examine the causal role of theta oscillations for TMR during non-rapid eye movement (non-REM) sleep. Thirty-seven healthy participants learned Dutch-German word pairs before sleep. During non-REM sleep, we applied either theta-tACS or control-tACS (23 Hz) in blocks (9 min) in a randomised order, according to a within-subject design. One group of participants received tACS coupled with TMR time-locked two seconds after the reminder cue (time-locked group). Another group received tACS in a continuous manner while TMR cues were presented (continuous group). Contrary to our predictions, we observed no frequency-specific benefit of theta-tACS coupled with TMR during sleep on memory performance, neither for continuous nor time-locked stimulation. In fact, both stimulation protocols blocked the TMR-induced memory benefits during sleep, resulting in no memory enhancement by TMR in both the theta and control conditions. No frequency-specific effect was found on the power analyses of the electroencephalogram. We conclude that tACS might have an unspecific blocking effect on memory benefits typically observed after TMR during non-REM sleep.

Dorsal hippocampus represents locations to avoid as well as locations to approach during approach-avoidance conflict

Worrying about perceived threats is a hallmark of multiple psychological disorders including anxiety. This concern about future events is particularly important when an individual is faced with an approach-avoidance conflict. Potential goals to approach are known to be represented in the dorsal hippocampus during theta sweeps. Similarly, important non-local information is represented during hippocampal high synchrony events (HSEs), which are correlated with sharp-wave ripples (SWRs). It is likely that potential future threats may be similarly represented. We examined how threats and rewards were represented within the hippocampus during approach-avoidance conflicts in rats faced with a predator-like robot guarding a food reward. We found representations of the pseudo-predator during HSEs when hesitating in the nest, and during theta prior to retreating as the rats approached the pseudo-predator. After the first attack, we observed new place fields appearing at the location of the robot (not the location the rat was when attacked). The anxiolytic diazepam reduced anxiety-like behavior and altered hippocampal local field potentials, including reducing SWRs, suggesting that one potential mechanism of diazepam's actions may be through altered representations of imagined threat. These results suggest that hippocampal representation of potential threats could be an important mechanism that underlies worry and a potential target for anxiolytics.

Grid cells: the missing link in understanding Parkinson's disease?

The mechanisms underlying Parkinson's disease (PD) are complex and not fully understood, and the box-and-arrow model among other current models present significant challenges. This paper explores the potential role of the allocentric brain and especially its grid cells in several PD motor symptoms, including bradykinesia, kinesia paradoxa, freezing of gait, the bottleneck phenomenon, and their dependency on cueing. It is argued that central hubs, like the locus coeruleus and the pedunculopontine nucleus, often narrowly interpreted in the context of PD, play an equally important role in governing the allocentric brain as the basal ganglia. Consequently, the motor and secondary motor (e.g., spatially related) symptoms of PD linked with dopamine depletion may be more closely tied to erroneous computation by grid cells than to the basal ganglia alone. Because grid cells and their associated central hubs introduce both spatial and temporal information to the brain influencing velocity perception they may cause bradykinesia or hyperkinesia as well. In summary, PD motor symptoms may primarily be an allocentric disturbance resulting from virtual faulty computation by grid cells revealed by dopamine depletion in PD.

A multi-stage anticipated surprise model with dynamic expectation for economic decision-making

There are many modeling works that aim to explain people's behaviors that violate classical economic theories. However, these models often do not take into full account the multi-stage nature of real-life problems and people's tendency in solving complicated problems sequentially. In this work, we propose a descriptive decision-making model for multi-stage problems with perceived post-decision information. In the model, decisions are chosen based on an entity which we call the 'anticipated surprise'. The reference point is determined by the expected value of the possible outcomes, which we assume to be dynamically changing during the mental simulation of a sequence of events. We illustrate how our formalism can help us understand prominent economic paradoxes and gambling behaviors that involve multi-stage or sequential planning. We also discuss how neuroscience findings, like prediction error signals and introspective neuronal replay, as well as psychological theories like affective forecasting, are related to the features in our model. This provides hints for future experiments to investigate the role of these entities in decision-making.

Reward Expectation Reduces Representational Drift in the Hippocampus

Spatial memory in the hippocampus involves dynamic neural patterns that change over days, termed representational drift. While drift may aid memory updating, excessive drift could impede retrieval. Memory retrieval is influenced by reward expectation during encoding, so we hypothesized that diminished reward expectation would exacerbate representational drift. We found that high reward expectation limited drift, with CA1 representations on one day gradually re-emerging over successive trials the following day. Conversely, the absence of reward expectation resulted in increased drift, as the gradual re-emergence of the previous day's representation did not occur. At the single cell level, lowering reward expectation caused an immediate increase in the proportion of place-fields with low trial-to-trial reliability. These place fields were less likely to be reinstated the following day, underlying increased drift in this condition. In conclusion, heightened reward expectation improves memory encoding and retrieval by maintaining reliable place fields that are gradually reinstated across days, thereby minimizing representational drift.

A cognitive map for value-guided choice in ventromedial prefrontal cortex

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Sensory cortical ensembles exhibit differential coupling to ripples in distinct hippocampal subregions

Cortical neurons activated during recent experiences often reactivate with dorsal hippocampal CA1 ripples during subsequent rest. Less is known about cortical interactions with intermediate hippocampal CA1, whose connectivity, functions, and ripple events differ from dorsal CA1. We identified three clusters of putative excitatory neurons in mouse visual cortex that are preferentially excited together with either dorsal or intermediate CA1 ripples or suppressed before both ripples. Neurons in each cluster were evenly distributed across primary and higher visual cortices and co-active even in the absence of ripples. These ensembles exhibited similar visual responses but different coupling to thalamus and pupil-indexed arousal. We observed a consistent activity sequence preceding and predicting ripples: (1) suppression of ripple-suppressed cortical neurons, (2) thalamic silence, and (3) activation of intermediate CA1-ripple-activated cortical neurons. We propose that coordinated dynamics of these ensembles relay visual experiences to distinct hippocampal subregions for incorporation into different cognitive maps.

What is sleep exactly? Global and local modulations of sleep oscillations all around the clock

Wakefulness, non-rapid eye-movement (NREM) and rapid eye-movement (REM) sleep differ from each other along three dimensions: behavioral, phenomenological, physiological. Although these dimensions often fluctuate in step, they can also dissociate. The current paradigm that views sleep as made of global NREM and REM states fail to account for these dissociations. This conundrum can be dissolved by stressing the existence and significance of the local regulation of sleep. We will review the evidence in animals and humans, healthy and pathological brains, showing different forms of local sleep and the consequences on behavior, cognition, and subjective experience. Altogether, we argue that the notion of local sleep provides a unified account for a host of phenomena: dreaming in REM and NREM sleep, NREM and REM parasomnias, intrasleep responsiveness, inattention and mind wandering in wakefulness. Yet, the physiological origins of local sleep or its putative functions remain unclear. Exploring further local sleep could provide a unique and novel perspective on how and why we sleep.

Memory consolidation: Building influence over the entorhinal cortex

Memory consolidation involves interactions between the hippocampus and other cortical areas. A new study identifies neurons in the medial entorhinal cortex that over learning increase their coordination with hippocampal replay events, suggesting a route for consolidation of spatial memories.

Theta-band phase locking during encoding leads to coordinated entorhinal-hippocampal replay

Precisely timed interactions between hippocampal and cortical neurons during replay epochs are thought to support learning. Indeed, research has shown that replay is associated with heightened hippocampal-cortical synchrony. Yet many caveats remain in our understanding. Namely, it remains unclear how this offline synchrony comes about, whether it is specific to particular behavioral states, and how-if at all-it relates to learning. In this study, we sought to address these questions by analyzing coordination between CA1 cells and neurons of the deep layers of the medial entorhinal cortex (dMEC) while rats learned a novel spatial task. During movement, we found a subset of dMEC cells that were particularly locked to hippocampal LFP theta-band oscillations and that were preferentially coordinated with hippocampal replay during offline periods. Further, dMEC synchrony with CA1 replay peaked ∼10 ms after replay initiation in CA1, suggesting that the distributed replay reflects extra-hippocampal information propagation and is specific to "offline" periods. Finally, theta-modulated dMEC cells showed a striking experience-dependent increase in synchronization with hippocampal replay trajectories, mirroring the animals' acquisition of the novel task and coupling to the hippocampal local field. Together, these findings provide strong support for the hypothesis that synergistic hippocampal-cortical replay supports learning and highlights phase locking to hippocampal theta oscillations as a potential mechanism by which such cross-structural synchrony comes about.

Diminished State Space Theory of Human Aging

Many new technologies, such as smartphones, computers, or public-access systems (like ticket-vending machines), are a challenge for older adults. One feature that these technologies have in common is that they involve underlying, partially observable, structures (state spaces) that determine the actions that are necessary to reach a certain goal (e.g., to move from one menu to another, to change a function, or to activate a new service). In this work we provide a theoretical, neurocomputational account to explain these behavioral difficulties in older adults. Based on recent findings from age-comparative computational- and cognitive-neuroscience studies, we propose that age-related impairments in complex goal-directed behavior result from an underlying deficit in the representation of state spaces of cognitive tasks. Furthermore, we suggest that these age-related deficits in adaptive decision-making are due to impoverished neural representations in the orbitofrontal cortex and hippocampus.

Sharp-wave-ripple-associated activity in the medial prefrontal cortex supports spatial rule switching

Previous studies have highlighted an important role for hippocampal sharp-wave ripples in spatial alternation learning, as well as in modulating activity in the medial prefrontal cortex (mPFC). However, the direct influence of hippocampal sharp-wave ripples on mPFC activity during spatial alternation learning has not been investigated. Here, we train Long Evans rats on a three-arm radial maze to perform a sequence of alternations. Three alternation sequences needed to be learned, and while learning a sequence, the activity in the mPFC was inhibited either directly following sharp-wave ripples in the hippocampus (on-time condition) or with a randomized delay (delayed condition). In the on-time condition, the behavioral performance is significantly worse compared to the same animals in the delayed inhibition condition, as measured by a lower correct alternation performance and more perseverative behavior. This indicates that the activity in the mPFC directly following hippocampal sharp-wave ripples is necessary for spatial rule switching.

Human REM sleep recalibrates neural activity in support of memory formation

The proposed mechanisms of sleep-dependent memory consolidation involve the overnight regulation of neural activity at both synaptic and whole-network levels. Now, there is a lack of in vivo data in humans elucidating if, and how, sleep and its varied stages balance neural activity, and if such recalibration benefits memory. We combined electrophysiology with in vivo two-photon calcium imaging in rodents as well as intracranial and scalp electroencephalography (EEG) in humans to reveal a key role for non-oscillatory brain activity during rapid eye movement (REM) sleep to mediate sleep-dependent recalibration of neural population dynamics. The extent of this REM sleep recalibration predicted the success of overnight memory consolidation, expressly the modulation of hippocampal-neocortical activity, favoring remembering rather than forgetting. The findings describe a non-oscillatory mechanism how human REM sleep modulates neural population activity to enhance long-term memory.

Habitual intake of fat and sugar is associated with poorer memory and greater impulsivity in humans

The vicious cycle model of obesity suggests that repeated habitual intake of a diet high in fat and sugar (HFS) results in impairment in hippocampal function which in turn increases impulsive behaviours, making it harder to resist unhealthy diet choices. Evidence from studies with rodents consistently show switching to a HFS diet impairs performance on hippocampally-sensitive memory tasks. The limited literature in humans also suggest impaired memory and increased impulsivity related to higher habitual HFS intake. However, these changes in memory and impulsivity have been looked at independently. To investigate how these effects are inter-related, three experiments were conducted where relative HFS intake was related to measures of memory and impulsivity. In Experiment 1 (90 female participants), HFS was associated with higher scores on the Everyday Memory Questionnaire-revised (EMQ), and higher scores on the total, Attention (BISatt) and Motor (BISmot) sub-scales of the Barratt Impulsiveness Scale (BIS11). Experiment 2 (84 women and 35 men), replicated the association between HFS and EMQ, and also found HFS related to poorer performance on the hippocampally-sensitive 4 mountain (4MT) memory task. The association between HFS intake and the BISatt replicated, but there were no significant associations with other BIS11 measures or delay-discounting for monetary rewards. Experiment 3 (199 women and 87 men) replicated the associations between DFS and 4MT and EMQ, and also found an association with overall recall, but not response inhibition, from a Remembering Causes Forgetting task: HFS was also significantly associated with BIS total, BISatt and BISmot. In all three studies these associations remained when potential confounds (BMI, age, gender, hunger state, restrained and disinhibited eating) were controlled for. Mediation analysis found that the effect of HFS on memory at least part mediated the relationship between HFS and impulsivity in Experiments 1 and 3, but not 2. Overall these data provide some support for the vicious cycle model, but also suggest that trait impulsivity may be a risk factor for poor dietary choice.

Individual-specific memory reinstatement patterns within human face-selective cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions involved in the sensory processing of prior events are reactivated to support this perception of the past. Recently, several studies have emphasized potential transformations in the spatial organization of reinstated activity patterns. In particular, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with those during perception. However, it is not clear that such transformations occur universally, with evidence lacking for other important stimulus categories, particularly faces. Critical to addressing these questions, and to studies of reinstatement more broadly, is the growing importance of considering meaningful variations in the organization of sensory systems across individuals. Therefore, we conducted a multi-session neuroimaging study to first carefully map individual participants face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the correspondence of activity patterns during face memory encoding and retrieval. Our results showed distinct configurations of face-selective regions within the VTC across individuals. While a significant degree of overlap was observed between face perception and memory encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions. Importantly, these activity patterns were consistently tied to individual-specific neural substrates, but did not show any consistent direction of spatial transformation (e.g., anteriorization). To provide further insight to these findings, we also report on unique human intracranial recordings from VTC under the same experimental conditions. Our findings highlight the importance of considering individual variations in functional neuroanatomy in the context of assessing the nature of cortical reinstatement. Consideration of such factors will be important for establishing general principles shaping the neural transformations that occur from perception to memory.

Offline memory replay in recurrent neuronal networks emerges from constraints on online dynamics

During spatial exploration, neural circuits in the hippocampus store memories of sequences of sensory events encountered in the environment. When sensory information is absent during 'offline' resting periods, brief neuronal population bursts can 'replay' sequences of activity that resemble bouts of sensory experience. These sequences can occur in either forward or reverse order, and can even include spatial trajectories that have not been experienced, but are consistent with the topology of the environment. The neural circuit mechanisms underlying this variable and flexible sequence generation are unknown. Here we demonstrate in a recurrent spiking network model of hippocampal area CA3 that experimental constraints on network dynamics such as population sparsity, stimulus selectivity, rhythmicity and spike rate adaptation, as well as associative synaptic connectivity, enable additional emergent properties, including variable offline memory replay. In an online stimulus-driven state, we observed the emergence of neuronal sequences that swept from representations of past to future stimuli on the timescale of the theta rhythm. In an offline state driven only by noise, the network generated both forward and reverse neuronal sequences, and recapitulated the experimental observation that offline memory replay events tend to include salient locations like the site of a reward. These results demonstrate that biological constraints on the dynamics of recurrent neural circuits are sufficient to enable memories of sensory events stored in the strengths of synaptic connections to be flexibly read out during rest and sleep, which is thought to be important for memory consolidation and planning of future behaviour. KEY POINTS: A recurrent spiking network model of hippocampal area CA3 was optimized to recapitulate experimentally observed network dynamics during simulated spatial exploration. During simulated offline rest, the network exhibited the emergent property of generating flexible forward, reverse and mixed direction memory replay events. Network perturbations and analysis of model diversity and degeneracy identified associative synaptic connectivity and key features of network dynamics as important for offline sequence generation. Network simulations demonstrate that population over-representation of salient positions like the site of reward results in biased memory replay.

Organizing memories for generalization in complementary learning systems

Memorization and generalization are complementary cognitive processes that jointly promote adaptive behavior. For example, animals should memorize safe routes to specific water sources and generalize from these memories to discover environmental features that predict new ones. These functions depend on systems consolidation mechanisms that construct neocortical memory traces from hippocampal precursors, but why systems consolidation only applies to a subset of hippocampal memories is unclear. Here we introduce a new neural network formalization of systems consolidation that reveals an overlooked tension-unregulated neocortical memory transfer can cause overfitting and harm generalization in an unpredictable world. We resolve this tension by postulating that memories only consolidate when it aids generalization. This framework accounts for partial hippocampal-cortical memory transfer and provides a normative principle for reconceptualizing numerous observations in the field. Generalization-optimized systems consolidation thus provides new insight into how adaptive behavior benefits from complementary learning systems specialized for memorization and generalization.

How Do You Build a Cognitive Map? The Development of Circuits and Computations for the Representation of Space in the Brain

In mammals, the activity of neurons in the entorhinal-hippocampal network is modulated by the animal's position and its movement through space. At multiple stages of this distributed circuit, distinct populations of neurons can represent a rich repertoire of navigation-related variables like the animal's location, the speed and direction of its movements, or the presence of borders and objects. Working together, spatially tuned neurons give rise to an internal representation of space, a cognitive map that supports an animal's ability to navigate the world and to encode and consolidate memories from experience. The mechanisms by which, during development, the brain acquires the ability to create an internal representation of space are just beginning to be elucidated. In this review, we examine recent work that has begun to investigate the ontogeny of circuitry, firing patterns, and computations underpinning the representation of space in the mammalian brain.

Rethinking the hippocampal cognitive map as a meta-learning computational module

A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition.

A deep network-based model of hippocampal memory functions under normal and Alzheimer's disease conditions

We present a deep network-based model of the associative memory functions of the hippocampus. The proposed network architecture has two key modules: (1) an autoencoder module which represents the forward and backward projections of the cortico-hippocampal projections and (2) a module that computes familiarity of the stimulus and implements hill-climbing over the familiarity which represents the dynamics of the loops within the hippocampus. The proposed network is used in two simulation studies. In the first part of the study, the network is used to simulate image pattern completion by autoassociation under normal conditions. In the second part of the study, the proposed network is extended to a heteroassociative memory and is used to simulate picture naming task in normal and Alzheimer's disease (AD) conditions. The network is trained on pictures and names of digits from 0 to 9. The encoder layer of the network is partly damaged to simulate AD conditions. As in case of AD patients, under moderate damage condition, the network recalls superordinate words ("odd" instead of "nine"). Under severe damage conditions, the network shows a null response ("I don't know"). Neurobiological plausibility of the model is extensively discussed.

An evolutionary conserved division-of-labor between archicortical and neocortical ripples organizes information transfer during sleep

Systems-level memory consolidation during sleep depends on the temporally precise interplay between cardinal sleep oscillations. Specifically, hippocampal ripples constitute a key substrate of the hippocampal-neocortical dialogue underlying memory formation. Recently, it became evident that ripples are not unique to archicortex, but constitute a wide-spread neocortical phenomenon. To date, little is known about the morphological similarities between archi- and neocortical ripples. Moreover, it remains undetermined if neocortical ripples fulfill distinct functional roles. Leveraging intracranial recordings from the human medial temporal lobe (MTL) and neocortex during sleep, our results reveal region-specific functional specializations, albeit a near-uniform morphology. While MTL ripples synchronize the memory network to trigger directional MTL-to-neocortical information flow, neocortical ripples reduce information flow to minimize interference. At the population level, MTL ripples confined population dynamics to a low-dimensional subspace, while neocortical ripples diversified the population response; thus, constituting an effective mechanism to functionally uncouple the MTL-neocortical network. Critically, we replicated the key findings in rodents, where the same division-of-labor between archi- and neocortical ripples was evident. In sum, these results uncover an evolutionary preserved mechanism where the precisely coordinated interplay between MTL and neocortical ripples temporally segregates MTL information transfer from subsequent neocortical processing during sleep.

Progress on the hippocampal circuits and functions based on sharp wave ripples

Sharp wave ripples (SWRs) are high-frequency synchronization events generated by hippocampal neuronal circuits during various forms of learning and reactivated during memory consolidation and recall. There is mounting evidence that SWRs are essential for storing spatial and social memories in rodents and short-term episodic memories in humans. Sharp wave ripples originate mainly from the hippocampal CA3 and subiculum, and can be transmitted to modulate neuronal activity in cortical and subcortical regions for long-term memory consolidation and behavioral guidance. Different hippocampal subregions have distinct functions in learning and memory. For instance, the dorsal CA1 is critical for spatial navigation, episodic memory, and learning, while the ventral CA1 and dorsal CA2 may work cooperatively to store and consolidate social memories. Here, we summarize recent studies demonstrating that SWRs are essential for the consolidation of spatial, episodic, and social memories in various hippocampal-cortical pathways, and review evidence that SWR dysregulation contributes to cognitive impairments in neurodegenerative and neurodevelopmental diseases.

Hippocampus experience inference for safety critical control of unknown multi-agent linear systems

Risk mitigation is usually addressed in simulated environments for safety critical control. The migration of the final controller requires further adjustments due to the simulation assumptions and constraints. This paper presents the design of an experience inference algorithm for safety critical control of unknown multi-agent linear systems. The approach is inspired in the close relationship between three main areas of the brain cortex that enables transfer learning and decision making: the hippocampus, the neocortex, and the striatum. The hippocampus is modelled as a stable linear model that communicates to the striatum how the real-world system is expected to behave. The hippocampus model is controlled by an adaptive dynamic programming (ADP) algorithm to achieve an optimal desired performance. The neocortex and the striatum are designed simultaneously by an actor control policy algorithm that ensures experience inference to the real-world system. Experimental and simulations studies are carried out to verify the proposed approach.

Spatiotemporal Precision of Neuroimaging in Psychiatry

Aberrant patterns of cognition, perception, and behavior seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between spatial and temporal resolutions inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography, often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illnesses such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications that seek to drive a mechanistic understanding of psychopathology and the realization of preclinical translation.

Differential replay of reward and punishment paths predicts approach and avoidance

Neural replay is implicated in planning, where states relevant to a task goal are rapidly reactivated in sequence. It remains unclear whether, during planning, replay relates to an actual prospective choice. Here, using magnetoencephalography (MEG), we studied replay in human participants while they planned to either approach or avoid an uncertain environment containing paths leading to reward or punishment. We find evidence for forward sequential replay during planning, with rapid state-to-state transitions from 20 to 90 ms. Replay of rewarding paths was boosted, relative to aversive paths, before a decision to avoid and attenuated before a decision to approach. A trial-by-trial bias toward replaying prospective punishing paths predicted irrational decisions to approach riskier environments, an effect more pronounced in participants with higher trait anxiety. The findings indicate a coupling of replay with planned behavior, where replay prioritizes an online representation of a worst-case scenario for approaching or avoiding.