On the methods for reactivation and replay analysis

Similarity of brain activity patterns during learning and subsequent resting state predicts memory consolidation

Spontaneous reactivation of brain activity from learning to a subsequent off-line period has been implicated as a neural mechanism underlying memory consolidation. However, similarities in brain activity may also emerge as a result of individual, trait-like characteristics. Here, we introduced a novel approach for analyzing continuous electroencephalography (EEG) data to investigate learning-induced changes as well as trait-like characteristics in brain activity underlying memory consolidation. Thirty-one healthy young adults performed a learning task, and their performance was retested after a short (∼1 h) delay. Consolidation of two distinct types of information (serial-order and probability) embedded in the task were tested to reveal similarities in functional networks that uniquely predict the changes in the respective memory performance. EEG was recorded during learning and pre- and post-learning rest periods. To investigate brain activity associated with consolidation, we quantified similarities in EEG functional connectivity between learning and pre-learning rest (baseline similarity) and learning and post-learning rest (post-learning similarity). While comparable patterns of these two could indicate trait-like similarities, changes from baseline to post-learning similarity could indicate learning-induced changes, possibly spontaneous reactivation. Higher learning-induced changes in alpha frequency connectivity (8.5-9.5 Hz) were associated with better consolidation of serial-order information, particularly for long-range connections across central and parietal sites. The consolidation of probability information was associated with learning-induced changes in delta frequency connectivity (2.5-3 Hz) specifically for more local, short-range connections. Furthermore, there was a substantial overlap between the baseline and post-learning similarities and their associations with consolidation performance, suggesting robust (trait-like) differences in functional connectivity networks underlying memory processes.

Self-avoidance dominates the selection of hippocampal replay

Spontaneous neural activity sequences are generated by the brain in the absence of external input 1-12 , yet how they are produced remains unknown. During immobility, hippocampal replay sequences depict spatial paths related to the animal's past experience or predicted future 13 . By recording from large ensembles of hippocampal place cells 14 in combination with optogenetic manipulation of cortical input in freely behaving rats, we show here that the selection of hippocampal replay is governed by a novel self-avoidance principle. Following movement cessation, replay of the animal's past path is strongly avoided, while replay of the future path predominates. Moreover, when the past and future paths overlap, early replays avoid both and depict entirely different trajectories. Further, replays avoid self-repetition, on a shorter timescale compared to the avoidance of previous behavioral trajectories. Eventually, several seconds into the stopping period, replay of the past trajectory dominates. This temporal organization contrasts with established and recent predictions 9,10,15,16 but is well-recapitulated by a symmetry-breaking attractor model of sequence generation in which individual neurons adapt their firing rates over time 26-35 . However, while the model is sufficient to produce avoidance of recently traversed or reactivated paths, it requires an additional excitatory input into recently activated cells to produce the later window of past-dominance. We performed optogenetic perturbations to demonstrate that this input is provided by medial entorhinal cortex, revealing its role in maintaining a memory of past experience that biases hippocampal replay. Together, these data provide specific evidence for how hippocampal replays are generated.

Sleep loss diminishes hippocampal reactivation and replay

Memories benefit from sleep1, and the reactivation and replay of waking experiences during hippocampal sharp-wave ripples (SWRs) are considered to be crucial for this process2. However, little is known about how these patterns are impacted by sleep loss. Here we recorded CA1 neuronal activity over 12 h in rats across maze exploration, sleep and sleep deprivation, followed by recovery sleep. We found that SWRs showed sustained or higher rates during sleep deprivation but with lower power and higher frequency ripples. Pyramidal cells exhibited sustained firing during sleep deprivation and reduced firing during sleep, yet their firing rates were comparable during SWRs regardless of sleep state. Despite the robust firing and abundance of SWRs during sleep deprivation, we found that the reactivation and replay of neuronal firing patterns was diminished during these periods and, in some cases, completely abolished compared to ad libitum sleep. Reactivation partially rebounded after recovery sleep but failed to reach the levels found in natural sleep. These results delineate the adverse consequences of sleep loss on hippocampal function at the network level and reveal a dissociation between the many SWRs elicited during sleep deprivation and the few reactivations and replays that occur during these events.

Retuning of hippocampal representations during sleep

Hippocampal representations that underlie spatial memory undergo continuous refinement following formation1. Here, to track the spatial tuning of neurons dynamically during offline states, we used a new Bayesian learning approach based on the spike-triggered average decoded position in ensemble recordings from freely moving rats. Measuring these tunings, we found spatial representations within hippocampal sharp-wave ripples that were stable for hours during sleep and were strongly aligned with place fields initially observed during maze exploration. These representations were explained by a combination of factors that included preconfigured structure before maze exposure and representations that emerged during θ-oscillations and awake sharp-wave ripples while on the maze, revealing the contribution of these events in forming ensembles. Strikingly, the ripple representations during sleep predicted the future place fields of neurons during re-exposure to the maze, even when those fields deviated from previous place preferences. By contrast, we observed tunings with poor alignment to maze place fields during sleep and rest before maze exposure and in the later stages of sleep. In sum, the new decoding approach allowed us to infer and characterize the stability and retuning of place fields during offline periods, revealing the rapid emergence of representations following new exploration and the role of sleep in the representational dynamics of the hippocampus.

Efficient, continual, and generalized learning in the brain - neural mechanism of Mental Schema 2.0

There has been tremendous progress in artificial neural networks (ANNs) over the past decade; however, the gap between ANNs and the biological brain as a learning device remains large. With the goal of closing this gap, this paper reviews learning mechanisms in the brain by focusing on three important issues in ANN research: efficiency, continuity, and generalization. We first discuss the method by which the brain utilizes a variety of self-organizing mechanisms to maximize learning efficiency, with a focus on the role of spontaneous activity of the brain in shaping synaptic connections to facilitate spatiotemporal learning and numerical processing. Then, we examined the neuronal mechanisms that enable lifelong continual learning, with a focus on memory replay during sleep and its implementation in brain-inspired ANNs. Finally, we explored the method by which the brain generalizes learned knowledge in new situations, particularly from the mathematical generalization perspective of topology. Besides a systematic comparison in learning mechanisms between the brain and ANNs, we propose "Mental Schema 2.0," a new computational property underlying the brain's unique learning ability that can be implemented in ANNs.

Hippocampal memory reactivation during sleep is correlated with specific cortical states of the retrosplenial and prefrontal cortices

Episodic memories are thought to be stabilized through the coordination of cortico-hippocampal activity during sleep. However, the timing and mechanism of this coordination remain unknown. To investigate this, we studied the relationship between hippocampal reactivation and slow-wave sleep up and down states of the retrosplenial cortex (RTC) and prefrontal cortex (PFC). We found that hippocampal reactivations are strongly correlated with specific cortical states. Reactivation occurred during sustained cortical Up states or during the transition from up to down state. Interestingly, the most prevalent interaction with memory reactivation in the hippocampus occurred during sustained up states of the PFC and RTC, while hippocampal reactivation and cortical up-to-down state transition in the RTC showed the strongest coordination. Reactivation usually occurred within 150-200 msec of a cortical Up state onset, indicating that a buildup of excitation during cortical Up state activity influences the probability of memory reactivation in CA1. Conversely, CA1 reactivation occurred 30-50 msec before the onset of a cortical down state, suggesting that memory reactivation affects down state initiation in the RTC and PFC, but the effect in the RTC was more robust. Our findings provide evidence that supports and highlights the complexity of bidirectional communication between cortical regions and the hippocampus during sleep.

Enhanced Reactivation of Remapping Place Cells during Aversive Learning

Study of the hippocampal place cell system has greatly enhanced our understanding of memory encoding for distinct places, but how episodic memories for distinct experiences occurring within familiar environments are encoded is less clear. We developed a spatial decision-making task in which male rats learned to navigate a multiarm maze to a goal location for food reward while avoiding maze arms in which aversive stimuli were delivered. Task learning induced partial remapping in CA1 place cells, allowing us to identify both remapping and stable cell populations. Remapping cells were recruited into sharp-wave ripples and associated replay events to a greater extent than stable cells, despite having similar firing rates during navigation of the maze. Our results suggest that recruitment into replay events may be a mechanism to incorporate new contextual information into a previously formed and stabilized spatial representation.SIGNIFICANCE STATEMENT Hippocampal place cells provide a map of space that animals use to navigate. This map can change to reflect changes in the physical properties of the environment in which the animal finds itself, and also in response to nonphysical contextual changes, such as changes in the valence of specific locations within that environment. We show here that cells which change their spatial tuning after a change in context are preferentially recruited into sharp-wave ripple-associated replay events compared with stable nonremapping cells. Thus, our data lend strong support to the hypothesis that replay is a mechanism for the storage of new spatial maps.

Memory reactivations during sleep: a neural basis of dream experiences?

Newly encoded memory traces are spontaneously reactivated during sleep. Since their discovery in the 1990s, these memory reactivations have been discussed as a potential neural basis for dream experiences. New results from animal and human research, as well as from the rapidly growing field of sleep and dream engineering, provide essential insights into this question, and reveal both strong parallels and disparities between the two phenomena. We suggest that, although memory reactivations may contribute to subjective experiences across different states of consciousness, they are not likely to be the primary neural basis of dreaming. We identify important limitations in current research paradigms and suggest novel strategies to address this question empirically.

How our understanding of memory replay evolves

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

Sleep loss diminishes hippocampal reactivation and replay

Memories benefit from sleep, and sleep loss immediately following learning has a negative impact on subsequent memory storage. Several prominent hypotheses ascribe a central role to hippocampal sharp-wave ripples (SWRs), and the concurrent reactivation and replay of neuronal patterns from waking experience, in the offline memory consolidation process that occurs during sleep. However, little is known about how SWRs, reactivation, and replay are affected when animals are subjected to sleep deprivation. We performed long duration (~12 h), high-density silicon probe recordings from rat hippocampal CA1 neurons, in animals that were either sleeping or sleep deprived following exposure to a novel maze environment. We found that SWRs showed a sustained rate of activity during sleep deprivation, similar to or higher than in natural sleep, but with decreased amplitudes for the sharp-waves combined with higher frequencies for the ripples. Furthermore, while hippocampal pyramidal cells showed a log-normal distribution of firing rates during sleep, these distributions were negatively skewed with a higher mean firing rate in both pyramidal cells and interneurons during sleep deprivation. During SWRs, however, firing rates were remarkably similar between both groups. Despite the abundant quantity of SWRs and the robust firing activity during these events in both groups, we found that reactivation of neurons was either completely abolished or significantly diminished during sleep deprivation compared to sleep. Interestingly, reactivation partially rebounded upon recovery sleep, but failed to reach the levels characteristic of natural sleep. Similarly, the number of replays were significantly lower during sleep deprivation and recovery sleep compared to natural sleep. These results provide a network-level account for the negative impact of sleep loss on hippocampal function and demonstrate that sleep loss impacts memory storage by causing a dissociation between the amount of SWRs and the replays and reactivations that take place during these events.

Neural ensembles in navigation: From single cells to population codes

The brain can represent behaviorally relevant information through the firing of individual neurons as well as the coordinated firing of ensembles of neurons. Neurons in the hippocampus and associated cortical regions participate in a variety of types of ensembles to support navigation. These ensemble types include single cell codes, population codes, time-compressed sequences, behavioral sequences, and engrams. We present the physiological basis and behavioral relevance of ensemble firing. We discuss how these traditional definitions of ensembles can constrain or expand potential analyses due to the underlying assumptions and abstractions made. We highlight how coding can change at the ensemble level while underlying single cell codes remain intact. Finally, we present how ensemble definitions could be broadened to better understand the full complexity of the brain.

Imagination as a fundamental function of the hippocampus

Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

A large majority of awake hippocampal sharp-wave ripples feature spatial trajectories with momentum

During periods of rest, hippocampal place cells feature bursts of activity called sharp-wave ripples (SWRs). Heuristic approaches have revealed that a small fraction of SWRs appear to "simulate" trajectories through the environment, called awake hippocampal replay. However, the functional role of a majority of these SWRs remains unclear. We find, using Bayesian model comparison of state-space models to characterize the spatiotemporal dynamics embedded in SWRs, that almost all SWRs of foraging rodents simulate such trajectories. Furthermore, these trajectories feature momentum, or inertia in their velocities, that mirrors the animals' natural movement, in contrast to replay events during sleep, which lack such momentum. Last, we show that past analyses of replayed trajectories for navigational planning were biased by the heuristic SWR sub-selection. Our findings thus identify the dominant function of awake SWRs as simulating trajectories with momentum and provide a principled foundation for future work on their computational function.

CA2 inhibition reduces the precision of hippocampal assembly reactivation

The structured reactivation of hippocampal neuronal ensembles during fast synchronous oscillatory events, termed sharp-wave ripples (SWRs), has been suggested to play a crucial role in the storage and use of memory. Activity in both the CA2 and CA3 subregions can precede this population activity in CA1, and chronic inhibition of either region alters SWR oscillations. However, the precise contribution of CA2 to the oscillation, as well as to the reactivation of CA1 neurons within it, remains unclear. Here, we employ chemogenetics to transiently silence CA2 pyramidal cells in mice, and we observe that although SWRs still occur, the reactivation of CA1 pyramidal cell ensembles within the events lose both temporal and informational precision. These observations suggest that CA2 activity contributes to the fidelity of experience-dependent hippocampal replay.

Brain neural patterns and the memory function of sleep

Sleep is crucial for healthy cognition, including memory. The two main phases of sleep, REM (rapid eye movement) and non-REM sleep, are associated with characteristic electrophysiological patterns that are recorded using surface and intracranial electrodes. These patterns include sharp-wave ripples, cortical slow oscillations, delta waves, and spindles during non-REM sleep and theta oscillations during REM sleep. They reflect the precisely timed activity of underlying neural circuits. Here, we review how these electrical signatures have been guiding our understanding of the circuits and processes sustaining memory consolidation during sleep, focusing on hippocampal theta oscillations and sharp-wave ripples and how they coordinate with cortical patterns. Finally, we highlight how these brain patterns could also sustain sleep-dependent homeostatic processes and evoke several potential future directions for research on the memory function of sleep.

Replay in Deep Learning: Current Approaches and Missing Biological Elements

Replay is the reactivation of one or more neural patterns that are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated in deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this letter, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be used to improve artificial neural networks.

Hippocampal replay of experience at real-world speeds

Representations related to past experiences play a critical role in memory and decision-making processes. The rat hippocampus expresses these types of representations during sharp-wave ripple (SWR) events, and previous work identified a minority of SWRs that contain ‘replay’ of spatial trajectories at ∼20x the movement speed of the animal. Efforts to understand replay typically make multiple assumptions about which events to examine and what sorts of representations constitute replay. We therefore lack a clear understanding of both the prevalence and the range of representational dynamics associated with replay. Here, we develop a state space model that uses a combination of movement dynamics of different speeds to capture the spatial content and time evolution of replay during SWRs. Using this model, we find that the large majority of replay events contain spatially coherent, interpretable content. Furthermore, many events progress at real-world, rather than accelerated, movement speeds, consistent with actual experiences.

Aperiodic sleep networks promote memory consolidation

Hierarchical synchronization of sleep oscillations establishes communication pathways to support memory reactivation, transfer, and consolidation. From an information-theoretical perspective, oscillations constitute highly structured network states that provide limited information-coding capacity. Recent findings indicate that sleep oscillations occur in transient bursts that are interleaved with aperiodic network states, which were previously considered to be random noise. We argue that aperiodic activity exhibits unique and variable spatiotemporal patterns, providing an ideal information-rich neurophysiological substrate for imprinting new mnemonic patterns onto existing circuits. We discuss novel avenues in conceptualizing and quantifying aperiodic network states during sleep to further understand their relevance and interplay with sleep oscillations in support of memory consolidation.

Temporally delayed linear modelling (TDLM) measures replay in both animals and humans

There are rich structures in off-task neural activity which are hypothesized to reflect fundamental computations across a broad spectrum of cognitive functions. Here, we develop an analysis toolkit - temporal delayed linear modelling (TDLM) - for analysing such activity. TDLM is a domain-general method for finding neural sequences that respect a pre-specified transition graph. It combines nonlinear classification and linear temporal modelling to test for statistical regularities in sequences of task-related reactivations. TDLM is developed on the non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. Notably, as a linear framework, TDLM can be easily extended, without loss of generality, to capture rodent replay in electrophysiology, including in continuous spaces, as well as addressing second-order inference questions, for example, its temporal and spatial varying pattern. We hope TDLM will advance a deeper understanding of neural computation and promote a richer convergence between animal and human neuroscience.

The evolving view of replay and its functions in wake and sleep

The term hippocampal replay originally referred to the temporally compressed reinstantiation, during rest, of sequential neural activity observed during prior active wake. Since its description in the 1990s, hippocampal replay has often been viewed as the key mechanism by which a memory trace is repeatedly rehearsed at high speeds during sleep and gradually transferred to neocortical circuits. However, the methods used to measure the occurrence of replay remain debated, and it is now clear that the underlying neural events are considerably more complicated than the traditional narratives had suggested. “Replay-like” activity happens during wake, can play out in reverse order, may represent trajectories never taken by the animal, and may have additional functions beyond memory consolidation, from learning values and solving the problem of credit assignment to decision-making and planning. Still, we know little about the role of replay in cognition, and to what extent it differs between wake and sleep. This may soon change, however, because decades-long efforts to explain replay in terms of reinforcement learning (RL) have started to yield testable predictions and possible explanations for a diverse set of observations. Here, we (1) survey the diverse features of replay, focusing especially on the latest findings; (2) discuss recent attempts at unifying disparate experimental results and putatively different cognitive functions under the banner of RL; (3) discuss methodological issues and theoretical biases that impede progress or may warrant a partial revaluation of the current literature, and finally; (4) highlight areas of considerable uncertainty and promising avenues of inquiry.

Point process models for sequence detection in high-dimensional neural spike trains

Sparse sequences of neural spikes are posited to underlie aspects of working memory [1], motor production [2], and learning [3, 4]. Discovering these sequences in an unsupervised manner is a longstanding problem in statistical neuroscience [5-7]. Promising recent work [4, 8] utilized a convolutive nonnegative matrix factorization model [9] to tackle this challenge. However, this model requires spike times to be discretized, utilizes a sub-optimal least-squares criterion, and does not provide uncertainty estimates for model predictions or estimated parameters. We address each of these shortcomings by developing a point process model that characterizes fine-scale sequences at the level of individual spikes and represents sequence occurrences as a small number of marked events in continuous time. This ultra-sparse representation of sequence events opens new possibilities for spike train modeling. For example, we introduce learnable time warping parameters to model sequences of varying duration, which have been experimentally observed in neural circuits [10]. We demonstrate these advantages on experimental recordings from songbird higher vocal center and rodent hippocampus.

A mechanism for learning with sleep spindles

Spindles are ubiquitous oscillations during non-rapid eye movement (NREM) sleep. A growing body of evidence points to a possible link with learning and memory, and the underlying mechanisms are now starting to be unveiled. Specifically, spindles are associated with increased dendritic activity and high intracellular calcium levels, a situation favourable to plasticity, as well as with control of spiking output by feed-forward inhibition. During spindles, thalamocortical networks become unresponsive to inputs, thus potentially preventing interference between memory-related internal information processing and extrinsic signals. At the system level, spindles are co-modulated with other major NREM oscillations, including hippocampal sharp wave-ripples (SWRs) and neocortical slow waves, both previously shown to be associated with learning and memory. The sequential occurrence of reactivation at the time of SWRs followed by neuronal plasticity-promoting spindles is a possible mechanism to explain NREM sleep-dependent consolidation of memories. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Memories replayed: reactivating past successes and new dilemmas

Our experiences continue to be processed 'offline' in the ensuing hours of both wakefulness and sleep. During these different brain states, the memory formed during our experience is replayed or reactivated. Here, we discuss the unique challenges in studying offline reactivation, the growth in both the experimental and analytical techniques available across different animals from rodents to humans to capture these offline events, the important challenges this innovation has brought, our still modest understanding of how reactivation drives diverse synaptic changes across circuits, and how these changes differ (if at all), and perhaps complement, those at memory formation. Together, these discussions highlight critical emerging issues vital for identifying how reactivation affects circuits, and, in turn, behaviour, and provides a broader context for the contributions in this special issue. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Progress and issues in second-order analysis of hippocampal replay

Patterns of neural activity that occur spontaneously during sharp-wave ripple (SWR) events in the hippocampus are thought to play an important role in memory formation, consolidation and retrieval. Typical studies examining the content of SWRs seek to determine whether the identity and/or temporal order of cell firing is different from chance. Such 'first-order' analyses are focused on a single time point and template (map), and have been used to show, for instance, the existence of preplay. The major methodological challenge in first-order analyses is the construction and interpretation of different chance distributions. By contrast, 'second-order' analyses involve a comparison of SWR content between different time points, and/or between different templates. Typical second-order questions include tests of experience-dependence (replay) that compare SWR content before and after experience, and comparisons or replay between different arms of a maze. Such questions entail additional methodological challenges that can lead to biases in results and associated interpretations. We provide an inventory of analysis challenges for second-order questions about SWR content, and suggest ways of preventing, identifying and addressing possible analysis biases. Given evolving interest in understanding SWR content in more complex experimental scenarios and across different time scales, we expect these issues to become increasingly pervasive. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.