Real-time classification of experience-related ensemble spiking patterns for closed-loop applications

How our understanding of memory replay evolves

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

Disrupting ripples: Methods, results, and caveats in closed-loop approaches in rodents

Hippocampal ripple oscillations have been associated with memory reactivations during wake and sleep. These reactivations should contribute to working memory and memory consolidation respectively. In the past decade studies have moved from being observational to actively disrupting ripple-related activity in closed-loop approaches to enable causal investigations into their function. All together these studies have been able to provide evidence that wake, task-related ripple activity is important for working memory and planning but less important for stabilisation of spatial representations. Rest and sleep-related ripple activity, in contrast, is important for long-term memory performance and thus memory consolidation. In this review, we summarise results from different closed-loop approaches in rodents. Further, we highlight differences in detection and stimulation methods as well as controls and discuss how these differences could influence outcomes.

High-resolution optogenetics in space and time

To understand the neural mechanisms of behavior, it is necessary to both monitor and perturb the activity of ensembles of neurons with high specificity. While neural ensemble recordings have been available for decades, progress in high-resolution manipulation techniques has lagged behind. Optogenetics has enabled the manipulation of genetically defined cell types in behaving animals, and recent developments, including multipoint nanofabricated light sources, provide spatiotemporal resolution on a par with that of physiological recordings. Here we review current advances in optogenetic methods for cellular-resolution stimulation and intervention, as well as their integration with real-time neural recordings for closed-loop experimentation. We discuss how these approaches open the door to new kinds of experiments aimed at dissecting the role of specific neural patterns and discrete cellular populations in orchestrating the activity of brain circuits that support behavior and cognition.

Uncovering spatial representations from spatiotemporal patterns of rodent hippocampal field potentials

Spatiotemporal patterns of large-scale spiking and field potentials of the rodent hippocampus encode spatial representations during maze runs, immobility, and sleep. Here, we show that multisite hippocampal field potential amplitude at ultra-high-frequency band (FPAuhf), a generalized form of multiunit activity, provides not only a fast and reliable reconstruction of the rodent's position when awake, but also a readout of replay content during sharp-wave ripples. This FPAuhf feature may serve as a robust real-time decoding strategy from large-scale recordings in closed-loop experiments. Furthermore, we develop unsupervised learning approaches to extract low-dimensional spatiotemporal FPAuhf features during run and ripple periods and to infer latent dynamical structures from lower-rank FPAuhf features. We also develop an optical flow-based method to identify propagating spatiotemporal LFP patterns from multisite array recordings, which can be used as a decoding application. Finally, we develop a prospective decoding strategy to predict an animal's future decision in goal-directed navigation.

Open-Source Software for Real-time Calcium Imaging and Synchronized Neuron Firing Detection

We developed Carignan, a real-time calcium imaging software that can automatically detect activity patterns of neurons. Carignan can activate an external device when synchronized neural activity is detected in calcium imaging obtained by a one-photon (1p) miniscope. Combined with optogenetics, our software enables closed-loop experiments for investigating functions of specific types of neurons in the brain. In addition to making existing pattern detection algorithms run in real-time seamlessly, we developed a new classification module that distinguishes neurons from false-positives using deep learning. We used a combination of convolutional and recurrent neural networks to incorporate both spatial and temporal features in activity patterns. Our method performed better than existing neuron detection methods for false-positive neuron detection in terms of the F1 score. Using Carignan, experimenters can activate or suppress a group of neurons when specific neural activity is observed. Because the system uses a 1p miniscope, it can be used on the brain of a freely-moving animal, making it applicable to a wide range of experimental paradigms.

Single-trial dynamics of hippocampal spatial representations are modulated by reward value

Reward value is known to modulate learning speed in spatial memory tasks, but little is known about its influence on the dynamical changes in hippocampal spatial representations. Here, we monitored the trial-to-trial changes in hippocampal place cell activity during the acquisition of place-reward associations with varying reward size. We show a faster reorganization and stabilization of the hippocampal place map when a goal location is associated with a large reward. The reorganization is driven by both rate changes and the appearance and disappearance of place fields. The occurrence of hippocampal replay activity largely followed the dynamics of changes in spatial representations. Replay patterns became more selectively tuned toward behaviorally relevant experiences over the course of learning via the refined contributions of specific cell subpopulations. These results suggest that high reward value enhances memory retention by accelerating the formation and stabilization of the hippocampal cognitive map and selectively enhancing its reactivation during learning.

Improving scalability in systems neuroscience

Emerging technologies to acquire data at increasingly greater scales promise to transform discovery in systems neuroscience. However, current exponential growth in the scale of data acquisition is a double-edged sword. Scaling up data acquisition can speed up the cycle of discovery but can also misinterpret the results or possibly slow down the cycle because of challenges presented by the curse of high-dimensional data. Active, adaptive, closed-loop experimental paradigms use hardware and algorithms optimized to enable time-critical computation to provide feedback that interprets the observations and tests hypotheses to actively update the stimulus or stimulation parameters. In this perspective, we review important concepts of active and adaptive experiments and discuss how selectively constraining the dimensionality and optimizing strategies at different stages of discovery loop can help mitigate the curse of high-dimensional data. Active and adaptive closed-loop experimental paradigms can speed up discovery despite an exponentially increasing data scale, offering a road map to timely and iterative hypothesis revision and discovery in an era of exponential growth in neuroscience.

The evolving view of replay and its functions in wake and sleep

The term hippocampal replay originally referred to the temporally compressed reinstantiation, during rest, of sequential neural activity observed during prior active wake. Since its description in the 1990s, hippocampal replay has often been viewed as the key mechanism by which a memory trace is repeatedly rehearsed at high speeds during sleep and gradually transferred to neocortical circuits. However, the methods used to measure the occurrence of replay remain debated, and it is now clear that the underlying neural events are considerably more complicated than the traditional narratives had suggested. “Replay-like” activity happens during wake, can play out in reverse order, may represent trajectories never taken by the animal, and may have additional functions beyond memory consolidation, from learning values and solving the problem of credit assignment to decision-making and planning. Still, we know little about the role of replay in cognition, and to what extent it differs between wake and sleep. This may soon change, however, because decades-long efforts to explain replay in terms of reinforcement learning (RL) have started to yield testable predictions and possible explanations for a diverse set of observations. Here, we (1) survey the diverse features of replay, focusing especially on the latest findings; (2) discuss recent attempts at unifying disparate experimental results and putatively different cognitive functions under the banner of RL; (3) discuss methodological issues and theoretical biases that impede progress or may warrant a partial revaluation of the current literature, and finally; (4) highlight areas of considerable uncertainty and promising avenues of inquiry.

Efficient Position Decoding Methods Based on Fluorescence Calcium Imaging in the Mouse Hippocampus

Large-scale fluorescence calcium imaging methods have become widely adopted for studies of long-term hippocampal and cortical neuronal dynamics. Pyramidal neurons of the rodent hippocampus show spatial tuning in freely foraging or head-fixed navigation tasks. Development of efficient neural decoding methods for reconstructing the animal's position in real or virtual environments can provide a fast readout of spatial representations in closed-loop neuroscience experiments. Here, we develop an efficient strategy to extract features from fluorescence calcium imaging traces and further decode the animal's position. We validate our spike inference-free decoding methods in multiple in vivo calcium imaging recordings of the mouse hippocampus based on both supervised and unsupervised decoding analyses. We systematically investigate the decoding performance of our proposed methods with respect to the number of neurons, imaging frame rate, and signal-to-noise ratio. Our proposed supervised decoding analysis is ultrafast and robust, and thereby appealing for real-time position decoding applications based on calcium imaging.

A Systematic Review of Closed-Loop Feedback Techniques in Sleep Studies-Related Issues and Future Directions

Advances in computer processing technology have enabled researchers to analyze real-time brain activity and build real-time closed-loop paradigms. In many fields, the effectiveness of these closed-loop protocols has proven to be better than that of the simple open-loop paradigms. Recently, sleep studies have attracted much attention as one possible application of closed-loop paradigms. To date, several studies that used closed-loop paradigms have been reported in the sleep-related literature and recommend a closed-loop feedback system to enhance specific brain activity during sleep, which leads to improvements in sleep's effects, such as memory consolidation. However, to the best of our knowledge, no report has reviewed and discussed the detailed technical issues that arise in designing sleep closed-loop paradigms. In this paper, we reviewed the most recent reports on sleep closed-loop paradigms and offered an in-depth discussion of some of their technical issues. We found 148 journal articles strongly related with 'sleep and stimulation' and reviewed 20 articles on closed-loop feedback sleep studies. We focused on human sleep studies conducting any modality of feedback stimulation. Then we introduced the main component of the closed-loop system and summarized several open-source libraries, which are widely used in closed-loop systems, with step-by-step guidelines for closed-loop system implementation for sleep. Further, we proposed future directions for sleep research with closed-loop feedback systems, which provide some insight into closed-loop feedback systems.

Five Decades of Hippocampal Place Cells and EEG Rhythms in Behaving Rats

Over the last 50 years, much has been learned about the physiology and functions of the hippocampus from studies in freely behaving rats. Two relatively early works in the field provided major insights that remain relevant today. Here, I revisit these studies and discuss how our understanding of the hippocampus has evolved over the last several decades.

Sub-second dynamics of theta-gamma coupling in hippocampal CA1

Oscillatory brain activity reflects different internal brain states including neurons’ excitatory state and synchrony among neurons. However, characterizing these states is complicated by the fact that different oscillations are often coupled, such as gamma oscillations nested in theta in the hippocampus, and changes in coupling are thought to reflect distinct states. Here, we describe a new method to separate single oscillatory cycles into distinct states based on frequency and phase coupling. Using this method, we identified four theta-gamma coupling states in rat hippocampal CA1. These states differed in abundance across behaviors, phase synchrony with other hippocampal subregions, and neural coding properties suggesting that these states are functionally distinct. We captured cycle-to-cycle changes in oscillatory coupling states and found frequent switching between theta-gamma states showing that the hippocampus rapidly shifts between different functional states. This method provides a new approach to investigate oscillatory brain dynamics broadly.

Validating the theoretical bases of sleep reactivation during sharp-wave ripples and their association with emotional valence

Sleep is important for memory consolidation, and an abundant literature suggests that reactivation in the hippocampus during sleep is instrumental to this process. Yet, the current interpretation of activity during sharp-waves ripples (SWRs), as replay of wake experiences, relies on hypotheses that, while widely accepted, have only recently begun to be tested directly. Moreover, this theory has been mainly studied in the context of pure spatial learning, and it is still not clear how emotional valence can fit into this conceptual framework when considering reward- or punishment-based learning. In this review, we will present recent experimental arguments validating the interpretation of sleep replay as reactivation of awake experiences and examine the evidence showing that the emotional valence is also replayed during sleep in a coordinated fashion with hippocampal SWRs. Finally, we will detail recent experiments showing that brain-computer interfaces can be used to modify the emotional valence associated with sleep replay.

Myopic control of neural dynamics

Manipulating the dynamics of neural systems through targeted stimulation is a frontier of research and clinical neuroscience; however, the control schemes considered for neural systems are mismatched for the unique needs of manipulating neural dynamics. An appropriate control method should respect the variability in neural systems, incorporating moment to moment “input” to the neural dynamics and behaving based on the current neural state, irrespective of the past trajectory. We propose such a controller under a nonlinear state-space feedback framework that steers one dynamical system to function as through it were another dynamical system entirely. This “myopic” controller is formulated through a novel variant of a model reference control cost that manipulates dynamics in a short-sighted manner that only sets a target trajectory of a single time step into the future (hence its myopic nature), which omits the need to pre-calculate a rigid and computationally costly neural feedback control solution. To demonstrate the breadth of this control’s utility, two examples with distinctly different applications in neuroscience are studied. First, we show the myopic control’s utility to probe the causal link between dynamics and behavior for cognitive processes by transforming a winner-take-all decision-making system to operate as a robust neural integrator of evidence. Second, an unhealthy motor-like system containing an unwanted beta-oscillation spiral attractor is controlled to function as a healthy motor system, a relevant clinical example for neurological disorders.

Stimulating a neural system and observing its effect through simultaneous observation offers the promise to better understand how neural systems perform computations, as well as for the treatment of neurological disorders. A powerful perspective for understanding a neural system’s behavior undergoing stimulation is to conceptualize them as dynamical systems, which considers the global effect that stimulation has on the brain, rather than only assessing what impact it has on the recorded signal from the brain. With this more comprehensive perspective comes a central challenge of determining what requirements need to be satisfied to harness neural observations and then stimulate to make one dynamical system function as another one entirely. This could lead to applications such as neural stimulators that make a diseased brain behave like its healthy counterpart, or to make a neural system previously capable of only hasty decision making to wait and accumulate more evidence for a more informed decision. In this work we explore the implications of this new perspective on neural stimulation and derive a simple prescription for using neural observations to inform stimulation protocol that makes one neural system behave like another one.

Real-Time Readout of Large-Scale Unsorted Neural Ensemble Place Codes

Uncovering spatial representations from large-scale ensemble spike activity in specific brain circuits provides valuable feedback in closed-loop experiments. We develop a graphics processing unit (GPU)-powered population-decoding system for ultrafast reconstruction of spatial positions from rodents' unsorted spatiotemporal spiking patterns, during run behavior or sleep. In comparison with an optimized quad-core central processing unit (CPU) implementation, our approach achieves an ∼20- to 50-fold increase in speed in eight tested rat hippocampal, cortical, and thalamic ensemble recordings, with real-time decoding speed (approximately fraction of a millisecond per spike) and scalability up to thousands of channels. By accommodating parallel shuffling in real time (computation time <15 ms), our approach enables assessment of the statistical significance of online-decoded "memory replay" candidates during quiet wakefulness or sleep. This open-source software toolkit supports the decoding of spatial correlates or content-triggered experimental manipulation in closed-loop neuroscience experiments.