Neural reactivations during sleep determine network credit assignment

Desiderata for Normative Models of Synaptic Plasticity

Normative models of synaptic plasticity use computational rationales to arrive at predictions of behavioral and network-level adaptive phenomena. In recent years, there has been an explosion of theoretical work in this realm, but experimental confirmation remains limited. In this review, we organize work on normative plasticity models in terms of a set of desiderata that, when satisfied, are designed to ensure that a given model demonstrates a clear link between plasticity and adaptive behavior, is consistent with known biological evidence about neural plasticity and yields specific testable predictions. As a prototype, we include a detailed analysis of the REINFORCE algorithm. We also discuss how new models have begun to improve on the identified criteria and suggest avenues for further development. Overall, we provide a conceptual guide to help develop neural learning theories that are precise, powerful, and experimentally testable.

Modulation of neural spiking in motor cortex-cerebellar networks during sleep spindles

Sleep spindles appear to play an important role in learning new motor skills. Motor skill learning engages several regions in the brain with two important areas being the motor cortex (M1) and the cerebellum. However, the neurophysiological processes in these areas during sleep, especially how spindle oscillations affect local and cross-region spiking, are not fully understood. We recorded activity from the M1 and cerebellar cortex in 8 rats during spontaneous activity to investigate how sleep spindles in these regions are related to local spiking as well as cross-region spiking. We found that M1 firing was significantly changed during both M1 and cerebellum spindles and this spiking occurred at a preferred phase of the spindle. On average, M1 and cerebellum neurons showed most spiking at the M1 or cerebellum spindle peaks. These neurons also developed a preferential phase-locking to local or cross-area spindles with the greatest phase-locking value at spindle peaks; however, this preferential phase-locking wasn't significant for cerebellar neurons when compared to cerebellum spindles. Additionally, we found the percentage of task-modulated cells in the M1 and cerebellum that fired with non-uniform spike-phase distribution during M1/ cerebellum spindle peaks were greater in the rats that learned a reach-to-grasp motor task robustly. Finally, we found that spindle-band LFP coherence (for M1 and cerebellum LFPs) showed a positive correlation with success rate in the motor task. These findings support the idea that sleep spindles in both the M1 and cerebellum recruit neurons that participate in the awake task to support motor memory consolidation.Significance Statement Neural processing during sleep spindles is linked to memory consolidation. However, little is known about sleep activity in the cerebellum and whether cerebellum spindles can affect spiking activity in local or distant areas. We report the effect of sleep spindles on neuron activity in the M1 and cerebellum-specifically their firing rate and phase-locking to spindle oscillations. Our results indicate that awake practice neuronal activity is tempered during local M1 and cerebellum spindles, and during cross-region spindles, which may support motor skill learning. We describe spiking dynamics in motor networks spindle oscillations that may aid in the learning of skills. Our results support the sleep reactivation hypothesis and suggest that awake M1 activity may be reactivated during cerebellum spindles.

Cortico-cerebellar coordination facilitates neuroprosthetic control

Temporally coordinated neural activity is central to nervous system function and purposeful behavior. Still, there is a paucity of evidence demonstrating how this coordinated activity within cortical and subcortical regions governs behavior. We investigated this between the primary motor (M1) and contralateral cerebellar cortex as rats learned a neuroprosthetic/brain-machine interface (BMI) task. In neuroprosthetic task, actuator movements are causally linked to M1 "direct" neurons that drive the decoder for successful task execution. However, it is unknown how task-related M1 activity interacts with the cerebellum. We observed a notable 3 to 6 hertz coherence that emerged between these regions' local field potentials (LFPs) with learning that also modulated task-related spiking. We identified robust task-related indirect modulation in the cerebellum, which developed a preferential relationship with M1 task-related activity. Inhibiting cerebellar cortical and deep nuclei activity through optogenetics led to performance impairments in M1-driven neuroprosthetic control. Together, these results demonstrate that cerebellar influence is necessary for M1-driven neuroprosthetic control.

Spindle oscillations in communicating axons within a reconstituted hippocampal formation are strongest in CA3 without thalamus

Spindle-shaped waves of oscillations emerge in EEG scalp recordings during human and rodent non-REM sleep. The association of these 10-16 Hz oscillations with events during prior wakefulness suggests a role in memory consolidation. Human and rodent depth electrodes in the brain record strong spindles throughout the cortex and hippocampus, with possible origins in the thalamus. However, the source and targets of the spindle oscillations from the hippocampus are unclear. Here, we employed an in vitro reconstruction of four subregions of the hippocampal formation with separate microfluidic tunnels for single axon communication between subregions assembled on top of a microelectrode array. We recorded spontaneous 400-1000 ms long spindle waves at 10-16 Hz in single axons passing between subregions as well as from individual neurons in those subregions. Spindles were nested within slow waves. The highest amplitudes and most frequent occurrence suggest origins in CA3 neurons that send feed-forward axons into CA1 and feedback axons into DG. Spindles had 50-70% slower conduction velocities than spikes and were not phase-locked to spikes suggesting that spindle mechanisms are independent of action potentials. Therefore, consolidation of declarative-cognitive memories in the hippocampus may be separate from the more easily accessible consolidation of memories related to thalamic motor function.

Learning leaves a memory trace in motor cortex

How are we able to learn new behaviors without disrupting previously learned ones? To understand how the brain achieves this, we used a brain-computer interface (BCI) learning paradigm, which enables us to detect the presence of a memory of one behavior while performing another. We found that learning to use a new BCI map altered the neural activity that monkeys produced when they returned to using a familiar BCI map in a way that was specific to the learning experience. That is, learning left a "memory trace" in the primary motor cortex. This memory trace coexisted with proficient performance under the familiar map, primarily by altering neural activity in dimensions that did not impact behavior. Forming memory traces might be how the brain is able to provide for the joint learning of multiple behaviors without interference.

Assistive sensory-motor perturbations influence learned neural representations

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Population dimensionality remained constant or increased with learning, counter to trends with non-adaptive BCIs. Yet, over time, task information was contained in a smaller subset of neurons or population modes. Moreover, task information was ultimately stored in neural modes that occupied a small fraction of the population variance. An artificial neural network model suggests the adaptive decoders contribute to forming these compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations, like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

Computational role of sleep in memory reorganization

Sleep is considered to play an essential role in memory reorganization. Despite its importance, classical theoretical models did not focus on some sleep characteristics. Here, we review recent theoretical approaches investigating their roles in learning and discuss the possibility that non-rapid eye movement (NREM) sleep selectively consolidates memory, and rapid eye movement (REM) sleep reorganizes the representations of memories. We first review the possibility that slow waves during NREM sleep contribute to memory selection by using sequential firing patterns and the existence of up and down states. Second, we discuss the role of dreaming during REM sleep in developing neuronal representations. We finally discuss how to develop these points further, emphasizing the connections to experimental neuroscience and machine learning.

Disturbed laterality of non-rapid eye movement sleep oscillations in post-stroke human sleep: a pilot study

Sleep is known to promote recovery post-stroke. However, there is a paucity of data profiling sleep oscillations in the post-stroke human brain. Recent rodent work showed that resurgence of physiologic spindles coupled to sleep slow oscillations (SOs) and concomitant decrease in pathological delta (δ) waves is associated with sustained motor performance gains during stroke recovery. The goal of this study was to evaluate bilaterality of non-rapid eye movement (NREM) sleep-oscillations (namely SOs, δ-waves, spindles, and their nesting) in post-stroke patients vs. healthy control subjects. We analyzed NREM-marked electroencephalography (EEG) data in hospitalized stroke-patients (n = 5) and healthy subjects (n = 3). We used a laterality index to evaluate symmetry of NREM oscillations across hemispheres. We found that stroke subjects had pronounced asymmetry in the oscillations, with a predominance of SOs, δ-waves, spindles, and nested spindles in affected hemisphere, when compared to the healthy subjects. Recent preclinical work classified SO-nested spindles as restorative post-stroke and δ-wave-nested spindles as pathological. We found that the ratio of SO-nested spindles laterality index to δ-wave-nested spindles laterality index was lower in stroke subjects. Using linear mixed models (which included random effects of concurrent pharmacologic drugs), we found large and medium effect size for δ-wave nested spindle and SO-nested spindle, respectively. Our results in this pilot study indicate that considering laterality index of NREM oscillations might be a useful metric for assessing recovery post-stroke and that factoring in pharmacologic drugs may be important when targeting sleep modulation for neurorehabilitation post-stroke.

Local field potentials and single unit dynamics in motor cortex of unconstrained macaques during different behavioral states

Different sleep stages have been shown to be vital for a variety of brain functions, including learning, memory, and skill consolidation. However, our understanding of neural dynamics during sleep and the role of prominent LFP frequency bands remain incomplete. To elucidate such dynamics and differences between behavioral states we collected multichannel LFP and spike data in primary motor cortex of unconstrained macaques for up to 24 h using a head-fixed brain-computer interface (Neurochip3). Each 8-s bin of time was classified into awake-moving (Move), awake-resting (Rest), REM sleep (REM), or non-REM sleep (NREM) by using dimensionality reduction and clustering on the average spectral density and the acceleration of the head. LFP power showed high delta during NREM, high theta during REM, and high beta when the animal was awake. Cross-frequency phase-amplitude coupling typically showed higher coupling during NREM between all pairs of frequency bands. Two notable exceptions were high delta-high gamma and theta-high gamma coupling during Move, and high theta-beta coupling during REM. Single units showed decreased firing rate during NREM, though with increased short ISIs compared to other states. Spike-LFP synchrony showed high delta synchrony during Move, and higher coupling with all other frequency bands during NREM. These results altogether reveal potential roles and functions of different LFP bands that have previously been unexplored.

Emergence of task-related spatiotemporal population dynamics in transplanted neurons

Loss of nervous system tissue after severe brain injury is a main determinant of poor functional recovery. Cell transplantation is a promising method to restore lost tissue and function, yet it remains unclear if transplanted neurons can demonstrate the population level dynamics important for movement control. Here we present a comprehensive approach for long-term single neuron monitoring and manipulation of transplanted embryonic cortical neurons after cortical injury in adult male mice performing a prehension task. The observed patterns of population activity in the transplanted network strongly resembled that of healthy networks. Specifically, the task-related spatiotemporal activity patterns of transplanted neurons could be represented by latent factors that evolve within a low dimensional manifold. We also demonstrate reliable modulation of the transplanted networks using minimally invasive epidural stimulation. Our approach may allow greater insight into how restoration of cell-type specific network dynamics in vivo can restore motor function.

[The role of sleep brain oscillations and neuronal patterns for memory]

Sleep is crucial for the selective processing and strengthening of information encoded during wakefulness, known as memory consolidation. The different phases of sleep are characterized by specific neuronal activities associated with memory consolidation and homeostatic regulation. In the hippocampus during non-REM sleep, neural patterns called sharp-wave ripple complexes are associated with reactivations of the neural activity that occurred during wakefulness. These reactivations, through their coordinations with cortical slow oscillations and thalamocortical spindles, contribute to the consolidation of spatial memories by strengthening neuronal connections. Cortical slow waves are also a marker of synaptic homeostasis, a regulatory phenomenon maintaining networks in a functional range of firing rates. Finally, REM sleep is also important for memory, although the underlying physiology and the role of theta waves deserves to be further explored.

Disturbed laterality of non-rapid eye movement sleep oscillations in post-stroke human sleep: a pilot study

Sleep is known to promote recovery post-stroke. However, there is a paucity of data profiling sleep oscillations post-stroke in the human brain. Recent rodent work showed that resurgence of physiologic spindles coupled to sleep slow oscillations(SOs) and concomitant decrease in pathological delta(δ) waves is associated with sustained motor performance gains during stroke recovery. The goal of this study was to evaluate bilaterality of non-rapid eye movement (NREM) sleep-oscillations (namely SOs, δ-waves, spindles and their nesting) in post-stroke patients versus healthy control subjects. We analyzed NREM-marked electroencephalography (EEG) data in hospitalized stroke-patients (n=5) and healthy subjects (n=3) from an open-sourced dataset. We used a laterality index to evaluate symmetry of NREM oscillations across hemispheres. We found that stroke subjects had pronounced asymmetry in the oscillations, with a predominance of SOs, δ-waves, spindles and nested spindles in one hemisphere, when compared to the healthy subjects. Recent preclinical work classified SO-nested spindles as restorative post-stroke and δ-wave-nested spindles as pathological. We found that the ratio of SO-nested spindles laterality index to δ-wave-nested spindles laterality index was lower in stroke subjects. Using linear mixed models (which included random effects of concurrent pharmacologic drugs), we found large and medium effect size for δ-wave nested spindle and SO-nested spindle, respectively. Our results indicate considering laterality index of NREM oscillations might be a useful metric for assessing recovery post-stroke and that factoring in pharmacologic drugs may be important when targeting sleep modulation for neurorehabilitation post-stroke.

Emergence of cortical network motifs for short-term memory during learning

Learning of adaptive behaviors requires the refinement of coordinated activity across multiple brain regions. However, how neural communications develop during learning remains poorly understood. Here, using two-photon calcium imaging, we simultaneously recorded the activity of layer 2/3 excitatory neurons in eight regions of the mouse dorsal cortex during learning of a delayed-response task. Across learning, while global functional connectivity became sparser, there emerged a subnetwork comprising of neurons in the anterior lateral motor cortex (ALM) and posterior parietal cortex (PPC). Neurons in this subnetwork shared a similar choice code during action preparation and formed recurrent functional connectivity across learning. Suppression of PPC activity disrupted choice selectivity in ALM and impaired task performance. Recurrent neural networks reconstructed from ALM activity revealed that PPC-ALM interactions rendered choice-related attractor dynamics more stable. Thus, learning constructs cortical network motifs by recruiting specific inter-areal communication channels to promote efficient and robust sensorimotor transformation.

Flexible regulation of representations on a drifting manifold enables long-term stable complex neuroprosthetic control

The nervous system needs to balance the stability of neural representations with plasticity. It is unclear what is the representational stability of simple actions, particularly those that are well-rehearsed in humans, and how it changes in new contexts. Using an electrocorticography brain-computer interface (BCI), we found that the mesoscale manifold and relative representational distances for a repertoire of simple imagined movements were remarkably stable. Interestingly, however, the manifold's absolute location demonstrated day-to-day drift. Strikingly, representational statistics, especially variance, could be flexibly regulated to increase discernability during BCI control without somatotopic changes. Discernability strengthened with practice and was specific to the BCI, demonstrating remarkable contextual specificity. Accounting for drift, and leveraging the flexibility of representations, allowed neuroprosthetic control of a robotic arm and hand for over 7 months without recalibration. Our study offers insight into how electrocorticography can both track representational statistics across long periods and allow long-term complex neuroprosthetic control.

Desiderata for normative models of synaptic plasticity

Normative models of synaptic plasticity use a combination of mathematics and computational simulations to arrive at predictions of behavioral and network-level adaptive phenomena. In recent years, there has been an explosion of theoretical work on these models, but experimental confirmation is relatively limited. In this review, we organize work on normative plasticity models in terms of a set of desiderata which, when satisfied, are designed to guarantee that a model has a clear link between plasticity and adaptive behavior, consistency with known biological evidence about neural plasticity, and specific testable predictions. We then discuss how new models have begun to improve on these criteria and suggest avenues for further development. As prototypes, we provide detailed analyses of two specific models - REINFORCE and the Wake-Sleep algorithm. We provide a conceptual guide to help develop neural learning theories that are precise, powerful, and experimentally testable.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Probing pathways by which rhynchophylline modifies sleep using spatial transcriptomics

BACKGROUND

Rhynchophylline (RHY) is an alkaloid component of Uncaria, which are plants extensively used in traditional Asian medicines. Uncaria treatments increase sleep time and quality in humans, and RHY induces sleep in rats. However, like many traditional natural treatments, the mechanisms of action of RHY and Uncaria remain evasive. Moreover, it is unknown whether RHY modifies key brain oscillations during sleep. We thus aimed at defining the effects of RHY on sleep architecture and oscillations throughout a 24-h cycle, as well as identifying the underlying molecular mechanisms. Mice received systemic RHY injections at two times of the day (beginning and end of the light period), and vigilance states were studied by electrocorticographic recordings.

RESULTS

RHY enhanced slow wave sleep (SWS) after both injections, suppressed paradoxical sleep (PS) in the light but enhanced PS in the dark period. Furthermore, RHY modified brain oscillations during both wakefulness and SWS (including delta activity dynamics) in a time-dependent manner. Interestingly, most effects were larger in females. A brain spatial transcriptomic analysis showed that RHY modifies the expression of genes linked to cell movement, apoptosis/necrosis, and transcription/translation in a brain region-independent manner, and changes those linked to sleep regulation (e.g., Hcrt, Pmch) in a brain region-specific manner (e.g., in the hypothalamus).

CONCLUSIONS

The findings provide support to the sleep-inducing effect of RHY, expose the relevance to shape wake/sleep oscillations, and highlight its effects on the transcriptome with a high spatial resolution. The exposed molecular mechanisms underlying the effect of a natural compound should benefit sleep- and brain-related medicine.

Sleep-A brain-state serving systems memory consolidation

Although long-term memory consolidation is supported by sleep, it is unclear how it differs from that during wakefulness. Our review, focusing on recent advances in the field, identifies the repeated replay of neuronal firing patterns as a basic mechanism triggering consolidation during sleep and wakefulness. During sleep, memory replay occurs during slow-wave sleep (SWS) in hippocampal assemblies together with ripples, thalamic spindles, neocortical slow oscillations, and noradrenergic activity. Here, hippocampal replay likely favors the transformation of hippocampus-dependent episodic memory into schema-like neocortical memory. REM sleep following SWS might balance local synaptic rescaling accompanying memory transformation with a sleep-dependent homeostatic process of global synaptic renormalization. Sleep-dependent memory transformation is intensified during early development despite the immaturity of the hippocampus. Overall, beyond its greater efficacy, sleep consolidation differs from wake consolidation mainly in that it is supported, rather than impaired, by spontaneous hippocampal replay activity possibly gating memory formation in neocortex.

How our understanding of memory replay evolves

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

Emergent Low-Frequency Activity in Cortico-Cerebellar Networks with Motor Skill Learning

The motor cortex controls skilled arm movement by recruiting a variety of targets in the nervous system, and it is important to understand the emergent activity in these regions as refinement of a motor skill occurs. One fundamental projection of the motor cortex (M1) is to the cerebellum. However, the emergent activity in the motor cortex and the cerebellum that appears as a dexterous motor skill is consolidated is incompletely understood. Here, we report on low-frequency oscillatory (LFO) activity that emerges in cortico-cerebellar networks with learning the reach-to-grasp motor skill. We chronically recorded the motor and the cerebellar cortices in rats, which revealed the emergence of coordinated movement-related activity in the local-field potentials as the reaching skill consolidated. Interestingly, we found this emergent activity only in the rats that gained expertise in the task. We found that the local and cross-area spiking activity was coordinated with LFOs in proficient rats. Finally, we also found that these neural dynamics were more prominently expressed during accurate behavior in the M1. This work furthers our understanding on emergent dynamics in the cortico-cerebellar loop that underlie learning and execution of precise skilled movement.

Cortical-hippocampal coupling during manifold exploration in motor cortex

Systems consolidation-a process for long-term memory stabilization-has been hypothesized to occur in two stages1-4. Whereas new memories require the hippocampus5-9, they become integrated into cortical networks over time10-12, making them independent of the hippocampus. How hippocampal-cortical dialogue precisely evolves during this and how cortical representations change in concert is unknown. Here, we use a skill learning task13,14 to monitor the dynamics of cross-area coupling during non-rapid eye movement sleep along with changes in primary motor cortex (M1) representational stability. Our results indicate that precise cross-area coupling between hippocampus, prefrontal cortex and M1 can demarcate two distinct stages of processing. We specifically find that each animal demonstrates a sharp increase in prefrontal cortex and M1 sleep slow oscillation coupling with stabilization of performance. This sharp increase then predicts a drop in hippocampal sharp-wave ripple (SWR)-M1 slow oscillation coupling-suggesting feedback to inform hippocampal disengagement and transition to a second stage. Notably, the first stage shows significant increases in hippocampal SWR-M1 slow oscillation coupling in the post-training sleep and is closely associated with rapid learning and variability of the M1 low-dimensional manifold. Strikingly, even after consolidation, inducing new manifold exploration by changing task parameters re-engages hippocampal-M1 coupling. We thus find evidence for dynamic hippocampal-cortical dialogue associated with manifold exploration during learning and adaptation.

Neural circuit plasticity for complex non-declarative sensorimotor memory consolidation during sleep

Evidence is accumulating that the brain actively consolidates long-term memory during sleep. Motor skill memory is a form of non-declarative procedural memory and can be coordinated with multi-sensory processing such as visual, tactile, and, auditory. Conversely, perception is affected by body movement signal from motor brain regions. Although both cortical and subcortical brain regions are involved in memory consolidation, cerebral cortex activity can be recorded and manipulated noninvasively or minimally invasively in humans and animals. NREM sleep, which is important for non-declarative memory consolidation, is characterized by slow and spindle waves representing thalamo-cortical population activity. In animals, electrophysiological recording, optical imaging, and manipulation approaches have revealed multi-scale cortical dynamics across learning and sleep. In the sleeping cortex, neural activity is affected by prior learning and neural circuits are continually reorganized. Here I outline how sensorimotor coordination is formed through awake learning and subsequent sleep.

Information maximization explains state-dependent synaptic plasticity and memory reorganization during non-rapid eye movement sleep

Slow waves during the non-rapid eye movement (NREM) sleep reflect the alternating up and down states of cortical neurons; global and local slow waves promote memory consolidation and forgetting, respectively. Furthermore, distinct spike-timing-dependent plasticity (STDP) operates in these up and down states. The contribution of different plasticity rules to neural information coding and memory reorganization remains unknown. Here, we show that optimal synaptic plasticity for information maximization in a cortical neuron model provides a unified explanation for these phenomena. The model indicates that the optimal synaptic plasticity is biased toward depression as the baseline firing rate increases. This property explains the distinct STDP observed in the up and down states. Furthermore, it explains how global and local slow waves predominantly potentiate and depress synapses, respectively, if the background firing rate of excitatory neurons declines with the spatial scale of waves as the model predicts. The model provides a unifying account of the role of NREM sleep, bridging neural information coding, synaptic plasticity, and memory reorganization.

Selective Enhancement of Post-Sleep Visual Motion Perception by Repetitive Tactile Stimulation during Sleep

Tactile sensations can bias visual perception in the awake state while visual sensitivity is known to be facilitated by sleep. It remains unknown, however, whether the tactile sensation during sleep can bias the visual improvement after sleep. Here, we performed nap experiments in human participants (n = 56, 18 males, 38 females) to demonstrate that repetitive tactile motion stimulation on the fingertip during slow wave sleep selectively enhanced subsequent visual motion detection. The visual improvement was associated with slow wave activity. The high activation at the high beta frequency was found in the occipital electrodes after the tactile motion stimulation during sleep, indicating a visual-tactile cross-modal interaction during sleep. Furthermore, a second experiment (n = 14, 14 females) to examine whether a hand- or head-centered coordination is dominant for the interpretation of tactile motion direction showed that the biasing effect on visual improvement occurs according to the hand-centered coordination. These results suggest that tactile information can be interpreted during sleep, and can induce the selective improvement of post-sleep visual motion detection.SIGNIFICANCE STATEMENT Tactile sensations can bias our visual perception as a form of cross-modal interaction. However, it was reported only in the awake state. Here we show that repetitive directional tactile motion stimulation on the fingertip during slow wave sleep selectively enhanced subsequent visual motion perception. Moreover, the visual improvement was positively associated with sleep slow wave activity. The tactile motion stimulation during slow wave activity increased the activation at the high beta frequency over the occipital electrodes. The visual improvement occurred in agreement with a hand-centered reference frame. These results suggest that our sleeping brain can interpret tactile information based on a hand-centered reference frame, which can cause the sleep-dependent improvement of visual motion detection.

Modulation of neural co-firing to enhance network transmission and improve motor function after stroke

Stroke is a leading cause of disability. While neurotechnology has shown promise for improving upper limb recovery after stroke, efficacy in clinical trials has been variable. Our central thesis is that to improve clinical translation, we need to develop a common neurophysiological framework for understanding how neurotechnology alters network activity. Our perspective discusses principles for how motor networks, both healthy and those recovering from stroke, subserve reach-to-grasp movements. We focus on neural processing at the resolution of single movements, the timescale at which neurotechnologies are applied, and discuss how this activity might drive long-term plasticity. We propose that future studies should focus on cross-area communication and bridging our understanding of timescales ranging from single trials within a session to across multiple sessions. We hope that this perspective establishes a combined path forward for preclinical and clinical research with the goal of more robust clinical translation of neurotechnology.

Heksor: The CNS substrate of an adaptive behavior

Over the past half‐century, the largely hardwired central nervous system (CNS) of 1970 has become the ubiquitously plastic CNS of today, in which change is the rule not the exception. This transformation complicates a central question in neuroscience: how are adaptive behaviours – behaviours that serve the needs of the individual – acquired and maintained through life? It poses a more basic question: how do many adaptive behaviours share the ubiquitously plastic CNS? This question compels neuroscience to adopt a new paradigm. The core of this paradigm is a CNS entity with unique properties, here given the name heksor from the Greek hexis. A heksor is a distributed network of neurons and synapses that changes itself as needed to maintain the key features of an adaptive behaviour, the features that make the behaviour satisfactory. Through their concurrent changes, the numerous heksors that share the CNS negotiate the properties of the neurons and synapses that they all use. Heksors keep the CNS in a state of negotiated equilibrium that enables each heksor to maintain the key features of its behaviour. The new paradigm based on heksors and the negotiated equilibrium they create is supported by animal and human studies of interactions among new and old adaptive behaviours, explains otherwise inexplicable results, and underlies promising new approaches to restoring behaviours impaired by injury or disease. Furthermore, the paradigm offers new and potentially important answers to extant questions, such as the generation and function of spontaneous neuronal activity, the aetiology of muscle synergies, and the control of homeostatic plasticity.

Transition from predictable to variable motor cortex and striatal ensemble patterning during behavioral exploration

Animals can capitalize on invariance in the environment by learning and automating highly consistent actions; however, they must also remain flexible and adapt to environmental changes. It remains unclear how primary motor cortex (M1) can drive precise movements, yet also support behavioral exploration when faced with consistent errors. Using a reach-to-grasp task in rats, along with simultaneous electrophysiological monitoring in M1 and dorsolateral striatum (DLS), we find that behavioral exploration to overcome consistent task errors is closely associated with tandem increases in M1 and DLS neural variability; subsequently, consistent ensemble patterning returns with convergence to a new successful strategy. We also show that compared to reliably patterned intracranial microstimulation in M1, variable stimulation patterns result in significantly greater movement variability. Our results thus indicate that motor and striatal areas can flexibly transition between two modes, reliable neural pattern generation for automatic and precise movements versus variable neural patterning for behavioral exploration.

It is not fully understood how behavioral flexibility is established in the context of automatic performance of a complex motor skill. Here the authors show that corticostriatal activity can flexibly transition between two modes during a reach to-grasp task in rats: reliable neural pattern generation for precise, automatic movements versus variable neural patterning for behavioral exploration.

Recovery of consolidation after sleep following stroke-interaction of slow waves, spindles, and GABA

Sleep is known to promote recovery after stroke. Yet it remains unclear how stroke affects neural processing during sleep. Using an experimental stroke model in rats along with electrophysiological monitoring of neural firing and sleep microarchitecture, here we show that sleep processing is altered by stroke. We find that the precise coupling of spindles to global slow oscillations (SOs), a phenomenon that is known to be important for memory consolidation, is disrupted by a pathological increase in "isolated" local delta waves. The transition from this pathological to a physiological state-with increased spindle coupling to SO-is associated with sustained performance gains during recovery. Interestingly, post-injury sleep could be pushed toward a physiological state via a pharmacological reduction of tonic γ-aminobutyric acid (GABA). Together, our results suggest that sleep processing after stroke is impaired due to an increase in delta waves and that its restoration can be important for recovery.

Timescales of Local and Cross-Area Interactions during Neuroprosthetic Learning

How does the brain integrate signals with different timescales to drive purposeful actions? Brain-machine interfaces (BMIs) offer a powerful tool to causally test how distributed neural networks achieve specific neural patterns. During neuroprosthetic learning, actuator movements are causally linked to primary motor cortex (M1) neurons, i.e., "direct" neurons that project to the decoder and whose firing is required to successfully perform the task. However, it is unknown how such direct M1 neurons interact with both "indirect" local (in M1 but not part of the decoder) and across area neural populations (e.g., in premotor cortex/M2), all of which are embedded in complex biological recurrent networks. Here, we trained male rats to perform a M1-BMI task and simultaneously recorded the activity of indirect neurons in both M2 and M1. We found that both M2 and M1 indirect neuron populations could be used to predict the activity of the direct neurons (i.e., "BMI-potent activity"). Interestingly, compared with M1 indirect activity, M2 neural activity was correlated with BMI-potent activity across a longer set of time lags, and the timescale of population activity patterns evolved more slowly. M2 units also predicted the activity of both M1 direct and indirect neural populations, suggesting that M2 population dynamics provide a continuous modulatory influence on M1 activity as a whole, rather than a moment-by-moment influence solely on neurons most relevant to a task. Together, our results indicate that longer timescale M2 activity provides modulatory influence over extended time lags on shorter-timescale control signals in M1.SIGNIFICANCE STATEMENT A central question in the study of motor control is whether primary motor cortex (M1) and premotor cortex (M2) interact through task-specific subpopulations of neurons, or whether tasks engage broader correlated networks. Brain-machine interfaces (BMIs) are powerful tools to study cross-area interactions. Here, we performed simultaneous recordings of M1 and M2 in a BMI task using a subpopulation of M1 neurons (direct neurons). We found that activity outside of direct neurons in M1 and M2 was predictive of M1-BMI task activity, and that M2 activity evolved at slower timescales than M1. These findings suggest that M2 provides a continuous modulatory influence on M1 as a whole, supporting a model of interactions through broad correlated networks rather than task-specific neural subpopulations.

Sleep-learning impairs subsequent awake-learning

Although we can learn new information while asleep, we usually cannot consciously remember the sleep-formed memories - presumably because learning occurred in an unconscious state. Here, we ask whether sleep-learning expedites the subsequent awake-learning of the same information. To answer this question, we reanalyzed data (Züst et al., 2019, Curr Biol) from napping participants, who learned new semantic associations between pseudowords and translation-words (guga-ship) while in slow-wave sleep. They retrieved sleep-formed associations unconsciously on an implicit memory test following awakening. Then, participants took five runs of paired-associative learning to probe carry-over effects of sleep-learning on awake-learning. Surprisingly, sleep-learning diminished awake-learning when participants learned semantic associations that were congruent to sleep-learned associations (guga-boat). Yet, learning associations that conflicted with sleep-learned associations (guga-coin) was unimpaired relative to learning new associations (resun-table; baseline). We speculate that the impeded wake-learning originated in a deficient synaptic downscaling and resulting synaptic saturation in neurons that were activated during both sleep-learning and awake-learning.

Brain neural patterns and the memory function of sleep

[Figure: see text].

Cortical Responses to Vagus Nerve Stimulation Are Modulated by Brain State in Nonhuman Primates

Vagus nerve stimulation (VNS) has been tested as therapy for several brain disorders and as a means to modulate cortical excitability and brain plasticity. Cortical effects of VNS, manifesting as vagal-evoked potentials (VEPs), are thought to arise from activation of ascending cholinergic and noradrenergic systems. However, it is unknown whether those effects are modulated by brain state at the time of stimulation. In 2 freely behaving macaque monkeys, we delivered short trains of 5 pulses to the left cervical vagus nerve at different frequencies (5-300 Hz) while recording local field potentials (LFPs) from sites in contralateral prefrontal, sensorimotor and parietal cortical areas. Brain states were inferred from spectral components of LFPs and the presence of overt movement: active awake, resting awake, REM sleep and NREM sleep. VNS elicited VEPs in all sampled cortical areas. VEPs comprised early (<70 ms), intermediate (70-250 ms) and late (>250 ms) components. The magnitude of the intermediate and late components was largest during NREM sleep and smallest during wakefulness, whereas that of the early component was not modulated by brain state. VEPs during NREM were larger for stimuli delivered at the depolarized phase of ongoing delta oscillations. Higher pulsing frequencies generated larger VEPs. These short VNS trains did not affect brain state transitions during wakefulness or sleep. Our findings suggest that ongoing brain state modulates the evoked effects of VNS on cortical activity. This has implications for the role of ongoing cortical activity and brain state in shaping cortical responses to peripheral stimuli, for the modulation of vagal interoceptive signaling by cortical activity, and for the dose calibration of VNS therapies.

Replay in Deep Learning: Current Approaches and Missing Biological Elements

Replay is the reactivation of one or more neural patterns that are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated in deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this letter, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be used to improve artificial neural networks.

The Influence of Frequency Bands and Brain Region on ECoG-Based BMI Learning Performance

Numerous brain–machine interface (BMI) studies have shown that various frequency bands (alpha, beta, and gamma bands) can be utilized in BMI experiments and modulated as neural information for machine control after several BMI learning trial sessions. In addition to frequency range as a neural feature, various areas of the brain, such as the motor cortex or parietal cortex, have been selected as BMI target brain regions. However, although the selection of target frequency and brain region appears to be crucial in obtaining optimal BMI performance, the direct comparison of BMI learning performance as it relates to various brain regions and frequency bands has not been examined in detail. In this study, ECoG-based BMI learning performances were compared using alpha, beta, and gamma bands, respectively, in a single rodent model. Brain area dependence of learning performance was also evaluated in the frontal cortex, the motor cortex, and the parietal cortex. The findings indicated that BMI learning performance was best in the case of the gamma frequency band and worst in the alpha band (one-way ANOVA, F = 4.41, p < 0.05). In brain area dependence experiments, better BMI learning performance appears to be shown in the primary motor cortex (one-way ANOVA, F = 4.36, p < 0.05). In the frontal cortex, two out of four animals failed to learn the feeding tube control even after a maximum of 10 sessions. In conclusion, the findings reported in this study suggest that the selection of target frequency and brain region should be carefully considered when planning BMI protocols and for performing optimized BMI.

Coupling between motor cortex and striatum increases during sleep over long-term skill learning

The strength of cortical connectivity to the striatum influences the balance between behavioral variability and stability. Learning to consistently produce a skilled action requires plasticity in corticostriatal connectivity associated with repeated training of the action. However, it remains unknown whether such corticostriatal plasticity occurs during training itself or 'offline' during time away from training, such as sleep. Here, we monitor the corticostriatal network throughout long-term skill learning in rats and find that non-rapid-eye-movement (NREM) sleep is a relevant period for corticostriatal plasticity. We first show that the offline activation of striatal NMDA receptors is required for skill learning. We then show that corticostriatal functional connectivity increases offline, coupled to emerging consistent skilled movements, and coupled cross-area neural dynamics. We then identify NREM sleep spindles as uniquely poised to mediate corticostriatal plasticity, through interactions with slow oscillations. Our results provide evidence that sleep shapes cross-area coupling required for skill learning.

Fear conditioning potentiates the hippocampal CA1 commissural pathway in vivo and increases awake phase sleep

The hippocampus is essential for spatial learning and memory. To assess learning we used contextual fear conditioning (cFC), where animals learn to associate a place with aversive events like foot‐shocks. Candidate memory mechanisms for cFC are long‐term potentiation (LTP) and long‐term depression (LTD), but there is little direct evidence of them operating in the hippocampus in vivo following cFC. Also, little is known about the behavioral state changes induced by cFC. To address these issues, we recorded local field potentials in freely behaving mice by stimulating in the left dorsal CA1 region and recording in the right dorsal CA1 region. Synaptic strength in the commissural pathway was monitored by measuring field excitatory postsynaptic potentials (fEPSPs) before and after cFC. After cFC, the commissural pathway's synaptic strength was potentiated. Although recordings occurred during the wake phase of the light/dark cycle, the mice slept more in the post‐conditioning period than in the pre‐conditioning period. Relative to awake periods, in non‐rapid eye movement (NREM) sleep the fEPSPs were larger in both pre‐ and post‐conditioning periods. We also found a significant negative correlation between the animal's speed and fEPSP size. Therefore, to avoid confounds in the fEFSP potentiation estimates, we controlled for speed‐related and sleep‐related fEPSP changes and still found that cFC induced long‐term potentiation, but no significant long‐term depression. Synaptic strength changes were not found in the control group that simply explored the fear‐conditioning chamber, indicating that exploration of the novel place did not produce the measurable effects caused by cFC. These results show that following cFC, the CA1 commissural pathway is potentiated, likely contributing to the functional integration of the left and right hippocampi in fear memory consolidation. In addition, the cFC paradigm produces significant changes in an animal's behavioral state, which are observable as proximal changes in sleep patterns.

Replay in minds and machines

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Shining a Light on the Mechanisms of Sleep for Memory Consolidation

Purpose of review

This paper reviews all optogenetic studies that directly test various sleep states, traits, and circuit-level activity profiles for the consolidation of different learning tasks.

Recent findings

Inhibiting or exciting neurons involved either in the production of sleep states or in the encoding and consolidation of memories reveals sleep states and traits that are essential for memory. REM sleep, NREM sleep, and the N2 transition to REM (characterized by sleep spindles) are integral to memory consolidation. Neural activity during sharp-wave ripples, slow oscillations, theta waves, and spindles are the mediators of this process.

Summary

These studies lend strong support to the hypothesis that sleep is essential to the consolidation of memories from the hippocampus and the consolidation of motor learning which does not necessarily involve the hippocampus. Future research can further probe the types of memory dependent on sleep-related traits and on the neurotransmitters and neuromodulators required.

Coordinated increase of reliable cortical and striatal ensemble activations during recovery after stroke

Skilled movements rely on a coordinated cortical and subcortical network, but how this network supports motor recovery after stroke is unknown. Previous studies focused on the perilesional cortex (PLC), but precisely how connected subcortical areas reorganize and coordinate with PLC is unclear. The dorsolateral striatum (DLS) is of interest because it receives monosynaptic inputs from motor cortex and is important for learning and generation of fast reliable actions. Using a rat focal stroke model, we perform chronic electrophysiological recordings in motor PLC and DLS during long-term recovery of a dexterous skill. We find that recovery is associated with the simultaneous emergence of reliable movement-related single-trial ensemble spiking in both structures along with increased cross-area alignment of spiking. Our study highlights the importance of consistent neural activity patterns across brain structures during recovery and suggests that modulation of cross-area coordination can be a therapeutic target for enhancing motor function post-stroke.

Chronic wireless neural population recordings with common marmosets

Marmosets are an increasingly important model system for neuroscience in part due to genetic tractability and enhanced cortical accessibility, due to a lissencephalic neocortex. However, many of the techniques generally employed to record neural activity in primates inhibit the expression of natural behaviors in marmosets precluding neurophysiological insights. To address this challenge, we have developed methods for recording neural population activity in unrestrained marmosets across multiple ethological behaviors, multiple brain states, and over multiple years. Notably, our flexible methodological design allows for replacing electrode arrays and removal of implants providing alternative experimental endpoints. We validate the method by recording sensorimotor cortical population activity in freely moving marmosets across their natural behavioral repertoire and during sleep.

Epidural cerebellar stimulation drives widespread neural synchrony in the intact and stroke perilesional cortex

BACKGROUND

Cerebellar electrical stimulation has shown promise in improving motor recovery post-stroke in both rodent and human studies. Past studies have used motor evoked potentials (MEPs) to evaluate how cerebellar stimulation modulates ongoing activity in the cortex, but the underlying mechanisms are incompletely understood. Here we used invasive electrophysiological recordings from the intact and stroke-injured rodent primary motor cortex (M1) to assess how epidural cerebellar stimulation modulates neural dynamics at the level of single neurons as well as at the level of mesoscale dynamics.

METHODS

We recorded single unit spiking and local field potentials (LFPs) in both the intact and acutely stroke-injured M1 contralateral to the stimulated cerebellum in adult Long-Evans rats under anesthesia. We analyzed changes in the firing rates of single units, the extent of synchronous spiking and power spectral density (PSD) changes in LFPs during and post-stimulation.

RESULTS

Our results show that post-stimulation, the firing rates of a majority of M1 neurons changed significantly with respect to their baseline rates. These firing rate changes were diverse in character, as the firing rate of some neurons increased while others decreased. Additionally, these changes started to set in during stimulation. Furthermore, cross-correlation analysis showed a significant increase in coincident firing amongst neuronal pairs. Interestingly, this increase in synchrony was unrelated to the direction of firing rate change. We also found that neuronal ensembles derived through principal component analysis were more active post-stimulation. Lastly, these changes occurred without a significant change in the overall spectral power of LFPs post-stimulation.

CONCLUSIONS

Our results show that cerebellar stimulation caused significant, long-lasting changes in the activity patterns of M1 neurons by altering firing rates, boosting neural synchrony and increasing neuronal assemblies' activation strength. Our study provides evidence that cerebellar stimulation can directly modulate cortical dynamics. Since these results are present in the perilesional cortex, our data might also help explain the facilitatory effects of cerebellar stimulation post-stroke.

Net decrease in spine-surface GluA1-containing AMPA receptors after post-learning sleep in the adult mouse cortex

The mechanisms by which sleep benefits learning and memory remain unclear. Sleep may further strengthen the synapses potentiated by learning or promote broad synaptic weakening while protecting the newly potentiated synapses. We tested these ideas by combining a motor task whose consolidation is sleep-dependent, a marker of synaptic AMPA receptor plasticity, and repeated two-photon imaging to track hundreds of spines in vivo with single spine resolution. In mouse motor cortex, sleep leads to an overall net decrease in spine-surface GluA1-containing AMPA receptors, both before and after learning. Molecular changes in single spines during post-learning sleep are correlated with changes in performance after sleep. The spines in which learning leads to the largest increase in GluA1 expression have a relative advantage after post-learning sleep compared to sleep deprivation, because sleep weakens all remaining spines. These results are obtained in adult mice, showing that sleep-dependent synaptic down-selection also benefits the mature brain.

Neural fatigue due to intensive learning is reversed by a nap but not by quiet waking

Do brain circuits become fatigued due to intensive neural activity or plasticity? Is sleep necessary for recovery? Well-rested subjects trained extensively in a visuo-motor rotation learning task (ROT) or a visuo-motor task without rotation learning (MOT), followed by sleep or quiet wake. High-density electroencephalography showed that ROT training led to broad increases in EEG power over a frontal cluster of electrodes, with peaks in the theta (mean ± SE: 24% ± 6%, p = 0.0013) and beta ranges (10% ± 3%, p = 0.01). These traces persisted in the spontaneous EEG (sEEG) between sessions (theta: 42% ± 8%, p = 0.0001; beta: 35% ± 7%, p = 0.002) and were accompanied by increased errors in a motor test with kinematic characteristics and neural substrates similar to ROT (81.8% ± 0.8% vs. 68.2% ± 2.3%; two-tailed paired t-test: p = 0.00001; Cohen's d = 1.58), as well as by score increases of subjective task-specific fatigue (4.00 ± 0.39 vs. 5.36 ± 0.39; p = 0.0007; Cohen's d = 0.60). Intensive practice with MOT did not affect theta sEEG or the motor test. A nap, but not quiet wake, induced a local sEEG decrease of theta power by 33% (SE: 8%, p = 0.02), renormalized test performance (70.9% ± 2.9% vs 79.1% ± 2.7%, p = 0.018, Cohen's d = 0.85), and improved learning ability in ROT (adaptation rate: 71.2 ± 1.2 vs. 73.4 ± 0.9, p = 0.024; Cohen's d = 0.60). Thus, sleep is necessary to restore plasticity-induced fatigue and performance.

Cell-Type-Specific Dynamics of Calcium Activity in Cortical Circuits over the Course of Slow-Wave Sleep and Rapid Eye Movement Sleep

Sleep shapes cortical network activity, fostering global homeostatic downregulation of excitability while maintaining or even upregulating excitability in selected networks in a manner that supports memory consolidation. Here, we used two-photon calcium imaging of cortical layer 2/3 neurons in sleeping male mice to examine how these seemingly opposing dynamics are balanced in cortical networks. During slow-wave sleep (SWS) episodes, mean calcium activity of excitatory pyramidal (Pyr) cells decreased. Simultaneously, however, variance in Pyr population calcium activity increased, contradicting the notion of a homogenous downregulation of network activity. Indeed, we identified a subpopulation of Pyr cells distinctly upregulating calcium activity during SWS, which were highly active during sleep spindles known to support mnemonic processing. Rapid eye movement (REM) episodes following SWS were associated with a general downregulation of Pyr cells, including the subpopulation of Pyr cells active during spindles, which persisted into following stages of sleep and wakefulness. Parvalbumin-positive inhibitory interneurons (PV-In) showed an increase in calcium activity during SWS episodes, while activity remained unchanged during REM sleep episodes. This supports the view that downregulation of Pyr calcium activity during SWS results from increased somatic inhibition via PV-In, whereas downregulation during REM sleep is achieved independently of such inhibitory activity. Overall, our findings show that SWS enables upregulation of select cortical circuits (likely those which were involved in mnemonic processing) through a spindle-related process, whereas REM sleep mediates general downregulation, possibly through synaptic re-normalization.SIGNIFICANCE STATEMENT Sleep is thought to globally downregulate cortical excitability and, concurrently, to upregulate synaptic connections in neuron ensembles with newly encoded memory, with upregulation representing a function of sleep spindles. Using in vivo two-photon calcium imaging in combination with surface EEG recordings, we classified cells based on their calcium activity during sleep spindles. Spindle-active pyramidal (Pyr) cells persistently increased calcium activity during slow-wave sleep (SWS) episodes while spindle-inactive cells decreased calcium activity. Subsequent rapid eye movement (REM) sleep episodes profoundly reduced calcium activity in both cell clusters. Results indicate that SWS allows for a spindle-related differential upregulation of ensembles whereas REM sleep functions to globally downregulate networks.

t-soft update of target network for deep reinforcement learning

This paper proposes a new robust update rule of target network for deep reinforcement learning (DRL), to replace the conventional update rule, given as an exponential moving average. The target network is for smoothly generating the reference signals for a main network in DRL, thereby reducing learning variance. The problem with its conventional update rule is the fact that all the parameters are smoothly copied with the same speed from the main network, even when some of them are trying to update toward the wrong directions. This behavior increases the risk of generating the wrong reference signals. Although slowing down the overall update speed is a naive way to mitigate wrong updates, it would decrease learning speed. To robustly update the parameters while keeping learning speed, a t-soft update method, which is inspired by Student-t distribution, is derived with reference to the analogy between the exponential moving average and the normal distribution. Through the analysis of the derived t-soft update, we show that it takes over the properties of the Student-t distribution. Specifically, with a heavy-tailed property of the Student-t distribution, the t-soft update automatically excludes extreme updates that differ from past experiences. In addition, when the updates are similar to the past experiences, it can mitigate the learning delay by increasing the amount of updates. In PyBullet robotics simulations for DRL, an online actor-critic algorithm with the t-soft update outperformed the conventional methods in terms of the obtained return and/or its variance. From the training process by the t-soft update, we found that the t-soft update is globally consistent with the standard soft update, and the update rates are locally adjusted for acceleration or suppression.

Plug-and-play control of a brain-computer interface through neural map stabilization

Brain-computer interfaces (BCIs) enable control of assistive devices in individuals with severe motor impairments. A limitation of BCIs that has hindered real-world adoption is poor long-term reliability and lengthy daily recalibration times. To develop methods that allow stable performance without recalibration, we used a 128-channel chronic electrocorticography (ECoG) implant in a paralyzed individual, which allowed stable monitoring of signals. We show that long-term closed-loop decoder adaptation, in which decoder weights are carried across sessions over multiple days, results in consolidation of a neural map and 'plug-and-play' control. In contrast, daily reinitialization led to degradation of performance with variable relearning. Consolidation also allowed the addition of control features over days, that is, long-term stacking of dimensions. Our results offer an approach for reliable, stable BCI control by leveraging the stability of ECoG interfaces and neural plasticity.

Competing Roles of Slow Oscillations and Delta Waves in Memory Consolidation versus Forgetting

Sleep has been implicated in both memory consolidation and forgetting of experiences. However, it is unclear what governs the balance between consolidation and forgetting. Here, we tested how activity-dependent processing during sleep might differentially regulate these two processes. We specifically examined how neural reactivations during non-rapid eye movement (NREM) sleep were causally linked to consolidation versus weakening of the neural correlates of neuroprosthetic skill. Strikingly, we found that slow oscillations (SOs) and delta (δ) waves have dissociable and competing roles in consolidation versus forgetting. By modulating cortical spiking linked to SOs or δ waves using closed-loop optogenetic methods, we could, respectively, weaken or strengthen consolidation and thereby bidirectionally modulate sleep-dependent performance gains. We further found that changes in the temporal coupling of spindles to SOs relative to δ waves could account for such effects. Thus, our results indicate that neural activity driven by SOs and δ waves have competing roles in sleep-dependent memory consolidation.

Dynamics of Cortical Local Connectivity during Sleep-Wake States and the Homeostatic Process

Sleep exerts modulatory effects on the cerebral cortex. Whether sleep modulates local connectivity in the cortex or only individual neural activity, however, is poorly understood. Here we investigated functional connectivity, that is, covarying activity between neurons, during spontaneous sleep-wake states and during and after sleep deprivation using calcium imaging of identified excitatory/inhibitory neurons in the motor cortex. Functional connectivity was estimated with a statistical learning approach glasso and quantified by "the probability of establishing connectivity (sparse/dense)" and "the strength of the established connectivity (weak/strong)." Local cortical connectivity was sparse in non-rapid eye movement (NREM) sleep and dense in REM sleep, which was similar in both excitatory and inhibitory neurons. The overall mean strength of the connectivity did not differ largely across spontaneous sleep-wake states. Sleep deprivation induced strong excitatory/inhibitory and dense inhibitory, but not excitatory, connectivity. Subsequent NREM sleep after sleep deprivation exhibited weak excitatory/inhibitory, sparse excitatory, and dense inhibitory connectivity. These findings indicate that sleep-wake states modulate local cortical connectivity, and the modulation is large and compensatory for stability of local circuits during the homeostatic control of sleep, which contributes to plastic changes in neural information flow.

Open-source raspberry Pi-based operant box for translational behavioral testing in rodents

BACKGROUND

Rodents have been used for decades to probe neural circuits involved in behavior. Increasingly, attempts have been developed to standardize training paradigms across labs; and to use visual/auditory paradigms that can be also tested in humans. Commercially available systems are expensive and thus do not scale easily, and are not optimized for electrophysiology.

NEW METHOD

Using the rich open-source technology built around Raspberry Pi, we were able to develop an inexpensive (<$1000) visual-screen based operant chamber with electrophysiological and optogenetic compatibility. The chamber is operated within MATLAB/Simulink, a commonly used scientific programming language allowing for rapid customization.

RESULTS

Here, we describe and provide all relevant details needed to develop and produce these chambers, and show examples of behavior and electrophysiology data collected using these chambers. We also include all of the tools needed to allow readers to build and develop their own boxes (CAD models for 3D printing and laser-cutting; PCB-board design; all bill of materials for required parts and supplies, and some examples of Simulink models to operate the boxes).

COMPARISON WITH EXISTING METHODS

The new boxes are far more cost-effective than commercially available environments and allow for the combination of automated behavioral testing with electrophysiological read-outs with high temporal precision.

CONCLUSION

These open-source boxes can be used for labs interested in developing high-throughput visual/auditory behavioral assays for ∼ 10th the cost of commercial systems.