Temporally Precise Cell-Specific Coherence Develops in Corticostriatal Networks during Learning

The complex affective and cognitive capacities of rats

For several decades, although studies of rat physiology and behavior have abounded, research on rat emotions has been limited in scope to fear, anxiety, and pain. Converging evidence for the capacity of many species to share others' affective states has emerged, sparking interest in the empathic capacities of rats. Recent research has demonstrated that rats are a highly cooperative species and are motivated by others' distress to prosocial actions, such as opening a door or pulling a chain to release trapped conspecifics. Studies of rat affect, cognition, and neural function provide compelling evidence that rats have some capacity to represent others' needs, to instrumentally act to improve their well-being, and are thus capable of forms of targeted helping. Rats' complex abilities raise the importance of integrating new measures of rat well-being into scientific research.

SHIELD: Skull-shaped hemispheric implants enabling large-scale electrophysiology datasets in the mouse brain

To understand the neural basis of behavior, it is essential to measure spiking dynamics across many interacting brain regions. Although new technologies, such as Neuropixels probes, facilitate multi-regional recordings, significant surgical and procedural hurdles remain for these experiments to achieve their full potential. Here, we describe skull-shaped hemispheric implants enabling large-scale electrophysiology datasets (SHIELD). These 3D-printed skull-replacement implants feature customizable insertion holes, allowing dozens of cortical and subcortical structures to be recorded in a single mouse using repeated multi-probe insertions over many days. We demonstrate the procedure's high success rate, biocompatibility, lack of adverse effects on behavior, and compatibility with imaging and optogenetics. To showcase SHIELD's scientific utility, we use multi-probe recordings to reveal novel insights into how alpha rhythms organize spiking activity across visual and sensorimotor networks. Overall, this method enables powerful, large-scale electrophysiological experiments for the study of distributed neural computation.

Volitional modulation of neuronal activity in the external globus pallidus by engagement of the cortical-basal ganglia circuit

Volitional modulation of neural activity is not confined to the cortex but extends to various brain regions. Yet, it remains unclear whether neurons in the basal ganglia structure, the external globus pallidus (GPe), can be volitionally controlled. Here, we employed a volitional conditioning task to compare the volitional modulation of GPe and primary motor cortex (M1) neurons as well as the underlying circuits and control mechanisms. The results revealed that the volitional modulation of GPe neuronal activity engaged both M1 and substantia nigra pars reticulata (SNr) neurons, indicating the involvement of the cortex-GPe-SNr loop. In contrast, the volitional modulation of M1 neurons primarily occurred through the engagement of M1 local circuitry. Furthermore, lesioning M1 neurons did not affect the volitional learning or volitional control signal in GPe, whereas lesioning of GPe neurons impaired the learning process for the volitional modulation of M1 neuronal activity at the intermediate stage. Additionally, lesion of GPe neurons enhanced M1 neuronal activity when performing the volitional control task without reward delivery and a random reward test. Taken together, our findings demonstrated that GPe neurons could be volitionally controlled by engagement of the cortical-basal ganglia circuit and inhibit learning process for the volitional modulation of M1 neuronal activity by regulating M1 neuronal activity. Thus, GPe neurons can be effectively harnessed for independent volitional modulation for neurorehabilitation in patients with cortical damage. KEY POINTS: The cortical-basal ganglia circuit contributes to the volitional modulation of GPe neurons. Volitional modulation of M1 neuronal activity mainly engages M1 local circuitry. Bilateral GPe lesioning impedes volitional learning at the intermediate stages. Lesioning of GPe neurons inhibits volitional learning process by regulating M1 neuronal activity.

Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity

The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.

Goal-directed learning is multidimensional and accompanied by diverse and widespread changes in neocortical signaling

New tasks are often learned in stages with each stage reflecting a different learning challenge. Accordingly, each learning stage is likely mediated by distinct neuronal processes. And yet, most rodent studies of the neuronal correlates of goal-directed learning focus on individual outcome measures and individual brain regions. Here, we longitudinally studied mice from naïve to expert performance in a head-fixed, operant conditioning whisker discrimination task. In addition to tracking the primary behavioral outcome of stimulus discrimination, we tracked and compared an array of object-based and temporal-based behavioral measures. These behavioral analyses identify multiple, partially overlapping learning stages in this task, consistent with initial response implementation, early stimulus-response generalization, and late response inhibition. To begin to understand the neuronal foundations of these learning processes, we performed widefield Ca2+ imaging of dorsal neocortex throughout learning and correlated behavioral measures with neuronal activity. We found distinct and widespread correlations between neocortical activation patterns and various behavioral measures. For example, improvements in sensory discrimination correlated with target stimulus evoked activations of response-related cortices along with distractor stimulus evoked global cortical suppression. Our study reveals multidimensional learning for a simple goal-directed learning task and generates hypotheses for the neuronal modulations underlying these various learning processes.

Refinement of efficient encodings of movement in the dorsolateral striatum throughout learning

The striatum is required for normal action selection, movement, and sensorimotor learning. Although action-specific striatal ensembles have been well documented, it is not well understood how these ensembles are formed and how their dynamics may evolve throughout motor learning. Here we used longitudinal 2-photon Ca2+ imaging of dorsal striatal neurons in head-fixed mice as they learned to self-generate locomotion. We observed a significant activation of both direct- and indirect-pathway spiny projection neurons (dSPNs and iSPNs, respectively) during early locomotion bouts and sessions that gradually decreased over time. For dSPNs, onset- and offset-ensembles were gradually refined from active motion-nonspecific cells. iSPN ensembles emerged from neurons initially active during opponent actions before becoming onset- or offset-specific. Our results show that as striatal ensembles are progressively refined, the number of active nonspecific striatal neurons decrease and the overall efficiency of the striatum information encoding for learned actions increases.

Information flow between motor cortex and striatum reverses during skill learning

The coordination of neural activity across brain areas during a specific behavior is often interpreted as neural communication involved in controlling the behavior. However, whether information relevant to the behavior is actually transferred between areas is often untested. Here, we used information-theoretic tools to quantify how motor cortex and striatum encode and exchange behaviorally relevant information about specific reach-to-grasp movement features during skill learning in rats. We found a temporal shift in the encoding of behaviorally relevant information during skill learning, as well as a reversal in the primary direction of behaviorally relevant information flow, from cortex-to-striatum during naive movements to striatum-to-cortex during skilled movements. Standard analytical methods that quantify the evolution of overall neural activity during learning-such as changes in neural signal amplitude or the overall exchange of information between areas-failed to capture these behaviorally relevant information dynamics. Using these standard methods, we instead found a consistent coactivation of overall neural signals during movement production and a bidirectional increase in overall information propagation between areas during learning. Our results show that skill learning is achieved through a transformation in how behaviorally relevant information is routed across cortical and subcortical brain areas and that isolating the components of neural activity relevant to and informative about behavior is critical to uncover directional interactions within a coactive and coordinated network.

Corticospinal Modulation of Precision Movements

Recently we demonstrated a critical role for temporal coding of corticospinal activity in a prehension movement requiring precise forelimb control. Learning of precision isometric pull drives large-scale remodeling of corticospinal motor networks. Optogenetic modulation of corticospinal activity and full transection of the corticospinal tract disrupted critical functions of the network in expert animals resulting in impaired modulation of precise movements. In contrast, we observed more widespread corticospinal co-activation and limited temporal coding on a similar, yet more simplistic prehension task, adaptive isometric pull. Disrupting corticospinal neuron activity had much more limited effects on adaptive isometric pull, which was found to be corticospinal independent by transection of the corticospinal tract. Here we discuss these results in context of known roles for corticospinal and corticostriatal neurons in motor control, as well as some of the questions our study raised.

Cortico-cerebellar coordination facilitates neuroprosthetic control

Temporally coordinated neural activity is central to nervous system function and purposeful behavior. Still, there is a paucity of evidence demonstrating how this coordinated activity within cortical and subcortical regions governs behavior. We investigated this between the primary motor (M1) and contralateral cerebellar cortex as rats learned a neuroprosthetic/brain-machine interface (BMI) task. In neuroprosthetic task, actuator movements are causally linked to M1 "direct" neurons that drive the decoder for successful task execution. However, it is unknown how task-related M1 activity interacts with the cerebellum. We observed a notable 3 to 6 hertz coherence that emerged between these regions' local field potentials (LFPs) with learning that also modulated task-related spiking. We identified robust task-related indirect modulation in the cerebellum, which developed a preferential relationship with M1 task-related activity. Inhibiting cerebellar cortical and deep nuclei activity through optogenetics led to performance impairments in M1-driven neuroprosthetic control. Together, these results demonstrate that cerebellar influence is necessary for M1-driven neuroprosthetic control.

Enhancement of low gamma oscillations by volitional conditioning of local field potential in the primary motor and visual cortex of mice

Volitional control of local field potential oscillations in low gamma band via brain machine interface can not only uncover the relationship between low gamma oscillation and neural synchrony but also suggest a therapeutic potential to reverse abnormal local field potential oscillation in neurocognitive disorders. In nonhuman primates, the volitional control of low gamma oscillations has been demonstrated by brain machine interface techniques in the primary motor and visual cortex. However, it is not clear whether this holds in other brain regions and other species, for which gamma rhythms might involve in highly different neural processes. Here, we established a closed-loop brain-machine interface and succeeded in training mice to volitionally elevate low gamma power of local field potential in the primary motor and visual cortex. We found that the mice accomplished the task in a goal-directed manner and spiking activity exhibited phase-locking to the oscillation in local field potential in both areas. Moreover, long-term training made the power enhancement specific to direct and adjacent channel, and increased the transcriptional levels of NMDA receptors as well as that of hypoxia-inducible factor relevant to metabolism. Our results suggest that volitionally generated low gamma rhythms in different brain regions share similar mechanisms and pave the way for employing brain machine interface in therapy of neurocognitive disorders.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodents has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguishes control types at the go cue and target acquisition, respectively, while M1 best predicts target-direction at both task events. We also find effective connectivity from DLPFC → M1 throughout both control types and Cd → M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Dynamic top-down biasing implements rapid adaptive changes to individual movements

Complex behaviors depend on the coordinated activity of neural ensembles in interconnected brain areas. The behavioral function of such coordination, often measured as co-fluctuations in neural activity across areas, is poorly understood. One hypothesis is that rapidly varying co-fluctuations may be a signature of moment-by-moment task-relevant influences of one area on another. We tested this possibility for error-corrective adaptation of birdsong, a form of motor learning which has been hypothesized to depend on the top-down influence of a higher-order area, LMAN (lateral magnocellular nucleus of the anterior nidopallium), in shaping moment-by-moment output from a primary motor area, RA (robust nucleus of the arcopallium). In paired recordings of LMAN and RA in singing birds, we discovered a neural signature of a top-down influence of LMAN on RA, quantified as an LMAN-leading co-fluctuation in activity between these areas. During learning, this co-fluctuation strengthened in a premotor temporal window linked to the specific movement, sequential context, and acoustic modification associated with learning. Moreover, transient perturbation of LMAN activity specifically within this premotor window caused rapid occlusion of pitch modifications, consistent with LMAN conveying a temporally localized motor-biasing signal. Combined, our results reveal a dynamic top-down influence of LMAN on RA that varies on the rapid timescale of individual movements and is flexibly linked to contexts associated with learning. This finding indicates that inter-area co-fluctuations can be a signature of dynamic top-down influences that support complex behavior and its adaptation.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodent BMI has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from the primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguish between control types at the go cue and target acquisition, respectively. We also found effective connectivity from DLPFC→M1 throughout trials across both control types and Cd→M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Neural representation and modulation of volitional motivation in response to escalating efforts

Task-dependent volitional control of the selected neural activity in the cortex is critical to neuroprosthetic learning to achieve reliable and robust control of the external device. The volitional control of neural activity is driven by a motivational factor (volitional motivation), which directly reinforces the target neurons via real-time biofeedback. However, in the absence of motor behaviour, how do we evaluate volitional motivation? Here, we defined the criterion (ΔF/F) of the calcium fluorescence signal in a volitionally controlled neural task, then escalated the efforts by progressively increasing the number of reaching the criterion or holding time after reaching the criterion. We devised calcium-based progressive threshold-crossing events (termed 'Calcium PTE') and calcium-based progressive threshold-crossing holding-time (termed 'Calcium PTH') for quantitative assessment of volitional motivation in response to progressively escalating efforts. Furthermore, we used this novel neural representation of volitional motivation to explore the neural circuit and neuromodulator bases for volitional motivation. As with behavioural motivation, chemogenetic activation and pharmacological blockade of the striatopallidal pathway decreased and increased, respectively, the breakpoints of the 'Calcium PTE' and 'Calcium PTH' in response to escalating efforts. Furthermore, volitional and behavioural motivation shared similar dopamine dynamics in the nucleus accumbens in response to trial-by-trial escalating efforts. In general, the development of a neural representation of volitional motivation may open a new avenue for smooth and effective control of brain-machine interface tasks. KEY POINTS: Volitional motivation is quantitatively evaluated by M1 neural activity in response to progressively escalating volitional efforts. The striatopallidal pathway and adenosine A_2A receptor modulate volitional motivation in response to escalating efforts. Dopamine dynamics encode prediction signal for reward in response to repeated escalating efforts during motor and volitional conditioning. Mice learn to modulate neural activity to compensate for repeated escalating efforts in volitional control.

Self-modulation of motor cortex activity after stroke: a randomized controlled trial

Real-time functional MRI neurofeedback allows individuals to self-modulate their ongoing brain activity. This may be a useful tool in clinical disorders that are associated with altered brain activity patterns. Motor impairment after stroke has previously been associated with decreased laterality of motor cortex activity. Here we examined whether chronic stroke survivors were able to use real-time fMRI neurofeedback to increase laterality of motor cortex activity and assessed effects on motor performance and on brain structure and function. We carried out a randomized, double-blind, sham-controlled trial (ClinicalTrials.gov: NCT03775915) in which 24 chronic stroke survivors with mild to moderate upper limb impairment experienced three training days of either Real (n = 12) or Sham (n = 12) neurofeedback. Assessments of brain structure, brain function and measures of upper-limb function were carried out before and 1 week after neurofeedback training. Additionally, measures of upper-limb function were repeated 1 month after neurofeedback training. Primary outcome measures were (i) changes in lateralization of motor cortex activity during movements of the stroke-affected hand throughout neurofeedback training days; and (ii) changes in motor performance of the affected limb on the Jebsen Taylor Test (JTT). Stroke survivors were able to use Real neurofeedback to increase laterality of motor cortex activity within (P = 0.019), but not across, training days. There was no group effect on the primary behavioural outcome measure, which was average JTT performance across all subtasks (P = 0.116). Secondary analysis found improvements in the performance of the gross motor subtasks of the JTT in the Real neurofeedback group compared to Sham (P = 0.010). However, there were no improvements on the Action Research Arm Test or the Upper Extremity Fugl-Meyer score (both P > 0.5). Additionally, decreased white-matter asymmetry of the corticospinal tracts was detected 1 week after neurofeedback training (P = 0.008), indicating that the tracts become more similar with Real neurofeedback. Changes in the affected corticospinal tract were positively correlated with participants neurofeedback performance (P = 0.002). Therefore, here we demonstrate that chronic stroke survivors are able to use functional MRI neurofeedback to self-modulate motor cortex activity in comparison to a Sham control, and that training is associated with improvements in gross hand motor performance and with white matter structural changes.

Clinical neuroscience and neurotechnology: An amazing symbiosis

In the last decades, clinical neuroscience found a novel ally in neurotechnologies, devices able to record and stimulate electrical activity in the nervous system. These technologies improved the ability to diagnose and treat neural disorders. Neurotechnologies are concurrently enabling a deeper understanding of healthy and pathological dynamics of the nervous system through stimulation and recordings during brain implants. On the other hand, clinical neurosciences are not only driving neuroengineering toward the most relevant clinical issues, but are also shaping the neurotechnologies thanks to clinical advancements. For instance, understanding the etiology of a disease informs the location of a therapeutic stimulation, but also the way stimulation patterns should be designed to be more effective/naturalistic. Here, we describe cases of fruitful integration such as Deep Brain Stimulation and cortical interfaces to highlight how this symbiosis between clinical neuroscience and neurotechnology is closer to a novel integrated framework than to a simple interdisciplinary interaction.

Selective modulation of cortical population dynamics during neuroprosthetic skill learning

Brain-machine interfaces (BMIs) provide a framework for studying how cortical population dynamics evolve over learning in a task in which the mapping between neural activity and behavior is precisely defined. Learning to control a BMI is associated with the emergence of coordinated neural dynamics in populations of neurons whose activity serves as direct input to the BMI decoder (direct subpopulation). While previous work shows differential modification of firing rate modulation in this population relative to a population whose activity was not directly input to the BMI decoder (indirect subpopulation), little is known about how learning-related changes in cortical population dynamics within these groups compare.To investigate this, we monitored both direct and indirect subpopulations as two macaque monkeys learned to control a BMI. We found that while the combined population increased coordinated neural dynamics, this increase in coordination was primarily driven by changes in the direct subpopulation. These findings suggest that motor cortex refines cortical dynamics by increasing neural variance throughout the entire population during learning, with a more pronounced coordination of firing activity in subpopulations that are causally linked to behavior.

Entrainment to sleep spindles reflects dissociable patterns of connectivity between cortex and basal ganglia

Sleep spindles are crucial for learning in the cortex and basal ganglia (BG) because they facilitate the reactivation of previously active neuronal ensembles. Studying field potentials (FPs) and spiking in the cortex and BG during sleep in non-human primates following pre-sleep learning, we show that FP sleep spindles are widespread in the BG and are similar to cortical spindles in morphology, spectral content, and response to the pre-sleep task. Further, BG spindles are concordant with electroencephalogram (EEG) spindles and associated with increased cortico-BG correlation. However, spindles across the BG differ markedly in their entrainment of local spiking. The spiking activity of striatal projection neurons exhibits consistent phase locking to striatal and EEG spindles, producing phase windows of peaked cross-region spindling. In contrast, firing in other BG nuclei is not entrained to either local or EEG sleep spindles. These results suggest corticostriatal synapses as the main hub for offline cortico-BG communication.

Neurofeedback in patients with frontal brain lesions: A randomized, controlled double-blind trial

Background

Frontal brain dysfunction is a major challenge in neurorehabilitation. Neurofeedback (NF), as an EEG-based brain training method, is currently applied in a wide spectrum of mental health conditions, including traumatic brain injury.

Objective

This study aimed to explore the capacity of Infra-Low Frequency Neurofeedback (ILF-NF) to promote the recovery of brain function in patients with frontal brain injury.

Materials and methods

Twenty patients hospitalized at a neurorehabilitation clinic in Switzerland with recently acquired, frontal and optionally other brain lesions were randomized to either receive NF or sham-NF. Cognitive improvement was assessed using the Frontal Assessment Battery (FAB) and the Test of Attentional Performance (TAP) tasks regarding intrinsic alertness, phasic alertness and impulse control.

Results

With respect to cognitive improvements, there was no significant difference between the two groups after 20 sessions of either NF or sham-NF. However, in a subgroup of patients with predominantly frontal brain lesions, the improvements measured by the FAB and intrinsic alertness were significantly higher in the NF-group.

Conclusion

This is the first double-blind controlled study using NF in recovery from brain injury, and thus also the first such study of ILF NF. Although the result of the subgroup has limited significance because of the small number of participants, it accentuates the trend seen in the whole group regarding the FAB and intrinsic alertness (p = 0.068, p = 0.079, respectively). We therefore conclude that NF could be a promising candidate promoting the recoveryfrom frontal brain lesions. Further studies with larger numbers of patients and less lesion heterogeneity are needed to verify the usefulness of NF in the neurorehabilitation of patients with frontal brain injury (NCT02957695 ClinicalTrials.gov).

One session of fMRI-Neurofeedback training on motor imagery modulates whole-brain effective connectivity and dynamical complexity

In the past decade, several studies have shown that Neurofeedback (NFB) by functional magnetic resonance imaging can alter the functional coupling of targeted and non-targeted areas. However, the causal mechanisms underlying these changes remain uncertain. Here, we applied a whole-brain dynamical model to estimate Effective Connectivity (EC) profiles of resting-state data acquired before and immediately after a single-session NFB training for 17 participants who underwent motor imagery NFB training and 16 healthy controls who received sham feedback. Within-group and between-group classification analyses revealed that only for the NFB group it was possible to accurately discriminate between the 2 resting-state sessions. NFB training-related signatures were reflected in a support network of direct connections between areas involved in reward processing and implicit learning, together with regions belonging to the somatomotor, control, attention, and default mode networks, identified through a recursive-feature elimination procedure. By applying a data-driven approach to explore NFB-induced changes in spatiotemporal dynamics, we demonstrated that these regions also showed decreased switching between different brain states (i.e. metastability) only following real NFB training. Overall, our findings contribute to the understanding of NFB impact on the whole brain’s structure and function by shedding light on the direct connections between brain areas affected by NFB training.

Diverse operant control of different motor cortex populations during learning

During motor learning,1 as well as during neuroprosthetic learning,2-4 animals learn to control motor cortex activity in order to generate behavior. Two different populations of motor cortex neurons, intra-telencephalic (IT) and pyramidal tract (PT) neurons, convey the resulting cortical signals within and outside the telencephalon. Although a large amount of evidence demonstrates contrasting functional organization among both populations,5,6 it is unclear whether the brain can equally learn to control the activity of either class of motor cortex neurons. To answer this question, we used a calcium-imaging-based brain-machine interface (CaBMI)3 and trained different groups of mice to modulate the activity of either IT or PT neurons in order to receive a reward. We found that the animals learned to control PT neuron activity faster and better than IT neuron activity. Moreover, our findings show that the advantage of PT neurons is the result of characteristics inherent to this population as well as their local circuitry and cortical depth location. Taken together, our results suggest that the motor cortex is more efficient at controlling the activity of pyramidal tract neurons, which are embedded deep in the cortex, and relaying motor commands outside the telencephalon.

A behavioral paradigm for cortical control of a robotic actuator by freely moving rats in a one-dimensional two-target reaching task

BACKGROUND

Controlling the trajectory of a neuroprosthesis to reach distant targets is a commonly used brain-machine interface (BMI) task in primates and has not been available for rodents yet.

NEW METHOD

Here, we describe a novel, fine-tuned behavioral paradigm and setup which enables this task for rats in one-dimensional space for reaching two distant targets depending on their limited cognitive and visual capabilities compared to those of primates. An online transform was used to convert the activity of a pair of primary motor cortex (M1) units into two robotic actions. The rats were shaped to adapt to the transform and direct the robotic actuator toward the selected target by modulating the activity of the M1 neurons.

RESULTS

All three rats involved in the study were capable of achieving randomly selected targets with at least 78% accuracy. A total of 9 out of 16 pairs of units examined were eligible for exceeding this success criterion. Two out of three rats were capable of reversal learning, where the mapping between the activity of the M1 units and the robotic actions were reversed.

COMPARISON WITH EXISTING METHODS

The present work is the first demonstration of trajectory-based control of a neuroprosthetic device by rodents to reach two distant targets using visual feedback.

CONCLUSION

The behavioral paradigm and setup introduced here can be used as a cost-effective platform for elucidating the information processing principles in the neural circuits related to neuroprosthetic control and for studying the performance of novel BMI technologies using freely moving rats.

Investigation of Neural Substrates of Erroneous Behavior in a Delayed-Response Task

Motor cortical neurons exhibit persistent selective activities (selectivity) during motor planning. Experimental perturbation of selectivity results in the failure of short-term memory retention and consequent behavioral biases, demonstrating selectivity as a neural characteristic of encoding previous sensory input or future action. However, even without experimental manipulation, animals occasionally fail to maintain short-term memory leading to erroneous choice. Here, we investigated neural substrates that lead to the incorrect formation of selectivity during short-term memory. We analyzed neuronal activities in anterior lateral motor cortex (ALM) of mice, a region known to be engaged in motor planning while mice performed the tactile delayed-response task. We found that highly selective neurons lost their selectivity while originally nonselective neurons showed selectivity during the error trials where mice licked toward incorrect direction. We assumed that those alternations would reflect changes in intrinsic properties of population activity. Thus, we estimated an intrinsic manifold shared by neuronal population (shared space), using factor analysis (FA) and measured the association of individual neurons with the shared space by communality, the variance of neuronal activity accounted for by the shared space. We found a positive correlation between selectivity and communality over ALM neurons, which disappeared in erroneous behavior. Notably, neurons showing selectivity alternations between correct and incorrect licking also underwent proportional changes in communality. Our results demonstrated that the extent to which an ALM neuron is associated with the intrinsic manifolds of population activity may elucidate its selectivity and that disruption of this association may alter selectivity, likely leading to erroneous behavior.

Development of an Ergonomic User Interface Design of Calcium Imaging Processing System

An optical brain-machine interface (O-BMI) system using calcium imaging has various advantages such as high resolution, a comprehensive view of large neural populations, abilities such as long-term stable recording, and applicability to freely behaving animals in neuroscience research. The present study developed an ergonomic user interface (UI) design, based on a use scenario for an O-BMI system that can be used for the acquisition and processing of calcium imaging in freely behaving rodents. The UI design was developed in three steps: (1) identification of design and function requirements of users, (2) establishment of a use scenario, and (3) development of a UI prototype. The UI design requirements were identified by a literature review, a benchmark of existing systems, and a focus group interview with five neuroscience researchers. Then, the use scenario was developed for tasks of data acquisition, feature extraction, and neural decoding for offline and online processing by considering the sequences of operations and needs of users. Lastly, a digital prototype incorporating an information architecture, graphic user interfaces, and simulated functions was fabricated. A usability test was conducted with five neuroscientists (work experience = 3.4 ± 1.1 years) and five ergonomic experts (work experience = 3.6 ± 2.7 years) to compare the digital prototypes with four existing systems (Miniscope, nVista, Mosaic, and Suite2p). The usability testing results showed that the ergonomic UI design was significantly preferred to the UI designs of the existing systems by reducing the task completion time by 10.1% to 70.2% on average, the scan path length by 14.4% to 88.7%, and perceived workload by 12.2% to 37.9%, increasing satisfaction by 11.3% to 74.3% in data acquisition and signal-extraction tasks. The present study demonstrates the significance of the user-centered design approach in the development of a system for neuroscience research. Further research is needed to validate the usability test results of the UI prototype as a corresponding real system is implemented.

Volitional control of individual neurons in the human brain

Brain-machine interfaces allow neuroscientists to causally link specific neural activity patterns to a particular behaviour. Thus, in addition to their current clinical applications, brain-machine interfaces can also be used as a tool to investigate neural mechanisms of learning and plasticity in the brain. Decades of research using such brain-machine interfaces have shown that animals (non-human primates and rodents) can be operantly conditioned to self-regulate neural activity in various motor-related structures of the brain. Here, we ask whether the human brain, a complex interconnected structure of over 80 billion neurons, can learn to control itself at the most elemental scale-a single neuron. We used the unique opportunity to record single units in 11 individuals with epilepsy to explore whether the firing rate of a single (direct) neuron in limbic and other memory-related brain structures can be brought under volitional control. To do this, we developed a visual neurofeedback task in which participants were trained to move a block on a screen by modulating the activity of an arbitrarily selected neuron from their brain. Remarkably, participants were able to volitionally modulate the firing rate of the direct neuron in these previously uninvestigated structures. We found that a subset of participants (learners), were able to improve their performance within a single training session. Successful learning was characterized by (i) highly specific modulation of the direct neuron (demonstrated by significantly increased firing rates and burst frequency); (ii) a simultaneous decorrelation of the activity of the direct neuron from the neighbouring neurons; and (iii) robust phase-locking of the direct neuron to local alpha/beta-frequency oscillations, which may provide some insights in to the potential neural mechanisms that facilitate this type of learning. Volitional control of neuronal activity in mnemonic structures may provide new ways of probing the function and plasticity of human memory without exogenous stimulation. Furthermore, self-regulation of neural activity in these brain regions may provide an avenue for the development of novel neuroprosthetics for the treatment of neurological conditions that are commonly associated with pathological activity in these brain structures, such as medically refractory epilepsy.

Basal Forebrain Cholinergic Neurons Selectively Drive Coordinated Motor Learning in Mice

Motor control requires precise temporal and spatial encoding across distinct motor centers that is refined through the repetition of learning. The recruitment of motor regions requires modulatory input to shape circuit activity. Here, we identify a role for the baso-cortical cholinergic pathway in the acquisition of a coordinated motor skill in mice. Targeted depletion of basal forebrain cholinergic neurons results in significant impairments in training on the rotarod task of coordinated movement. Cholinergic neuromodulation is required during training sessions as chemogenetic inactivation of cholinergic neurons also impairs task acquisition. Rotarod learning is known to drive refinement of corticostriatal neurons arising in both medial prefrontal cortex (mPFC) and motor cortex, and we have found that cholinergic input to both motor regions is required for task acquisition. Critically, the effects of cholinergic neuromodulation are restricted to the acquisition stage, as depletion of basal forebrain cholinergic neurons after learning does not affect task execution. Our results indicate a critical role for cholinergic neuromodulation of distant cortical motor centers during coordinated motor learning.SIGNIFICANCE STATEMENT Acetylcholine release from basal forebrain cholinergic neuron terminals rapidly modulates neuronal excitability, circuit dynamics, and cortical coding; all processes required for processing complex sensory information, cognition, and attention. We found that depletion or transient silencing of cholinergic inputs to anatomically isolated motor areas, medial prefrontal cortex (mPFC) and motor cortex, selectively led to significant impairments on coordinated motor learning; disrupting this baso-cortical network after acquisition elicited no effect on task execution. Our results indicate a pivotal role for cholinergic neuromodulation of distant cortical motor centers during coordinated motor learning. These findings support the concept that cognitive components (such as attention) are indispensable in the adjustment of motor output and training-induced improvements in motor performance.

Timescales of Local and Cross-Area Interactions during Neuroprosthetic Learning

How does the brain integrate signals with different timescales to drive purposeful actions? Brain-machine interfaces (BMIs) offer a powerful tool to causally test how distributed neural networks achieve specific neural patterns. During neuroprosthetic learning, actuator movements are causally linked to primary motor cortex (M1) neurons, i.e., "direct" neurons that project to the decoder and whose firing is required to successfully perform the task. However, it is unknown how such direct M1 neurons interact with both "indirect" local (in M1 but not part of the decoder) and across area neural populations (e.g., in premotor cortex/M2), all of which are embedded in complex biological recurrent networks. Here, we trained male rats to perform a M1-BMI task and simultaneously recorded the activity of indirect neurons in both M2 and M1. We found that both M2 and M1 indirect neuron populations could be used to predict the activity of the direct neurons (i.e., "BMI-potent activity"). Interestingly, compared with M1 indirect activity, M2 neural activity was correlated with BMI-potent activity across a longer set of time lags, and the timescale of population activity patterns evolved more slowly. M2 units also predicted the activity of both M1 direct and indirect neural populations, suggesting that M2 population dynamics provide a continuous modulatory influence on M1 activity as a whole, rather than a moment-by-moment influence solely on neurons most relevant to a task. Together, our results indicate that longer timescale M2 activity provides modulatory influence over extended time lags on shorter-timescale control signals in M1.SIGNIFICANCE STATEMENT A central question in the study of motor control is whether primary motor cortex (M1) and premotor cortex (M2) interact through task-specific subpopulations of neurons, or whether tasks engage broader correlated networks. Brain-machine interfaces (BMIs) are powerful tools to study cross-area interactions. Here, we performed simultaneous recordings of M1 and M2 in a BMI task using a subpopulation of M1 neurons (direct neurons). We found that activity outside of direct neurons in M1 and M2 was predictive of M1-BMI task activity, and that M2 activity evolved at slower timescales than M1. These findings suggest that M2 provides a continuous modulatory influence on M1 as a whole, supporting a model of interactions through broad correlated networks rather than task-specific neural subpopulations.

Entrainment of Astrocytic and Neuronal Ca2+ Population Dynamics During Information Processing of Working Memory in Mice

Astrocytes are increasingly recognized to play an active role in learning and memory, but whether neural inputs can trigger event-specific astrocytic Ca²⁺ dynamics in real time to participate in working memory remains unclear due to the difficulties in directly monitoring astrocytic Ca²⁺ dynamics in animals performing tasks. Here, using fiber photometry, we showed that population astrocytic Ca²⁺ dynamics in the hippocampus were gated by sensory inputs (centered at the turning point of the T-maze) and modified by the reward delivery during the encoding and retrieval phases. Notably, there was a strong inter-locked and antagonistic relationship between the astrocytic and neuronal Ca²⁺ dynamics with a 3-s phase difference. Furthermore, there was a robust synchronization of astrocytic Ca²⁺ at the population level among the hippocampus, medial prefrontal cortex, and striatum. The inter-locked, bidirectional communication between astrocytes and neurons at the population level may contribute to the modulation of information processing in working memory.

Distributed cell assemblies spanning prefrontal cortex and striatum

Highly synchronous neuronal assembly activity is deemed essential for cognitive brain function. In theory, such synchrony could coordinate multiple brain areas performing complementary processes. However, cell assemblies have been observed only in single structures, typically cortical areas, and little is known about their synchrony with downstream subcortical structures, such as the striatum. Here, we demonstrate distributed cell assemblies activated at high synchrony (∼10 ms) spanning prefrontal cortex and striatum. In addition to including neurons at different brain hierarchical levels, surprisingly, they synchronized functionally distinct limbic and associative sub-regions. These assembly activations occurred when members shifted their firing phase relative to ongoing 4 Hz and theta rhythms, in association with high gamma oscillations. This suggests that these rhythms could mediate the emergence of cross-structural assemblies. To test for the role of assemblies in behavior, we trained the rats to perform a task requiring cognitive flexibility, alternating between two different rules in a T-maze. Overall, assembly activations were correlated with task-relevant parameters, including impending choice, reward, rule, or rule order. Moreover, these behavioral correlates were more robustly expressed by assemblies than by their individual member neurons. Finally, to verify whether assemblies can be endogenously generated, we found that they were indeed spontaneously reactivated during sleep and quiet immobility. Thus, cell assemblies are a more general coding mechanism than previously envisioned, linking distributed neocortical and subcortical areas at high synchrony.

Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life

The rapid evolution of Brain-Computer Interface (BCI) technology and the exponential growth of BCI literature during the past 20 years is a consequence of increasing computational power and the achievements of statistical learning theory and machine learning since the 1960s. Despite this rapid scientific progress, the range of successful clinical and societal applications remained limited, with some notable exceptions in the rehabilitation of chronic stroke and first steps towards BCI-based assisted verbal communication in paralysis. In this contribution, we focus on the effects of noninvasive and invasive BCI-based verbal communication on the quality of life (QoL) of patients with amyotrophic lateral sclerosis (ALS) in the locked-in state (LIS) and the completely locked-in state (CLIS). Despite a substantial lack of replicated scientific data, this paper complements the existing methodological knowledge and focuses future investigators' attention on (1) Social determinants of QoL and (2) Brain reorganization and behavior. While it is not documented in controlled studies that the good QoL in these patients is a consequence of BCI-based neurorehabilitation, the proposed determinants of QoL might become the theoretical background needed to develop clinically more useful BCI systems and to evaluate the effects of BCI-based communication on QoL for advanced ALS patients and other forms of severe paralysis.

Coupling between motor cortex and striatum increases during sleep over long-term skill learning

The strength of cortical connectivity to the striatum influences the balance between behavioral variability and stability. Learning to consistently produce a skilled action requires plasticity in corticostriatal connectivity associated with repeated training of the action. However, it remains unknown whether such corticostriatal plasticity occurs during training itself or 'offline' during time away from training, such as sleep. Here, we monitor the corticostriatal network throughout long-term skill learning in rats and find that non-rapid-eye-movement (NREM) sleep is a relevant period for corticostriatal plasticity. We first show that the offline activation of striatal NMDA receptors is required for skill learning. We then show that corticostriatal functional connectivity increases offline, coupled to emerging consistent skilled movements, and coupled cross-area neural dynamics. We then identify NREM sleep spindles as uniquely poised to mediate corticostriatal plasticity, through interactions with slow oscillations. Our results provide evidence that sleep shapes cross-area coupling required for skill learning.

Concomitant Processing of Choice and Outcome in Frontal Corticostriatal Ensembles Correlates with Performance of Rats

The frontal cortex-basal ganglia network plays a pivotal role in adaptive goal-directed behaviors. Medial frontal cortex (MFC) encodes information about choices and outcomes into sequential activation of neural population, or neural trajectory. While MFC projects to the dorsal striatum (DS), whether DS also displays temporally coordinated activity remains unknown. We studied this question by simultaneously recording neural ensembles in the MFC and DS of rodents performing an outcome-based alternative choice task. We found that the two regions exhibited highly parallel evolution of neural trajectories, transforming choice information into outcome-related information. When the two trajectories were highly correlated, spike synchrony was task-dependently modulated in some MFC-DS neuron pairs. Our results suggest that neural trajectories concomitantly process decision-relevant information in MFC and DS with increased spike synchrony between these regions.

Coordinated increase of reliable cortical and striatal ensemble activations during recovery after stroke

Skilled movements rely on a coordinated cortical and subcortical network, but how this network supports motor recovery after stroke is unknown. Previous studies focused on the perilesional cortex (PLC), but precisely how connected subcortical areas reorganize and coordinate with PLC is unclear. The dorsolateral striatum (DLS) is of interest because it receives monosynaptic inputs from motor cortex and is important for learning and generation of fast reliable actions. Using a rat focal stroke model, we perform chronic electrophysiological recordings in motor PLC and DLS during long-term recovery of a dexterous skill. We find that recovery is associated with the simultaneous emergence of reliable movement-related single-trial ensemble spiking in both structures along with increased cross-area alignment of spiking. Our study highlights the importance of consistent neural activity patterns across brain structures during recovery and suggests that modulation of cross-area coordination can be a therapeutic target for enhancing motor function post-stroke.

Interpreting the role of the striatum during multiple phases of motor learning

The synaptic pathways in the striatum are central to basal ganglia functions including motor control, learning and organization, action selection, acquisition of motor skills, cognitive function, and emotion. Here, we review the role of the striatum and its connections in motor learning and performance. The development of new techniques to record neuronal activity and animal models of motor disorders using neurotoxin, pharmacological, and genetic manipulations are revealing pathways that underlie motor performance and motor learning, as well as how they are altered by pathophysiological mechanisms. We discuss approaches that can be used to analyze complex motor skills, particularly in rodents, and identify specific questions central to understanding how striatal circuits mediate motor learning.

Dorsal striatum and the temporal expectancy of an aversive event in Pavlovian odor fear learning

Interval timing, the ability to encode and retrieve the memory of intervals from seconds to minutes, guides fundamental animal behaviors across the phylogenetic tree. In Pavlovian fear conditioning, an initially neutral stimulus (conditioned stimulus, CS) predicts the arrival of an aversive unconditioned stimulus (US, generally a mild foot-shock) at a fixed time interval. Although some studies showed that temporal relations between CS and US events are learned from the outset of conditioning, the question of the memory of time and its underlying neural network in fear conditioning is still poorly understood. The aim of the present study was to investigate the role of the dorsal striatum in timing intervals in odor fear conditioning in male rats. To assess the animal's interval timing ability in this paradigm, we used the respiratory frequency. This enabled us to detect the emergence of temporal patterns related to the odor-shock time interval from the early stage of learning, confirming that rats are able to encode the odor-shock time interval after few training trials. We carried out reversible inactivation of the dorsal striatum before the acquisition session and before a shift in the learned time interval, and measured the effects of this treatment on the temporal pattern of the respiratory rate. In addition, using intracerebral microdialysis, we monitored extracellular dopamine level in the dorsal striatum throughout odor-shock conditioning and in response to a shift of the odor-shock time interval. Contrary to our initial predictions based on the existing literature on interval timing, we found evidence suggesting that transient inactivation of the dorsal striatum may favor a more precocious buildup of the respiratory frequency's temporal pattern during the odor-shock interval in a manner that reflected the duration of the interval. Our data further suggest that the conditioning and the learning of a novel time interval were associated with a decrease in dopamine level in the dorsal striatum, but not in the nucleus accumbens. These findings prompt a reassessment of the role of the striatum and striatal dopamine in interval timing, at least when considering Pavlovian aversive conditioning.

The role of the anterior nuclei of the thalamus in human memory processing

Extensive neuroanatomical connectivity between the anterior thalamic nuclei (ATN) and hippocampus and neocortex renders them well-placed for a role in memory processing, and animal, lesion, and neuroimaging studies support such a notion. The deep location and small size of the ATN have precluded their real-time electrophysiological investigation during human memory processing. However, ATN electrophysiological recordings from patients receiving electrodes implanted for deep brain stimulation for pharmacoresistant focal epilepsy have enabled high temporal resolution study of ATN activity. Theta frequency synchronization of ATN and neocortical oscillations during successful memory encoding, enhanced phase alignment, and coupling between ATN local gamma frequency activity and frontal neocortical and ATN theta oscillations provide evidence of an active role for the ATN in memory encoding, potentially integrating information from widespread neocortical sources. Greater coupling of a broader gamma frequency range with theta oscillations at rest than during memory encoding provides additional support for the hypothesis that the ATN play a role in selecting local, task-relevant high frequency activity associated with particular features of a memory trace.

Striatal activity topographically reflects cortical activity

The cortex projects to the dorsal striatum topographically^1,2 to regulate behaviour^3-5, but spiking activity in the two structures has previously been reported to have markedly different relations to sensorimotor events^6-9. Here we show that the relationship between activity in the cortex and striatum is spatiotemporally precise, topographic, causal and invariant to behaviour. We simultaneously recorded activity across large regions of the cortex and across the width of the dorsal striatum in mice that performed a visually guided task. Striatal activity followed a mediolateral gradient in which behavioural correlates progressed from visual cue to response movement to reward licking. The summed activity in each part of the striatum closely and specifically mirrored activity in topographically associated cortical regions, regardless of task engagement. This relationship held for medium spiny neurons and fast-spiking interneurons, whereas the activity of tonically active neurons differed from cortical activity with stereotypical responses to sensory or reward events. Inactivation of the visual cortex abolished striatal responses to visual stimuli, supporting a causal role of cortical inputs in driving the striatum. Striatal visual responses were larger in trained mice than untrained mice, with no corresponding change in overall activity in the visual cortex. Striatal activity therefore reflects a consistent, causal and scalable topographical mapping of cortical activity.

A behavioral paradigm for cortical control of a robotic actuator by freely moving rats in a one-dimensional two-target reaching task.. [DOI: 10.1101/2021.01.24.427325] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Controlling the trajectory of a neuroprosthetic device to reach multiple targets is a commonly used brain-machine interface (BMI) task in primates and has not been available for rodents yet. Here, we describe a novel experimental paradigm which enables this task for rats in one-dimensional space for reaching two distant targets depending on their limited cognitive and visual capabilities compared to primates. An online transform was used to convert the activity of a pair of primary motor cortex (M1) units into two robotic actions. The rats were shaped to adapt to the transform and direct the robotic actuator toward the selected target by modulating the activity of the M1 neurons. All three rats involved in the study were capable of achieving randomly selected targets with at least 78% accuracy. A total of 9 out of 16 pairs of units examined were eligible for exceeding this success criterion. Two out of three rats were capable of reversal learning, where the mapping between the activity of the unit pairs and the robotic actions were reversed. The present work is the first demonstration of trajectory-based control of a neuroprosthetic device by rodents to reach two distant targets using visual feedback. The paradigm introduced here may be used as a cost-effective platform for elucidating the information processing principles in the neural circuits related to neuroprosthetic control and for studying the performance of novel BMI technologies using freely moving rats.

Adenosine A2A receptor blockade improves neuroprosthetic learning by volitional control of population calcium signal in M1 cortical neurons

Volitional control is at the core of brain-machine interfaces (BMI) adaptation and neuroprosthetic-driven learning to restore motor function for disabled patients, but neuroplasticity changes and neuromodulation underlying volitional control of neuroprosthetic learning are largely unexplored. To better study volitional control at annotated neural population, we have developed an operant neuroprosthetic task with closed-loop feedback system by volitional conditioning of population calcium signal in the M1 cortex using fiber photometry recording. Importantly, volitional conditioning of the population calcium signal in M1 neurons did not improve within-session adaptation, but specifically enhanced across-session neuroprosthetic skill learning with reduced time-to-target and the time to complete 50 successful trials. With brain-behavior causality of the neuroprosthetic paradigm, we revealed that proficiency of neuroprosthetic learning by volitional conditioning of calcium signal was associated with the stable representational (plasticity) mapping in M1 neurons with the reduced calcium peak. Furthermore, pharmacological blockade of adenosine A_2A receptors facilitated volitional conditioning of neuroprosthetic learning and converted an ineffective volitional conditioning protocol to be the effective for neuroprosthetic learning. These findings may help to harness neuroplasticity for better volitional control of neuroprosthetic training and suggest a novel pharmacological strategy to improve neuroprosthetic learning in BMI adaptation by targeting striatal A_2A receptors.

Neural reinforcement: re-entering and refining neural dynamics leading to desirable outcomes

How do organisms learn to do again, on-demand, a behavior that led to a desirable outcome? Dopamine-dependent cortico-striatal plasticity provides a framework for learning behavior's value, but it is less clear how it enables the brain to re-enter desired behaviors and refine them over time. Reinforcing behavior is achieved by re-entering and refining the neural patterns that produce it. We review studies using brain-machine interfaces which reveal that reinforcing cortical population activity requires cortico-basal ganglia circuits. Then, we propose a formal framework for how reinforcement in cortico-basal ganglia circuits acts on the neural dynamics of cortical populations. We propose two parallel mechanisms: i) fast reinforcement which selects the inputs that permit the re-entrance of the particular cortical population dynamics which naturally produced the desired behavior, and ii) slower reinforcement which leads to refinement of cortical population dynamics and more reliable production of neural trajectories driving skillful behavior on-demand.

Aberrant features of in vivo striatal dynamics in Parkinson's disease

The striatum plays an important role in learning, selecting, and executing actions. As a major input hub of the basal ganglia, it receives and processes a diverse array of signals related to sensory, motor, and cognitive information. Aberrant neural activity in this area is implicated in a wide variety of neurological and psychiatric disorders. It is therefore important to understand the hallmarks of disrupted striatal signal processing. This review surveys literature examining how in vivo striatal microcircuit dynamics are impacted in animal models of one of the most widely studied movement disorders, Parkinson's disease. The review identifies four major features of aberrant striatal dynamics: altered relative levels of direct and indirect pathway activity, impaired information processing by projection neurons, altered information processing by interneurons, and increased synchrony.

Gain Modulation by Corticostriatal and Thalamostriatal Input Signals during Reward-Conditioned Behavior

The cortex and thalamus send excitatory projections to the striatum, but little is known about how these inputs, either individually or collectively, regulate striatal dynamics during behavior. The lateral striatum receives overlapping input from the secondary motor cortex (M2), an area involved in licking, and the parafascicular thalamic nucleus (PF). Using neural recordings, together with optogenetic terminal inhibition, we examine the contribution of M2 and PF projections on medium spiny projection neuron (MSN) activity as mice performed an anticipatory licking task. Each input has a similar contribution to striatal activity. By comparing how suppressing single or multiple projections altered striatal activity, we find that cortical and thalamic input signals modulate MSN gain and that this effect is more pronounced in a temporally specific period of the task following the cue presentation. These results demonstrate that cortical and thalamic inputs synergistically regulate striatal output during reward-conditioned behavior.

Near Perfect Neural Critic from Motor Cortical Activity Toward an Autonomously Updating Brain Machine Interface

We are developing an autonomously updating brain machine interface (BMI) utilizing reinforcement learning principles. One aspect of this system is a neural critic that determines reward expectations from neural activity. This critic is then used to update a BMI decoder toward an improved performance from the user's perspective. Here we demonstrate the ability of a neural critic to classify trial reward value given activity from the primary motor cortex (M1), using neural features from single/multi units (SU/MU), and local field potentials (LFPs) with prediction accuracies up to 97% correct. A nonhuman primate subject conducted a cued center out reaching task, either manually, or observationally. The cue indicated the reward value of a trial. Features such as power spectral density (PSD) of the LFPs and spike-field coherence (SFC) between SU/MU and corresponding LFPs were calculated and used as inputs to several classifiers. We conclude that hybrid features of PSD and SFC show higher classification performance than PSD or SFC alone (accuracy was 92% for manual tasks, and 97% for observational). In the future, we will employ these hybrid features toward our autonomously updating BMI.

Reward Expectation Modulates Local Field Potentials, Spiking Activity and Spike-Field Coherence in the Primary Motor Cortex

Reward modulation (M1) could be exploited in developing an autonomously updating brain-computer interface (BCI) based on a reinforcement learning (RL) architecture. For an autonomously updating RL-based BCI system, we would need a reward prediction error, or a state-value representation from the user's neural activity, which the RL-BCI agent could use to update its BCI decoder. In order to understand the multifaceted effects of reward on M1 activity, we investigated how neural spiking, oscillatory activities and their functional interactions are modulated by conditioned stimuli related reward expectation. To do so, local field potentials (LFPs) and single/multi-unit activities were recorded simultaneously and bilaterally from M1 cortices while four non-human primates (NHPs) performed cued center-out reaching or grip force tasks either manually using their right arm/hand or observed passively. We found that reward expectation influenced the strength of α (8-14 Hz) power, α-γ comodulation, α spike-field coherence (SFC), and firing rates (FRs) in general in M1. Furthermore, we found that an increase in α-band power was correlated with a decrease in neural spiking activity, that FRs were highest at the trough of the α-band cycle and lowest at the peak of its cycle. These findings imply that α oscillations modulated by reward expectation have an influence on spike FR and spike timing during both reaching and grasping tasks in M1. These LFP, spike, and spike-field interactions could be used to follow the M1 neural state in order to enhance BCI decoding (An et al., 2018; Zhao et al., 2018).

Bridging the gap between striatal plasticity and learning

The striatum, the main input nucleus of the basal ganglia, controls goal-directed behavior and procedural learning. Striatal projection neurons integrate glutamatergic inputs from cortex and thalamus together with neuromodulatory systems, and are subjected to plasticity. Striatal projection neurons exhibit bidirectional plasticity (LTP and LTD) when exposed to Hebbian paradigms. Importantly, correlative and even causal links between procedural learning and striatal plasticity have recently been shown. This short review summarizes the current view on striatal plasticity (with a focus on spike-timing-dependent plasticity), recent studies aiming at bridging in vivo skill acquisition and striatal plasticity, the temporal credit-assignment problem, and the gaps that remain to be filled.