Neural mechanisms of reward-related motor learning

Sensory encoding and memory in the mushroom body: signals, noise, and variability

To survive in changing environments, animals need to learn to associate specific sensory stimuli with positive or negative valence. How do they form stimulus-specific memories to distinguish between positively/negatively associated stimuli and other irrelevant stimuli? Solving this task is one of the functions of the mushroom body, the associative memory center in insect brains. Here we summarize recent work on sensory encoding and memory in the Drosophila mushroom body, highlighting general principles such as pattern separation, sparse coding, noise and variability, coincidence detection, and spatially localized neuromodulation, and placing the mushroom body in comparative perspective with mammalian memory systems.

Novel voxelwise residual analysis of [11C]raclopride PET data improves detection of low-amplitude dopamine release

Existing methods for voxelwise transient dopamine (DA) release detection rely on explicit kinetic modeling of the [11C]raclopride PET time activity curve, which at the voxel level is typically confounded by noise, leading to poor performance for detection of low-amplitude DA release-induced signals. Here we present a novel data-driven, task-informed method-referred to as Residual Space Detection (RSD)-that transforms PET time activity curves to a residual space where DA release-induced perturbations can be isolated and processed. Using simulations, we demonstrate that this method significantly increases detection performance compared to existing kinetic model-based methods for low-magnitude DA release (simulated +100% peak increase in basal DA concentration). In addition, results from nine healthy controls injected with a single bolus of [11C]raclopride performing a finger tapping motor task are shown as proof-of-concept. The ability to detect relatively low magnitudes of dopamine release in the human brain using a single bolus injection, while achieving higher statistical power than previous methods, may additionally enable more complex analyses of neurotransmitter systems. Moreover, RSD is readily generalizable to multiple tasks performed during a single PET scan, further extending the capabilities of task-based single-bolus protocols.

Balancing risk-return decisions by manipulating the mesofrontal circuits in primates

Decision-making is always coupled with some level of risk, with more pathological forms of risk-taking decisions manifesting as gambling disorders. In macaque monkeys trained in a high risk-high return (HH) versus low risk-low return (LL) choice task, we found that the reversible pharmacological inactivation of ventral Brodmann area 6 (area 6V) impaired the risk dependency of decision-making. Selective optogenetic activation of the mesofrontal pathway from the ventral tegmental area (VTA) to the ventral aspect of 6V resulted in stronger preference for HH, whereas activation of the pathway from the VTA to the dorsal aspect of 6V led to LL preference. Finally, computational decoding captured the modulations of behavioral preference. Our results suggest that VTA inputs to area 6V determine the decision balance between HH and LL.

Implicit Selective Attention: The Role of the Mesencephalic-basal Ganglia System

The ability of the brain to recognize and orient attention to relevant stimuli appearing in the visual field is highlighted by a tuning process, which involves modulating the early visual system by both cortical and subcortical brain areas. Selective attention is coordinated not only by the output of stimulus-based saliency maps but is also influenced by top-down cognitive factors, such as internal states, goals, or previous experiences. The basal ganglia system plays a key role in implicitly modulating the underlying mechanisms of selective attention, favouring the formation and maintenance of implicit sensory-motor memories that are capable of automatically modifying the output of priority maps in sensory-motor structures of the midbrain, such as the superior colliculus. The article presents an overview of the recent literature outlining the crucial contribution of several subcortical structures to the processing of different sources of salient stimuli. In detail, we will focus on how the mesencephalic- basal ganglia closed loops contribute to implicitly addressing and modulating selective attention to prioritized stimuli. We conclude by discussing implicit behavioural responses observed in clinical populations in which awareness is compromised at some level. Implicit (emergent) awareness in clinical conditions that can be accompanied by manifest anosognosic symptomatology (i.e., hemiplegia) or involving abnormal conscious processing of visual information (i.e., unilateral spatial neglect and blindsight) represents interesting neurocognitive "test cases" for inferences about mesencephalicbasal ganglia closed-loops involvement in the formation of implicit sensory-motor memories.

Reward expectations direct learning and drive operant matching in Drosophila

Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein's operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly's sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.

DRD1 signaling modulates TrkB turnover and BDNF sensitivity in direct pathway striatal medium spiny neurons

Disturbed motor control is a hallmark of Parkinson's disease (PD). Cortico-striatal synapses play a central role in motor learning and adaption, and brain-derived neurotrophic factor (BDNF) from cortico-striatal afferents modulates their plasticity via TrkB in striatal medium spiny projection neurons (SPNs). We studied the role of dopamine in modulating the sensitivity of direct pathway SPNs (dSPNs) to BDNF in cultures of fluorescence-activated cell sorting (FACS)-enriched D1-expressing SPNs and 6-hydroxydopamine (6-OHDA)-treated rats. DRD1 activation causes enhanced TrkB translocation to the cell surface and increased sensitivity for BDNF. In contrast, dopamine depletion in cultured dSPN neurons, 6-OHDA-treated rats, and postmortem brain of patients with PD reduces BDNF responsiveness and causes formation of intracellular TrkB clusters. These clusters associate with sortilin related VPS10 domain containing receptor 2 (SORCS-2) in multivesicular-like structures, which apparently protects them from lysosomal degradation. Thus, impaired TrkB processing might contribute to disturbed motor function in PD.

Virtual reality-based gait rehabilitation intervention for stroke individuals: a scoping review

Virtual reality (VR)-based rehabilitation is rapidly gaining interest and has been shown to be an intervention to facilitate motor learning in balance and gait rehabilitation. A review of the current literature is needed to provide an overview of the current state of knowledge of VR-based gait physiotherapy for stroke patients. A systematic literature search was performed in PubMed and Scopus. Search terms included: "virtual reality," "stroke," "gait," and "physical therapy." Articles published in a peer-reviewed journal between 2017 and 2021 were considered. The intervention was mainly related to the use of VR as a therapeutic modality, and the outcome was gait performance. The initial search identified 329 articles. After an eligibility review, 13 articles that met the inclusion criteria were included in the study. Most of participants were in a chronic stage and were between 14 and 85 years old. The VR-based gait training ranged from nonimmersive to immersive, was mostly performed on a treadmill, and was usually combined with conventional physiotherapy. The duration of the program varied from 10 to 60 min, and there were about 9 to 30 sessions. VR-based gait rehabilitation has a positive effect on gait ability. The existing literature suggests that VR-based rehabilitation combined with conventional physiotherapy could improve gait ability of people with stroke, especially in the chronic stage. However, the duration of VR-based programs should be customized to suit individuals to avoid stimulation sickness. Further research is needed to investigate the long-term effects of this approach.

R1441C and G2019S LRRK2 knockin mice have distinct striatal molecular, physiological, and behavioral alterations

LRRK2 mutations are closely associated with Parkinson's disease (PD). Convergent evidence suggests that LRRK2 regulates striatal function. Here, by using knock-in mouse lines expressing the two most common LRRK2 pathogenic mutations-G2019S and R1441C-we investigated how LRRK2 mutations altered striatal physiology. While we found that both R1441C and G2019S mice displayed reduced nigrostriatal dopamine release, hypoexcitability in indirect-pathway striatal projection neurons, and alterations associated with an impaired striatal-dependent motor learning were observed only in the R1441C mice. We also showed that increased synaptic PKA activities in the R1441C and not G2019S mice underlie the specific alterations in motor learning deficits in the R1441C mice. In summary, our data argue that LRRK2 mutations' impact on the striatum cannot be simply generalized. Instead, alterations in electrochemical, electrophysiological, molecular, and behavioral levels were distinct between LRRK2 mutations. Our findings offer mechanistic insights for devising and optimizing treatment strategies for PD patients.

Motor constellation theory: A model of infants' phonological development

Every normally developing human infant solves the difficult problem of mapping their native-language phonology, but the neural mechanisms underpinning this behavior remain poorly understood. Here, motor constellation theory, an integrative neurophonological model, is presented, with the goal of explicating this issue. It is assumed that infants' motor-auditory phonological mapping takes place through infants' orosensory "reaching" for phonological elements observed in the language-specific ambient phonology, via reference to kinesthetic feedback from motor systems (e.g., articulators), and auditory feedback from resulting speech and speech-like sounds. Attempts are regulated by basal ganglion-cerebellar speech neural circuitry, and successful attempts at reproduction are enforced through dopaminergic signaling. Early in life, the pace of anatomical development constrains mapping such that complete language-specific phonological mapping is prohibited by infants' undeveloped supralaryngeal vocal tract and undescended larynx; constraints gradually dissolve with age, enabling adult phonology. Where appropriate, reference is made to findings from animal and clinical models. Some implications for future modeling and simulation efforts, as well as clinical settings, are also discussed.

Vagus nerve stimulation drives selective circuit modulation through cholinergic reinforcement

Vagus nerve stimulation (VNS) is a neuromodulation therapy for a broad and expanding set of neurologic conditions. However, the mechanism through which VNS influences central nervous system circuitry is not well described, limiting therapeutic optimization. VNS leads to widespread brain activation, but the effects on behavior are remarkably specific, indicating plasticity unique to behaviorally engaged neural circuits. To understand how VNS can lead to specific circuit modulation, we leveraged genetic tools including optogenetics and in vivo calcium imaging in mice learning a skilled reach task. We find that VNS enhances skilled motor learning in healthy animals via a cholinergic reinforcement mechanism, producing a rapid consolidation of an expert reach trajectory. In primary motor cortex (M1), VNS drives precise temporal modulation of neurons that respond to behavioral outcome. This suggests that VNS may accelerate motor refinement in M1 via cholinergic signaling, opening new avenues for optimizing VNS to target specific disease-relevant circuitry.

Different Patterns of Neural Activity Characterize Motor Skill Performance During Acquisition and Retention

Motor performance and learning have distinct behavioral and neural signatures and can be uniquely modulated by various informational and motivational factors. Contemporary frameworks describe four different motor learning mechanisms mapped onto specific neural regions which are key for motor skill acquisition: error-based learning (cerebellum), reinforcement learning (basal ganglia), cognitive strategies (prefrontal cortex), and use-dependent learning (motor cortex). However, little is known about the neural circuits engaged during skill acquisition that are modulated specifically by practice-based performance improvement and those that predict recall performance. Based on previous work, we hypothesize that brain activity during practice in primary motor cortex and basal ganglia (1) is associated with trial-by-trial practice performance and (2) is predictive of immediate recall performance. Leveraging the contemporary framework, we use a well-known task paradigm that primarily relies upon cognitive strategy, reinforcement, and use-based learning mechanisms to test our hypotheses. Forty neurotypical young adults were asked to practice a pinch force tracking task. Participants received performance feedback after each trial during practice. We used whole brain analysis of functional magnetic resonance imaging (fMRI) and behavioral performance measures (i.e., time-on-target and self-efficacy) during the practice phase to determine which brain activation patterns are (1) associated with trial-by-trial tracking performance and (2) predictive of immediate no-feedback retention performance. We observed brain activations in the frontal orbital cortex, putamen, amygdala, and insula correlated with tracking performance improvement during practice. In contrast, a different set of performance-related activated regions were observed that were associated with immediate retention performance that included the primary motor cortex, superior frontal gyrus, somatosensory cortex, angular gyrus, and parietal gyrus. Our findings demonstrate that improved practice performance and recall of a sensorimotor skill are correlated with distinct neural activity patterns during acquisition, drawing on different motor learning mechanisms during encoding. While motor performance improvements depend on both cortical and subcortical regions, motor skill recall depends primarily on prefrontal and motor cortices. We discuss possible interpretations for why our hypothesis regarding basal ganglia activity and retention performance was not supported. Understanding the different neural mechanisms engaged in motor performance and learning may inform novel interventions to enhance motor skill learning.

Gene expression and chromatin conformation differs between worker bees performing different tasks

BACKGROUND

Revealing the effect of transcriptomic regulation on behavioral differences is a fundamental goal in biology, but the relationship between gene regulatory networks and individual behavior differences remains largely unknown. Honey bees are considered as good models for studying the mechanisms underlying gene expression changes and behavioral differences since they exhibit strong and obvious differences in tasks between individuals. The cis-regulatory regions usually contain the binding sites of diverse transcription factor (TFs) influencing bee behavior. Thus, the identification of cis-regulatory elements in the brains across different behavioral states is important for understanding how genomic and transcriptomic variations affect different tasks in honeybees.

METHODS

In this study, we employed transcriptome and genome-wide chromatin accessibility assays to analyze brain tissues of honey bees in different behavioral states for examining the relationship between individual behavior differences and brain gene expression changes. We also used the obtained open chromatin regions to identify cis-motifs associating differentially expressed TFs and genes in order to reveal the transcriptional regulatory mechanism related to the different tasks.

RESULTS

We identified genetic regulatory modules regulating different tasks that contained key TFs (CTCF, Trl and schlank) associated with open chromatin regions enriched for DNA sequence motifs belonging to the family of the corresponding TFs. The most prominent transcriptomic changes, which correlated with chromatin accessibility modifications within their proximal promoter regions, occurred in nervous system development, and were associated with behavior switch.

CONCLUSIONS

Our results revealed the regulatory landscape among three behavioral stages in honeybees and identified interactive molecular networks regulating different tasks. These results provide a comprehensive insight into behavioral differences of honeybees, which offers reference for future study.

Reward System Dysfunction and the Motoric-Cognitive Risk Syndrome in Older Persons

During aging, many physiological systems spontaneously change independent of the presence of chronic diseases. The reward system is not an exception and its dysfunction generally includes a reduction in dopamine and glutamate activities and the loss of neurons of the ventral tegmental area (VTA). These impairments are even more pronounced in older persons who have neurodegenerative diseases and/or are affected by cognitive and motoric frailty. All these changes may result in the occurrence of cognitive and motoric frailty and accelerated progression of neurodegenerative diseases, such as Alzheimer’s and Parkinson’s diseases. In particular, the loss of neurons in VTA may determine an acceleration of depressive symptoms and cognitive and motor frailty trajectory, producing an increased risk of disability and mortality. Thus, we hypothesize the existence of a loop between reward system dysfunction, depression, and neurodegenerative diseases in older persons. Longitudinal studies are needed to evaluate the determinant role of the reward system in the onset of motoric-cognitive risk syndrome.

Benefits of exergame training for female patients with fibromyalgia syndrome: a systematic review and meta-analysis of randomized controlled trials

OBJECTIVE

The objective of this systematic review and meta-analysis was to evaluate the effects of exergame training on female patients with fibromyalgia syndrome (FMS), and outcome measurements mainly include overall functioning, pain perception, quality of life, exercise capacity, health perception, kinesiophobia and fatigue severity.

DATA SOURCES

Five electronic databases (PubMed, Web of Science, PsycInfo, Scopus, and Cochrane Library) were searched from inception until June 24, 2021. In addition to search reviewing the reference lists of relevant papers by hand. Cochrane Collaboration's tool for assessing risk of bias was used to evaluate the risk of bias in the included studies.

STUDY SELECTION

Articles were collected with the following study inclusion criteria: (1) randomized controlled trials (RCTs) design. (2) participants were female patients with FMS aged > 18 years. (3) participants from experiments groups received exergame training using any modality. And (4) outcome measures include overall functioning, quality of life, exercise capacity, health perception, kinesiophobia and fatigue severity.

DATA SYNTHESIS

Nine randomized controlled trials, including 466 female patients with FMS, were included in the analysis. Exergame training had significant positive effects on overall functioning (standardized mean difference [SMD] = -0.52; 95% confidence interval [CI] = -0.77 to 0.27; p < .0001), pain perception (SMD = -0.49; 95% CI = -0.97 to -0.02; p = .04), quality of life (SMD = 0.77; 95% CI = 0.44 to1.10; p < .00001), exercise capacity (SMD = 0.58; 95% CI = 0.32 to 0.84; p < .0001), health perception (SMD = 0.69; 95% CI: 0.38 to 1.01; p < .0001), and fatigue severity (SMD = -0.97; 95% CI = -1.55 to -0.38; p = .001). However, exergame training did not have significant effects on kinesiophobia (SMD = -1.13; 95% CI = -2.88 to 0.62; p = .21) .

CONCLUSIONS

Exergame training has beneficial effects on the overall functioning, pain perception, quality of life, exercise capacity, health perception and fatigue severity of female patients with FMS. Exergame training is a potential non-drug therapy for the treatment of patients with FMS.

Central nervous system physiology

This is the second chapter of the series on the use of clinical neurophysiology for the study of movement disorders. It focusses on methods that can be used to probe neural circuits in brain and spinal cord. These include use of spinal and supraspinal reflexes to probe the integrity of transmission in specific pathways; transcranial methods of brain stimulation such as transcranial magnetic stimulation and transcranial direct current stimulation, which activate or modulate (respectively) the activity of populations of central neurones; EEG methods, both in conjunction with brain stimulation or with behavioural measures that record the activity of populations of central neurones; and pure behavioural measures that allow us to build conceptual models of motor control. The methods are discussed mainly in relation to work on healthy individuals. Later chapters will focus specifically on changes caused by pathology.

Concomitant Processing of Choice and Outcome in Frontal Corticostriatal Ensembles Correlates with Performance of Rats

The frontal cortex-basal ganglia network plays a pivotal role in adaptive goal-directed behaviors. Medial frontal cortex (MFC) encodes information about choices and outcomes into sequential activation of neural population, or neural trajectory. While MFC projects to the dorsal striatum (DS), whether DS also displays temporally coordinated activity remains unknown. We studied this question by simultaneously recording neural ensembles in the MFC and DS of rodents performing an outcome-based alternative choice task. We found that the two regions exhibited highly parallel evolution of neural trajectories, transforming choice information into outcome-related information. When the two trajectories were highly correlated, spike synchrony was task-dependently modulated in some MFC-DS neuron pairs. Our results suggest that neural trajectories concomitantly process decision-relevant information in MFC and DS with increased spike synchrony between these regions.

Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function

Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function.

Dopaminergic signaling supports auditory social learning

Explicit rewards are commonly used to reinforce a behavior, a form of learning that engages the dopaminergic neuromodulatory system. In contrast, skill acquisition can display dramatic improvements from a social learning experience, even though the observer receives no explicit reward. Here, we test whether a dopaminergic signal contributes to social learning in naïve gerbils that are exposed to, and learn from, a skilled demonstrator performing an auditory discrimination task. Following five exposure sessions, naïve observer gerbils were allowed to practice the auditory task and their performance was assessed across days. We first tested the effect of an explicit food reward in the observer's compartment that was yoked to the demonstrator's performance during exposure sessions. Naïve observer gerbils with the yoked reward learned the discrimination task significantly faster, as compared to unrewarded observers. The effect of this explicit reward was abolished by administration of a D1/D5 dopamine receptor antagonist during the exposure sessions. Similarly, the D1/D5 antagonist reduced the rate of learning in unrewarded observers. To test whether a dopaminergic signal was sufficient to enhance social learning, we administered a D1/D5 receptor agonist during the exposure sessions in which no reward was present and found that the rate of learning occurred significantly faster. Finally, a quantitative analysis of vocalizations during the exposure sessions suggests one behavioral strategy that contributes to social learning. Together, these results are consistent with a dopamine-dependent reward signal during social learning.

Early Sensory Deprivation Leads to Differential Inhibitory Changes in the Striatum During Learning

The corticostriatal circuit has been identified as a vital pathway for associative learning. However, how learning is implemented when the sensory striatum is permanently impaired remains unclear. Using chemogenetic techniques to suppress layer five auditory cortex (AC) input to the auditory striatum, learning of a sound discrimination task was significantly impacted in freely moving Mongolian gerbils, in particular when this suppression occurs early on during learning. Whole-cell recordings sampled throughout learning revealed a transient reduction in postsynaptic (GABAA) inhibition in both striatal D1 and D2 cells in normal-hearing gerbils during task acquisition. In contrast, when the baseline striatal inhibitory strengths and firing rates were permanently reduced by a transient period of developmental sensory deprivation, learning was accompanied by augmented inhibition and increased firing rates. Direct manipulation of striatal inhibition in vivo and in vitro revealed a key role of the transient inhibitory changes in task acquisition. Together, these results reveal a flexible corticostriatal inhibitory synaptic plasticity mechanism that accompanies associative auditory learning.

Interpreting the role of the striatum during multiple phases of motor learning

The synaptic pathways in the striatum are central to basal ganglia functions including motor control, learning and organization, action selection, acquisition of motor skills, cognitive function, and emotion. Here, we review the role of the striatum and its connections in motor learning and performance. The development of new techniques to record neuronal activity and animal models of motor disorders using neurotoxin, pharmacological, and genetic manipulations are revealing pathways that underlie motor performance and motor learning, as well as how they are altered by pathophysiological mechanisms. We discuss approaches that can be used to analyze complex motor skills, particularly in rodents, and identify specific questions central to understanding how striatal circuits mediate motor learning.

Dopaminergic Modulation of Forced Running Performance in Adolescent Rats: Role of Striatal D1 and Extra-striatal D2 Dopamine Receptors

Improving exercise capacity during adolescence impacts positively on cognitive and motor functions. However, the neural mechanisms contributing to enhance physical performance during this sensitive period remain poorly understood. Such knowledge could help to optimize exercise programs and promote a healthy physical and cognitive development in youth athletes. The central dopamine system is of great interest because of its role in regulating motor behavior through the activation of D1 and D2 receptors. Thus, the aim of the present study is to determine whether D1 or D2 receptor signaling contributes to modulate the exercise capacity during adolescence and if this modulation takes place through the striatum. To test this, we used a rodent model of forced running wheel that we implemented recently to assess the exercise capacity. Briefly, rats were exposed to an 8-day period of habituation in the running wheel before assessing their locomotor performance in response to an incremental exercise test, in which the speed was gradually increased until exhaustion. We found that systemic administration of D1-like (SCH23390) and/or D2-like (raclopride) receptor antagonists prior to the incremental test reduced the duration of forced running in a dose-dependent manner. Similarly, locomotor activity in the open field was decreased by the dopamine antagonists. Interestingly, this was not the case following intrastriatal infusion of an effective dose of SCH23390, which decreased motor performance during the incremental test without disrupting the behavioral response in the open field. Surprisingly, intrastriatal delivery of raclopride failed to impact the duration of forced running. Altogether, these results indicate that the level of locomotor response to incremental loads of forced running in adolescent rats is dopamine dependent and mechanistically linked to the activation of striatal D1 and extra-striatal D2 receptors.

Dopamine, Prediction Error and Beyond

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.

Therapeutic potential of targeting G protein-gated inwardly rectifying potassium (GIRK) channels in the central nervous system

G protein-gated inwardly rectifying potassium channels (Kir3/GirK) are important for maintaining resting membrane potential, cell excitability and inhibitory neurotransmission. Coupled to numerous G protein-coupled receptors (GPCRs), they mediate the effects of many neurotransmitters, neuromodulators and hormones contributing to the general homeostasis and particular synaptic plasticity processes, learning, memory and pain signaling. A growing number of behavioral and genetic studies suggest a critical role for the appropriate functioning of the central nervous system, as well as their involvement in many neurologic and psychiatric conditions, such as neurodegenerative diseases, mood disorders, attention deficit hyperactivity disorder, schizophrenia, epilepsy, alcoholism and drug addiction. Hence, GirK channels emerge as a very promising tool to be targeted in the current scenario where these conditions already are or will become a global public health problem. This review examines recent findings on the physiology, function, dysfunction, and pharmacology of GirK channels in the central nervous system and highlights the relevance of GirK channels as a worthful potential target to improve therapies for related diseases.

History of conditioned reward association disrupts inhibitory control: an examination of neural correlates

The neural processes that support inhibitory control in the face of stimuli with a history of reward association are not yet well understood. Yet, the ability to flexibly adapt behavior to changing reward-contingency contexts is important for daily functioning and warrants further investigation. This study aimed to characterize neural and behavioral impacts of stimuli with a history of conditioned reward association on motor inhibitory control in healthy young adults by investigating group-level effects as well as individual variation in the ability to inhibit responses to stimuli with a reward history. Participants (N = 41) first completed a reward conditioning phase, during which responses to rewarded stimuli were associated with money and responses to unrewarded stimuli were not. Rewarded and unrewarded stimuli from training were carried forward as No-Go targets in a subsequent go/no-go task to test the effect of reward history on inhibitory control. Participants underwent functional brain imaging during the go/no-go portion of the task. On average, a history of reward conditioning disrupted inhibitory control. Compared to inhibition of responses to stimuli with no reward history, trials that required inhibition of responses to previously rewarded stimuli were associated with greater activity in frontal and striatal regions, including the inferior frontal gyrus, insula, striatum, and thalamus. Activity in the insula and thalamus during false alarms and in the ventromedial prefrontal cortex during correctly withheld trials predicted behavioral performance on the task. Overall, these results suggest that reward history serves to disrupt inhibitory control and provide evidence for diverging roles of the insula and ventromedial prefrontal cortex while inhibiting responses to stimuli with a reward history.

A Phonomotor Approach to Apraxia of Speech Treatment

Purpose This study continued Phase I investigation of a modified Phonomotor Treatment (PMT) Program on motor planning in two individuals with apraxia of speech (AOS) and aphasia and, with support from prior work, refined Phase I methodology for treatment intensity and duration, a measure of communicative participation, and the use of effect size benchmarks specific to AOS. Method A single-case experimental design with multiple baselines across behaviors and participants was used to examine acquisition, generalization, and maintenance of treatment effects 8-10 weeks posttreatment. Treatment was distributed 3 days a week, and duration of treatment was specific to each participant (criterion based). Experimental stimuli consisted of target sounds or clusters embedded nonwords and real words, specific to each participants' deficit. Results Findings show improved repetition accuracy for targets in trained nonwords, generalization to targets in untrained nonwords and real words, and maintenance of treatment effects at 10 weeks posttreatment for one participant and more variable outcomes for the other participant. Conclusions Results indicate that a modified version of PMT can promote generalization and maintenance of treatment gains for trained speech targets via a multimodal approach emphasizing repeated exposure and practice. While these results are promising, the frequent co-occurrence of AOS and aphasia warrants a treatment that addresses both motor planning and linguistic deficits. Thus, the application of traditional PMT with participant-specific modifications for AOS embedded into the treatment program may be a more effective approach. Future work will continue to examine and maximize improvements in motor planning, while also treating anomia in aphasia.

Dopamine and Addiction

Addiction is commonly identified with habitual nonmedical self-administration of drugs. It is usually defined by characteristics of intoxication or by characteristics of withdrawal symptoms. Such addictions can also be defined in terms of the brain mechanisms they activate; most addictive drugs cause elevations in extracellular levels of the neurotransmitter dopamine. Animals unable to synthesize or use dopamine lack the conditioned reflexes discussed by Pavlov or the appetitive behavior discussed by Craig; they have only unconditioned consummatory reflexes. Burst discharges (phasic firing) of dopamine-containing neurons are necessary to establish long-term memories associating predictive stimuli with rewards and punishers. Independent discharges of dopamine neurons (tonic or pacemaker firing) determine the motivation to respond to such cues. As a result of habitual intake of addictive drugs, dopamine receptors expressed in the brain are decreased, thereby reducing interest in activities not already stamped in by habitual rewards.

Cell-Type-Specific Outcome Representation in the Primary Motor Cortex

Adaptive movements are critical for animal survival. To guide future actions, the brain monitors various outcomes, including achievement of movement and appetitive goals. The nature of these outcome signals and their neuronal and network realization in the motor cortex (M1), which directs skilled movements, is largely unknown. Using a dexterity task, calcium imaging, optogenetic perturbations, and behavioral manipulations, we studied outcome signals in the murine forelimb M1. We found two populations of layer 2-3 neurons, termed success- and failure-related neurons, that develop with training, and report end results of trials. In these neurons, prolonged responses were recorded after success or failure trials independent of reward and kinematics. In addition, the initial state of layer 5 pyramidal tract neurons contained a memory trace of the previous trial's outcome. Intertrial cortical activity was needed to learn new task requirements. These M1 layer-specific performance outcome signals may support reinforcement motor learning of skilled behavior.

A neural network model of basal ganglia's decision-making circuitry

The basal ganglia have been increasingly recognized as an important structure involved in decision making. Neurons in the basal ganglia were found to reflect the evidence accumulation process during decision making. However, it is not well understood how the direct and indirect pathways of the basal ganglia work together for decision making. Here, we create a recurrent neural network model that is composed of the direct and indirect pathways and test it with the classic random dot motion discrimination task. The direct pathway drives the outputs, which are modulated through a gating mechanism controlled by the indirect pathway. We train the network to learn the task and find that the network reproduces the accuracy and reaction time patterns of previous animal studies. Units in the model exhibit ramping activities that reflect evidence accumulation. Finally, we simulate manipulations of the direct and indirect pathways and find that the manipulations of the direct pathway mainly affect the choice while the manipulations of the indirect pathway affect the model's reaction time. These results suggest a potential circuitry mechanism of the basal ganglia's role in decision making with predictions that can be tested experimentally in the future.

Age-related reduction in motor adaptation: brain structural correlates and the role of explicit memory

The adaption of movement to changes in the environment varies across life span. Recent evidence has linked motor adaptation and its reduction with age to differences in "explicit" learning processes. We examine differences in brain structure and cognition underlying motor adaptation in a population-based cohort (n = 322, aged 18-89 years) using a visuomotor learning task and structural magnetic resonance imaging. Reduced motor adaptation with age was associated with reduced volume in striatum, prefrontal, and sensorimotor cortical regions, but not cerebellum. Medial temporal lobe volume, including the hippocampus, became a stronger determinant of motor adaptation with age. Consistent with the role of the medial temporal lobes, declarative long-term memory showed a similar interaction, whereby memory was more positively correlated with motor adaptation with increasing age. By contrast, visual short-term memory was related to motor adaptation, independently of age. These results support the hypothesis that cerebellar learning is largely unaffected in old age, and the reduction in motor adaptation with age is driven by a decline in explicit memory systems.

Video games as rich environments to foster brain plasticity

This chapter highlights the key role of two main factors, attentional control and reward processing, in unlocking brain plasticity. We first review the evidence for the role that each of these mechanisms plays in neuroplasticity, and then make the case that tools and technologies that combine these two are likely to result in maximal and broad, generalized benefits. In this context, we review the evidence concerning the impact of video game play on brain plasticity, with an eye toward plasticity-driving methods such as the seamless integration of neurofeedback into the video game platforms.

An integrative framework for perceptual disturbances in psychosis

Perceptual disturbances in psychosis, such as auditory verbal hallucinations, are associated with increased baseline activity in the associative auditory cortex and increased dopamine transmission in the associative striatum. Perceptual disturbances are also associated with perceptual biases that suggest increased reliance on prior expectations. We review theoretical models of perceptual inference and key supporting physiological evidence, as well as the anatomy of associative cortico-striatal loops that may be relevant to auditory perceptual inference. Integrating recent findings, we outline a working framework that bridges neurobiology and the phenomenology of perceptual disturbances via theoretical models of perceptual inference.

[Exergames for patients with complex regional pain syndrome : A feasibility study]

BACKGROUND

Complex regional pain syndrome (CRPS) is a disease of the limbs composed of various disorders and defined by the cardinal symptom of pain. So-called exergames with a combination of physical activity and fun are increasingly being offered as part of treatment. Exergame therapy could also provide CRPS patients with repetitive training, reward and motivation.

METHOD

In this study 10 adult patients with CRPS of the hand (50% acute) received a 30 min therapy session using MindMotion™GO. MindMotion™GO is a software that enables control of the integrated games through visual feedback. Outcomes were the subjectively perceived workload (National Aeronautics and Space Administration-task load index, NASA-TLX), user-friendliness (system usability scale, SUS) and pain (numeric rating scale, NRS).

RESULTS

The CRPS patients rated the average workload as appropriate with a total score of 50.9 points (SD ± 18.13). The user-friendliness of the system was judged to be acceptable with an average total score of 89.5 ± 7.53 points. There were no significant changes in pain intensity after the exergames. The subgroup analysis (acute versus chronic) showed differences in the assessment of the individual dimensions of the workload.

CONCLUSION

In this study the use of exergames proved to be a suitable tool for rehabilitation of the hand in adult CRPS patients. Whether exergames represent an effective rehabilitation strategy should be examined by means of functional and activity-related target criteria in a representative sample in a randomized controlled study.

Potential benefits of music playing in stroke upper limb motor rehabilitation

Music-based interventions have emerged as a promising tool in stroke motor rehabilitation as they integrate most of the principles of motor training and multimodal stimulation. This paper aims to review the use of music in the rehabilitation of upper extremity motor function after stroke. First, we review the evidence supporting current music-based interventions including Music-supported Therapy, Music glove, group music therapy, Rhythm- and music-based intervention, and Musical sonification. Next, we describe the mechanisms that may be responsible for the effectiveness of these interventions, focusing on motor learning aspects, how multimodal stimulation may boost motor performance, and emotional and motivational aspects related to music. Then, we discuss methodological concerns in music therapy research related to modifications of therapy protocols, evaluation of patients and study designs. Finally, we highlight clinical considerations for the implementation of music-based interventions in clinical settings.

Principles of Neurorehabilitation After Stroke Based on Motor Learning and Brain Plasticity Mechanisms

What are the principles underlying effective neurorehabilitation? The aim of neurorehabilitation is to exploit interventions based on human and animal studies about learning and adaptation, as well as to show that the activation of experience-dependent neuronal plasticity augments functional recovery after stroke. Instead of teaching compensatory strategies that do not reduce impairment but allow the patient to return home as soon as possible, functional recovery might be more sustainable as it ensures a long-term reduction in impairment and an improvement in quality of life. At the same time, neurorehabilitation permits the scientific community to collect valuable data, which allows inferring about the principles of brain organization. Hence neuroscience sheds light on the mechanisms of learning new functions or relearning lost ones. However, current rehabilitation methods lack the exact operationalization of evidence gained from skill learning literature, leading to an urgent need to bridge motor learning theory and present clinical work in order to identify a set of ingredients and practical applications that could guide future interventions. This work aims to unify the neuroscientific literature relevant to the recovery process and rehabilitation practice in order to provide a synthesis of the principles that constitute an effective neurorehabilitation approach. Previous attempts to achieve this goal either focused on a subset of principles or did not link clinical application to the principles of motor learning and recovery. We identified 15 principles of motor learning based on existing literature: massed practice, spaced practice, dosage, task-specific practice, goal-oriented practice, variable practice, increasing difficulty, multisensory stimulation, rhythmic cueing, explicit feedback/knowledge of results, implicit feedback/knowledge of performance, modulate effector selection, action observation/embodied practice, motor imagery, and social interaction. We comment on trials that successfully implemented these principles and report evidence from experiments with healthy individuals as well as clinical work.

Learning Similar Actions by Reinforcement or Sensory-Prediction Errors Rely on Distinct Physiological Mechanisms

Humans can acquire knowledge of new motor behavior via different forms of learning. The two forms most commonly studied have been the development of internal models based on sensory-prediction errors (error-based learning) and success-based feedback (reinforcement learning). Human behavioral studies suggest these are distinct learning processes, though the neurophysiological mechanisms that are involved have not been characterized. Here, we evaluated physiological markers from the cerebellum and the primary motor cortex (M1) using noninvasive brain stimulations while healthy participants trained finger-reaching tasks. We manipulated the extent to which subjects rely on error-based or reinforcement by providing either vector or binary feedback about task performance. Our results demonstrated a double dissociation where learning the task mainly via error-based mechanisms leads to cerebellar plasticity modifications but not long-term potentiation (LTP)-like plasticity changes in M1; while learning a similar action via reinforcement mechanisms elicited M1 LTP-like plasticity but not cerebellar plasticity changes. Our findings indicate that learning complex motor behavior is mediated by the interplay of different forms of learning, weighing distinct neural mechanisms in M1 and the cerebellum. Our study provides insights for designing effective interventions to enhance human motor learning.

The role of the nucleus accumbens in learned approach behavior diminishes with training

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Effects of Acupuncture on Behavioral Stereotypies and Brain Dopamine System in Mice as a Model of Tourette Syndrome

Tourette syndrome (TS), a developmental neurobehavioral disorder, is characterized by involuntary behavioral stereotypies. Clinical studies have confirmed the positive effect of acupuncture on treating TS, but the underlying mechanisms are not fully understood. In the present study, we used behavioral tests, Western blotting, double-immunofluorescence labeling, and fluorescence spectrophotometry to investigate whether acupuncture performed at acupoints "Baihui" (GV20) and "Yintang" (GV29) affected behavioral stereotypies and regulated the dopamine (DA) system in three different brain regions in Balb/c mice injected with 3,3'-iminodipropionitrile (IDPN) as a model for TS. We found that acupuncture alleviated behavioral stereotypies, down-regulated the expression of D1R and D2R in the striatum (STR) and substantia nigra pars compacta (SNpc), and decreased the concentration of DA in the STR, SNpc, and prefrontal cortex (PFC) as well. Moreover, acupuncture reduced the expression of tyrosine hydroxylase (TH) in the SNpc. Conclusively, acupuncture ameliorated behavioral stereotypies by regulating the DA system in the STR, SNpc, and PFC. Our findings provide novel evidence for the therapeutic effect of acupuncture on TS.

Contributions of the basal ganglia to action sequence learning and performance

Animals engage in intricately woven and choreographed action sequences that are constructed from trial-and-error learning. The mechanisms by which the brain links together individual actions which are later recalled as fluid chains of behavior are not fully understood, but there is broad consensus that the basal ganglia play a crucial role in this process. This paper presents a comprehensive review of the role of the basal ganglia in action sequencing, with a focus on whether the computational framework of reinforcement learning can capture key behavioral features of sequencing and the neural mechanisms that underlie them. While a simple neurocomputational model of reinforcement learning can capture key features of action sequence learning, this model is not sufficient to capture goal-directed control of sequences or their hierarchical representation. The hierarchical structure of action sequences, in particular, poses a challenge for building better models of action sequencing, and it is in this regard that further investigations into basal ganglia information processing may be informative.

Differential Impact of Social and Monetary Reward on Procedural Learning and Consolidation in Aging and Its Structural Correlates

In young (n = 36, mean ± SD: 24.8 ± 4.5 years) and older (n = 34, mean ± SD: 65.1 ± 6.5 years) healthy participants, we employed a modified version of the Serial Reaction Time task to measure procedural learning (PL) and consolidation while providing monetary and social reward. Using voxel-based morphometry (VBM), we additionally determined the structural correlates of reward-related motor performance (RMP) and PL. Monetary reward had a beneficial effect on PL in the older subjects only. In contrast, social reward significantly enhanced PL in the older and consolidation in the young participants. VBM analyses revealed that motor performance related to monetary reward was associated with larger grey matter volume (GMV) of the left striatum in the young, and motor performance related to social reward with larger GMV of the medial orbitofrontal cortex in the older group. The differential effects of social reward in young (improved consolidation) and both social and monetary rewards in older (enhanced PL) healthy subjects point to the potential of rewards for interventions targeting aging-associated motor decline or stroke-induced motor deficits.

Changes in Magnitude and Variability of Corticospinal Excitability During Rewarded Time-Sensitive Behavior

Reward expectation and time estimation are important for behavior and affect corticospinal excitability. This study investigated changes in corticospinal excitability during rewarded time-sensitive behavioral tasks. The rewarded time-sensitive task comprised three fixed-ratio (FR) schedules: FR_A contained a reward stimulus after every response, FR_B after every two responses, and FR_C after every four responses. The participants were instructed to press a left button with the index finger as quickly as possible in response to the appearance of a red circle. Just after the left button press, the word “10-yen” (approximately $0.1) or “no pay” was presented as feedback. Then, the participant had to mentally estimate/wait for 2.5 s from pressing the left button to pressing the right button. One second after the reward stimulus, transcranial magnetic stimulation (TMS) was delivered to the primary motor cortex at the hotspot of the first dorsal interosseous (FDI) muscle. Each participant received items corresponding to the total monetary reward accumulated at the end of the experiment. The variability of motor evoked potential (MEP) amplitudes transformed from a random process during the resting state into an autoregressive process during the rewarded time-sensitive behavioral task. Additionally, the random variation of MEP amplitudes in the FR_C, FR_B, and FR_A schedules increased in a stepwise fashion. However, the magnitude of MEP amplitudes significantly increased for the FR_B and FR_C schedules compared to the FR_A schedule. The time estimation lag was negative for the three FR schedules but there was no difference among the three FR schedules. The magnitude of corticospinal excitability increased in low reward probability, whereas the variability of corticospinal excitability transformed into an autoregressive process in high reward probability. These results imply that the magnitude and variability of expectation-related corticospinal excitabilities can be differentially altered by reward probability.

Is increased sensitivity to punishment a common characteristic of attention deficit/hyperactivity disorder? An experimental study of response allocation in Japanese children

Research on motivational processes in attention deficit/hyperactivity disorder (ADHD) focuses on reward. Studies with punishment are limited and findings mixed. This study evaluated the effects of punishment on response allocation in Japanese children with and without ADHD. Thirty-four children meeting DSM-IV criteria for ADHD and 59 typically developing control-group children completed an operant task in which they choose between playing two simultaneously available games. Reward was arranged symmetrically across the games under concurrent variable interval schedules. Asymmetric punishment schedules were superimposed with responses on one game punished four times as often as responses on the other. Children with ADHD showed greater behavioral sensitivity to punishment than controls. They allocated significantly more responses to the less frequently punished alternative and were more likely to play this game on consecutive trials and responded more slowly to the more punished game. Control group children allocated their responses evenly across games. Punishment exerted greater control over the behavior of Japanese children with ADHD than controls, similar to findings with children from Western countries, suggesting this is a common characteristic of the disorder. The behavior of typically developing Japanese children, while demonstrating awareness of punishment, was not controlled by the frequency of its occurrence.

Auditory Thalamostriatal and Corticostriatal Pathways Convey Complementary Information about Sound Features

Multiple parallel neural pathways link sound-related signals to behavioral responses. For instance, the striatum, a brain structure involved in action selection and reward-related learning, receives neuronal projections from both the auditory thalamus and auditory cortex. It is not clear whether sound information that reaches the striatum through these two pathways is redundant or complementary. We used an optogenetic approach in awake mice of both sexes to identify thalamostriatal and corticostriatal neurons during extracellular recordings, and characterized neural responses evoked by sounds of different frequencies and amplitude modulation rates. We found that neurons in both pathways encode sound frequency with similar fidelity, but display different coding strategies for amplitude modulated noise. Whereas corticostriatal neurons provide a more accurate representation of amplitude modulation rate in their overall firing rate, thalamostriatal neurons convey information about the precise timing of acoustic events. These results demonstrate that auditory thalamus and auditory cortex neurons provide complementary information to the striatum, and suggest that these pathways could be differentially recruited depending on the requirements of a sound-driven behavior.SIGNIFICANCE STATEMENT Sensory signals from the cerebral cortex and the thalamus converge onto the striatum, a nucleus implicated in reward-related learning. It is not clear whether these two sensory inputs convey redundant or complementary information. By characterizing the sound-evoked responses of thalamostriatal and corticostriatal neurons, our work demonstrates that these neural pathways convey complementary information about the temporal features of sounds. This work opens new avenues for investigating how these pathways could be selectively recruited depending on task demands, and provides a framework for studying convergence of cortical and thalamic information onto the striatum in other sensory systems.

Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning

The brain’s dopaminergic pathways are crucially important for adaptive behavior. They are thought to enable us to approach rewards and stay away from punishments. During learning, dopaminergic reward prediction errors are thought to reinforce previously rewarded actions, so they become easier to repeat. This dopaminergic activity could lead to a systematic bias by which rewarded actions are more readily learned than rewarded inactions. We present two findings. First, dopamine receptors in cortex, dorsal striatum, and nucleus accumbens provide distinct sources of variance in the human brain. Second, the boost in an individual’s learning rate from previously rewarded actions is dependent on the dopamine receptor density in dorsal striatum, a central structure in the dopaminergic circuit.

Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [¹¹C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.

Error-related Persistence of Motor Activity in Resting-state Networks

The relationship between neural activation during movement training and the plastic changes that survive beyond movement execution is not well understood. Here we ask whether the changes in resting-state functional connectivity observed following motor learning overlap with the brain networks that track movement error during training. Human participants learned to trace an arched trajectory using a computer mouse in an MRI scanner. Motor performance was quantified on each trial as the maximum distance from the prescribed arc. During learning, two brain networks were observed, one showing increased activations for larger movement error, comprising the cerebellum, parietal, visual, somatosensory, and cortical motor areas, and the other being more activated for movements with lower error, comprising the ventral putamen and the OFC. After learning, changes in brain connectivity at rest were found predominantly in areas that had shown increased activation for larger error during task, specifically the cerebellum and its connections with motor, visual, and somatosensory cortex. The findings indicate that, although both errors and accurate movements are important during the active stage of motor learning, the changes in brain activity observed at rest primarily reflect networks that process errors. This suggests that error-related networks are represented in the initial stages of motor memory formation.

A subcortical circuit linking the cerebellum to the basal ganglia engaged in vocal learning

Speech is a complex sensorimotor skill, and vocal learning involves both the basal ganglia and the cerebellum. These subcortical structures interact indirectly through their respective loops with thalamo-cortical and brainstem networks, and directly via subcortical pathways, but the role of their interaction during sensorimotor learning remains undetermined. While songbirds and their song-dedicated basal ganglia-thalamo-cortical circuitry offer a unique opportunity to study subcortical circuits involved in vocal learning, the cerebellar contribution to avian song learning remains unknown. We demonstrate that the cerebellum provides a strong input to the song-related basal ganglia nucleus in zebra finches. Cerebellar signals are transmitted to the basal ganglia via a disynaptic connection through the thalamus and then conveyed to their cortical target and to the premotor nucleus controlling song production. Finally, cerebellar lesions impair juvenile song learning, opening new opportunities to investigate how subcortical interactions between the cerebellum and basal ganglia contribute to sensorimotor learning.

Place preference and vocal learning rely on distinct reinforcers in songbirds

In reinforcement learning (RL) agents are typically tasked with maximizing a single objective function such as reward. But it remains poorly understood how agents might pursue distinct objectives at once. In machines, multiobjective RL can be achieved by dividing a single agent into multiple sub-agents, each of which is shaped by agent-specific reinforcement, but it remains unknown if animals adopt this strategy. Here we use songbirds to test if navigation and singing, two behaviors with distinct objectives, can be differentially reinforced. We demonstrate that strobe flashes aversively condition place preference but not song syllables. Brief noise bursts aversively condition song syllables but positively reinforce place preference. Thus distinct behavior-generating systems, or agencies, within a single animal can be shaped by correspondingly distinct reinforcement signals. Our findings suggest that spatially segregated vocal circuits can solve a credit assignment problem associated with multiobjective learning.