Brain structural substrates of reward dependence during behavioral performance

Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

Elevated integration within the reward network underlies vulnerability to distress

Distress tolerance (DT), the capability to persist under negative circumstances, underlies a range of psychopathologies. It has been proposed that DT may originate from the activity and connectivity in diverse neural networks integrated by the reward system. To test this hypothesis, we examined the link between DT and integration and segregation in the reward network as derived from resting-state functional connectivity data. DT was measured in 147 participants from a large community sample using the Behavioral Indicator of Resiliency to Distress task. Prior to DT evaluation, participants underwent a resting-state functional magnetic resonance imaging scan. For each participant, we constructed a whole-brain functional connectivity network and calculated the degree of reward network integration and segregation based on the extent to which reward network nodes showed functional connections within and outside their network. We found that distress-intolerant participants demonstrated heightened reward network integration relative to the distress-tolerant participants. In addition, these differences in integration were higher relative to the rest of the brain and, more specifically, the somatomotor network, which has been implicated in impulsive behavior. These findings support the notion that increased integration in large-scale brain networks may constitute a risk for distress intolerance and its psychopathological correlates.

Reward Dependence-Moderated Noradrenergic and Hormonal Responses During Noncompetitive and Competitive Physical Activities

The aim of this study was to reveal whether increased reward dependence (RD) plays a role in the catecholamine neurotransmitter release and testosterone hormone regulation during physical activities among healthy trained participants. Twenty-two male participants (mean age: 40.27 ± 5.4 years) participated in this study. Two conditions were constructed, namely, a noncompetitive and a competitive running task (RT), which were separated by a 2-week interval. Urine and blood samples were collected prior to and following the running tasks. Noradrenaline (NA), adrenaline (A), dopamine (D), and their metabolites, vanillylmandelic acid (VMA) and homovanillic acid (HVA), were measured from urine, while testosterone levels were analyzed from blood samples. RD was assessed using the Cloninger's Personality Inventory (PI). Mental health was evaluated using the WHO Well-Being, Beck Depression, and Perceived Life Stress Questionnaires. According to our findings, levels of NA, A, D, VMA, and testosterone released underwent an increase following physical exertion, independently from the competitive condition of the RT, while HVA levels experienced a decrease. However, we found that testosterone levels showed a significantly lower tendency to elevate in the competitive RT, compared with the noncompetitive condition (p = 0.02). In contrast, HVA values were higher in the competitive compared with the noncompetitive condition (p = 0.031), both before and after the exercise. Considering the factor RD, in noncompetitive RT, its higher values were associated with elevated NA levels (p = 0.007); however, this correlation could not be detected during the competitive condition (p = 0.233). Among male runners, the NA and testosterone levels could be predicted to the degree of RD by analyzing competitive and noncompetitive physical exercises.

Linking interindividual variability in brain structure to behaviour

What are the brain structural correlates of interindividual differences in behaviour? More than a decade ago, advances in structural MRI opened promising new avenues to address this question. The initial wave of research then progressively led to substantial conceptual and methodological shifts, and a replication crisis unveiled the limitations of traditional approaches, which involved searching for associations between local measurements of neuroanatomy and behavioural variables in small samples of healthy individuals. Given these methodological issues and growing scepticism regarding the idea of one-to-one mapping of psychological constructs to brain regions, new perspectives emerged. These not only embrace the multivariate nature of brain structure-behaviour relationships and promote generalizability but also embrace the representation of the relationships between brain structure and behavioural data by latent dimensions of interindividual variability. Here, we examine the past and present of the study of brain structure-behaviour associations in healthy populations and address current challenges and open questions for future investigations.

Reward and plasticity: Implications for neurorehabilitation

Neuroplasticity follows nervous system injury in the presence or absence of rehabilitative treatments. Rehabilitative interventions can be used to modulate adaptive neuroplasticity, reducing motor impairment and improving activities of daily living in patients with brain lesions. Learning principles guide some rehabilitative interventions. While basic science research has shown that reward combined with training enhances learning, this principle has been only recently explored in the context of neurorehabilitation. Commonly used reinforcers may be more or less rewarding depending on the individual or the context in which the task is performed. Studies in healthy humans showed that both reward and punishment can enhance within-session motor performance; but reward, and not punishment, improves consolidation and retention of motor skills. On the other hand, neurorehabilitative training after brain lesions involves complex tasks (e.g., walking and activities of daily living). The contribution of reward to neurorehabilitation is incompletely understood. Here, we discuss recent research on the role of reward in neurorehabilitation and the needed directions of future research.

Intrinsic Functional Connectivity of the Anterior Cingulate Cortex Is Associated with Tolerance to Distress

The ability to adapt under significant adversity, defined as psychological resilience, is instrumental in preventing stress-related disorders. An important aspect of resilience is the capacity to endure affective distress when in pursuit of goals, also known as distress tolerance. Evidence that links intrinsic baseline interactions within large-scale functional networks with performance under distress remains missing. We hypothesized that the anterior cingulate cortex (ACC) may engage in distress tolerance because of its involvement in attention and emotion regulation. Accordingly, we tested whether behavioral performance under distress is associated with baseline resting-state ACC functional connectivity (FC). Distress tolerance was measured in 97 participants using the behavioral indicator of resiliency to distress (BIRD) task. Analyses contrasted participants who quit the task before its designated termination (n = 51) with those who persisted throughout it (n = 46). Seed-based FC analysis indicated greater connectivity between the ACC and dorsolateral prefrontal cortex (DLPFC) in subjects who persisted throughout the task, along with greater FC between the ACC and the precentral gyrus in those who quit before its termination. The results shed light on the mechanisms underlying interindividual differences in the ability to handle distress.

Reward boosts reinforcement-based motor learning

Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we investigated the net effect of motivation by reward on motor learning while controlling for the sensory and reinforcement feedback received by the participants. Reward improved motor skill learning beyond performance-based reinforcement feedback. Importantly, the beneficial effect of reward involved a specific potentiation of reinforcement-related adjustments in motor commands, which concerned primarily the most relevant motor component for task success and persisted on the following day in the absence of reward. We propose that the long-lasting effects of motivation on motor learning may entail a form of associative learning resulting from the repetitive pairing of the reinforcement feedback and reward during training, a mechanism that may be exploited in future rehabilitation protocols.

The Bright and Dark Sides of Performance-Dependent Monetary Rewards: Evidence From Visual Perception Tasks

Studies have shown that performance-dependent monetary rewards facilitate visual perception. However, no study has examined whether such a positive effect is limited to the rewarded task or may be generalized to other tasks. In the current study, two groups of people were asked to perform two visual perception tasks, one being a reward-relevant task and the other being a reward-irrelevant task. For the reward-relevant task, the experimental group received performance-dependent monetary rewards, whereas the control group did not. For the reward-irrelevant task, both groups were not rewarded. The two tasks were randomly intermixed trial by trial (Experiment 1) or presented block by block (Experiment 2) or session by session (Experiments 3a, 3b, and 3c). Results showed that performance-dependent monetary rewards improved participants' performance on the relevant task in all experiments and impaired their performance on the irrelevant task in Experiments 2, 3a, 3b, and 3c. These results suggested that monetary rewards might incur a cost on reward-irrelevant tasks. Finally, the benefit of monetary rewards disappeared when they were no longer provided during the final session. This is the first study that reveals both the bright and dark sides of the performance-dependent monetary rewards in visual perception.

Neuromodulation of reinforced skill learning reveals the causal function of prefrontal cortex

Accumulating evidence has suggested functional interactions between prefrontal cortex (PFC) and dissociable large-scale networks. However, how these networks interact in the human brain to enable complex behaviors is not well-understood. Here, using a combination of behavioral, brain stimulation and neuroimaging paradigms, we tested the hypothesis that human PFC is required for successful reinforced skill formation. We additionally tested the extent to which PFC-dependent skill formation is related to intrinsic functional communication with this region. We report that inhibitory noninvasive transcranial magnetic stimulation over lateral PFC, a hub region with a diverse connectivity profile, causally modulated effective reinforcement-based motor skill acquisition. Furthermore, PFC-dependent skill formation was strongly related to the strength of functional connectivity between the PFC and regions in the sensorimotor network. These results point to the involvement of lateral PFC in the neural architecture that underlies the acquisition of complex skills, and suggest that, in relation to skill acquisition, this region may be involved in functional interactions with sensorimotor networks.

Pharmacological Dopamine Manipulation Does Not Alter Reward-Based Improvements in Memory Retention during a Visuomotor Adaptation Task

Motor adaptation tasks investigate our ability to adjust motor behaviors to an ever-changing and unpredictable world. Previous work has shown that punishment-based feedback delivered during a visuomotor adaptation task enhances error-reduction, whereas reward increases memory retention. While the neural underpinnings of the influence of punishment on the adaptation phase remain unclear, reward has been hypothesized to increase retention through dopaminergic mechanisms. We directly tested this hypothesis through pharmacological manipulation of the dopaminergic system. A total of 96 young healthy human participants were tested in a placebo-controlled double-blind between-subjects design in which they adapted to a 40° visuomotor rotation under reward or punishment conditions. We confirmed previous evidence that reward enhances retention, but the dopamine (DA) precursor levodopa (LD) or the DA antagonist haloperidol failed to influence performance. We reason that such a negative result could be due to experimental limitations or it may suggest that the effect of reward on motor memory retention is not driven by dopaminergic processes. This provides further insight regarding the role of motivational feedback in optimizing motor learning, and the basis for further decomposing the effect of reward on the subprocesses known to underlie motor adaptation paradigms.

Neural Responses to Smoking Cues in Schizophrenia

The high prevalence of nicotine dependence contributes to excess mortality in schizophrenia. Cue reactivity, or the encounter of drug-related cues or contexts, triggers craving, drug-seeking, and relapse. Prior functional magnetic resonance imaging (fMRI) research indicates that individuals with schizophrenia have blunted neural responses to rewarding stimuli in association with more severe negative symptoms. The objectives of this study are to determine if smokers with schizophrenia have altered neural reactivity to smoking cues compared with non-psychiatrically ill smokers and to evaluate the influence of negative symptoms on cue reactivity. Twenty smokers with schizophrenia and 19 control smokers underwent fMRI while viewing smoking-related and neutral cues. The primary analysis was group comparison of Smoking-Neutral contrast using whole-brain analysis (Pcorrected < .05). Smokers with schizophrenia had significantly greater baseline carbon monoxide levels and longer duration of smoking, suggesting more nicotine use. While both groups had greater brain reactivity to smoking vs neutral cues, smokers with schizophrenia had significantly decreased cue reactivity (Smoking-Neutral) compared to controls in bilateral frontal midline regions. There were significant negative correlations between negative symptoms and frontal midline reactivity. Despite greater nicotine use, smokers with schizophrenia exhibited decreased smoking cue-induced neural reactivity in frontal midline regions, suggesting that increased smoking and low cessation rates in schizophrenia are not primarily driven by responses to smoking-related cues. The finding of negative correlations between cue reactivity and negative symptoms is consistent with previous research demonstrating decreased neural responses to rewarding cues, particularly in patients with negative symptoms.

Somatic and Reinforcement-Based Plasticity in the Initial Stages of Human Motor Learning

As one learns to dance or play tennis, the desired somatosensory state is typically unknown. Trial and error is important as motor behavior is shaped by successful and unsuccessful movements. As an experimental model, we designed a task in which human participants make reaching movements to a hidden target and receive positive reinforcement when successful. We identified somatic and reinforcement-based sources of plasticity on the basis of changes in functional connectivity using resting-state fMRI before and after learning. The neuroimaging data revealed reinforcement-related changes in both motor and somatosensory brain areas in which a strengthening of connectivity was related to the amount of positive reinforcement during learning. Areas of prefrontal cortex were similarly altered in relation to reinforcement, with connectivity between sensorimotor areas of putamen and the reward-related ventromedial prefrontal cortex strengthened in relation to the amount of successful feedback received. In other analyses, we assessed connectivity related to changes in movement direction between trials, a type of variability that presumably reflects exploratory strategies during learning. We found that connectivity in a network linking motor and somatosensory cortices increased with trial-to-trial changes in direction. Connectivity varied as well with the change in movement direction following incorrect movements. Here the changes were observed in a somatic memory and decision making network involving ventrolateral prefrontal cortex and second somatosensory cortex. Our results point to the idea that the initial stages of motor learning are not wholly motor but rather involve plasticity in somatic and prefrontal networks related both to reward and exploration.

SIGNIFICANCE STATEMENT

In the initial stages of motor learning, the placement of the limbs is learned primarily through trial and error. In an experimental analog, participants make reaching movements to a hidden target and receive positive feedback when successful. We identified sources of plasticity based on changes in functional connectivity using resting-state fMRI. The main finding is that there is a strengthening of connectivity between reward-related prefrontal areas and sensorimotor areas in the basal ganglia and frontal cortex. There is also a strengthening of connectivity related to movement exploration in sensorimotor circuits involved in somatic memory and decision making. The results indicate that initial stages of motor learning depend on plasticity in somatic and prefrontal networks related to reward and exploration.

The many facets of motor learning and their relevance for Parkinson's disease

The final goal of motor learning, a complex process that includes both implicit and explicit (or declarative) components, is the optimization and automatization of motor skills. Motor learning involves different neural networks and neurotransmitters systems depending on the type of task and on the stage of learning. After the first phase of acquisition, a motor skill goes through consolidation (i.e., becoming resistant to interference) and retention, processes in which sleep and long-term potentiation seem to play important roles. The studies of motor learning in Parkinson's disease have yielded controversial results that likely stem from the use of different experimental paradigms. When a task's characteristics, instructions, context, learning phase and type of measures are taken into consideration, it is apparent that, in general, only learning that relies on attentional resources and cognitive strategies is affected by PD, in agreement with the finding of a fronto-striatal deficit in this disease. Levodopa administration does not seem to reverse the learning deficits in PD, while deep brain stimulation of either globus pallidus or subthalamic nucleus appears to be beneficial. Finally and most importantly, patients with PD often show a decrease in retention of newly learned skill, a problem that is present even in the early stages of the disease. A thorough dissection and understanding of the processes involved in motor learning is warranted to provide solid bases for effective medical, surgical and rehabilitative approaches in PD.

Motor Learning Enhances Use-Dependent Plasticity

Motor behaviors are shaped not only by current sensory signals but also by the history of recent experiences. For instance, repeated movements toward a particular target bias the subsequent movements toward that target direction. This process, called use-dependent plasticity (UDP), is considered a basic and goal-independent way of forming motor memories. Most studies consider movement history as the critical component that leads to UDP (Classen et al., 1998; Verstynen and Sabes, 2011). However, the effects of learning (i.e., improved performance) on UDP during movement repetition have not been investigated. Here, we used transcranial magnetic stimulation in two experiments to assess plasticity changes occurring in the primary motor cortex after individuals repeated reinforced and nonreinforced actions. The first experiment assessed whether learning a skill task modulates UDP. We found that a group that successfully learned the skill task showed greater UDP than a group that did not accumulate learning, but made comparable repeated actions. The second experiment aimed to understand the role of reinforcement learning in UDP while controlling for reward magnitude and action kinematics. We found that providing subjects with a binary reward without visual feedback of the cursor led to increased UDP effects. Subjects in the group that received comparable reward not associated with their actions maintained the previously induced UDP. Our findings illustrate how reinforcing consistent actions strengthens use-dependent memories and provide insight into operant mechanisms that modulate plastic changes in the motor cortex.SIGNIFICANCE STATEMENT Performing consistent motor actions induces use-dependent plastic changes in the motor cortex. This plasticity reflects one of the basic forms of human motor learning. Past studies assumed that this form of learning is exclusively affected by repetition of actions. However, here we showed that success-based reinforcement signals could affect the human use-dependent plasticity (UDP) process. Our results indicate that learning augments and interacts with UDP. This effect is important to the understanding of the interplay between the different forms of motor learning and suggests that reinforcement is not only important to learning new behaviors, but can shape our subsequent behavior via its interaction with UDP.

Reward disrupts reactivated human skill memory

Accumulating evidence across species and memory domains shows that when an existing memory is reactivated, it becomes susceptible to modifications. However, the potential role of reward signals in these mechanisms underlying human memory dynamics is unknown. Leaning on a wealth of findings on the role of reward in reinforcing memory, we tested the impact of reinforcing a skill memory trace with monetary reward following memory reactivation, on strengthening of the memory trace. Reinforcing reactivated memories did not strengthen the memory, but rather led to disruption of the memory trace, breaking down the link between memory reactivation and subsequent memory strength. Statistical modeling further revealed a strong mediating role for memory reactivation in linking between memory encoding and subsequent memory strength only when the memory was replayed without reinforcement. We suggest that, rather than reinforcing the existing memory trace, reward creates a competing memory trace, impairing expression of the original reward-free memory. This mechanism sheds light on the processes underlying skill acquisition, having wide translational implications.

Visual Feedback of Tongue Movement for Novel Speech Sound Learning

Pronunciation training studies have yielded important information concerning the processing of audiovisual (AV) information. Second language (L2) learners show increased reliance on bottom-up, multimodal input for speech perception (compared to monolingual individuals). However, little is known about the role of viewing one's own speech articulation processes during speech training. The current study investigated whether real-time, visual feedback for tongue movement can improve a speaker's learning of non-native speech sounds. An interactive 3D tongue visualization system based on electromagnetic articulography (EMA) was used in a speech training experiment. Native speakers of American English produced a novel speech sound (/ɖ/; a voiced, coronal, palatal stop) before, during, and after trials in which they viewed their own speech movements using the 3D model. Talkers' productions were evaluated using kinematic (tongue-tip spatial positioning) and acoustic (burst spectra) measures. The results indicated a rapid gain in accuracy associated with visual feedback training. The findings are discussed with respect to neural models for multimodal speech processing.