Momentary reward induce changes in excitability of primary motor cortex

The Mesocortical System Encodes the Strength of Subsequent Force Generation

Our minds impact motor outputs. Such mind-motor interactions are critical for understanding motor control mechanisms and optimizing motor performance. In particular, incentive motivation strongly enhances motor performance. Dopaminergic neurons located in the ventral midbrain (VM) are believed to be the center of incentive motivation. Direct projections from the VM to the primary motor cortex constitute a mesocortical pathway. However, the functional role of this pathway in humans remains unclear. Recently, we demonstrated the functional role of the mesocortical pathway in human motor control in the context of incentive motivation by using functional magnetic resonance imaging (fMRI). Incentive motivation remarkably improved not only reaction times but also the peak grip force in subsequent grip responses. Although the reaction time has been used as a proxy for incentive motivation mediated by dopaminergic midbrain activity, the premovement activity of the mesocortical pathway is involved in controlling the force strength rather than the initiation of subsequent force generation. In this commentary, we review our recent findings and discuss remaining questions regarding the functional role of the mesocortical pathway in mind-motor interactions.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Premovement activity in the mesocortical system links peak force but not initiation of force generation under incentive motivation

Motivation facilitates motor performance; however, the neural substrates of the psychological effects on motor performance remain unclear. We conducted a functional magnetic resonance imaging experiment while human subjects performed a ready-set-go task with monetary incentives. Although subjects were only motivated to respond quickly, increasing the incentives improved not only reaction time but also peak grip force. However, the trial-by-trial correlation between reaction time and peak grip force was weak. Extensive areas in the mesocortical system, including the ventral midbrain (VM) and cortical motor-related areas, exhibited motivation-dependent activity in the premovement "Ready" period when the anticipated monetary reward was displayed. This premovement activity in the mesocortical system correlated only with subsequent peak grip force, whereas the activity in motor-related areas alone was associated with subsequent reaction time and peak grip force. These findings suggest that the mesocortical system linking the VM and motor-related regions plays a role in controlling the peak of force generation indirectly associated with incentives but not the initiation of force generation.

The intracortical excitability changes underlying the enhancing effects of rewards and punishments on motor performance

Monetary rewards and punishments enhance motor performance and are associated with corticospinal excitability (CSE) increases within the motor cortex (M1) during movement preparation. However, such CSE changes have unclear origins. Based on converging evidence, one possibility is that they stem from increased glutamatergic (GLUTergic) facilitation and/or decreased type A gamma-aminobutyric acid (GABAA)-mediated inhibition within M1. To investigate this, paired-pulse transcranial magnetic stimulation was used over the left M1 to evaluate intracortical facilitation (ICF) and short intracortical inhibition (SICI), indirect assays of GLUTergic activity and GABAA-mediated inhibition, in an index finger muscle during the preparation of sequences initiated by either the right index or little finger. Behaviourally, rewards and punishments enhanced both reaction and movement time. During movement preparation, regardless of rewards or punishments, ICF increased when the index finger initiated sequences, whereas SICI decreased when both the index and little fingers initiated sequences. This finding suggests that GLUTergic activity increases in a finger-specific manner whilst GABAA-mediated inhibition decreases in a finger-unspecific manner during preparation. In parallel, both rewards and punishments non-specifically increased ICF, but only rewards non-specifically decreased SICI as compared to neutral. This suggests that to enhance performance rewards both increase GLUTergic activity and decrease GABAA-mediated inhibition, whereas punishments selectively increase GLUTergic activity. A control experiment revealed that such changes were not observed post-movement as participants processed reward and punishment feedback, indicating they were selective to movement preparation. Collectively, these results map the intracortical excitability changes in M1 by which incentives enhance motor performance.

Dissociable effects of positive feedback on the capture and inhibition of impulsive behavior in adolescents with ADHD versus typically developing adolescents

The present study investigated how enhancing motivation by delivering positive feedback (a smiley) after a successful trial could affect interference control in adolescents with Attention Deficit Hyperactivity Disorder (ADHD) and in their typically developing (TD) peers. By using a Simon task within the theoretical framework of the "activation-suppression" model, we were able to separately investigate the expression and the inhibition of impulsive motor behavior. The experiment included 19 adolescents with ADHD and 20 TD adolescents in order to explore whether data found in adolescents with ADHD were similar to those found in TD adolescents. Participants performed the Simon task in two conditions: a condition with feedback delivered after each successful trial and a condition with no feedback. The main findings were that increasing motivation by delivering positive feedback increased impulsive response in both groups of adolescents. It also improved the efficiency of impulsive motor action inhibition in adolescents with ADHD but deteriorated it in TD adolescents. We suggest that 1/increased motivation could lead adolescents to favor fast responses even if incorrect, and 2/the differential effect of feedback on the selective suppression of impulsive motor action in both groups could be due to different baseline DA levels.

Face emotional expressions influence interhemispheric inhibition

The processing of face expressions is a key ability to perform social interactions. Recently, it has been demonstrated that the excitability of the hand primary motor cortex (M1) increases following the view of negative faces expressions. Interhemispheric interactions and sensory-motor integration are cortical processes involving M1, which are known to be modulated by emotional and social behaviors. Whether these processes may mediate the effects of face emotional expressions on M1 excitability is unknown. Therefore, the aim of this study was to investigate the influence of the passive viewing of face emotional expressions on M1 interhemispheric connections and sensory-motor integration using standardized transcranial magnetic stimulation (TMS) protocols. Nineteen healthy subjects participated in the study. Interhemispheric inhibition (IHI) and short-afferent inhibition (SAI) were probed in the right first dorsal interosseous (FDI) muscle 300 ms after the randomized presentation of seven different face expressions (neutral, sadness, fear, disgust, surprise and happiness). Results showed a significantly reduced IHI following the passive viewing of fearful faces compared to neutral (p = .001) and happy (p = .035) faces and following the view of sad faces compared to neutral faces (p = .008). No effect of emotional faces was detected on SAI. Data suggest that sensory-motor integration process does not mediate the increased excitability of M1 induced by the view of negative face expressions. By contrast, it may be underpinned by a depression of IHI, which from a functional point of view may promote symmetrical avoiding movements of the hands in response to aversive stimuli.

Cell-type specific responses to associative learning in the primary motor cortex

The primary motor cortex (M1) is known to be a critical site for movement initiation and motor learning. Surprisingly, it has also been shown to possess reward-related activity, presumably to facilitate reward-based learning of new movements. However, whether reward-related signals are represented among different cell types in M1, and whether their response properties change after cue–reward conditioning remains unclear. Here, we performed longitudinal in vivo two-photon Ca²⁺ imaging to monitor the activity of different neuronal cell types in M1 while mice engaged in a classical conditioning task. Our results demonstrate that most of the major neuronal cell types in M1 showed robust but differential responses to both the conditioned cue stimulus (CS) and reward, and their response properties undergo cell-type-specific modifications after associative learning. PV-INs’ responses became more reliable to the CS, while VIP-INs’ responses became more reliable to reward. Pyramidal neurons only showed robust responses to novel reward, and they habituated to it after associative learning. Lastly, SOM-INs’ responses emerged and became more reliable to both the CS and reward after conditioning. These observations suggest that cue- and reward-related signals are preferentially represented among different neuronal cell types in M1, and the distinct modifications they undergo during associative learning could be essential in triggering different aspects of local circuit reorganization in M1 during reward-based motor skill learning.

Modulation of neural activity in frontopolar cortex drives reward-based motor learning

The frontopolar cortex (FPC) contributes to tracking the reward of alternative choices during decision making, as well as their reliability. Whether this FPC function extends to reward gradients associated with continuous movements during motor learning remains unknown. We used anodal transcranial direct current stimulation (tDCS) over the right FPC to investigate its role in reward-based motor learning. Nineteen healthy human participants practiced novel sequences of finger movements on a digital piano with corresponding auditory feedback. Their aim was to use trialwise reward feedback to discover a hidden performance goal along a continuous dimension: timing. We additionally modulated the contralateral motor cortex (left M1) activity, and included a control sham stimulation. Right FPC-tDCS led to faster learning compared to lM1-tDCS and sham through regulation of motor variability. Bayesian computational modelling revealed that in all stimulation protocols, an increase in the trialwise expectation of reward was followed by greater exploitation, as shown previously. Yet, this association was weaker in lM1-tDCS suggesting a less efficient learning strategy. The effects of frontopolar stimulation were dissociated from those induced by lM1-tDCS and sham, as motor exploration was more sensitive to inferred changes in the reward tendency (volatility). The findings suggest that rFPC-tDCS increases the sensitivity of motor exploration to updates in reward volatility, accelerating reward-based motor learning.

Motor cortex modulation and reward in children with attention-deficit/hyperactivity disorder

Attention-deficit/hyperactivity disorder, the most prevalent developmental disorder in childhood, is a biologically heterogenous condition characterized by impaired attention and impulse control as well as motoric hyperactivity and anomalous motor skill development. Neuropsychological testing often demonstrates impairments in motivation and reward-related decision making in attention-deficit/hyperactivity disorder, believed to indicate dysfunction of the dopamine reward pathway. Development of reliable, non-invasive, easily obtained and quantitative biomarkers correlating with the presence and severity of clinical symptoms and impaired domains of function could aid in identifying meaningful attention-deficit/hyperactivity disorder subgroups and targeting appropriate treatments. To this end, 55 (37 male) 8–12-year-old children with attention-deficit/hyperactivity disorder and 50 (32 male) age-matched, typically-developing controls were enrolled in a transcranial magnetic stimulation protocol—used previously to quantify cortical disinhibition in both attention-deficit/hyperactivity disorder and Parkinson’s Disease—with a child-friendly reward motivation task. The primary outcomes were reward task-induced changes in short interval cortical inhibition and up-modulation of motor evoked potential amplitudes, evaluated using mixed model, repeated measure regression. Our results show that both reward cues and reward receipt reduce short-interval cortical inhibition, and that baseline differences by diagnosis (less inhibition in attention-deficit/hyperactivity disorder) were no longer present when reward was cued or received. Similarly, both reward cues and reward receipt up-modulated motor evoked potential amplitudes, but, differentiating the two groups, this Task-Related-Up-Modulation was decreased in children with attention-deficit/hyperactivity disorder. Furthermore, more severe hyperactive/impulsive symptoms correlated significantly with less up-modulation with success in obtaining reward. These results suggest that in children with attention-deficit/hyperactivity disorder, short interval cortical inhibition may reflect baseline deficiencies as well as processes that normalize performance under rewarded conditions. Task-Related-Up-Modulation may reflect general hypo-responsiveness in attention-deficit/hyperactivity disorder to both reward cue and, especially in more hyperactive/impulsive children, to successful reward receipt. These findings support transcranial magnetic stimulation evoked cortical inhibition and task-induced excitability as biomarkers of clinically relevant domains of dysfunction in childhood attention-deficit/hyperactivity disorder.

Association of short- and long-latency afferent inhibition with human behavior

Transcranial magnetic stimulation (TMS) paired with nerve stimulation evokes short-latency afferent inhibition (SAI) and long-latency afferent inhibition (LAI), which are non-invasive assessments of the excitability of the sensorimotor system. SAI and LAI are abnormally reduced in various special populations in comparison to healthy controls. However, the relationship between afferent inhibition and human behavior remains unclear. The purpose of this review is to survey the current literature and synthesize observations and patterns that affect the interpretation of SAI and LAI in the context of human behavior. We discuss human behaviour across the motor and cognitive domains, and in special and control populations. Further, we discuss future considerations for research in this field and the potential for clinical applications. By understanding how human behavior is mediated by changes in SAI and LAI, this can allow us to better understand the neurophysiological underpinnings of human motor control.

Reward-driven enhancements in motor control are robust to TMS manipulation

A wealth of evidence describes the strong positive impact that reward has on motor control at the behavioural level. However, surprisingly little is known regarding the neural mechanisms which underpin these effects, beyond a reliance on the dopaminergic system. In recent work, we developed a task that enabled the dissociation of the selection and execution components of an upper limb reaching movement. Our results demonstrated that both selection and execution are concommitently enhanced by immediate reward availability. Here, we investigate what the neural underpinnings of each component may be. To this end, we aimed to alter the cortical excitability of the ventromedial prefrontal cortex and supplementary motor area using continuous theta-burst transcranial magnetic stimulation (cTBS) in a within-participant design (N = 23). Both cortical areas are involved in determining an individual’s sensitivity to reward and physical effort, and we hypothesised that a change in excitability would result in the reward-driven effects on action selection and execution to be altered, respectively. To increase statistical power, participants were pre-selected based on their sensitivity to reward in the reaching task. While reward did lead to enhanced performance during the cTBS sessions and a control sham session, cTBS was ineffective in altering these effects. These results may provide evidence that other areas, such as the primary motor cortex or the premotor area, may drive the reward-based enhancements of motor performance.

Reward-Based Improvements in Motor Control Are Driven by Multiple Error-Reducing Mechanisms

Reward has a remarkable ability to invigorate motor behavior, enabling individuals to select and execute actions with greater precision and speed. However, if reward is to be exploited in applied settings, such as rehabilitation, a thorough understanding of its underlying mechanisms is required. In a series of experiments, we first demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Specifically, reward promoted the selection of the correct action in the presence of distractors, while also improving execution through increased speed and maintenance of accuracy. These results led to a shift in the speed-accuracy functions for both selection and execution. In addition, punishment had a similar impact on action selection and execution, although it enhanced execution performance across all trials within a block, that is, its impact was noncontingent to trial value. Although the reward-driven enhancement of movement execution has been proposed to occur through enhanced feedback control, an untested possibility is that it is also driven by increased arm stiffness, an energy-consuming process that enhances limb stability. Computational analysis revealed that reward led to both an increase in feedback correction in the middle of the movement and a reduction in motor noise near the target. In line with our hypothesis, we provide novel evidence that this noise reduction is driven by a reward-dependent increase in arm stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate motor performance without compromising accuracy.SIGNIFICANCE STATEMENT While reward is well-known for enhancing motor performance, how the nervous system generates these improvements is unclear. Despite recent work indicating that reward leads to enhanced feedback control, an untested possibility is that it also increases arm stiffness. We demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Furthermore, we show that punishment has a similar positive impact on performance. Importantly, by combining computational and biomechanical approaches, we show that reward leads to both improved feedback correction and an increase in stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate performance without compromising accuracy. This work suggests that stiffness control plays a vital, and underappreciated, role in the reward-based imporvemenets in motor control.

Altered Recruitment of Motor Cortex Neuronal Activity During the Grasping Phase of Skilled Reaching in a Chronic Rat Model of Unilateral Parkinsonism

Parkinson's disease causes prominent difficulties in the generation and execution of voluntary limb movements, including regulation of distal muscles and coordination of proximal and distal movement components to achieve accurate grasping. Difficulties with manual dexterity have a major impact on activities of daily living. We used extracellular single neuron recordings to investigate the neural underpinnings of parkinsonian movement deficits in the motor cortex of chronic unilateral 6-hydroxydopamine lesion male rats performing a skilled reach-to-grasp task the. Both normal movements and parkinsonian deficits in this task have striking homology to human performance. In lesioned animals there were several differences in the activity of cortical neurons during reaches by the affected limb compared with control rats. These included an increase in proportions of neurons showing rate decreases, along with increased amplitude of their average rate-decrease response at specific times during the reach, suggesting a shift in the balance of net excitation and inhibition of cortical neurons; a significant increase in the duration of rate-increase responses, which could result from reduced coupling of cortical activity to specific movement components; and changes in the timing and incidence of neurons with pure rate-increase or biphasic responses, particularly at the end of reach when grasping would normally be occurring. The changes in cortical activity may account for the deficits that occur in skilled distal motor control following dopamine depletion, and highlight the need for treatment strategies targeted toward modulating cortical mechanisms for fine distal motor control in patients.SIGNIFICANCE STATEMENT We show for the first time in a chronic lesion rat model of Parkinson's disease movement deficits that there are specific changes in motor cortex neuron activity associated with the grasping phase of a skilled motor task. Such changes provide a possible mechanism underpinning the problems with manual dexterity seen in Parkinson's patients and highlight the need for treatment strategies targeted toward distal motor control.

Motor Memory: Revealing Conditioned Action Tendencies Using Transcranial Magnetic Stimulation

Action tendencies can be elicited by motivationally salient stimuli (e.g., appetitive rewards) or objects that support utilization behaviors. These action tendencies can benefit behavioral performance through speeded RTs in response tasks and improve detection accuracy in attentional capture tasks. However, action tendencies can be counterproductive when goals change (e.g., refraining from junk foods or abstaining from alcohol). Maintaining control over cue-elicited action tendencies is therefore critical for successful behavior modification. To better understand this relationship, we used transcranial magnetic stimulation to investigate the neural signatures of action tendencies in the presence of previously trained response cues. Participants were presented with a continuous letter stream and instructed to respond quickly to two target letters using two different response keys. Following this training phase, the target letters were embedded in a new task (test phase), and we applied transcranial magnetic stimulation to the motor cortex and measured motor evoked potentials as an index of corticospinal excitability (CSE). We found that CSE could be potentiated by a former response cue trained within a single experimental session, even when participants were instructed to withhold responses during the test phase. Critically, attention to the previously trained response cue was required to elicit the primed modulation in CSE, and successful control of this activity was accompanied by CSE suppression. These findings suggest that well-trained response cues can come to prime a conditioned action tendency and provide a model for understanding how the implementation of cognitive control can override action automaticity.

Changes in Magnitude and Variability of Corticospinal Excitability During Rewarded Time-Sensitive Behavior

Reward expectation and time estimation are important for behavior and affect corticospinal excitability. This study investigated changes in corticospinal excitability during rewarded time-sensitive behavioral tasks. The rewarded time-sensitive task comprised three fixed-ratio (FR) schedules: FR_A contained a reward stimulus after every response, FR_B after every two responses, and FR_C after every four responses. The participants were instructed to press a left button with the index finger as quickly as possible in response to the appearance of a red circle. Just after the left button press, the word “10-yen” (approximately $0.1) or “no pay” was presented as feedback. Then, the participant had to mentally estimate/wait for 2.5 s from pressing the left button to pressing the right button. One second after the reward stimulus, transcranial magnetic stimulation (TMS) was delivered to the primary motor cortex at the hotspot of the first dorsal interosseous (FDI) muscle. Each participant received items corresponding to the total monetary reward accumulated at the end of the experiment. The variability of motor evoked potential (MEP) amplitudes transformed from a random process during the resting state into an autoregressive process during the rewarded time-sensitive behavioral task. Additionally, the random variation of MEP amplitudes in the FR_C, FR_B, and FR_A schedules increased in a stepwise fashion. However, the magnitude of MEP amplitudes significantly increased for the FR_B and FR_C schedules compared to the FR_A schedule. The time estimation lag was negative for the three FR schedules but there was no difference among the three FR schedules. The magnitude of corticospinal excitability increased in low reward probability, whereas the variability of corticospinal excitability transformed into an autoregressive process in high reward probability. These results imply that the magnitude and variability of expectation-related corticospinal excitabilities can be differentially altered by reward probability.

Motor cortex inhibition and modulation in children with ADHD

OBJECTIVE

Compared to typically developing (TD) peers, children with attention-deficit/hyperactivity disorder (ADHD) consistently demonstrate impaired transcranial magnetic stimulation (TMS)-evoked short interval cortical inhibition (SICI) of motor evoked potentials (MEPs) in resting motor cortex (M1). To determine whether perturbed M1 physiology also reflects clinically relevant behavioral dysfunction, we evaluated M1 physiology during a cognitive control task taxing motor response selection/inhibition.

METHODS

In this case-control study, behavioral ratings, motor skill (assessed using standardized examination), and left M1 physiology were evaluated in 131 right-handed, 8- to 12-year-old children (66 ADHD: mean 10.5 years, 43 male; 65 TD: mean 10.6 years, 42 male). The primary outcomes were MEP amplitudes and SICI, evaluated during rest and during a modified "racecar" Slater-Hammel stop signal reaction task, with TMS pulses administered 150 ms prior to the target go action and after the dynamic stop cue.

RESULTS

Go responses were significantly slower (p = 0.01) and more variable (p = 0.002) in ADHD. Children with ADHD showed less M1 SICI at rest (p = 0.02) and during go (p = 0.03) and stop trials (p = 0.02). Rest M1 excitability increased during response inhibition task engagement (p < 0.0001). This Task-Related Up-Modulation (TRUM) was less robust across and within groups, with diminished task upmodulation associated with significantly more severe ADHD behavioral ratings and slower stop signal reaction times.

CONCLUSION

Children with ADHD show anomalous motor cortex physiology, with deficient SICI across behavioral states and less TRUM from rest to action selection. Associations of these physiologic measures with ADHD symptoms and cognitive control measures support further investigation into biological mechanisms.

The role of stimulus-driven versus goal-directed processes in fight and flight tendencies measured with motor evoked potentials induced by Transcranial Magnetic Stimulation

This study examines two contrasting explanations for early tendencies to fight and flee. According to a stimulus-driven explanation, goal-incompatible stimuli that are easy/difficult to control lead to the tendency to fight/flee. According to a goal-directed explanation, on the other hand, the tendency to fight/flee occurs when the expected utility of fighting/fleeing is the highest. Participants did a computer task in which they were confronted with goal-incompatible stimuli that were (a) easy to control and fighting had the highest expected utility, (b) easy to control and fleeing had the highest expected utility, and (c) difficult to control and fleeing and fighting had zero expected utility. After participants were trained to use one hand to fight and another hand to flee, they either had to choose a response or merely observe the stimuli. During the observation trials, single-pulse Transcranial Magnetic Stimulation (TMS) was applied to the primary motor cortex 450 ms post-stimulus onset and motor-evoked potentials (MEPs) were measured from the hand muscles. Results showed that participants chose to fight/flee when the expected utility of fighting/fleeing was the highest, and that they responded late when the expected utility of both responses was low. They also showed larger MEPs for the right/left hand when the expected utility of fighting/fleeing was the highest. This result can be interpreted as support for the goal-directed account, but only if it is assumed that we were unable to override the presumed natural mapping between hand (right/left) and response (fight/flight).

Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings

Background

Reward-based feedback given during motor learning has been shown to improve the retention of the behaviour being acquired. Interestingly, applying transcranial direct current stimulation (tDCS) during learning over the primary motor cortex (M1), an area associated with motor retention, also results in enhanced retention of the newly formed motor memories. However, it remains unknown whether combining these distinct interventions result in an additive benefit of motor retention.

Methods

We investigated whether combining both interventions while participants learned to account for a visuomotor transformation results in enhanced motor retention (total n = 56; each group n = 14). To determine whether these interventions share common physiological mechanisms underpinning learning, we assessed motor cortical excitability and inhibition (i.e. SICI) on a hand muscle before and after all participants learned the visuomotor rotation using their entire arm and hand.

Results

We found that both the Reward-Stim (i.e. reward + tDCS) and Reward-Sham (i.e. reward-only) groups had increased retention at the beginning of the retention phase, indicating an immediate effect of reward on behaviour. However, each intervention on their own did not enhance retention when compared to sham, but rather, only the combination of both reward and tDCS demonstrated prolonged retention. We also found that only the Reward-Stim group had a significant reduction in SICI after exposure to the perturbation.

Conclusions

We show that combining both interventions are additive in providing stronger retention of motor adaptation. These results indicate that the reliability and validity of using tDCS within a clinical context may depend on the type of feedback individuals receive when learning a new motor pattern.

•

Concurrently administering reward and M1 tDCS during learning results in enhanced motor retention.

•

The combination of these interventions also leads to a reduction in M1 inhibitory mechanisms.

•

No benefits of motor retention were found when reward or M1 tDCS were given alone.

Phase-Locked Stimulation during Cortical Beta Oscillations Produces Bidirectional Synaptic Plasticity in Awake Monkeys

The functional role of cortical beta oscillations, if any, remains unresolved. During oscillations, the periodic fluctuation in excitability of entrained cells modulates transmission of neural impulses and periodically enhances synaptic interactions. The extent to which oscillatory episodes affect activity-dependent synaptic plasticity remains to be determined. In nonhuman primates, we delivered single-pulse electrical cortical stimulation to a "stimulated" site in sensorimotor cortex triggered on a specific phase of ongoing beta (12-25 Hz) field potential oscillations recorded at a separate "triggering" site. Corticocortical connectivity from the stimulated to the triggering site as well as to other (non-triggering) sites was assessed by cortically evoked potentials elicited by test stimuli to the stimulated site, delivered outside of oscillatory episodes. In separate experiments, connectivity was assessed by intracellular recordings of evoked excitatory postsynaptic potentials. The conditioning paradigm produced transient (1-2 s long) changes in connectivity between the stimulated and the triggering site that outlasted the duration of the oscillatory episodes. The direction of the plasticity effect depended on the phase from which stimulation was triggered: potentiation in depolarizing phases, depression in hyperpolarizing phases. Plasticity effects were also seen at non-triggering sites that exhibited oscillations synchronized with those at the triggering site. These findings indicate that cortical beta oscillations provide a spatial and temporal substrate for short-term, activity-dependent synaptic plasticity in primate neocortex and may help explain the role of oscillations in attention, learning, and cortical reorganization.

Goal Achievement Failure Drives Corticospinal Modulation in Promotion and Prevention Contexts

When making decisions, people are typically differently sensitive to gains and losses according to the motivational context in which the choice is performed. As hypothesized by Regulatory Focus Theory (RFT), indeed, goals are supposed to change in relation to the set of possible outcomes. In particular, in a promotion context, the goal is achieving the maximal gain, whereas in a prevention context it turns into avoiding the greatest loss. We explored the neurophysiological counterpart of this phenomenon, by applying Transcranial Magnetic Stimulation (TMS) and recording the motor evoked potentials (MEPs) in participants taking part in an economic game, in which they observed actions conveying different goal attainment levels, framed in different motivational contexts. More than the actual value of the economic exchange involved in the game, what affected motor cortex excitability was the goal attainment failure, corresponding to not achieving the maximal payoff in a promotion context and not avoiding the greatest snatch in a prevention context. Therefore, the results provide support for the key predictions of RFT, identifying a neural signature for the goal attainment failure.

Nonequivalent modulation of corticospinal excitability by positive and negative outcomes

OBJECTIVE

The difference between positive and negative outcomes is important in trial-and-error decision-making processes and affects corticospinal excitability. This study investigated corticospinal excitability during the performance of trial-and-error decision-making tasks with varying competing behavioral outcomes.

METHODS

Each trial began with one of five colored circles presented as a cue. Each color represented a different reward probability, ranging from 10% to 90%. The subjects were instructed to decide whether to perform wrist flexion in response to the cue. Two seconds after the presentation of the cue, a reward stimulus (picture of a coin) or penalty stimulus (mauve circle) was randomly presented to the subject. If the picture of a coin appeared, the subjects received the coin after the experiment if they had performed wrist flexion, but not if they had not performed wrist flexion. If a mauve circle appeared, a coin was deducted from the total reward if the subjects had performed wrist flexion, but not if they had not performed wrist flexion. One second after the reward or penalty stimulus, transcranial magnetic stimulation was delivered to the primary motor cortex at the midpoint between the centers of gravity of the flexor carpi radialis (agonist) and extensor carpi radialis (antagonist) muscles.

RESULTS

Cumulative wrist flexions were positively correlated with reward probabilities. Motor evoked potential (MEP) amplitudes in agonist muscles were significantly higher when wrist flexion incurred a penalty than when it incurred a reward, but there was no difference in the MEP amplitudes of antagonist muscles.

CONCLUSION

Positive and negative behavioral outcomes differentially altered behavior and corticospinal excitability, and unexpected penalties had a stronger effect on corticospinal excitability for agonist muscles.

Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning

For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible.

Modulation of corticospinal excitability by reward depends on task framing

Findings from previous transcranial magnetic stimulation (TMS) experiments suggest that the primary motor cortex (M1) is sensitive to reward conditions in the environment. However, the nature of this influence on M1 activity is poorly understood. The dopamine neuron response to conditioned stimuli encodes reward probability and outcome uncertainty, or the extent to which the outcome of a situation is known. Reward uncertainty and probability are related: uncertainty is maximal when probability is 0.5 and minimal when probability is 0 or 1 (i.e., certain outcome). Previous TMS-reward studies did not examine these factors independently. Here, we used single-pulse TMS to measure corticospinal excitability in 40 individuals while they performed a simple computer task, making guesses to find or avoid a hidden target. The task stimuli implied three levels of reward probability and two levels of uncertainty. We found that reward probability level interacted with the trial search condition. That is, motor evoked potential (MEP) amplitude, a measure of corticospinal neuron excitability, increased with increasing reward probability when participants were instructed to "find" a target, but not when they were instructed to "avoid" a target. There was no effect of uncertainty on MEPs. Response times varied with the number of choices. A subset of participants also received paired-pulse stimulation to evaluate changes in short-intracortical inhibition (SICI). No effects of SICI were observed. Taken together, the results suggest that the reward-contingent modulation of M1 activity reflects reward probability or a related aspect of utility, not outcome uncertainty, and that this effect is sensitive to the conceptual framing of the task.

Voltage-sensitive dye imaging of primary motor cortex activity produced by ventral tegmental area stimulation

The primary motor cortex (M1) receives dopaminergic projections from the ventral tegmental area (VTA) through the mesocortical dopamine pathway. However, few studies have focused on changes in M1 neuronal activity caused by VTA activation. To address this issue, we used voltage-sensitive dye imaging (VSD) to reveal the spatiotemporal dynamics of M1 activity induced by single-pulse stimulation of VTA in anesthetized rats. VSD imaging showed that brief electrical stimulation of unilateral VTA elicited a short-latency excitatory-inhibitory sequence of neuronal activity not only in the ipsilateral but also in the contralateral M1. The contralateral M1 response was not affected by pharmacological blockade of ipsilateral M1 activity, but it was completely abolished by corpus callosum transection. Although the VTA-evoked neuronal activity extended throughout the entire M1, we found the most prominent activity in the forelimb area of M1. The 6-OHDA-lesioned VTA failed to evoke M1 activity. Furthermore, both excitatory and inhibitory intact VTA-induced activity was entirely extinguished by blocking glutamate receptors in the target M1. When intracortical microstimulation of M1 was paired with VTA stimulation, the evoked forelimb muscle activity was facilitated or inhibited, depending on the interval between the two stimuli. These findings suggest that VTA neurons directly modulate the excitability of M1 neurons via fast glutamate signaling and, consequently, may control the last cortical stage of motor command processing.

Reward in the mirror neuron system, social context, and the implications on psychopathology

Positive and negative reinforcers guide our behaviors as we interact with others in our social environment. Here, we present evidence that highlights a central role for reward in the general functioning of the mirror neuron system (MNS). We also discuss the relevance of reward-related modulation on other previous findings revealing certain properties of the MNS, and on social context and psychopathology.

Motor cortex-evoked activity in reciprocal muscles is modulated by reward probability

Horizontal intracortical projections for agonist and antagonist muscles exist in the primary motor cortex (M1), and reward may induce a reinforcement of transmission efficiency of intracortical circuits. We investigated reward-induced change in M1 excitability for agonist and antagonist muscles. Participants were 8 healthy volunteers. Probabilistic reward tasks comprised 3 conditions of 30 trials each: 30 trials contained 10% reward, 30 trials contained 50% reward, and 30 trials contained 90% reward. Each trial began with a cue (red fixation cross), followed by blue circle for 1 s. The subjects were instructed to perform wrist flexion and press a button with the dorsal aspect of middle finger phalanx as quickly as possible in response to disappearance of the blue circle without looking at their hand or the button. Two seconds after the button press, reward/non-reward stimulus was randomly presented for 2-s duration. The reward stimulus was a picture of Japanese 10-yen coin, and each subject received monetary reward at the end of experiment. Subjects were not informed of the reward probabilities. We delivered transcranial magnetic stimulation of the left M1 at the midpoint between center of gravities of agonist flexor carpi radialis (FCR) and antagonist extensor carpi radialis (ECR) muscles at 2 s after the red fixation cross and 1 s after the reward/non-reward stimuli. Relative motor evoked potential (MEP) amplitudes at 2 s after the red fixation cross were significantly higher for 10% reward probability than for 90% reward probability, whereas relative MEP amplitudes at 1 s after reward/non-reward stimuli were significantly higher for 90% reward probability than for 10% and 50% reward probabilities. These results implied that reward could affect the horizontal intracortical projections in M1 for agonist and antagonist muscles, and M1 excitability including the reward-related circuit before and after reward stimulus could be differently altered by reward probability.

Single pulse TMS differentially modulates reward behavior

Greater knowledge of cortical brain regions in reward processing may set the stage for using transcranial magnetic stimulation (TMS) as a treatment in patients with avolition, apathy or other drive-related symptoms. This study examined the effects of single pulse (sp) TMS to two reward circuit targets on drive in healthy subjects. Fifteen healthy subjects performed the monetary incentive delay task (MID) while receiving fMRI-guided spTMS to either inferior parietal lobe (IPL) or supplemental motor area (SMA). The study demonstrated decreasing reaction times (RT) for increasing reward. It also showed significant differences in RT modulation for TMS pulses to the IPL versus the SMA. TMS pulses during the delay period produced significantly more RT slowing when targeting the IPL than those to the SMA. This RT slowing carried over into subsequent trials without TMS stimulation, with significantly slower RTs in sessions that had targeted the IPL compared to those targeting SMA. The results of this study suggest that both SMA and IPL are involved in reward processing, with opposite effects on RT in response to TMS stimulation. TMS to these target cortical regions may be useful in modulating reward circuit deficits in psychiatric populations.

Visual attentional load influences plasticity in the human motor cortex

Neural plasticity plays a critical role in learning, memory, and recovery from injury to the nervous system. Although much is known about the physical and physiological determinants of plasticity, little is known about the influence of cognitive factors. In this study, we investigated whether selective attention plays a role in modifying changes in neural excitability reflecting long-term potentiation (LTP)-like plasticity. We induced LTP-like effects in the hand area of the human motor cortex using transcranial magnetic stimulation (TMS). During the induction of plasticity, participants engaged in a visual detection task with either low or high attentional demands. Changes in neural excitability were assessed by measuring motor-evoked potentials in a small hand muscle before and after the TMS procedures. In separate experiments plasticity was induced either by paired associative stimulation (PAS) or intermittent theta-burst stimulation (iTBS). Because these procedures induce different forms of LTP-like effects, they allowed us to investigate the generality of any attentional influence on plasticity. In both experiments reliable changes in motor cortex excitability were evident under low-load conditions, but this effect was eliminated under high-attentional load. In a third experiment we investigated whether the attentional task was associated with ongoing changes in the excitability of motor cortex, but found no difference in evoked potentials across the levels of attentional load. Our findings indicate that in addition to their role in modifying sensory processing, mechanisms of attention can also be a potent modulator of cortical plasticity.

The role of prediction in social neuroscience

Research has shown that the brain is constantly making predictions about future events. Theories of prediction in perception, action and learning suggest that the brain serves to reduce the discrepancies between expectation and actual experience, i.e., by reducing the prediction error. Forward models of action and perception propose the generation of a predictive internal representation of the expected sensory outcome, which is matched to the actual sensory feedback. Shared neural representations have been found when experiencing one's own and observing other's actions, rewards, errors, and emotions such as fear and pain. These general principles of the “predictive brain” are well established and have already begun to be applied to social aspects of cognition. The application and relevance of these predictive principles to social cognition are discussed in this article. Evidence is presented to argue that simple non-social cognitive processes can be extended to explain complex cognitive processes required for social interaction, with common neural activity seen for both social and non-social cognitions. A number of studies are included which demonstrate that bottom-up sensory input and top-down expectancies can be modulated by social information. The concept of competing social forward models and a partially distinct category of social prediction errors are introduced. The evolutionary implications of a “social predictive brain” are also mentioned, along with the implications on psychopathology. The review presents a number of testable hypotheses and novel comparisons that aim to stimulate further discussion and integration between currently disparate fields of research, with regard to computational models, behavioral and neurophysiological data. This promotes a relatively new platform for inquiry in social neuroscience with implications in social learning, theory of mind, empathy, the evolution of the social brain, and potential strategies for treating social cognitive deficits.

Impaired savings despite intact initial learning of motor adaptation in Parkinson's disease

In motor adaptation, the occurrence of savings (faster relearning of a previously learned motor adaptation task) has been explained in terms of operant reinforcement learning (Huang et al. in Neuron 70(4):787-801, 2011), which is thought to associate an adapted motor command with outcome success during repeated execution of the adapted movement. There is some evidence for deficient savings in Parkinson's Disease (PD), which might result from deficient operant reinforcement processes. However, this evidence is compromised by limited adaptation training during initial learning and by multi-target adaptation, which reduces the number of reinforced movement repetitions for each target. Here, we examined savings in PD patients and controls following overlearning with a single target. PD patients showed less savings than controls after successive adaptation and deadaptation blocks within the same test session, as well as less savings across test sessions separated by a 24-h delay. It is argued that impaired blunted dopaminergic signals in PD impairs the modulation of dopaminergic signals to the motor cortex in response to rewarding motor outcomes, thus impairing the association of the adapted motor command with rewarding motor outcomes. Consequently, the previously adapted motor command is not preferentially selected during relearning, and savings is impaired.