Reward-related activity in the human motor cortex

Improving rehabilitation motivation and motor learning ability of stroke patients using different reward strategies: study protocol for a single-center, randomized controlled trial

Background

Stroke survivors often face challenges in motor learning and motivation during rehabilitation, which can impede their recovery progress. Traditional rehabilitation methods vary in effectiveness, prompting the exploration of novel approaches such as reward strategies. Previous research indicates that rewards can enhance rehabilitation motivation and facilitate motor learning. However, most reward paradigms have utilized fixed reward amounts, which also have limitations. Exploring alternative, more effective reward strategies, such as probabilistic rewards, is warranted to optimize stroke patient rehabilitation.

Methods

A total of 81 stroke patients will be recruited and randomly assigned to control, fixed reward, or probabilistic reward groups at a ratio of 1:1:1 using a randomized number table method. Participants will undergo 10 days of daily hand motor function rehabilitation training, with sessions lasting 20 min each. The training will involve pegboard tests and box and block tests. Control group participants will receive standard training, while fixed reward group members will receive monetary incentives for completing tests, and probabilistic reward group members will have the chance to win monetary rewards through a lottery box. Rehabilitation motivation and motor performance and functional near-infrared spectroscopy brain imaging will be conducted at designated time points. The primary outcome measure is the stroke rehabilitation motivation scale, and the second outcome measures include motor performance, simple test for evaluating hand function, motivation and pleasure scale self-report, and Pittsburgh rehabilitation participation scale.

Discussion

Reward-based training enhance rehabilitation participation and adherence, it also improve motor learning speed and memory retention of stroke patients. The fixed reward applied in the past studies could diminish the sensitivity of stroke patients to rewards, while probabilistic reward may provide unpredictable or variable incentives or reinforcements for motor rehabilitation. This study will compare the efficacy of different reward strategies in enhancing motor learning ability and rehabilitation motivation among stroke patients. By conducting a randomized controlled trial, the study seeks to provide valuable insights into optimizing stroke rehabilitation protocols and improving patient outcomes.Clinical Trial Registration:https://www.chictr.org.cn/, ChiCTR2400082419.

The Mesocortical System Encodes the Strength of Subsequent Force Generation

Our minds impact motor outputs. Such mind-motor interactions are critical for understanding motor control mechanisms and optimizing motor performance. In particular, incentive motivation strongly enhances motor performance. Dopaminergic neurons located in the ventral midbrain (VM) are believed to be the center of incentive motivation. Direct projections from the VM to the primary motor cortex constitute a mesocortical pathway. However, the functional role of this pathway in humans remains unclear. Recently, we demonstrated the functional role of the mesocortical pathway in human motor control in the context of incentive motivation by using functional magnetic resonance imaging (fMRI). Incentive motivation remarkably improved not only reaction times but also the peak grip force in subsequent grip responses. Although the reaction time has been used as a proxy for incentive motivation mediated by dopaminergic midbrain activity, the premovement activity of the mesocortical pathway is involved in controlling the force strength rather than the initiation of subsequent force generation. In this commentary, we review our recent findings and discuss remaining questions regarding the functional role of the mesocortical pathway in mind-motor interactions.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Premovement activity in the mesocortical system links peak force but not initiation of force generation under incentive motivation

Motivation facilitates motor performance; however, the neural substrates of the psychological effects on motor performance remain unclear. We conducted a functional magnetic resonance imaging experiment while human subjects performed a ready-set-go task with monetary incentives. Although subjects were only motivated to respond quickly, increasing the incentives improved not only reaction time but also peak grip force. However, the trial-by-trial correlation between reaction time and peak grip force was weak. Extensive areas in the mesocortical system, including the ventral midbrain (VM) and cortical motor-related areas, exhibited motivation-dependent activity in the premovement "Ready" period when the anticipated monetary reward was displayed. This premovement activity in the mesocortical system correlated only with subsequent peak grip force, whereas the activity in motor-related areas alone was associated with subsequent reaction time and peak grip force. These findings suggest that the mesocortical system linking the VM and motor-related regions plays a role in controlling the peak of force generation indirectly associated with incentives but not the initiation of force generation.

The intracortical excitability changes underlying the enhancing effects of rewards and punishments on motor performance

Monetary rewards and punishments enhance motor performance and are associated with corticospinal excitability (CSE) increases within the motor cortex (M1) during movement preparation. However, such CSE changes have unclear origins. Based on converging evidence, one possibility is that they stem from increased glutamatergic (GLUTergic) facilitation and/or decreased type A gamma-aminobutyric acid (GABAA)-mediated inhibition within M1. To investigate this, paired-pulse transcranial magnetic stimulation was used over the left M1 to evaluate intracortical facilitation (ICF) and short intracortical inhibition (SICI), indirect assays of GLUTergic activity and GABAA-mediated inhibition, in an index finger muscle during the preparation of sequences initiated by either the right index or little finger. Behaviourally, rewards and punishments enhanced both reaction and movement time. During movement preparation, regardless of rewards or punishments, ICF increased when the index finger initiated sequences, whereas SICI decreased when both the index and little fingers initiated sequences. This finding suggests that GLUTergic activity increases in a finger-specific manner whilst GABAA-mediated inhibition decreases in a finger-unspecific manner during preparation. In parallel, both rewards and punishments non-specifically increased ICF, but only rewards non-specifically decreased SICI as compared to neutral. This suggests that to enhance performance rewards both increase GLUTergic activity and decrease GABAA-mediated inhibition, whereas punishments selectively increase GLUTergic activity. A control experiment revealed that such changes were not observed post-movement as participants processed reward and punishment feedback, indicating they were selective to movement preparation. Collectively, these results map the intracortical excitability changes in M1 by which incentives enhance motor performance.

Dissociable effects of positive feedback on the capture and inhibition of impulsive behavior in adolescents with ADHD versus typically developing adolescents

The present study investigated how enhancing motivation by delivering positive feedback (a smiley) after a successful trial could affect interference control in adolescents with Attention Deficit Hyperactivity Disorder (ADHD) and in their typically developing (TD) peers. By using a Simon task within the theoretical framework of the "activation-suppression" model, we were able to separately investigate the expression and the inhibition of impulsive motor behavior. The experiment included 19 adolescents with ADHD and 20 TD adolescents in order to explore whether data found in adolescents with ADHD were similar to those found in TD adolescents. Participants performed the Simon task in two conditions: a condition with feedback delivered after each successful trial and a condition with no feedback. The main findings were that increasing motivation by delivering positive feedback increased impulsive response in both groups of adolescents. It also improved the efficiency of impulsive motor action inhibition in adolescents with ADHD but deteriorated it in TD adolescents. We suggest that 1/increased motivation could lead adolescents to favor fast responses even if incorrect, and 2/the differential effect of feedback on the selective suppression of impulsive motor action in both groups could be due to different baseline DA levels.

Contribution of the sensorimotor beta oscillations and the cortico-basal ganglia-thalamic circuitry during value-based decision making: A simultaneous EEG-fMRI investigation

In decision neuroscience, the motor system has primarily been considered to be involved in executing choice actions. However, a competing perspective suggests its engagement in the evaluation of options, traditionally considered to be performed by the brain's valuation system. Here, we investigate the role of the motor system in value-based decision making by determining the neural circuitries associated with the sensorimotor beta oscillations previously identified to encode decision options. In a simultaneous EEG-fMRI study, participants evaluated reward and risk associated with a forthcoming action. A significant sensorimotor beta desynchronization was identified prior to and independent of response. The level of beta desynchronization showed evidence of encoding the reward levels. This beta desynchronization covaried, on a trial-by-trial level, with BOLD activity in the cortico-basal ganglia-thalamic circuitry. In contrast, there was only a weak covariation within the valuation network, despite significant modulation of its BOLD activity by reward levels. These results suggest that the way in which decision variables are processed differs in the valuation network and in the cortico-basal ganglia-thalamic circuitry. We propose that sensorimotor beta oscillations indicate incentive motivational drive towards a choice action computed from the decision variables even prior to making a response, and it arises from the cortico-basal ganglia-thalamic circuitry.

Motor cortex modulation and reward in children with attention-deficit/hyperactivity disorder

Attention-deficit/hyperactivity disorder, the most prevalent developmental disorder in childhood, is a biologically heterogenous condition characterized by impaired attention and impulse control as well as motoric hyperactivity and anomalous motor skill development. Neuropsychological testing often demonstrates impairments in motivation and reward-related decision making in attention-deficit/hyperactivity disorder, believed to indicate dysfunction of the dopamine reward pathway. Development of reliable, non-invasive, easily obtained and quantitative biomarkers correlating with the presence and severity of clinical symptoms and impaired domains of function could aid in identifying meaningful attention-deficit/hyperactivity disorder subgroups and targeting appropriate treatments. To this end, 55 (37 male) 8–12-year-old children with attention-deficit/hyperactivity disorder and 50 (32 male) age-matched, typically-developing controls were enrolled in a transcranial magnetic stimulation protocol—used previously to quantify cortical disinhibition in both attention-deficit/hyperactivity disorder and Parkinson’s Disease—with a child-friendly reward motivation task. The primary outcomes were reward task-induced changes in short interval cortical inhibition and up-modulation of motor evoked potential amplitudes, evaluated using mixed model, repeated measure regression. Our results show that both reward cues and reward receipt reduce short-interval cortical inhibition, and that baseline differences by diagnosis (less inhibition in attention-deficit/hyperactivity disorder) were no longer present when reward was cued or received. Similarly, both reward cues and reward receipt up-modulated motor evoked potential amplitudes, but, differentiating the two groups, this Task-Related-Up-Modulation was decreased in children with attention-deficit/hyperactivity disorder. Furthermore, more severe hyperactive/impulsive symptoms correlated significantly with less up-modulation with success in obtaining reward. These results suggest that in children with attention-deficit/hyperactivity disorder, short interval cortical inhibition may reflect baseline deficiencies as well as processes that normalize performance under rewarded conditions. Task-Related-Up-Modulation may reflect general hypo-responsiveness in attention-deficit/hyperactivity disorder to both reward cue and, especially in more hyperactive/impulsive children, to successful reward receipt. These findings support transcranial magnetic stimulation evoked cortical inhibition and task-induced excitability as biomarkers of clinically relevant domains of dysfunction in childhood attention-deficit/hyperactivity disorder.

Reward-driven enhancements in motor control are robust to TMS manipulation

A wealth of evidence describes the strong positive impact that reward has on motor control at the behavioural level. However, surprisingly little is known regarding the neural mechanisms which underpin these effects, beyond a reliance on the dopaminergic system. In recent work, we developed a task that enabled the dissociation of the selection and execution components of an upper limb reaching movement. Our results demonstrated that both selection and execution are concommitently enhanced by immediate reward availability. Here, we investigate what the neural underpinnings of each component may be. To this end, we aimed to alter the cortical excitability of the ventromedial prefrontal cortex and supplementary motor area using continuous theta-burst transcranial magnetic stimulation (cTBS) in a within-participant design (N = 23). Both cortical areas are involved in determining an individual’s sensitivity to reward and physical effort, and we hypothesised that a change in excitability would result in the reward-driven effects on action selection and execution to be altered, respectively. To increase statistical power, participants were pre-selected based on their sensitivity to reward in the reaching task. While reward did lead to enhanced performance during the cTBS sessions and a control sham session, cTBS was ineffective in altering these effects. These results may provide evidence that other areas, such as the primary motor cortex or the premotor area, may drive the reward-based enhancements of motor performance.

Local and global consequences of reward-evoked striatal dopamine release

The neurotransmitter dopamine is required for the reinforcement of actions by rewarding stimuli¹. Neuroscientists have tried to define dopamine’s functions in concise conceptual terms², but the practical significance of dopamine release depends on its diverse brain-wide consequences. Although the molecular and cellular effects of dopaminergic signaling have been extensively studied³, its impact on larger-scale neural activity profiles is less understood. Here we combine dynamic dopamine-sensitive molecular imaging⁴ and functional magnetic resonance imaging (fMRI) to determine how striatal dopamine release shapes local and global responses to rewarding stimulation in the rodent brain. We find that dopamine consistently alters the duration but not the magnitude of stimulus responses across much of striatum, via quantifiable postsynaptic effects that vary across subregions. Striatal dopamine release also potentiates a network of distal responses we delineate using neurochemically-dependent functional connectivity analyses. Hot spots of dopaminergic drive notably include cortical regions associated with both limbic and motor function. Our results thus reveal distinct neuromodulatory actions of striatal dopamine that extend well beyond its sites of peak release, and that result in enhanced activation of remote neural populations necessary for performance of motivated actions. Our findings also suggest brain-wide biomarkers of dopaminergic function and could provide a basis for improved interpretation of neuroimaging results relevant to learning and addiction.

Motor Memory: Revealing Conditioned Action Tendencies Using Transcranial Magnetic Stimulation

Action tendencies can be elicited by motivationally salient stimuli (e.g., appetitive rewards) or objects that support utilization behaviors. These action tendencies can benefit behavioral performance through speeded RTs in response tasks and improve detection accuracy in attentional capture tasks. However, action tendencies can be counterproductive when goals change (e.g., refraining from junk foods or abstaining from alcohol). Maintaining control over cue-elicited action tendencies is therefore critical for successful behavior modification. To better understand this relationship, we used transcranial magnetic stimulation to investigate the neural signatures of action tendencies in the presence of previously trained response cues. Participants were presented with a continuous letter stream and instructed to respond quickly to two target letters using two different response keys. Following this training phase, the target letters were embedded in a new task (test phase), and we applied transcranial magnetic stimulation to the motor cortex and measured motor evoked potentials as an index of corticospinal excitability (CSE). We found that CSE could be potentiated by a former response cue trained within a single experimental session, even when participants were instructed to withhold responses during the test phase. Critically, attention to the previously trained response cue was required to elicit the primed modulation in CSE, and successful control of this activity was accompanied by CSE suppression. These findings suggest that well-trained response cues can come to prime a conditioned action tendency and provide a model for understanding how the implementation of cognitive control can override action automaticity.

Changes in Magnitude and Variability of Corticospinal Excitability During Rewarded Time-Sensitive Behavior

Reward expectation and time estimation are important for behavior and affect corticospinal excitability. This study investigated changes in corticospinal excitability during rewarded time-sensitive behavioral tasks. The rewarded time-sensitive task comprised three fixed-ratio (FR) schedules: FR_A contained a reward stimulus after every response, FR_B after every two responses, and FR_C after every four responses. The participants were instructed to press a left button with the index finger as quickly as possible in response to the appearance of a red circle. Just after the left button press, the word “10-yen” (approximately $0.1) or “no pay” was presented as feedback. Then, the participant had to mentally estimate/wait for 2.5 s from pressing the left button to pressing the right button. One second after the reward stimulus, transcranial magnetic stimulation (TMS) was delivered to the primary motor cortex at the hotspot of the first dorsal interosseous (FDI) muscle. Each participant received items corresponding to the total monetary reward accumulated at the end of the experiment. The variability of motor evoked potential (MEP) amplitudes transformed from a random process during the resting state into an autoregressive process during the rewarded time-sensitive behavioral task. Additionally, the random variation of MEP amplitudes in the FR_C, FR_B, and FR_A schedules increased in a stepwise fashion. However, the magnitude of MEP amplitudes significantly increased for the FR_B and FR_C schedules compared to the FR_A schedule. The time estimation lag was negative for the three FR schedules but there was no difference among the three FR schedules. The magnitude of corticospinal excitability increased in low reward probability, whereas the variability of corticospinal excitability transformed into an autoregressive process in high reward probability. These results imply that the magnitude and variability of expectation-related corticospinal excitabilities can be differentially altered by reward probability.

The role of stimulus-driven versus goal-directed processes in fight and flight tendencies measured with motor evoked potentials induced by Transcranial Magnetic Stimulation

This study examines two contrasting explanations for early tendencies to fight and flee. According to a stimulus-driven explanation, goal-incompatible stimuli that are easy/difficult to control lead to the tendency to fight/flee. According to a goal-directed explanation, on the other hand, the tendency to fight/flee occurs when the expected utility of fighting/fleeing is the highest. Participants did a computer task in which they were confronted with goal-incompatible stimuli that were (a) easy to control and fighting had the highest expected utility, (b) easy to control and fleeing had the highest expected utility, and (c) difficult to control and fleeing and fighting had zero expected utility. After participants were trained to use one hand to fight and another hand to flee, they either had to choose a response or merely observe the stimuli. During the observation trials, single-pulse Transcranial Magnetic Stimulation (TMS) was applied to the primary motor cortex 450 ms post-stimulus onset and motor-evoked potentials (MEPs) were measured from the hand muscles. Results showed that participants chose to fight/flee when the expected utility of fighting/fleeing was the highest, and that they responded late when the expected utility of both responses was low. They also showed larger MEPs for the right/left hand when the expected utility of fighting/fleeing was the highest. This result can be interpreted as support for the goal-directed account, but only if it is assumed that we were unable to override the presumed natural mapping between hand (right/left) and response (fight/flight).

Goal Achievement Failure Drives Corticospinal Modulation in Promotion and Prevention Contexts

When making decisions, people are typically differently sensitive to gains and losses according to the motivational context in which the choice is performed. As hypothesized by Regulatory Focus Theory (RFT), indeed, goals are supposed to change in relation to the set of possible outcomes. In particular, in a promotion context, the goal is achieving the maximal gain, whereas in a prevention context it turns into avoiding the greatest loss. We explored the neurophysiological counterpart of this phenomenon, by applying Transcranial Magnetic Stimulation (TMS) and recording the motor evoked potentials (MEPs) in participants taking part in an economic game, in which they observed actions conveying different goal attainment levels, framed in different motivational contexts. More than the actual value of the economic exchange involved in the game, what affected motor cortex excitability was the goal attainment failure, corresponding to not achieving the maximal payoff in a promotion context and not avoiding the greatest snatch in a prevention context. Therefore, the results provide support for the key predictions of RFT, identifying a neural signature for the goal attainment failure.

Nonequivalent modulation of corticospinal excitability by positive and negative outcomes

OBJECTIVE

The difference between positive and negative outcomes is important in trial-and-error decision-making processes and affects corticospinal excitability. This study investigated corticospinal excitability during the performance of trial-and-error decision-making tasks with varying competing behavioral outcomes.

METHODS

Each trial began with one of five colored circles presented as a cue. Each color represented a different reward probability, ranging from 10% to 90%. The subjects were instructed to decide whether to perform wrist flexion in response to the cue. Two seconds after the presentation of the cue, a reward stimulus (picture of a coin) or penalty stimulus (mauve circle) was randomly presented to the subject. If the picture of a coin appeared, the subjects received the coin after the experiment if they had performed wrist flexion, but not if they had not performed wrist flexion. If a mauve circle appeared, a coin was deducted from the total reward if the subjects had performed wrist flexion, but not if they had not performed wrist flexion. One second after the reward or penalty stimulus, transcranial magnetic stimulation was delivered to the primary motor cortex at the midpoint between the centers of gravity of the flexor carpi radialis (agonist) and extensor carpi radialis (antagonist) muscles.

RESULTS

Cumulative wrist flexions were positively correlated with reward probabilities. Motor evoked potential (MEP) amplitudes in agonist muscles were significantly higher when wrist flexion incurred a penalty than when it incurred a reward, but there was no difference in the MEP amplitudes of antagonist muscles.

CONCLUSION

Positive and negative behavioral outcomes differentially altered behavior and corticospinal excitability, and unexpected penalties had a stronger effect on corticospinal excitability for agonist muscles.

Somatic and Reinforcement-Based Plasticity in the Initial Stages of Human Motor Learning

As one learns to dance or play tennis, the desired somatosensory state is typically unknown. Trial and error is important as motor behavior is shaped by successful and unsuccessful movements. As an experimental model, we designed a task in which human participants make reaching movements to a hidden target and receive positive reinforcement when successful. We identified somatic and reinforcement-based sources of plasticity on the basis of changes in functional connectivity using resting-state fMRI before and after learning. The neuroimaging data revealed reinforcement-related changes in both motor and somatosensory brain areas in which a strengthening of connectivity was related to the amount of positive reinforcement during learning. Areas of prefrontal cortex were similarly altered in relation to reinforcement, with connectivity between sensorimotor areas of putamen and the reward-related ventromedial prefrontal cortex strengthened in relation to the amount of successful feedback received. In other analyses, we assessed connectivity related to changes in movement direction between trials, a type of variability that presumably reflects exploratory strategies during learning. We found that connectivity in a network linking motor and somatosensory cortices increased with trial-to-trial changes in direction. Connectivity varied as well with the change in movement direction following incorrect movements. Here the changes were observed in a somatic memory and decision making network involving ventrolateral prefrontal cortex and second somatosensory cortex. Our results point to the idea that the initial stages of motor learning are not wholly motor but rather involve plasticity in somatic and prefrontal networks related both to reward and exploration.

SIGNIFICANCE STATEMENT

In the initial stages of motor learning, the placement of the limbs is learned primarily through trial and error. In an experimental analog, participants make reaching movements to a hidden target and receive positive feedback when successful. We identified sources of plasticity based on changes in functional connectivity using resting-state fMRI. The main finding is that there is a strengthening of connectivity between reward-related prefrontal areas and sensorimotor areas in the basal ganglia and frontal cortex. There is also a strengthening of connectivity related to movement exploration in sensorimotor circuits involved in somatic memory and decision making. The results indicate that initial stages of motor learning depend on plasticity in somatic and prefrontal networks related to reward and exploration.

Cortical neurons multiplex reward-related signals along with sensory and motor information

Rewards are known to influence neural activity associated with both motor preparation and execution. This influence can be exerted directly upon the primary motor (M1) and somatosensory (S1) cortical areas via the projections from reward-sensitive dopaminergic neurons of the midbrain ventral tegmental areas. However, the neurophysiological manifestation of reward-related signals in M1 and S1 are not well understood. Particularly, it is unclear how the neurons in these cortical areas multiplex their traditional functions related to the control of spatial and temporal characteristics of movements with the representation of rewards. To clarify this issue, we trained rhesus monkeys to perform a center-out task in which arm movement direction, reward timing, and magnitude were manipulated independently. Activity of several hundred cortical neurons was simultaneously recorded using chronically implanted microelectrode arrays. Many neurons (9-27%) in both M1 and S1 exhibited activity related to reward anticipation. Additionally, neurons in these areas responded to a mismatch between the reward amount given to the monkeys and the amount they expected: A lower-than-expected reward caused a transient increase in firing rate in 60-80% of the total neuronal sample, whereas a larger-than-expected reward resulted in a decreased firing rate in 20-35% of the neurons. Moreover, responses of M1 and S1 neurons to reward omission depended on the direction of movements that led to those rewards. These observations suggest that sensorimotor cortical neurons corepresent rewards and movement-related activity, presumably to enable reward-based learning.

Sexual motivation is reflected by stimulus-dependent motor cortex excitability

Sexual behavior involves motivational processes. Findings from both animal models and neuroimaging in humans suggest that the recruitment of neural motor networks is an integral part of the sexual response. However, no study so far has directly linked sexual motivation to physiologically measurable changes in cerebral motor systems in humans. Using transcranial magnetic stimulation in hetero- and homosexual men, we here show that sexual motivation modulates cortical excitability. More specifically, our results demonstrate that visual sexual stimuli corresponding with one's sexual orientation, compared with non-corresponding visual sexual stimuli, increase the excitability of the motor cortex. The reflection of sexual motivation in motor cortex excitability provides evidence for motor preparation processes in sexual behavior in humans. Moreover, such interrelationship links theoretical models and previous neuroimaging findings of sexual behavior.

Modulation of corticospinal excitability by reward depends on task framing

Findings from previous transcranial magnetic stimulation (TMS) experiments suggest that the primary motor cortex (M1) is sensitive to reward conditions in the environment. However, the nature of this influence on M1 activity is poorly understood. The dopamine neuron response to conditioned stimuli encodes reward probability and outcome uncertainty, or the extent to which the outcome of a situation is known. Reward uncertainty and probability are related: uncertainty is maximal when probability is 0.5 and minimal when probability is 0 or 1 (i.e., certain outcome). Previous TMS-reward studies did not examine these factors independently. Here, we used single-pulse TMS to measure corticospinal excitability in 40 individuals while they performed a simple computer task, making guesses to find or avoid a hidden target. The task stimuli implied three levels of reward probability and two levels of uncertainty. We found that reward probability level interacted with the trial search condition. That is, motor evoked potential (MEP) amplitude, a measure of corticospinal neuron excitability, increased with increasing reward probability when participants were instructed to "find" a target, but not when they were instructed to "avoid" a target. There was no effect of uncertainty on MEPs. Response times varied with the number of choices. A subset of participants also received paired-pulse stimulation to evaluate changes in short-intracortical inhibition (SICI). No effects of SICI were observed. Taken together, the results suggest that the reward-contingent modulation of M1 activity reflects reward probability or a related aspect of utility, not outcome uncertainty, and that this effect is sensitive to the conceptual framing of the task.

Reward in the mirror neuron system, social context, and the implications on psychopathology

Positive and negative reinforcers guide our behaviors as we interact with others in our social environment. Here, we present evidence that highlights a central role for reward in the general functioning of the mirror neuron system (MNS). We also discuss the relevance of reward-related modulation on other previous findings revealing certain properties of the MNS, and on social context and psychopathology.

Motor cortex-evoked activity in reciprocal muscles is modulated by reward probability

Horizontal intracortical projections for agonist and antagonist muscles exist in the primary motor cortex (M1), and reward may induce a reinforcement of transmission efficiency of intracortical circuits. We investigated reward-induced change in M1 excitability for agonist and antagonist muscles. Participants were 8 healthy volunteers. Probabilistic reward tasks comprised 3 conditions of 30 trials each: 30 trials contained 10% reward, 30 trials contained 50% reward, and 30 trials contained 90% reward. Each trial began with a cue (red fixation cross), followed by blue circle for 1 s. The subjects were instructed to perform wrist flexion and press a button with the dorsal aspect of middle finger phalanx as quickly as possible in response to disappearance of the blue circle without looking at their hand or the button. Two seconds after the button press, reward/non-reward stimulus was randomly presented for 2-s duration. The reward stimulus was a picture of Japanese 10-yen coin, and each subject received monetary reward at the end of experiment. Subjects were not informed of the reward probabilities. We delivered transcranial magnetic stimulation of the left M1 at the midpoint between center of gravities of agonist flexor carpi radialis (FCR) and antagonist extensor carpi radialis (ECR) muscles at 2 s after the red fixation cross and 1 s after the reward/non-reward stimuli. Relative motor evoked potential (MEP) amplitudes at 2 s after the red fixation cross were significantly higher for 10% reward probability than for 90% reward probability, whereas relative MEP amplitudes at 1 s after reward/non-reward stimuli were significantly higher for 90% reward probability than for 10% and 50% reward probabilities. These results implied that reward could affect the horizontal intracortical projections in M1 for agonist and antagonist muscles, and M1 excitability including the reward-related circuit before and after reward stimulus could be differently altered by reward probability.

Opposing effects of appetitive and aversive cues on go/no-go behavior and motor excitability

Everyday life, as well as psychiatric illness, is replete with examples where appetitive and aversive stimuli hijack the will, leading to maladaptive behavior. Yet the mechanisms underlying this phenomenon are not well understood. Here we investigate how motivational cues influence action tendencies in healthy individuals with a novel paradigm. Behaviorally, we observed that an appetitive cue biased go behavior (making a response), whereas an aversive cue biased no-go behavior (withholding a response). We hypothesized that the origin of this behavioral go/no-go bias occurs at the motor system level. To test this, we used single-pulse TMS as a motor system probe (rather than a disruptive tool) to index motivational biasing. We found that the appetitive cue biased the participants to go more by relatively increasing motor system excitability, and that the aversive cue biased participants to no-go more by relatively decreasing motor system excitability. These results show, first, that maladaptive behaviors arise from motivational cues quickly spilling over into the motor system and biasing behavior even before action selection and, second, that this occurs in opposing directions for appetitive and aversive cues.

Neurophysiology of robot-mediated training and therapy: a perspective for future use in clinical populations

The recovery of functional movements following injury to the central nervous system (CNS) is multifaceted and is accompanied by processes occurring in the injured and non-injured hemispheres of the brain or above/below a spinal cord lesion. The changes in the CNS are the consequence of functional and structural processes collectively termed neuroplasticity and these may occur spontaneously and/or be induced by movement practice. The neurophysiological mechanisms underlying such brain plasticity may take different forms in different types of injury, for example stroke vs. spinal cord injury (SCI). Recovery of movement can be enhanced by intensive, repetitive, variable, and rewarding motor practice. To this end, robots that enable or facilitate repetitive movements have been developed to assist recovery and rehabilitation. Here, we suggest that some elements of robot-mediated training such as assistance and perturbation may have the potential to enhance neuroplasticity. Together the elemental components for developing integrated robot-mediated training protocols may form part of a neurorehabilitation framework alongside those methods already employed by therapists. Robots could thus open up a wider choice of options for delivering movement rehabilitation grounded on the principles underpinning neuroplasticity in the human CNS.

Single pulse TMS differentially modulates reward behavior

Greater knowledge of cortical brain regions in reward processing may set the stage for using transcranial magnetic stimulation (TMS) as a treatment in patients with avolition, apathy or other drive-related symptoms. This study examined the effects of single pulse (sp) TMS to two reward circuit targets on drive in healthy subjects. Fifteen healthy subjects performed the monetary incentive delay task (MID) while receiving fMRI-guided spTMS to either inferior parietal lobe (IPL) or supplemental motor area (SMA). The study demonstrated decreasing reaction times (RT) for increasing reward. It also showed significant differences in RT modulation for TMS pulses to the IPL versus the SMA. TMS pulses during the delay period produced significantly more RT slowing when targeting the IPL than those to the SMA. This RT slowing carried over into subsequent trials without TMS stimulation, with significantly slower RTs in sessions that had targeted the IPL compared to those targeting SMA. The results of this study suggest that both SMA and IPL are involved in reward processing, with opposite effects on RT in response to TMS stimulation. TMS to these target cortical regions may be useful in modulating reward circuit deficits in psychiatric populations.

Influence of reward on corticospinal excitability during movement preparation

Current models of decision making postulate that action selection entails a competition within motor-related areas. According to this view, during action selection, motor activity should integrate cognitive information (e.g., reward) that drives our decisions. We tested this hypothesis in humans by measuring motor-evoked potentials (MEPs) in a left finger muscle during motor preparation in a hand selection task, in which subjects performed left or right key presses according to an imperative signal. This signal was either obvious or ambiguous, but subjects were always asked to react as fast as possible. When the signal was really indistinct, any key press was regarded as correct, so subjects could respond "at random" in those trials. A score based on reaction times was provided after each correct response, and subjects were told they would receive a monetary reward proportional to their final score. Importantly, the scores were either equitable for both hands or favored implicitly left responses (reward(neutral) and reward(biased) blocks, respectively). We found that subjects selected their left hand more often in the reward(biased) than in the reward(neutral) condition, particularly after ambiguous signals. Moreover, left MEPs were larger, as soon as the signal appeared, in the reward(biased) than in the reward(neutral) conditions. During the course of motor preparation, this effect became strongest following ambiguous signals, a condition in which subjects' choices relied strongly on the reward. These results indicate that motor activity is shaped by a cognitive variable that drives our choices, possibly in the context of a competition taking place within motor-related areas.

Different mechanisms contributing to savings and anterograde interference are impaired in Parkinson's disease

Reinforcement and use-dependent plasticity mechanisms have been proposed to be involved in both savings and anterograde interference in adaptation to a visuomotor rotation (cf. Huang et al., 2011). In Parkinson's disease (PD), dopamine dysfunction is known to impair reinforcement mechanisms, and could also affect use-dependent plasticity. Here, we assessed savings and anterograde interference in PD with an A1-B-A2 paradigm in which movement repetition was (1) favored by the use of a single-target, and (2) manipulated through the amount of initial training. PD patients and controls completed either limited or extended training in A1 where they adapted movement to a 30° counter-clockwise rotation of visual feedback of the movement trajectory, and then adapted to a 30° clockwise rotation in B. After subsequent washout, participants readapted to the first 30° counter-clockwise rotation in A2. Controls showed significant anterograde interference from A1 to B only after extended training, and significant A1-B-A2 savings after both limited and extended training. However, despite similar A1 adaptation to controls, PD patients showed neither anterograde interference nor savings. That extended training was necessary in controls to elicit anterograde interference but not savings suggests that savings and anterograde interference do not result from equal contributions of the same underlying mechanism(s). It is suggested that use-dependent plasticity mechanisms contributes to anterograde interference but not to savings, while reinforcement mechanisms contribute to both. As both savings and anterograde interference were impaired in PD, dopamine dysfunction in PD might impair both reinforcement and use-dependent plasticity mechanisms during adaptation to a visuomotor rotation.

Common mechanisms of human perceptual and motor learning

The adult mammalian brain has a remarkable capacity to learn in both the perceptual and motor domains through the formation and consolidation of memories. Such practice-enabled procedural learning results in perceptual and motor skill improvements. Here, we examine evidence supporting the notion that perceptual and motor learning in humans exhibit analogous properties, including similarities in temporal dynamics and the interactions between primary cortical and higher-order brain areas. These similarities may point to the existence of a common general mechanism for learning in humans.

Time-dependent changes in human corticospinal excitability reveal value-based competition for action during decision processing

Our choices often require appropriate actions to obtain a preferred outcome, but the neural underpinnings that link decision making and action selection remain largely undetermined. Recent theories propose that action selection occurs simultaneously, i.e., parallel in time, with the decision process. Specifically, it is thought that action selection in motor regions originates from a competitive process that is gradually biased by evidence signals originating in other regions, such as those specialized in value computations. Biases reflecting the evaluation of choice options should thus emerge in the motor system before the decision process is complete. Using transcranial magnetic stimulation, we sought direct physiological evidence for this prediction by measuring changes in corticospinal excitability in human motor cortex during value-based decisions. We found that excitability for chosen versus unchosen actions distinguishes the forthcoming choice before completion of the decision process. Both excitability and reaction times varied as a function of the subjective value-difference between chosen and unchosen actions, consistent with this effect being value-driven. This relationship was not observed in the absence of a decision. Our data provide novel evidence in humans that internally generated value-based decisions influence the competition between action representations in motor cortex before the decision process is complete. This is incompatible with models of serial processing of stimulus, decision, and action.

The role of prediction in social neuroscience

Research has shown that the brain is constantly making predictions about future events. Theories of prediction in perception, action and learning suggest that the brain serves to reduce the discrepancies between expectation and actual experience, i.e., by reducing the prediction error. Forward models of action and perception propose the generation of a predictive internal representation of the expected sensory outcome, which is matched to the actual sensory feedback. Shared neural representations have been found when experiencing one's own and observing other's actions, rewards, errors, and emotions such as fear and pain. These general principles of the “predictive brain” are well established and have already begun to be applied to social aspects of cognition. The application and relevance of these predictive principles to social cognition are discussed in this article. Evidence is presented to argue that simple non-social cognitive processes can be extended to explain complex cognitive processes required for social interaction, with common neural activity seen for both social and non-social cognitions. A number of studies are included which demonstrate that bottom-up sensory input and top-down expectancies can be modulated by social information. The concept of competing social forward models and a partially distinct category of social prediction errors are introduced. The evolutionary implications of a “social predictive brain” are also mentioned, along with the implications on psychopathology. The review presents a number of testable hypotheses and novel comparisons that aim to stimulate further discussion and integration between currently disparate fields of research, with regard to computational models, behavioral and neurophysiological data. This promotes a relatively new platform for inquiry in social neuroscience with implications in social learning, theory of mind, empathy, the evolution of the social brain, and potential strategies for treating social cognitive deficits.

Is motor cortex excitability associated with personality factors? A replication study

In a previous study an association has been reported between motor cortex excitability, as measured by paired-pulse transcranial magnetic stimulation (TMS) and neuroticism of the NEO personality inventory; this correlation was carried by the men. The aim of the present study was to replicate these findings in a larger sample and with additional measures of motor cortex excitability. Eighty-nine healthy volunteers filled in the NEO-FFI and underwent several measures of motor cortex excitability (resting and active motor thresholds, double-pulse TMS with interstimulus intervals of 1-20ms, and cortical silent period). We did not find any systematic significant correlations of personality factors with motor cortex excitability. Dividing the samples by sex or controlling for confounders such as age, sex and education level by partial correlations did not reveal any significant associations either. Reasons for the failure of replication may include differences in sample characteristics, personality measures, and TMS methodology. However, synopsis of literature indicates that association of personality and motor cortex excitability might be mediated rather by state than by trait factors.

Reward processing abnormalities in Parkinson's disease

The primary motor cortex is important for motor learning and response selection, functions that require information on the expected and actual outcomes of behavior. Therefore, it should receive signals related to reward. Pathways from reward centers to motor cortex exist in primates. Previously, we showed that gamma aminobutyric acid-A-mediated inhibition in the motor cortex, measured by paired transcranial magnetic stimulation, changes with expectation and uncertainty of money rewards generated by a slot machine simulation. We examined the role of dopamine in this phenomenon by testing 13 mildly affected patients with Parkinson's disease, off and on dopaminergic medications, and 13 healthy, age-matched controls. Consistent with a dopaminergic mechanism, reward expectation or predictability modulated the response to paired transcranial magnetic stimulation in controls, but not in unmedicated patients. A single dose of pramipexole restored this effect of reward, mainly by increasing the paired transcranial magnetic stimulation response amplitude during low expectation. Levodopa produced no such effect. Both pramipexole and levodopa increased risk-taking behavior on the Iowa Gambling Task. However, pramipexole increased risk-taking behavior more in patients showing lower paired transcranial magnetic stimulation response amplitude during low expectation. These results provide evidence that modulation of motor cortex inhibition by reward is mediated by dopamine signaling and that the physiological state of the motor cortex changes with risk-taking tendency in patients on pramipexole. The cortical response to reward expectation may represent an endophenotype for risk-taking behavior in patients on agonist treatment.

Motor cortex inhibition: a marker of ADHD behavior and motor development in children

OBJECTIVE

Attention-deficit/hyperactivity disorder (ADHD) is a childhood-onset behavioral diagnosis in which children often fail to meet age norms in development of motor control, particularly timed repetitive and sequential movements, motor overflow, and balance. The neural substrate of this motor delay may include mechanisms of synaptic inhibition in or adjacent to the motor cortex. The primary objective of this study was to determine whether transcranial magnetic stimulation (TMS)-evoked measures, particularly short interval cortical inhibition (SICI), in motor cortex correlate with the presence and severity of ADHD in childhood as well as with commonly observed delays in motor control.

METHODS

In this case-control study, behavioral ratings, motor skills, and motor cortex physiology were evaluated in 49 children with ADHD (mean age 10.6 years, 30 boys) and 49 typically developing children (mean age 10.5 years, 30 boys), all right-handed, aged 8-12 years. Motor skills were evaluated with the Physical and Neurological Examination for Subtle Signs (PANESS) and the Motor Assessment Battery for Children version 2. SICI and other physiologic measures were obtained using TMS in the left motor cortex.

RESULTS

In children with ADHD, mean SICI was reduced by 40% (p < 0.0001) and less SICI correlated with higher ADHD severity (r = -0.52; p = 0.002). Mean PANESS motor development scores were 59% worse in children with ADHD (p < 0.0001). Worse PANESS scores correlated modestly with less SICI (r = -.30; p = 0.01).

CONCLUSION

Reduced TMS-evoked SICI correlates with ADHD diagnosis and symptom severity and also reflects motor skill development in children.

Momentary reward induce changes in excitability of primary motor cortex

OBJECTIVE

To investigate the human primary motor cortex (M1) excitability changes induced by momentary reward.

METHODS

To test the changes in excitatory and inhibitory functions of M1, motor-evoked potentials (MEPs), short-interval intracortical inhibition (SICI) and short-latency afferent inhibition (SAI) were tested in the abductor pollicis brevis (APB) muscle of non-dominant hand in 14 healthy volunteers by transcranial magnetic stimulation (TMS) during a behavioral task in which subjects were pseudorandomly received either reward target or non-target stimuli in response to a cue. To control sensorimotor and attention effects, a sensorimotor control task was done replacing the reward target with non-reward target.

RESULTS

The SICI was increased, and the SAI was decreased significantly during the presentation of the reward target stimuli. Those changes were not evident during non-reward target stimuli in the sensorimotor control task, indicating that this change is specific to momentary reward.

CONCLUSIONS

Momentary rewarding is associated with change in intracortical inhibitory circuits of M1.

SIGNIFICANCE

TMS may be a useful probe to study the reward system in health and in many diseases in which its dysfunction is suspected.

Reward improves long-term retention of a motor memory through induction of offline memory gains

In humans, training in which good performance is rewarded or bad performance punished results in transient behavioral improvements. The relative effects of reward and punishment on consolidation and long-term retention, critical behavioral stages for successful learning, are not known. Here, we investigated the effects of reward and punishment on these different stages of human motor skill learning. We studied healthy subjects who trained on a motor task under rewarded, punished, or neutral control conditions. Performance was tested before and immediately, 6 hr, 24 hr, and 30 days after training in the absence of reward or punishment. Performance improvements immediately after training were comparable in the three groups. At 6 hr, the rewarded group maintained performance gains, whereas the other two groups experienced significant forgetting. At 24 hr, the reward group showed significant offline (posttraining) improvements, whereas the other two groups did not. At 30 days, the rewarded group retained the gains identified at 24 hr, whereas the other two groups experienced significant forgetting. We conclude that training under rewarded conditions is more effective than training under punished or neutral conditions in eliciting lasting motor learning, an advantage driven by offline memory gains that persist over time.

Urges for food and money spill over into motor system excitability before action is taken

Much human behavior is driven by urges. Yet research into urges is hampered by a paucity of tools to objectively index their strength, timing and control. Here we used transcranial magnetic stimulation (TMS) and concurrent electromyography to examine whether urges for food and money are detectable via motor system excitability. In Experiment 1, we used a naturalistic food paradigm to show that food items that were most strongly wanted elicited the largest motor excitability, even before participants knew which response to make to get them. In Experiment 2a, we replicated the results using money - motor excitability was greater for larger monetary amounts. In Experiment 2b we show that monetary amount does not modulate motor excitability when participants simply observe, without having to take action. As the chief effect occurred prior to the subject knowing which motor response to make, it is not merely related to response preparation, and as the effect was present only when action was required, it is not merely related to increased arousal. Instead, the increased motor excitability likely indexes the degree of motivation a subject has to perform an action. Thus, we have used TMS to demonstrate that urges for food and money 'spill over' into the motor system. This is likely mediated by interactions between the limbic system (including the orbital frontal cortex) and the motor system, probably at the level of the basal ganglia. Implications are discussed for theories of embodied cognition and for methodological progress in studying urge control.

Mechanisms of attention for appetitive and aversive outcomes in Pavlovian conditioning

Different mechanisms of attention controlling learning have been proposed in appetitive and aversive conditioning. The aim of the present study was to compare attention and learning in a Pavlovian conditioning paradigm using visual stimuli of varying predictive value of either monetary reward (appetitive conditioning; 10p or 50p) or blast of white noise (aversive conditioning; 97 dB or 102 dB). Outcome values were matched across the two conditions with regard to their emotional significance. Sixty-four participants were allocated to one of the four conditions matched for age and gender. All participants underwent a discriminative learning task using pairs of visual stimuli that signalled a 100%, 50%, or 0% probability of receiving an outcome. Learning was measured using a 9-point Likert scale of expectancy of the outcome, while attention using an eyetracker device. Arousal and emotional conditioning were also evaluated. Dwell time was greatest for the full predictor in the noise groups, while in the money groups attention was greatest for the partial predictor over the other two predictors. The progression of learning was the same for both groups. These findings suggest that in aversive conditioning attention is driven by the predictive salience of the stimulus while in appetitive conditioning attention is error-driven, when emotional value of the outcome is comparable.

Transcranial magnetic stimulation in Clinical Pharmacology

Transcranial magnetic stimulation (TMS) is a safe and easy technique for stimulating neurons in the human central nervous system. Studies combining TMS with drugs in healthy subjects and patients have advanced our knowledge of how TMS activates brain circuits and led to new techniques for evaluating the function of specific systems. For example, TMS techniques can detect effects on axon membranes, glutamatergic and GABAergic synapses and the influence of catecholaminergic systems, as well as group differences due to genetic variations in the response to drugs. With this knowledge base, TMS can now be used to explore and compare the effects of drugs on brain systems and may also serve as a surrogate for behavioral responses in clinical trials.