Reward anticipation modulates primary motor cortex excitability during task preparation

Proactive reward in conflict tasks: Does it only enhance general performance or also modulate conflict effects?

In the present study, we investigated the influence of performance-contingent reward prospects on task performance across three visual conflict tasks with manual responses (Experiments 1 & 2: Simon and Stroop tasks; Experiment 3: Simon and Eriksen flanker task) using block-wise (Experiment 1) and trial-wise (Experiments 2 & 3) manipulations to signal the possibility of reward. Across all experiments, task performance (in reaction time and/or error rates) generally improved in reward compared with no-reward conditions in each conflict task. However, there was, if any, little evidence that the reward manipulation modulated the size of the mean conflict effects, and there was also no evidence for conflict-specific effects of reward when controlling for time-varying fluctuations in conflict processing via distributional analyses (delta plots). Thus, the results provide no evidence for conflict-specific accounts and instead favor performance-general accounts, where reward anticipation leads to overall performance improvements without affecting conflict effects. We discuss possible implications for how proactive control might modulate the interplay between target- and distractor-processing in conflict tasks.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Corticospinal excitability reductions during action preparation and action stopping in humans: Different sides of the same inhibitory coin?

Motor functions and cognitive processes are closely associated with each other. In humans, this linkage is reflected in motor system state changes both when an action must be prepared and stopped. Single-pulse transcranial magnetic stimulation showed that both action preparation and action stopping are accompanied by a reduction of corticospinal excitability, referred to as preparatory and response inhibition, respectively. While previous efforts have been made to describe both phenomena extensively, an updated and comprehensive comparison of the two phenomena is lacking. To ameliorate such deficit, this review focuses on the role and interpretation of single-coil (single-pulse and paired-pulse) and dual-coil TMS outcome measures during action preparation and action stopping in humans. To that effect, it aims to identify commonalities and differences, detailing how TMS-based outcome measures are affected by states, traits, and psychopathologies in both processes. Eventually, findings will be compared, and open questions will be addressed to aid future research.

The role of ventral tegmental area in chronic stroke rehabilitation: an exploratory study

Introduction

The acknowledged role of external rewards in chronic stroke rehabilitation, offering positive reinforcement and motivation, has significantly contributed to patient engagement and perseverance. However, the exploration of self-reward's importance in this context remains limited. This study aims to investigate the functional connectivity of the ventral tegmental area (VTA), a key node in the brain's reward circuitry, during motor task-based rehabilitation and its correlation with the recovery process.

Methods

Twelve right-handed healthy volunteers (4 men, 8 women, aged 57.4 ± 11.3 years) and twelve chronic stroke patients (5 men, 7 women, aged 48.1 ± 11.1 years) with clinically significant right-sided motor impairment (mean FM-UE score of 27.6 ± 8.7) participated. The analysis employed the CONN toolbox to assess the association between motor tasks and VTA connectivity using psychophysiological interaction (PPI).

Results

PPI analysis revealed motor-dependent changes in VTA connectivity, particularly with regions within the motor circuitry, cerebellum, and prefrontal cortex. Notably, stronger connectivity between the ipsilesional VTA and cerebellum was observed in healthy controls compared to chronic stroke patients, highlighting the importance of VTA-cerebellum interactions in motor function. Stroke patients' motor performance was associated with VTA modulation in areas related to both motor tasks and reward processing, emphasizing the role of self-reward processes in rehabilitation. Changes in VTA influence on motor circuitry were linked to improvements in motor performance resulting from rehabilitation.

Discussion

Our findings underscore the potential of neuroimaging techniques in quantifying and predicting rehabilitation outcomes by examining self-reward processes. The observed associations between VTA connectivity and motor performance in both healthy and stroke-affected individuals emphasize the role of psychological factors, particularly self-reward, in the rehabilitation process. This study contributes valuable insights into the intricate interplay between reward circuits and motor function, highlighting the importance of addressing psychological dimensions in neurorehabilitation strategies.

The intracortical excitability changes underlying the enhancing effects of rewards and punishments on motor performance

Monetary rewards and punishments enhance motor performance and are associated with corticospinal excitability (CSE) increases within the motor cortex (M1) during movement preparation. However, such CSE changes have unclear origins. Based on converging evidence, one possibility is that they stem from increased glutamatergic (GLUTergic) facilitation and/or decreased type A gamma-aminobutyric acid (GABAA)-mediated inhibition within M1. To investigate this, paired-pulse transcranial magnetic stimulation was used over the left M1 to evaluate intracortical facilitation (ICF) and short intracortical inhibition (SICI), indirect assays of GLUTergic activity and GABAA-mediated inhibition, in an index finger muscle during the preparation of sequences initiated by either the right index or little finger. Behaviourally, rewards and punishments enhanced both reaction and movement time. During movement preparation, regardless of rewards or punishments, ICF increased when the index finger initiated sequences, whereas SICI decreased when both the index and little fingers initiated sequences. This finding suggests that GLUTergic activity increases in a finger-specific manner whilst GABAA-mediated inhibition decreases in a finger-unspecific manner during preparation. In parallel, both rewards and punishments non-specifically increased ICF, but only rewards non-specifically decreased SICI as compared to neutral. This suggests that to enhance performance rewards both increase GLUTergic activity and decrease GABAA-mediated inhibition, whereas punishments selectively increase GLUTergic activity. A control experiment revealed that such changes were not observed post-movement as participants processed reward and punishment feedback, indicating they were selective to movement preparation. Collectively, these results map the intracortical excitability changes in M1 by which incentives enhance motor performance.

The influence of reward in the Simon task: Differences and similarities to the Stroop and Eriksen flanker tasks

Previous studies have suggested that performance-contingent reward can modulate cognitive control by biasing irrelevant location-response associations in the Simon task. However, the influence of reward in the case of irrelevant words (Stroop task) or irrelevant flankers (Eriksen Flanker task) remains unclear. Across two preregistered experiments, the present study investigated the influence of reward on conflict processing with different types of distractors. Conflict effects on mean reaction time (RT) were reduced in the Simon task (Experiments 1 and 2) when incongruent versus congruent trials were rewarded, and this modulating effect of reward on conflict processing was also observed in the Eriksen flanker task (Experiment 2), but not in the Stroop task (Experiment 1). We propose that cognitive control adjustments to distractor-specific reward contingencies can be generalized across distractor types producing both perceptual-related (Flanker task) and motor-related (Simon task) conflict, but, if any, to a limited degree when distractors produce additional higher-level task conflict (Stroop task). In addition, distributional RT analyses (delta plots) revealed that rewarded distractor-response associations modulate cognitive control not only via biasing the strength (Simon and Eriksen tasks) but also the time-course of suppressing distractor processing (Eriksen task). Overall, the present study dissociated distractor-general and distractor-specific effects of reward on cognitive control.

The reward for placebos: mechanisms underpinning placebo-induced effects on motor performance

Different from the most popular thinking, the placebo effect is not a purely psychological phenomenon. A body of knowledge from multidisciplinary fields has shown that the expectation of a potential benefit when receiving a treatment induces a cascade of neurochemical-electrophysiological alterations in brain reward areas, including motor-related ones. Alterations in the dopamine, opioid, and glutamate metabolism are the neural representation converting reward-derived declarative forms into an attractive and wanted behavior, thereby changing the activation in reward subcortical and cortical structures involved in motor planning, motor execution, and emotional-cognitive attributes of decision-making. We propose that the expectation of receiving a treatment that is beneficial to motor performance triggers a cascade of activations in brain reward areas that travels from motor planning and motor command areas, passing through corticospinal pathways until driving the skeletal muscles, therefore facilitating the motor performance. Although alternative explanations cannot be totally ruled out, this mechanistic route is robust in explaining the results of placebo-induced effects on motor performance and could lead to novel insights and applications in the exercise sciences. Factors such as sex differences in reward-related mechanisms and aversion-induced nocebo effects should also be addressed.

Action Selection and Motor Decision Making: Insights from Transcranial Magnetic Stimulation

In everyday life, goal-oriented motor behaviour relies on the estimation of the rewards/costs associated with alternative actions and on the appropriate selection of movements. Motor decision making is defined as the process by which a motor plan is chosen among a set of competing actions based on the expected value. In the present literature review we discuss evidence from transcranial magnetic stimulation (TMS) studies of motor control. We focus primarily on studies of action selection for instructed movements and motor decision making. In the first section, we delve into the usefulness of various TMS paradigms to characterise the contribution of motor areas and distributed brain networks to cued action selection. Then, we address the influence of motivational information (e.g., reward and biomechanical cost) in guiding action choices based on TMS findings. Finally, we conclude that TMS represents a powerful tool for elucidating the neurophysiological mechanisms underlying action choices in humans.

Response bias reveals the role of interhemispheric inhibitory networks in movement preparation and execution

Human movement is influenced by various cognitive processes, such as bias, that dynamically shape competing movement representations. However, the neurophysiological mechanisms underlying the effects of bias on movement selection across the lifespan remains poorly understood. Healthy young (n = 21) and older (n = 20) adults completed a choice reaction-time task necessitating left- or right-hand responses to imperative stimuli (IS). Response bias was manipulated via a cue that informed participants a particular response was 70% likely (i.e., the IS was either congruent, or incongruent, with the cue); biasing was either fixed for blocks of trials (block-wise bias) or varied from trial-to-trial (trial-wise bias). As well as assessing the behavioural manifestations of bias, we used transcranial magnetic stimulation to determine changes in corticospinal excitability (CSE) and short- and long-interval interhemispheric inhibition (SIHI, LIHI) during movement preparation and execution. Participants responded more quickly, and accurately, in congruent compared to incongruent trials. CSE decreases occurred in both hands following the cue, consistent with the 'inhibition for impulse control' hypothesis of preparatory inhibition. In contrast, IHI modulations occurred in a hand-specific manner. Greater SIHI was observed during movement preparation in the hand biased away from, compared to the hand biased towards, the cue; furthermore, greater SIHI was observed during movement execution in the hand biased towards the cue when it was not required to respond (i.e., incongruent trial) compared to when it was required to respond (congruent trial). Additionally, during the movement preparation period, the LIHI ratio of the hand biased towards, compared to the hand biased away from, the cue was greatest when the cue varied trial-by-trial. Overall, the IHI results provide support for the 'inhibition for competition resolution' hypothesis, with hand specific modulation of inhibition during movement preparation and execution.

Reward facilitates response conflict resolution via global motor inhibition: Electromyography evidence

It is crucial for humans to coordinate between behavioural tendencies that can lead to reward but are in conflict with each other. This response conflict can be measured in a reward-modulated Simon task, in which a discriminative response to the identity of a lateral target is required and the target is associated with either high- or low-reward. Critically, the lateral target is presented either congruent or incongruent with the location of the responding hand. It has been shown that relative to the low-reward target, the high-reward target induced a larger response conflict when the target was incongruent with the position of the task-required, reward-obtaining hand. Here we investigated how this response conflict is resolved by acquiring 24 healthy participants' electromyography (EMG) signals from both the task-required responding hand (i.e., goal-directed effector) and the alternative hand (i.e., inappropriate effector). During the coping with the response conflict, motor inhibition (indexed by reduction in EMG signals between conditions) was observed not only at the inappropriate effector but also at the goal-directed effector. Individuals who showed stronger inhibition on the inappropriate effector suffered less from the inhibition on the goal-directed effector, and had more efficient implementation of the reward-obtaining response. Our findings suggest a global motor inhibition that may function to increase the signal-noise ratio in the motor system so as to implement reward-guided behavior.

Reward does not modulate corticospinal excitability in anticipation of a Stroop trial

Action preparation is associated with a transient decrease of corticospinal excitability just before target onset. We have previously shown that the prospect of reward modulates preparatory corticospinal excitability in a Simon task. While the conflict in the Simon task strongly implicates the motor system, it is unknown whether reward prospect modulates preparatory corticospinal excitability in tasks that implicate the motor system less directly. To that effect, we examined reward-modulated preparatory corticospinal excitability in the Stroop task. We administered a rewarded cue-target delay paradigm using Stroop stimuli that afforded a left or right index finger response. Single-pulse transcranial magnetic stimulation was administered over the left primary motor cortex and electromyography was obtained from the right first dorsal interosseous muscle. In line with previous findings, there was a preparatory decrease in corticospinal excitability during the delay period. In contrast to our previous study using the Simon task, preparatory corticospinal excitability was not modulated by reward. Our results indicate that reward-modulated changes in the motor system depend on specific task-demands, possibly related to varying degrees of motor conflict.

Neural Dynamics of Reward-Induced Response Activation and Inhibition

Reward-predictive stimuli can increase an automatic response tendency, which needs to be counteracted by effortful response inhibition when this tendency is inappropriate for the current task. Here we investigated how the human brain implements this dynamic process by adopting a reward-modulated Simon task while acquiring EEG and fMRI data in separate sessions. In the Simon task, a lateral target stimulus triggers an automatic response tendency of the spatially corresponding hand, which needs to be overcome if the activated hand is opposite to what the task requires, thereby delaying the response. We associated high or low reward with different targets, the location of which could be congruent or incongruent with the correct response hand. High-reward targets elicited larger Simon effects than low-reward targets, suggesting an increase in the automatic response tendency induced by the stimulus location. This tendency was accompanied by modulations of the lateralized readiness potential over the motor cortex, and was inhibited soon after if the high-reward targets were incongruent with the correct response hand. Moreover, this process was accompanied by enhanced theta oscillations in medial frontal cortex and enhanced activity in a frontobasal ganglia network. With dynamical causal modeling, we further demonstrated that the connection from presupplementary motor area (pre-SMA) to right inferior frontal cortex (rIFC) played a crucial role in modulating the reward-modulated response inhibition. Our results support a dynamic neural model of reward-induced response activation and inhibition, and shed light on the neural communication between reward and cognitive control in generating adaptive behaviors.

Reward-driven enhancements in motor control are robust to TMS manipulation

A wealth of evidence describes the strong positive impact that reward has on motor control at the behavioural level. However, surprisingly little is known regarding the neural mechanisms which underpin these effects, beyond a reliance on the dopaminergic system. In recent work, we developed a task that enabled the dissociation of the selection and execution components of an upper limb reaching movement. Our results demonstrated that both selection and execution are concommitently enhanced by immediate reward availability. Here, we investigate what the neural underpinnings of each component may be. To this end, we aimed to alter the cortical excitability of the ventromedial prefrontal cortex and supplementary motor area using continuous theta-burst transcranial magnetic stimulation (cTBS) in a within-participant design (N = 23). Both cortical areas are involved in determining an individual’s sensitivity to reward and physical effort, and we hypothesised that a change in excitability would result in the reward-driven effects on action selection and execution to be altered, respectively. To increase statistical power, participants were pre-selected based on their sensitivity to reward in the reaching task. While reward did lead to enhanced performance during the cTBS sessions and a control sham session, cTBS was ineffective in altering these effects. These results may provide evidence that other areas, such as the primary motor cortex or the premotor area, may drive the reward-based enhancements of motor performance.

Reward-Based Improvements in Motor Control Are Driven by Multiple Error-Reducing Mechanisms

Reward has a remarkable ability to invigorate motor behavior, enabling individuals to select and execute actions with greater precision and speed. However, if reward is to be exploited in applied settings, such as rehabilitation, a thorough understanding of its underlying mechanisms is required. In a series of experiments, we first demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Specifically, reward promoted the selection of the correct action in the presence of distractors, while also improving execution through increased speed and maintenance of accuracy. These results led to a shift in the speed-accuracy functions for both selection and execution. In addition, punishment had a similar impact on action selection and execution, although it enhanced execution performance across all trials within a block, that is, its impact was noncontingent to trial value. Although the reward-driven enhancement of movement execution has been proposed to occur through enhanced feedback control, an untested possibility is that it is also driven by increased arm stiffness, an energy-consuming process that enhances limb stability. Computational analysis revealed that reward led to both an increase in feedback correction in the middle of the movement and a reduction in motor noise near the target. In line with our hypothesis, we provide novel evidence that this noise reduction is driven by a reward-dependent increase in arm stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate motor performance without compromising accuracy.SIGNIFICANCE STATEMENT While reward is well-known for enhancing motor performance, how the nervous system generates these improvements is unclear. Despite recent work indicating that reward leads to enhanced feedback control, an untested possibility is that it also increases arm stiffness. We demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Furthermore, we show that punishment has a similar positive impact on performance. Importantly, by combining computational and biomechanical approaches, we show that reward leads to both improved feedback correction and an increase in stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate performance without compromising accuracy. This work suggests that stiffness control plays a vital, and underappreciated, role in the reward-based imporvemenets in motor control.

Motor Memory: Revealing Conditioned Action Tendencies Using Transcranial Magnetic Stimulation

Action tendencies can be elicited by motivationally salient stimuli (e.g., appetitive rewards) or objects that support utilization behaviors. These action tendencies can benefit behavioral performance through speeded RTs in response tasks and improve detection accuracy in attentional capture tasks. However, action tendencies can be counterproductive when goals change (e.g., refraining from junk foods or abstaining from alcohol). Maintaining control over cue-elicited action tendencies is therefore critical for successful behavior modification. To better understand this relationship, we used transcranial magnetic stimulation to investigate the neural signatures of action tendencies in the presence of previously trained response cues. Participants were presented with a continuous letter stream and instructed to respond quickly to two target letters using two different response keys. Following this training phase, the target letters were embedded in a new task (test phase), and we applied transcranial magnetic stimulation to the motor cortex and measured motor evoked potentials as an index of corticospinal excitability (CSE). We found that CSE could be potentiated by a former response cue trained within a single experimental session, even when participants were instructed to withhold responses during the test phase. Critically, attention to the previously trained response cue was required to elicit the primed modulation in CSE, and successful control of this activity was accompanied by CSE suppression. These findings suggest that well-trained response cues can come to prime a conditioned action tendency and provide a model for understanding how the implementation of cognitive control can override action automaticity.

Reward anticipation changes corticospinal excitability during task preparation depending on response requirements and time pressure

The preparation of an action is accompanied by transient corticospinal (CS) excitability changes. Motivation can modulate these changes. Specifically, when a cue indicates that a reward can be obtained, CS excitability initially increases, followed by a pronounced decrease. This dynamic could reflect processes related to reward expectancy, processes related to action preparation, or a combination of both. Here we set up two experiments to dissociate these accounts. A rewarded choice reaction time task was used in which individuals were cued at the beginning of each trial whether or not a response would be required at target onset and whether or not a reward could be obtained. We used single-pulse transcranial magnetic stimulation (spTMS) over the left primary motor cortex (M1) early (shortly after cue onset) or late (shortly before target onset) preceding target onset to examine CS excitability during motivated action preparation. Electromyography (EMG) was obtained from the right first dorsal interosseous (FDI) muscle. In the first experiment, we used a lenient response deadline, whereas a strict response time-out procedure was employed in the second experiment. Reward modulated CS excitability differentially only in the second experiment: CS excitability was highest during reward anticipation for the early stimulation epoch and was reduced for the late stimulation epoch when individuals were required to prepare a response, while CS excitability remained unchanged during non-reward anticipation. Our findings suggest that the reward effect on CS excitability is dependent on the actual implementation of effort to attain reward (i.e., the preparation of an actual action), as well as on temporal requirements (i.e., time pressure) invoked by the task.

Reward anticipation and punishment anticipation are instantiated in the brain via opponent mechanisms

fMRI investigations have examined the extent to which reward and punishment motivation are associated with common or opponent neural systems, but such investigations have been limited by confounding variables and methodological constraints. The present study aimed to address limitations of earlier approaches and more comprehensively evaluate the extent to which neural activation associated with reward and punishment motivation reflects opponent or shared systems. Participants completed a modified monetary incentive delay task, which involved the presentation of a cue followed by a target to which participants were required to make a speeded button press. Using a factorial design, cues indicated whether monetary reward and/or loss (i.e., cues signaled probability of reward, punishment, both, or neither) could be expected depending upon response speed. Neural analyses evaluated evidence of (a) directionally opposing effects by testing for regions of differential activation for reward and punishment anticipation, (b) mutual inhibition by testing for interactive effects of reward and punishment anticipation within a factorial design, and (c) opposing effects on shared outputs via a psychophysiological interaction analysis. Evidence supporting all three criteria for opponent systems was obtained. Collectively, present findings support conceptualizing reward and punishment motivation as opponent forces influencing brain and behavior and indicate that shared activation does not suggest the operation of a common neural mechanism instantiating reward and punishment motivation.