Added value of money on motor performance feedback: Increased left central beta-band power for rewards and fronto-central theta-band power for punishments

Context Modulates Perceived Fairness in Altruistic Punishment: Neural Signatures from ERPs and EEG Oscillations

Social norms and altruistic punitive behaviours are both based on the integration of information from multiple contexts. Individual behavioural performance can be altered by loss and gain contexts, which produce different mental states and subjective perceptions. In this study, we used event-related potential and time-frequency techniques to examine performance on a third-party punishment task and to explore the neural mechanisms underlying context-dependent differences in punishment decisions. The results indicated that individuals were more likely to reject unfairness in the context of loss (vs. gain) and to increase punishment as unfairness increased. In contrast, fairness appeared to cause an early increase in cognitive control signal enhancement, as indicated by the P2 amplitude and theta oscillations, and a later increase in emotional and motivational salience during decision-making in gain vs. loss contexts, as indicated by the medial frontal negativity and beta oscillations. In summary, individuals were more willing to sanction violations of social norms in the loss context than in the gain context and rejecting unfair losses induced more equity-related cognitive conflict than accepting unfair gains, highlighting the importance of context (i.e., gain vs. loss) in equity-related social decision-making processes.

The effects of reward and punishment on the performance of ping-pong ball bouncing

Introduction

Reward and punishment modulate behavior. In real-world motor skill learning, reward and punishment have been found to have dissociable effects on optimizing motor skill learning, but the scientific basis for these effects is largely unknown.

Methods

In the present study, we investigated the effects of reward and punishment on the performance of real-world motor skill learning. Specifically, three groups of participants were trained and tested on a ping-pong ball bouncing task for three consecutive days. The training and testing sessions were identical across the three days: participants were trained with their right (dominant) hand each day under conditions of either reward, punishment, or a neutral control condition (neither). Before and after the training session, all participants were tested with their right and left hands without any feedback.

Results

We found that punishment promoted early learning, while reward promoted late learning. Reward facilitated short-term memory, while punishment impaired long-term memory. Both reward and punishment interfered with long-term memory gains. Interestingly, the effects of reward and punishment transferred to the left hand.

Discussion

The results show that reward and punishment have different effects on real-world motor skill learning. The effects change with training and transfer readily to novel contexts. The results suggest that reward and punishment may act on different learning processes and engage different neural mechanisms during real-world motor skill learning. In addition, high-level metacognitive processes may be enabled by the additional reinforcement feedback during real-world motor skill learning. Our findings provide new insights into the mechanisms underlying motor learning, and may have important implications for practical applications such as sports training and motor rehabilitation.

Reinforcement feedback impairs locomotor adaptation and retention

Introduction

Locomotor adaptation is a motor learning process used to alter spatiotemporal elements of walking that are driven by prediction errors, a discrepancy between the expected and actual outcomes of our actions. Sensory and reward prediction errors are two different types of prediction errors that can facilitate locomotor adaptation. Reward and punishment feedback generate reward prediction errors but have demonstrated mixed effects on upper extremity motor learning, with punishment enhancing adaptation, and reward supporting motor memory. However, an in-depth behavioral analysis of these distinct forms of feedback is sparse in locomotor tasks.

Methods

For this study, three groups of healthy young adults were divided into distinct feedback groups [Supervised, Reward, Punishment] and performed a novel locomotor adaptation task where each participant adapted their knee flexion to 30 degrees greater than baseline, guided by visual supervised or reinforcement feedback (Adaptation). Participants were then asked to recall the new walking pattern without feedback (Retention) and after a washout period with feedback restored (Savings).

Results

We found that all groups learned the adaptation task with external feedback. However, contrary to our initial hypothesis, enhancing sensory feedback with a visual representation of the knee angle (Supervised) accelerated the rate of learning and short-term retention in comparison to monetary reinforcement feedback. Reward and Punishment displayed similar rates of adaptation, short-term retention, and savings, suggesting both types of reinforcement feedback work similarly in locomotor adaptation. Moreover, all feedback enhanced the aftereffect of locomotor task indicating changes to implicit learning.

Discussion

These results demonstrate the multi-faceted nature of reinforcement feedback on locomotor adaptation and demonstrate the possible different neural substrates that underly reward and sensory prediction errors during different motor tasks.

Differential effects of reward and punishment on reinforcement-based motor learning and generalization

Reward and punishment have long been recognized as potent modulators of human behavior. Although reinforcement learning is a significant motor learning process, the exact mechanisms underlying how the brain learns movements through reward and punishment are not yet fully understood. Beyond the memory of specific examples, investigating the ability to generalize to new situations offers a better understanding of motor learning. This study hypothesizes that reward and punishment engage qualitatively different motivational systems with different neurochemical and neuroanatomical substrates, which would have differential effects on reinforcement-based motor learning and generalization. To test this hypothesis, two groups of participants learn a motor task in one direction and then relearn the same task in a new direction, receiving only performance-based reward or punishment score feedback. Our findings support our hypothesis, showing that reward led to slower learning but promoted generalization. On the other hand, punishment led to faster learning but impaired generalization. These behavioral differences may be due to different tendencies of movement variability in each group. The punishment group tended to explore more actively than the reward group during the initial learning phase, possibly due to loss aversion. In contrast, the reward group tended to explore more actively than the initial learning phase during the generalization test phase, seemingly recalling the strategy that led to the reward. These results suggest that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization, with important implications for practical applications such as sports training and motor rehabilitation.NEW & NOTEWORTHY Although reinforcement learning is a significant motor learning process, the mechanisms underlying how the brain learns movements through reward and punishment are not fully understood. We modified a well-established motor adaptation task and used savings (faster relearning) to measure generalization. We found reward led to slower learning but promoted generalization, whereas punishment led to faster learning but impaired generalization, suggesting that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization.

The intracortical excitability changes underlying the enhancing effects of rewards and punishments on motor performance

Monetary rewards and punishments enhance motor performance and are associated with corticospinal excitability (CSE) increases within the motor cortex (M1) during movement preparation. However, such CSE changes have unclear origins. Based on converging evidence, one possibility is that they stem from increased glutamatergic (GLUTergic) facilitation and/or decreased type A gamma-aminobutyric acid (GABAA)-mediated inhibition within M1. To investigate this, paired-pulse transcranial magnetic stimulation was used over the left M1 to evaluate intracortical facilitation (ICF) and short intracortical inhibition (SICI), indirect assays of GLUTergic activity and GABAA-mediated inhibition, in an index finger muscle during the preparation of sequences initiated by either the right index or little finger. Behaviourally, rewards and punishments enhanced both reaction and movement time. During movement preparation, regardless of rewards or punishments, ICF increased when the index finger initiated sequences, whereas SICI decreased when both the index and little fingers initiated sequences. This finding suggests that GLUTergic activity increases in a finger-specific manner whilst GABAA-mediated inhibition decreases in a finger-unspecific manner during preparation. In parallel, both rewards and punishments non-specifically increased ICF, but only rewards non-specifically decreased SICI as compared to neutral. This suggests that to enhance performance rewards both increase GLUTergic activity and decrease GABAA-mediated inhibition, whereas punishments selectively increase GLUTergic activity. A control experiment revealed that such changes were not observed post-movement as participants processed reward and punishment feedback, indicating they were selective to movement preparation. Collectively, these results map the intracortical excitability changes in M1 by which incentives enhance motor performance.

Exploring the role of task success in implicit motor adaptation

We learn to improve our motor skills using different forms of feedback: sensory-prediction error, task success, and reward/punishment. While implicit motor adaptation is driven by sensory-prediction errors, recent work has shown that task success modulates this process. Task success is often confounded with reward, so we sought to determine if the effects of these two signals on adaptation can be dissociated. To address this question, we conducted five experiments that isolated implicit learning using error-clamp visuomotor reach adaptation paradigms. Task success was manipulated by changing the size and position of the target relative to the cursor providing visual feedback, and reward expectation was established using monetary cues and auditory feedback. We found that neither monetary cues nor auditory feedback affected implicit adaptation, suggesting that task success influences implicit adaptation via mechanisms distinct from conventional reward-related processes. Additionally, we found that changes in target size, which caused the target to either exclude or fully envelop the cursor, only affected implicit adaptation for a narrow range of error sizes, while jumping the target to overlap with the cursor more reliably and robustly affected implicit adaptation. Taken together, our data indicate that, while task success exerts a small effect on implicit adaptation, these effects are susceptible to methodological variations and unlikely to be mediated by reward.

Losses resulting from deliberate exploration trigger beta oscillations in frontal cortex

We examined the neural signature of directed exploration by contrasting MEG beta (16-30 Hz) power changes between disadvantageous and advantageous choices in the two-choice probabilistic reward task. We analyzed the choices made after the participants have learned the probabilistic contingency between choices and their outcomes, i.e., acquired the inner model of choice values. Therefore, rare disadvantageous choices might serve explorative, environment-probing purposes. The study brought two main findings. Firstly, decision making leading to disadvantageous choices took more time and evidenced greater large-scale suppression of beta oscillations than its advantageous alternative. Additional neural resources recruited during disadvantageous decisions strongly suggest their deliberately explorative nature. Secondly, an outcome of disadvantageous and advantageous choices had qualitatively different impact on feedback-related beta oscillations. After the disadvantageous choices, only losses-but not gains-were followed by late beta synchronization in frontal cortex. Our results are consistent with the role of frontal beta oscillations in the stabilization of neural representations for selected behavioral rule when explorative strategy conflicts with value-based behavior. Punishment for explorative choice being congruent with its low value in the reward history is more likely to strengthen, through punishment-related beta oscillations, the representation of exploitative choices consistent with the inner utility model.

Midfrontal theta power encodes the value of haptic delay

The use of haptic technologies in modern life scenarios is becoming the new normal particularly in rehabilitation, medical training, and entertainment applications. An evident challenge in haptic telepresence systems is the delay in haptic information. How humans perceive delayed visual and audio information has been extensively studied, however, the same for haptically delayed environments remains largely unknown. Here, we develop a visuo-haptic experimental setting that simulates pick and place task and involves continuous haptic feedback stimulation with four possible haptic delay levels. The setting is built using a haptic device and a computer screen. We use electroencephalography (EEG) to study the neural correlates that could be used to identify the amount of the experienced haptic delay. EEG data were collected from 34 participants. Results revealed that midfrontal theta oscillation plays a pivotal role in quantifying the amount of haptic delay while parietal alpha showed a significant modulation that encodes the presence of haptic delay. Based on the available literature, these results suggest that the amount of haptic delay is proportional to the neural activation that is associated with conflict detection and resolution as well as for multi-sensory divided attention.

Novelty processing associated with neural beta oscillations improves recognition memory in young and older adults

Novelty anticipation activates the mesolimbic system and promotes subsequent long-term memory in younger adults. Importantly, mesolimbic structures typically degenerate with age, which might reduce positive effects of novelty anticipation. Here, we used electroencephalography in combination with an established paradigm in healthy young (19-33 years old, n = 28) and older (53-84, n = 27) humans. Colored cues predicted the subsequent presentation of either a novel or previously familiarized image (75% cue validity). On the subsequent day, recognition memory for the novel images was tested. Behaviorally, novelty anticipation improved recollection-based but not familiarity-based recognition memory in both groups, and this effect was more pronounced in older subjects. Furthermore, novelty and familiarity cues increased theta (4-8 Hz) and decreased alpha/beta power (9-20 Hz); at outcome, expected novel and familiar images both increased beta power (13-25 Hz). Finally, a subsequent memory effect for expected novel images was associated with increases in beta power independent of age. Together, novelty anticipation drives hippocampus-dependent long-term recognition memory across the life span, and this effect appears to be related to neural beta oscillations.

Punishments and rewards both modestly impair visuomotor memory retention

While the effects of rewards on memory appear well documented, the effects of punishments remain uncertain. Based on neuroimaging data, this study tested the hypothesis that, as compared to a neutral condition, a context allowing successful punishment avoidance would enhance memory to a similar extent as rewards. In a fully within-subject and counter-balanced design, participants (n = 18) took part in 3 distinct learning sessions during which the delivery of performance-contingent monetary punishments and rewards was manipulated. Specifically, participants had to reach towards visual targets while compensating for a gradually introduced visual deviation. Accuracy at achieving targets was either punished (Hit: "+0$"; Miss: "-0.5$), rewarded (Hit: "+0.5$"; Miss: "-0$"), or associated with neutral binary feedback (Hit: "Hit"; Miss: "Miss"). Retention was assessed through reach aftereffects both immediately and 24 h after initial acquisition. The results disconfirmed the hypothesis by showing that the punishment and reward learning sessions both impaired retention as compared to the neutral session, suggesting that both types of incentives similarly impaired memory formation and consolidation. Two alternative but complementary interpretations are discussed. One interpretation is that the presence of punishments and rewards induced a negative learning context, which - based on neurobiological data - could have been sufficient to interfere with memory formation and consolidation. Another interpretation is that punishments and rewards emphasized the disrupting effects of target hits on implicit learning processes, therefore yielding retention impairments. Altogether, these results suggest that incentives can have counter-productive effects on memory.

Reward boosts reinforcement-based motor learning

Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we investigated the net effect of motivation by reward on motor learning while controlling for the sensory and reinforcement feedback received by the participants. Reward improved motor skill learning beyond performance-based reinforcement feedback. Importantly, the beneficial effect of reward involved a specific potentiation of reinforcement-related adjustments in motor commands, which concerned primarily the most relevant motor component for task success and persisted on the following day in the absence of reward. We propose that the long-lasting effects of motivation on motor learning may entail a form of associative learning resulting from the repetitive pairing of the reinforcement feedback and reward during training, a mechanism that may be exploited in future rehabilitation protocols.

Association of short- and long-latency afferent inhibition with human behavior

Transcranial magnetic stimulation (TMS) paired with nerve stimulation evokes short-latency afferent inhibition (SAI) and long-latency afferent inhibition (LAI), which are non-invasive assessments of the excitability of the sensorimotor system. SAI and LAI are abnormally reduced in various special populations in comparison to healthy controls. However, the relationship between afferent inhibition and human behavior remains unclear. The purpose of this review is to survey the current literature and synthesize observations and patterns that affect the interpretation of SAI and LAI in the context of human behavior. We discuss human behaviour across the motor and cognitive domains, and in special and control populations. Further, we discuss future considerations for research in this field and the potential for clinical applications. By understanding how human behavior is mediated by changes in SAI and LAI, this can allow us to better understand the neurophysiological underpinnings of human motor control.

Reward-Based Improvements in Motor Control Are Driven by Multiple Error-Reducing Mechanisms

Reward has a remarkable ability to invigorate motor behavior, enabling individuals to select and execute actions with greater precision and speed. However, if reward is to be exploited in applied settings, such as rehabilitation, a thorough understanding of its underlying mechanisms is required. In a series of experiments, we first demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Specifically, reward promoted the selection of the correct action in the presence of distractors, while also improving execution through increased speed and maintenance of accuracy. These results led to a shift in the speed-accuracy functions for both selection and execution. In addition, punishment had a similar impact on action selection and execution, although it enhanced execution performance across all trials within a block, that is, its impact was noncontingent to trial value. Although the reward-driven enhancement of movement execution has been proposed to occur through enhanced feedback control, an untested possibility is that it is also driven by increased arm stiffness, an energy-consuming process that enhances limb stability. Computational analysis revealed that reward led to both an increase in feedback correction in the middle of the movement and a reduction in motor noise near the target. In line with our hypothesis, we provide novel evidence that this noise reduction is driven by a reward-dependent increase in arm stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate motor performance without compromising accuracy.SIGNIFICANCE STATEMENT While reward is well-known for enhancing motor performance, how the nervous system generates these improvements is unclear. Despite recent work indicating that reward leads to enhanced feedback control, an untested possibility is that it also increases arm stiffness. We demonstrate that reward simultaneously improves the selection and execution components of a reaching movement. Furthermore, we show that punishment has a similar positive impact on performance. Importantly, by combining computational and biomechanical approaches, we show that reward leads to both improved feedback correction and an increase in stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate performance without compromising accuracy. This work suggests that stiffness control plays a vital, and underappreciated, role in the reward-based imporvemenets in motor control.

Differential motor learning via reward and punishment

Visuomotor adaptation involves multiple processes such as explicit learning, implicit learning from sensory prediction errors, and model-free mechanisms like use-dependent plasticity. Recent findings show that reward and punishment differently affect visuomotor adaptation. This study examined whether punishment and reward had distinct effects on explicit learning. When participants practised adapting to a large, abrupt visual rotation during reaching for a virtual visual target, visual feedback of the cursor was not provided. Only performance-based scalar reward or punishment feedback (money gained or lost) was used, thereby emphasising explicit processes during adaptation. The results revealed that punishment, compared with reward, induced faster adaptation and greater variability of reaching in the initial phase of adaptation. We interpret these findings as reflecting enhanced explicit learning, likely due to loss aversion.

Luring the Motor System: Impact of Performance-Contingent Incentives on Pre-Movement Beta-Band Activity and Motor Performance

It has been shown that when incentives are provided during movement preparation, activity in parieto-frontal regions reflects both expected value and motivational salience. Yet behavioral work suggests that the processing of rewards is faster than for punishments, raising the possibility that expected value and motivational salience manifest at different latencies during movement planning. Given the role of beta oscillations (13-30 Hz) in movement preparation and in communication within the reward circuit, this study investigated how beta activity is modulated by positive and negative monetary incentives during reach planning, and in particular whether it reflects expected value and motivational salience at different latencies. Electroencephalography was recorded while male and female humans performed a reaching task in which reward or punishment delivery depended on movement accuracy. Before a preparatory delay period, participants were informed of the consequences of hitting or missing the target, according to four experimental conditions: Neutral (hit/miss:+0/-0¢), Reward (hit/miss:+5/-0¢), Punish (hit/miss:+0/-5¢) and Mixed (hit/miss:+5/-5¢). Results revealed that beta power over parieto-frontal regions was strongly modulated by incentives during the delay period, with power positively correlating with movement times. Interestingly, beta power was selectively sensitive to potential rewards early in the delay period, after which it came to reflect motivational salience as movement onset neared. These results demonstrate that beta activity reflects expected value and motivational salience on different time scales during reach planning. They also provide support for models that link beta activity with basal ganglia and dopamine for the allocation of neural resources according to behavioral salience.SIGNIFICANCE STATEMENT The present work demonstrates that pre-movement parieto-frontal beta power is modulated by monetary incentives in a goal-directed reaching task. Specifically, beta power transiently scaled with the availability of rewards early in movement planning, before reflecting motivational salience as movement onset neared. Moreover, pre-movement beta activity correlated with the vigor of the upcoming movement. These findings suggest that beta oscillations reflect neural processes that mediate the invigorating effect of incentives on motor performance, possibly through dopamine-mediated interactions with the basal ganglia.