Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters

Reward actively engages both implicit and explicit components in dual force field adaptation

Motor learning occurs through multiple mechanisms, including unsupervised, supervised (error based), and reinforcement (reward based) learning. Although studies have shown that reward leads to an overall better motor adaptation, the specific processes by which reward influences adaptation are still unclear. Here, we examine how the presence of reward affects dual adaptation to novel dynamics and distinguish its influence on implicit and explicit learning. Participants adapted to two opposing force fields in an adaptation/deadaptation/error-clamp paradigm, where five levels of reward (a score and a digital face) were provided as participants reduced their lateral error. Both reward and control (no reward provided) groups simultaneously adapted to both opposing force fields, exhibiting a similar final level of adaptation, which was primarily implicit. Triple-rate models fit to the adaptation process found higher learning rates in the fast and slow processes and a slightly increased fast retention rate for the reward group. Whereas differences in the slow learning rate were only driven by implicit learning, the large difference in the fast learning rate was mainly explicit. Overall, we confirm previous work showing that reward increases learning rates, extending this to dual-adaptation experiments and demonstrating that reward influences both implicit and explicit adaptation. Specifically, we show that reward acts primarily explicitly on the fast learning rate and implicitly on the slow learning rates.NEW & NOTEWORTHY Here we show that rewarding participants' performance during dual force field adaptation primarily affects the initial rate of learning and the early timescales of adaptation, with little effect on the final adaptation level. However, reward affects both explicit and implicit components of adaptation. Whereas the learning rate of the slow process is increased implicitly, the fast learning and retention rates are increased through both implicit components and the use of explicit strategies.

Differential effects of reward and punishment on reinforcement-based motor learning and generalization

Reward and punishment have long been recognized as potent modulators of human behavior. Although reinforcement learning is a significant motor learning process, the exact mechanisms underlying how the brain learns movements through reward and punishment are not yet fully understood. Beyond the memory of specific examples, investigating the ability to generalize to new situations offers a better understanding of motor learning. This study hypothesizes that reward and punishment engage qualitatively different motivational systems with different neurochemical and neuroanatomical substrates, which would have differential effects on reinforcement-based motor learning and generalization. To test this hypothesis, two groups of participants learn a motor task in one direction and then relearn the same task in a new direction, receiving only performance-based reward or punishment score feedback. Our findings support our hypothesis, showing that reward led to slower learning but promoted generalization. On the other hand, punishment led to faster learning but impaired generalization. These behavioral differences may be due to different tendencies of movement variability in each group. The punishment group tended to explore more actively than the reward group during the initial learning phase, possibly due to loss aversion. In contrast, the reward group tended to explore more actively than the initial learning phase during the generalization test phase, seemingly recalling the strategy that led to the reward. These results suggest that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization, with important implications for practical applications such as sports training and motor rehabilitation.NEW & NOTEWORTHY Although reinforcement learning is a significant motor learning process, the mechanisms underlying how the brain learns movements through reward and punishment are not fully understood. We modified a well-established motor adaptation task and used savings (faster relearning) to measure generalization. We found reward led to slower learning but promoted generalization, whereas punishment led to faster learning but impaired generalization, suggesting that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization.

Motor potentials evoked by transcranial magnetic stimulation: interpreting a simple measure of a complex system

Transcranial magnetic stimulation (TMS) is a non-invasive technique that is increasingly used to study the human brain. One of the principal outcome measures is the motor-evoked potential (MEP) elicited in a muscle following TMS over the primary motor cortex (M1), where it is used to estimate changes in corticospinal excitability. However, multiple elements play a role in MEP generation, so even apparently simple measures such as peak-to-peak amplitude have a complex interpretation. Here, we summarize what is currently known regarding the neural pathways and circuits that contribute to the MEP and discuss the factors that should be considered when interpreting MEP amplitude measured at rest in the context of motor processing and patients with neurological conditions. In the last part of this work, we also discuss how emerging technological approaches can be combined with TMS to improve our understanding of neural substrates that can influence MEPs. Overall, this review aims to highlight the capabilities and limitations of TMS that are important to recognize when attempting to disentangle sources that contribute to the physiological state-related changes in corticomotor excitability.

Dopamine facilitates the translation of physical exertion into assessments of effort

Our assessments of effort are critically shaped by experiences of exertion. However, it is unclear how the nervous system transforms physical exertion into assessments of effort. Availability of the neuromodulator dopamine influences features of motor performance and effort-based decision-making. To test dopamine's role in the translation of effortful exertion into assessments of effort, we had participants with Parkinson's disease, in dopamine depleted (OFF dopaminergic medication) and elevated (ON dopaminergic medication) states, exert levels of physical exertion and retrospectively assess how much effort they exerted. In a dopamine-depleted state, participants exhibited increased exertion variability and over-reported their levels of exertion, compared to the dopamine-supplemented state. Increased exertion variability was associated with less accurate effort assessment and dopamine had a protective influence on this effect, reducing the extent to which exertion variability corrupted assessments of effort. Our findings provide an account of dopamine's role in the translation of features of motor performance into judgments of effort, and a potential therapeutic target for the increased sense of effort observed across a range of neurologic and psychiatric conditions.

Variations induced by the use of unstable surface do not facilitate motor adaptation to a throwing skill

Induced variability by the use of unstable surfaces has been proposed to enhance proprioceptive control to deal with perturbations in the support base better. However, there is a lack of evidence about its benefits facilitating motor adaptions in upper body skills. In this experiment, practice on an unstable surface was applied to analyze the adaptations in an upper limb precision throwing skill. After a pretest, twenty-one participants were randomly allocated into two groups: one group practiced the throwing task on a stable surface and the other group practiced the same task on an unstable support base. Differences in throwing performance between pre- and post-practice were analyzed in accuracy, hand movement kinematics and variability of the throw in both surface conditions. Fuzzy entropy of the horizontal force was calculated to assess the complexity dynamics of postural sway. Participants improved their performance on the stable and the unstable surface. Induced variability using an unstable surface reduced participants' variability and the complexity of postural sway, but it did not facilitate a superior adaptation of the throwing task. The results suggest that the variations induced by unstable surfaces would fall far from the family of specific motor solutions and would not facilitate additional motor performance of the throwing task.

Exploration of sensory-motor tradeoff behavior in Parkinson's disease

While slowness of movement is an obligatory characteristic of Parkinson's disease (PD), there are conditions in which patients move uncharacteristically fast, attributed to deficient motor inhibition. Here we investigate deficient inhibition in an optimal sensory-motor integration framework, using a game in which subjects used a paddle to catch a virtual ball. Display of the ball was extinguished as soon as the catching movement started, segregating the task into a sensing and acting phase. We analyzed the behavior of 9 PD patients (ON medication) and 10 age-matched controls (HC). The switching times (between sensing and acting phase) were compared to the predicted optimal switching time, based on the individual estimates of sensory and motor uncertainties. The comparison showed that deviation from predicted optimal switching times were similar between groups. However, PD patients showed a weaker correlation between variability in switching time and sensory-motor uncertainty, indicating a reduced propensity to generate exploratory behavior for optimizing goal-directed movements. Analysis of the movement kinematics revealed that PD patients, compared to controls, used a lower peak velocity of the paddle and intercepted the ball with greater velocity. Adjusting the trial duration to the time for the paddle to stop moving, we found that PD patients spent a smaller proportion of the trial duration for observing the ball. Altogether, the results do not show the premature movement initiation and truncated sensory processing that we predicted to ensue from deficient inhibition in PD.

Get to grips with motivation: Slipping and gripping movements are biased by approach-avoidance context

People are better at approaching appetitive cues signaling reward and avoiding aversive cues signaling punishment than vice versa. This action bias has previously been shown in approach-avoidance tasks involving arm movements in response to appetitive or aversive cues. It is not known whether appetitive or aversive stimuli also bias more distal dexterous actions, such as gripping and slipping, in a similar manner. To test this hypothesis, we designed a novel task involving grip force control (gripping and slipping) to probe gripping-related approach and avoidance behavior. 32 male volunteers, aged 18–40 years, were instructed to either grip (“approach”) or slip (”avoid”) a grip-force device with their right thumb and index finger at the sight of positive or negative images. In one version of this pincer grip task, participants were responding to graspable objects and in another version of the task they were responding to happy or angry faces. Bayesian repeated measures Analysis of variance revealed extreme evidence for an interaction between response type and cue valence (Bayes factor = 296). Participants were faster to respond in affect-congruent conditions (“approach appetitive,” “avoid aversive”) than in affect-incongruent conditions (“approach aversive,” “avoid appetitive”). This bias toward faster response times for affect-congruent conditions was present regardless of whether it was a graspable object or a face signaling valence. Since our results mirror the approach and avoidance effects previously observed for arm movements, we conclude that a tendency favoring affectively congruent cue-response mappings is an inherent feature of motor control and thus also includes precision grip.

Trial-to-trial Variability and Cortical Processing Depend on Recent Outcomes During Human Reinforcement Motor Learning

The history of our actions and their outcomes represent important information, informing choices and efficiently guiding future behavior. While unsuccessful (S-) outcomes are expected to lead to more explorative motor states and increased behavioral variability, successful (S+) outcomes are expected to reinforce the use of the previous action. Here, we show that humans attribute different values to previous actions during reinforcement motor learning when they experience S- compared to S+ outcomes. Behavioral variability after an S- outcome is influenced more by the previous outcome than after S+ outcomes. Using electroencephalography, we show that theta band oscillations of the prefrontal cortex are most prominent during changes in two consecutive outcomes, potentially reflecting the need for enhanced cognitive control. Our results suggest that S+ experiences 'overwrite' previous motor states to a greater extent than S- experiences and that modulations in neural oscillations in the prefrontal cortex play a potential role in encoding changes in movement variability state during reinforcement motor learning.

Motor Synergies Measurement Reveals the Relevant Role of Variability in Reward-Based Learning

Currently, it is not fully understood how motor variability is regulated to ease of motor learning processes during reward-based tasks. This study aimed to assess the potential relationship between different dimensions of motor variability (i.e., the motor variability structure and the motor synergies variability) and the learning rate in a reward-based task developed using a two-axis force sensor in a computer environment. Forty-four participants performed a pretest, a training period, a posttest, and three retests. They had to release a virtual ball to hit a target using a vertical handle attached to a dynamometer in a computer-simulated reward-based task. The participants' throwing performance, learning ratio, force applied, variability structure (detrended fluctuation analysis, DFA), and motor synergy variability (good and bad variability ratio, GV/BV) were calculated. Participants with higher initial GV/BV displayed greater performance improvements than those with lower GV/BV. DFA did not show any relationship with the learning ratio. These results suggest that exploring a broader range of successful motor synergy combinations to achieve the task goal can facilitate further learning during reward-based tasks. The evolution of the motor variability synergies as an index of the individuals' learning stages seems to be supported by our study.

Reward boosts reinforcement-based motor learning

Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we investigated the net effect of motivation by reward on motor learning while controlling for the sensory and reinforcement feedback received by the participants. Reward improved motor skill learning beyond performance-based reinforcement feedback. Importantly, the beneficial effect of reward involved a specific potentiation of reinforcement-related adjustments in motor commands, which concerned primarily the most relevant motor component for task success and persisted on the following day in the absence of reward. We propose that the long-lasting effects of motivation on motor learning may entail a form of associative learning resulting from the repetitive pairing of the reinforcement feedback and reward during training, a mechanism that may be exploited in future rehabilitation protocols.

Striatal dopamine synthesis capacity reflects smartphone social activity

Striatal dopamine and smartphone behavior have both been linked with behavioral variability. Here, we leverage day-to-day logs of natural, unconstrained smartphone behavior and establish a correlation between a measure of smartphone social activity previously linked with behavioral variability and a measure of striatal dopamine synthesis capacity using [¹⁸F]-DOPA PET in (N = 22) healthy adult humans. Specifically, we find that a higher proportion of social app interactions correlates with lower dopamine synthesis capacity in the bilateral putamen. Permutation tests and penalized regressions provide evidence that this link between dopamine synthesis capacity and social versus non-social smartphone interactions is specific. These observations provide a key empirical grounding for current speculations about dopamine's role in digital social behavior.

•

Putamen dopamine synthesis capacity correlates with smartphone social app use.

•

The correlation parallels a prior link between social app use and motor variability.

•

It is selective to social app use, controlling for multiple smartphone use factors.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

Interactions between motor exploration and reinforcement learning

Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning.NEW & NOTEWORTHY Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.

The role of stimulus-driven versus goal-directed processes in fight and flight tendencies measured with motor evoked potentials induced by Transcranial Magnetic Stimulation

This study examines two contrasting explanations for early tendencies to fight and flee. According to a stimulus-driven explanation, goal-incompatible stimuli that are easy/difficult to control lead to the tendency to fight/flee. According to a goal-directed explanation, on the other hand, the tendency to fight/flee occurs when the expected utility of fighting/fleeing is the highest. Participants did a computer task in which they were confronted with goal-incompatible stimuli that were (a) easy to control and fighting had the highest expected utility, (b) easy to control and fleeing had the highest expected utility, and (c) difficult to control and fleeing and fighting had zero expected utility. After participants were trained to use one hand to fight and another hand to flee, they either had to choose a response or merely observe the stimuli. During the observation trials, single-pulse Transcranial Magnetic Stimulation (TMS) was applied to the primary motor cortex 450 ms post-stimulus onset and motor-evoked potentials (MEPs) were measured from the hand muscles. Results showed that participants chose to fight/flee when the expected utility of fighting/fleeing was the highest, and that they responded late when the expected utility of both responses was low. They also showed larger MEPs for the right/left hand when the expected utility of fighting/fleeing was the highest. This result can be interpreted as support for the goal-directed account, but only if it is assumed that we were unable to override the presumed natural mapping between hand (right/left) and response (fight/flight).

Neural signatures of reward and sensory error feedback processing in motor learning

At least two distinct processes have been identified by which motor commands are adapted according to movement-related feedback: reward-based learning and sensory error-based learning. In sensory error-based learning, mappings between sensory targets and motor commands are recalibrated according to sensory error feedback. In reward-based learning, motor commands are associated with subjective value, such that successful actions are reinforced. We designed two tasks to isolate reward- and sensory error-based motor adaptation, and we used electroencephalography in humans to identify and dissociate the neural correlates of reward and sensory error feedback processing. We designed a visuomotor rotation task to isolate sensory error-based learning that was induced by altered visual feedback of hand position. In a reward learning task, we isolated reward-based learning induced by binary reward feedback that was decoupled from the visual target. A fronto-central event-related potential called the feedback-related negativity (FRN) was elicited specifically by binary reward feedback but not sensory error feedback. A more posterior component called the P300 was evoked by feedback in both tasks. In the visuomotor rotation task, P300 amplitude was increased by sensory error induced by perturbed visual feedback and was correlated with learning rate. In the reward learning task, P300 amplitude was increased by reward relative to nonreward and by surprise regardless of feedback valence. We propose that during motor adaptation the FRN specifically reflects a reward-based learning signal whereas the P300 reflects feedback processing that is related to adaptation more generally. NEW & NOTEWORTHY We studied the event-related potentials evoked by feedback stimuli during motor adaptation tasks that isolate reward- and sensory error-based learning mechanisms. We found that the feedback-related negativity was specifically elicited by binary reward feedback, whereas the P300 was observed in both tasks. These results reveal neural processes associated with different learning mechanisms and elucidate which classes of errors, from a computational standpoint, elicit the feedback-related negativity and P300.

Attractor-like Dynamics in Belief Updating in Schizophrenia

Subjects with a diagnosis of schizophrenia (Scz) overweight unexpected evidence in probabilistic inference: such evidence becomes "aberrantly salient." A neurobiological explanation for this effect is that diminished synaptic gain (e.g., hypofunction of cortical NMDARs) in Scz destabilizes quasi-stable neuronal network states (or "attractors"). This attractor instability account predicts that (1) Scz would overweight unexpected evidence but underweight consistent evidence, (2) belief updating would be more vulnerable to stochastic fluctuations in neural activity, and (3) these effects would correlate. Hierarchical Bayesian belief updating models were tested in two independent datasets (n = 80 male and n = 167 female) comprising human subjects with Scz, and both clinical and nonclinical controls (some tested when unwell and on recovery) performing the "probability estimates" version of the beads task (a probabilistic inference task). Models with a standard learning rate, or including a parameter increasing updating to "disconfirmatory evidence," or a parameter encoding belief instability were formally compared. The "belief instability" model (based on the principles of attractor dynamics) had most evidence in all groups in both datasets. Two of four parameters differed between Scz and nonclinical controls in each dataset: belief instability and response stochasticity. These parameters correlated in both datasets. Furthermore, the clinical controls showed similar parameter distributions to Scz when unwell, but were no different from controls once recovered. These findings are consistent with the hypothesis that attractor network instability contributes to belief updating abnormalities in Scz, and suggest that similar changes may exist during acute illness in other psychiatric conditions.SIGNIFICANCE STATEMENT Subjects with a diagnosis of schizophrenia (Scz) make large adjustments to their beliefs following unexpected evidence, but also smaller adjustments than controls following consistent evidence. This has previously been construed as a bias toward "disconfirmatory" information, but a more mechanistic explanation may be that in Scz, neural firing patterns ("attractor states") are less stable and hence easily altered in response to both new evidence and stochastic neural firing. We model belief updating in Scz and controls in two independent datasets using a hierarchical Bayesian model, and show that all subjects are best fit by a model containing a belief instability parameter. Both this and a response stochasticity parameter are consistently altered in Scz, as the unstable attractor hypothesis predicts.

Nonequivalent modulation of corticospinal excitability by positive and negative outcomes

OBJECTIVE

The difference between positive and negative outcomes is important in trial-and-error decision-making processes and affects corticospinal excitability. This study investigated corticospinal excitability during the performance of trial-and-error decision-making tasks with varying competing behavioral outcomes.

METHODS

Each trial began with one of five colored circles presented as a cue. Each color represented a different reward probability, ranging from 10% to 90%. The subjects were instructed to decide whether to perform wrist flexion in response to the cue. Two seconds after the presentation of the cue, a reward stimulus (picture of a coin) or penalty stimulus (mauve circle) was randomly presented to the subject. If the picture of a coin appeared, the subjects received the coin after the experiment if they had performed wrist flexion, but not if they had not performed wrist flexion. If a mauve circle appeared, a coin was deducted from the total reward if the subjects had performed wrist flexion, but not if they had not performed wrist flexion. One second after the reward or penalty stimulus, transcranial magnetic stimulation was delivered to the primary motor cortex at the midpoint between the centers of gravity of the flexor carpi radialis (agonist) and extensor carpi radialis (antagonist) muscles.

RESULTS

Cumulative wrist flexions were positively correlated with reward probabilities. Motor evoked potential (MEP) amplitudes in agonist muscles were significantly higher when wrist flexion incurred a penalty than when it incurred a reward, but there was no difference in the MEP amplitudes of antagonist muscles.

CONCLUSION

Positive and negative behavioral outcomes differentially altered behavior and corticospinal excitability, and unexpected penalties had a stronger effect on corticospinal excitability for agonist muscles.

The impact of reward and punishment on skill learning depends on task demands

Reward and punishment motivate behavior, but it is unclear exactly how they impact skill performance and whether the effect varies across skills. The present study investigated the effect of reward and punishment in both a sequencing skill and a motor skill context. Participants trained on either a sequencing skill (serial reaction time task) or a motor skill (force-tracking task). Skill knowledge was tested immediately after training, and again 1 hour, 24–48 hours, and 30 days after training. We found a dissociation of the effects of reward and punishment on the tasks, primarily reflecting the impact of punishment. While punishment improved serial reaction time task performance, it impaired force-tracking task performance. In contrast to prior literature, neither reward nor punishment benefitted memory retention, arguing against the common assumption that reward ubiquitously benefits skill retention. Collectively, these results suggest that punishment impacts skilled behavior more than reward in a complex, task dependent fashion.

How feedback, motor imagery, and reward influence brain self-regulation using real-time fMRI

The learning process involved in achieving brain self-regulation is presumed to be related to several factors, such as type of feedback, reward, mental imagery, duration of training, among others. Explicitly instructing participants to use mental imagery and monetary reward are common practices in real-time fMRI (rtfMRI) neurofeedback (NF), under the assumption that they will enhance and accelerate the learning process. However, it is still not clear what the optimal strategy is for improving volitional control. We investigated the differential effect of feedback, explicit instructions and monetary reward while training healthy individuals to up-regulate the blood-oxygen-level dependent (BOLD) signal in the supplementary motor area (SMA). Four groups were trained in a two-day rtfMRI-NF protocol: GF with NF only, GF,I with NF + explicit instructions (motor imagery), GF,R with NF + monetary reward, and GF,I,R with NF + explicit instructions (motor imagery) + monetary reward. Our results showed that GF increased significantly their BOLD self-regulation from day-1 to day-2 and GF,R showed the highest BOLD signal amplitude in SMA during the training. The two groups who were instructed to use motor imagery did not show a significant learning effect over the 2 days. The additional factors, namely motor imagery and reward, tended to increase the intersubject variability in the SMA during the course of training. Whole brain univariate and functional connectivity analyses showed common as well as distinct patterns in the four groups, representing the varied influences of feedback, reward, and instructions on the brain. Hum Brain Mapp 37:3153-3171, 2016. © 2016 Wiley Periodicals, Inc.

Lack of reliability in the disruption of cognitive performance following exposure to protons

A series of three replications were run to determine the reliability with which exposure to protons produces a disruption of cognitive performance, using a novel object recognition task and operant responding on an ascending fixed-ratio task. For the first two replications, rats were exposed to head-only exposures to 1000 MeV/n protons at the NASA Space Radiation Laboratory. For the third replication, subjects were given head-only or whole-body exposures to both 1000 and 150 MeV/n protons. The results were characterized by a lack of consistency in the effects of exposure to protons on the performance of these cognitive tasks, both within and between replications. The factors that might influence the lack of consistency and the implications for exploratory class missions are discussed.

Reward-dependent modulation of movement variability

Movement variability is often considered an unwanted byproduct of a noisy nervous system. However, variability can signal a form of implicit exploration, indicating that the nervous system is intentionally varying the motor commands in search of actions that yield the greatest success. Here, we investigated the role of the human basal ganglia in controlling reward-dependent motor variability as measured by trial-to-trial changes in performance during a reaching task. We designed an experiment in which the only performance feedback was success or failure and quantified how reach variability was modulated as a function of the probability of reward. In healthy controls, reach variability increased as the probability of reward decreased. Control of variability depended on the history of past rewards, with the largest trial-to-trial changes occurring immediately after an unrewarded trial. In contrast, in participants with Parkinson's disease, a known example of basal ganglia dysfunction, reward was a poor modulator of variability; that is, the patients showed an impaired ability to increase variability in response to decreases in the probability of reward. This was despite the fact that, after rewarded trials, reach variability in the patients was comparable to healthy controls. In summary, we found that movement variability is partially a form of exploration driven by the recent history of rewards. When the function of the human basal ganglia is compromised, the reward-dependent control of movement variability is impaired, particularly affecting the ability to increase variability after unsuccessful outcomes.

Modulation of motor performance by a monetary incentive: A pilot study

It is commonly acknowledged that movement performance is determined by a trade-off between accuracy requirements and energetic expenditure. However, their relative weights are subjective and depend on the perceived benefit (or cost) associated to successful movement completion. A deeper knowledge on how this trade-off affects motor behavior may suggest ways to manipulate it in pathologies, like Parkinson's disease, in which the mechanisms underlying the selection of motor response are believed to be defective. In this preliminary study, we associate a monetary incentive to successful completion of a full-body reaching task and look at the determinants of motor performance. Our preliminary results suggest that motor performance (measured as the absolute average acceleration of hand movements) increases with movement amplitude/target elevation. Overall, performance also increases with the amount of monetary incentive and with the average reward experienced in previous trials. In addition, subjects with a greater sensitivity to incentive exhibit a low sensitivity to the average reward. In contrast, subjects with a negative sensitivity to incentive exhibit a smaller sensitivity to the average reward. These results suggest that motor performance has a complex relation with its perceived benefits, and this relation is probably subject-dependent.

The Role of Dopamine in Motor Flexibility

Humans carry out many daily tasks in a seemingly automatic fashion. However, when unexpected changes in the environment occur, we have the capacity to inhibit prepotent behavior and replace it with an alternative one. Such behavioral flexibility is a hallmark of executive functions. The neurotransmitter dopamine is known to be crucial for fast, efficient, and accurate cognitive flexibility. Despite the perceived similarities between cognitive and motor flexibility, less is known regarding the role of dopamine within the motor domain. Therefore, the aim of this study was to determine the role of dopamine in motor flexibility. In a double-blind, five-session, within-subject pharmacological experiment, human participants performed an RT task within a probabilistic context that was either predictable or unpredictable. The probabilistic nature of the predictable context resulted in prediction errors. This required participants to replace the prepotent or prepared action with an unprepared action (motor flexibility). The task was overlearned, and changes in context were explicitly instructed, thus controlling for contributions from other dopamine-related processes such as probabilistic or reversal learning and interactions with other types of uncertainty. We found that dopamine receptor blockade by high-dose haloperidol (D1/D2 dopamine receptors) impaired participants' ability to react to unexpected events occurring in a predictable context, which elicit large prediction errors and necessitate motor flexibility. This effect was not observed with selective D2 receptor blockade (sulpiride), with a general increase in tonic dopamine levels (levodopa), or during an unpredictable context, which evoked minimal prediction error. We propose that dopamine is vital in responding to low-level prediction errors about stimulus outcome that requires motor flexibility.