Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients

Behaviourally inhibited temperament and female sex, two vulnerability factors for anxiety disorders, facilitate conditioned avoidance (also) in humans

Acquisition and maintenance of avoidance behaviour is a key feature of all human anxiety disorders. Animal models have been useful in understanding how anxiety vulnerability could translate into avoidance learning. For example, behaviourally inhibited temperament and female sex, two vulnerability factors for clinical anxiety, are associated with faster acquisition of avoidance responses in rodents. However, to date, the translation of such empirical data to human populations has been limited since many features of animal avoidance paradigms are not typically captured in human research. Here, using a computer-based task that captures many features of rodent escape-avoidance learning paradigms, we investigated whether avoidance learning would be faster in humans with inhibited temperament and/or female sex and, if so, whether this facilitation would take the same form. Results showed that, as in rats, both vulnerability factors were associated with facilitated acquisition of avoidance behaviour in humans. Specifically, inhibited temperament was associated with higher rate of avoidance responding, while female sex was associated with longer avoidance duration. These findings strengthen the direct link between animal avoidance work and human anxiety vulnerability, further motivating the study of animal models while also providing a simple testbed for a direct human testing.

Dopaminergic influences on emotional decision making in euthymic bipolar patients

We recently reported that the D2/D3 agonist pramipexole may have pro-cognitive effects in euthymic patients with bipolar disorder (BPD); however, the emergence of impulse-control disorders has been documented in Parkinson's disease (PD) after pramipexole treatment. Performance on reward-based tasks is altered in healthy subjects after a single dose of pramipexole, but its potential to induce abnormalities in BPD patients is unknown. We assessed reward-dependent decision making in euthymic BPD patients pre- and post 8 weeks of treatment with pramipexole or placebo by using the Iowa Gambling Task (IGT). The IGT requires subjects to choose among four card decks (two risky and two conservative) and is designed to promote learning to make advantageous (conservative) choices over time. Thirty-four BPD patients completed both assessments (18 placebo and 16 pramipexole). Baseline performance did not differ by treatment group (F=0.63; p=0.64); however, at week 8, BPD patients on pramipexole demonstrated a significantly greater tendency to make increasingly high-risk, high-reward choices across the five blocks, whereas the placebo group's pattern was similar to that reported in healthy individuals (treatment × time × block interaction, p<0.05). Analyses of choice strategy using the expectancy valence model revealed that after 8 weeks on pramipexole, BPD patients attended more readily to feedback related to gains than to losses, which could explain the impaired learning. There were no significant changes in mood symptoms over the 8 weeks, and no increased propensity toward manic-like behaviors were reported. Our results suggest that the enhancement of dopaminergic activity influences risk-associated decision-making performance in euthymic BPD. The clinical implications remain unknown.

Dopaminergic stimulation increases selfish behavior in the absence of punishment threat

RATIONALE

People often face decisions that pit self-interested behavior aimed at maximizing personal reward against normative behavior such as acting cooperatively, which benefits others. The threat of social sanctions for defying the fairness norm prevents people from behaving overly selfish. Thus, normative behavior is influenced by both seeking rewards and avoiding punishment. However, the neurochemical processes mediating the impact of these influences remain unknown. Several lines of evidence link the dopaminergic system to reward and punishment processing, respectively, but this evidence stems from studies in non-social contexts.

OBJECTIVES

The present study investigates dopaminergic drug effects on individuals' reward seeking and punishment avoidance in social interaction.

METHODS

Two-hundred one healthy male participants were randomly assigned to receive 300 mg of L-3,4-dihydroxyphenylalanine (L-DOPA) or a placebo before playing an economic bargaining game. This game involved two conditions, one in which unfair behavior could be punished and one in which unfair behavior could not be punished.

RESULTS

In the absence of punishment threats, L-DOPA administration led to more selfish behavior, likely mediated through an increase in reward seeking. In contrast, L-DOPA administration had no significant effect on behavior when faced with punishment threats.

CONCLUSIONS

The results of this study broaden the role of the dopaminergic system in reward seeking to human social interactions. We could show that even a single dose of a dopaminergic drug may bring selfish behavior to the fore, which in turn may shed new light on potential causal relationships between the dopaminergic system and norm abiding behaviors in certain clinical subpopulations.

Acute stress-induced cortisol elevations mediate reward system activity during subconscious processing of sexual stimuli

Stress is thought to alter motivational processes by increasing dopamine (DA) secretion in the brain's "reward system", and its key region, the nucleus accumbens (NAcc). However, stress studies using functional magnetic resonance imaging (fMRI), mainly found evidence for stress-induced decreases in NAcc responsiveness toward reward cues. Results from both animal and human PET studies indicate that the stress hormone cortisol may be crucial in the interaction between stress and dopaminergic actions. In the present study we therefore investigated whether cortisol mediated the effect of stress on DA-related responses to -subliminal-presentation of reward cues using the Trier Social Stress Test (TSST), which is known to reliably enhance cortisol levels. Young healthy males (n = 37) were randomly assigned to the TSST or control condition. After stress induction, brain activation was assessed using fMRI during a backward-masking paradigm in which potentially rewarding (sexual), emotionally negative and neutral stimuli were presented subliminally, masked by pictures of inanimate objects. A region of interest analysis showed that stress decreased activation in the NAcc in response to masked sexual cues (voxel-corrected, p<05). Furthermore, with mediation analysis it was found that high cortisol levels were related to stronger NAcc activation, showing that cortisol acted as a suppressor variable in the negative relation between stress and NAcc activation. The present findings indicate that cortisol is crucially involved in the relation between stress and the responsiveness of the reward system. Although generally stress decreases activation in the NAcc in response to rewarding stimuli, high stress-induced cortisol levels suppress this relation, and are associated with stronger NAcc activation. Individuals with a high cortisol response to stress might on one hand be protected against reductions in reward sensitivity, which has been linked to anhedonia and depression, but they may ultimately be more vulnerable to increased reward sensitivity, and addictions. Future studies investigating effects of stress on reward sensitivity should take into account the severity of the stressor and the individual cortisol response to stress.

Association of hair iron levels with creativity and psychological variables related to creativity

Creativity generally involves the conception of original and valuable ideas. Previous studies have suggested an association between creativity and the dopaminergic system, and that physical activity facilitates creativity. Iron plays a key role in the dopaminergic system and physical activity. Here, we newly investigated the associations between hair iron levels and creativity, dopamine-related traits and states [novelty seeking, extraversion, and vigor (motivational state)], as well as the physical activity level. In the present study, we addressed this issue by performing a hair mineral analysis to determine iron levels and a behavioral creativity test of divergent thinking and related psychological measures among young adults (254 men, 88 women; mean age 20.79 ± 2.03 years). Iron levels did not show any significant association with creativity but displayed significant positive associations with novelty seeking, extraversion, and physical activity level. These results may be partly congruent with the notion that iron plays a key role in the dopaminergic system and imply that iron is important for traits and physical activity, which facilitate creativity. Future interventional or longitudinal studies are warranted to identify any causal effects.

Love to win or hate to Lose? Asymmetry of dopamine D2 receptor binding predicts sensitivity to reward versus punishment

Humans show consistent differences in the extent to which their behavior reflects a bias toward appetitive approach-related behavior or avoidance of aversive stimuli [Elliot, A. J. Approach and avoidance motivation. In A. J. Elliot (Ed.), Handbook of approach and avoidance motivation (pp. 3-14). New York: Psychology Press, 2008]. We examined the hypothesis that in healthy participants this motivational bias (assessed by self-report and by a probabilistic learning task that allows direct comparison of the relative sensitivity to reward and punishment) reflects lateralization of dopamine signaling. Using [F-18]fallypride to measure D2/D3 binding, we found that self-reported motivational bias was predicted by the asymmetry of frontal D2 binding. Similarly, striatal and frontal asymmetries in D2 dopamine receptor binding, rather than absolute binding levels, predicted individual differences in learning from reward versus punishment. These results suggest that normal variation in asymmetry of dopamine signaling may, in part, underlie human personality and cognition.

Impulsive and compulsive behaviors in Parkinson's disease

Impulsive-compulsive behaviors (ICBs) in Parkinson's disease (PD) are a common and devastating side effect of dopamine replacement therapy. In this review we describe the phenomenology, prevalence, and risk factors of patients with PD. Results of behavioral studies assessing the neuropsychological profile of patients with PD emphasize that the ICBs, which are behavioral addictions, are not hedonically motivated. Rather, other factors such as the inability to cope with uncertainty may be triggering ICBs. New insights from functional imaging studies, strengthening the incentive salience hypothesis, are discussed, and therapeutic guidelines for the management of ICBs in PD are given.

Depression and impulse control disorders in Parkinson's disease: two sides of the same coin?

Depression and impulse control disorders (ICD) are two common neuropsychiatric features in Parkinson's disease (PD). Studies have revealed that both phenomena are associated with aberrations in ventral striatal dopamine signaling and concomitant dysfunction of the reward-related (limbic) cortico-striatal-thalamocortical (CSTC) circuit. Depression in PD seems associated with decreased activity in the limbic CSTC circuit, whereas ICD seem associated with increased limbic CSTC circuit activity, usually after commencing dopamine replacement therapy (DRT). Not all DRT using PD patients, however, develop symptoms of ICD, suggesting an additional underlying neurobiological susceptibility. Furthermore, the symptoms of depression and ICD frequently coincide even though they are related to seemingly contrasting limbic CSTC circuit activation states. The aim of this review is to provide an overview of the currently available literature on the neurobiology of PD-related depression and ICD and discusses possible susceptibility factors. Finally, we propose a neurobiological model that identifies ventral striatal dopaminergic denervation as a common underlying neurobiological substrate of depression and ICD and subsequent dysfunction of reward and motivation-related brain areas.

Asymmetric right/left encoding of emotions in the human subthalamic nucleus

Emotional processing is lateralized to the non-dominant brain hemisphere. However, there is no clear spatial model for lateralization of emotional domains in the basal ganglia. The subthalamic nucleus (STN), an input structure in the basal ganglia network, plays a major role in the pathophysiology of Parkinson's disease (PD). This role is probably not limited only to the motor deficits of PD, but may also span the emotional and cognitive deficits commonly observed in PD patients. Beta oscillations (12–30 Hz), the electrophysiological signature of PD, are restricted to the dorsolateral part of the STN that corresponds to the anatomically defined sensorimotor STN. The more medial, more anterior and more ventral parts of the STN are thought to correspond to the anatomically defined limbic and associative territories of the STN. Surprisingly, little is known about the electrophysiological properties of the non-motor domains of the STN, nor about electrophysiological differences between right and left STNs. In this study, microelectrodes were utilized to record the STN spontaneous spiking activity and responses to vocal non-verbal emotional stimuli during deep brain stimulation (DBS) surgeries in human PD patients. The oscillation properties of the STN neurons were used to map the dorsal oscillatory and the ventral non-oscillatory regions of the STN. Emotive auditory stimulation evoked activity in the ventral non-oscillatory region of the right STN. These responses were not observed in the left ventral STN or in the dorsal regions of either the right or left STN. Therefore, our results suggest that the ventral non-oscillatory regions are asymmetrically associated with non-motor functions, with the right ventral STN associated with emotional processing. These results suggest that DBS of the right ventral STN may be associated with beneficial or adverse emotional effects observed in PD patients and may relieve mental symptoms in other neurological and psychiatric diseases.

Neural mechanisms of attention-deficit/hyperactivity disorder symptoms are stratified by MAOA genotype

BACKGROUND

Attention-deficit/hyperactivity disorder (ADHD) is characterized by deficits in reward sensitivity and response inhibition. The relative contribution of these frontostriatal mechanisms to ADHD symptoms and their genetic determinants is largely unexplored.

METHODS

Using functional magnetic resonance imaging and genetic analysis of the monoamine oxidase A (MAOA) gene, we investigated how striatal and inferior frontal activation patterns contribute to ADHD symptoms depending on MAOA genotype in a sample of adolescent boys (n = 190).

RESULTS

We demonstrate an association of ADHD symptoms with distinct blood oxygen level-dependent (BOLD) responses depending on MAOA genotype. In A hemizygotes of the expression single nucleotide polymorphism rs12843268, which express lower levels of MAOA, ADHD symptoms are associated with lower ventral striatal BOLD response during the monetary incentive delay task and lower inferior frontal gyrus BOLD response during the stop signal task. In G hemizygotes, ADHD symptoms are associated with increased inferior frontal gyrus BOLD response during the stop signal task in the presence of increased ventral striatal BOLD response during the monetary incentive delay task.

CONCLUSIONS

Depending on MAOA genotype, ADHD symptoms in adolescent boys are associated with either reward deficiency or insufficient response inhibition. Apart from its mechanistic interest, our finding may aid in developing pharmacogenetic markers for ADHD.

Subthalamic stimulation modulates self-estimation of patients with Parkinson's disease and induces risk-seeking behaviour

Patients with Parkinson's disease with deep brain stimulation in the subthalamic nucleus postoperatively often display higher impulsivity and therefore may experience difficulties in social interactions. Here, we examined social interactions of patients with Parkinson's disease with and without deep brain stimulation in the subthalamic nucleus in competitive situations. We hypothesized altered self-estimation and risk-seeking behaviour in this patient group induced by deep brain stimulation in the subthalamic nucleus. To test the hypothesis, an experimental setting was used in which participants performed a calculation task and chose their preferred compensation. Based on their actual calculation performance, more patients with Parkinson's disease with deep brain stimulation chose a competitive tournament compensation. Assuming rational behaviour, this self-selection pattern reflects increased risk tolerance. Since patients who performed in the lowest quartile chose the tournament option, the data suggest that deep brain stimulation in the subthalamic nucleus results in a loss of the correct reference frame against which patients with Parkinson's disease evaluate their performance. The stimulation-induced combination of overestimation of their own performance, increased risk-taking, and preference for competitive environments despite poor performance is likely to impact considerably on the patients' social and work life.

Learning from negative feedback in patients with major depressive disorder is attenuated by SSRI antidepressants

One barrier to interpreting past studies of cognition and major depressive disorder (MDD) has been the failure in many studies to adequately dissociate the effects of MDD from the potential cognitive side effects of selective serotonin reuptake inhibitors (SSRIs) use. To better understand how remediation of depressive symptoms affects cognitive function in MDD, we evaluated three groups of subjects: medication-naïve patients with MDD, medicated patients with MDD receiving the SSRI paroxetine, and healthy control (HC) subjects. All were administered a category-learning task that allows for dissociation between learning from positive feedback (reward) vs. learning from negative feedback (punishment). Healthy subjects learned significantly better from positive feedback than medication-naïve and medicated MDD groups, whose learning accuracy did not differ significantly. In contrast, medicated patients with MDD learned significantly less from negative feedback than medication-naïve patients with MDD and healthy subjects, whose learning accuracy was comparable. A comparison of subject’s relative sensitivity to positive vs. negative feedback showed that both the medicated MDD and HC groups conform to Kahneman and Tversky’s (1979) Prospect Theory, which expects losses (negative feedback) to loom psychologically slightly larger than gains (positive feedback). However, medicated MDD and HC profiles are not similar, which indicates that the state of medicated MDD is not “normal” when compared to HC, but rather balanced with less learning from both positive and negative feedback. On the other hand, medication-naïve patients with MDD violate Prospect Theory by having significantly exaggerated learning from negative feedback. This suggests that SSRI antidepressants impair learning from negative feedback, while having negligible effect on learning from positive feedback. Overall, these findings shed light on the importance of dissociating the cognitive consequences of MDD from those of SSRI treatment, and from cognitive evaluation of MDD subjects in a medication-naïve state before the administration of antidepressants. Future research is needed to correlate the mood-elevating effects and the cognitive balance between reward- and punishment-based learning related to SSRIs.

Extraversion differentiates between model-based and model-free strategies in a reinforcement learning task

Prominent computational models describe a neural mechanism for learning from reward prediction errors, and it has been suggested that variations in this mechanism are reflected in personality factors such as trait extraversion. However, although trait extraversion has been linked to improved reward learning, it is not yet known whether this relationship is selective for the particular computational strategy associated with error-driven learning, known as model-free reinforcement learning, vs. another strategy, model-based learning, which the brain is also known to employ. In the present study we test this relationship by examining whether humans' scores on an extraversion scale predict individual differences in the balance between model-based and model-free learning strategies in a sequentially structured decision task designed to distinguish between them. In previous studies with this task, participants have shown a combination of both types of learning, but with substantial individual variation in the balance between them. In the current study, extraversion predicted worse behavior across both sorts of learning. However, the hypothesis that extraverts would be selectively better at model-free reinforcement learning held up among a subset of the more engaged participants, and overall, higher task engagement was associated with a more selective pattern by which extraversion predicted better model-free learning. The findings indicate a relationship between a broad personality orientation and detailed computational learning mechanisms. Results like those in the present study suggest an intriguing and rich relationship between core neuro-computational mechanisms and broader life orientations and outcomes.

Working memory capacity predicts effects of methylphenidate on reversal learning

Increased use of stimulant medication, such as methylphenidate, by healthy college students has raised questions about its cognitive-enhancing effects. Methylphenidate acts by increasing extracellular catecholamine levels and is generally accepted to remediate cognitive and reward deficits in patients with attention deficit hyperactivity disorder. However, the cognitive-enhancing effects of such 'smart drugs' in the healthy population are still unclear. Here, we investigated effects of methylphenidate (Ritalin, 20  mg) on reward and punishment learning in healthy students (N=19) in a within-subject, double-blind, placebo-controlled cross-over design. Results revealed that methylphenidate effects varied both as a function of task demands and as a function of baseline working memory capacity. Specifically, methylphenidate improved reward vs punishment learning in high-working memory subjects, whereas it impaired reward vs punishment learning in low-working memory subjects. These results contribute to our understanding of individual differences in the cognitive-enhancing effects of methylphenidate in the healthy population. Moreover, they highlight the importance of taking into account both inter- and intra-individual differences in dopaminergic drug research.

Learning to obtain reward, but not avoid punishment, is affected by presence of PTSD symptoms in male veterans: empirical data and computational model

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD.

Dopaminergic and clinical correlates of pathological gambling in Parkinson's disease: a case report

Dopaminergic medication for motor symptoms in Parkinson's disease (PD) recently has been linked with impulse control disorders, including pathological gambling (PG), which affects up to 8% of patients. PG often is considered a behavioral addiction associated with disinhibition, risky decision-making, and altered striatal dopaminergic neurotransmission. Using [(11)C]raclopride with positron emission tomography, we assessed dopaminergic neurotransmission during Iowa Gambling Task performance. Here we present data from a single patient with PD and concomitant PG. We noted a marked decrease in [(11)C]raclopride binding in the left ventral striatum upon gambling, indicating a gambling-induced dopamine release. The results imply that PG in PD is associated with a high dose of dopaminergic medication, pronounced motor symptomatology, young age at disease onset, high propensity for sensation seeking, and risky decision-making. Overall, the findings are consistent with the hypothesis of medication-related PG in PD and underscore the importance of taking clinical variables, such as age and personality, into account when patients with PD are medicated, to reduce the risk of PG.

Enhanced avoidance learning in behaviorally inhibited young men and women

Behavioral inhibition (BI) is a temperamental tendency to avoid or withdraw from novel social and nonsocial situations, and has been shown to predispose individuals to anxiety disorders. However, adequate means to assess individual differences in avoidance learning in humans are presently limited. Here, we tested whether individuals with high self-reported BI show faster associative learning on a purely cognitive task and whether such inhibited individuals are more prone to avoid aversive outcomes. In Experiment 1, we tested 74 healthy undergraduate students (mean age 19.5 years; 55.4% female) on a computer-based probabilistic classification task, where participants were asked to classify four distinct visual stimuli into two categories. Two stimuli were associated with reward (point gain) and two were associated with punishment (point loss). In Experiment 2, 79 participants from the same population (mean age 19.8 years; 62% female) were tested on a novel modification of the same task, where they also had the option to opt out of responding on each trial, thus avoiding any chance of being punished (or rewarded) on that trial. Results show that inhibited participants demonstrated better associative learning in Experiment 1, while exhibiting a greater tendency to opt out in Experiment 2 (repeated-measures analysis of variance, main effects of BI, both p < 0.05). These results indicate that the facilitated classically conditioned learning previously observed in inhibited individuals can be extended to a cognitive task, and also highlight a specific preference in inhibited individuals for withdrawal ("opting out") as a response strategy, when multiple strategies are available to avoid punishment.

Hemispheric dissociation of reward processing in humans: insights from deep brain stimulation

Rewards have various effects on human behavior and multiple representations in the human brain. Behaviorally, rewards notably enhance response vigor in incentive motivation paradigms and bias subsequent choices in instrumental learning paradigms. Neurally, rewards affect activity in different fronto-striatal regions attached to different motor effectors, for instance in left and right hemispheres for the two hands. Here we address the question of whether manipulating reward-related brain activity has local or general effects, with respect to behavioral paradigms and motor effectors. Neuronal activity was manipulated in a single hemisphere using unilateral deep brain stimulation (DBS) in patients with Parkinson's disease. Results suggest that DBS amplifies the representation of reward magnitude within the targeted hemisphere, so as to affect the behavior of the contralateral hand specifically. These unilateral DBS effects on behavior include both boosting incentive motivation and biasing instrumental choices. Furthermore, using computational modeling we show that DBS effects on incentive motivation can predict DBS effects on instrumental learning (or vice versa). Thus, we demonstrate the feasibility of causally manipulating reward-related neuronal activity in humans, in a manner that is specific to a class of motor effectors but that generalizes to different computational processes. As these findings proved independent from therapeutic effects on parkinsonian motor symptoms, they might provide insight into DBS impact on non-motor disorders, such as apathy or hypomania.

Parkinsonian apathy responds to dopaminergic stimulation of D2/D3 receptors with piribedil

Apathy is one of the most common symptoms encountered in Parkinson's disease, and is defined as a lack of motivation accompanied by reduced goal-directed cognition, behaviour and emotional involvement. In a previous study we have described a delayed withdrawal syndrome after successful motor improvement related to subthalamic stimulation allowing for a major decrease in dopaminergic treatment. This withdrawal syndrome correlated with a diffuse mesolimbic dopaminergic denervation. To confirm our hypothesis of parkinsonian apathy being related to mesolimbic dopaminergic denervation, we performed a randomized controlled study using piribedil, a relatively selective D2/D3 dopamine agonist to treat parkinsonian apathy, using the model of postoperative apathy. A 12-week prospective, placebo-controlled, randomized, double-blinded trial was conducted in 37 patients with Parkinson's disease presenting with apathy (Starkstein Apathy Scale score > 14) following subthalamic nucleus stimulation. Patients received either piribedil up to 300 mg per day (n = 19) or placebo (n = 18) for 12 weeks. The primary end point was the improvement of apathy under treatment, as assessed by the reduction of the Starkstein Apathy Scale score in both treatment groups. Secondary end points included alleviation in depression (Beck Depression Inventory), anxiety (Beck Anxiety Inventory), improvement of quality of life (PDQ39) and anhedonia (Snaith-Hamilton Pleasure Scale). Exploratory endpoints consisted in changes of the Robert Inventory score and Hamilton depression scales. An intention to treat analysis of covariance analysis was performed to compare treatment effects (P < 0.05). The number of premature study dropouts was seven in the placebo and five in the piribedil groups, mostly related to intolerance to hypodopaminergic symptoms. At follow-up evaluation, the apathy score was reduced by 34.6% on piribedil versus 3.2% on placebo (P = 0.015). With piribedil, modifications in the Beck depression and anxiety scores were -19.8% and -22.8%, respectively versus +1.4% and -8.3% with placebo, without reaching significance level. Piribedil led to a trend towards improvement in quality of life (-16.2% versus +6.7% on placebo; P = 0.08) and anhedonia (-49% versus -5.6% on the placebo; P = 0.08). Apathy, assessed by the Robert Inventory score, improved by 46.6% on piribedil and worsened by 2.3% on placebo (P = 0.005). Depression, measured by the Hamilton score, improved in the piribedil group (P = 0.05). No significant side effects were observed. The present study provides a class II evidence of the efficacy of the dopamine agonist piribedil in the treatment of apathy in Parkinson's disease.

Attenuated neural response to gamble outcomes in drug-naive patients with Parkinson's disease

Parkinson's disease results from the degeneration of dopaminergic neurons in the substantia nigra, manifesting as a spectrum of motor, cognitive and affective deficits. Parkinson's disease also affects reward processing, but disease-related deficits in reinforcement learning are thought to emerge at a slower pace than motor symptoms as the degeneration progresses from dorsal to ventral striatum. Dysfunctions in reward processing are difficult to study in Parkinson's disease as most patients have been treated with dopaminergic drugs, which sensitize reward responses in the ventral striatum, commonly resulting in impulse control disorders. To circumvent this treatment confound, we assayed the neural basis of reward processing in a group of newly diagnosed patients with Parkinson's disease that had never been treated with dopaminergic drugs. Thirteen drug-naive patients with Parkinson's disease and 12 healthy age-matched control subjects underwent whole-brain functional magnetic resonance imaging while they performed a simple two-choice gambling task resulting in stochastic and parametrically variable monetary gains and losses. In patients with Parkinson's disease, the neural response to reward outcome (as reflected by the blood oxygen level-dependent signal) was attenuated in a large group of mesolimbic and mesocortical regions, comprising the ventral putamen, ventral tegmental area, thalamus and hippocampus. Although these regions showed a linear response to reward outcome in healthy individuals, this response was either markedly reduced or undetectable in drug-naive patients with Parkinson's disease. The results show that the core regions of the meso-cortico-limbic dopaminergic system, including the ventral tegmental area, ventral striatum, and medial orbitofrontal cortex, are already significantly compromised in the early stages of the disease and that these deficits cannot be attributed to the contaminating effect of dopaminergic treatment.

Decomposing effects of dopaminergic medication in Parkinson's disease on probabilistic action selection--learning or performance?

Dopamine has long been implicated in reward-based learning and the expression of such learned associations on performance. Robust evidence supports its effects on learning and performance, but teasing these apart has proved challenging. Here we have adapted a classic test of value-based learning, the probabilistic selection task, to disentangle effects of dopamine on value-based performance from effects on value-based learning. Valence-specific effects of dopamine on this specific task cannot be accounted for by modulation of learning, and therefore must reflect modulation of performance. We found that dopaminergic medication, consisting of levodopa and/or dopamine agonists taken at own dose, in 18 patients with mild Parkinson's disease (Hoehn and Yahr < 2.5) potentiated reward-based approach in terms of both accuracy and reaction times, while leaving punishment-based avoidance unaffected. These data demonstrate that the effects of dopamine on probabilistic action selection are at least partly mediated by effects on the expression of learned associations rather than on learning itself, and help refine current models of dopamine's role in reward.

Dopamine agonists rather than deep brain stimulation cause reflection impulsivity in Parkinson's disease

BACKGROUND

Dopamine agonist therapy is the main risk factor for impulse control disorders in Parkinson's disease (PD). However, it is unclear whether bilateral deep brain stimulation of the subthalamic nucleus also causes impairment in decision making.

OBJECTIVES

To assess the role of dopamine agonist therapy and deep brain stimulation on reflection impulsivity in non-demented patients with PD.

METHODS

We recruited 61 PD patients, 20 treated with L-dopa in combination with a dopamine agonist, 14 taking L-dopa monotherapy, a further 16 PD patients with bilateral subthalamic nucleus deep brain stimulation treated with L-dopa in combination with a dopamine agonist, and 11 PD patients with bilateral subthalamic nucleus deep brain stimulation taking L-dopa but not a dopamine agonist. Results were compared with 18 healthy controls. Patients who had evidence of impulsive compulsive behaviour were excluded. Reflection impulsivity was assessed with the beads task, which is a validated information sampling task.

RESULTS

All patients treated with a dopamine agonist gathered significantly less information and made more irrational decisions than all other groups regardless of whether they had surgical treatment.

CONCLUSIONS

Our results imply that dopamine agonist therapy but not deep brain stimulation leads to "reflection impulsivity" in PD.

Effect of D-amphetamine on emotion-potentiated startle in healthy humans: implications for psychopathy and antisocial behaviour

RATIONALE

An emerging literature associates increased dopaminergic neurotransmission with altered brain response to aversive stimuli in humans. The direction of the effect of dopamine on aversive motivation, however, remains unclear, with some studies reporting increased and others decreased amygdala activation to aversive stimuli following the administration of dopamine agonists. Potentiation of the startle response by aversive foreground stimuli provides an objective and directional measure of emotional reactivity and is considered useful as an index of the emotional effects of different drugs.

OBJECTIVE

We investigated the effects of two doses of D-amphetamine (5 and 10 mg), compared to placebo, for the first time to our knowledge, using the affect-startle paradigm.

METHOD

The study employed a between-subjects, double-blind design, with three conditions: 0 mg (placebo), and 5 and 10 mg D-amphetamine (initially n = 20/group; final sample: n = 18, placebo; n = 18, 5 mg; n = 16, 10 mg). After drug/placebo administration, startle responses (eyeblinks) to intermittent noise probes were measured during viewing of pleasant, neutral and unpleasant images. Participants' general and specific impulsivity and fear-related personality traits were also assessed.

RESULTS

The three groups were comparable on personality traits. Only the placebo group showed significant startle potentiation by unpleasant, relative to neutral, images; this effect was absent in both 5- and 10-mg D-amphetamine groups (i.e. the same effect of D-amphetamine observed at different doses in different people).

CONCLUSIONS

Our findings demonstrate a reduced aversive emotional response under D-amphetamine and may help to account for the known link between the use of psychostimulant drugs and antisocial behaviour.

Reinforcement learning and Tourette syndrome

In this chapter, we report the first experimental explorations of reinforcement learning in Tourette syndrome, realized by our team in the last few years. This report will be preceded by an introduction aimed to provide the reader with the state of the art of the knowledge concerning the neural bases of reinforcement learning at the moment of these studies and the scientific rationale beyond them. In short, reinforcement learning is learning by trial and error to maximize rewards and minimize punishments. This decision-making and learning process implicates the dopaminergic system projecting to the frontal cortex-basal ganglia circuits. A large body of evidence suggests that the dysfunction of the same neural systems is implicated in the pathophysiology of Tourette syndrome. Our results show that Tourette condition, as well as the most common pharmacological treatments (dopamine antagonists), affects reinforcement learning performance in these patients. Specifically, the results suggest a deficit in negative reinforcement learning, possibly underpinned by a functional hyperdopaminergia, which could explain the persistence of tics, despite their evident inadaptive (negative) value. This idea, together with the implications of these results in Tourette therapy and the future perspectives, is discussed in Section 4 of this chapter.

A role for dopamine-mediated learning in the pathophysiology and treatment of Parkinson's disease

Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade. We hypothesize that an inappropriate corticostriatal potentiation in striatopallidal cells of the indirect pathway underlies aberrant learning. We demonstrate synaptic potentiation in striatopallidal neurons induced by D2 blockade and diminished by application of an adenosine antagonist, consistent with behavioral observations. A neurocomputational model of the basal ganglia recapitulates the behavioral pattern and further links aberrant learning to plasticity in the indirect pathway. Thus, D2-mediated aberrant learning may contribute to motor deficits in PD, suggesting new avenues for the development of therapeutics.

Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning

Drugs that alter dopamine transmission have opposite effects on reward and punishment learning. These opposite effects have been suggested to depend on dopamine in the striatum. Here, we establish for the first time the neurochemical specificity of such drug effects, during reward and punishment learning in humans, by adopting a coadministration design. Participants (N = 22) were scanned on 4 occasions using functional magnetic resonance imaging, following intake of placebo, bromocriptine (dopamine-receptor agonist), sulpiride (dopamine-receptor antagonist), or a combination of both drugs. A reversal-learning task was employed, in which both unexpected rewards and punishments signaled reversals. Drug effects were stratified with baseline working memory to take into account individual variations in drug response. Sulpiride induced parallel span-dependent changes on striatal blood oxygen level-dependent (BOLD) signal during unexpected rewards and punishments. These drug effects were found to be partially dopamine-dependent, as they were blocked by coadministration with bromocriptine. In contrast, sulpiride elicited opposite effects on behavioral measures of reward and punishment learning. Moreover, sulpiride-induced increases in striatal BOLD signal during both outcomes were associated with behavioral improvement in reward versus punishment learning. These results provide a strong support for current theories, suggesting that drug effects on reward and punishment learning are mediated via striatal dopamine.

Dopamine enhances model-based over model-free choice behavior

Decision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice.

[Imaging non motor signs in Parkinson's disease]

Parkinson's disease is mainly considered as a motor disorder defined by a motor triad. However, various non-motor manifestations may be encountered in Parkinson's disease, including hyposmia, pain, fatigue, sleep disorders, cognitive and behavioral disorders. The pathophysiology of these signs is complex, not univocal and remains poorly understood. Functional imaging techniques either by positron emission tomography, single photon emission tomography or functional magnetic resonance imaging provide an invaluable opportunity to better understand the pathophysiology of these signs. In this paper, we present a review of the recent advances provided by functional imaging in this area.

K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration

Phasic activation of the dopamine (DA) midbrain system in response to unexpected reward or novelty is critical for adaptive behavioral strategies. This activation of DA midbrain neurons occurs via a synaptically triggered switch from low-frequency background spiking to transient high-frequency burst firing. We found that, in medial DA neurons of the substantia nigra (SN), activity of ATP-sensitive potassium (K-ATP) channels enabled NMDA-mediated bursting in vitro as well as spontaneous in vivo burst firing in anesthetized mice. Cell-selective silencing of K-ATP channel activity in medial SN DA neurons revealed that their K-ATP channel-gated burst firing was crucial for novelty-dependent exploratory behavior. We also detected a transcriptional upregulation of K-ATP channel and NMDA receptor subunits, as well as high in vivo burst firing, in surviving SN DA neurons from Parkinson's disease patients, suggesting that burst-gating K-ATP channel function in DA neurons affects phenotypes in both disease and health.

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.

Impulse control disorders in Parkinson's disease: background and update on prevention and management

Given that impulse control disorders (ICDs) have been identified among a considerable minority of Parkinson's disease (PD) patients, these conditions have gained increased clinical and research attention in the past decade. Dopamine-replacement therapies, taken to ameliorate PD symptoms, have been associated with ICDs in PD. Unfortunately, there are relatively sparse empirical data regarding how best to address ICDs in PD patients. Conversely, progress has been made in understanding the clinical, neurobiological and cognitive correlates of ICDs in PD. Some of these findings may inform possible courses of action for care providers working with PD patients with ICDs. The literature on ICDs in non-PD populations may also be informative in this regard. The goals of the present article are to outline important clinical characteristics of ICDs in PD, briefly review relevant neurocognitive and neurobiological studies and discuss possible ways to prevent and manage ICDs in PD.

Similar improvement of reward and punishment learning by serotonin reuptake inhibitors in obsessive-compulsive disorder

BACKGROUND

The role of dopamine in reinforcement learning has been extensively studied, but the role of other major neuromodulators, particularly serotonin, remains poorly understood. An influential theory has suggested that dopamine and serotonin represent opponent systems respectively driving reward and punishment learning.

METHODS

To test this theory, we compared two groups of patients with obsessive-compulsive disorder, one unmedicated (n = 12) and one treated with serotonin reuptake inhibitors (SRI; n = 13). To avoid confounding basic reinforcement learning with strategic conscious reasoning, we used a subliminal conditioning task that involves subjects learning to associate masked cues with gambling outcomes to maximize their payoff. The same task was used in a previous study to demonstrate opposite effects of dopaminergic medication on reward and punishment learning.

RESULTS

Unmedicated obsessive-compulsive disorder patients exhibited an instrumental learning deficit that was fully alleviated under SRI treatment. Contrary to dopaminergic medication, SRIs similarly modulated reward and punishment learning.

CONCLUSIONS

Thus, departing from the opponency model, our results support a beneficial role of serotonin in instrumental learning that is independent of outcome valence.

Decision making, impulsivity, and addictions: do Parkinson's disease patients jump to conclusions?

Links between impulsive-compulsive behaviors (ICBs) in treated Parkinson's disease (PD), behavioral addictions, and substance abuse have been postulated, but no direct comparisons have been carried out so far. We directly compared patients with PD with and without ICBs with illicit drug abusers, pathological gamblers, and age-matched healthy controls using the beads task, a test of reflection impulsivity, and a working memory task. We found that all patients with PD made more impulsive and irrational choices than the control group. PD patients who had an ICB showed similar behavior to illicit substance abusers, whereas patients without ICBs more closely resembled pathological gamblers. In contrast, we found no difference in working memory performance within the PD groups. However, PD patients without ICBs remembered distractors significantly less than all other patients during working memory tests. We were able to correctly classify 96% of the PD patients with respect to whether or not they had an ICB by analyzing three trials of the 80/20 loss condition of the beads task with a negative prediction value of 92.3%, and we propose that this task may prove to be a powerful screening tool to detect an ICB in PD. Our results also suggest that intact cortical processing and less distractibility in PD patients without ICBs may protect them from developing behavioral addictions.

A tribute to charlie chaplin: induced positive affect improves reward-based decision-learning in Parkinson's disease

Reward-based decision-learning refers to the process of learning to select those actions that lead to rewards while avoiding actions that lead to punishments. This process, known to rely on dopaminergic activity in striatal brain regions, is compromised in Parkinson’s disease (PD). We hypothesized that such decision-learning deficits are alleviated by induced positive affect, which is thought to incur transient boosts in midbrain and striatal dopaminergic activity. Computational measures of probabilistic reward-based decision-learning were determined for 51 patients diagnosed with PD. Previous work has shown these measures to rely on the nucleus caudatus (outcome evaluation during the early phases of learning) and the putamen (reward prediction during later phases of learning). We observed that induced positive affect facilitated learning, through its effects on reward prediction rather than outcome evaluation. Viewing a few minutes of comedy clips served to remedy dopamine-related problems associated with frontostriatal circuitry and, consequently, learning to predict which actions will yield reward.

A neurocomputational account of cognitive deficits in Parkinson's disease

Parkinson's disease (PD) is caused by the accelerated death of dopamine (DA) producing neurons. Numerous studies documenting cognitive deficits of PD patients have revealed impairments in a variety of tasks related to memory, learning, visuospatial skills, and attention. While there have been several studies documenting cognitive deficits of PD patients, very few computational models have been proposed. In this article, we use the COVIS model of category learning to simulate DA depletion and show that the model suffers from cognitive symptoms similar to those of human participants affected by PD. Specifically, DA depletion in COVIS produced deficits in rule-based categorization, non-linear information-integration categorization, probabilistic classification, rule maintenance, and rule switching. These were observed by simulating results from younger controls, older controls, PD patients, and severe PD patients in five well-known tasks. Differential performance among the different age groups and clinical populations was modeled simply by changing the amount of DA available in the model. This suggests that COVIS may not only be an adequate model of the simulated tasks and phenomena but also more generally of the role of DA in these tasks and phenomena.

Pramipexole-induced increased probabilistic discounting: comparison between a rodent model of Parkinson's disease and controls

The dopamine agonist pramipexole (PPX) can increase impulsiveness, and PPX therapy for neurological diseases (Parkinson's disease (PD) and restless leg syndrome) is associated with impulse control disorders (ICDs) in subpopulations of treated patients. A commonly reported ICD is pathological gambling of which risk taking is a prominent feature. Probability discounting is a measurable aspect of risk taking. We recently developed a probability discounting paradigm wherein intracranial self-stimulation (ICSS) serves as the positive reinforcer. Here we used this paradigm to determine the effects of PPX on discounting. We included assessments of a rodent model of PD, wherein 6-OHDA was injected into the dorsolateral striatum of both hemispheres, which produced persistent PD-like deficits in posture adjustment. Rats were trained to perform ICSS-mediated probability discounting, in which PD-like and control groups exhibited similar profiles. Rats were treated twice daily for 2 weeks with 2 mg/kg (±)PPX (ie, 1 mg/kg of the active form), a dose that improved lesion-induced motor deficits. In both groups, (±)PPX increased discounting; preference for the large reinforcer was enhanced 30-45% at the most uncertain probabilities. Tolerance did not develop with repeated treatments. Increased discounting subsided within 2 weeks of (±)PPX cessation, and re-exposure to (±)PPX reinstated heightened discounting. Such findings emulate the clinical scenario; therefore, ICSS for discounting assessments in rats exhibited high face validity. This model should prove useful in medication development where assessment of the propensity of a putative therapy to induce risk-taking behaviors is of interest.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson’s disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

Sensitivity and bias in decision-making under risk: evaluating the perception of reward, its probability and value

Background

There are few clinical tools that assess decision-making under risk. Tests that characterize sensitivity and bias in decisions between prospects varying in magnitude and probability of gain may provide insights in conditions with anomalous reward-related behaviour.

Objective

We designed a simple test of how subjects integrate information about the magnitude and the probability of reward, which can determine discriminative thresholds and choice bias in decisions under risk.

Design/Methods

Twenty subjects were required to choose between two explicitly described prospects, one with higher probability but lower magnitude of reward than the other, with the difference in expected value between the two prospects varying from 3 to 23%.

Results

Subjects showed a mean threshold sensitivity of 43% difference in expected value. Regarding choice bias, there was a ‘risk premium’ of 38%, indicating a tendency to choose higher probability over higher reward. An analysis using prospect theory showed that this risk premium is the predicted outcome of hypothesized non-linearities in the subjective perception of reward value and probability.

Conclusions

This simple test provides a robust measure of discriminative value thresholds and biases in decisions under risk. Prospect theory can also make predictions about decisions when subjective perception of reward or probability is anomalous, as may occur in populations with dopaminergic or striatal dysfunction, such as Parkinson's disease and schizophrenia.