Differential influence of levodopa on reward-based learning in Parkinson's disease

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's Disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

Impulse control disorder in Parkinson's disease is associated with abnormal frontal value signalling

Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson's disease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the mechanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson's disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disorders and depression. One hundred and ninety-nine patients with Parkinson's disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain signalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine's effects on reinforcement learning in Parkinson's disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of value in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article.

Role of serotonin in modulation of decision-making in Parkinson's disease

BACKGROUND

Dysfunction of dopaminergic pathways has been considered to play a pivotal role in Parkinson's disease (PD), affecting the processing of emotional and rewarding information, and potentially leading to symptoms of depression or apathy. However, some aspects of motivation in PD might be affected by non-dopaminergic mechanisms.

AIM AND METHOD

The objective of this experimental medicine study was to investigate the contribution of serotonergic modulation via administration of citalopram (20 mg) for 7 days on motivated decision-making in twenty PD patients, measured using several different computerised tasks and clinical questionnaires that probe different aspects of decision-making. Twenty healthy controls were additionally tested without medication to assess any baseline differences between the two groups.

RESULTS

Results indicated that PD patients were overall less motivated than controls on an effort- and reward-based decision-making task. Citalopram increased or decreased willingness to exert effort for reward, depending on whether baseline motivation was high or low, respectively. A task assessing decision-making under risk revealed higher levels of risk aversion for potential losses in PD patients, which neither serotonin nor the patient's regular dopaminergic medication seemed to restore. However, citalopram in PD was associated with more risk-seeking choices for gains, although patients and controls did not differ on this at baseline.

CONCLUSION

The results provide evidence for a role of the serotonergic system in influencing some aspects of motivated decision-making in PD processes.

No effect of subthalamic deep brain stimulation on metacognition in Parkinson's disease

Deep brain stimulation of the subthalamic nucleus (STN-DBS) is a powerful treatment in Parkinson's disease (PD), which provides a positive effect on motor symptoms although the way it operates on high cognitive processes such as metacognition remains unclear. To address this issue, we recorded electroencephalogram (EEG) of PD patients treated with STN-DBS that performed a reversal learning (RL) paradigm endowed with metacognitive self-assessment. We considered two stimulation conditions, namely DBS-ON (stimulation on) and DBS-OFF (stimulation off), and focused our EEG-analysis on the frontal brain region due to its involvement on high cognitive processes. We found a trend towards a significant difference in RL ability between stimulation conditions. STN-DBS showed no effect on metacognition, although a significant association between accuracy and decision confidence level held for DBS OFF, but not in the case of DBS ON. In summary, our study revealed no significant effect of STN-DBS on RL or metacognition.

Dopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients

Engaging in demanding mental activities requires the allocation of cognitive control, which can be effortful and aversive. Individuals thus tend to avoid exerting cognitive effort if less demanding behavioral options are available. Recent accounts propose a key role for dopamine in motivating behavior by increasing the sensitivity to rewards associated with effort exertion. Whether dopamine additionally plays a specific role in modulating the sensitivity to the costs of cognitive effort, even in the absence of any incentives, is much less clear. To address this question, we assessed cognitive effort avoidance in patients (n = 38) with Parkinson's disease, a condition characterized by loss of midbrain dopaminergic neurons, both ON and OFF dopaminergic medication and compared them to healthy controls (n = 24). Effort avoidance was assessed using the Demand Selection Task (DST), in which participants could freely choose between performing a high-demand or a low-demand version of a task-switching paradigm. Critically, participants were not offered any incentives to choose the more effortful option, nor for good performance. While healthy controls and patients OFF their dopaminergic medications consistently preferred the low-demand option, effort avoidance in patients ON dopaminergic medications was reduced compared to patients OFF, a difference which seems to lessen over trials. These differences in preference could not be explained by altered task-switching performance. Although patients ON were less accurate at detecting the different effort levels, as measured during instructed forced-choice blocks, their detection ability was not associated with effort avoidance, unlike in the healthy controls and the patients OFF. Our findings provide evidence that dopamine replacement in Parkinson's patients increases the willingness to engage in cognitively demanding behavior, and that this cannot be explained by possible effects of dopamine replacement on performance nor on the ability to detect effort demands. These results suggest that dopamine plays a role in reducing the sensitivity to effort costs that is independent of its role in enhancing the sensitivity to the benefits of effort exertion.

Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) is commonly treated with dopaminergic medication, which enhances some, while impairing other cognitive functions. It can even contribute to impulse control disorder and addiction. We describe the history of research supporting the dopamine overdose hypothesis, which accounts for the large within-patient variability in dopaminergic medication effects across different tasks by referring to the spatially non-uniform pattern of dopamine depletion in dorsal versus ventral striatum. However, there is tremendous variability in dopaminergic medication effects not just within patients across distinct tasks, but also across different patients. In the second part of this chapter we review recent studies addressing the large individual variability in the negative side effects of dopaminergic medication on functions that implicate dopamine, such as value-based learning and choice. These studies begin to unravel the mechanisms of dopamine overdosing, thus revising the strict version of the overdose hypothesis. For example, the work shows that the canonical boosting of reward-versus punishment-based choice by medication is greater in patients with depression and a non-tremor phenotype, which both implicate, among other pathology, more rather than less severe dysregulation of the mesolimbic dopamine system. Future longitudinal cohort studies are needed to identify how to optimally combine different clinical, personality, cognitive, neural, genetic and molecular predictors of detrimental medication effects in order to account for as much of the relevant variability as possible. This will provide a useful tool for precision neurology, allowing individual and contextual tailoring of (the dose of) dopaminergic medication in order to maximize its cognitive benefits, yet minimize its side effects.

Increased emissions of 50-kHz ultrasonic vocalizations in hemiparkinsonian rats repeatedly treated with dopaminomimetic drugs: A potential preclinical model for studying the affective properties of dopamine replacement therapy in Parkinson's disease

Dopamine replacement therapy used in Parkinson's disease (PD) may induce alterations in the emotional state that can underlie the manifestation of iatrogenic psychiatric-like disturbances. The preclinical investigation of these disturbances is limited, also because few reliable paradigms are available to study the affective properties of dopaminomimetic drugs in parkinsonian animals. To provide a relevant experimental tool in this respect, we evaluated whether dopaminomimetic drugs modified the emission of 50-kHz ultrasonic vocalizations (USVs), a behavioral marker of positive affect, in rats bearing a unilateral lesion with 6-hydroxydopamine in the medial forebrain bundle. Apomorphine (2 or 4 mg/kg, i.p.), L-3,4-dihydroxyphenilalanine (L-DOPA, 6 or 12 mg/kg, i.p.), or pramipexole (2 or 4 mg/kg, i.p.) were administered in a test cage (× 5 administrations) on alternate days. Seven days after treatment discontinuation, rats were re-exposed to the test cage to measure conditioned calling behavior and thereafter received a drug challenge. Hemiparkinsonian rats treated with either apomorphine or L-DOPA, but not pramipexole, markedly vocalized during repeated treatment and after challenge, and showed conditioned calling behavior. Moreover, apomorphine, L-DOPA and pramipexole elicited different patterns of 50-kHz USV emissions and rotational behavior, indicating that calling behavior in hemiparkinsonian rats treated with dopaminomimetic drugs is not a byproduct of motor activation. Taken together, these results suggest that measuring 50-kHz USV emissions may be a relevant experimental tool for studying how dopaminomimetic drugs modify the affective state in parkinsonian rats, with possible implications for the preclinical investigation of iatrogenic psychiatric-like disturbances in PD.

Null effects of levodopa on reward- and error-based motor adaptation, savings, and anterograde interference

Dopamine signaling is thought to mediate reward-based learning. We tested for a role of dopamine in motor adaptation by administering the dopamine precursor levodopa to healthy participants in two experiments involving reaching movements. Levodopa has been shown to impair reward-based learning in cognitive tasks. Thus, we hypothesized that levodopa would selectively impair aspects of motor adaptation that depend on the reinforcement of rewarding actions. In the first experiment, participants performed two separate tasks in which adaptation was driven either by visual error-based feedback of the hand position or binary reward feedback. We used EEG to measure event-related potentials evoked by task feedback. We hypothesized that levodopa would specifically diminish adaptation and the neural responses to feedback in the reward learning task. However, levodopa did not affect motor adaptation in either task nor did it diminish event-related potentials elicited by reward outcomes. In the second experiment, participants learned to compensate for mechanical force field perturbations applied to the hand during reaching. Previous exposure to a particular force field can result in savings during subsequent adaptation to the same force field or interference during adaptation to an opposite force field. We hypothesized that levodopa would diminish savings and anterograde interference, as previous work suggests that these phenomena result from a reinforcement learning process. However, we found no reliable effects of levodopa. These results suggest that reward-based motor adaptation, savings, and interference may not depend on the same dopaminergic mechanisms that have been shown to be disrupted by levodopa during various cognitive tasks.NEW & NOTEWORTHY Motor adaptation relies on multiple processes including reinforcement of successful actions. Cognitive reinforcement learning is impaired by levodopa-induced disruption of dopamine function. We administered levodopa to healthy adults who participated in multiple motor adaptation tasks. We found no effects of levodopa on any component of motor adaptation. This suggests that motor adaptation may not depend on the same dopaminergic mechanisms as cognitive forms or reinforcement learning that have been shown to be impaired by levodopa.

Maladaptive avoidance patterns in Parkinson's disease are exacerbated by symptoms of depression

Parkinson's disease (PD) is a chronic, progressive neurodegenerative disorder, characterized by a loss of dopaminergic neurons in the substantia nigra pars compacta. Given that dopamine is critically involved in learning and other cognitive processes, such as working memory, dopamine loss in PD has been linked both to learning abnormalities and to cognitive dysfunction more generally in the disease. It is unclear, however, whether avoidance behavior is impacted in PD. This is significant, as this type of instrumental behavior plays an important role in both decision-making and emotional (dys) function. Consequently, the aim of the present study was to examine avoidance learning and operant extinction in PD using a computer-based task. On this task, participants control a spaceship and attempt to shoot an enemy spaceship to gain points. They also learn to hide in safe areas to protect from (i.e., avoid) aversive events (on-screen explosions and point loss). The results showed that patients with PD (N = 25) acquired an avoidance response during aversive periods to the same extent as healthy age-matched controls (N = 19); however, patients demonstrated greater hiding during safe periods not associated with aversive events, which could represent maladaptive generalization of the avoidance response. Furthermore, this impairment was more pronounced during the extinction phase, and in patients who reported higher levels of depression. These results demonstrate for the first time that PD is associated with maladaptive avoidance patterns, which could possibly contribute to the emergence of depression in the disease.

The effect of aging, Parkinson's disease, and exogenous dopamine on the neural response associated with auditory regularity processing

Processing regular patterns in auditory scenes is important for navigating complex environments. Electroencephalography studies find enhancement of sustained brain activity, correlating with the emergence of a regular pattern in sounds. How aging, aging-related diseases such as Parkinson's disease (PD), and treatment of PD with dopaminergic therapy affect this fundamental function remain unknown. We addressed this knowledge gap. Healthy younger and older adults and patients with PD listened to sounds that contained or were devoid of regular patterns. Healthy older adults and patients with PD were tested twice-off and on dopaminergic medication, in counterbalanced order. Regularity-evoked, sustained electroencephalography activity was reduced in older, compared with younger adults. Patients with PD and older controls evidenced comparable attenuation of the sustained response. Dopaminergic therapy further weakened the sustained response in both older controls and patients with PD. These findings suggest that fundamental regularity processing is impacted by aging but not specifically by PD. The finding that dopaminergic therapy attenuates rather than improves the sustained response coheres with the dopamine overdose response and is in line with previous findings that regularity processing implicates brain regions receiving dopamine from the ventral tegmental area that is relatively spared in PD and normal aging.

Effects of subthalamic nucleus stimulation and levodopa on decision-making in Parkinson's disease

BACKGROUND

Parkinson's disease (PD) is frequently associated with behavioral disorders, particularly within the spectrum of motivated behaviors such as apathy or impulsivity. Both pharmacological and neurosurgical treatments have an impact on these impairments. However, there still is controversy as to whether subthalamic nucleus deep brain stimulation (STN-DBS) can cause or reduce impulsive behaviors.

OBJECTIVES

We aimed to identify the influence of functional surgery on decision-making processes in PD.

METHODS

We studied 13 PD patients and 13 healthy controls. The experimental task involved squeezing a dynamometer with variable force to obtain rewards of various values under four conditions: without treatment, with l-dopa or subthalamic stimulation alone, and with both l-dopa and subthalamic stimulation. Statistical analyses consisted of generalized linear mixed models including treatment condition, reward value, level of effort, and their interactions. We analyzed acceptance rate (the percentage of accepted trials), decision time, and force applied.

RESULTS

Comparatively to controls, patients without treatment exhibited lower acceptance rate and force applied. Patients under l-dopa alone did not exhibit increased acceptance rate. With subthalamic stimulation, either with or without added l-dopa, all measures were improved so that patients' behaviors were undistinguishable from healthy controls'.

CONCLUSIONS

Our study shows that l-dopa administration does not fully restore cost-benefit decision-making processes, whereas STN-DBS fully normalizes patients' behaviors. These findings suggest that dopamine is partly involved in cost-benefit valuation, and that STN-DBS can have a beneficial effect on motivated behaviors in PD and may improve certain forms of impulsive behaviors. © 2019 International Parkinson and Movement Disorder Society.

Assessment of Metacognition and Reversal Learning in Parkinson's Disease: Preliminary Results

Reversal learning (RL) has been widely used for assessment of behavioral adaptation, impulsivity, obsession, and compulsion in healthy controls as well as people suffering from psychiatric and neurological disorders such as Parkinson’s disease (PD). Nevertheless, studies addressing high cognitive functions such as metacognition in PD are scarce. Here, we address for the first time the effect of levodopa and PD on metacognition within the framework of a RL paradigm. In agreement with previous reports, PD patients exhibited reversal shifting impairment with respect to healthy controls (CTRL) regardless of medication condition (MED-ON and MED-OFF), which was supported by a well-known model of learning conditioning (Rescorla–Wagner). In spite that we found a significant association between accuracy and decision confidence level for MED-OFF and CTRL, analysis of metacognitive sensitivity assessed by type 2 signal detection theory (SDT) revealed only a significant underperformance for patients without medication (MED-OFF). This finding points toward a non-compromising positive effect of dopaminergic medication on metacognition for PD.

Pharmacological Dopamine Manipulation Does Not Alter Reward-Based Improvements in Memory Retention during a Visuomotor Adaptation Task

Motor adaptation tasks investigate our ability to adjust motor behaviors to an ever-changing and unpredictable world. Previous work has shown that punishment-based feedback delivered during a visuomotor adaptation task enhances error-reduction, whereas reward increases memory retention. While the neural underpinnings of the influence of punishment on the adaptation phase remain unclear, reward has been hypothesized to increase retention through dopaminergic mechanisms. We directly tested this hypothesis through pharmacological manipulation of the dopaminergic system. A total of 96 young healthy human participants were tested in a placebo-controlled double-blind between-subjects design in which they adapted to a 40° visuomotor rotation under reward or punishment conditions. We confirmed previous evidence that reward enhances retention, but the dopamine (DA) precursor levodopa (LD) or the DA antagonist haloperidol failed to influence performance. We reason that such a negative result could be due to experimental limitations or it may suggest that the effect of reward on motor memory retention is not driven by dopaminergic processes. This provides further insight regarding the role of motivational feedback in optimizing motor learning, and the basis for further decomposing the effect of reward on the subprocesses known to underlie motor adaptation paradigms.

Independent effects of age and levodopa on reversal learning in healthy volunteers

The dopamine overdose hypothesis has provided an important theoretical framework for understanding cognition in Parkinson's disease. It posits that effects of dopaminergic therapy on cognition in Parkinson's disease depend on baseline dopamine levels in brain regions that support different functions. Although functions performed by more severely dopamine-depleted brain regions improve with medication, those associated with less dopamine deficient areas are actually worsened. It is presumed that medication-related worsening of cognition owes to dopamine overdose. We investigated whether age-related changes in baseline dopamine levels would modulate effects of dopaminergic therapy on reward learning in healthy volunteers. In a double-blind, crossover design, healthy younger and older adults completed a probabilistic reversal learning task after treatment with 100/25 mg of levodopa/carbidopa versus placebo. Older adults learned more poorly than younger adults at baseline, being more likely to shift responses after misleading punishment. Levodopa worsened stimulus-reward learning relative to placebo to the same extent in both groups, irrespective of differences in baseline performance and expected dopamine levels. When order effects were eliminated, levodopa induced response shifts after reward more often than placebo. Our results reveal independent deleterious effects of age group and exogenous dopamine on reward learning, suggesting a more complex scenario than predicted by the dopamine overdose hypothesis.

Exploring genetic influences underlying acute aerobic exercise effects on motor learning

The objective of the current work was to evaluate whether the effects of acute aerobic exercise on motor learning were dependent on genetic variants impacting brain-derived neurotrophic factor (BDNF val66met polymorphism) and the dopamine D2 receptor (DRD2/ANKK1 glu713lys polymorphism) in humans. A retrospective analysis was performed to determine whether these polymorphisms influence data from our two previous studies, which both demonstrated that a single bout of aerobic exercise prior to motor practice enhanced implicit motor learning. Here, our main finding was that the effect of acute aerobic exercise on motor learning was dependent on DRD2/ANKK1 genotype. Motor learning was enhanced when aerobic exercise was performed prior to skill practice in glu/glu homozygotes, but not lys allele carriers. In contrast, the BDNF val66met polymorphism did not impact the exercise effect. The results suggest that the dopamine D2 receptor may be involved in acute aerobic exercise effects on motor learning. Such genetic information could inform the development of individualized aerobic exercise strategies to promote motor learning.

Modelling ADHD: A review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning

Attention deficit hyperactivity disorder (ADHD) is characterized by altered decision-making (DM) and reinforcement learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic mechanisms behind ADHD. This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as described by the drift-diffusion model of DM (DDM) and a basic RL model. Empirical studies employing these models are also reviewed. While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique combination of predictions. Empirical studies agree with the theories' assumptions of a lowered DDM drift rate in ADHD, while findings are less conclusive for boundary separation. The few studies employing RL models support a lower choice sensitivity in ADHD, but not an altered learning rate. The discussion outlines research areas for further theoretical refinement in the ADHD field.

Levodopa impairs probabilistic reversal learning in healthy young adults

RATIONALE

Dopaminergic therapy improves some cognitive functions and worsens others in patients with Parkinson's disease (PD). These paradoxical effects are explained by the dopamine overdose hypothesis, which proposes that effects of dopaminergic therapy on a cognitive function is determined by the baseline dopamine levels in brain regions mediating that function.

OBJECTIVES

We directly tested this prevalent hypothesis, evaluating the effects of levodopa on stimulus-reward learning in healthy young adults, who presumably have optimal baseline dopamine levels and dopamine regulation.

METHODS

Twenty-six healthy, young adults completed a probabilistic reversal learning task in a randomized, double-blind, placebo-controlled, crossover design. Participants completed one session on levodopa 100 mg/carbidopa 25 mg and another session on placebo.

RESULTS

We found that levodopa impaired reversal learning relative to placebo. Further analyses revealed that levodopa impaired learning from both punishment and reward.

CONCLUSIONS

Exogenous dopamine impairs stimulus-reward learning, independent of PD pathology and prior to sensitization through repeated exposure, in healthy adults with normal cognition and baseline dopamine function. Our findings support the dopamine overdose hypothesis and caution clinicians about detrimental effects of levodopa in all clinical populations (e.g., early PD, restless leg syndrome) regardless of baseline cognitive and dopaminergic system function.

Psychosocial Modulators of Motor Learning in Parkinson's Disease

Using the remarkable overlap between brain circuits affected in Parkinson’s disease (PD) and those underlying motor sequence learning, we may improve the effectiveness of motor rehabilitation interventions by identifying motor learning facilitators in PD. For instance, additional sensory stimulation and task cueing enhanced motor learning in people with PD, whereas exercising using musical rhythms or console computer games improved gait and balance, and reduced some motor symptoms, in addition to increasing task enjoyment. Yet, despite these advances, important knowledge gaps remain. Most studies investigating motor learning in PD used laboratory-specific tasks and equipment, with little resemblance to real life situations. Thus, it is unknown whether similar results could be achieved in more ecological setups and whether individual’s task engagement could further improve motor learning capacity. Moreover, the role of social interaction in motor skill learning process has not yet been investigated in PD and the role of mind-set and self-regulatory mechanisms have been sporadically examined. Here, we review evidence suggesting that these psychosocial factors may be important modulators of motor learning in PD. We propose their incorporation in future research, given that it could lead to development of improved non-pharmacological interventions aimed to preserve or restore motor function in PD.

Neuroimaging studies of striatum in cognition part II: Parkinson's disease

In recent years a gradual shift in the definition of Parkinson's disease (PD) has been established, from a classical akinetic-rigid movement disorder to a multi-system neurodegenerative disease. While the pathophysiology of PD is complex and goes much beyond the nigro-striatal degeneration, the striatum has been shown to be responsible for many cognitive functions. Patients with PD develop impairments in multiple cognitive domains and the PD model is probably the most extensively studied regarding striatum dysfunction and its influence on cognition. Up to 40% of PD patients present cognitive impairment even in the early stages of disease development. Thus, understanding the key patterns of striatum and connecting regions' influence on cognition will help develop more specific approaches to alleviate cognitive impairment and slow down its decline. This review focuses on the contribution of neuroimaging studies in understanding how striatum impairment affects cognition in PD.

Reversal learning and dopamine: a bayesian perspective

Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (l-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.

Dopaminergic medication impairs feedback-based stimulus-response learning but not response selection in Parkinson's disease

Cognitive dysfunction is a feature of Parkinson's Disease (PD). Some cognitive functions are impaired by dopaminergic medications prescribed to address the movement symptoms that typify PD. Learning appears to be the cognitive function most frequently worsened by dopaminergic therapy. However, this result could reflect either impairments in learning (i.e., acquisition of associations among stimuli, responses, and outcomes) or deficits in performance based on learning (e.g., selecting responses). We sought to clarify the specific effects of dopaminergic medication on (a) stimulus-response association learning from outcome feedback and (b) response selection based on learning, in PD. We tested 28 PD patients on and/or off dopaminergic medication along with 32 healthy, age- and education-matched controls. In Session 1, participants learned to associate abstract images with specific key-press responses through trial and error via outcome feedback. In Session 2, participants provided specific responses to abstract images learned in Session 1, without feedback, precluding new feedback-based learning. By separating Sessions 1 and 2 by 24 h, we could distinguish the effect of dopaminergic medication on (a) feedback-based learning and response selection processes in Session 1 as well as on (b) response selection processes when feedback-based learning could not occur in Session 2. Accuracy achieved at the end of Session 1 were comparable across groups. PD patients on medication learned stimulus-response associations more poorly than PD patients off medication and controls. Medication did not influence decision performance in Session 2. We confirm that dopaminergic therapy impairs feedback-based learning in PD, discounting an alternative explanation that warranted consideration.

Performance of a motor task learned on levodopa deteriorates when subsequently practiced off

Studies in animals and in people with Parkinson's disease (PD) demonstrate complex effects of dopamine on learning motor tasks; its effect on retention of motor learning has received little attention. Recent animal studies demonstrate that practicing a task in the off state, when initially learned in the on state, leads to progressive deterioration in performance. We measured the acquisition and retention of 3 different motor tasks in the presence and absence of levodopa. Twenty individuals with Hoehn and Yahr Stage 1.5 to 3 PD practiced the tasks daily for two 4-day weeks, one half practicing on L-dopa the first week and off the second week. The other half practiced off l-dopa both weeks. The tasks were (1) alternate tapping of 2 keys, (2) moving the body toward 2 targets on a posturography device, and (3) mirror drawing of a star. For the tapping and body movement tests, those who practiced on the first week had a progressive decline in performance with practice during week 2, while subjects off during week 1 maintained or improved. In contrast, for the mirror task, subjects on L-dopa initially had much more difficulty completing the task compared to subjects who practiced off. Both groups improved with practice the first week and had flat performance the second week. These data suggest that performance of speed-accuracy tasks learned in the on state may progressively worsen if subsequently practiced in the off state. In addition, performance, but not learning, of some tasks may be impeded by L-dopa.

Dopamine overdose hypothesis: evidence and clinical implications

About a half a century has passed since dopamine was identified as a neurotransmitter, and it has been several decades since it was established that people with Parkinson's disease receive motor symptom relief from oral levodopa. Despite the evidence that levodopa can reduce motor symptoms, there has been a developing body of literature that dopaminergic therapy can improve cognitive functions in some patients but make them worse in others. Over the past two decades, several laboratories have shown that dopaminergic medications can impair the action of intact neural structures and impair the behaviors associated with these structures. In this review, we consider the evidence that has accumulated in the areas of reversal learning, motor sequence learning, and other cognitive tasks. The purported inverted-U shaped relationship between dopamine levels and performance is complex and includes many contributory factors. The regional striatal topography of nigrostriatal denervation is a critical factor, as supported by multimodal neuroimaging studies. A patient's individual genotype will determine the relative baseline position on this inverted-U curve. Dopaminergic pharmacotherapy and individual gene polymorphisms can affect the mesolimbic and prefrontal cortical dopaminergic functions in a comparable, inverted-U dose-response relationship. Depending on these factors, a patient can respond positively or negatively to levodopa when performing reversal learning and motor sequence learning tasks. These tasks may continue to be relevant as our society moves to increased technological demands of a digital world that requires newly learned motor sequences and adaptive behaviors to manage daily life activities.

Differential effects of Parkinson's disease and dopamine replacement on memory encoding and retrieval

Increasingly memory deficits are recognized in Parkinson's disease (PD). In PD, the dopamine-producing cells of the substantia nigra (SN) are significantly degenerated whereas those in the ventral tegmental area (VTA) are relatively spared. Dopamine-replacement medication improves cognitive processes that implicate the SN-innervated dorsal striatum but is thought to impair those that depend upon the VTA-supplied ventral striatum, limbic and prefrontal cortices. Our aim was to examine memory encoding and retrieval in PD and how they are affected by dopamine replacement. Twenty-nine PD patients performed the Rey Auditory Verbal Learning Test (RAVLT) and a non-verbal analogue, the Aggie Figures Learning Test (AFLT), both on and off dopaminergic medications. Twenty-seven, age-matched controls also performed these memory tests twice and their data were analyzed to correspond to the ON-OFF order of the PD patients to whom they were matched. We contrasted measures that emphasized with those that accentuated retrieval and investigated the effect of PD and dopamine-replacement on these processes separately. For PD patients relative to controls, encoding performance was normal in the off state and was impaired on dopaminergic medication. Retrieval was impaired off medication and improved by dopamine repletion. This pattern of findings suggests that VTA-innervated brain regions such as ventral striatum, limbic and prefrontal cortices are implicated in encoding, whereas the SN-supplied dorsal striatum mediates retrieval. Understanding this pattern of spared functions and deficits in PD, and the effect of dopamine replacement on these distinct memory processes, should prompt closer scrutiny of patients' cognitive complaints to inform titration of dopamine replacement dosages along with motor symptoms.

Genetic variation in the human brain dopamine system influences motor learning and its modulation by L-Dopa

Dopamine is important to learning and plasticity. Dopaminergic drugs are the focus of many therapies targeting the motor system, where high inter-individual differences in response are common. The current study examined the hypothesis that genetic variation in the dopamine system is associated with significant differences in motor learning, brain plasticity, and the effects of the dopamine precursor L-Dopa. Skilled motor learning and motor cortex plasticity were assessed using a randomized, double-blind, placebo-controlled, crossover design in 50 healthy adults during two study weeks, one with placebo and one with L-Dopa. The influence of five polymorphisms with established effects on dopamine neurotransmission was summed using a gene score, with higher scores corresponding to higher dopaminergic neurotransmission. Secondary hypotheses examined each polymorphism individually. While training on placebo, higher gene scores were associated with greater motor learning (p = .03). The effect of L-Dopa on learning varied with the gene score (gene score*drug interaction, p = .008): participants with lower gene scores, and thus lower endogenous dopaminergic neurotransmission, showed the largest learning improvement with L-Dopa relative to placebo (p<.0001), while L-Dopa had a detrimental effect in participants with higher gene scores (p = .01). Motor cortex plasticity, assessed via transcranial magnetic stimulation (TMS), also showed a gene score*drug interaction (p = .02). Individually, DRD2/ANKK1 genotype was significantly associated with motor learning (p = .02) and its modulation by L-Dopa (p<.0001), but not with any TMS measures. However, none of the individual polymorphisms explained the full constellation of findings associated with the gene score. These results suggest that genetic variation in the dopamine system influences learning and its modulation by L-Dopa. A polygene score explains differences in L-Dopa effects on learning and plasticity most robustly, thus identifying distinct biological phenotypes with respect to L-Dopa effects on learning and plasticity. These findings may have clinical applications in post-stroke rehabilitation or the treatment of Parkinson's disease.

Trends and issues in characterizing early cognitive changes in Parkinson's disease

In this review, we first discuss trends and issues in measuring cognitive changes in PD, including recent efforts to define the diagnostic classification of "PD Mild Cognitive Impairment" (PD-MCI). After reviewing some limitations associated with this diagnosis, we discuss how measures derived from the neurocognitive sciences offer better precision in detecting early cognitive changes in PD. To support this idea, we highlight 2 influential lines of current investigation that are unveiling novel insights about specific cognitive processes that are vulnerable early in PD and of critical importance to clinicians involved in treating PD: action control and reward learning and decision making. We conclude by highlighting some extant issues and unresolved questions for future investigations.

Effect of D-amphetamine on emotion-potentiated startle in healthy humans: implications for psychopathy and antisocial behaviour

RATIONALE

An emerging literature associates increased dopaminergic neurotransmission with altered brain response to aversive stimuli in humans. The direction of the effect of dopamine on aversive motivation, however, remains unclear, with some studies reporting increased and others decreased amygdala activation to aversive stimuli following the administration of dopamine agonists. Potentiation of the startle response by aversive foreground stimuli provides an objective and directional measure of emotional reactivity and is considered useful as an index of the emotional effects of different drugs.

OBJECTIVE

We investigated the effects of two doses of D-amphetamine (5 and 10 mg), compared to placebo, for the first time to our knowledge, using the affect-startle paradigm.

METHOD

The study employed a between-subjects, double-blind design, with three conditions: 0 mg (placebo), and 5 and 10 mg D-amphetamine (initially n = 20/group; final sample: n = 18, placebo; n = 18, 5 mg; n = 16, 10 mg). After drug/placebo administration, startle responses (eyeblinks) to intermittent noise probes were measured during viewing of pleasant, neutral and unpleasant images. Participants' general and specific impulsivity and fear-related personality traits were also assessed.

RESULTS

The three groups were comparable on personality traits. Only the placebo group showed significant startle potentiation by unpleasant, relative to neutral, images; this effect was absent in both 5- and 10-mg D-amphetamine groups (i.e. the same effect of D-amphetamine observed at different doses in different people).

CONCLUSIONS

Our findings demonstrate a reduced aversive emotional response under D-amphetamine and may help to account for the known link between the use of psychostimulant drugs and antisocial behaviour.

Dissociating the cognitive effects of levodopa versus dopamine agonists in a neurocomputational model of learning in Parkinson's disease

BACKGROUND/AIMS

Levodopa and dopamine agonists have different effects on the motor, cognitive, and psychiatric aspects of Parkinson's disease (PD).

METHODS

Using a computational model of basal ganglia (BG) and prefrontal cortex (PFC) dopamine, we provide a theoretical synthesis of the dissociable effects of these dopaminergic medications on brain and cognition. Our model incorporates the findings that levodopa is converted by dopamine cells into dopamine, and thus activates prefrontal and striatal D(1) and D(2) dopamine receptors, whereas antiparkinsonian dopamine agonists directly stimulate D(2) receptors in the BG and PFC (although some have weak affinity to D(1) receptors).

RESULTS

In agreement with prior neuropsychological studies, our model explains how levodopa enhances, but dopamine agonists impair or have no effect on, stimulus-response learning and working memory.

CONCLUSION

Our model explains how levodopa and dopamine agonists have differential effects on motor and cognitive processes in PD.

The effect of motivation on movement: a study of bradykinesia in Parkinson's disease

BACKGROUND

Bradykinesia is a cardinal feature of Parkinson's disease (PD). Despite its disabling impact, the precise cause of this symptom remains elusive. Recent thinking suggests that bradykinesia may be more than simply a manifestation of motor slowness, and may in part reflect a specific deficit in the operation of motivational vigour in the striatum. In this paper we test the hypothesis that movement time in PD can be modulated by the specific nature of the motivational salience of possible action-outcomes.

METHODOLOGY/PRINCIPAL FINDINGS

We developed a novel movement time paradigm involving winnable rewards and avoidable painful electrical stimuli. The faster the subjects performed an action the more likely they were to win money (in appetitive blocks) or to avoid a painful shock (in aversive blocks). We compared PD patients when OFF dopaminergic medication with controls. Our key finding is that PD patients OFF dopaminergic medication move faster to avoid aversive outcomes (painful electric shocks) than to reap rewarding outcomes (winning money) and, unlike controls, do not speed up in the current trial having failed to win money in the previous one. We also demonstrate that sensitivity to distracting stimuli is valence specific.

CONCLUSIONS/SIGNIFICANCE

We suggest this pattern of results can be explained in terms of low dopamine levels in the Parkinsonian state leading to an insensitivity to appetitive outcomes, and thus an inability to modulate movement speed in the face of rewards. By comparison, sensitivity to aversive stimuli is relatively spared. Our findings point to a rarely described property of bradykinesia in PD, namely its selective regulation by everyday outcomes.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson’s disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

The effect of dopamine agonists on adaptive and aberrant salience in Parkinson's disease

Clinical evidence suggests that after initiation of dopaminergic medications some patients with Parkinson's disease (PD) develop psychotic symptoms, such as hallucinations and delusions. Here, we tested the hypothesis that the neurocognitive basis of this phenomenon can be defined as the formation of arbitrary and illusory associations between conditioned stimuli and reward signals, called aberrant salience. Young, never-medicated PD patients and matched controls were assessed on a speeded reaction time task in which the probe stimulus was preceded by conditioned stimuli that could signal monetary reward by color or shape. The patients and controls were re-evaluated after 12 weeks during which the patients received a dopamine agonist (pramipexole or ropinirole). Results indicated that dopamine agonists increased both adaptive and aberrant salience in PD patients, that is, formation of real and illusory associations between conditioned stimuli and reward, respectively. This effect was present when associations were assessed by means of faster responding after conditioned stimuli signaling reward (implicit salience) and overt rating of stimulus-reward links (explicit salience). However, unusual feelings and experiences, which are subclinical manifestations of psychotic-like symptoms, were specifically related to irrelevant and illusory stimulus-reward associations (aberrant salience) in PD patients receiving dopamine agonists. The learning of relevant and real stimulus-reward associations (adaptive salience) was not related to unusual experiences. These results suggest that dopamine agonists may increase psychotic-like experiences in young patients with PD, possibly by facilitating dopaminergic transmission in the ventral striatum, which results in aberrant associations between conditioned stimuli and reward.

Dose dependent dopaminergic modulation of reward-based learning in Parkinson's disease

Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward associations, such that, at the instant a specific stimulus is presented, it activates a specific action, based on the expectation that that particular action will likely incur reward (or avoid punishment); and (2) the comparison of predicted and actual outcomes to determine whether the specific stimulus-action association yielded the intended outcome or needs revision. Animal electrophysiology and human fMRI studies converge on the notion that dissociable neural circuitries centered on the striatum are differentially involved in different components of this learning process. The modulatory role of dopamine (DA) in these respective circuits and component processes is of particular relevance to the study of reward-based learning in patients diagnosed with Parkinson's disease (PD). Here we show that the first component process, learning to predict which actions yield reward (supported by the anterior putamen and associated motor circuitry) is impaired when PD patients are taken off their DA medication, whereas DA medication has no systematic effects on the second processes, outcome evaluation (supported by caudate and ventral striatum and associated frontal circuitries). However, the effects of DA medication on these processes depend on dosage, with larger daily doses leading to a decrease in predictability of stimulus-action-reward relations and increase in reward-prediction errors.

Modulation of habit formation by levodopa in Parkinson's disease

Dopamine promotes the execution of positively reinforced actions, but its role for the formation of behaviour when feedback is unavailable remains open. To study this issue, the performance of treated/untreated patients with Parkinson's disease and controls was analysed in an implicit learning task, hypothesising dopamine-dependent adherence to hidden task rules. Sixteen patients on/off levodopa and fourteen healthy subjects engaged in a Go/NoGo paradigm comprising four equiprobable stimuli. One of the stimuli was defined as target which was first consistently preceded by one of the three non-target stimuli (conditioning), whereas this coupling was dissolved thereafter (deconditioning). Two task versions were presented: in a 'Go version', only the target cue required the execution of a button press, whereas non-target stimuli were not instructive of a response; in a 'NoGo version', only the target cue demanded the inhibition of the button press which was demanded upon any non-target stimulus. Levodopa influenced in which task version errors grew from conditioning to deconditioning: in unmedicated patients just as controls errors only rose in the NoGo version with an increase of incorrect responses to target cues. Contrarily, in medicated patients errors went up only in the Go version with an increase of response omissions to target cues. The error increases during deconditioning can be understood as a perpetuation of reaction tendencies acquired during conditioning. The levodopa-mediated modulation of this carry-over effect suggests that dopamine supports habit conditioning under the task demand of response execution, but dampens it when inhibition is required. However, other than in reinforcement learning, supporting dopaminergic actions referred to the most frequent, i. e., non-target behaviour. Since this is passive whenever selective actions are executed against an inactive background, dopaminergic treatment could in according scenarios contribute to passive behaviour in patients with Parkinson's disease.

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson's disease: implications for cognitive function

Cognitive abnormalities are a feature of Parkinson's disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment.

Computational cognitive models of prefrontal-striatal-hippocampal interactions in Parkinson's disease and schizophrenia

Disruption to different components of the prefrontal cortex, basal ganglia, and hippocampal circuits leads to various psychiatric and neurological disorders including Parkinson's disease (PD) and schizophrenia. Medications used to treat these disorders (such as levodopa, dopamine agonists, antipsychotics, among others) affect the prefrontal-striatal-hippocampal circuits in a complex fashion. We have built models of prefrontal-striatal and striatal-hippocampal interactions which simulate cognitive dysfunction in PD and schizophrenia. In these models, we argue that the basal ganglia is key for stimulus-response learning, the hippocampus for stimulus-stimulus representational learning, and the prefrontal cortex for stimulus selection during learning about multidimensional stimuli. In our models, PD is associated with reduced dopamine levels in the basal ganglia and prefrontal cortex. In contrast, the cognitive deficits in schizophrenia are associated primarily with hippocampal dysfunction, while the occurrence of negative symptoms is associated with frontostriatal deficits in a subset of patients. In this paper, we review our past models and provide new simulation results for both PD and schizophrenia. We also describe an extended model that includes simulation of the different functional role of D1 and D2 dopamine receptors in the basal ganglia and prefrontal cortex, a dissociation we argue is essential for understanding the non-uniform effects of levodopa, dopamine agonists, and antipsychotics on cognition. Motivated by clinical and physiological data, we discuss model limitations and challenges to be addressed in future models of these brain disorders.