Novelty seeking behaviour in Parkinson's disease

Impulse Control Disorders in Parkinson's Disease: An Overview of Risk Factors, Pathogenesis and Pharmacological Management

Impulse control disorders in Parkinson's disease are relatively common drug-induced addictive behaviours that are usually triggered by the dopamine agonists pramipexole, ropinirole and rotigotine. This narrative review aimed to provide a comprehensive overview of the current knowledge of impulse control disorders in Parkinson's disease. We summarised the prevalence, clinical features, risk factors and potential underlying mechanisms of impulse control disorders in Parkinson's disease. Moreover, recent advances in behavioural and imaging characteristics and management strategies are discussed. Early detection as well as a tailored multidisciplinary approach, which typically includes careful adjustment of the dopaminergic therapy and the treatment of associated neuropsychiatric symptoms, are necessary. In some cases, a continuous delivery of levodopa via a pump or the dopamine D1 receptor agonist, apomorphine, can be considered. In selected patients without cognitive or speech impairment, deep brain stimulation of the subthalamic nucleus can also improve addictions. Finding the right balance of tapering dopaminergic dose (usually dopamine agonists) without worsening motor symptoms is essential for a beneficial long-term outcome.

Multiple and subject-specific roles of uncertainty in reward-guided decision-making

Decision-making in noisy, changing, and partially observable environments entails a basic tradeoff between immediate reward and longer-term information gain, known as the exploration-exploitation dilemma. Computationally, an effective way to balance this tradeoff is by leveraging uncertainty to guide exploration. Yet, in humans, empirical findings are mixed, from suggesting uncertainty-seeking to indifference and avoidance. In a novel bandit task that better captures uncertainty-driven behavior, we find multiple roles for uncertainty in human choices. First, stable and psychologically meaningful individual differences in uncertainty preferences actually range from seeking to avoidance, which can manifest as null group-level effects. Second, uncertainty modulates the use of basic decision heuristics that imperfectly exploit immediate rewards: a repetition bias and win-stay-lose-shift heuristic. These heuristics interact with uncertainty, favoring heuristic choices under higher uncertainty. These results, highlighting the rich and varied structure of reward-based choice, are a step to understanding its functional basis and dysfunction in psychopathology.

Curiosity: primate neural circuits for novelty and information seeking

For many years, neuroscientists have investigated the behavioural, computational and neurobiological mechanisms that support value-based decisions, revealing how humans and animals make choices to obtain rewards. However, many decisions are influenced by factors other than the value of physical rewards or second-order reinforcers (such as money). For instance, animals (including humans) frequently explore novel objects that have no intrinsic value solely because they are novel and they exhibit the desire to gain information to reduce their uncertainties about the future, even if this information cannot lead to reward or assist them in accomplishing upcoming tasks. In this Review, I discuss how circuits in the primate brain responsible for detecting, predicting and assessing novelty and uncertainty regulate behaviour and give rise to these behavioural components of curiosity. I also briefly discuss how curiosity-related behaviours arise during postnatal development and point out some important reasons for the persistence of curiosity across generations.

Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors

Deciding whether to forego immediate rewards or explore new opportunities is a key component of flexible behavior and is critical for the survival of the species. Although previous studies have shown that different cortical and subcortical areas, including the amygdala and ventral striatum (VS), are implicated in representing the immediate (exploitative) and future (explorative) value of choices, the effect of the motor system used to make choices has not been examined. Here, we tested male rhesus macaques with amygdala or VS lesions on two versions of a three-arm bandit task where choices were registered with either a saccade or an arm movement. In both tasks we presented the monkeys with explore-exploit tradeoffs by periodically replacing familiar options with novel options that had unknown reward probabilities. We found that monkeys explored more with saccades but showed better learning with arm movements. VS lesions caused the monkeys to be more explorative with arm movements and less explorative with saccades, although this may have been due to an overall decrease in performance. VS lesions affected the monkeys' ability to learn novel stimulus-reward associations in both tasks, while after amygdala lesions this effect was stronger when choices were made with saccades. Further, on average, VS and amygdala lesions reduced the monkeys' ability to choose better options only when choices were made with a saccade. These results show that learning reward value associations to manage explore-exploit behaviors is motor system dependent and they further define the contributions of amygdala and VS to reinforcement learning.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Electrophysiological Markers of Aberrant Cue-Specific Exploration in Hazardous Drinkers

Background

Hazardous drinking is associated with maladaptive alcohol-related decision-making. Existing studies have often focused on how participants learn to exploit familiar cues based on prior reinforcement, but little is known about the mechanisms that drive hazardous drinkers to explore novel alcohol cues when their value is not known.

Methods

We investigated exploration of novel alcohol and non-alcohol cues in hazardous drinkers (N = 27) and control participants (N = 26) during electroencephalography (EEG). A normative computational model with two free parameters was fit to estimate participants' weighting of the future value of exploration and immediate value of exploitation.

Results

Hazardous drinkers demonstrated increased exploration of novel alcohol cues, and conversely, increased probability of exploiting familiar alternatives instead of exploring novel non-alcohol cues. The motivation to explore novel alcohol stimuli in hazardous drinkers was driven by an elevated relative future valuation of uncertain alcohol cues. P3a predicted more exploratory decision policies driven by an enhanced relative future valuation of novel alcohol cues. P3b did not predict choice behavior, but computational parameter estimates suggested that hazardous drinkers with enhanced P3b to alcohol cues were likely to learn to exploit their immediate expected value.

Conclusions

Hazardous drinkers did not display atypical choice behavior, different P3a/P3b amplitudes, or computational estimates to novel non-alcohol cues-diverging from previous studies in addiction showing atypical generalized explore-exploit decisions with non-drug-related cues. These findings reveal that cue-specific neural computations may drive aberrant alcohol-related decision-making in hazardous drinkers-highlighting the importance of drug-relevant cues in studies of decision-making in addiction.

Characteristics of behavioural addiction in Parkinson's disease patients with self-reported impulse control disorder and controls matched for levodopa equivalent dose: a matched case-control study

Impulse control disorders (ICD) in Parkinson's disease (PD) frequently occur, not always as a direct consequence of dopaminergic medication. This study investigated premorbid personality traits and behavioural characteristics in non-demented PD patients with self-reported symptoms of ICD (PD-srICD). From a total of 200 non-demented PD patients who filled out questionnaires assessing symptoms and severity of ICD, those were classified as PD-srICD (n = 32) who reported current occurrence of at least one compulsive behaviour (gambling, sexual behaviour, buying behaviour, or eating). As a control group, 32 patients with no self-reported ICD symptoms were matched for levodopa equivalent daily dose. The demographic, clinical, and premorbid personality profiles were compared between both groups. Frequency of psychological characteristics indicating substance use disorder was evaluated in patients with PD-srICD. Patients with PD-srICD were more frequently male, younger at examination, had earlier PD onset, more depression, higher non-motor burden, less quality of life (p < 0.05, respectively), and more frequently reported premorbid sensation seeking/novelty orientation (p = 0.03) and joyful experience of stress (p = 0.04) than patients in the control group. Of patients with PD-srICD, 90.6% reported at least one behavioural characteristic of substance use disorder, most frequently positive expectations following ICD behaviour and illusional beliefs about its behavioural control. Signs of addiction were common among patients with PD-srICD. Therefore, the profile of psychological characteristics in patients with PD-srICD resembled that of patients with substance use disorder. It can be concluded that dopamine replacement therapy (DRT) alone does not account for PD-srICD and that thorough psychological diagnostics are recommended.

A circuit-based approach to modulate hypersexuality in Parkinson's disease

AIM

Impulse-control disorder is a common neuropsychiatric complication in Parkinson's disease (PD) under dopamine replacement therapy. Prior studies tested the balance between enhanced desire towards reward and cognitive control deficits, hypothesized to be biased towards the former in impulse control disorders. We provide evidence for this hypothesis by measuring behavioral and neural patterns behind the influence of sexual desire over response inhibition and tools towards functional restoration using repetitive transcranial stimulation in patients with hypersexuality as predominant impulsive disorder.

METHODS

The effect of sexual cues on inhibition was measured with a novel erotic stop-signal task under on and off dopaminergic medication. Task-related functional and anatomical connectivity models were estimated in 16 hypersexual and 17 nonhypersexual patients with PD as well as in 17 healthy controls. Additionally, excitatory neuromodulation using intermittent theta-burst stimulation (sham-controlled) was applied over the pre-supplementary motor area in 20 additional hypersexual patients with PD aiming to improve response inhibition.

RESULTS

Compared with their nonhypersexual peers, patients with hypersexuality recruited caudate, pre-supplementary motor area, ventral tegmental area, and anterior cingulate cortex while on medication. Reduced connectivity was found between pre-supplementary motor area and caudate nucleus in hypersexual compared with nonhypersexual patients (while medicated), a result paralleled by compensatory enhanced anatomical connectivity. Furthermore, stimulation over the pre-supplementary motor area improved response inhibition in hypersexual patients with PD when exposed to sexual cues.

CONCLUSION

This study, therefore, has identified a specific fronto-striatal and mesolimbic circuitry underlying uncontrolled sexual responses in medicated patients with PD where cortical neuromodulation halts its expression.

The neurocomputational bases of explore-exploit decision-making

Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives. Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans. We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded. Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex. These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

Aberrant Salient and Corticolimbic Connectivity in Hypersexual Parkinson's Disease

Introduction: Impulse control disorders (ICDs) represent a side effect of dopaminergic medication in Parkinson's disease (PD). Patients experience an excessive desire toward natural rewards paired with uncontrolled actions. Yet, the precise neural and behavioral mechanisms associated with ICDs and, importantly, each specific subdomain remain unclear. We aim to decipher resting-state and corticolimbic functional connectivity in PD patients with and without hypersexual ICD. Materials and Methods: Seventeen PD patients with hypersexuality (PD+HS) and 15 PD patients without hypersexuality (PD-HS) underwent two sessions (with and without medication) of resting-state functional magnetic resonance imaging and were compared with 17 healthy controls. Dual-regression independent component analyses extracted salience, sensorimotor, default-mode, and central executive networks. Seed-based functional connectivity with three striatal subdivisions (motor, associative, and limbic) was obtained and significant changes were correlated with key impulsivity and inhibitory measures. Results: Enhanced salience network (SN) activity represented by a significant rise in the right inferior frontal gyrus was found in PD+HS compared with PD-HS. Connectivity analyses revealed a functional disconnection between associative and limbic striatum with precuneus and superior parietal lobe in PD+HS, some connections explained by abnormal sexual behavior and inhibition in PD+HS. Conclusions: Hypersexual ICD is associated with enhanced SN signaling and corticolimbic disconnections, including striatal associative and limbic loops that contribute to altered control of sexually driven behavior and overall severity in PD and ICD.

Individual associations of adolescent alcohol use disorder versus cannabis use disorder symptoms in neural prediction error signaling and the response to novelty

Two of the most commonly used illegal substances by adolescents are alcohol and cannabis. Alcohol use disorder (AUD) and cannabis use disorder (CUD) are associated with poorer decision-making in adolescents. In adolescents, level of AUD symptomatology has been negatively associated with striatal reward responsivity. However, little work has explored the relationship with striatal reward prediction error (RPE) representation and the extent to which any augmentation of RPE by novel stimuli is impacted. One-hundred fifty-one adolescents participated in the Novelty Task while undergoing functional magnetic resonance imaging (fMRI). In this task, participants learn to choose novel or non-novel stimuli to gain monetary reward. Level of AUD symptomatology was negatively associated with both optimal decision-making and BOLD response modulation by RPE within striatum and regions of prefrontal cortex. The neural alterations in RPE representation were particularly pronounced when participants were exploring novel stimuli. Level of CUD symptomatology moderated the relationship between novelty propensity and RPE representation within inferior parietal lobule and dorsomedial prefrontal cortex. These data expand on an emerging literature investigating individual associations of AUD symptomatology levels versus CUD symptomatology levels and RPE representation during reinforcement processing and provide insight on the role of neuro-computational processes underlying reinforcement learning/decision-making in adolescents.

Co-occurrence of apathy and impulse control disorders in Parkinson disease

OBJECTIVE

To empirically test whether apathy and impulse control disorders (ICDs) represent independent, opposite ends of a motivational spectrum.

METHODS

In this single-center, cross-sectional study, we obtained retrospective demographics and clinical data for 887 patients with idiopathic Parkinson disease (PD) seen at a tertiary care center. Mood and motivation disturbances were classified using recommended cutoff scores from self-reported measures of apathy, ICD, anxiety, and depression.

RESULTS

Prevalence rates included 29.0% of patients with PD with depression, 40.7% with anxiety, 41.3% with apathy, 27.6% with ICDs, and 17.0% with both apathy and ICD. The majority (61.6%) of people reporting clinically significant ICDs also reported clinically significant apathy, and more than a third of patients with apathy (41.3%) also reported elevated ICD. Anxiety and depression were highest in patients with both apathy and ≥1 ICDs. Dopamine agonist use was higher in people with only ICD compared to people with only apathy. Mood significantly interacted with demographic variables to predict motivational disturbances.

CONCLUSIONS

Motivational disturbances are common comorbid conditions in patients with PD. In addition, these complex behavioral syndromes interact with mood in clinically important ways that may influence the design of future clinical trials and the development of novel therapies. This study challenges the concept of apathy and ICD in PD as opposite ends of a spectrum.

Unravelling the effects of methylphenidate on the dopaminergic and noradrenergic functional circuits

Functional magnetic resonance imaging (fMRI) can be combined with drugs to investigate the system-level functional responses in the brain to such challenges. However, most psychoactive agents act on multiple neurotransmitters, limiting the ability of fMRI to identify functional effects related to actions on discrete pharmacological targets. We recently introduced a multimodal approach, REACT (Receptor-Enriched Analysis of functional Connectivity by Targets), which offers the opportunity to disentangle effects of drugs on different neurotransmitters and clarify the biological mechanisms driving clinical efficacy and side effects of a compound. Here, we focus on methylphenidate (MPH), which binds to the dopamine transporter (DAT) and the norepinephrine transporter (NET), to unravel its effects on dopaminergic and noradrenergic functional circuits in the healthy brain at rest. We then explored the relationship between these target-enriched resting state functional connectivity (FC) maps and inter-individual variability in behavioural responses to a reinforcement-learning task encompassing a novelty manipulation to disentangle the molecular systems underlying specific cognitive/behavioural effects. Our main analysis showed a significant MPH-induced FC increase in sensorimotor areas in the functional circuit associated with DAT. In our exploratory analysis, we found that MPH-induced regional variations in the DAT and NET-enriched FC maps were significantly correlated with some of the inter-individual differences on key behavioural responses associated with the reinforcement-learning task. Our findings show that main MPH-related FC changes at rest can be understood through the distribution of DAT in the brain. Furthermore, they suggest that when compounds have mixed pharmacological profiles, REACT may be able to capture regional functional effects that are underpinned by the same cognitive mechanism but are related to engagement of distinct molecular targets.

Linking brain activity during sequential gambling to impulse control in Parkinson's disease

Dopaminergic treatment may impair the ability to suppress impulsive behaviours in patients with Parkinson's disease, triggering impulse control disorders. It is unclear how dopaminergic medication affects the neural networks that contribute to withholding inappropriate actions. To address this question, we mapped task-related brain activity with whole-brain functional magnetic resonance imaging at 3 Tesla in 26 patients with Parkinson's disease. Patients performed a sequential gambling task while being ON and OFF their regular dopaminergic treatment. During a gambling round, patients repeatedly decided between the option to continue with gambling and accumulate more monetary reward under increasing risk or the option to bank the current balance and start a new round. 13 patients had an impulse control disorder (ICD + group). These patients did not differ in risk-taking attitude during sequential gambling from 13 patients without impulse control disorder (ICD - group), but they displayed differences in gambling-related activity in cortico-subcortical brain areas supporting inhibitory control. First, the ICD + group showed reduced "continue-to-gamble" activity in right inferior frontal gyrus and subthalamic nucleus. Second, the individual risk-attitude scaled positively with "continue-to-gamble" activity in right subthalamic nucleus and striatum in the ICD - group only. Third, ICD + patients differed in their functional neural responses to dopaminergic treatment from ICD - patients: dopaminergic therapy reduced functional connectivity between inferior frontal gyrus and subthalamic nucleus during "continue-to-gamble" decisions and attenuated striatal responses towards accumulating reward and risk. Together, the medication-independent (trait) and medication-related (state) differences in neural activity may set a permissive stage for the emergence of impulse control disorders during dopamine replacement therapy in Parkinson's disease.

Primate Orbitofrontal Cortex Codes Information Relevant for Managing Explore-Exploit Tradeoffs

Reinforcement learning (RL) refers to the behavioral process of learning to obtain reward and avoid punishment. An important component of RL is managing explore-exploit tradeoffs, which refers to the problem of choosing between exploiting options with known values and exploring unfamiliar options. We examined correlates of this tradeoff, as well as other RL related variables, in orbitofrontal cortex (OFC) while three male monkeys performed a three-armed bandit learning task. During the task, novel choice options periodically replaced familiar options. The values of the novel options were unknown, and the monkeys had to explore them to see if they were better than other currently available options. The identity of the chosen stimulus and the reward outcome were strongly encoded in the responses of single OFC neurons. These two variables define the states and state transitions in our model that are relevant to decision-making. The chosen value of the option and the relative value of exploring that option were encoded at intermediate levels. We also found that OFC value coding was stimulus specific, as opposed to coding value independent of the identity of the option. The location of the option and the value of the current environment were encoded at low levels. Therefore, we found encoding of the variables relevant to learning and managing explore-exploit tradeoffs in OFC. These results are consistent with findings in the ventral striatum and amygdala and show that this monosynaptically connected network plays an important role in learning based on the immediate and future consequences of choices.SIGNIFICANCE STATEMENT Orbitofrontal cortex (OFC) has been implicated in representing the expected values of choices. Here we extend these results and show that OFC also encodes information relevant to managing explore-exploit tradeoffs. Specifically, OFC encodes an exploration bonus, which characterizes the relative value of exploring novel choice options. OFC also strongly encodes the identity of the chosen stimulus, and reward outcomes, which are necessary for computing the value of novel and familiar options.

Wait and you shall see: sexual delay discounting in hypersexual Parkinson's disease

Patients with Parkinson's disease may develop impulse control disorders under dopaminergic treatments. Impulse control disorders include a wide spectrum of behaviours, such as hypersexuality, pathological gambling or compulsive shopping. Yet, the neural systems engaged in specific impulse control disorders remain poorly characterized. Here, using model-based functional MRI, we aimed to determine the brain systems involved during delay-discounting of erotic rewards in hypersexual patients with Parkinson's disease (PD+HS), patients with Parkinson's disease without hypersexuality (PD - HS) and controls. Patients with Parkinson's disease were evaluated ON and OFF levodopa (counterbalanced). Participants had to decide between two options: (i) wait for 1.5 s to briefly view an erotic image; or (ii) wait longer to see the erotic image for a longer period of time. At the time of decision-making, we investigated which brain regions were engaged with the subjective valuation of the delayed erotic reward. At the time of the rewarded outcome, we searched for the brain regions responding more robustly after waiting longer to view the erotic image. PD+HS patients showed reduced discounting of erotic delayed rewards, compared to both patients with Parkinson's disease and controls, suggesting that they accepted waiting longer to view erotic images for a longer period of time. Thus, when using erotic stimuli that motivate PD+HS, these patients were less impulsive for the immediate reward. At the brain system level, this effect was paralleled by the fact that PD+HS, as compared to controls and PD - HS, showed a negative correlation between subjective value of the delayed reward and activity of medial prefrontal cortex and ventral striatum. Consistent with the incentive salience hypothesis combining learned cue-reward associations with current relevant physiological state, dopaminergic treatment in PD+HS boosted excessive 'wanting' of rewards and heightened activity in the anterior medial prefrontal cortex and the posterior cingulate cortex, as reflected by higher correlation with subjective value of the option associated to the delayed reward when ON medication as compared to the OFF medication state. At the time of outcome, the anterior medial prefrontal/rostral anterior cingulate cortex showed an interaction between group (PD+HS versus PD - HS) and medication (ON versus OFF), suggesting that dopaminergic treatment boosted activity of this brain region in PD+HS when viewing erotic images after waiting for longer periods of time. Our findings point to reduced delay discounting of erotic rewards in PD+HS, both at the behavioural and brain system levels, and abnormal reinforcing effect of levodopa when PD+HS patients are confronted with erotic stimuli.10.1093/brain/awy298_video1awy298media15983845074001.

Subcortical Substrates of Explore-Exploit Decisions in Primates

The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

A neurocomputational account of reward and novelty processing and effects of psychostimulants in attention deficit hyperactivity disorder

Computational models of reinforcement learning have helped dissect discrete components of reward-related function and characterize neurocognitive deficits in psychiatric illnesses. Stimulus novelty biases decision-making, even when unrelated to choice outcome, acting as if possessing intrinsic reward value to guide decisions toward uncertain options. Heightened novelty seeking is characteristic of attention deficit hyperactivity disorder, yet how this influences reward-related decision-making is computationally encoded, or is altered by stimulant medication, is currently uncertain. Here we used an established reinforcement-learning task to model effects of novelty on reward-related behaviour during functional MRI in 30 adults with attention deficit hyperactivity disorder and 30 age-, sex- and IQ-matched control subjects. Each participant was tested on two separate occasions, once ON and once OFF stimulant medication. OFF medication, patients with attention deficit hyperactivity disorder showed significantly impaired task performance (P = 0.027), and greater selection of novel options (P = 0.004). Moreover, persistence in selecting novel options predicted impaired task performance (P = 0.025). These behavioural deficits were accompanied by a significantly lower learning rate (P = 0.011) and heightened novelty signalling within the substantia nigra/ventral tegmental area (family-wise error corrected P < 0.05). Compared to effects in controls, stimulant medication improved attention deficit hyperactivity disorder participants' overall task performance (P = 0.011), increased reward-learning rates (P = 0.046) and enhanced their ability to differentiate optimal from non-optimal novel choices (P = 0.032). It also reduced substantia nigra/ventral tegmental area responses to novelty. Preliminary cross-sectional evidence additionally suggested an association between long-term stimulant treatment and a reduction in the rewarding value of novelty. These data suggest that aberrant substantia nigra/ventral tegmental area novelty processing plays an important role in the suboptimal reward-related decision-making characteristic of attention deficit hyperactivity disorder. Compared to effects in controls, abnormalities in novelty processing and reward-related learning were improved by stimulant medication, suggesting that they may be disorder-specific targets for the pharmacological management of attention deficit hyperactivity disorder symptoms.

Impact of Cognitive Profile on Impulse Control Disorders Presence and Severity in Parkinson's Disease

Background: Impulse control disorders (ICDs) and related behaviors are frequent in Parkinson's disease (PD). Mild cognitive impairment (PD-MCI) and dementia (PDD), both characterized by heterogeneous cognitive phenotypes, are also commonly reported in PD. However, the frequency and severity of ICD within PD cognitive states is unknown.

Methods: Three hundred and twenty-six PD patients completed a comprehensive neuropsychological assessment and were classified as PD-MCI, PDD, or without cognitive alterations (PD-NC). The Minnesota impulsive disorders interview was used to ascertain the presence (ICD+) or absence (ICD–) of ICD. The Questionnaire for Impulsive-Compulsive Disorders in Parkinson's Disease-Rating Scale was used to assess ICD severity. A subsample of 286 patients evaluated with the same cognitive tasks was selected in order to investigate the characteristics of ICD in PD cognitive phenotypes.

Results: ICDs were present in 55% of PD-NC, in 50% of PD-MCI, and in 42% of PDD patients. Frequencies of ICD+ with attentive (ICD+: 20% vs. ICD–: 4%; p = 0.031) and executive impairments (ICD+: 44% vs. ICD–: 30%; p = 0.027) were higher in the PD-MCI and PDD subgroups, respectively. As expected, no differences were observed in the PD-NC. PD-MCI with attentive impairments presented higher percentage of ICD+ with deficits in the Trail Making Test B-A but not in the Digit Span Sequencing task. In PDD, executive failures concerned Similarities task (ICD+: 67%; ICD–: 29%; p = 0.035), with no differences between ICD+ and ICD– in the Stroop task.

Conclusions: Prevalence and severity of ICDs and related behaviors do not differ in PD with different cognitive states. However, ICD+ are more likely to show deficits, respectively in attentive and in executive domains, specifically in the Trail Making Test B-A task for the attention and working memory domain in PD-MCI and in the Similarities task for the executive function domain in PDD. Prospective studies should evaluate if these tests can be used as screening tool for ICDs in PD.

Medications, Deep Brain Stimulation, and Other Factors Influencing Impulse Control Disorders in Parkinson's Disease

Impulse control disorders (ICDs) in Parkinson's disease (PD) have a high cumulative incidence and negatively impact quality of life. ICDs are influenced by a complex interaction of multiple factors. Although it is now well-recognized that dopaminergic treatments and especially dopamine agonists underpin many ICDs, medications alone are not the sole cause. Susceptibility to ICD is increased in the setting of PD. While causality can be challenging to ascertain, a wide range of modifiable and non-modifiable risk factors have been linked to ICDs. Common characteristics of PD patients with ICDs have been consistently identified across many studies; for example, males with an early age of PD onset and dopamine agonist use have a higher risk of ICD. However, not all cases of ICDs in PD can be directly attributable to dopamine, and studies have concluded that additional factors such as genetics, smoking, and/or depression may be more predictive. Beyond dopamine, other ICD associations have been described but remain difficult to explain, including deep brain stimulation surgery, especially in the setting of a reduction in dopaminergic medication use. In this review, we will summarize the demographic, genetic, behavioral, and clinical contributions potentially influencing ICD onset in PD. These associations may inspire future preventative or therapeutic strategies.

Neurocognitive correlates of medication-induced addictive behaviours in Parkinson's disease: A systematic review

Dopaminergic medication can induce severe addictive behaviours (e.g., pathological gambling) in susceptible Parkinson's disease (PD) patients. It is still unknown which particular neurocognitive processes become exacerbated or dysfunctional in PD patients with addictive behaviours. We sought to systematically review the relevant literature to identity potential neurocognitive correlates of medication-induced addictive behaviours in PD. We framed our review around neurocognitive processes central to four dominant accounts of substance addiction: 'aberrant learning', 'incentive sensitization', 'impulsivity to compulsivity' and 'impaired response inhibition and salience attribution'. Searches of the PubMed and Scopus databases were completed on June 23, 2017. To be included, studies were required to involve: (a) medicated PD patients, without a history of deep brain stimulation, with and without addictive behaviours; (b) a reward-related or decision-making task; and (c) statistical comparison of addictive and non-addictive groups' 'on' medication performance on the task(s). Studies were summarised qualitatively with statistically significant (p<.05) group differences and effect sizes (Cohen's d) highlighted. 35 studies were included. Findings showed that the extant literature is highly heterogeneous. The domains of reward and punishment learning, reflection impulsivity and disadvantageous decision-making exemplify this. More homogeneity exists in domains in which (a) neurocognitive dysfunction is not apparent (motor control, cognitive/attentional flexibility and cognitive control) or (b) typical neurocognitive processes appear exacerbated by medication (reward motivation and choice impulsivity). Future large-scale neurocognitive studies are still required to develop our scientific understanding of addictive behaviours in PD and aid their clinical treatment and prediction.

The value of novelty in schizophrenia

Influential models of schizophrenia suggest that patients experience incoming stimuli as excessively novel and motivating, with important consequences for hallucinatory experience and delusional belief. However, whether schizophrenia patients exhibit excessive novelty value and whether this interferes with adaptive behaviour has not yet been formally tested. Here, we employed a three-armed bandit task to investigate this hypothesis. Schizophrenia patients and healthy controls were first familiarised with a group of images and then asked to repeatedly choose between familiar and unfamiliar images associated with different monetary reward probabilities. By fitting a reinforcement-learning model we were able to estimate the values attributed to familiar and unfamiliar images when first presented in the context of the decision-making task. In line with our hypothesis, we found increased preference for newly introduced images (irrespective of whether these were familiar or unfamiliar) in patients compared to healthy controls and this to correlate with severity of hallucinatory experience. In addition, we found a correlation between value assigned to novel images and task performance, suggesting that excessive novelty value may interfere with optimal learning in patients, putatively through the disruption of the mechanisms regulating exploration versus exploitation. Our results suggest excessive novelty value in patients, whereby even previously seen stimuli acquire higher value as the result of their exposure in a novel context - a form of 'hyper novelty' which may explain why patients are often attracted by familiar stimuli experienced as new.

Impulse Control and Related Disorders in Parkinson's Disease

Impulse control disorders (ICDs), such as compulsive gambling, buying, sexual, and eating behaviors, are a serious and increasingly recognized complication in Parkinson's disease (PD), occurring in up to 20% of PD patients over the course of their illness. Related behaviors include punding (stereotyped, repetitive, purposeless behaviors), dopamine dysregulation syndrome (DDS) (compulsive medication overuse), and hobbyism (e.g., compulsive internet use, artistic endeavors, and writing). These disorders have a significant impact on quality of life and function, strain interpersonal relationships, and worsen caregiver burden, and are associated with significant psychiatric comorbidity. ICDs have been most closely related to the use of dopamine agonists (DAs), while DDS is primarily associated with shorter acting, higher potency dopamine replacement therapy (DRT), such as levodopa. However, in preliminary research ICDs have also been reported to occur with monoamine oxidase inhibitor-B and amantadine treatment, and after deep brain stimulation (DBS) surgery. Other risk factors for ICDs may include sex (e.g., male sex for compulsive sexual behavior, and female sex for compulsive buying behavior); younger age overall at PD onset; a pre-PD history of an ICD; personal or family history of substance abuse, bipolar disorder, or gambling problems; and impulsive personality traits. Dysregulation of the mesocorticolimbic dopamine system is thought to be the major neurobiological substrate for ICDs in PD, but there is preliminary evidence for alterations in opiate and serotonin systems too. The primary treatment of ICDs in PD is discontinuation of the offending treatment, but not all patients can tolerate this due to worsening motor symptoms or DA withdrawal syndrome. While psychiatric medications and psychosocial treatments are frequently used to treat ICDs in the general population, there is limited empirical evidence for their use in PD, so it is critical for patients to be monitored closely for ICDs from disease onset and routine throughout its course. In the future, it may be possible to use a precision medicine approach to decrease the incidence of ICDs in PD by avoiding DA use in patients determined to be at highest risk based on their clinical and neurobiological (e.g., motor presentation, behavioral measures of medication response, genetics, dopamine transporter neuroimaging) profile. Additionally, as empirically validated treatments for ICDs and similar disorders (e.g., substance use disorders) emerge, it will also be important to examine their efficacy and tolerability in individuals with comorbid PD.

Cognitive Psychophysiological Substrates of Affective Temperaments

Affective temperaments are the subclinical manifestations or phenotypes of mood states and hypothetically represent one healthy end of the mood disorder spectrum. However, there is a scarcity of studies investigating the neurobiological basis of affective temperaments. One fundamental aspect of temperament is the behavioral reactivity to environmental stimuli, which can be effectively evaluated by use of cognitive event-related potentials (ERPs) reflecting the diversity of information processing. The aim of the present study is to explore the associations between P300 and the affective temperamental traits in healthy individuals. We recorded the P300 ERP waves using an auditory oddball paradigm in 50 medical student volunteers (23 females, 27 males). Participants' affective temperaments were evaluated using the Temperament Evaluation of Memphis, Pisa, Paris, and San Diego-auto questionnaire version (TEMPS-A). In bivariate analyses, depressive temperament score was significantly correlated with P300 latency ( r_s = 0.37, P < .01). In a multiple linear regression analysis, P300 latency showed a significant positive correlation with scores of depressive temperament (β = 0.40, P < .01) and a significant negative one with scores of cyclothymic temperament (β = -0.29, P = .03). Affective temperament scores were not associated with P300 amplitude and reaction times. These results indicate that affective temperaments are related to information processing in the brain. Depressive temperament may be characterized by decreased physiological arousal and slower information processing, while the opposite was observed for cyclothymic temperament.

A Comparison of Treatment-Seeking Behavioral Addiction Patients with and without Parkinson's Disease

The administration of dopaminergic medication to treat the symptoms of Parkinson's disease (PD) is associated with addictive behaviors and impulse control disorders. Little is known, however, on how PD patients differ from other patients seeking treatments for behavioral addictions. The aim of this study was to compare the characteristics of behavioral addiction patients with and without PD. N = 2,460 treatment-seeking men diagnosed with a behavioral addiction were recruited from a university hospital. Sociodemographic, impulsivity [Barratt Impulsiveness Scale (BIS-11)], and personality [Temperament and Character Inventory-Revised (TCI-R)] measures were taken upon admission to outpatient treatment. Patients in the PD group were older and had a higher prevalence of mood disorders than patients without PD. In terms of personality characteristics and impulsivity traits, PD patients appeared to present a more functional profile than PD-free patients with a behavioral addiction. Our results suggest that PD patients with a behavioral addiction could be more difficult to detect than their PD-free counterparts in behavioral addiction clinical setting due to their reduced levels of impulsivity and more standard personality traits. As a whole, this suggests that PD patients with a behavioral addiction may have different needs from PD-free behavioral addiction patients and that they could potentially benefit from targeted interventions.

Executive functioning and risk-taking behavior in Parkinson's disease patients with impulse control disorders

Impulse control disorders (ICD) are common in Parkinson's disease (PD) and are associated with dopaminergic medication. The purpose of this study was to investigate executive function and risk-taking behavior in PD patients with ICD. 17 PD patients with ICD (ICD-PD) were compared to 20 PD patients without ICD (CTRL-PD) using neuropsychological and experimental tasks. Executive functions were assessed using standard executive testing (Conner's Performance Test, Modified Wisconsin Card Sorting Test, Trail Making Test and phonological verbal fluency). Subjects were also submitted to an experimental gambling task consisted of three decks of money cards: neutral deck (equal opportunity for gains as losses), winning deck (small amount of money with a positive balance) and loser deck (high amount of money with a negative balance), evaluating risk-taking behavior (number of cards picked in each deck) and valuation of the reward (subjective appreciation of the value of each deck). There was no significant difference in executive functioning between groups. Both groups selected more cards in the losing deck (high amount of money) as compared to the neutral deck (Mann-Whitney test, ICD-PD, p = 0.02; CTRL-PD, p = 0.003) and to the winning deck (Mann-Whitney test, ICD-PD p = 0.0001; CTRL-PD p = 0.003), suggesting an increased risk-taking behavior. Interestingly, we found that ICD-PD patients estimated the value of decks differently from CTRL-PD patients, taking into account mainly the positive reinforced value of the decks (Mann-Whitney test, p = 0.04). This study showed that executive pattern and risk-taking behavior are similar between ICD-PD and CTRL-PD patients. However, ICD-PD patients showed a specific deficit of the subjective estimation of the reward. Links between this deficit and metacognitive skills are discussed.

Psychiatric Conditions in Parkinson Disease: A Comparison With Classical Psychiatric Disorders

Psychiatric conditions often complicate the outcome of patients affected by Parkinson disease (PD), but they differ from classical psychiatric disorders in terms of underlying biological mechanisms, clinical presentation, and treatment response. The purpose of the present review is to illustrate the biological and clinical aspects of psychiatric conditions associated with PD, with particular reference to the differences with respect to classical psychiatric disorders. A careful search of articles on main databases was performed in order to obtain a comprehensive review about the main psychiatric conditions associated with PD. A manual selection of the articles was then performed in order to consider only those articles that concerned with the topic of the review. Psychiatric conditions in patients with PD present substantial differences with respect to classical psychiatric disorders. Their clinical presentation does not align with the symptom profiles represented by Diagnostic and Statistical Manual for Mental Disorders and International Classification of Diseases. Furthermore, psychiatry treatment guidelines are of poor help in managing psychiatric symptoms of patients with PD. Specific diagnostic tools and treatment guidelines are needed to allow early diagnosis and adequate treatment of psychiatric conditions in comorbidity with PD.

Novelty, conditioning and attentional bias to sexual rewards

The Internet provides a large source of novel and rewarding stimuli, particularly with respect to sexually explicit materials. Novelty-seeking and cue-conditioning are fundamental processes underlying preference and approach behaviors implicated in disorders of addiction. Here we examine these processes in individuals with compulsive sexual behaviors (CSB), hypothesizing a greater preference for sexual novelty and stimuli conditioned to sexual rewards relative to healthy volunteers. Twenty-two CSB males and forty age-matched male volunteers were tested in two separate behavioral tasks focusing on preferences for novelty and conditioned stimuli. Twenty subjects from each group were also assessed in a third conditioning and extinction task using functional magnetic resonance imaging. CSB was associated with enhanced novelty preference for sexual, as compared to control images, and a generalized preference for cues conditioned to sexual and monetary versus neutral outcomes compared to healthy volunteers. CSB individuals also had greater dorsal cingulate habituation to repeated sexual versus monetary images with the degree of habituation correlating with enhanced preference for sexual novelty. Approach behaviors to sexually conditioned cues dissociable from novelty preference were associated with an early attentional bias to sexual images. This study shows that CSB individuals have a dysfunctional enhanced preference for sexual novelty possibly mediated by greater cingulate habituation along with a generalized enhancement of conditioning to rewards. We further emphasize a dissociable role for cue-conditioning and novelty preference on the early attentional bias for sexual cues. These findings have wider relevance as the Internet provides a broad range of novel and potentially rewarding stimuli.

Pathophysiology of L-dopa-induced motor and non-motor complications in Parkinson's disease

Involuntary movements, or dyskinesia, represent a debilitating complication of levodopa (L-dopa) therapy for Parkinson's disease (PD). L-dopa-induced dyskinesia (LID) are ultimately experienced by the vast majority of patients. In addition, psychiatric conditions often manifested as compulsive behaviours, are emerging as a serious problem in the management of L-dopa therapy. The present review attempts to provide an overview of our current understanding of dyskinesia and other L-dopa-induced dysfunctions, a field that dramatically evolved in the past twenty years. In view of the extensive literature on LID, there appeared a critical need to re-frame the concepts, to highlight the most suitable models, to review the central nervous system (CNS) circuitry that may be involved, and to propose a pathophysiological framework was timely and necessary. An updated review to clarify our understanding of LID and other L-dopa-related side effects was therefore timely and necessary. This review should help in the development of novel therapeutic strategies aimed at preventing the generation of dyskinetic symptoms.

Theory of choice in bandit, information sampling and foraging tasks

Decision making has been studied with a wide array of tasks. Here we examine the theoretical structure of bandit, information sampling and foraging tasks. These tasks move beyond tasks where the choice in the current trial does not affect future expected rewards. We have modeled these tasks using Markov decision processes (MDPs). MDPs provide a general framework for modeling tasks in which decisions affect the information on which future choices will be made. Under the assumption that agents are maximizing expected rewards, MDPs provide normative solutions. We find that all three classes of tasks pose choices among actions which trade-off immediate and future expected rewards. The tasks drive these trade-offs in unique ways, however. For bandit and information sampling tasks, increasing uncertainty or the time horizon shifts value to actions that pay-off in the future. Correspondingly, decreasing uncertainty increases the relative value of actions that pay-off immediately. For foraging tasks the time-horizon plays the dominant role, as choices do not affect future uncertainty in these tasks.

Patterns of cortical thickness associated with impulse control disorders in Parkinson's disease

Previous functional neuroimaging studies in Parkinson's disease (PD) patients with impulse control disorders (ICDs) demonstrated dysfunction of the reward network, although the extent of anatomical changes is unclear. The aim of this study was to measure brain cortical thickness and subcortical volumes, and to assess their relationship with presence and severity of symptoms, in PD patients with and without ICDs. We studied 110 PD patients (N=58 with ICDs) and 33 healthy controls (all negative for ICDs) who underwent an extensive neurological, neuropsychological, and behavioral assessment as well as structural 1.5 Tesla magnetic resonance imaging (MRI). Between-group differences in brain cortical thickness and subcortical volumes, assessed with the FreeSurfer 5.1 tool, were analyzed. In patients with ICDs, we found significant cortical thinning in fronto-striatal circuitry, specifically in the right superior orbitofrontal, left rostral middle frontal, bilateral caudal middle frontal region, and corpus callosum, as well as volume reduction in the right accumbens and increase in the left amygdala. Finally, we observed a positive association relationship between severity of impulsive symptoms and left rostral middle frontal, inferior parietal, and supramarginal areas. These results support the involvement of both reward and response inhibition networks in PD patients with ICDs. Moreover, their severity is associated with alterations in brain regions linked with reward and top-down control networks. Increased understanding of the mechanisms underlying impulsive and compulsive behaviors might help improve therapeutic strategies for these important disorders.

Linking neuroscience with modern concepts of impulse control disorders in Parkinson's disease

Patients with Parkinson's disease (PD) may experience impulse control disorders (ICDs) when on dopamine agonist therapy for their motor symptoms. In the last few years, a rapid growth of interest for the recognition of these aberrant behaviors and their neurobiological correlates has occurred. Recent advances in neuroimaging are helping to identify the neuroanatomical networks responsible for these ICDs, and together with psychopharmacological assessments are providing new insights into the brain status of impulsive behavior. The genetic associations that may be unique to ICDs in PD are also being identified. Complementing human studies, electrophysiological and biochemical studies in animal models are providing insights into neuropathological mechanisms associated with these disorders. New animal models of ICDs in PD patients are being implemented that should provide critical means to identify efficacious therapies for PD-related motor deficits while avoiding ICD side effects. Here, we provide an overview of these recent advances, with a particular emphasis on the neurobiological correlates reported in animal models and patients along with their genetic underpinnings.

Psychiatric issues in cognitive impairment

Neuropsychiatric symptoms (NPS) such as depression, hallucinations and apathy commonly occur in Parkinson's disease (PD) and have major clinical consequences including a negative impact on quality of life. This review discusses the epidemiology, clinical features, diagnostic procedures and treatment issues of NPS in PD and related disorders in the perspective of cognitive impairment, focusing on depression, anxiety, visual hallucinations, apathy, sleep disturbances, impulse control disorder and non-motor fluctuations. The majority of NPS are more common in PD patients with dementia, possibly related to shared underlying pathologies. Recent studies also suggest that NPS are associated with mild cognitive impairment in PD, in particular with the amnestic type. Accurate diagnosis of NPS is important but can be difficult, due to overlapping symptoms and similar appearance of symptoms of motor symptoms of parkinsonism, cognitive impairment, mood disorders and apathy. There are few systematic studies focusing on the management of NPS in PD with cognitive impairment.

Perceptual decision-making in patients with Parkinson's disease

Impulsive choice and poor information sampling have been found to be key behavioural mechanisms linked to impulse control disorders (ICDs) in Parkinson's disease (PD). Perceptual decision-making is intimately related to information sampling. Therefore, we wanted to determine whether dopaminergic medication or ICDs influence perceptual decision-making in PD. All participants performed two tasks. One was a simple reaction time task, where subjects needed to respond as quickly as possible. The second was a perceptual decision-making task, in which participants had to estimate whether a stimulus contained either more red or more blue pixels. We tested three groups of patients, one treated with levodopa monotherapy, one additionally treated with dopamine agonists, and a third group had ICDs. Results were compared to healthy controls. We found that all patients made more errors than controls. Further, patients with ICDs responded fastest on the reaction time task and also in incorrect trials on the perceptual decision-making task. Similarly, patients with dopamine agonists responded faster than those on levodopa monotherapy and controls. Our results demonstrate that all patients have deficits in perceptual decision-making. However, patients treated with dopamine agonists closely resembled patients with ICDs.

Do Parkinson's Disease Patients Have Deficits in Sequential Sampling Tasks?

The aim of this study was to assess the neuropsychological behavior of Parkinson's disease (PD) patients with addictive behaviors. Characteristically, these patients have younger onset of PD, higher novelty-seeking personality traits, jump to conclusions, and often make irrational choices. We assessed whether PD patients with and without addictive behaviors have deficits in a sequential sampling task, often called the secretary problem. In this task, participants needed to pick the best out of multiple offers. Critically, once participants rejected a deal, this option became unavailable. Thus, decisions needed to be balanced not to stop too soon or sample for too long and miss the best deal. We tested 13 PD patients with and 13 patients without addictive behaviors. Results were compared to healthy volunteers. We found that all patients declined fewer options before committing to a deal. There was, however, no difference between the two patient groups. Furthermore, there was no difference in overall choice rank between patients and controls. These results suggest that, compared to controls, PD patients gather less evidence before committing to an offer, but have no deficits in recognizing the best deal out of many options, regardless of whether or not they have addictive behaviors.

Dopamine modulates novelty seeking behavior during decision making

Novelty seeking refers to the tendency of humans and animals to explore novel and unfamiliar stimuli and environments. The idea that dopamine modulates novelty seeking is supported by evidence that novel stimuli excite dopamine neurons and activate brain regions receiving dopaminergic input. In addition, dopamine is shown to drive exploratory behavior in novel environments. It is not clear whether dopamine promotes novelty seeking when it is framed as the decision to explore novel options versus the exploitation of familiar options. To test this hypothesis, we administered systemic injections of saline or GBR-12909, a selective dopamine transporter (DAT) inhibitor, to monkeys and assessed their novelty seeking behavior during a probabilistic decision making task. The task involved pseudorandom introductions of novel choice options. This allowed monkeys the opportunity to explore novel options or to exploit familiar options that they had already sampled. We found that DAT blockade increased the monkeys' preference for novel options. A reinforcement learning (RL) model fit to the monkeys' choice data showed that increased novelty seeking after DAT blockade was driven by an increase in the initial value the monkeys assigned to novel options. However, blocking DAT did not modulate the rate at which the monkeys learned which cues were most predictive of reward or their tendency to exploit that knowledge. These data demonstrate that dopamine enhances novelty-driven value and imply that excessive novelty seeking-characteristic of impulsivity and behavioral addictions-might be caused by increases in dopamine, stemming from less reuptake.

Multiple modes of impulsivity in Parkinson's disease

Cognitive problems are a major factor determining quality of life of patients with Parkinson's disease. These include deficits in inhibitory control, ranging from subclinical alterations in decision-making to severe impulse control disorders. Based on preclinical studies, we proposed that Parkinson's disease does not cause a unified disorder of inhibitory control, but rather a set of impulsivity factors with distinct psychological profiles, anatomy and pharmacology. We assessed a broad set of measures of the cognitive, behavioural and temperamental/trait aspects of impulsivity. Sixty adults, including 30 idiopathic Parkinson's disease patients (Hoehn and Yahr stage I–III) and 30 healthy controls, completed a neuropsychological battery, objective behavioural measures and self-report questionnaires. Univariate analyses of variance confirmed group differences in nine out of eleven metrics. We then used factor analysis (principal components method) to identify the structure of impulsivity in Parkinson's disease. Four principal factors were identified, consistent with four different mechanisms of impulsivity, explaining 60% of variance. The factors were related to (1) tests of response conflict, interference and self assessment of impulsive behaviours on the Barrett Impulsivity Scale, (2) tests of motor inhibitory control, and the self-report behavioural approach system, (3) time estimation and delay aversion, and (4) reflection in hypothetical scenarios including temporal discounting. The different test profiles of these four factors were consistent with human and comparative studies of the pharmacology and functional anatomy of impulsivity. Relationships between each factor and clinical and demographic features were examined by regression against factor loadings. Levodopa dose equivalent was associated only with factors (2) and (3). The results confirm that impulsivity is common in Parkinson's disease, even in the absence of impulse control disorders, and that it is not a unitary phenomenon. A better understanding of the structure of impulsivity in Parkinson's disease will support more evidence-based and effective strategies to treat impulsivity.

Impulsive and compulsive behaviors in Parkinson's disease

Impulsive-compulsive behaviors (ICBs) in Parkinson's disease (PD) are a common and devastating side effect of dopamine replacement therapy. In this review we describe the phenomenology, prevalence, and risk factors of patients with PD. Results of behavioral studies assessing the neuropsychological profile of patients with PD emphasize that the ICBs, which are behavioral addictions, are not hedonically motivated. Rather, other factors such as the inability to cope with uncertainty may be triggering ICBs. New insights from functional imaging studies, strengthening the incentive salience hypothesis, are discussed, and therapeutic guidelines for the management of ICBs in PD are given.

The detection of novelty relies on dopaminergic signaling: evidence from apomorphine's impact on the novelty N2

Despite much research, it remains unclear if dopamine is directly involved in novelty detection or plays a role in orchestrating the subsequent cognitive response. This ambiguity stems in part from a reliance on experimental designs where novelty is manipulated and dopaminergic activity is subsequently observed. Here we adopt the alternative approach: we manipulate dopamine activity using apomorphine (D1/D2 agonist) and measure the change in neurological indices of novelty processing. In separate drug and placebo sessions, participants completed a von Restorff task. Apomorphine speeded and potentiated the novelty-elicited N2, an Event-Related Potential (ERP) component thought to index early aspects of novelty detection, and caused novel-font words to be better recalled. Apomorphine also decreased the amplitude of the novelty-P3a. An increase in D1/D2 receptor activation thus appears to potentiate neural sensitivity to novel stimuli, causing this content to be better encoded.

Impulsivity and apathy in Parkinson's disease

Impulse control disorders (ICDs) and apathy are recognized as two important neuropsychiatric syndromes associated with Parkinson's disease (PD), but as yet we understand very little about the cognitive mechanisms underlying them. Here, we review emerging findings, from both human and animal studies, that suggest that impulsivity and apathy are opposite extremes of a dopamine-dependent spectrum of motivated decision making. We first argue that there is strong support for a hypodopaminergic state in PD patients with apathy, as well as for an association between dopamine therapy and development of ICDs. However, there is little evidence for a clear dose-response relationship, and great heterogeneity of findings. We argue that dopaminergic state on its own is an insufficient explanation, and suggest instead that there is now substantial evidence that both apathy and impulsivity are in fact multi-dimensional syndromes, with separate, dissociable mechanisms underlying their 'surface' manifestations. Some of these mechanisms might be dopamine-dependent. According to this view, individuals diagnosed as impulsive or apathetic may have very different mechanisms underlying their clinical states. We propose that impulsivity and apathy can arise from dissociable deficits in option generation, option selection, action initiation or inhibition and learning. Review of the behavioural and neurobiological evidence leads us to a new conceptual framework that might help understand the variety of functional deficits seen in PD.

Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: evidence from Parkinson's disease

A subset of patients with Parkinson's disease (PD) develops behavioral addictions, which may be due to their dopamine replacement therapy. Recently, several groups have been comparing PD patients with and without behavioral addictions on tasks that are thought to measure aspects of impulsivity. Several of these experiments, including information sampling, a bias toward novel stimuli and temporal discounting, have shown differences between PD patients with and without behavioral addictions. We have developed a unifying theoretical framework that allows us to model behavior in all three of these tasks. By exploring the performance of the patient groups on the three tasks with a single framework, we can ask questions about common mechanisms that underlie all three. Our results suggest that the effects seen in all three tasks can be accounted for by uncertainty about the ability to map future actions into rewards. More specifically, the modeling is consistent with the hypothesis that the group with behavioral addictions behaves as if they cannot use information provided within the experimental context to improve future reward guided actions. Future studies will be necessary to more firmly establish (or refute) this hypothesis. We discuss this result in light of what is known about the pathology that underlies the behavioral addictions in the PD patients.

Action selection and action value in frontal-striatal circuits

The role that frontal-striatal circuits play in normal behavior remains unclear. Two of the leading hypotheses suggest that these circuits are important for action selection or reinforcement learning. To examine these hypotheses, we carried out an experiment in which monkeys had to select actions in two different task conditions. In the first (random) condition, actions were selected on the basis of perceptual inference. In the second (fixed) condition, the animals used reinforcement from previous trials to select actions. Examination of neural activity showed that the representation of the selected action was stronger in lateral prefrontal cortex (lPFC), and occurred earlier in the lPFC than it did in the dorsal striatum (dSTR). In contrast to this, the representation of action values, in both the random and fixed conditions, was stronger in the dSTR. Thus, the dSTR contains an enriched representation of action value, but it followed frontal cortex in action selection.

Effects of dopamine on sensitivity to social bias in Parkinson's disease

Patients with Parkinson's disease (PD) sometimes develop impulsive compulsive behaviours (ICBs) due to their dopaminergic medication. We compared 26 impulsive and 27 non-impulsive patients with PD, both on and off medication, on a task that examined emotion bias in decision making. No group differences were detected, but patients on medication were less biased by emotions than patients off medication and the strongest effects were seen in patients with ICBs. PD patients with ICBs on medication also showed more learning from negative feedback and less from positive feedback, whereas off medication they showed the opposite effect.

Pathological gambling in Parkinson's disease--a review of the literature

The prevalence of pathological gambling is 3.4% to 6% in treated Parkinson's disease, which is higher than the background population rate. In this review we discuss current evidence to indicate that dopamine agonists are much more likely to trigger this behavior than either L-dopa or selective monoamine oxidase B inhibitor monotherapy. New insights from recent behavioral and functional imaging studies and possible treatment approaches are also covered. A PubMed literature search using the terms "gambling" and "Parkinson's disease," "impulse control disorder," "impulsive compulsive behaviour," "dopamine agonist," of individual dopamine agonists, and of ongoing drug trials, using http://www.clinicaltrials.gov, was carried out for the period up to January 2011.