Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients

Mouse Model of Parkinson's Disease with Bilateral Dorsal Striatum Lesion with 6-Hydroxydopamine Exhibits Cognitive Apathy-like Behavior

Among the symptoms of Parkinson's disease (PD), apathy comprises a set of behavioral, affective, and cognitive features that can be classified into several subtypes. However, the pathophysiology and brain regions that are involved in these different apathy subtypes are still poorly characterized. We examined which subtype of apathy is elicited in a mouse model of PD with 6-hydroxydopamine (6-OHDA) lesions and the behavioral symptoms that are exhibited. Male C57/BL6J mice were allocated to sham (n = 8) and 6-OHDA (n = 13) groups and locally injected with saline or 4 µg 6-OHDA bilaterally in the dorsal striatum. We then conducted motor performance tests and apathy-related behavioral experiments. We then pathologically evaluated tyrosine hydroxylase (TH) immunostaining. The 6-OHDA group exhibited significant impairments in motor function. In the behavioral tests of apathy, significant differences were observed between the sham and 6-OHDA groups in the hole-board test and novelty-suppressed feeding test. The 6-OHDA group exhibited impairments in inanimate novel object preference, whereas social preference was maintained in the three-chamber test. The number of TH+ pixels in the caudate putamen and substantia nigra compacta was significantly reduced in the 6-OHDA group. The present mouse model of PD predominantly showed dorsal striatum dopaminergic neuronal loss and a decrease in novelty seeking as a symptom that is related to the cognitive apathy component.

Reward and Punishment Learning as Predictors of Cognitive Behavioral Therapy Response in Parkinson's Disease Comorbid with Clinical Depression

Depression is highly comorbid among individuals with Parkinson's Disease (PD), who often experience unique challenges to accessing and benefitting from empirically supported interventions like Cognitive Behavioral Therapy (CBT). Given the role of reward processing in both depression and PD, this study analyzed a subset (N = 25) of participants who participated in a pilot telemedicine intervention of PD-informed CBT, and also completed a Reward- and Punishment-Learning Task (RPLT) at baseline. At the conclusion of CBT, participants were categorized into treatment responders (n = 14) and non-responders (n = 11). Responders learned more optimally from negative rather than positive feedback on the RPLT, while this pattern was reversed in non-responders. Computational modeling suggested group differences in learning rate to negative feedback may drive the observed differences. Overall, the results suggest that a within-subject bias for punishment-based learning might help to predict response to CBT intervention for depression in those with PD.Plain Language Summary Performance on a Computerized Task may predict which Parkinson's Disease Patients benefit from Cognitive Behavioral Treatment of Clinical DepressionWhy was the study done? Clinical depression regularly arises in individuals with Parkinson's Disease (PD) due to the neurobiological changes with the onset and progression of the disease as well as the unique psychosocial difficulties associated with living with a chronic condition. Nonetheless, psychiatric disorders among individuals with PD are often underdiagnosed and likewise undertreated for a variety of reasons. The results of our study have implications about how to improve the accuracy and specificity of mental health treatment recommendations in the future to maximize benefits for individuals with PD, who often face additional barriers to accessing quality mental health treatment.What did the researchers do? We explored whether performance on a computerized task called the Reward- and Punishment-Learning Task (RPLT) helped to predict response to Cognitive Behavioral Therapy (CBT) for depression better than other predictors identified in previous studies. Twenty-five individuals with PD and clinical depression that completed a 10-week telehealth CBT program were assessed for: Demographics (Age, gender, etc.); Clinical information (PD duration, mental health diagnoses, levels of anxiety/depression, etc.); Neurocognitive performance (Memory, processing speed, impulse control, etc.); and RPLT performance.What did the researchers find? A total of 14 participants significantly benefitted from CBT treatment while 11 did not significantly benefit from treatment.There were no differences before treatment in the demographics, clinical information, and neurocognitive performance of those participants who ended up benefitting from the treatment versus those who did not.There were, however, differences before treatment in RPLT performance so that those individuals that benefitted from CBT seemed to learn better from negative feedback.What do the findings mean? Our results suggest that the CBT program benefitted those PD patients with clinical depression that seemed to overall learn best from avoiding punishment rather than obtaining reward which was targeted in CBT by focusing on increasing engagement in rewarding activities. The Reward- and Punishment-Learning Task hence may be a useful tool to help predict treatment response and provide more individualized recommendations on how to best maximize the benefits of psychotherapy for individuals with PD that may struggle to connect to mental health care. Caution is recommended about interpretating these results beyond this study as the overall number of participants was small and the data for this study were collected as part of a previous study so there was no opportunity to include additional measurements of interest.

The effects of pramipexole on motivational vigour during a saccade task: a placebo-controlled study in healthy adults

Motivation allows us to energise actions when we expect reward and is reduced in depression. This effect, termed motivational vigour, has been proposed to rely on central dopamine, with dopaminergic agents showing promise in the treatment of depression. This suggests that dopaminergic agents might act to reduce depression by increasing the effects of reward or by helping energise actions. The aim of the current study was to investigate whether the dopamine agonist pramipexole enhanced motivational vigour during a rewarded saccade task. In addition, we asked whether the effects of pramipexole on vigour differ between reward contingent on performance and guaranteed reward. Healthy adult participants were randomised to receive either pramipexole (n = 19) or placebo (controls n = 18) for 18 days. The vigour of saccades was measured twice, once before the administration of study medication (Time 1) and after taking it for 12-15 days (Time 2). To separate motivation by contingency vs. reward, saccadic vigour was separately measured when (1) rewards were contingent on performance (2) delivered randomly with matched frequency, (3) when reward was guaranteed, (4) when reward was not present at all. Motivation increased response vigour, as expected. Relative to placebo, pramipexole also increased response vigour. However, there was no interaction, meaning that the effects of reward were not modulated by drug, and there was no differential drug effect on contingent vs. guaranteed rewards. The effect of pramipexole on vigour could not be explained by a speed/accuracy trade-off, nor by autonomic arousal as indexed by pupillary dilation. Chronic D2 stimulation increases general vigour, energising movements in healthy adults irrespective of extrinsic reward.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

Impulse Control Disorders in Parkinson's Disease: An Overview of Risk Factors, Pathogenesis and Pharmacological Management

Impulse control disorders in Parkinson's disease are relatively common drug-induced addictive behaviours that are usually triggered by the dopamine agonists pramipexole, ropinirole and rotigotine. This narrative review aimed to provide a comprehensive overview of the current knowledge of impulse control disorders in Parkinson's disease. We summarised the prevalence, clinical features, risk factors and potential underlying mechanisms of impulse control disorders in Parkinson's disease. Moreover, recent advances in behavioural and imaging characteristics and management strategies are discussed. Early detection as well as a tailored multidisciplinary approach, which typically includes careful adjustment of the dopaminergic therapy and the treatment of associated neuropsychiatric symptoms, are necessary. In some cases, a continuous delivery of levodopa via a pump or the dopamine D1 receptor agonist, apomorphine, can be considered. In selected patients without cognitive or speech impairment, deep brain stimulation of the subthalamic nucleus can also improve addictions. Finding the right balance of tapering dopaminergic dose (usually dopamine agonists) without worsening motor symptoms is essential for a beneficial long-term outcome.

Transitioning from Subtyping to Precision Medicine in Parkinson's Disease: A Purpose-Driven Approach

The International Parkinson and Movement Disorder Society (MDS) created a task force (TF) to provide a critical overview of the Parkinson's disease (PD) subtyping field and develop a guidance on future research in PD subtypes. Based on a literature review, we previously concluded that PD subtyping requires an ultimate alignment with principles of precision medicine, and consequently novel approaches were needed to describe heterogeneity at the individual patient level. In this manuscript, we present a novel purpose-driven framework for subtype research as a guidance to clinicians and researchers when proposing to develop, evaluate, or use PD subtypes. Using a formal consensus methodology, we determined that the key purposes of PD subtyping are: (1) to predict disease progression, for both the development of therapies (use in clinical trials) and prognosis counseling, (2) to predict response to treatments, and (3) to identify therapeutic targets for disease modification. For each purpose, we describe the desired product and the research required for its development. Given the current state of knowledge and data resources, we see purpose-driven subtyping as a pragmatic and necessary step on the way to precision medicine. © 2024 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Pramipexole Enhances Reward Learning by Preserving Value Estimates

BACKGROUND

Dopamine D2-like agonists show promise as treatments for depression. They are thought to act by enhancing reward learning; however, the mechanisms by which they achieve this are not clear. Reinforcement learning accounts describe 3 distinct candidate mechanisms: increased reward sensitivity, increased inverse decision-temperature, and decreased value decay. As these mechanisms produce equivalent effects on behavior, arbitrating between them requires measurement of how expectations and prediction errors are altered. We characterized the effects of 2 weeks of the D2-like agonist pramipexole on reward learning and used functional magnetic resonance imaging measures of expectation and prediction error to assess which of these 3 mechanistic processes were responsible for the behavioral effects.

METHODS

Forty healthy volunteers (50% female) were randomized to 2 weeks of pramipexole (titrated to 1 mg/day) or placebo in a double-blind, between-subject design. Participants completed a probabilistic instrumental learning task before and after the pharmacological intervention, with functional magnetic resonance imaging data collected at the second visit. Asymptotic choice accuracy and a reinforcement learning model were used to assess reward learning.

RESULTS

Pramipexole increased choice accuracy in the reward condition with no effect on losses. Participants who received pramipexole had increased blood oxygen level-dependent response in the orbital frontal cortex during the expectation of win trials but decreased blood oxygen level-dependent response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.

CONCLUSIONS

The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible mechanism for pramipexole's antidepressant effect.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Pramipexole for the Treatment of Depression: Efficacy and Mechanisms

Dopaminergic mechanisms are a plausible treatment target for patients with clinical depression but are relatively underexplored in conventional antidepressant medications. There is continuing interest in the potential antidepressant effects of the dopamine receptor agonist, pramipexole, with data from both case series and controlled trials indicating that this agent may produce benefit for patients with difficult-to-treat depression. Pramipexole's therapeutic utility in depression is likely to be expressed through alterations in reward mechanisms which are strongly influenced by dopamine pathways and are known to function abnormally in depressed patients. Our work in healthy participants using brain imaging in conjunction with computational modelling suggests that repeated pramipexole facilitates reward learning by inhibiting value decay. This mechanism needs to be confirmed in larger clinical trials in depressed patients. Such studies will also allow assessment of whether baseline performance in reward learning in depression predicts therapeutic response to pramipexole treatment.

Jumping to conclusions bias, psychosis and impulsivity in early stages of Parkinson's disease

OBJECTIVES

The aim was to explore the correlations between Jumping to Conclusions (JtC) tendency and neuropsychiatric features in patients with early Parkinson's disease (PD).

BACKGROUND

According to few reports, PD patients with impulsive-compulsive behaviors (ICBs) are prone to working memory difficulties including JtC bias. The correlation of psychotic features and JtC tendency remains still unclear.

METHODS

Healthy controls and patients within 3 years of PD onset were recruited. Participants were examined for psychotic symptoms using a 10 question PD-specific psychosis severity scale. JtC was measured by a probalistic reasoning scenario (beads task). In PD group, medication use, motor and non-motor symptoms were documented. Impulsivity was evaluated using the Questionnaire for Impulsive-Compulsive Disorders in PD (QUIP).

RESULTS

The prevalence of JtC bias was 9% (6/70) in healthy individuals, compared to 32% (22/68) of PD group [p = 0.001]. No association was detected between the presence of JtC tendency and PD-associated psychosis (p = 0.216). Patients with JtC had shorter duration of PD, more tremor-dominant PD subtype and higher QUIP scores, regardless of the dopaminergic therapy (p = 0.043, p = 0.015, p = 0.007, respectively). A trend towards attention and inhibition control deficit was noticed in JtC patients.

CONCLUSIONS

We found a high prevalence of JtC bias in early, cognitively intact PD population and a potential link between subthreshold ICBs and poor performance on beads task. Additional studies are needed to confirm our results and elaborate on the mechanisms that correlate impulsivity with JtC tendency, which are likely to be different from those mediating psychotic features in early PD.

Impulse control disorder in Parkinson's disease is associated with abnormal frontal value signalling

Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson's disease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the mechanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson's disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disorders and depression. One hundred and ninety-nine patients with Parkinson's disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain signalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine's effects on reinforcement learning in Parkinson's disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of value in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article.

Apathy in Parkinson's Disease: Clinical Patterns and Neurobiological Basis

Apathy is commonly defined as a loss of motivation leading to a reduction in goal-directed behaviors. This multidimensional syndrome, which includes cognitive, emotional and behavioral components, is one of the most prevalent neuropsychiatric features of Parkinson's disease (PD). It has been established that the prevalence of apathy increases as PD progresses. However, the pathophysiology and anatomic substrate of this syndrome remain unclear. Apathy seems to be underpinned by impaired anatomical structures that link the prefrontal cortex with the limbic system. It can be encountered in the prodromal stage of the disease and in fluctuating PD patients receiving bilateral chronic subthalamic nucleus stimulation. In these stages, apathy may be considered as a disorder of motivation that embodies amotivational behavioral syndrome, is underpinned by combined dopaminergic and serotonergic denervation and is dopa-responsive. In contrast, in advanced PD patients, apathy may be considered as cognitive apathy that announces cognitive decline and PD dementia, is underpinned by diffuse neurotransmitter system dysfunction and Lewy pathology spreading and is no longer dopa-responsive. In this review, we discuss the clinical patterns of apathy and their treatment, the neurobiological basis of apathy, the potential role of the anatomical structures involved and the pathways in motivational and cognitive apathy.

Weaker situations: Uncertainty reveals individual differences in learning: Implications for PTSD

Few individuals who experience trauma develop posttraumatic stress disorder (PTSD). Therefore, the identification of individual differences that signal increased risk for PTSD is important. Lissek et al. (2006) proposed using a weak rather than a strong situation to identify individual differences. A weak situation involves less-salient cues as well as some degree of uncertainty, which reveal individual differences. A strong situation involves salient cues with little uncertainty, which produce consistently strong responses. Results from fear conditioning studies that support this hypothesis are discussed briefly. This review focuses on recent findings from three learning tasks: classical eyeblink conditioning, avoidance learning, and a computer-based task. These tasks are interpreted as weaker learning situations in that they involve some degree of uncertainty. Individual differences in learning based on behavioral inhibition, which is a risk factor for PTSD, are explored. Specifically, behaviorally inhibited individuals and rodents (i.e., Wistar Kyoto rats), as well as individuals expressing PTSD symptoms, exhibit enhanced eyeblink conditioning. Behaviorally inhibited rodents also demonstrate enhanced avoidance responding (i.e., lever pressing). Both enhanced eyeblink conditioning and avoidance are most evident with schedules of partial reinforcement. Behaviorally inhibited individuals also performed better on reward and punishment trials than noninhibited controls in a probabilistic category learning task. Overall, the use of weaker situations with uncertain relationships may be more ecologically valid than learning tasks in which the aversive event occurs on every trial and may provide more sensitivity for identifying individual differences in learning for those at risk for, or expressing, PTSD symptoms.

The impact of ADHD symptom severity on reinforcement and punishment learning among adults

Introduction: Aberrations in feedback learning are hypothesised to contribute to the behavioural disruptions and impairment of attention-deficit/hyperactivity disorder (ADHD). However, few studies have evaluated the relation of reward/punishment feedback and ADHD symptom severity on learning. The current study evaluates the differential effects of reward and punishment feedback on learning among adults with elevated ADHD. Methods: One hundred five participants self-reported their level of current ADHD symptoms and completed an innovative instrumental learning task. Results: Consistent with predictions, participants with low self-reported ADHD symptom severity benefitted equally from reward and punishment feedback during the learning task, whereas participants with high self-reported symptom severity performed better (indexed by accuracy on learning task) from reward than punishment feedback trials. Conclusions: Overall, adults with high self-reported symptom severity of ADHD learned more from reward-based feedback, which provides critical implications for motivational theories about ADHD, as well as for treatment protocols. Future work should examine the translatability of results within a treatment setting.

Prefrontal transcranial magnetic stimulation boosts response vigour during reinforcement learning in healthy adults

A 10-Hz repetitive transcranial magnetic stimulation to the left dorsal lateral prefrontal cortex has been shown to increase dopaminergic activity in the dorsal striatum, a region strongly implicated in reinforcement learning. However, the behavioural influence of this effect remains largely unknown. We tested the causal effects of 10-Hz stimulation on behavioural and computational characteristics of reinforcement learning. A total of 40 healthy individuals were randomized into active and sham (placebo) stimulation groups. Each participant underwent one stimulation session (1500 pulses) in which stimulation was applied over the left dorsal lateral prefrontal cortex using a robotic arm. Participants then completed a reinforcement learning task sensitive to striatal dopamine functioning. Participants' choices were modelled using a reinforcement learning model (Q-learning) that calculates separate learning rates associated with positive and negative reward prediction errors. Subjects receiving active stimulation exhibited increased reward rate (number of correct responses per second of task activity) compared with those in sham. Computationally, although no group differences were observed, the active group displayed a higher learning rate for correct trials (αG) compared with incorrect trials (αL). Finally, when tested with novel pairs of stimuli, the active group displayed extremely fast reaction times, and a trend towards a higher reward rate. This study provided specific behavioural and computational accounts of altered striatal-mediated behaviour, particularly response vigour, induced by a proposed increase of dopamine activity by 10-Hz stimulation to the left dorsal lateral prefrontal cortex. Together, these findings bolster the use of repetitive transcranial magnetic stimulation to target neurocognitive disturbances attributed to the dysregulation of dopaminergic-striatal circuits.

Neural correlates of impulsive compulsive behaviors in Parkinson's disease: A Japanese retrospective study

BACKGROUND

Impulsive compulsive behaviors (ICBs) often disturb patients with Parkinson's Disease (PD), of which impulse control disorder (ICD) and dopamine dysregulation syndrome (DDS) are two major subsets. The nucleus accumbens (NAcc) is involved in ICB; however, it remains unclear how the NAcc affects cortical function and defines the different behavioral characteristics of ICD and DDS.

OBJECTIVES

To identify the cortico-striatal network primarily involved in ICB and the differences in these networks between patients with ICD and DDS using structural and resting-state functional magnetic resonance imaging.

METHODS

Patients with PD were recruited using data from a previous cohort study and divided into those with ICB (ICB group) and without ICB (non-ICB group) using the Japanese version of the Questionnaire for Impulsive Compulsive Disorders in Parkinson's Disease (J-QUIP). From these two groups, we extracted 37 pairs matched for age, sex, disease duration, and levodopa equivalent daily dose of dopamine agonists. Patients with ICB were further classified as having ICD or DDS based on the J-QUIP subscore. General linear models were used to compare gray matter volume and functional connectivity (FC) of the NAcc, caudate, and putamen between the ICB and non-ICB groups and between patients with ICD and those with DDS.

RESULTS

We found no significant differences in gray matter volumebetween the ICB and non-ICB groups or between patients with ICD and those with DDS. Compared with the non-ICB group, the FC of the right NAcc in the ICB group was lower in the bilateral ventromedial prefrontal cortex and higher in the left middle occipital gyrus. Furthermore, patients with DDS showed higher FC between the right putamen and left superior temporal gyrus and higher FC between the left caudate and bilateral middle occipital gyrus than patients with ICD. In contrast, patients with ICD exhibited higher FC between the left NAcc and the right posterior cingulate cortex than patients with DDS.

CONCLUSIONS

The functionally altered network between the right NAcc and ventromedial prefrontal cortex was associated with ICB in PD. In addition, the surrounding cortico-striatal networks may differentiate the behavioral characteristics of patients with ICD and those with DDS.

Altered Reinforcement Learning from Reward and Punishment in Anorexia Nervosa: Evidence from Computational Modeling

OBJECTIVES

Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior.

METHODS

This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge.

RESULTS

AN had lower learning rates than HC following both positive and negative PE (p < .02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p < .001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome.

CONCLUSIONS

This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

Defining the unknowns for cell therapies in Parkinson's disease

First-in-human clinical trials have commenced to test the safety and efficacy of cell therapies for people with Parkinson's disease (PD). Proof of concept that this neural repair strategy is efficacious is based on decades of preclinical studies and clinical trials using primary foetal cells, as well as a significant literature exploring more novel stem cell-derived products. Although several measures of efficacy have been explored, including the successful in vitro differentiation of stem cells to dopamine neurons and consistent alleviation of motor dysfunction in rodent models, many unknowns still remain regarding the long-term clinical implications of this treatment strategy. Here, we consider some of these outstanding questions, including our understanding of the interaction between anti-Parkinsonian medication and the neural transplant, the impact of the cell therapy on cognitive or neuropsychiatric symptoms of PD, the role of neuroinflammation in the therapeutic process and the development of graft-induced dyskinesias. We identify questions that are currently pertinent to the field that require further exploration, and pave the way for a more holistic understanding of this neural repair strategy for treatment of PD.

Contribution of the sensorimotor beta oscillations and the cortico-basal ganglia-thalamic circuitry during value-based decision making: A simultaneous EEG-fMRI investigation

In decision neuroscience, the motor system has primarily been considered to be involved in executing choice actions. However, a competing perspective suggests its engagement in the evaluation of options, traditionally considered to be performed by the brain's valuation system. Here, we investigate the role of the motor system in value-based decision making by determining the neural circuitries associated with the sensorimotor beta oscillations previously identified to encode decision options. In a simultaneous EEG-fMRI study, participants evaluated reward and risk associated with a forthcoming action. A significant sensorimotor beta desynchronization was identified prior to and independent of response. The level of beta desynchronization showed evidence of encoding the reward levels. This beta desynchronization covaried, on a trial-by-trial level, with BOLD activity in the cortico-basal ganglia-thalamic circuitry. In contrast, there was only a weak covariation within the valuation network, despite significant modulation of its BOLD activity by reward levels. These results suggest that the way in which decision variables are processed differs in the valuation network and in the cortico-basal ganglia-thalamic circuitry. We propose that sensorimotor beta oscillations indicate incentive motivational drive towards a choice action computed from the decision variables even prior to making a response, and it arises from the cortico-basal ganglia-thalamic circuitry.

Serotonin modulates asymmetric learning from reward and punishment in healthy human volunteers

Instrumental learning is driven by a history of outcome success and failure. Here, we examined the impact of serotonin on learning from positive and negative outcomes. Healthy human volunteers were assessed twice, once after acute (single-dose), and once after prolonged (week-long) daily administration of the SSRI citalopram or placebo. Using computational modelling, we show that prolonged boosting of serotonin enhances learning from punishment and reduces learning from reward. This valence-dependent learning asymmetry increases subjects’ tendency to avoid actions as a function of cumulative failure without leading to detrimental, or advantageous, outcomes. By contrast, no significant modulation of learning was observed following acute SSRI administration. However, differences between the effects of acute and prolonged administration were not significant. Overall, these findings may help explain how serotonergic agents impact on mood disorders.

Two factors can drive learning: punishment of failures and reward of successes. Serotonin induces a valence-dependent learning asymmetry, as revealed by prolonged administering of SSRIs to healthy participants in a gambling task.

Early Parkinson's Disease Phenotypes Tailored by Personality, Behavior, and Motor Symptoms

BACKGROUND

Previous studies described a parkinsonian personality characterized as rigid, introverted, and cautious; however, little is known about personality traits in de novo Parkinson's disease (PD) patients and their relationships with motor and neuropsychiatric symptoms.

OBJECTIVE

To investigate personality in de novo PD and explore its relationship with PD symptoms.

METHODS

Using Cloninger's biosocial model, we assessed personality in 193 de novo PD patients. Motor and non-motor symptoms were measured using several validated scales. Cluster analysis was conducted to investigate the interrelationship of personality traits, motor, and non-motor symptoms.

RESULTS

PD patients showed low novelty seeking, high harm avoidance, and normal reward dependence and persistence scores. Harm avoidance was positively correlated with the severity of depression, anxiety, and apathy (rs = [0.435, 0.676], p < 0.001) and negatively correlated with quality of life (rs = -0.492, p < 0.001). Novelty seeking, reward dependence, and persistence were negatively correlated with apathy (rs = [-0.274, -0.375], p < 0.001). Classification of patients according to personality and PD symptoms revealed 3 distinct clusters: i) neuropsychiatric phenotype (with high harm avoidance and low novelty seeking, hypodopaminergic neuropsychiatric symptoms and higher impulsivity), ii) motor phenotype (with low novelty seeking and higher motor severity), iii) benign phenotype (with low harm avoidance and high novelty seeking, reward dependence, and persistence traits clustered with lower symptoms severity and low impulsivity).

CONCLUSION

Personality in early PD patients allows us to recognize 3 patients' phenotypes. Identification of such subgroups may help to better understand their natural history. Their longitudinal follow-up will allow confirming whether some personality features might influence disease evolution and treatment.

Decreased sensitivity to loss of options in patients with Parkinson's disease

Humans prefer to have many options when making decisions. When there is a threat of options disappearing, humans invest more to keep these options available, indicating that they are sensitive to the loss of options. This study examined whether patients with Parkinson's disease (PD), a disease characterized by dopamine depletion, try to keep options available when options are disappearing. Twenty-seven PD patients without dementia and 27 healthy controls (HCs) performed the door game, in which participants were presented with multiple alternatives in the form of three doors, each associated with a different point distribution. The participants were asked to maximize their point earnings by finding the best door. The task included two conditions. In the shutter condition, shutters gradually closed on doors that were not chosen; once the shutters completely closed, the door was no longer available. There were no shutters in the control condition. The results revealed that the HCs switched doors more often in the shutter condition than in the control condition, indicating a tendency to keep options available. However, the PD patients did not show such differences between the two conditions. The difference in the number of switches between the shutter and control conditions in the PD patients was significantly positively correlated with the distribution of dopamine transporters in the left striatum, as measured by ¹²³I-ioflupane-SPECT (DaTSCAN) images. These results suggest that PD patients are less sensitive to the loss of options, and this decreased sensitivity may be caused by a decline in dopaminergic neurotransmission.

Dopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients

Engaging in demanding mental activities requires the allocation of cognitive control, which can be effortful and aversive. Individuals thus tend to avoid exerting cognitive effort if less demanding behavioral options are available. Recent accounts propose a key role for dopamine in motivating behavior by increasing the sensitivity to rewards associated with effort exertion. Whether dopamine additionally plays a specific role in modulating the sensitivity to the costs of cognitive effort, even in the absence of any incentives, is much less clear. To address this question, we assessed cognitive effort avoidance in patients (n = 38) with Parkinson's disease, a condition characterized by loss of midbrain dopaminergic neurons, both ON and OFF dopaminergic medication and compared them to healthy controls (n = 24). Effort avoidance was assessed using the Demand Selection Task (DST), in which participants could freely choose between performing a high-demand or a low-demand version of a task-switching paradigm. Critically, participants were not offered any incentives to choose the more effortful option, nor for good performance. While healthy controls and patients OFF their dopaminergic medications consistently preferred the low-demand option, effort avoidance in patients ON dopaminergic medications was reduced compared to patients OFF, a difference which seems to lessen over trials. These differences in preference could not be explained by altered task-switching performance. Although patients ON were less accurate at detecting the different effort levels, as measured during instructed forced-choice blocks, their detection ability was not associated with effort avoidance, unlike in the healthy controls and the patients OFF. Our findings provide evidence that dopamine replacement in Parkinson's patients increases the willingness to engage in cognitively demanding behavior, and that this cannot be explained by possible effects of dopamine replacement on performance nor on the ability to detect effort demands. These results suggest that dopamine plays a role in reducing the sensitivity to effort costs that is independent of its role in enhancing the sensitivity to the benefits of effort exertion.

The effects of acute oral naltrexone pretreatment on the abuse potential of intranasal methamphetamine, and the relationship between reward/punishment sensitivity and methamphetamine's effects

One potential medication for treating methamphetamine use disorder is the opioid antagonist naltrexone (NLTX). Despite encouraging preclinical findings, the results of clinical studies have been mixed. The primary aim of the current trial was to examine the effects of acute NLTX pretreatment on the subjective and reinforcing effects of intranasal methamphetamine. Nonmedical psychostimulant users completed outpatient testing sessions in which they received oral placebo (0 mg) or NLTX (50 mg) before intranasal methamphetamine (30 mg/70 kg). Primary outcome measures were peak positive subjective effects (e.g. drug 'Liking') assessed on a visual analog scale (0-100), and methamphetamine self-administration using an operant self-administration task. Participants also completed a probabilistic categorization task to assess reward and punishment learning sensitivity. Complete data were available from 13 male and 1 transgender (male-to-female) participant (age: 33.4 ± 7.6 years). Intranasal methamphetamine significantly increased subjective ratings of drug 'Liking', 'Good Effect' and 'High' from baseline (P's < 0.01), but did not significantly vary as a function of placebo or NLTX pretreatment. Similarly, methamphetamine self-administration did not vary between the placebo and NLTX pretreatment conditions. This sample did not demonstrate a significant 'bias' in learning from positive and negative outcomes (i.e. reward and punishment sensitivity), and reward/punishment sensitivity was not correlated with the effects of methamphetamine or the effects of NLTX on methamphetamine. The current study argues against the use of NLTX as a stand-alone medication for treating methamphetamine use disorder.

Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) is commonly treated with dopaminergic medication, which enhances some, while impairing other cognitive functions. It can even contribute to impulse control disorder and addiction. We describe the history of research supporting the dopamine overdose hypothesis, which accounts for the large within-patient variability in dopaminergic medication effects across different tasks by referring to the spatially non-uniform pattern of dopamine depletion in dorsal versus ventral striatum. However, there is tremendous variability in dopaminergic medication effects not just within patients across distinct tasks, but also across different patients. In the second part of this chapter we review recent studies addressing the large individual variability in the negative side effects of dopaminergic medication on functions that implicate dopamine, such as value-based learning and choice. These studies begin to unravel the mechanisms of dopamine overdosing, thus revising the strict version of the overdose hypothesis. For example, the work shows that the canonical boosting of reward-versus punishment-based choice by medication is greater in patients with depression and a non-tremor phenotype, which both implicate, among other pathology, more rather than less severe dysregulation of the mesolimbic dopamine system. Future longitudinal cohort studies are needed to identify how to optimally combine different clinical, personality, cognitive, neural, genetic and molecular predictors of detrimental medication effects in order to account for as much of the relevant variability as possible. This will provide a useful tool for precision neurology, allowing individual and contextual tailoring of (the dose of) dopaminergic medication in order to maximize its cognitive benefits, yet minimize its side effects.

Exploring reward-related attention selectivity deficits in Parkinson's disease

An important aspect of managing a limited cognitive resource like attention is to use the reward value of stimuli to prioritize the allocation of attention to higher-value over lower-value stimuli. Recent evidence suggests this depends on dopaminergic signaling of reward. In Parkinson's disease, both reward sensitivity and attention are impaired, but whether these deficits are directly related to one another is unknown. We tested whether Parkinson's patients use reward information when automatically allocating their attention and whether this is modulated by dopamine replacement. We compared patients, tested both ON and OFF dopamine replacement medication, to older controls using a standard attention capture task. First, participants learned the different reward values of stimuli. Then, these reward-associated stimuli were used as distractors in a visual search task. We found that patients were generally distracted by the presence of the distractors but that the degree of distraction caused by the high-value and low-value distractors was similar. Furthermore, we found no evidence to support the possibility that dopamine replacement modulates the effect of reward on automatic attention allocation. Our results suggest a possible inability in Parkinson's patients to use the reward value of stimuli when automatically allocating their attention, and raise the possibility that reward-driven allocation of resources may affect the adaptive modulation of other cognitive processes.

Neuroticism, Smoking, and the Risk of Parkinson's Disease

Background:

The relationship among neuroticism, smoking, and Parkinson’s disease (PD) is less examined.

Objective:

To examine the causal associations between neuroticism, smoking initiation, and the risk of PD.

Methods:

We performed a two-sample Mendelian randomization (MR) design in a network framework. Summary statistics from meta-analyses of genome-wide association studies (GWAS) were based on large cohorts of European ancestry. Study participants were from various cohort studies for neuroticism and smoking initiation, and case-control studies or cohort studies of PD from previously published GWAS meta-analyses. Patients with PD were ascertained from either clinical visit or self-reported.

Results:

The two-sample MR analysis showed no evidence for a causal association between neuroticism and PD risk (odds ratio [OR] 0.86, 95%confidence intervals [CIs] 0.67 to 1.12). While we did not find a significant association between neuroticism and PD, one SNP, rs58879558 (located in MAPT region), was associated with both neuroticism and PD. We found a significant association of neuroticism on smoking initiation (OR: 1.10, 95%CI: 1.05 to 1.14). Further, our results provided evidence for a protective effect of smoking initiation on the risk of PD (OR: 0.75, 95%CI: 0.62 to 0.91).

Conclusion:

These findings do not support a causal association of neuroticism on PD risk. However, they provide evidence for a causal relationship between neuroticism and smoking initiation and a strong causal effect of smoking initiation on a reduced risk of PD.

Snus and Alcohol: Mutually Rewarding Effects in the Brain? A Matched Controlled Population Study

Background:

The use of moist smokeless tobacco (snus) is increasing in the U.S. and other Western countries, and especially among young people. Snus is associated with several health problems, but the relationship between use of snus and alcohol is scarcely explored. Neuro-cognitive and psychological research suggest an association due to possible mutually rewarding effects in the limbic brain. We investigated this issue in a matched controlled population study.

Methods:

Matched control group design where drinking habits and alcohol consumption in a group of users of snus (n = 1043, mean age = 35.20; n men = 749, n women = 294) were compared to a control group of non-users matched on age and gender (n = 1043, mean age = 35.65; n men = 749, n women = 294). In addition, we registered background variables such as level of education, income, self-perceived general, dental health, mental health, current depressive symptoms, and BMI. In estimation of alcohol consumption, the background variables were used as covariates in factorial analyses of variance (ANCOVA).

Results:

Users of snus had lower level of education, lower income, poorer general, dental, and mental health status than non-users, but there were no differences in BMI. Differences in mental health status were related to drinking habits. Users of snus had a higher frequency of drinking, higher frequency of intoxication, and showed more excess drinking. Controlled for background variables users of snus had a 25.2% higher estimated yearly consumption of alcohol in terms of standard units of alcohol on the weekdays, 26.4% higher on weekends and a 60.2% higher yearly excess consumption.

Conclusion:

Users of snus had an elevated alcohol consumption and another drinking style than non-users. The findings are discussed according to neuro-cognitive and psychopharmacological mechanisms, reward learning and conditioning. The results have implications for prevention, treatment and rehabilitation of alcohol and nicotine dependence.

Reinforcement-based cognitive biases as vulnerability factors in alcohol addiction: From humans to animal models

Alcohol use disorder (AUD) is one of the most common, but still poorly treated, psychiatric conditions. Developing new treatments requires a better understanding of the aetiology of symptoms and evaluation of novel therapeutic targets in preclinical studies. Recent developments in our understanding of the reinforcement-based cognitive biases (RBCBs) that contribute to the development of AUD and its treatment offer new opportunities for both clinical and preclinical research. In this review, we first briefly describe psychological and cognitive theories that involve various aspects of reinforcement sensitivity in the development, maintenance, and recurrence of alcohol addiction. Furthermore, in separate sections, we describe studies investigating RBCBs and their neural, neurochemical, and pharmacological correlates, and we discuss possible interactions between RBCBs and trajectories of AUD. Finally, we describe how recent translational studies using state-of-the-art animal models can facilitate our understanding of the role of reinforcement sensitivity and RBCBs in various aspects of AUD.

The Role of User Behaviour in Improving Cyber Security Management

Information security has for long time been a field of study in computer science, software engineering, and information communications technology. The term 'information security' has recently been replaced with the more generic term cybersecurity. The goal of this paper is to show that, in addition to computer science studies, behavioural sciences focused on user behaviour can provide key techniques to help increase cyber security and mitigate the impact of attackers' social engineering and cognitive hacking methods (i.e., spreading false information). Accordingly, in this paper, we identify current research on psychological traits and individual differences among computer system users that explain vulnerabilities to cyber security attacks and crimes. Our review shows that computer system users possess different cognitive capabilities which determine their ability to counter information security threats. We identify gaps in the existing research and provide possible psychological methods to help computer system users comply with security policies and thus increase network and information security.

Neuropsychiatric and Cognitive Deficits in Parkinson's Disease and Their Modeling in Rodents

Parkinson’s disease (PD) is associated with a large burden of non-motor symptoms including olfactory and autonomic dysfunction, as well as neuropsychiatric (depression, anxiety, apathy) and cognitive disorders (executive dysfunctions, memory and learning impairments). Some of these non-motor symptoms may precede the onset of motor symptoms by several years, and they significantly worsen during the course of the disease. The lack of systematic improvement of these non-motor features by dopamine replacement therapy underlines their multifactorial origin, with an involvement of monoaminergic and cholinergic systems, as well as alpha-synuclein pathology in frontal and limbic cortical circuits. Here we describe mood and neuropsychiatric disorders in PD and review their occurrence in rodent models of PD. Altogether, toxin-based rodent models of PD indicate a significant but non-exclusive contribution of mesencephalic dopaminergic loss in anxiety, apathy, and depressive-like behaviors, as well as in learning and memory deficits. Gene-based models display significant deficits in learning and memory, as well as executive functions, highlighting the contribution of alpha-synuclein pathology to these non-motor deficits. Collectively, neuropsychiatric and cognitive deficits are recapitulated to some extent in rodent models, providing partial but nevertheless useful options to understand the pathophysiology of non-motor symptoms and develop therapeutic options for these debilitating symptoms of PD.

Anatomical dissociation of intracerebral signals for reward and punishment prediction errors in humans

Whether maximizing rewards and minimizing punishments rely on distinct brain systems remains debated, given inconsistent results coming from human neuroimaging and animal electrophysiology studies. Bridging the gap across techniques, we recorded intracerebral activity from twenty participants while they performed an instrumental learning task. We found that both reward and punishment prediction errors (PE), estimated from computational modeling of choice behavior, correlate positively with broadband gamma activity (BGA) in several brain regions. In all cases, BGA scaled positively with the outcome (reward or punishment versus nothing) and negatively with the expectation (predictability of reward or punishment). However, reward PE were better signaled in some regions (such as the ventromedial prefrontal and lateral orbitofrontal cortex), and punishment PE in other regions (such as the anterior insula and dorsolateral prefrontal cortex). These regions might therefore belong to brain systems that differentially contribute to the repetition of rewarded choices and the avoidance of punished choices.

Whether maximizing rewards and minimizing punishments rely on distinct brain learning systems remains debated. Here, using intracerebral recordings in humans, the authors provide evidence for brain regions differentially engaged in signaling reward and punishment prediction errors that prescribe repetition versus avoidance of past choices.

Nafion and Multiwall Carbon Nanotube Modified Ultrananocrystalline Diamond Microelectrodes for Detection of Dopamine and Serotonin

Neurochemicals play a critical role in the function of the human brain in healthy and diseased states. Here, we have investigated three types of microelectrodes, namely boron-doped ultrananocrystalline diamond (BDUNCD), nafion-modified BDUNCD, and nafion-multi-walled carbon nanotube (MWCNT)-modified BDUNCD microelectrodes for long-term neurochemical detection. A ~50 nm-thick nafion-200-nm-thick MWCNT-modified BDUNCD microelectrode provided an excellent combination of sensitivity and selectivity for the detection of dopamine (DA; 6.75 μA μM-1 cm-2) and serotonin (5-HT; 4.55 μA μM-1 cm-2) in the presence of excess amounts of ascorbic acid (AA), the most common interferent. Surface stability studies employing droplet-based microfluidics demonstrate rapid response time (<2 s) and low limits of detection (5.4 ± 0.40 nM). Furthermore, we observed distinguishable DA and 5-HT current peaks in a ternary mixture during long-term stability studies (up to 9 h) with nafion-MWCNT-modified BDUNCD microelectrodes. Reduced fouling on the modified BDUNCD microelectrode surface offers significant advantages for their use in long-term neurochemical detection as compared to those of prior-art microelectrodes.

Role of SHH in Patterning Human Pluripotent Cells towards Ventral Forebrain Fates

The complexities of human neurodevelopment have historically been challenging to decipher but continue to be of great interest in the contexts of healthy neurobiology and disease. The classic animal models and monolayer in vitro systems have limited the types of questions scientists can strive to answer in addition to the technical ability to answer them. However, the tridimensional human stem cell-derived organoid system provides the unique opportunity to model human development and mimic the diverse cellular composition of human organs. This strategy is adaptable and malleable, and these neural organoids possess the morphogenic sensitivity to be patterned in various ways to generate the different regions of the human brain. Furthermore, recapitulating human development provides a platform for disease modeling. One master regulator of human neurodevelopment in many regions of the human brain is sonic hedgehog (SHH), whose expression gradient and pathway activation are responsible for conferring ventral identity and shaping cellular phenotypes throughout the neural axis. This review first discusses the benefits, challenges, and limitations of using organoids for studying human neurodevelopment and disease, comparing advantages and disadvantages with other in vivo and in vitro model systems. Next, we explore the range of control that SHH exhibits on human neurodevelopment, and the application of SHH to various stem cell methodologies, including organoids, to expand our understanding of human development and disease. We outline how this strategy will eventually bring us much closer to uncovering the intricacies of human neurodevelopment and biology.

Impulse Control Disorders in Parkinson's Disease: From Bench to Bedside

Parkinson’s disease (PD) is a neurodegenerative disorder that is characterized by symptoms that impact both motor and non-motor domains. Outside of motor impairments, PD patients are at risk for impulse control disorders (ICDs), which include excessively disabling impulsive and compulsive behaviors. ICD symptoms in PD (PD + ICD) can be broadly conceptualized as a synergistic interaction between dopamine agonist therapy and the many molecular and circuit-level changes intrinsic to PD. Aside from discontinuing dopamine agonist treatment, there remains a lack of consensus on how to best address ICD symptoms in PD. In this review, we explore recent advances in the molecular and neuroanatomical mechanisms underlying ICD symptoms in PD by summarizing a rapidly accumulating body of clinical and preclinical studies, with a special focus on the utility of rodent models in gaining new insights into the neurochemical basis of PD + ICD. We also discuss the relevance of these findings to the broader problem of impulsive and compulsive behaviors that impact a range of neuropsychiatric syndromes.

A Causal Role for the Pedunculopontine Nucleus in Human Instrumental Learning

A critical mechanism for maximizing reward is instrumental learning. In standard instrumental learning models, action values are updated on the basis of reward prediction errors (RPEs), defined as the discrepancy between expectations and outcomes. A wealth of evidence across species and experimental techniques has established that RPEs are signaled by midbrain dopamine neurons. However, the way dopamine neurons receive information about reward outcomes remains poorly understood. Recent animal studies suggest that the pedunculopontine nucleus (PPN), a small brainstem structure considered as a locomotor center, is sensitive to reward and sends excitatory projection to dopaminergic nuclei. Here, we examined the hypothesis that the PPN could contribute to reward learning in humans. To this aim, we leveraged a clinical protocol that assessed the therapeutic impact of PPN deep-brain stimulation (DBS) in three patients with Parkinson disease. PPN local field potentials (LFPs), recorded while patients performed an instrumental learning task, showed a specific response to reward outcomes in a low-frequency (alpha-beta) band. Moreover, PPN DBS selectively improved learning from rewards but not from punishments, a pattern that is typically observed following dopaminergic treatment. Computational analyses indicated that the effect of PPN DBS on instrumental learning was best captured by an increase in subjective reward sensitivity. Taken together, these results support a causal role for PPN-mediated reward signals in human instrumental learning.

Human moral decision-making through the lens of Parkinson's disease

Parkinson's disease (PD) is a neurodegenerative disorder characterized by the loss of dopaminergic neurons in the basal ganglia (BG) and thalamocortical circuitry. While defective motor control has long been considered the defining symptom of PD, mounting evidence indicates that the BG are fundamentally important for a multitude of cognitive, emotional, and motivational processes in addition to motor function. Here, we review alterations in moral decision-making in people with PD, specifically in the context of deceptive behavior. We report that PD patients exhibit two opposite behavioral patterns: hyper- and hypo-honesty. The hyper-honest subgroup engages in deception less often than matched controls, even when lying is associated with a monetary payoff. This behavioral pattern seems to be linked to dopaminergic hypo-activity, implying enhanced harm avoidance, risk aversion, non-impulsivity, and reduced reward sensitivity. On the contrary, the hypo-honest subgroup-often characterized by the additional diagnosis of impulse control disorders (ICDs) and dopamine dysregulation syndrome (DDS)-deceives more often than both PD patients without ICDs/DDS and controls. This behavioral pattern appears to be associated with dopaminergic hyperactivity, which underpins enhanced novelty-seeking, risk-proneness, impulsivity, and reward sensitivity. We posit that these two complementary behavioral patterns might be related to dysfunction of the dopaminergic reward system, leading to reduced or enhanced motivation to deceive. Only a few studies have directly investigated moral decision-making in PD and other neurodegenerative disorders affecting the BG, and further research on the causal role of subcortical structures in shaping moral behavior is needed.

The dynamics, processes, mechanisms, and functioning of personality: An overview of the field

Personality psychology has long focused on structural trait models, but it can also offer a rich understanding of the dynamics, processes, mechanisms, and functioning of individual differences or entire persons. The field of personality dynamics, which works towards such an understanding, has experienced a renaissance in the last two decades. This review article seeks to act as a primer of that field. It covers its historical roots, summarizes current research strands - along with their theoretical backbones and methodologies - in an accessible way, and sketches some considerations for the future. In doing so, we introduce relevant concepts, give an overview of different topics and phenomena subsumed under the broad umbrella term 'dynamics', and highlight the interdisciplinarity as well as applied relevance of the field. We hope this article can serve as a useful overview for scholars within and outside of personality psychology who are interested in the dynamic nature of human behaviour and experience.

Does the Personality of Patients with Parkinson's Disease Affect the Decision to Perform Deep Brain Stimulation Surgery? A Cross-Sectional Study in a Chinese Cohort

We investigated whether the personality of patients with Parkinson's disease (PD) before subthalamic brain stimulation differed from patients receiving drug treatments and whether the personality of patients affected surgical decisions. We recruited 38 patients with advanced PD scheduled for deep brain stimulation (DBS), 40 patients with PD receiving the very best medical treatment, and 51 healthy control subjects. All participants were evaluated by the Minnesota multiphasic personality inventory-1 (MMPI-1). PD patients who were candidates for DBS did not exhibit any significant differences in personality when compared with PD patients who were treated with drugs. Compared with healthy controls, patients with PD had remarkably higher MMPI-1 scores for spiritual quality, neuroticism, and introversion, but significantly lower scores for socialization. In addition, patients with PD were more submissive, more dependent on others, and less active in social activities. Our data indicated that the main deciding factor relating to whether to undergo DBS was the disease itself and not the pathological personality. However, neurotic and psychotic symptoms accompanying PD may influence the effect of DBS. We found that greater benefit is obtained by surgical or medical interventions if abnormal neurotic characteristics are considered early in the course of PD.

Intrinsic motivation in patients with Parkinson's disease: a neuropsychological investigation of curiosity using dopamine transporter imaging

Both intrinsic and extrinsic motivation are believed to involve brain regions that are innervated by the dopaminergic pathway. Although dopaminergic neurons in the midbrain deteriorate in Parkinson's disease (PD), it remains unclear whether intrinsic motivation is impaired in PD patients. To address this issue, we investigated intrinsic motivation in PD patients using a task designed to assess the "Pandora effect," which constitutes a curiosity for resolving uncertainty, even if this curiosity is likely to result in negative consequences. Twenty-seven PD patients and 27 age-matched healthy controls (HCs) completed a curiosity task in which they were required to decide either to view or skip negative pictures (e.g., snakes, spiders) and an examination battery that included the Mini-Mental State Examination, a verbal fluency test, the Trail Making Test, 10-word recall tests, and questionnaires for behavioral inhibition/activation and depression. DaTSCAN images to assess the distribution of dopamine transporters in the striatum were acquired only from PD patients. The results revealed that PD patients, relative to the HCs, viewed the pictures less frequently under both the certain and uncertain conditions. However, both the PD patients and HCs viewed the pictures at a higher frequency under the uncertain condition than under the certain condition. In the PD patients, the proportion of pictures viewed under the certain condition was positively correlated with the distribution of dopamine transporters in the striatum. These results suggest that despite the overall decreasing level of interest in viewing negative pictures, the motivation to resolve uncertainty is relatively intact in PD patients.

Personality dimensions of patients can change during the course of parkinson's disease

BACKGROUND

Studies assessing personality dimensions by the "Temperament and Character Inventory" (TCI) have previously found an association between Parkinson's disease (PD) and lower Novelty Seeking and higher Harm Avoidance scores. Here, we aimed to describe personality dimensions of PD patients with motor fluctuations and compare them to a normative population and other PD populations.

METHODS

All PD patients awaiting Deep Brain Stimulation (DBS) answered the TCI before neurosurgery. Their results were compared to those of historical cohorts (a French normative population, a de novo PD population, and a PD population with motor fluctuations).

RESULTS

Most personality dimensions of our 333 included PD patients with motor fluctuations who are candidates for DBS were different from those of the normative population and some were also different from those of the De Novo PD population, whereas they were similar to those of another population of PD patients with motor fluctuations.

CONCLUSIONS

During the course of PD, personality dimensions can change in parallel with the development of motor fluctuations, either due to the evolution of the disease and/or dopaminergic treatments.

The Reward System and Post-Traumatic Stress Disorder: Does Trauma Affect the Way We Interact With Positive Stimuli?

Post-traumatic stress disorder (PTSD) is a highly prevalent disorder and a highly debilitating condition. Although anhedonia is an important construct of the disorder, the relationship between PTSD and reward functioning is still under-researched. To date, the majority of research on PTSD has focused on fear: fear learning, maintenance, and extinction. Here we review the relevant literature-including clinical observations, self-report data, neuroimaging research, and animal studies-in order to examine the potential effects of post-traumatic stress disorder on the reward system. Our current lack of sufficient insight into how trauma affects the reward system is one possible hindrance to clinical progress. The current review highlights the need for further investigation into the complex relationship between exposure to trauma and the reward system to further our understandings of the ethology of PTSD.

Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype

Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.

Impaired working memory, cognitive flexibility and reward processing in mice genetically lacking Gpr88: Evidence for a key role for Gpr88 in multiple cortico-striatal-thalamic circuits

The GPR88 orphan G protein-coupled receptor is expressed throughout the striatum, being preferentially localised in medium spiny neurons. It is also present in lower densities in frontal cortex and thalamus. Rare mutations in humans suggest a role in cognition and motor function, while common variants are associated with psychosis. Here we evaluate the influence of genetic deletion of GPR88 upon performance in translational tasks interrogating motivation, reward evaluation and cognitive function. In an automated radial arm maze 'N-back' working memory task, Gpr88 KO mice showed impaired correct responding, suggesting a role for GPR88 receptors in working memory circuitry. Associative learning performance was similar to wild-type controls in a touchscreen task but performance was impaired at the reversal learning stage, suggesting cognitive inflexibility. Gpr88 KO mice showed higher breakpoints, reduced latencies and lengthened session time in a progressive ratio task consistent with enhanced motivation. Simultaneously, locomotor hyperactivity was apparent in this task, supporting previous findings of actions of GPR88 in a cortico-striatal-thalamic motor loop. Evidence for a role of GPR88 in reward processing was demonstrated in a touchscreen-based equivalent of the Iowa gambling task. Although both Gpr88 KO and wild-type mice showed a preference for an optimum contingency choice, Gpr88 KO mice selected more risky choices at the expense of more advantageous lower risk options. Together these novel data suggest that striatal GPR88 receptors influence activity in a range of procedures integrated by prefrontal, orbitofrontal and anterior cingulate cortico-striatal-thalamic loops leading to altered cognitive, motivational and reward evaluation processes.

A new approach toward gait training in patients with Parkinson's Disease

BACKGROUND

Typically, people with Parkinson's Disease (PD) progress to develop a gait pattern that is characterized by quick, short and shuffling steps. Gait cycle is altered and lacks definition and fluidity. Gait training combined with a variety of feedback modalities for PD are usually based on non-immediate and externally-based cues but none of these provide real-time feedback on gait quality and acquired gains tend to abate shortly after rehabilitation. Based on principals of motor learning, our team has developed the Heel2Toe sensor to provide real-time auditory feedback during gait training.

RESEARCH QUESTION

Is a short-term training using the Heel2Toe sensor feasible and efficient to improve gait in people with PD? Our objectives are to identify the extent of the immediate response to the feedback within the same session and the carry-over response to training and; 2) to identify patients' perceived effects, pleasures and challenges of using the Heel2Toe.

METHODS

Single-arm, proof-of-concept study. Six people received five sessions of gait training over a 2-3-week period using the Heel2Toe augmented with mobility exercises as an adjunct to gait training. The main outcomes were technically assessed gait parameters collected over a 2-minute walk test, without and with feedback. Heel2Toe signals were analyzed to extract angular velocity(AV), percentage of good steps, average cadence, and AV coefficient of variation(CV).

RESULTS

An immediate response to the Heel2Toe use and a carry-over response to the short-term training with the sensor were observed: an increase in AV with a reduction in CV (better heel strike and gait regularity); an increase in %good steps; and a near-optimal and homogeneous cadence (∼100 steps/min), which is equivalent to a moderate-intensity walking.

SIGNIFICANCE

Gait training using the Heel2Toe sensor is feasible and potentially effective for improving gait quality in people with PD. A definitive trial is a logical next step.

Dopamine agonist treatment increases sensitivity to gamble outcomes in the hippocampus in de novo Parkinson's disease

BACKGROUND

Parkinson's disease is associated with severe nigro-striatal dopamine depletion, leading to motor dysfunction and altered reward processing. We previously showed that drug-naïve patients with Parkinson's disease had a consistent attenuation of reward signalling in the mesolimbic and mesocortical system. Here, we address the neurobiological effects of dopaminergic therapy on reward sensitivity in the mesolimbic circuitry, and how this may contribute to neuropsychiatric symptoms.

OBJECTIVES

We tested the hypothesis that (1) dopaminergic treatment would restore the attenuated, mesolimbic and mesocortical responses to reward; and (2) restoration of reward responsivity by dopaminergic treatment would predict motor performance and the emergence of impulse control symptoms.

METHODS

In 11 drug-naïve Parkinson patients, we prospectively assessed treatment-induced changes in reward processing before, and eight weeks after initiation of monotherapy with dopamine agonists. They were compared to 10 non-medicated healthy controls who were also measured longitudinally. We used whole-brain functional magnetic resonance imaging at 3 Tesla to assess the reward responsivity of the brain to monetary gains and losses, while participants performed a simple consequential gambling task.

RESULTS

In patients, dopaminergic treatment improved clinical motor symptoms without significantly changing task performance. Dopamine agonist therapy induced a stronger reward responsivity in the right hippocampus with higher doses being less effective. None of the patients developed impulse control disorders in the follow-up period of four years.

CONCLUSIONS

Short-term treatment with first-ever dopaminergic medication partially restores deficient reward-related processing in the hippocampus in de novo Parkinson's disease.

From apathy to addiction: Insights from neurology and psychiatry

The tendency to engage in addictive behaviors has long been tied to the actions of the dopamine system. Early theories were based on the fact that all addictive drugs and behaviors (such as gambling) increase dopamine levels in the striatum, and the evidence that dopamine signaled reward or reward prediction error. However, with a changing emphasis of addiction away from purely pharmacological models that emphasize tolerance and withdrawal, towards one of behavioral dyscontrol, is there still a place for abnormal dopamine signaling in addiction? Here we recast the dopamine theory of addiction based on the idea that tonic dopamine may index a continuous phenotype that goes from apathy to impulsivity and compulsivity. Higher tonic dopamine signaling would make individuals vulnerable to drug reinforcement and cue-induced craving. We relate this to computational models of dopamine signaling, and review clinical and neuroimaging evidence from Parkinson's Disease, schizophrenia and bipolar disorder in support of this model.

Spectrum of impulse control behaviours in Parkinson's disease: pathophysiology and management

Impulse control behaviours (ICBs) are a range of behaviours linked by their reward-based, repetitive natures. They can be precipitated in Parkinson's disease (PD) by dopamine replacement therapy, often with detrimental consequences for patients and caregivers. While now a well-recognised non-motor feature of treated PD, much remains unknown about the influence of risk factors, pathophysiological mechanisms, vulnerability factors for specific types of behaviour and the optimal management strategies. Imaging studies have identified structural and functional changes in striatal and prefrontal brain regions, among others. Gene association studies indicate a role for genetic predisposition to PD-ICB. Clinical observational studies have identified potential modifiable and non-modifiable risk factors. Psychological studies shed light on the neurocognitive domains implicated in PD-ICBs and identify psychosocial determinants that may perpetuate the cycle of impulsive and harm-avoidance behaviours. Based on these results, a range of pharmacological and non-pharmacological management strategies have been trialled in PD-ICBs with varying success. The purpose of this review is to update clinicians on the evidence around the pathophysiology of PD-ICB. We aim to translate our findings into an interpretable biopsychosocial model that can be applied to the clinical assessment and management of individual cases of PD-ICB.

Impaired context-sensitive adjustment of behaviour in Parkinson's disease patients tested on and off medication: An fMRI study

The brain's sensitivity to and accentuation of unpredicted over predicted sensory signals plays a fundamental role in learning. According to recent theoretical models of the predictive coding framework, dopamine is responsible for balancing the interplay between bottom-up input and top-down predictions by controlling the precision of surprise signals that guide learning. Using functional MRI, we investigated whether patients with Parkinson's disease (PD) show impaired learning from prediction errors requiring either adaptation or stabilisation of current predictions. Moreover, we were interested in whether deficits in learning over a specific time scale would be accompanied by altered surprise responses in dopamine-related brain structures. To this end, twenty-one PD patients tested on and off dopaminergic medication and twenty-one healthy controls performed a digit prediction paradigm. During the task, violations of sequence-based predictions either signalled the need to update or to stabilise the current prediction and, thus, to react to them or ignore them, respectively. To investigate contextual adaptation to prediction errors, the probability (or its inverse, surprise) of the violations fluctuated across the experiment. When the probability of prediction errors over a specific time scale increased, healthy controls but not PD patients off medication became more flexible, i.e., error rates at violations requiring a motor response decreased in controls but increased in patients. On the neural level, this learning deficit in patients was accompanied by reduced signalling in the substantia nigra and the caudate nucleus. In contrast, differences between the groups regarding the probabilistic modulation of behaviour and neural responses were much less pronounced at prediction errors requiring only stabilisation but no adaptation. Interestingly, dopaminergic medication could neither improve learning from prediction errors nor restore the physiological, neurotypical pattern. Our findings point to a pivotal role of dysfunctions of the substantia nigra and caudate nucleus in deficits in learning from flexibility-demanding prediction errors in PD. Moreover, the data witness poor effects of dopaminergic medication on learning in PD.