Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: evidence from Parkinson's disease

Multiple and subject-specific roles of uncertainty in reward-guided decision-making

Decision-making in noisy, changing, and partially observable environments entails a basic tradeoff between immediate reward and longer-term information gain, known as the exploration-exploitation dilemma. Computationally, an effective way to balance this tradeoff is by leveraging uncertainty to guide exploration. Yet, in humans, empirical findings are mixed, from suggesting uncertainty-seeking to indifference and avoidance. In a novel bandit task that better captures uncertainty-driven behavior, we find multiple roles for uncertainty in human choices. First, stable and psychologically meaningful individual differences in uncertainty preferences actually range from seeking to avoidance, which can manifest as null group-level effects. Second, uncertainty modulates the use of basic decision heuristics that imperfectly exploit immediate rewards: a repetition bias and win-stay-lose-shift heuristic. These heuristics interact with uncertainty, favoring heuristic choices under higher uncertainty. These results, highlighting the rich and varied structure of reward-based choice, are a step to understanding its functional basis and dysfunction in psychopathology.

Assessing alexithymia: the proposal of a psychometric tool based on spheric videos

The present perspective introduces a novel psychometric tool designed to enhance the evaluation of alexithymia. Alexithymia, a condition marked by difficulties in recognizing and expressing emotions, along with a propensity to direct attention outside rather than toward one's own interior experiences, is commonly investigated through self-report questionnaires. These instruments assume that individuals have sufficient self-awareness and abstraction capabilities, which restricts the understanding of the underlying mechanisms of emotional recognition in individuals who do not possess these capacities. To address this lack, emerging technologies like virtual reality (VR) and 360° videos facilitate the recreation of immersive contexts, enabling subjects to engage with scenarios even remotely. Our innovative tool employs spherical video technology to recreate social and non-social scenarios that elicit emotions. Psychophysiological measures are collected during video observation; then, questions are asked to investigate how the subject consciously processes the emotions they experienced. This multimodal approach aims to capture both implicit and explicit emotion processing, providing a comprehensive assessment. Overall, the proposed psychometric tool offers the potential for a more nuanced understanding of alexithymic traits and their real-life impact, empowering clinicians to tailor treatment processes to individual needs based on a richer set of information.

Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors

Deciding whether to forego immediate rewards or explore new opportunities is a key component of flexible behavior and is critical for the survival of the species. Although previous studies have shown that different cortical and subcortical areas, including the amygdala and ventral striatum (VS), are implicated in representing the immediate (exploitative) and future (explorative) value of choices, the effect of the motor system used to make choices has not been examined. Here, we tested male rhesus macaques with amygdala or VS lesions on two versions of a three-arm bandit task where choices were registered with either a saccade or an arm movement. In both tasks we presented the monkeys with explore-exploit tradeoffs by periodically replacing familiar options with novel options that had unknown reward probabilities. We found that monkeys explored more with saccades but showed better learning with arm movements. VS lesions caused the monkeys to be more explorative with arm movements and less explorative with saccades, although this may have been due to an overall decrease in performance. VS lesions affected the monkeys' ability to learn novel stimulus-reward associations in both tasks, while after amygdala lesions this effect was stronger when choices were made with saccades. Further, on average, VS and amygdala lesions reduced the monkeys' ability to choose better options only when choices were made with a saccade. These results show that learning reward value associations to manage explore-exploit behaviors is motor system dependent and they further define the contributions of amygdala and VS to reinforcement learning.

Altered delay discounting in neurodegeneration: insight into the underlying mechanisms and perspectives for clinical applications

Steeper delay discounting (i.e., the extent to which future rewards are perceived as less valuable than immediate ones) has been proposed as a transdiagnostic process across different health conditions, in particular psychiatric disorders. Impulsive decision-making is a hallmark of different neurodegenerative conditions but little is known about delay discounting in the domain of neurodegenerative conditions. We reviewed studies on delay discounting in patients with Parkinson's disease (PD) and in patients with dementia (Alzheimer's disease / AD or frontotemporal dementia / FTD). We proposed that delay discounting could be an early marker of the neurodegenerative process. We developed the idea that altered delay discounting is associated with overlapping but distinct neurocognitive mechanisms across neurodegenerative diseases: dopaminergic-related disorders of reward processing in PD, memory/projection deficits due to medial temporal atrophy in AD, modified reward processing due to orbitofrontal atrophy in FTD. Neurodegeneration could provide a framework to decipher the neuropsychological mechanisms of value-based decision-making. Further, delay discounting could become a marker of interest in clinical practice, in particular for differential diagnosis.

Environmental uncertainty and the advantage of impulsive choice strategies

Choice impulsivity is characterized by the choice of immediate, smaller reward options over future, larger reward options, and is often thought to be associated with negative life outcomes. However, some environments make future rewards more uncertain, and in these environments impulsive choices can be beneficial. Here we examined the conditions under which impulsive vs. non-impulsive decision strategies would be advantageous. We used Markov Decision Processes (MDPs) to model three common decision-making tasks: Temporal Discounting, Information Sampling, and an Explore-Exploit task. We manipulated environmental variables to create circumstances where future outcomes were relatively uncertain. We then manipulated the discount factor of an MDP agent, which affects the value of immediate versus future rewards, to model impulsive and non-impulsive behavior. This allowed us to examine the performance of impulsive and non-impulsive agents in more or less predictable environments. In Temporal Discounting, we manipulated the transition probability to delayed rewards and found that the agent with the lower discount factor (i.e. the impulsive agent) collected more average reward than the agent with a higher discount factor (the non-impulsive agent) by selecting immediate reward options when the probability of receiving the future reward was low. In the Information Sampling task, we manipulated the amount of information obtained with each sample. When sampling led to small information gains, the impulsive MDP agent collected more average reward than the non-impulsive agent. Third, in the Explore-Exploit task, we manipulated the substitution rate for novel options. When the substitution rate was high, the impulsive agent again performed better than the non-impulsive agent, as it explored the novel options less and instead exploited options with known reward values. The results of these analyses show that impulsivity can be advantageous in environments that are unexpectedly uncertain.

The neurocomputational bases of explore-exploit decision-making

Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives. Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans. We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded. Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex. These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

Evidence and implications of abnormal predictive coding in dementia

The diversity of cognitive deficits and neuropathological processes associated with dementias has encouraged divergence in pathophysiological explanations of disease. Here, we review an alternative framework that emphasizes convergent critical features of cognitive pathophysiology. Rather than the loss of ‘memory centres’ or ‘language centres’, or singular neurotransmitter systems, cognitive deficits are interpreted in terms of aberrant predictive coding in hierarchical neural networks. This builds on advances in normative accounts of brain function, specifically the Bayesian integration of beliefs and sensory evidence in which hierarchical predictions and prediction errors underlie memory, perception, speech and behaviour. We describe how analogous impairments in predictive coding in parallel neurocognitive systems can generate diverse clinical phenomena, including the characteristics of dementias. The review presents evidence from behavioural and neurophysiological studies of perception, language, memory and decision-making. The reformulation of cognitive deficits in terms of predictive coding has several advantages. It brings diverse clinical phenomena into a common framework; it aligns cognitive and movement disorders; and it makes specific predictions on cognitive physiology that support translational and experimental medicine studies. The insights into complex human cognitive disorders from the predictive coding framework may therefore also inform future therapeutic strategies.

Balancing exploration and exploitation with information and randomization

Explore-exploit decisions require us to trade off the benefits of exploring unknown options to learn more about them, with exploiting known options, for immediate reward. Such decisions are ubiquitous in nature, but from a computational perspective, they are notoriously hard. There is therefore much interest in how humans and animals make these decisions and recently there has been an explosion of research in this area. Here we provide a biased and incomplete snapshot of this field focusing on the major finding that many organisms use two distinct strategies to solve the explore-exploit dilemma: a bias for information ('directed exploration') and the randomization of choice ('random exploration'). We review evidence for the existence of these strategies, their computational properties, their neural implementations, as well as how directed and random exploration vary over the lifespan. We conclude by highlighting open questions in this field that are ripe to both explore and exploit.

Individual associations of adolescent alcohol use disorder versus cannabis use disorder symptoms in neural prediction error signaling and the response to novelty

Two of the most commonly used illegal substances by adolescents are alcohol and cannabis. Alcohol use disorder (AUD) and cannabis use disorder (CUD) are associated with poorer decision-making in adolescents. In adolescents, level of AUD symptomatology has been negatively associated with striatal reward responsivity. However, little work has explored the relationship with striatal reward prediction error (RPE) representation and the extent to which any augmentation of RPE by novel stimuli is impacted. One-hundred fifty-one adolescents participated in the Novelty Task while undergoing functional magnetic resonance imaging (fMRI). In this task, participants learn to choose novel or non-novel stimuli to gain monetary reward. Level of AUD symptomatology was negatively associated with both optimal decision-making and BOLD response modulation by RPE within striatum and regions of prefrontal cortex. The neural alterations in RPE representation were particularly pronounced when participants were exploring novel stimuli. Level of CUD symptomatology moderated the relationship between novelty propensity and RPE representation within inferior parietal lobule and dorsomedial prefrontal cortex. These data expand on an emerging literature investigating individual associations of AUD symptomatology levels versus CUD symptomatology levels and RPE representation during reinforcement processing and provide insight on the role of neuro-computational processes underlying reinforcement learning/decision-making in adolescents.

Alexithymia

Humans are highly adept at differentiating, regulating, and responding to their emotions. At the core of all these functions is emotional awareness: the conscious feeling states that are central to human mental life. Disrupted emotional awareness-a subclinical construct commonly referred to as alexithymia-is present in a range of psychiatric and neurological disorders and can have a deleterious impact on functional outcomes and treatment response. This chapter is a selective review of the current state of the science on alexithymia. We focus on two separate but related issues: (i) the functional deficits associated with alexithymia and what they reveal about the importance of emotional awareness for shaping normative human functioning, and (ii) the neural correlates of alexithymia and what they can inform us about the biological bases of emotional awareness. Lastly, we outline challenges and opportunities for alexithymia research, focusing on measurement issues and the potential utility of formal computational models of emotional awareness for advancing the fields of clinical and affective science.

Time will tell: Decision making in premanifest and manifest Huntington's disease

OBJECTIVE

To investigate cognitive flexibility in premanifest and manifest Huntington's disease (HD).

BACKGROUND

HD is an autosomal dominant neurodegenerative disease characterized by motor, cognitive, and behavioral abnormalities with typical motor symptoms. In this study, we wanted to assess decision making in premanifest (pre-HD) and manifest HD patients.

METHODS

A total of 77 non-demented subjects including 29 pre-HD, 22 manifest HD patients, and 26 healthy controls (HC) were included. We stratified the pre-HD group based on their estimated years to disease onset into a far (FAR, n = 13) and a near (NEAR, n = 16) group. Furthermore, participants performed the Montreal cognitive assessment battery (MoCA), the trail making task part A and B (TMT A, TMT B), the Symbol digit modalities test (SDMT), and the beads task.

RESULTS

In the beads task, HD patients gathered less information than all other groups (all p-values < .001). Furthermore, the NEAR group gathered less information than the FAR group (p < .001) and HC (p = .001). There was no difference between the HC and the FAR group (p = 1.0). In the TMT and the SDMT, HD patients were slower than all other groups (all p-values < .01) but there were no other significant differences.

CONCLUSIONS

Decision making with a higher degree of uncertainty may be an early neuropsychological sign to indicate the disease process prior to reaching criteria for motor diagnosis of HD.

Primate Orbitofrontal Cortex Codes Information Relevant for Managing Explore-Exploit Tradeoffs

Reinforcement learning (RL) refers to the behavioral process of learning to obtain reward and avoid punishment. An important component of RL is managing explore-exploit tradeoffs, which refers to the problem of choosing between exploiting options with known values and exploring unfamiliar options. We examined correlates of this tradeoff, as well as other RL related variables, in orbitofrontal cortex (OFC) while three male monkeys performed a three-armed bandit learning task. During the task, novel choice options periodically replaced familiar options. The values of the novel options were unknown, and the monkeys had to explore them to see if they were better than other currently available options. The identity of the chosen stimulus and the reward outcome were strongly encoded in the responses of single OFC neurons. These two variables define the states and state transitions in our model that are relevant to decision-making. The chosen value of the option and the relative value of exploring that option were encoded at intermediate levels. We also found that OFC value coding was stimulus specific, as opposed to coding value independent of the identity of the option. The location of the option and the value of the current environment were encoded at low levels. Therefore, we found encoding of the variables relevant to learning and managing explore-exploit tradeoffs in OFC. These results are consistent with findings in the ventral striatum and amygdala and show that this monosynaptically connected network plays an important role in learning based on the immediate and future consequences of choices.SIGNIFICANCE STATEMENT Orbitofrontal cortex (OFC) has been implicated in representing the expected values of choices. Here we extend these results and show that OFC also encodes information relevant to managing explore-exploit tradeoffs. Specifically, OFC encodes an exploration bonus, which characterizes the relative value of exploring novel choice options. OFC also strongly encodes the identity of the chosen stimulus, and reward outcomes, which are necessary for computing the value of novel and familiar options.

Subjective estimates of uncertainty during gambling and impulsivity after subthalamic deep brain stimulation for Parkinson's disease

Subthalamic deep brain stimulation (DBS) for Parkinson’s disease (PD) may modulate chronometric and instrumental aspects of choice behaviour, including motor inhibition, decisional slowing, and value sensitivity. However, it is not well known whether subthalamic DBS affects more complex aspects of decision-making, such as the influence of subjective estimates of uncertainty on choices. In this study, 38 participants with PD played a virtual casino prior to subthalamic DBS (whilst ‘on’ medication) and again, 3-months postoperatively (whilst ‘on’ stimulation). At the group level, there was a small but statistically significant decrease in impulsivity postoperatively, as quantified by the Barratt Impulsiveness Scale (BIS). The gambling behaviour of participants (bet increases, slot machine switches and double or nothing gambles) was associated with this self-reported measure of impulsivity. However, there was a large variance in outcome amongst participants, and we were interested in whether individual differences in subjective estimates of uncertainty (specifically, volatility) were related to differences in pre- and postoperative impulsivity. To examine these individual differences, we fit a computational model (the Hierarchical Gaussian Filter, HGF), to choices made during slot machine game play as well as a simpler reinforcement learning model based on the Rescorla-Wagner formalism. The HGF was superior in accounting for the behaviour of our participants, suggesting that participants incorporated beliefs about environmental uncertainty when updating their beliefs about gambling outcome and translating these beliefs into action. A specific aspect of subjective uncertainty, the participant’s estimate of the tendency of the slot machine’s winning probability to change (volatility), increased subsequent to DBS. Additionally, the decision temperature of the response model decreased post-operatively, implying greater stochasticity in the belief-to-choice mapping of participants. Model parameter estimates were significantly associated with impulsivity; specifically, increased uncertainty was related to increased postoperative impulsivity. Moreover, changes in these parameter estimates were significantly associated with the maximum post-operative change in impulsivity over a six month follow up period. Our findings suggest that impulsivity in PD patients may be influenced by subjective estimates of uncertainty (environmental volatility) and implicate a role for the subthalamic nucleus in the modulation of outcome certainty. Furthermore, our work outlines a possible approach to characterising those persons who become more impulsive after subthalamic DBS, an intervention in which non-motor outcomes can be highly variable.

Subcortical Substrates of Explore-Exploit Decisions in Primates

The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

Disruption of Multiple Distinctive Neural Networks Associated With Impulse Control Disorder in Parkinson's Disease

The phenomenon of impulsivity in Parkinson's disease appears as an arduous side effect of dopaminergic therapy with potentially detrimental consequences for the life of the patients. Although conceptualized as a result of non-physiologic chronic dopaminergic stimulation, recent advances speculate on combined disruption of other networks as well. In the search for neuroanatomical correlates of this multifaceted disturbance, this study employs two distinct, well-defined tasks of close association to motor inhibition and decision-making impulsivity, Go/No Go and Delay discounting. The fMRI and functional connectivity analysis in 21 Parkinson's disease patients, including 8 patients suffering from severe impulse control disorder, and 28 healthy controls, revealed in impulsive Parkinson's disease patients not only decreased fMRI activation in the dorsolateral prefrontal cortex and bilateral striatum, but also vast functional connectivity changes of both caudate nuclei as decreased connectivity to the superior parietal cortex and increased connectivity to the insular area, clearly beyond the commonly stated areas, which indicates that orbitofronto-striatal and mesolimbic functional disruptions are not the sole mechanisms underlying impulse control disorder in Parkinson's disease. Ergo, our results present a refinement and synthesis of gradually developing ideas about the nature of impulsive control disorder in Parkinson's disease—an umbrella term encompassing various behavioral deviations related to distinct neuronal networks and presumably neurotransmitter systems, which greatly exceed the previously envisioned dopaminergic pathways as the only culprit.

Neurocognitive correlates of medication-induced addictive behaviours in Parkinson's disease: A systematic review

Dopaminergic medication can induce severe addictive behaviours (e.g., pathological gambling) in susceptible Parkinson's disease (PD) patients. It is still unknown which particular neurocognitive processes become exacerbated or dysfunctional in PD patients with addictive behaviours. We sought to systematically review the relevant literature to identity potential neurocognitive correlates of medication-induced addictive behaviours in PD. We framed our review around neurocognitive processes central to four dominant accounts of substance addiction: 'aberrant learning', 'incentive sensitization', 'impulsivity to compulsivity' and 'impaired response inhibition and salience attribution'. Searches of the PubMed and Scopus databases were completed on June 23, 2017. To be included, studies were required to involve: (a) medicated PD patients, without a history of deep brain stimulation, with and without addictive behaviours; (b) a reward-related or decision-making task; and (c) statistical comparison of addictive and non-addictive groups' 'on' medication performance on the task(s). Studies were summarised qualitatively with statistically significant (p<.05) group differences and effect sizes (Cohen's d) highlighted. 35 studies were included. Findings showed that the extant literature is highly heterogeneous. The domains of reward and punishment learning, reflection impulsivity and disadvantageous decision-making exemplify this. More homogeneity exists in domains in which (a) neurocognitive dysfunction is not apparent (motor control, cognitive/attentional flexibility and cognitive control) or (b) typical neurocognitive processes appear exacerbated by medication (reward motivation and choice impulsivity). Future large-scale neurocognitive studies are still required to develop our scientific understanding of addictive behaviours in PD and aid their clinical treatment and prediction.

Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes

Advantageous foraging choices benefit from an estimation of two aspects of a resource's value: its current desirability and availability. Both orbitofrontal and ventrolateral prefrontal areas contribute to updating these valuations, but their precise roles remain unclear. To explore their specializations, we trained macaque monkeys on two tasks: one required updating representations of a predicted outcome's desirability, as adjusted by selective satiation, and the other required updating representations of an outcome's availability, as indexed by its probability. We evaluated performance on both tasks in three groups of monkeys: unoperated controls and those with selective, fiber-sparing lesions of either the OFC or VLPFC. Representations that depend on the VLPFC but not the OFC play a necessary role in choices based on outcome availability; in contrast, representations that depend on the OFC but not the VLPFC play a necessary role in choices based on outcome desirability.

Intranasal oxytocin does not modulate jumping to conclusions in schizophrenia: Potential interactions with caudate volume and baseline social functioning

BACKGROUND

Patients with schizophrenia (SCZ) tend to sample less information when making a decision, jumping to conclusions (JTC) without sufficient evidence. This "JTC bias" may be a trait marker of the disease and may not improve with antipsychotic treatment. We conducted a double-blind, placebo-controlled trial to test whether intranasal oxytocin could reduce JTC in stable, medicated patients with SCZ and healthy controls (HCs). We also explored whether striatal volume, clinical symptoms, and baseline social functioning (SF) was related to JTC performance.

METHODS

Forty-three male, medicated SCZ patients (Mean Age: 40.81±11.44) and sixteen HCs (Mean Age: 30.38±9.85) participated in a double-blind, placebo-controlled, cross-over study. Participants completed the Beads Task on two separate visits (minimum 20days apart). Participants were randomized to receive either intranasal oxytocin (50IU in solution) or intranasal placebo (saline). Twenty of the SCZ patients and all sixteen HCs also provided T1 MRIs (3-T).

RESULTS

Patients with SCZ took fewer draws to decision (DTD) than HCs (t(57)=2.78, p=0.007). Oxytocin did not significantly change DTD in patients (t(42)=-1.11, p=0.27), nor in HCs (t(15)=-0.62, p=0.55). Exploratory analyses found ventral caudate volumes were negatively correlated with DTD (r(18)=-0.50, p=0.03) in patients. Moreover, oxytocin was more likely to improve JTC in patients with lower baseline SF. However, these exploratory findings did not survive correction for multiple comparisons.

CONCLUSIONS

We replicate increased JTC in SCZ. However, acute intranasal oxytocin did not modify JTC. Future studies with larger samples should explore how brain morphology and SF are related to JTC performance in patients with SCZ.

Decisional impulsivity and the associative-limbic subthalamic nucleus in obsessive-compulsive disorder: stimulation and connectivity

Why do we make hasty decisions for short-term gain? Rapid decision-making with limited accumulation of evidence and delay discounting are forms of decisional impulsivity. The subthalamic nucleus is implicated in inhibitory function but its role in decisional impulsivity is less well-understood. Here we assess decisional impulsivity in subjects with obsessive compulsive disorder who have undergone deep brain stimulation of the limbic and associative subthalamic nucleus. We show that stimulation of the subthalamic nucleus is causally implicated in increasing decisional impulsivity with less accumulation of evidence during probabilistic uncertainty and in enhancing delay discounting. Subthalamic stimulation shifts evidence accumulation in subjects with obsessive-compulsive disorder towards a functional less cautious style closer to that of healthy controls emphasizing its adaptive nature. Thus, subjects with obsessive compulsive disorder on subthalamic stimulation may be less likely to check for evidence (e.g. checking that the stove is on) with no difference in subjective confidence (or doubt). In a separate study, we replicate in humans (154 healthy controls) using resting state functional connectivity, tracing studies conducted in non-human primates dissociating limbic, associative and motor frontal hyper-direct connectivity with anterior and posterior subregions of the subthalamic nucleus. We show lateralization of functional connectivity of bilateral ventral striatum to right anterior ventromedial subthalamic nucleus consistent with previous observations of lateralization of emotionally evoked activity to right ventral subthalamic nucleus. We use a multi-echo sequence with independent components analysis, which has been shown to have enhanced signal-to-noise ratio, thus optimizing visualization of small subcortical structures. These findings in healthy controls converge with the effective contacts in obsessive compulsive disorder patients localized within the anterior and ventral subthalamic nucleus. We further show that evidence accumulation is associated with anterior associative-limbic subthalamic nucleus and right dorsolateral prefrontal functional connectivity in healthy controls, a region implicated in decision-making under uncertainty. Together, our findings highlight specificity of the anterior associative-limbic subthalamic nucleus in decisional impulsivity. Given increasing interest in the potential for subthalamic stimulation in psychiatric disorders and the neuropsychiatric symptoms of Parkinson’s disease, these findings have clinical implications for behavioural symptoms and cognitive effects as a function of localization of subthalamic stimulation.

Youth with substance abuse histories exhibit dysfunctional representation of expected value during a passive avoidance task

Individuals with substance abuse (SA) histories show impairment in the computations necessary for decision-making, including expected value (EV) and prediction error (PE). Neuroimaging findings, however, have been inconsistent. Sixteen youth with (SApositive) and 29 youth without (SAnegative) substance abuse histories completed a passive avoidance task while undergoing functional MRI. The groups did not significantly differ on age, gender composition or IQ. Behavioral results indicated that SApositive youth showed significantly less learning than SAnegative youth over the task. SApositive youth show problems representing EV information when attempting to avoid sub-optimal choices in bilateral inferior frontal gyrus and striatum. Furthermore, SApositive youth showed a significantly increased differential response to reward versus punishment feedback modulated by PE in posterior cingulate cortex relative to SAnegative youth. Disrupted decision-making is likely to exacerbate SA as a failure to represent EV during the avoidance of sub-optimal choices is likely to increase the likelihood of SA. With respect to the representation of PE, future work will be needed to clarify the impact of different substances on the neural systems underpinning PE representation. Moreover, interaction of age/development and substance abuse on PE signaling will need to be explored.

Impacts of religious semantic priming on an intertemporal discounting task: Response time effects and neural correlates

The purpose of this study is to test the hypothesis that religious primes would influence intertemporal discounting behaviors in neurotypical older adults, but not in participants with Parkinson's disease (PD). Furthermore, we predicted that this priming effect would be related to functional connectivity within neural networks mediating religious cognition, decision-making, reward valuing, and prospection processes. Contrary to past research with young adults, we found a significant positive relationship between religiosity and discounting rates. Religious semantic primes did not reliably shift individual discounting rates. But religious controls did respond more quickly to intertemporal decisions under the religious priming condition than the neutral condition, compared to response time differences among the participants with PD. Differences in response time were significantly associated with functional connectivity between the nucleus accumbens and various regions, including the left anterior cingulate cortex and Brodmann areas 10 and 46 in the right dorsolateral prefrontal cortex. These results suggest that religious primes influence discounting behavior via dopaminergic meso-limbic and right dorsolateral prefrontal supporting cognitive valuation and prospection processes.

Impulsivity and predictive control are associated with suboptimal action-selection and action-value learning in regular gamblers

Heightened impulsivity and cognitive biases are risk factors for gambling problems. However, little is known about precisely how these factors increase the risks of gambling-related harm in vulnerable individuals. Here, we modelled the behaviour of eighty-seven community-recruited regular, but not clinically problematic, gamblers during a binary-choice reinforcement-learning game, to characterise the relationships between impulsivity, cognitive biases, and the capacity to make optimal action selections and learn about action-values. Impulsive gamblers showed diminished use of an optimal (Bayesian-derived) probability estimate when selecting between candidate actions, and showed slower learning rates and enhanced non-linear probability weighting while learning action values. Critically, gamblers who believed that it is possible to predict winning outcomes (as 'predictive control') failed to use the game's reinforcement history to guide their action selections. Extensive evidence attests to the ease with which gamblers can erroneously perceive structure in the reinforcement history of games when there is none. Our findings demonstrate that the generic and specific risk factors of impulsivity and cognitive biases can interfere with the capacity of some gamblers to utilise structure when it is available in the reinforcement history of games, potentially increasing their risks of sustaining gambling-related harms.

Impulsivity in disorders of food and drug misuse

BACKGROUND

Evidence suggests some overlap between the pathological use of food and drugs, yet how impulsivity compares across these different clinical disorders remains unclear. Substance use disorders are commonly characterized by elevated impulsivity, and impulsivity subtypes may show commonalities and differences in various conditions. We hypothesized that obese subjects with binge-eating disorder (BED) and abstinent alcohol-dependent cohorts would have relatively more impulsive profiles compared to obese subjects without BED. We also predicted decision impulsivity impairment in obesity with and without BED.

METHOD

Thirty obese subjects with BED, 30 without BED and 30 abstinent alcohol-dependent subjects and age- and gender-matched controls were tested on delay discounting (preference for a smaller immediate reward over a larger delayed reward), reflection impulsivity (rapid decision making prior to evidence accumulation) and motor response inhibition (action cancellation of a prepotent response).

RESULTS

All three groups had greater delay discounting relative to healthy volunteers. Both obese subjects without BED and alcohol-dependent subjects had impaired motor response inhibition. Only obese subjects without BED had impaired integration of available information to optimize outcomes over later trials with a cost condition.

CONCLUSIONS

Delay discounting appears to be a common core impairment across disorders of food and drug intake. Unexpectedly, obese subjects without BED showed greater impulsivity than obese subjects with BED. We highlight the dissociability and heterogeneity of impulsivity subtypes and add to the understanding of neurocognitive profiles across disorders involving food and drugs. Our results have therapeutic implications suggesting that disorder-specific patterns of impulsivity could be targeted.

Theory of choice in bandit, information sampling and foraging tasks

Decision making has been studied with a wide array of tasks. Here we examine the theoretical structure of bandit, information sampling and foraging tasks. These tasks move beyond tasks where the choice in the current trial does not affect future expected rewards. We have modeled these tasks using Markov decision processes (MDPs). MDPs provide a general framework for modeling tasks in which decisions affect the information on which future choices will be made. Under the assumption that agents are maximizing expected rewards, MDPs provide normative solutions. We find that all three classes of tasks pose choices among actions which trade-off immediate and future expected rewards. The tasks drive these trade-offs in unique ways, however. For bandit and information sampling tasks, increasing uncertainty or the time horizon shifts value to actions that pay-off in the future. Correspondingly, decreasing uncertainty increases the relative value of actions that pay-off immediately. For foraging tasks the time-horizon plays the dominant role, as choices do not affect future uncertainty in these tasks.

Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.

Linking neuroscience with modern concepts of impulse control disorders in Parkinson's disease

Patients with Parkinson's disease (PD) may experience impulse control disorders (ICDs) when on dopamine agonist therapy for their motor symptoms. In the last few years, a rapid growth of interest for the recognition of these aberrant behaviors and their neurobiological correlates has occurred. Recent advances in neuroimaging are helping to identify the neuroanatomical networks responsible for these ICDs, and together with psychopharmacological assessments are providing new insights into the brain status of impulsive behavior. The genetic associations that may be unique to ICDs in PD are also being identified. Complementing human studies, electrophysiological and biochemical studies in animal models are providing insights into neuropathological mechanisms associated with these disorders. New animal models of ICDs in PD patients are being implemented that should provide critical means to identify efficacious therapies for PD-related motor deficits while avoiding ICD side effects. Here, we provide an overview of these recent advances, with a particular emphasis on the neurobiological correlates reported in animal models and patients along with their genetic underpinnings.

Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.

A model-based analysis of impulsivity using a slot-machine gambling paradigm

Impulsivity plays a key role in decision-making under uncertainty. It is a significant contributor to problem and pathological gambling (PG). Standard assessments of impulsivity by questionnaires, however, have various limitations, partly because impulsivity is a broad, multi-faceted concept. What remains unclear is which of these facets contribute to shaping gambling behavior. In the present study, we investigated impulsivity as expressed in a gambling setting by applying computational modeling to data from 47 healthy male volunteers who played a realistic, virtual slot-machine gambling task. Behaviorally, we found that impulsivity, as measured independently by the 11th revision of the Barratt Impulsiveness Scale (BIS-11), correlated significantly with an aggregate read-out of the following gambling responses: bet increases (BIs), machines switches (MS), casino switches (CS), and double-ups (DUs). Using model comparison, we compared a set of hierarchical Bayesian belief-updating models, i.e., the Hierarchical Gaussian Filter (HGF) and Rescorla-Wagner reinforcement learning (RL) models, with regard to how well they explained different aspects of the behavioral data. We then examined the construct validity of our winning models with multiple regression, relating subject-specific model parameter estimates to the individual BIS-11 total scores. In the most predictive model (a three-level HGF), the two free parameters encoded uncertainty-dependent mechanisms of belief updates and significantly explained BIS-11 variance across subjects. Furthermore, in this model, decision noise was a function of trial-wise uncertainty about winning probability. Collectively, our results provide a proof of concept that hierarchical Bayesian models can characterize the decision-making mechanisms linked to the impulsive traits of an individual. These novel indices of gambling mechanisms unmasked during actual play may be useful for online prevention measures for at-risk players and future assessments of PG.

Dopamine modulates novelty seeking behavior during decision making

Novelty seeking refers to the tendency of humans and animals to explore novel and unfamiliar stimuli and environments. The idea that dopamine modulates novelty seeking is supported by evidence that novel stimuli excite dopamine neurons and activate brain regions receiving dopaminergic input. In addition, dopamine is shown to drive exploratory behavior in novel environments. It is not clear whether dopamine promotes novelty seeking when it is framed as the decision to explore novel options versus the exploitation of familiar options. To test this hypothesis, we administered systemic injections of saline or GBR-12909, a selective dopamine transporter (DAT) inhibitor, to monkeys and assessed their novelty seeking behavior during a probabilistic decision making task. The task involved pseudorandom introductions of novel choice options. This allowed monkeys the opportunity to explore novel options or to exploit familiar options that they had already sampled. We found that DAT blockade increased the monkeys' preference for novel options. A reinforcement learning (RL) model fit to the monkeys' choice data showed that increased novelty seeking after DAT blockade was driven by an increase in the initial value the monkeys assigned to novel options. However, blocking DAT did not modulate the rate at which the monkeys learned which cues were most predictive of reward or their tendency to exploit that knowledge. These data demonstrate that dopamine enhances novelty-driven value and imply that excessive novelty seeking-characteristic of impulsivity and behavioral addictions-might be caused by increases in dopamine, stemming from less reuptake.

Impulsive and compulsive behaviors in Parkinson's disease

Impulsive-compulsive behaviors (ICBs) in Parkinson's disease (PD) are a common and devastating side effect of dopamine replacement therapy. In this review we describe the phenomenology, prevalence, and risk factors of patients with PD. Results of behavioral studies assessing the neuropsychological profile of patients with PD emphasize that the ICBs, which are behavioral addictions, are not hedonically motivated. Rather, other factors such as the inability to cope with uncertainty may be triggering ICBs. New insights from functional imaging studies, strengthening the incentive salience hypothesis, are discussed, and therapeutic guidelines for the management of ICBs in PD are given.

Emergence of nonmotor symptoms as the focus of research and treatment of Parkinson's disease: introduction to the special section on nonmotor dysfunctions in Parkinson's disease

Parkinson's disease (PD) is traditionally characterized by the cardinal motor symptoms of tremor, rigidity, slowness of movement, and impairments of posture, gait, and balance. A relatively new focus of research and treatment is the nonmotor symptoms of the disease, following from recent understanding of the neuropathological stages. Disruptions of arousal, mood, sleep, and autonomic function before the first motor signs of PD implicate the lower brainstem, which is affected before the substantia nigra and dopaminergic system. In later stages of the disease, the pathology extends to the cortex, accompanied by impairments in cognition and perception. The articles in this special section advance our knowledge of the brain bases of the nonmotor symptoms of PD, including disrupted visual perception, impaired cognition across a range of domains, and psychiatric and artistic manifestations. Subtypes under investigation include those described by side of disease onset (left or right body side), predominant cognitive profile, and gender. Taken together, the articles in this special section reflect the field's growing focus on the nonmotor symptoms of PD, their brain bases, and the corresponding potential for their treatment.