Constraints on decision making: Implications from genetics, personality, and addiction

Dysfunctional feedback processing in male methamphetamine abusers: Evidence from neurophysiological and computational approaches

Methamphetamine use disorder (MUD) as a major public health risk is associated with dysfunctional neural feedback processing. Although dysfunctional feedback processing in people who are substance dependent has been explored in several behavioral, computational, and electrocortical studies, this mechanism in MUDs requires to be well understood. Furthermore, the current understanding of latent components of their behavior such as learning speed and exploration-exploitation dilemma is still limited. In addition, the association between the latent cognitive components and the related neural mechanisms also needs to be explored. Therefore, in this study, the underlying neurocognitive mechanisms of feedback processing of such impairment, and age/gender-matched healthy controls are evaluated within a probabilistic learning task with rewards and punishments. Mathematical modeling results based on the Q-learning paradigm suggested that MUDs show less sensitivity in distinguishing optimal options. Additionally, it may be worth noting that MUDs exhibited a slight decrease in their ability to learn from negative feedback compared to healthy controls. Also through the lens of underlying neural mechanisms, MUDs showed lower theta power at the medial-frontal areas while responding to negative feedback. However, other EEG measures of reinforcement learning including feedback-related negativity, parietal-P300, and activity flow from the medial frontal to lateral prefrontal regions, remained intact in MUDs. On the other hand, the elimination of the linkage between value sensitivity and medial-frontal theta activity in MUDs was observed. The observed dysfunction could be due to the adverse effects of methamphetamine on the cortico-striatal dopamine circuit, which is reflected in the anterior cingulate cortex activity as the most likely region responsible for efficient behavior adjustment. These findings could help us to pave the way toward tailored therapeutic approaches.

Learning bias predicts fear acquisition under stress but not cognitive flexibility

Individuals differ in their ability to learn from reinforcement and in avoiding punishment, which can be measured by the Probabilistic Selection Task (PST). Recently, some studies have demonstrated that this learning bias is regulated by the dopaminergic system, and that stress can differentially affect the use of positive (i.e., reinforcement) and negative (i.e., avoiding punishment) feedback. The current two studies examined whether performance on the PST can predict measures of goal-directed behaviour as assessed by a cognitive flexibility task (Wisconsin Card Sorting Test) and the acquisition of fear responses, when individuals are exposed to a stressor (Socially Evaluated Cold Pressor Test). A total of 26 and 59 healthy participants completed Experiments I and II, respectively. In those who were best at learning from reinforcement, stress increased the processing (i.e., higher skin conductance responses) of non-threatening stimuli during fear acquisition compared to the non-stressful condition, which was not recapitulated in those who were best at avoiding punishment. Additionally, PST performance did not interact with stress to modulate cognitive flexibility, although stress negatively impaired this domain, consistent with previous findings. Furthermore, independent of stress, both positive and negative learning biases were correlated with cognitive flexibility errors. Our results demonstrate that the PST has predictive value for better understanding the determinants of reinforcement and avoidance learning.

Prefrontal transcranial magnetic stimulation boosts response vigour during reinforcement learning in healthy adults

A 10-Hz repetitive transcranial magnetic stimulation to the left dorsal lateral prefrontal cortex has been shown to increase dopaminergic activity in the dorsal striatum, a region strongly implicated in reinforcement learning. However, the behavioural influence of this effect remains largely unknown. We tested the causal effects of 10-Hz stimulation on behavioural and computational characteristics of reinforcement learning. A total of 40 healthy individuals were randomized into active and sham (placebo) stimulation groups. Each participant underwent one stimulation session (1500 pulses) in which stimulation was applied over the left dorsal lateral prefrontal cortex using a robotic arm. Participants then completed a reinforcement learning task sensitive to striatal dopamine functioning. Participants' choices were modelled using a reinforcement learning model (Q-learning) that calculates separate learning rates associated with positive and negative reward prediction errors. Subjects receiving active stimulation exhibited increased reward rate (number of correct responses per second of task activity) compared with those in sham. Computationally, although no group differences were observed, the active group displayed a higher learning rate for correct trials (αG) compared with incorrect trials (αL). Finally, when tested with novel pairs of stimuli, the active group displayed extremely fast reaction times, and a trend towards a higher reward rate. This study provided specific behavioural and computational accounts of altered striatal-mediated behaviour, particularly response vigour, induced by a proposed increase of dopamine activity by 10-Hz stimulation to the left dorsal lateral prefrontal cortex. Together, these findings bolster the use of repetitive transcranial magnetic stimulation to target neurocognitive disturbances attributed to the dysregulation of dopaminergic-striatal circuits.

Recovering Reliable Idiographic Biological Parameters from Noisy Behavioral Data: the Case of Basal Ganglia Indices in the Probabilistic Selection Task

Behavioral data, despite being a common index of cognitive activity, is under scrutiny for having poor reliability as a result of noise or lacking replications of reliable effects. Here, we argue that cognitive modeling can be used to enhance the test-retest reliability of the behavioral measures by recovering individual-level parameters from behavioral data. We tested this empirically with the Probabilistic Stimulus Selection (PSS) task, which is used to measure a participant’s sensitivity to positive or negative reinforcement. An analysis of 400,000 simulations from an Adaptive Control of Thought-Rational (ACT-R) model of this task showed that the poor reliability of the task is due to the instability of the end-estimates: because of the way the task works, the same participants might sometimes end up having apparently opposite scores. To recover the underlying interpretable parameters and enhance reliability, we used a Bayesian Maximum A Posteriori (MAP) procedure. We were able to obtain reliable parameters across sessions (intraclass correlation coefficient ≈ 0.5). A follow-up study on a modified version of the task also found the same pattern of results, with very poor test-retest reliability in behavior but moderate reliability in recovered parameters (intraclass correlation coefficient ≈ 0.4). Collectively, these results imply that this approach can further be used to provide superior measures in terms of reliability, and bring greater insights into individual differences.

Transition to substance use disorders: impulsivity for reward and learning from reward

Substance dependence constitutes a profound societal burden. Although large numbers of individuals use licit or illicit substances, few transition to dependence. The specific factors influencing this transition are not well understood. Substance-dependent individuals tend to be swayed by the immediate rewards of drug taking, but are often insensitive to delayed negative consequences of their behavior. Dependence is consequently associated with impulsivity for reward and atypical learning from feedback. Behavioral impulsivity is indexed using tasks measuring spontaneous decision-making and capacity to control impulses. While evidence indicates drug taking exacerbates behavioral impulsivity for reward, animal and human studies of drug naïve populations demonstrate it might precede any drug-related problems. Research suggests dependent individuals are also more likely to learn from rewarding (relative to punishing) feedback. This may partly explain why substance-dependent individuals fail to modify their behavior in response to negative outcomes. This enhanced learning from reward may constitute a further pre-existing risk factor for substance dependence. Although impulsivity for reward and preferential learning from rewarding feedback are both underpinned by a compromised dopaminergic system, few studies have examined the relationship between these two mechanisms. The interplay of these processes may help enrich understanding of why some individuals transition to substance dependence.

Predicting Attention-Shaping Response in People With Schizophrenia

ABSTRACT

People with schizophrenia often experience attentional impairments that hinder learning during psychological interventions. Attention shaping is a behavioral technique that improves attentiveness in this population. Because reinforcement learning (RL) is thought to be the mechanism by which attention shaping operates, we investigated if preshaping RL performance predicted level of response to attention shaping in people with schizophrenia. Contrary to hypotheses, a steeper attentiveness growth curve was predicted by less intact pretreatment RL ability and lower baseline attentiveness, accounting for 59% of the variance. Moreover, baseline attentiveness accounted for over 13 times more variance in response to attention shaping than did RL ability. Results suggest attention shaping is most effective for lower-functioning patients, and those high in RL ability may already be close to ceiling in terms of their response to reinforcers. Attention shaping may not be a primarily RL-driven intervention, and other mechanisms of its effects should be considered.

Neural mechanisms of affective instability and cognitive control in substance use

OBJECTIVE

We explored the impact of affect on cognitive control as this relates to individual differences in affective instability and substance use. Toward this end, we examined how different dimensions of affective instability interact to predict substance misuse and the effect of this on two event-related potential components, the reward positivity and the late positive potential, which are said to reflect the neural mechanisms of reward and emotion processing, respectively.

METHODS

We recorded the ongoing electroencephalogram from undergraduate students as they navigated two T-maze tasks in search of rewards. One of the tasks included neutral, pleasant, and unpleasant pictures from the International Affective Picture System. Participants also completed several questionnaires pertaining to substance use and personality.

RESULTS

A principal components analysis revealed a factor related to affective instability, which we named reactivity. This factor significantly predicted increased substance use. Individuals reporting higher levels of affective reactivity also displayed a larger reward positivity following stimuli with emotional content.

CONCLUSION

The current study uncovered a group of high-risk substance users who were characterized by greater levels of affective reactivity and context-specific increased sensitivity to rewards.

SIGNIFICANCE

These results help to elucidate the complex factors underlying substance use and may facilitate the creation of individually-tailored treatment programs for those struggling with substance use disorders.

Modulation of orbitofrontal-striatal reward activity by dopaminergic functional polymorphisms contributes to a predisposition to alcohol misuse in early adolescence

BACKGROUND

Abnormalities in reward circuit function are considered a core feature of addiction. Yet, it is still largely unknown whether these abnormalities stem from chronic drug use, a genetic predisposition, or both.

METHODS

In the present study, we investigated this issue using a large sample of adolescent children by applying structural equation modeling to examine the effects of several dopaminergic polymorphisms of the D1 and D2 receptor type on the reward function of the ventral striatum (VS) and orbital frontal cortex (OFC), and whether this relationship predicted the propensity to engage in early alcohol misuse behaviors at 14 years of age and again at 16 years of age.

RESULTS

The results demonstrated a regional specificity with which the functional polymorphism rs686 of the D1 dopamine receptor (DRD1) gene and Taq1A of the ANKK1 gene influenced medial and lateral OFC activation during reward anticipation, respectively. Importantly, our path model revealed a significant indirect relationship between the rs686 of the DRD1 gene and early onset of alcohol misuse through a medial OFC × VS interaction.

CONCLUSIONS

These findings highlight the role of D1 and D2 in adjusting reward-related activations within the mesocorticolimbic circuitry, as well as in the susceptibility to early onset of alcohol misuse.

Individual differences in learning from probabilistic reward and punishment predicts smoking status

INTRODUCTION

The ability to update reward and punishment contingencies is a fundamental aspect of effective decision-making, requiring the ability to successfully adapt to the changing demands of one's environment. In the case of nicotine addiction, research has predominantly focused on reward- and punishment-based learning processes among current smokers relative to non-smokers, whereas less is known about these processes in former smokers.

METHODS

In a total sample of 105 students, we used the Probabilistic Selection Task to examine differences in reinforcement learning among 41 current smokers, 29 ex-smokers, and 35 non-smokers. The PST was comprised of a training and test phase that allowed for the comparison of learning from positive versus negative feedback.

RESULTS

The test phase of the Probabilistic Selection Task significantly predicted smoking status. Current and non-smokers were classified with moderate accuracy, whereas ex-smokers were typically misclassified as smokers. Lower rates of learning from rewards were associated with an increased likelihood of being a smoker or an ex-smoker compared with being a non-smoker. Higher rates of learning from punishment were associated with an increased likelihood of being a smoker relative to non-smoker. However, learning from punishment did not predict ex-smoker status.

CONCLUSIONS

Current smokers and ex-smokers were less likely to learn from rewards, supporting the hypothesis that deficient reward processing is a feature of chronic addiction. In addition, current smokers were more sensitive to punishment than ex-smokers, contradicting some recent findings.

Meditation experience predicts negative reinforcement learning and is associated with attenuated FRN amplitude

Focused attention meditation (FAM) practices are cognitive control exercises where meditators learn to maintain focus and attention in the face of distracting stimuli. Previous studies have shown that FAM is both activating and causing plastic changes to the mesolimbic dopamine system and some of its target structures, particularly the anterior cingulate cortex (ACC) and striatum. Feedback-based learning also depends on these systems and is known to be modulated by tonic dopamine levels. Capitalizing on previous findings that FAM practices seem to cause dopamine release, the present study shows that FAM experience predicts learning from negative feedback on a probabilistic selection task. Furthermore, meditators exhibited attenuated feedback-related negativity (FRN) as compared with nonmeditators and this effect scales with meditation experience. Given that reinforcement learning and FRN are modulated by dopamine levels, a possible explanation for our findings is that FAM practice causes persistent increases in tonic dopamine levels which scale with amount of practice, thus altering feedback processing.

A Combination of Impulsivity Subdomains Predict Alcohol Intoxication Frequency

BACKGROUND

Impulsivity, broadly characterized as the tendency to act prematurely without foresight, is linked to alcohol misuse in college students. However, impulsivity is a multidimensional construct and different subdomains likely underlie different patterns of alcohol misuse. Here, we quantified the association between alcohol intoxication frequency and alcohol consumption frequency and choice, action, cognitive, and trait domains of impulsivity.

METHODS

University student drinkers (n = 106) completed a battery of demographic and alcohol-related items, as well as self-report and task-based measures indexing different facets of impulsivity. Two orthogonal latent factors, intoxication frequency and alcohol consumption frequency, were generated. Their validity was demonstrated with respect to adverse consequences of alcohol use. Machine learning with penalized regression and feature selection was then utilized to predict intoxication and alcohol consumption frequency using all impulsivity subdomains. Out-of-sample validation was used to quantify model performance.

RESULTS

Impulsivity measures alone were significant predictors of intoxication frequency, but not consumption frequency. Propensity for increased intoxication frequency was characterized by increased trait impulsivity, including the Disinhibition subscale of the Sensation Seeking Scale, Attentional and Non-planning subscales of the Barratt Impulsiveness Scale, increased task-based cognitive impulsivity (response time variability), and increased choice impulsivity (steeper delay discounting on a delay discounting questionnaire). A model combining impulsivity domains with other risk factors (gender; nicotine, cannabis, and other drug use; executive functioning; and learning processes) was also significant but did not outperform the model comprising of impulsivity alone.

CONCLUSIONS

Intoxication frequency, but not consumption frequency, was characterized by a number of impulsivity subdomains.

Effects of dopamine on reinforcement learning and consolidation in Parkinson's disease

Emerging evidence suggests that dopamine may modulate learning and memory with important implications for understanding the neurobiology of memory and future therapeutic targeting. An influential hypothesis posits that dopamine biases reinforcement learning. More recent data also suggest an influence during both consolidation and retrieval. Eighteen Parkinson's disease patients learned through feedback ON or OFF medication, with memory tested 24 hr later ON or OFF medication (4 conditions, within-subjects design with matched healthy control group). Patients OFF medication during learning decreased in memory accuracy over the following 24 hr. In contrast to previous studies, however, dopaminergic medication during learning and testing did not affect expression of positive or negative reinforcement. Two further experiments were run without the 24 hr delay, but they too failed to reproduce effects of dopaminergic medication on reinforcement learning. While supportive of a dopaminergic role in consolidation, this study failed to replicate previous findings on reinforcement learning.

Linking Individual Learning Styles to Approach-Avoidance Motivational Traits and Computational Aspects of Reinforcement Learning

Learning how to gain rewards (approach learning) and avoid punishments (avoidance learning) is fundamental for everyday life. While individual differences in approach and avoidance learning styles have been related to genetics and aging, the contribution of personality factors, such as traits, remains undetermined. Moreover, little is known about the computational mechanisms mediating differences in learning styles. Here, we used a probabilistic selection task with positive and negative feedbacks, in combination with computational modelling, to show that individuals displaying better approach (vs. avoidance) learning scored higher on measures of approach (vs. avoidance) trait motivation, but, paradoxically, also displayed reduced learning speed following positive (vs. negative) outcomes. These data suggest that learning different types of information depend on associated reward values and internal motivational drives, possibly determined by personality traits.

Reward Sensitivity of ACC as an Intermediate Phenotype between DRD4-521T and Substance Misuse

The development and expression of the midbrain dopamine system is determined in part by genetic factors that vary across individuals such that dopamine-related genes are partly responsible for addiction vulnerability. However, a complete account of how dopamine-related genes predispose individuals to drug addiction remains to be developed. Adopting an intermediate phenotype approach, we investigated whether reward-related electrophysiological activity of ACC—a cortical region said to utilize dopamine reward signals to learn the value of extended, context-specific sequences of goal-directed behaviors—mediates the influence of multiple dopamine-related functional polymorphisms over substance use. We used structural equation modeling to examine whether two related electrophysiological phenomena associated with the control and reinforcement learning functions of ACC—theta power and the reward positivity—mediated the relationship between the degree of substance misuse and genetic polymorphisms that regulate dopamine processing in frontal cortex. Substance use data were collected from 812 undergraduate students. One hundred ninety-six returned on a subsequent day to participate in an electrophysiological experiment and to provide saliva samples for DNA analysis. We found that these electrophysiological signals mediated a relationship between the DRD4-521T dopamine receptor genotype and substance misuse. Our results provide a theoretical framework that bridges the gap between genes and behavior in drug addiction and illustrate how future interventions might be individually tailored for specific genetic and neurocognitive profiles.

Hemispheric Asymmetries in Striatal Reward Responses Relate to Approach-Avoidance Learning and Encoding of Positive-Negative Prediction Errors in Dopaminergic Midbrain Regions

Some individuals are better at learning about rewarding situations, whereas others are inclined to avoid punishments (i.e., enhanced approach or avoidance learning, respectively). In reinforcement learning, action values are increased when outcomes are better than predicted (positive prediction errors [PEs]) and decreased for worse than predicted outcomes (negative PEs). Because actions with high and low values are approached and avoided, respectively, individual differences in the neural encoding of PEs may influence the balance between approach-avoidance learning. Recent correlational approaches also indicate that biases in approach-avoidance learning involve hemispheric asymmetries in dopamine function. However, the computational and neural mechanisms underpinning such learning biases remain unknown. Here we assessed hemispheric reward asymmetry in striatal activity in 34 human participants who performed a task involving rewards and punishments. We show that the relative difference in reward response between hemispheres relates to individual biases in approach-avoidance learning. Moreover, using a computational modeling approach, we demonstrate that better encoding of positive (vs negative) PEs in dopaminergic midbrain regions is associated with better approach (vs avoidance) learning, specifically in participants with larger reward responses in the left (vs right) ventral striatum. Thus, individual dispositions or traits may be determined by neural processes acting to constrain learning about specific aspects of the world.

Atypical valuation of monetary and cigarette rewards in substance dependent smokers

OBJECTIVE

Substance dependent (SD) relative to non-dependent (ND) individuals exhibit an attenuated reward positivity, an electrophysiological signal believed to index sensitivity of anterior cingulate cortex (ACC) to rewards. Here we asked whether this altered neural response reflects a specific devaluation of monetary rewards relative to drug-related rewards by ACC.

METHODS

We recorded the reward positivity from SD and ND individuals who currently smoke, following an overnight period of abstinence, while they engaged in two feedback tasks. In a money condition the feedback indicated either a monetary reward or no reward, and in a cigarette condition the feedback indicated either a drug-related reward or no reward.

RESULTS

Overall, cigarette relative to monetary rewards elicited a larger reward positivity. Further, for the subjects who engaged in the money condition first, the reward positivity was smaller for the SD compared to the ND participants, but for the subjects who engaged in the cigarette condition first, the reward positivity was larger for the SD compared to the ND participants.

CONCLUSIONS

Our results suggest that the initial category of feedback "primed" the response of the ACC to the alternative feedback type on subsequent trials, and that SD and ND individuals responded differently to this priming effect.

SIGNIFICANCE

We propose that for people who misuse addictive substances, the prospect of obtaining drug-related rewards engages the ACC to exert control over extended behaviors.