Dissociable responses to punishment in distinct striatal regions during reversal learning

Punishment Leads to Greater Sensorimotor Learning But Less Movement Variability Compared to Reward

When a musician practices a new song, hitting a correct note sounds pleasant while striking an incorrect note sounds unpleasant. Such reward and punishment feedback has been shown to differentially influence the ability to learn a new motor skill. Recent work has suggested that punishment leads to greater movement variability, which causes greater exploration and faster learning. To further test this idea, we collected 102 participants over two experiments. Unlike previous work, in Experiment 1 we found that punishment did not lead to faster learning compared to reward (n = 68), but did lead to a greater extent of learning. Surprisingly, we also found evidence to suggest that punishment led to less movement variability, which was related to the extent of learning. We then designed a second experiment that did not involve adaptation, allowing us to further isolate the influence of punishment feedback on movement variability. In Experiment 2, we again found that punishment led to significantly less movement variability compared to reward (n = 34). Collectively our results suggest that punishment feedback leads to less movement variability. Future work should investigate whether punishment feedback leads to a greater knowledge of movement variability and or increases the sensitivity of updating motor actions.

Activity in the Dorsomedial Striatum Underlies Serial Reversal Learning Performance Under Probabilistic Uncertainty

Background

Corticostriatal circuits, particularly the dorsomedial striatum (DMS) and lateral orbitofrontal cortex, are critical for navigating reversal learning under probabilistic uncertainty. These same areas are implicated in the reversal learning impairments observed in individuals with psychosis as well as their psychotic symptoms, suggesting that they may share a common neurobiological substrate. To address this question, we used psychostimulant exposure and specific activation of the DMS during reversal learning in mice to assess corticostriatal activity.

Methods

We used amphetamine treatment to induce psychosis-relevant neurobiology in male mice during reversal learning and to examine pathway-specific corticostriatal activation. To determine the causal role of DMS activity, we used chemogenetics to drive midbrain inputs during a range of probabilistic contingencies.

Results

Mice treated with amphetamine showed altered punishment learning, which was associated with decreased shifting after losses and increased perseverative errors after reversals. Reversal learning performance and strategies were dependent on increased activity in lateral orbitofrontal cortex to DMS circuits as well as in the DMS itself. Specific activation of midbrain to DMS circuits also decreased shifting after losses and reversal learning performance. However, these alterations were dependent on the probabilistic contingency.

Conclusions

Our work suggests that the DMS plays a multifaceted role in reversal learning. Increasing DMS activity impairs multiple reversal learning processes dependent on the level of uncertainty, confirming its role in the maintenance and selection of incoming cortical inputs. Together, these outcomes suggest that elevated dopamine levels in the DMS could contribute to decision-making impairments in individuals with psychosis.

Dissociable feedback valence effects on frontal midline theta during reward gain versus threat avoidance learning

While frontal midline theta (FMθ) has been associated with threat processing, with cognitive control in the context of anxiety, and with reinforcement learning, most reinforcement learning studies on FMθ have used reward rather than threat-related stimuli as reinforcer. Accordingly, the role of FMθ in threat-related reinforcement learning is largely unknown. Here, n = 23 human participants underwent one reward-, and one punishment-, based reversal learning task, which differed only with regard to the kind of reinforcers that feedback was tied to (i.e., monetary gain vs. loud noise burst, respectively). In addition to single-trial EEG, we assessed single-trial feedback expectations based on both a reinforcement learning computational model and trial-by-trial subjective feedback expectation ratings. While participants' performance and feedback expectations were comparable between the reward and punishment tasks, FMθ was more reliably amplified to negative vs. positive feedback in the reward vs. punishment task. Regressions with feedback valence, computationally derived, and self-reported expectations as predictors and FMθ as criterion further revealed that trial-by-trial variations in FMθ specifically relate to reward-related feedback-valence and not to threat-related feedback or to violated expectations/prediction errors. These findings suggest that FMθ as measured in reinforcement learning tasks may be less sensitive to the processing of events with direct relevance for fear and anxiety.

Cognitive [Computational] Neuroscience Test Reliability and Clinical Applications for Serious Mental Illness (CNTRaCS) Consortium: Progress and Future Directions

The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.

Neural Responsivity to Reward versus Punishment Shortly after Trauma Predicts Long-term Development of Post-Traumatic Stress Symptoms

BACKGROUND

Processing negative and positive valenced stimuli involve multiple brain regions including the amygdala and ventral striatum (VS). Post-Traumatic Stress Disorder (PTSD) is often associated with hyper-responsivity to negatively valenced, yet recent evidence also points to deficient positive valence functioning. It is yet unclear what is the relative contribution of such opposing valence processing shortly after trauma to the development of chronic PTSD.

METHODS

Neurobehavioral indicators of motivational positive vs. negative valence sensitivities were longitudinally assessed in 171 adults (87 females, age=34.19±11.47 years) at 1-, 6-, and 14-months following trauma exposure (TP1, TP2, TP3). Using a gambling fMRI paradigm, amygdala and VS functionality (activity and functional connectivity with the prefrontal cortex) in response to rewards vs. punishments were assessed with relation to PTSD severity at different time-points. The effect of valence processing was depicted behaviorally by the amount of risk taken to maximize reward.

RESULTS

PTSD severity at TP1 was associated with greater neural functionality in the amygdala (but not the VS) towards punishments vs. rewards, and fewer risky choices. PTSD severity at TP3 was associated with decreased neural functionality in both the VS and amygdala towards rewards vs. punishments at TP1 (but not with risky behavior). Explainable machine learning revealed the primacy of VS biased processing, over the amygdala, in predicting PTSD severity at TP3.

CONCLUSIONS

These results highlight the importance of biased neural responsivity to positive relative to negative motivational outcomes in PTSD development. Novel therapeutic strategies early after trauma may thus target both valence fronts.

Cortical hemodynamic mechanisms of reversal learning using high-resolution functional near-infrared spectroscopy: A pilot study

OBJECTIVES

Reversal learning is widely used to analyze cognitive flexibility and characterize behavioral abnormalities associated with impulsivity and disinhibition. Recent studies using fMRI have focused on regions involved in reversal learning with negative and positive reinforcers. Although the frontal cortex has been consistently implicated in reversal learning, few studies have focused on whether reward and punishment may have different effects on lateral frontal structures in these tasks.

METHODS

During this pilot study on eight healthy subjects, we used functional near infra-red spectroscopy (fNIRS) to characterize brain activity dynamics and differentiate the involvement of frontal structures in learning driven by reward and punishment.

RESULTS

We observed functional hemispheric asymmetries between punishment and reward processing by fNIRS following reversal of a learned rule. Moreover, the left dorsolateral prefrontal cortex (l-DLPFC) and inferior frontal gyrus (IFG) were activated under the reward condition only, whereas the orbito-frontal cortex (OFC) was significantly activated under the punishment condition, with a tendency towards activation for the right cortical hemisphere (r-DLPFC and r-IFG). Our results are compatible with the suggestion that the DLPFC is involved in the detection of contingency change. We propose a new representation for reward and punishment, with left lateralization for the reward process.

CONCLUSIONS

The results of this pilot study provide insights into the indirect neural mechanisms of reversal learning and behavioral flexibility and confirm the use of fNIRS imaging in reversal-learning tasks as a translational strategy, particularly in subjects who cannot undergo fMRI recordings.

Morphine exposure alters Fos expression in a sex-, age-, and brain region-specific manner during adolescence

Data in both humans and preclinical animal models clearly indicate drug exposure during adolescence, when the "reward" circuitry of the brain develops, increases the risk of substance use and other mental health disorders later in life. Human data indicate that different neural and behavioral sequelae can be observed in early versus late adolescence. However, most studies with rodent models examine a single adolescent age compared to a mature adult age, and often only in males. Herein, we sought to determine whether the acute response to the opioid morphine would also differ across adolescence, and by sex. By quantifying Fos positive cells, a proxy for neural activity, at different stages during adolescence (pre-, early, mid-, and late adolescence) and in multiple reward regions (prefrontal cortex, nucleus accumbens, caudate/putamen), we determined that the neural response to acute morphine is highly dependent on adolescent age, sex, and brain region. These data suggest that heterogeneity in the consequences of adolescent opioid exposure may be due to age- and sex-specific developmental profiles in individual reward processing regions. In future studies, it will be important to add age within adolescence as an independent variable for a holistic view of healthy or abnormal reward-related neural development.

Impaired probabilistic reversal learning in anxiety: Evidence from behavioral and ERP findings

•

High and low anxious participants finish a probabilistic reversal learning task.

•

High anxious participants showed a worse performance and less likely to lose-shift.

•

Feedback-related negativity (FRN) was correlated with the frequency of lose-shift.

•

High anxious participants showed a smaller FRN in response to lose-shift.

•

Anxious people’s reversal learning is affected by impaired sensitivity to losses.

Background

Reversal learning reflects an individual’s capacity to adapt to a dynamic environment with changing stimulus–reward contingencies. This study focuses on the potential influence of anxiety on reversal learning skills.

Methods

We asked 40 participants with a high level of trait anxiety (HTA) and 40 counterparts with a low anxiety level (LTA) to finish a probabilistic reversal learning task with event-related potential (ERP) recording, during which stimulus–reward contingencies are reversed after players have learned the optimal choice.

Results

We found that compared to their LTA counterparts, the HTA participants showed worse learning performance and were less likely to make lose-shift choices. The FRN amplitude might help interpret these behavioral results, which is suggested to be associated with punishment sensitivity and was positively correlated with the number of lose-shift in this study. Seeing that anxiety level predicted the FRN amplitude for lose-shift, we explain that anxious individuals’ inflexible behavioral responses to losses are due to their impaired sensitivity to negative feedback.

Conclusions

A higher level of anxiety is associated with weaker reversal learning performance, possibly because of abnormal sensitivity to negative outcomes. These findings have implications for the understanding of behavioral symptoms in anxiety.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

From Freedom From to Freedom To: New Perspectives on Intentional Action

There are few concepts as relevant as that of intentional action in shaping our sense of self and the interaction with the environment. At the same time, few concepts are so elusive. Indeed, both conceptual and neuroscientific accounts of intentional agency have proven to be problematic. On the one hand, most conceptual views struggle in defining how agents can adequately exert control over their actions. On the other hand, neuroscience settles for definitions by exclusion whereby key features of human intentional actions, including goal-directness, remain underspecified. This paper reviews the existing literature and sketches how this gap might be filled. In particular, we defend a gradualist notion of intentional behavior, which revolves around the following key features: autonomy, flexibility in the integration of causal vectors, and control.

The uncertain brain: A co-ordinate based meta-analysis of the neural signatures supporting uncertainty during different contexts

Uncertainty is often inevitable in everyday life and can be both stressful and exciting. Given its relevance to psychopathology and wellbeing, recent research has begun to address the brain basis of uncertainty. In the current review we examined whether there are discrete and shared neural signatures for different uncertain contexts. From the literature we identified three broad categories of uncertainty currently empirically studied using functional MRI (fMRI): basic threat and reward uncertainty, decision-making under uncertainty, and associative learning under uncertainty. We examined the neural basis of each category by using a coordinate based meta-analysis, where brain activation foci from previously published fMRI experiments were drawn together (1998-2017; 87 studies). The analyses revealed shared and discrete patterns of neural activation for uncertainty, such as the insula and amygdala, depending on the category. Such findings will have relevance for researchers attempting to conceptualise uncertainty, as well as clinical researchers examining the neural basis of uncertainty in relation to psychopathology.

Learning and Choice in Mood Disorders: Searching for the Computational Parameters of Anhedonia

Computational approaches are increasingly being used to model behavioral and neural processes in mood and anxiety disorders. Here we explore the extent to which the parameters of popular learning and decision-making models are implicated in anhedonic symptoms of major depression. We first highlight the parameters of reinforcement learning that have been implicated in anhedonia, focusing, in particular, on the role that choice variability (i.e., "temperature") may play in explaining heterogeneity across previous findings. We then turn to neuroimaging findings implicating attenuated ventral striatum response in anhedonic responses and discuss possible causes of the heterogeneity in the literature. Taken together, the reviewed findings highlight the potential of the computational approach in teasing apart the observed heterogeneity in both behavioral and functional imaging results. Nevertheless, considerable challenges remain, and we conclude with five unresolved questions that seek to address issues highlighted by the reviewed data.

Cardiac Concomitants of Feedback and Prediction Error Processing in Reinforcement Learning

Successful learning hinges on the evaluation of positive and negative feedback. We assessed differential learning from reward and punishment in a monetary reinforcement learning paradigm, together with cardiac concomitants of positive and negative feedback processing. On the behavioral level, learning from reward resulted in more advantageous behavior than learning from punishment, suggesting a differential impact of reward and punishment on successful feedback-based learning. On the autonomic level, learning and feedback processing were closely mirrored by phasic cardiac responses on a trial-by-trial basis: (1) Negative feedback was accompanied by faster and prolonged heart rate deceleration compared to positive feedback. (2) Cardiac responses shifted from feedback presentation at the beginning of learning to stimulus presentation later on. (3) Most importantly, the strength of phasic cardiac responses to the presentation of feedback correlated with the strength of prediction error signals that alert the learner to the necessity for behavioral adaptation. Considering participants' weight status and gender revealed obesity-related deficits in learning to avoid negative consequences and less consistent behavioral adaptation in women compared to men. In sum, our results provide strong new evidence for the notion that during learning phasic cardiac responses reflect an internal value and feedback monitoring system that is sensitive to the violation of performance-based expectations. Moreover, inter-individual differences in weight status and gender may affect both behavioral and autonomic responses in reinforcement-based learning.

Reward learning deficits in Parkinson's disease depend on depression

BACKGROUND

Depression is one of the most common and debilitating non-motor symptoms of Parkinson's disease (PD). The neurocognitive mechanisms underlying depression in PD are unclear and treatment is often suboptimal.

METHODS

We investigated the role of striatal dopamine in reversal learning from reward and punishment by combining a controlled medication withdrawal procedure with functional magnetic resonance imaging in 22 non-depressed PD patients and 19 PD patients with past or present depression.

RESULTS

PD patients with a depression (history) exhibited impaired reward v. punishment reversal learning as well as reduced reward v. punishment-related BOLD signal in the striatum (putamen) compared with non-depressed PD patients. No effects of dopaminergic medication were observed.

CONCLUSIONS

The present findings demonstrate that impairments in reversal learning from reward v. punishment and associated striatal signalling depend on the presence of (a history of) depression in PD.

Processing of Positive and Negative Feedback in Patients with Cerebellar Lesions

It is well accepted that the cerebellum plays a crucial role in the prediction of the sensory consequences of movements. Recent findings of altered error processing in patients with selective cerebellar lesions led to the hypothesis that feedback processing and feedback-based learning might be affected by cerebellar damage as well. Thus, the present study investigated learning from and processing of positive and negative feedback in 12 patients with selective cerebellar lesions and healthy control subjects. Participants performed a monetary feedback learning task. The processing of positive and negative feedback was assessed by means of event-related potentials (ERPs) during the learning task and during a separate task in which the frequencies of positive and negative feedback were balanced. Patients did not show a general learning deficit compared to controls. Relative to the control group, however, patients with cerebellar lesions showed significantly higher ERP difference wave amplitudes (rewards-losses) in a time window between 250 and 450 ms after feedback presentation, possibly indicating impaired outcome prediction. The analysis of the original waveforms suggested that patients and controls primarily differed in their pattern of feedback-related negativity and P300 amplitudes. Our results add to recent findings on altered performance monitoring associated with cerebellar damage and demonstrate, for the first time, alterations of feedback processing in patients with cerebellar damage. Unaffected learning performance appears to suggest that chronic cerebellar lesions can be compensated in behaviour.

Striatal dopamine D1 receptor suppression impairs reward-associative learning

Dopamine (DA) is required for reinforcement learning. Hence, disruptions in DA signaling may contribute to the learning deficits associated with psychiatric disorders. The DA D1 receptor (D1R) has been linked to learning and is a target for cognitive/motivational enhancement in patients with schizophrenia. Separating the striatal D1R contribution to learning vs. motivation, however, has been challenging. We suppressed striatal D1R expression in mice using a D1R-targeting short hairpin RNA (shRNA), delivered locally to the striatum via an adeno-associated virus (AAV). We then assessed reward- and punishment-associative learning using a probabilistic learning task and motivation using a progressive-ratio breakpoint procedure. We confirmed suppression of striatal D1Rs immunohistochemically and by testing locomotor activity after the administration of (+)-doxanthrine, a full D1R agonist, in control mice and those treated with the D1RshRNA. D1RshRNA-treated mice exhibited impaired reward-associative learning, while punishment-associative learning was spared. This deficit was unrelated to general learning impairments or amotivation, because the D1shRNA-treated mice exhibited normal Barnes maze learning and normal motivation in the progressive-ratio breakpoint procedure. Suppression of striatal D1Rs selectively impaired reward-associative learning whereas punishment-associative learning, aversion-motivated learning, and appetitive motivation were spared. Because patients with schizophrenia exhibit similar reward-associative learning deficits, D1R-targeted treatments should be investigated to improve reward learning in these patients.

Cognitive emotion regulation modulates the balance of competing influences on ventral striatal aversive prediction error signals

Cognitive emotion regulation (CER) is a critical human ability to face aversive emotional stimuli in a flexible way, via recruitment of specific prefrontal brain circuits. Animal research reveals a central role of ventral striatum in emotional behavior, for both aversive conditioning, with striatum signaling aversive prediction errors (aPE), and for integrating competing influences of distinct striatal inputs from regions such as the prefrontal cortex (PFC), amygdala, hippocampus and ventral tegmental area (VTA). Translating these ventral striatal findings from animal research to human CER, we hypothesized that successful CER would affect the balance of competing influences of striatal afferents on striatal aPE signals, in a way favoring PFC as opposed to 'subcortical' (i.e., non-isocortical) striatal inputs. Using aversive Pavlovian conditioning with and without CER during fMRI, we found that during CER, superior regulators indeed reduced the modulatory impact of 'subcortical' striatal afferents (hippocampus, amygdala and VTA) on ventral striatal aPE signals, while keeping the PFC impact intact. In contrast, inferior regulators showed an opposite pattern. Our results demonstrate that ventral striatal aPE signals and associated competing modulatory inputs are critical mechanisms underlying successful cognitive regulation of aversive emotions in humans.

Motivational Context Modulates Prediction Error Response in Schizophrenia

BACKGROUND

Recent findings demonstrate that patients with schizophrenia are worse at learning to predict rewards than losses, suggesting that motivational context modulates learning in this disease. However, these findings derive from studies in patients treated with antipsychotic medications, D2 receptor antagonists that may interfere with the neural systems that underlie motivation and learning. Thus, it remains unknown how motivational context affects learning in schizophrenia, separate from the effects of medication.

METHODS

To examine the impact of motivational context on learning in schizophrenia, we tested 16 unmedicated patients with schizophrenia and 23 matched controls on a probabilistic learning task while they underwent functional magnetic resonance imaging (fMRI) under 2 conditions: one in which they pursued rewards, and one in which they avoided losses. Computational models were used to derive trial-by-trial prediction error responses to feedback.

RESULTS

Patients performed worse than controls on the learning task overall, but there were no behavioral effects of condition. FMRI revealed an attenuated prediction error response in patients in the medial prefrontal cortex, striatum, and medial temporal lobe when learning to predict rewards, but not when learning to avoid losses.

CONCLUSIONS

Patients with schizophrenia showed differences in learning-related brain activity when learning to predict rewards, but not when learning to avoid losses. Together with prior work, these results suggest that motivational deficits related to learning in schizophrenia are characteristic of the disease and not solely a result of antipsychotic treatment.

Evidence that Illness-Compatible Cues Are Rewarding in Women Recovered from Anorexia Nervosa: A Study of the Effects of Dopamine Depletion on Eye-Blink Startle Responses

In anorexia nervosa (AN), motivational salience is attributed to illness-compatible cues (e.g., underweight and active female bodies) and this is hypothesised to involve dopaminergic reward circuitry. We investigated the effects of reducing dopamine (DA) transmission on the motivational processing of AN-compatible cues in women recovered from AN (AN REC, n = 17) and healthy controls (HC, n = 15). This involved the acute phenylalanine and tyrosine depletion (APTD) procedure and a startle eye-blink modulation (SEM) task. In a balanced amino acid state, AN REC showed an increased appetitive response (decreased startle potentiation) to illness-compatible cues (underweight and active female body pictures (relative to neutral and non-active cues, respectively)). The HC had an aversive response (increased startle potentiation) to the same illness-compatible stimuli (relative to neutral cues). Importantly, these effects, which may be taken to resemble symptoms observed in the acute stage of illness and healthy behaviour respectively, were not present when DA was depleted. Thus, AN REC implicitly appraised underweight and exercise cues as more rewarding than did HC and the process may, in part, be DA-dependent. It is proposed that the positive motivational salience attributed to cues of emaciation and physical activity is, in part, mediated by dopaminergic reward processes and this contributes to illness pathology. These observations are consistent with the proposal that, in AN, aberrant reward-based learning contributes to the development of habituation of AN-compatible behaviours.

Anticipatory reward processing among cocaine-dependent individuals with and without concurrent methadone-maintenance treatment: Relationship to treatment response

BACKGROUND

Cocaine dependence among opioid-dependent methadone-maintained individuals is a significant public health problem and is particularly challenging to treat. The neurobiology of this clinically complex population has not been previously assessed using fMRI.

METHODS

fMRI data from cocaine-dependent, methadone-maintained (CD-MM) patients (n=24), cocaine-dependent (CD) patients (n=20) and healthy comparison (HC) participants (n=21) were acquired during monetary incentive delay task performance. All patients were scanned prior to treatment for cocaine dependence. Between-group differences in anticipatory reward and loss processing were assessed using whole-brain ANOVAs in SPM12 (pFWE<0.05). Correlations between durations of abstinence during treatment and BOLD responses within the insula and caudate were also explored.

RESULTS

Main effects of diagnostic group, primarily involving decreased BOLD responses among CD-MM patients in comparison to HCs, were observed during anticipatory reward and loss processing within regions of posterior cingulate cortex, precuneus, inferior frontal gyrus and dorsolateral prefrontal cortex. BOLD responses within the right caudate were negatively associated with percentage of cocaine-negative urines during treatment among CD-MM patients, but not among non-methadone-maintained CD patients.

CONCLUSIONS

These data suggest neurofunctional differences that may be related to treatment outcomes for behavioral therapies between cocaine-dependent individuals with and without methadone-maintenance treatment. These findings may relate to differences in treatment efficacies and to the elevated relapse rates observed in methadone-maintained populations.

Compulsivity Across the Pathological Misuse of Drug and Non-Drug Rewards

Behavioral adaptation is required for the successful navigation of a constantly changing environment. Impairments in behavioral flexibility are commonly observed in psychiatric disorders including those of addiction. This study investigates two distinct facets of compulsivity, namely reversal learning and attentional set shifting, implicating orbitofrontal and lateral prefrontal regions respectively, across disorders of primary and secondary rewards. Obese subjects with and without binge eating disorder (BED), individuals with compulsive sexual behaviors (CSB), alcohol dependence (AD) and pathological video-gaming (VG) were tested with two computerized tasks: the probabilistic reversal task (trials to criterion and win-stay/lose-shift errors) and the intra/extra-dimensional set shift task (IED). Individuals with AD and pathological VG were slower at reversal learning irrespective of valence, with AD subjects more likely to perseverate after losses. Compared to obese subjects without BED, BED subjects were worse at reversal learning to wins but better at losses highlighting valence effects as a function of binge eating. CSB subjects demonstrated enhanced sensitivity to reward outcomes with faster acquisition and greater perseveration with higher magnitude rewards. We further show an impairment in attentional set shifting in individuals with BED and AD relative to healthy volunteers (HV). This study provides evidence for commonalities and differences in two distinct dimensions of behavioral inflexibility across disorders of compulsivity. We summarize studies on compulsivity subtypes within this same patient population. We emphasize commonalities in AD and BED with impairments across a range of compulsivity indices, perhaps supporting pathological binge eating as a form of behavioral addiction. We further emphasize commonalities in reversal learning across disorders and the crucial role of valence effects. These findings highlight the role of behavioral inflexibility and compulsivity as a relevant domain in defining dimensional psychiatry and the identification of relevant cognitive endophenotypes as targets for therapeutic modulation.

Rewarding feedback promotes motor skill consolidation via striatal activity

Knowledge of performance can activate the striatum, a key region of the reward system and highly relevant for motivated behavior. Using functional magnetic resonance imaging, striatal activity linked to knowledge of performance was measured during the training of a repetitive arc-tracking task. Knowledge of performance was given after a random selection of trials or after good performance. The third group received knowledge of performance after good performance plus a monetary reward. Skill learning was measured from pre- to post- (acquisition) and from post- to 24h posttraining (consolidation). Our results demonstrate an influence of feedback on motor skill learning. Adding a monetary reward after good performance leads to better consolidation and higher ventral striatal activation than knowledge of performance alone. In turn, rewarding strategies that increase ventral striatal response during training of a motor skill may be utilized to improve skill consolidation.

Levodopa impairs probabilistic reversal learning in healthy young adults

RATIONALE

Dopaminergic therapy improves some cognitive functions and worsens others in patients with Parkinson's disease (PD). These paradoxical effects are explained by the dopamine overdose hypothesis, which proposes that effects of dopaminergic therapy on a cognitive function is determined by the baseline dopamine levels in brain regions mediating that function.

OBJECTIVES

We directly tested this prevalent hypothesis, evaluating the effects of levodopa on stimulus-reward learning in healthy young adults, who presumably have optimal baseline dopamine levels and dopamine regulation.

METHODS

Twenty-six healthy, young adults completed a probabilistic reversal learning task in a randomized, double-blind, placebo-controlled, crossover design. Participants completed one session on levodopa 100 mg/carbidopa 25 mg and another session on placebo.

RESULTS

We found that levodopa impaired reversal learning relative to placebo. Further analyses revealed that levodopa impaired learning from both punishment and reward.

CONCLUSIONS

Exogenous dopamine impairs stimulus-reward learning, independent of PD pathology and prior to sensitization through repeated exposure, in healthy adults with normal cognition and baseline dopamine function. Our findings support the dopamine overdose hypothesis and caution clinicians about detrimental effects of levodopa in all clinical populations (e.g., early PD, restless leg syndrome) regardless of baseline cognitive and dopaminergic system function.

The neural underpinnings of cognitive flexibility and their disruption in psychotic illness

Schizophrenia (SZ) has long been associated with a variety of cognitive deficits, including reduced cognitive flexibility. More recent findings, however, point to tremendous inter-individual variability among patients on measures of cognitive flexibility/set-shifting. With an eye toward shedding light on potential sources of variability in set-shifting abilities among SZ patients, I examine the neural substrates of underlying probabilistic reversal learning (PRL) - a paradigmatic measure of cognitive flexibility - as well as neuromodulatory influences upon these systems. Finally, I report on behavioral and neuroimaging studies of PRL in SZ patients, discussing the potentially influences of illness profile and antipsychotic medications on cognitive flexibility in SZ.

The temporal dynamics of reversal learning: P3 amplitude predicts valence-specific behavioral adjustment

Adapting behavior to dynamic stimulus-reward contingences is a core feature of reversal learning and a capacity thought to be critical to socio-emotional behavior. Impairment in reversal learning has been linked to multiple psychiatric outcomes, including depression, Parkinson's disorder, and substance abuse. A recent influential study introduced an innovative laboratory reversal-learning paradigm capable of disentangling the roles of feedback valence and expectancy. Here, we sought to use this paradigm in order to examine the time-course of reward and punishment learning using event-related potentials among a large, representative sample (N=101). Three distinct phases of processing were examined: initial feedback evaluation (reward positivity, or RewP), allocation of attention (P3), and sustained processing (late positive potential, or LPP). Results indicate a differential pattern of valence and expectancy across these processing stages: the RewP was uniquely related to valence (i.e., positive vs. negative feedback), the P3 was uniquely associated with expectancy (i.e., unexpected vs. expected feedback), and the LPP was sensitive to both valence and expectancy (i.e., main effects of each, but no interaction). The link between ERP amplitudes and behavioral performance was strongest for the P3, and this association was valence-specific. Overall, these findings highlight the potential utility of the P3 as a neural marker for feedback processing in reversal-based learning and establish a foundation for future research in clinical populations.

Cognitive Translation Using the Rodent Touchscreen Testing Approach

The development of novel therapeutic avenues for the treatment of cognitive deficits in psychiatric and neurodegenerative disease is of high importance, yet progress in this field has been slow. One reason for this lack of success may lie in discrepancies between how cognitive functions are assessed in experimental animals and humans. In an attempt to bridge this translational gap, the rodent touchscreen testing platform is suggested as a translational tool. Specific examples of successful cross-species translation are discussed focusing on paired associate learning (PAL), the 5-choice serial reaction time task (5-CSRTT), the rodent continuous performance task (rCPT) and reversal learning. With ongoing research assessing the neurocognitive validity of tasks, the touchscreen approach is likely to become increasingly prevalent in translational cognitive research.

Cognitive impairment in a young marmoset reveals lateral ventriculomegaly and a mild hippocampal atrophy: a case report

The number of studies that use the common marmoset (Callithrix jacchus) in various fields of neurosciences is increasing dramatically. In general, animals enter the study when their health status is considered satisfactory on the basis of classical clinical investigations. In behavioral studies, variations of score between individuals are frequently observed, some of them being considered as poor performers or outliers. Experimenters rarely consider the fact that it could be related to some brain anomaly. This raises the important issue of the reliability of such classical behavioral approaches without using complementary imaging, especially in animals lacking striking external clinical signs. Here we report the case of a young marmoset which presented a set of cognitive impairments in two different tasks compared to other age-matched animals. Brain imaging revealed a patent right lateral ventricular enlargement with a mild hippocampal atrophy. This abnormality could explain the cognitive impairments of this animal. Such a case points to the importance of complementing behavioral studies by imaging explorations to avoid experimental bias.

Valence-separated representation of reward prediction error in feedback-related negativity and positivity

Feedback-related negativity (FRN) is an event-related brain potential (ERP) component elicited by errors and negative outcomes. Previous studies proposed that FRN reflects the activity of a general error-processing system that incorporates reward prediction error (RPE). However, other studies reported inconsistent results on this issue - namely, that FRN only reflects the valence of feedback and that the magnitude of RPE is reflected by the other ERP component called P300. The present study focused on the relationship between the FRN amplitude and RPE. ERPs were recorded during a reversal learning task performed by the participants, and a computational model was used to estimate trial-by-trial RPEs, which we correlated with the ERPs. The results indicated that FRN and P300 reflected the magnitude of RPE in negative outcomes and positive outcomes, respectively. In addition, the correlation between RPE and the P300 amplitude was stronger than the correlation between RPE and the FRN amplitude. These differences in the correlation between ERP and RPE components may explain the inconsistent results reported by previous studies; the asymmetry in the correlations might make it difficult to detect the effect of the RPE magnitude on the FRN and makes it appear that the FRN only reflects the valence of feedback.

Cognitive emotion regulation enhances aversive prediction error activity while reducing emotional responses

Cognitive emotion regulation is a powerful way of modulating emotional responses. However, despite the vital role of emotions in learning, it is unknown whether the effect of cognitive emotion regulation also extends to the modulation of learning. Computational models indicate prediction error activity, typically observed in the striatum and ventral tegmental area, as a critical neural mechanism involved in associative learning. We used model-based fMRI during aversive conditioning with and without cognitive emotion regulation to test the hypothesis that emotion regulation would affect prediction error-related neural activity in the striatum and ventral tegmental area, reflecting an emotion regulation-related modulation of learning. Our results show that cognitive emotion regulation reduced emotion-related brain activity, but increased prediction error-related activity in a network involving ventral tegmental area, hippocampus, insula and ventral striatum. While the reduction of response activity was related to behavioral measures of emotion regulation success, the enhancement of prediction error-related neural activity was related to learning performance. Furthermore, functional connectivity between the ventral tegmental area and ventrolateral prefrontal cortex, an area involved in regulation, was specifically increased during emotion regulation and likewise related to learning performance. Our data, therefore, provide first-time evidence that beyond reducing emotional responses, cognitive emotion regulation affects learning by enhancing prediction error-related activity, potentially via tegmental dopaminergic pathways.

The rat's not for turning: Dissociating the psychological components of cognitive inflexibility

•

Non-rewarded or irrelevant prior associations are important for flexible responding.

•

Associations of reward and non-reward in reversal learning are neurally dissociable.

•

Disruption of prior irrelevant or rewarded associations cause pathological deficits.

•

Experimental paradigms of cognitive flexibility can be improved to aid translation.

Executive function is commonly assessed by assays of cognitive flexibility such as reversal learning and attentional set-shifting. Disrupted performance in these assays, apparent in many neuropsychiatric disorders, is frequently interpreted as inability to overcome prior associations with reward.

However, non-rewarded or irrelevant associations may be of considerable importance in both discrimination learning and cognitive flexibility. Non-rewarded associations can have greater influence on choice behaviour than rewarded associations in discrimination learning. Pathology-related deficits in cognitive flexibility can produce selective disruptions to both the processing of irrelevant associations and associations with reward. Genetic and pharmacological animal models demonstrate that modulation of reversal learning may result from alterations in either rewarded or non-rewarded associations.

Successful performance in assays of cognitive flexibility can therefore depend on a combination of rewarded, non-rewarded, and irrelevant associations derived from previous learning, accounting for some inconsistencies observed in the literature. Taking this combination into account may increase the validity of animal models and may also reveal pathology-specific differences in problem solving and executive function.

Dopamine, Salience, and Response Set Shifting in Prefrontal Cortex

Dopamine is implicated in multiple functions, including motor execution, action learning for hedonically salient outcomes, maintenance, and switching of behavioral response set. Here, we used a novel within-subject psychopharmacological and combined functional neuroimaging paradigm, investigating the interaction between hedonic salience, dopamine, and response set shifting, distinct from effects on action learning or motor execution. We asked whether behavioral performance in response set shifting depends on the hedonic salience of reversal cues, by presenting these as null (neutral) or salient (monetary loss) outcomes. We observed marked effects of reversal cue salience on set-switching, with more efficient reversals following salient loss outcomes. L-Dopa degraded this discrimination, leading to inappropriate perseveration. Generic activation in thalamus, insula, and striatum preceded response set switches, with an opposite pattern in ventromedial prefrontal cortex (vmPFC). However, the behavioral effect of hedonic salience was reflected in differential vmPFC deactivation following salient relative to null reversal cues. l-Dopa reversed this pattern in vmPFC, suggesting that its behavioral effects are due to disruption of the stability and switching of firing patterns in prefrontal cortex. Our findings provide a potential neurobiological explanation for paradoxical phenomena, including maintenance of behavioral set despite negative outcomes, seen in impulse control disorders in Parkinson's disease.

Persistent cocaine-induced reversal learning deficits are associated with altered limbic cortico-striatal local field potential synchronization

Repeated exposure to cocaine is known to produce persistent deficits in behavioral flexibility. Evidence suggests that these deficits are mediated in part by a circuit involving the medial prefrontal and orbitofrontal cortices (PFC and OFC), nucleus accumbens (NAC), and basolateral amygdala (BLA). To assess the effects of cocaine on this circuit, we treated rats with cocaine daily for 14 d, followed by 4 weeks of abstinence. Animals were then tested on a cross-maze-based reversal learning and set-shifting task, after which they were anesthetized to allow for recording of spontaneous local field potential (LFP) activity simultaneously from all four regions, in addition to activity evoked from acute BLA stimulation. Cocaine-treated (COC) animals showed specific deficits in reversal learning; furthermore, spontaneous LFP oscillation power was reduced and BLA-induced oscillation power was increased in all regions compared with saline-treated (SAL) rats. Theta-burst stimulation of BLA potentiated BLA-evoked responses in all regions and cocaine challenge reduced spontaneous oscillation power and evoked response amplitude, with no COC/SAL group differences. Notably, cocaine challenge produced differential changes in coherence between OFC-BLA, BLA-NAC, and OFC-NAC in COC and SAL groups. These data indicate that repeated exposure to cocaine can produce changes in oscillatory LFP synchronization along limbic cortico-striatal circuits that persist long into abstinence. Furthermore, the regional specificity of these changes strongly correlates with the observed behavioral deficits. Aberrant synchronization within and between regions and consequent dysregulation of the neurocircuitry involved in executive control may contribute to the long-lasting maladaptive decision making seen in cocaine abusers.

Common neural mechanisms underlying reversal learning by reward and punishment

Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.

Aversive stimuli and loss in the mesocorticolimbic dopamine system

There is mounting evidence that the mesolimbic dopamine system carries valuation signals not only for appetitive or gain-related stimuli, with which it is traditionally associated, but also for aversive and loss-related stimuli. Cellular-level studies demonstrate that the neuronal architecture to support aversive stimuli encoding in this system does exist. Both cellular-level and human neuroimaging research suggest the co-existence of appetitive and aversive prediction-error signals within the mesocorticolimbic system. These findings shift the view of the mesocorticolimbic system as a singular pathway for reward to a system with multiple signals across a wide range of domains that drive value-based decision making.

Cocaine Dependent Individuals and Gamblers Present Different Associative Learning Anomalies in Feedback-Driven Decision Making: A Behavioral and ERP Study

Several recent studies have demonstrated that addicts behave less flexibly than healthy controls in the probabilistic reversal learning task (PRLT), in which participants must gradually learn to choose between a probably rewarded option and an improbably rewarded one, on the basis of corrective feedback, and in which preferences must adjust to abrupt reward contingency changes (reversals). In the present study, pathological gamblers (PG) and cocaine dependent individuals (CDI) showed different learning curves in the PRLT. PG also showed a reduced electroencephalographic response to feedback (Feedback-Related Negativity, FRN) when compared to controls. CDI's FRN was not significantly different either from PG or from healthy controls. Additionally, according to Standardized Low-Resolution Electromagnetic Tomography analysis, cortical activity in regions of interest (previously selected by virtue of their involvement in FRN generation in controls) strongly differed between CDI and PG. However, the nature of such anomalies varied within-groups across individuals. Cocaine use severity had a strong deleterious impact on the learning asymptote, whereas gambling intensity significantly increased reversal cost. These two effects have remained confounded in most previous studies, which can be hiding important associative learning differences between different populations of addicts.

Stress increases aversive prediction error signal in the ventral striatum

From job interviews to the heat of battle, it is evident that people think and learn differently when stressed. In fact, learning under stress may have long-term consequences; stress facilitates aversive conditioning and associations learned during extreme stress may result in debilitating emotional responses in posttraumatic stress disorder. The mechanisms underpinning such stress-related associations, however, are unknown. Computational neuroscience has successfully characterized several mechanisms critical for associative learning under normative conditions. One such mechanism, the detection of a mismatch between expected and observed outcomes within the ventral striatum (i.e., "prediction errors"), is thought to be a critical precursor to the formation of new stimulus-outcome associations. An untested possibility, therefore, is that stress may affect learning via modulation of this mechanism. Here we combine a translational model of stress with a cognitive neuroimaging paradigm to demonstrate that stress significantly increases ventral striatum aversive (but not appetitive) prediction error signal. This provides a unique account of the propensity to form threat-related associations under stress with direct implications for our understanding of both normal stress and stress-related disorders.

Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning

Drugs that alter dopamine transmission have opposite effects on reward and punishment learning. These opposite effects have been suggested to depend on dopamine in the striatum. Here, we establish for the first time the neurochemical specificity of such drug effects, during reward and punishment learning in humans, by adopting a coadministration design. Participants (N = 22) were scanned on 4 occasions using functional magnetic resonance imaging, following intake of placebo, bromocriptine (dopamine-receptor agonist), sulpiride (dopamine-receptor antagonist), or a combination of both drugs. A reversal-learning task was employed, in which both unexpected rewards and punishments signaled reversals. Drug effects were stratified with baseline working memory to take into account individual variations in drug response. Sulpiride induced parallel span-dependent changes on striatal blood oxygen level-dependent (BOLD) signal during unexpected rewards and punishments. These drug effects were found to be partially dopamine-dependent, as they were blocked by coadministration with bromocriptine. In contrast, sulpiride elicited opposite effects on behavioral measures of reward and punishment learning. Moreover, sulpiride-induced increases in striatal BOLD signal during both outcomes were associated with behavioral improvement in reward versus punishment learning. These results provide a strong support for current theories, suggesting that drug effects on reward and punishment learning are mediated via striatal dopamine.

Enhanced striatal sensitivity to aversive reinforcement in adolescents versus adults

Neurodevelopmental changes in mesolimbic regions are associated with adolescent risk-taking behavior. Numerous studies have shown exaggerated activation in the striatum in adolescents compared with children and adults during reward processing. However, striatal sensitivity to aversion remains elusive. Given the important role of the striatum in tracking both appetitive and aversive events, addressing this question is critical to understanding adolescent decision-making, as both positive and negative factors contribute to this behavior. In this study, human adult and adolescent participants performed a task in which they received squirts of appetitive or aversive liquid while undergoing fMRI, a novel approach in human adolescents. Compared with adults, adolescents showed greater behavioral and striatal sensitivity to both appetitive and aversive stimuli, an effect that was exaggerated in response to delivery of the aversive stimulus. Collectively, these findings contribute to understanding how neural responses to positive and negative outcomes differ between adolescents and adults and how they may influence adolescent behavior.

Specific neural correlates of successful learning and adaptation during social exchanges

Cooperation and betrayal are universal features of social interactions, and knowing who to trust is vital in human society. Previous studies have identified brain regions engaged by decision making during social encounters, but the mechanisms supporting modification of future behaviour by utilizing social experience are not well characterized. Using functional magnetic resonance imaging (fMRI), we show that cooperation and betrayal during social exchanges elicit specific patterns of neural activity associated with future behaviour. Unanticipated cooperation leads to greater behavioural adaptation than unexpected betrayal, and is signalled by specific neural responses in the striatum and midbrain. Neural responses to betrayal and willingness to trust novel partners both decrease as the number of individuals encountered during repeated social encounters increases. We propose that, as social groups increase in size, uncooperative or untrustworthy behaviour becomes progressively less surprising, with cooperation becoming increasingly important as a stimulus for social learning. Effects on reputation of non-trusting decisions may also act to drive pro-social behaviour. Our findings characterize the dynamic neural processes underlying social adaptation, and suggest that the brain is optimized to cooperate with trustworthy partners, rather than avoiding those who might betray us.

Prediction-error in the context of real social relationships modulates reward system activity

The human reward system is sensitive to both social (e.g., validation) and non-social rewards (e.g., money) and is likely integral for relationship development and reputation building. However, data is sparse on the question of whether implicit social reward processing meaningfully contributes to explicit social representations such as trust and attachment security in pre-existing relationships. This event-related fMRI experiment examined reward system prediction-error activity in response to a potent social reward—social validation—and this activity's relation to both attachment security and trust in the context of real romantic relationships. During the experiment, participants' expectations for their romantic partners' positive regard of them were confirmed (validated) or violated, in either positive or negative directions. Primary analyses were conducted using predefined regions of interest, the locations of which were taken from previously published research. Results indicate that activity for mid-brain and striatal reward system regions of interest was modulated by social reward expectation violation in ways consistent with prior research on reward prediction-error. Additionally, activity in the striatum during viewing of disconfirmatory information was associated with both increases in post-scan reports of attachment anxiety and decreases in post-scan trust, a finding that follows directly from representational models of attachment and trust.

Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice

To behave adaptively, we must learn from the consequences of our actions. Studies using event-related potentials (ERPs) have been informative with respect to the question of how such learning occurs. These studies have revealed a frontocentral negativity termed the feedback-related negativity (FRN) that appears after negative feedback. According to one prominent theory, the FRN tracks the difference between the values of actual and expected outcomes, or reward prediction errors. As such, the FRN provides a tool for studying reward valuation and decision making. We begin this review by examining the neural significance of the FRN. We then examine its functional significance. To understand the cognitive processes that occur when the FRN is generated, we explore variables that influence its appearance and amplitude. Specifically, we evaluate four hypotheses: (1) the FRN encodes a quantitative reward prediction error; (2) the FRN is evoked by outcomes and by stimuli that predict outcomes; (3) the FRN and behavior change with experience; and (4) the system that produces the FRN is maximally engaged by volitional actions.

Learning from positive and negative monetary feedback in patients with alcohol dependence

BACKGROUND

Chronic and excessive consumption of alcohol is associated with structural, physiological, and functional changes in multiple regions of the human brain including the prefrontal cortex, the medial temporal lobe, and the structures of the reward system. The present study aimed to assess the ability of alcohol-dependent patients (ADP) to learn probabilistic stimulus-reward contingencies and to transfer the acquired knowledge to new contexts. During transfer, the relative preference to learn from positive or negative feedback was also assessed.

METHODS

Twenty-four recently detoxified ADP and 20 healthy controls engaged in a feedback learning task with monetary rewards. The learning performance per se and transfer performance including positive versus negative learning were examined, as well as the relationship between different learning variables and variables comprising alcohol and nicotine consumption patterns, depression, and personality traits (harm avoidance and impulsivity).

RESULTS

Patients did not show a significant general learning deficit in the acquisition of stimulus-response-outcome associations. Fifteen healthy subjects and 13 patients reached the transfer phase, in which ADP showed generally lower performance than healthy controls. There was no specific deficit with regard to learning from positive or negative feedback. The only near-significant (negative) correlation between learning variables and drug consumption patterns, depression, and personality traits emerged for harm avoidance and positive learning in controls.

CONCLUSIONS

Impaired transfer performance suggests that ADP had problems applying their acquired knowledge in a new context. Potential relations to dysfunctions of specific brain structures and implications of the finding for therapy are discussed.

Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive disorder

OBJECTIVE

Affective biases may underlie many of the key symptoms of major depressive disorder, from anhedonia to altered cognitive performance. Understanding the cause of these biases is therefore critical in the quest for improved treatments. Depression is associated, for example, with a negative affective bias in reversal learning. However, despite the fact that reversal learning is associated with striatal response in healthy individuals and depressed individuals exhibit attenuated striatal function on multiple tasks, studies to date have not demonstrated striatal involvement in the negative bias in reversal learning in depression. In this study, the authors sought to determine whether this may be because reversal learning tasks conventionally used to study behavior examine reversals only on the basis of unexpected punishment and therefore do not adequately separate reward- and punishment-based behavior.

METHOD

The authors used functional MRI to compare the hemodynamic response to a reversal learning task with mixed reward- and punishment-based reversal stages between individuals with unmedicated major depressive disorder (N=13) and healthy comparison subjects (N=14).

RESULTS

Impaired reward (but not punishment) reversal accuracy was found alongside attenuated anteroventral striatal response to unexpected reward in depression.

CONCLUSIONS

Attenuated neurophysiological response of the anteroventral striatum may reflect dysfunction in circuits involving afferent projections from the orbitofrontal, limbic, and/or mesostriatal dopaminergic pathways, which conceivably may, together with the ventral striatum, underlie anhedonia in depression. Learning to appreciate and enjoy positive life experiences is critical for recovery from depression. This study pinpoints a neural target for such recovery.

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback--either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum--separated along both a dorsal/ventral and anterior/posterior axis--differentially participate in the learning of associations through reward and punishment.

Is all motivation good for learning? Dissociable influences of approach and avoidance motivation in declarative memory

The present study investigated the effects of approach versus avoidance motivation on declarative learning. Human participants navigated a virtual reality version of the Morris water task, a classic spatial memory paradigm, adapted to permit the experimental manipulation of motivation during learning. During this task, participants were instructed to navigate to correct platforms while avoiding incorrect platforms. To manipulate motivational states participants were either rewarded for navigating to correct locations (approach) or punished for navigating to incorrect platforms (avoidance). Participants' skin conductance levels (SCLs) were recorded during navigation to investigate the role of physiological arousal in motivated learning. Behavioral results revealed that, overall, approach motivation enhanced and avoidance motivation impaired memory performance compared to nonmotivated spatial learning. This advantage was evident across several performance indices, including accuracy, learning rate, path length, and proximity to platform locations during probe trials. SCL analysis revealed three key findings. First, within subjects, arousal interacted with approach motivation, such that high arousal on a given trial was associated with performance deficits. In addition, across subjects, high arousal negated or reversed the benefits of approach motivation. Finally, low-performing, highly aroused participants showed SCL responses similar to those of avoidance-motivation participants, suggesting that for these individuals, opportunities for reward may evoke states of learning similar to those typically evoked by threats of punishment. These results provide a novel characterization of how approach and avoidance motivation influence declarative memory and indicate a critical and selective role for arousal in determining how reinforcement influences goal-oriented learning.