Hemispheric Asymmetries in Striatal Reward Responses Relate to Approach-Avoidance Learning and Encoding of Positive-Negative Prediction Errors in Dopaminergic Midbrain Regions

Motivated with joy or anxiety: Does approach-avoidance goal framing elicit differential reward-network activation in the brain?

Psychological research on human motivation repeatedly observed that approach goals (i.e., goals to attain success) increase task enjoyment and intrinsic motivation more strongly than avoidance goals (i.e., goals to avoid failure). The present study sought to address how the reward network in the brain-including the striatum and ventromedial prefrontal cortex-is involved when individuals engage in the same task with a focus on approach or avoidance goals. Participants reported stronger positive emotions when they focused on approach goals, but stronger anxiety and disappointment when they focused on avoidance goals. The fMRI analyses revealed that the reward network in the brain showed similar levels of activity to cues predictive of approach and avoidance goals. In contrast, the two goal states were associated with different patterns of activity in the visual cortex, hippocampus, and cerebellum during success and failure outcomes. Representation similarity analysis further revealed shared and different representations within the striatum and vmPFC between the approach and avoidance goal states, suggesting both the similarity and uniqueness of the mechanisms behind the two goal states. In addition, the distinct patterns of activation in the striatum were associated with distinct subjective experiences participants reported between the approach and the avoidance conditions. These results suggest the importance of examining the pattern of striatal activity in understanding the mechanisms behind different motivational states in humans.

Imbalanced motivated behaviors according to motor sign asymmetry in drug-naïve Parkinson's disease

Few studies have considered the influence of motor sign asymmetry on motivated behaviors in de novo drug-naïve Parkinson's disease (PD). We tested whether motor sign asymmetry could be associated with different motivated behavior patterns in de novo drug-naïve PD. We performed a cross-sectional study in 128 de novo drug-naïve PD patients and used the Ardouin Scale of Behavior in Parkinson's disease (ASBPD) to assess a set of motivated behaviors. We assessed motor asymmetry based on (i) side of motor onset and (ii) MDS-UPDRS motor score, then we compared right hemibody Parkinson's disease to left hemibody Parkinson's disease. According to the MDS-UPDRS motor score, patients with de novo right hemibody PD had significantly lower frequency of approach behaviors (p = 0.031), including nocturnal hyperactivity (p = 0.040), eating behavior (p = 0.040), creativity (p = 0.040), and excess of motivation (p = 0.017) than patients with de novo left hemibody PD. Patients with de novo left hemibody PD did not significantly differ from those with de novo right hemibody PD regarding avoidance behaviors including apathy, anxiety and depression. Our findings suggest that motor sign asymmetry may be associated with an imbalance between motivated behaviors in de novo drug-naïve Parkinson's disease.

No evidence from a negative mood induction fMRI task for frontal functional asymmetry as a suitable neurofeedback target

Frontal functional asymmetry (FA) has been proposed as a potential target for neurofeedback (NFB) training for mental disorders but most FA NFB studies used electroencephalography while the investigations of FA NFB in functional magnetic resonance imaging (fMRI) are rather limited. In this study, we aimed at identifying functional asymmetry effects in fMRI and exploring its potential as a target for fMRI NFB studies by re-analyzing an existing data set containing a resting state measurement and a sad mood induction task of n = 30 participants with remitted major depressive disorder and n = 30 matched healthy controls. We applied low-frequency fluctuations (ALFF), fractional ALFF, and regional homogeneity and estimated functional asymmetry in both a voxel-wise and regional manner. We assessed functional asymmetry during rest and negative mood induction as well as functional asymmetry changes between the phases, and associated the induced mood change with the change in functional asymmetry. Analyses were conducted within as well as between groups. Despite extensive analyses, we identified only very limited effects. While some tests showed nominal significance, our results did not contain any clear identifiable patterns of effects that would be expected if a true underlying effect would be present. In conclusion, we do not find evidence for FA effects related to negative mood in fMRI, which questions the usefulness of FA measures for real-time fMRI neurofeedback as a treatment approach for affective disorders.

Neuropsychiatric symptoms and imbalance of atrophy in behavioral variant frontotemporal dementia

Behavioral variant frontotemporal dementia is characterized by heterogeneous frontal, insular, and anterior temporal atrophy patterns that vary along left-right and dorso-ventral axes. Little is known about how these structural imbalances impact clinical symptomatology. The goal of this study was to assess the frequency of frontotemporal asymmetry (right- or left-lateralization) and dorsality (ventral or dorsal predominance of atrophy) and to investigate their clinical correlates. Neuropsychiatric symptoms and structural images were analyzed for 250 patients with behavioral variant frontotemporal dementia. Frontotemporal atrophy was most often symmetric while left-lateralized (9%) and right-lateralized (17%) atrophy were present in a minority of patients. Atrophy was more often ventral (32%) than dorsal (3%) predominant. Patients with right-lateralized atrophy were characterized by higher severity of abnormal eating behavior and hallucinations compared to those with left-lateralized atrophy. Subsequent analyses clarified that eating behavior was associated with right atrophy to a greater extent than a lack of left atrophy, and hallucinations were driven mainly by right atrophy. Dorsality analyses showed that anxiety, euphoria, and disinhibition correlated with ventral-predominant atrophy. Agitation, irritability, and depression showed greater severity with a lack of regional atrophy, including in dorsal regions. Aberrant motor behavior and apathy were not explained by asymmetry or dorsality. This study provides additional insight into how anatomical heterogeneity influences the clinical presentation of patients with behavioral variant frontotemporal dementia. Behavioral symptoms can be associated not only with the presence or absence of focal atrophy, but also with right/left or dorsal/ventral imbalance of gray matter volume.

Irrelevant Threats Linger and Affect Behavior in High Anxiety

Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.

Average reward rates enable motivational transfer across independent reinforcement learning tasks

Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and was found to interact with dopamine-sensitive cognitive processes, such as vigilance and associative memory encoding. The ARR could therefore provide a bridge between independent tasks when these are performed in temporal proximity, such that the reward rate obtained in one task could influence performance in a second subsequent task. Reinforcement learning depends on the coding of prediction error signals by dopamine neurons and their downstream targets, in particular the nucleus accumbens. Because these brain regions also respond to changes in ARR, reinforcement learning may be vulnerable to changes in ARR. To test this hypothesis, we designed a novel paradigm in which participants (n = 245) performed two probabilistic reinforcement learning tasks presented in interleaved trials. The ARR was controlled by an "induction" task which provided feedback with a low (p = 0.58), a medium (p = 0.75), or a high probability of reward (p = 0.92), while the impact of ARR on reinforcement learning was tested by a second "reference" task with a constant reward probability (p = 0.75). We find that performance was significantly lower in the reference task when the induction task provided low reward probabilities (i.e., during low levels of ARR), as compared to the medium and high ARR conditions. Behavioral modeling further revealed that the influence of ARR is best described by models which accumulates average rewards (rather than average prediction errors), and where the ARR directly modulates the prediction error signal (rather than affecting learning rates or exploration). Our results demonstrate how affective information in one domain may transfer and affect motivated behavior in other domains. These findings are particularly relevant for understanding mood disorders, but may also inform abnormal behaviors attributed to dopamine dysfunction.

Aminopeptidase Activities Interact Asymmetrically between Brain, Plasma and Systolic Blood Pressure in Hypertensive Rats Unilaterally Depleted of Dopamine

Brain dopamine, in relation to the limbic system, is involved in cognition and emotion. These functions are asymmetrically processed. Hypertension not only alters such functions but also their asymmetric brain pattern as well as their bilateral pattern of neurovisceral integration. The central and peripheral renin-angiotensin systems, particularly the aminopeptidases involved in its enzymatic cascade, play an important role in blood pressure control. In the present study, we report how these aminopeptidases from left and right cortico-limbic locations, plasma and systolic blood pressure interact among them in spontaneously hypertensive rats (SHR) unilaterally depleted of dopamine. The study comprises left and right sham and left and right lesioned (dopamine-depleted) rats as research groups. Results revealed important differences in the bilateral behavior comparing sham left versus sham right, lesioned left versus lesioned right, and sham versus lesioned animals. Results also suggest an important role for the asymmetrical functioning of the amygdala in cardiovascular control and an asymmetrical behavior in the interaction between the medial prefrontal cortex, hippocampus and amygdala with plasma, depending on the left or right depletion of dopamine. Compared with previous results of a similar study in Wistar-Kyoto (WKY) normotensive rats, the asymmetrical behaviors differ significantly between both WKY and SHR strains.

Pupil dilation predicts modulation of direct gaze on action value calculations

Perceiving direct gaze facilitates social cognition and behaviour. We hypothesized that direct gaze modulates decision-making, particularly calculations of action values. To test our hypothesis, we used the reinforcement learning paradigm in situations with or without direct gaze. Forty adults were recruited and participated in pupil size measurements and a two-armed bandit task. The task was conducted with 70% and 30% reward probabilities for each option. During the task, a female showing the Direct Gaze (DG) or Closed Eyes (CE) condition was presented from the start of each trial. The results showed that behavioural bias to choices with 70% reward probability increased more in the DG condition than in the CE condition and the expected reward value. This bias to choices with 70% reward in the DG condition was predicted by pupil dilation to DG. These results suggest that participants over-evaluated the expected reward value in the DG condition, and this DG effect may be related to subjective expectations of rewarding events indexed by pupil dilations. It is considered that perceiving direct gaze is a driver of reward expectations that modulate action value calculations and then cognitive processing and behaviours are facilitated.

Mapping frontostriatal white matter tracts and their association with reward-related ventral striatum activation in adolescence

The ventral striatum (VS) is implicated in reward processing and motivation. Human and non-human primate studies demonstrate that the VS and prefrontal cortex (PFC), which comprise the frontostriatal circuit, interact to influence motivated behavior. However, there is a lack of research that precisely maps and quantifies VS-PFC white matter tracts. Moreover, no studies have linked frontostriatal white matter to VS activation. Using a multimodal neuroimaging approach with diffusion MRI (dMRI) and functional MRI (fMRI), the present study had two objectives: 1) to chart white matter tracts between the VS and specific PFC structures and 2) assess the association between the degree of VS-PFC white matter tract connectivity and VS activation in 187 adolescents. White matter connectivity was assessed with probabilistic tractography and functional activation was examined with two fMRI tasks (one task with social reward and another task using monetary reward). We found widespread but variable white matter connectivity between the VS and areas of the PFC, with the anterior insula and subgenual cingulate cortex demonstrating the greatest degree of connectivity with the VS. VS-PFC structural connectivity was related to functional activation in the VS though activation depended on the specific PFC region and reward task.

Examining learning coherence in group decision-making: triads vs. tetrads

This study examined whether three heads are better than four in terms of performance and learning properties in group decision-making. It was predicted that learning incoherence took place in tetrads because the majority rule could not be applied when two subgroups emerged. As a result, tetrads underperformed triads. To examine this hypothesis, we adopted a reinforcement learning framework using simple Q-learning and estimated learning parameters. Overall, the results were consistent with the hypothesis. Further, this study is one of a few attempts to apply a computational approach to learning behavior in small groups. This approach enables the identification of underlying learning parameters in group decision-making.

Three heads are better than two: Comparing learning properties and performances across individuals, dyads, and triads through a computational approach

Although it is considered that two heads are better than one, related studies argued that groups rarely outperform their best members. This study examined not only whether two heads are better than one but also whether three heads are better than two or one in the context of two-armed bandit problems where learning plays an instrumental role in achieving high performance. This research revealed that a U-shaped correlation exists between performance and group size. The performance was highest for either individuals or triads, but the lowest for dyads. Moreover, this study estimated learning properties and determined that high inverse temperature (exploitation) accounted for high performance. In particular, it was shown that group effects regarding the inverse temperatures in dyads did not generate higher values to surpass the averages of their two group members. In contrast, triads gave rise to higher values of the inverse temperatures than their averages of their individual group members. These results were consistent with our proposed hypothesis that learning coherence is likely to emerge in individuals and triads, but not in dyads, which in turn leads to higher performance. This hypothesis is based on the classical argument by Simmel stating that while dyads are likely to involve more emotion and generate greater variability, triads are the smallest structure which tends to constrain emotions, reduce individuality, and generate behavioral convergences or uniformity because of the ‘‘two against one” social pressures. As a result, three heads or one head were better than two in our study.

Brain Microstructural Properties Related to Subjective Well-Being: Diffusion Tensor Imaging Analysis

Although it is known that health is not merely the absence of disease, the positive aspects of mental health have been less comprehensively researched compared with its negative aspects. Subjective well-being (SWB) is one of the indicators of positive psychology, and high SWB is considered to benefit individuals in multiple ways. However, the neural mechanisms underlying individual differences in SWB remain unclear, particularly in terms of brain microstructural properties as detected by diffusion tensor imaging. The present study aimed to investigate the relationship between measurements of diffusion tensor imaging [mean diffusivity (MD) and fractional anisotropy] and the degree of SWB as measured using a questionnaire. Voxel-based analysis was used to investigate the association between MD and SWB scores in healthy young adults (age, 20.7 ± 1.8 years; 695 males and 514 females). Higher levels of SWB were found to be associated with lower MD in areas surrounding the right putamen, insula, globus pallidus, thalamus and caudate. These results indicated that individual SWB is associated with variability in brain microstructural properties.

Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing

Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.

Hemispheric stroke: Mood disorders

Stroke causes many forms of disability, including emotional and mood disorders. Depression is the most common of these, affecting approximately one-third of stroke patients. Other disorders like mania, bipolar disorder, anxiety disorder, or apathy may also develop following stroke, although they are less common. The development of mood and emotional disorders is dependent on the severity of brain injury, the side of injury, and hemispheric location. Whereas a left hemispheric stroke often results in depression or a catastrophic reaction with anxiety, injury to the right hemisphere has predominantly been associated with the development of emotional indifference (anosodiaphoria) or euphoria. In this chapter, we discuss the mood disorders associated with hemispheric strokes and the neuropsychological mechanisms that might account for the clinical manifestations of these affective disorders.

Prior Reward Conditioning Dampens Hippocampal and Striatal Responses during an Associative Memory Task

Offering reward during encoding typically leads to better memory [Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S.,Knutson, B., & Gabrieli, J. D. E. Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50, 507-517, 2006]. Whether such memory benefit persists when tested in a different task context remains, however, largely understudied [Wimmer, G. E., & Buechel, C. Reactivation of reward-related patterns from single past episodes supports memory-based decision making. Journal of Neuroscience, 36, 2868-2880, 2016]. Here, we ask whether reward at encoding leads to a generalized advantage across learning episodes, a question of high importance for any everyday life applications, from education to patient rehabilitation. Although we confirmed that offering monetary reward increased responses in the ventral striatum and pleasantness judgments for pictures used as stimuli, this immediate beneficial effect of reward did not carry over to a subsequent and different picture-location association memory task during which no reward was delivered. If anything, a trend for impaired memory accuracy was observed for the initially high-rewarded pictures as compared to low-rewarded ones. In line with this trend in behavioral performance, fMRI activity in reward (i.e., ventral striatum) and in memory (i.e., hippocampus) circuits was reduced during the encoding of new associations using previously highly rewarded pictures (compared to low-reward pictures). These neural effects extended to new pictures from same, previously highly rewarded semantic category. Twenty-four hours later, delayed recall of associations involving originally highly rewarded items was accompanied by decreased functional connectivity between the hippocampus and two brain regions implicated in value-based learning, the ventral striatum and the ventromedial PFC. We conclude that acquired reward value elicits a downward value-adjustment signal in the human reward circuit when reactivated in a novel nonrewarded context, with a parallel disengagement of memory-reward (hippocampal-striatal) networks, likely to undermine new associative learning. Although reward is known to promote learning, here we show how it may subsequently hinder hippocampal and striatal responses during new associative memory formation.

Learning From Success or Failure? - Positivity Biases Revisited

The purpose of this study was to reexamine positivity learning biases through a Q learning computation model and relate them to behavioral characteristics of exploitation and exploration. It was found that while the positivity learning biases existed in the simple asymmetric Q learning model, they completely disappeared once the time-varying nature of learning rates was incorporated. In the time-varying model, learning rates depended on the magnitudes of success and failure. The corresponding positive and negative learning rates were related to high and low performance, respectively, indicating that successes and failures were accounted for by positive and negative learning rates. Moreover, these learning rates were related to both exploitation and exploration in somewhat balanced ways. In contrast, under the constant learning parameter model, positivity biases were associated only with exploration. Therefore, the results in the time-varying model are more intuitively appealing than the simple asymmetric model. However, the statistical tests indicated that participants eclectically selected between the asymmetric learning model and its time-varying version, a frequency of which differed across participants.

Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding

Rewarding events enhance memory encoding via dopaminergic influences on hippocampal plasticity. Phasic dopamine release depends on immediate reward magnitude, but presumably also on tonic dopamine levels, which may vary as a function of the average accumulation of reward over time. Using model-based fMRI in combination with a novel associative memory task, we show that immediate reward magnitude exerts a monotonically increasing influence on the nucleus accumbens, ventral tegmental area (VTA), and hippocampal activity during encoding, and enhances memory. By contrast, average reward levels modulate feedback-related responses in the VTA and hippocampus in a non-linear (inverted U-shape) fashion, with similar effects on memory performance. Additionally, the dorsal anterior cingulate cortex (dACC) monotonically tracks average reward levels, while VTA-dACC functional connectivity is non-linearly modulated (inverted U-shape) by average reward. We propose that the dACC computes the net behavioral impact of average reward and relays this information to memory circuitry via the VTA.

Neurocomputational correlates of learned irrelevance in humans

Inappropriate behaviors may result from acquiring maladaptive associations between irrelevant information in the environment and important events, such as reward or punishment. Pre-exposure effects are believed to prevent the expression of irrelevant associations. For example, learned irrelevance delays the expression of associations between conditioned (CS) and unconditioned (US) stimuli following their uncorrelated presentation. The neuronal substrates of pre-exposure effects in humans are largely unknown because these effects rapidly attenuate when using traditional pre-exposure paradigms. The latter are therefore incompatible with neuroimaging approaches that require many trial repetitions. Moreover, large methodological differences between animal and human research on pre-exposure effects challenge the presumption of shared neurocognitive substrates, and question the prevalent use of pre-exposure effects in animals to model symptoms of human mental disorders. To overcome these limitations, we combined a novel learned irrelevance task with model-based fMRI. We report the results of a model that describes learned irrelevance as a dynamic process, which evolves across trials and integrates the weighting between two state-action values pertaining to 'CS-no US' associations (acquired during pre-exposure) and 'CS-US' associations (acquired during subsequent conditioning). This relative weighting correlated i) positively with the learned irrelevance effect observed in the behavioral task, ii) positively with activity in the entorhinal cortex, and iii) negatively with activity in the nucleus accumbens (NAcc). Furthermore, the model updates the relative weighting of the two state-action values via two separate prediction error (PE) signals that allow the dynamic accumulation of evidence for the CS to predict the 'US' or a 'no US' outcome. One PE signal, designed to increase the relative weight of 'CS-US' associations following 'US' outcomes, correlated with activity in the NAcc, while another PE signal, designed to increase the relative weight of 'CS-no US' associations following 'no US' outcomes, correlated with activity in the basolateral amygdala. By extending previous animal observations to humans, the present study provides a novel approach to foster translational research on pre-exposure effects.

Driven by Pain, Not Gain: Computational Approaches to Aversion-Related Decision Making in Psychiatry

Although it is well known that "losses loom larger than gains," computational approaches to aversion-related decision making (ARDM) for psychiatric disorders is an underdeveloped area. Computational models of ARDM have been implemented primarily as state-dependent reinforcement learning models with bias parameters to quantify Pavlovian associations, and differential learning rates to quantify instrumental updating have been shown to depend on context, involve complex cost calculations, and include the consideration of counterfactual outcomes. Little is known about how individual differences influence these models relevant to anxiety-related conditions or addiction-related dysfunction. It is argued that model parameters reflecting 1) Pavlovian biases in the context of reinforcement learning or 2) hyperprecise prior beliefs in the context of active inference play an important role in the emergence of dysfunctional avoidance behaviors. The neural implementation of ARDM includes brain areas that are important for valuation (ventromedial prefrontal cortex) and positive reinforcement-related prediction errors (ventral striatum), but also aversive processing (insular cortex and cerebellum). Computational models of ARDM will help to establish a quantitative explanatory account of the development of anxiety disorders and addiction, but such models also face several challenges, including limited evidence for stability of individual differences, relatively low reliability of tasks, and disorder heterogeneity. Thus, it will be necessary to develop robust, reliable, and model-based experimental probes; recruit larger sample sizes; and use single case experimental designs for better pragmatic and explanatory biological models of psychiatric disorders.

Unilateral accumbal dopamine depletion affects decision-making in a side-specific manner

Mechanisms underlying affective and cognitive deficits in Parkinson's disease (PD) remain less studied than motor symptoms. Nucleus accumbens (NAc) is affected in PD and due to its well-known involvement in motivation is an interesting target in this context. Furthermore, PD is frequently asymmetrical, with side-specific deficits aligning with evidences of accumbal laterality. We therefore used a 6-hydroxydopamine (6-OHDA) model to study the role of left and right NAc dopamine depletion in a battery of behavioral tasks. 2 months old male rats were used in all experiments. Habitual-based and goal-directed decision-making, impulsivity, anxiety- and depressive-like behavior and motor performance were tested 3 weeks after left (6-OHDA L) or right (6-OHDA R) NAc lesion was induced. Upon contingency degradation, 6-OHDA R decrease their lever press rate less than Sham and 6-OHDA L, indicating an impairment in the shift from habit-based to goal-directed strategies. On the other hand, 6-OHDA L lesions lead to increased rates of premature responding when delays where increased in the variable delay-to-signal test. Importantly, in both paradigms task acquisition was similar between groups. In the same line we found no differences in the amount of sugared pellets eaten when freely available as well as in both general and fine motor behaviors. In conclusion, left and right NAc play distinct roles in the contingency degradation and impulsivity. More studies are needed to understand the mechanisms behind this functional lateralization and its implications for PD patients.

Hemispheric asymmetries in resting-state EEG and fMRI are related to approach and avoidance behaviour, but not to eating behaviour or BMI

Much of our behaviour is driven by two motivational dimensions—approach and avoidance. These have been related to frontal hemispheric asymmetries in clinical and resting‐state EEG studies: Approach was linked to higher activity of the left relative to the right hemisphere, while avoidance was related to the opposite pattern. Increased approach behaviour, specifically towards unhealthy foods, is also observed in obesity and has been linked to asymmetry in the framework of the right‐brain hypothesis of obesity. Here, we aimed to replicate previous EEG findings of hemispheric asymmetries for self‐reported approach/avoidance behaviour and to relate them to eating behaviour. Further, we assessed whether resting fMRI hemispheric asymmetries can be detected and whether they are related to approach/avoidance, eating behaviour and BMI. We analysed three samples: Sample 1 (n = 117) containing EEG and fMRI data from lean participants, and Samples 2 (n = 89) and 3 (n = 152) containing fMRI data from lean, overweight and obese participants. In Sample 1, approach behaviour in women was related to EEG, but not to fMRI hemispheric asymmetries. In Sample 2, approach/avoidance behaviours were related to fMRI hemispheric asymmetries. Finally, hemispheric asymmetries were not related to either BMI or eating behaviour in any of the samples. Our study partly replicates previous EEG findings regarding hemispheric asymmetries and indicates that this relationship could also be captured using fMRI. Our findings suggest that eating behaviour and obesity are likely to be mediated by mechanisms not directly relating to frontal asymmetries in neuronal activation quantified with EEG and fMRI.

Probabilistic reinforcement learning abnormalities and their correlates in adolescent bipolar disorders

Theoretical models of bipolar disorders (BD) posit core deficits in reward system function. However, specifying which among the multiple reward system's neurobehavioral processes are abnormal in BD is necessary to develop appropriately targeted interventions. Research on probabilistic-reinforcement learning deficits in BD is limited, particularly during adolescence, a period of significant neurodevelopmental changes in the reward system. The present study investigated probabilistic-reinforcement learning, using a probabilistic selection task (PST), and its correlates, using self-reported reward/threat sensitivities and cognitive tasks, in 104 adolescents with and without BD. Compared with healthy peers, adolescents with BD were less likely to persist with their choices based on prior positive feedback (i.e., lower win-stay rates) in the PST's acquisition phase. Across groups, a greater win-stay rate appeared to be a more efficient learning strategy-associated with fewer acquisition trials and better testing phase performance. Win-stay rates were also related to verbal learning indices, but not self-reported reward/threat sensitivities. Finally, lower win-stay rates had significant incremental validity in predicting a BD diagnosis, after accounting for effects of current symptoms, reward sensitivities, verbal learning, and IQ. The present findings support multiple dysfunctional processes of the reward system in adolescent BD that require additional examinations. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Left-shifting prism adaptation boosts reward-based learning

Visuospatial cognition has an inherent lateralized bias. Individual differences in the direction and magnitude of this bias are associated with asymmetrical D2/3 dopamine binding and dopamine system genotypes. Dopamine level affects feedback-based learning and dopamine signaling asymmetry is related to differential learning from reward and punishment. High D2 binding in the left hemisphere is associated with preference for reward. Prism adaptation (PA) is a simple sensorimotor technique, which modulates visuospatial bias according to the direction of the deviation. Left-deviating prism adaptation (LPA) induces rightward bias in healthy subjects. It is therefore possible that the right side of space increases in saliency along with left hemisphere dopaminergic activity. Right-deviating prism adaptation (RPA) has been used mainly as a control condition because it does not modulate behavior in healthy individuals. Since LPA induces a rightward visuospatial bias as a result of left hemisphere modulation, and higher dopaminergic activity in the left hemisphere is associated with preference for rewarding events we hypothesized that LPA would increase the preference for learning with reward. Healthy volunteers performed a computer-based probabilistic classification task before and after LPA or RPA. Consistent with our predictions, PA altered the preference for rewarded versus punished learning, with the LPA group exhibiting increased learning from reward. These results suggest that PA modulates dopaminergic activity in a lateralized fashion.

Financial gain- and loss-related BOLD signals in the human ventral tegmental area and substantia nigra pars compacta

Neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNC) play central roles in reward-related behaviours. Nonhuman animal studies suggest that these neurons also process aversive events. However, our understanding of how the human VTA and SNC responds to such events is limited and has been hindered by the technical challenge of using functional magnetic resonance imaging (fMRI) to investigate a small structure where the signal is particularly vulnerable to physiological noise. Here we show, using methods optimized specifically for the midbrain (including high-resolution imaging, a novel registration protocol, and physiological noise modelling), a BOLD (blood-oxygen-level dependent) signal to both financial gain and loss in the VTA and SNC, along with a response to nil outcomes that are better or worse than expected in the VTA. Taken together, these findings suggest that the human VTA and SNC are involved in the processing of both appetitive and aversive financial outcomes in humans.

Effects of dopamine on reinforcement learning and consolidation in Parkinson's disease

Emerging evidence suggests that dopamine may modulate learning and memory with important implications for understanding the neurobiology of memory and future therapeutic targeting. An influential hypothesis posits that dopamine biases reinforcement learning. More recent data also suggest an influence during both consolidation and retrieval. Eighteen Parkinson's disease patients learned through feedback ON or OFF medication, with memory tested 24 hr later ON or OFF medication (4 conditions, within-subjects design with matched healthy control group). Patients OFF medication during learning decreased in memory accuracy over the following 24 hr. In contrast to previous studies, however, dopaminergic medication during learning and testing did not affect expression of positive or negative reinforcement. Two further experiments were run without the 24 hr delay, but they too failed to reproduce effects of dopaminergic medication on reinforcement learning. While supportive of a dopaminergic role in consolidation, this study failed to replicate previous findings on reinforcement learning.

Distinct neural processes support post-success and post-error slowing in the stop signal task

Executive control requires behavioral adaptation to environmental contingencies. In the stop signal task (SST), participants exhibit slower go trial reaction time (RT) following a stop trial, whether or not they successfully interrupt the motor response. In previous fMRI studies, we demonstrated activation of the right-hemispheric ventrolateral prefrontal cortex, in the area of inferior frontal gyrus, pars opercularis (IFGpo) and anterior insula (AI), during post-error slowing (PES). However, in similar analyses we were not able to identify regional activities during post-success slowing (PSS). Here, we revisited this issue in a larger sample of participants (n=100) each performing the SST for 40 min during fMRI. We replicated IFGpo/AI activation to PES (p≤0.05, FWE corrected). Further, PSS engages decreased activation in a number of cortical regions including the left inferior frontal cortex (IFC; p≤0.05, FWE corrected). We employed Granger causality mapping to identify areas that provide inputs each to the right IFGpo/AI and left IFC, and computed single-trial amplitude (STA) of stop trials of these input regions as well as the STA of post-stop trials of the right IFGpo/AI and left IFC. The STAs of the right inferior precentral sulcus and supplementary motor area (SMA) and right IFGpo/AI were positively correlated and the STAs of the left SMA and left IFC were positively correlated (slope>0, p's≤0.01, one-sample t test), linking regional responses during stop success and error trials to those during PSS and PES. These findings suggest distinct neural mechanisms to support PSS and PES.

Trial-by-Trial Modulation of Associative Memory Formation by Reward Prediction Error and Reward Anticipation as Revealed by a Biologically Plausible Computational Model

Anticipation and delivery of rewards improves memory formation, but little effort has been made to disentangle their respective contributions to memory enhancement. Moreover, it has been suggested that the effects of reward on memory are mediated by dopaminergic influences on hippocampal plasticity. Yet, evidence linking memory improvements to actual reward computations reflected in the activity of the dopaminergic system, i.e., prediction errors and expected values, is scarce and inconclusive. For example, different previous studies reported that the magnitude of prediction errors during a reinforcement learning task was a positive, negative, or non-significant predictor of successfully encoding simultaneously presented images. Individual sensitivities to reward and punishment have been found to influence the activation of the dopaminergic reward system and could therefore help explain these seemingly discrepant results. Here, we used a novel associative memory task combined with computational modeling and showed independent effects of reward-delivery and reward-anticipation on memory. Strikingly, the computational approach revealed positive influences from both reward delivery, as mediated by prediction error magnitude, and reward anticipation, as mediated by magnitude of expected value, even in the absence of behavioral effects when analyzed using standard methods, i.e., by collapsing memory performance across trials within conditions. We additionally measured trait estimates of reward and punishment sensitivity and found that individuals with increased reward (vs. punishment) sensitivity had better memory for associations encoded during positive (vs. negative) prediction errors when tested after 20 min, but a negative trend when tested after 24 h. In conclusion, modeling trial-by-trial fluctuations in the magnitude of reward, as we did here for prediction errors and expected value computations, provides a comprehensive and biologically plausible description of the dynamic interplay between reward, dopamine, and associative memory formation. Our results also underline the importance of considering individual traits when assessing reward-related influences on memory.

Linking Individual Learning Styles to Approach-Avoidance Motivational Traits and Computational Aspects of Reinforcement Learning

Learning how to gain rewards (approach learning) and avoid punishments (avoidance learning) is fundamental for everyday life. While individual differences in approach and avoidance learning styles have been related to genetics and aging, the contribution of personality factors, such as traits, remains undetermined. Moreover, little is known about the computational mechanisms mediating differences in learning styles. Here, we used a probabilistic selection task with positive and negative feedbacks, in combination with computational modelling, to show that individuals displaying better approach (vs. avoidance) learning scored higher on measures of approach (vs. avoidance) trait motivation, but, paradoxically, also displayed reduced learning speed following positive (vs. negative) outcomes. These data suggest that learning different types of information depend on associated reward values and internal motivational drives, possibly determined by personality traits.

The left hemisphere learns what is right: Hemispatial reward learning depends on reinforcement learning processes in the contralateral hemisphere

Orienting biases refer to consistent, trait-like direction of attention or locomotion toward one side of space. Recent studies suggest that such hemispatial biases may determine how well people memorize information presented in the left or right hemifield. Moreover, lesion studies indicate that learning rewarded stimuli in one hemispace depends on the integrity of the contralateral striatum. However, the exact neural and computational mechanisms underlying the influence of individual orienting biases on reward learning remain unclear. Because reward-based behavioural adaptation depends on the dopaminergic system and prediction error (PE) encoding in the ventral striatum, we hypothesized that hemispheric asymmetries in dopamine (DA) function may determine individual spatial biases in reward learning. To test this prediction, we acquired fMRI in 33 healthy human participants while they performed a lateralized reward task. Learning differences between hemispaces were assessed by presenting stimuli, assigned to different reward probabilities, to the left or right of central fixation, i.e. presented in the left or right visual hemifield. Hemispheric differences in DA function were estimated through differential fMRI responses to positive vs. negative feedback in the left vs. right ventral striatum, and a computational approach was used to identify the neural correlates of PEs. Our results show that spatial biases favoring reward learning in the right (vs. left) hemifield were associated with increased reward responses in the left hemisphere and relatively better neural encoding of PEs for stimuli presented in the right (vs. left) hemifield. These findings demonstrate that trait-like spatial biases implicate hemisphere-specific learning mechanisms, with individual differences between hemispheres contributing to reinforcing spatial biases.