The left hemisphere learns what is right: Hemispatial reward learning depends on reinforcement learning processes in the contralateral hemisphere

Irrelevant Threats Linger and Affect Behavior in High Anxiety

Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.

Mapping frontostriatal white matter tracts and their association with reward-related ventral striatum activation in adolescence

The ventral striatum (VS) is implicated in reward processing and motivation. Human and non-human primate studies demonstrate that the VS and prefrontal cortex (PFC), which comprise the frontostriatal circuit, interact to influence motivated behavior. However, there is a lack of research that precisely maps and quantifies VS-PFC white matter tracts. Moreover, no studies have linked frontostriatal white matter to VS activation. Using a multimodal neuroimaging approach with diffusion MRI (dMRI) and functional MRI (fMRI), the present study had two objectives: 1) to chart white matter tracts between the VS and specific PFC structures and 2) assess the association between the degree of VS-PFC white matter tract connectivity and VS activation in 187 adolescents. White matter connectivity was assessed with probabilistic tractography and functional activation was examined with two fMRI tasks (one task with social reward and another task using monetary reward). We found widespread but variable white matter connectivity between the VS and areas of the PFC, with the anterior insula and subgenual cingulate cortex demonstrating the greatest degree of connectivity with the VS. VS-PFC structural connectivity was related to functional activation in the VS though activation depended on the specific PFC region and reward task.

A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety

Exploration reduces uncertainty about the environment and improves the quality of future decisions, but at the cost of provisional uncertain and suboptimal outcomes. Although anxiety promotes intolerance to uncertainty, it remains unclear whether and by which mechanisms anxiety relates to exploratory decision-making. We use a dynamic three-armed-bandit task and find that higher trait-anxiety is associated with increased exploration, which in turn harms overall performance. We identify two distinct behavioral sources: first, decisions made by anxious individuals are guided toward reduction of uncertainty; and second, decisions are less guided by immediate value gains. These findings are similar in both loss and gain domains, and further demonstrate that an affective trait relates to exploration and results in an inverse-U-shaped relationship between anxiety and overall performance. Additional imaging data (fMRI) suggests that normative anxiety correlates negatively with the representation of expected-value in the dorsal-anterior-cingulate-cortex, and in contrast, positively with the representation of uncertainty in the anterior-insula. We conclude that a trade-off between value-gains and uncertainty-reduction entails maladaptive decision-making in individuals with higher normal-range anxiety.

Amygdala Circuitry During Neurofeedback Training and Symptoms' Change in Adolescents With Varying Depression

Typical adolescents have increased limbic engagement unchecked by regulatory medial prefrontal cortex (PFC) activity as well as heightened self-focus. The resulting emotion dysregulation and self-focused rumination make adolescents more susceptible to depression and suicide attempts. Heightened self-focus converges with mental illness among depressed adolescents, who deploy exaggerated attention to negative self-relevant stimuli and neglect positive ones as part of depression's phenomenology. This results in rigid negative self-representations during an identity formative period with potential lifetime repercussions. Current empirically supported treatments fail to allay recurrent depression. Evidence-based interventions for illnesses linked to suicide ideation and attempts (e.g., depression) underperform across the lifespan. This could be because current treatments are not successful in altering pervasive negative self-representations and affect dysregulation, which is known to be a risk factor of chronic depression. This study departs from the premise that increasing positive self-processing might be protective against chronic depression particularly during adolescence. The present research is a novel investigation of neurofeedback as a potential treatment alternative for adolescent depression. To enhance positive self-processing, we used the happy self-face as a cue to initiate neurofeedback from the bilateral amygdala and hippocampus and adolescents attempted to upregulate that limbic activity through the recall of positive autobiographical memories. We identified limbic functional circuitry engaged during neurofeedback and links to short-term symptoms' change in depression and rumination. We found that depressed youth showed greater right amygdala to right frontocortical connectivity and lower left amygdala to right frontocortical connectivity compared to healthy controls during neurofeedback vs. control conditions. Depressed youth also showed significant symptom reduction. Connectivity between the right amygdala and frontocortical regions was positively correlated with rumination and depression change, but connectivity between frontocortical regions and the left amygdala was negatively correlated with depression change. The results suggest that depressed youth might engage implicit emotion regulation circuitry while healthy youth recruit explicit emotion regulation circuits during neurofeedback. Our findings support a compensatory approach (i.e., target the right amygdala) during future neurofeedback interventions in depressed youth. Future work ought to include a placebo condition or group.

Neurocomputational correlates of learned irrelevance in humans

Inappropriate behaviors may result from acquiring maladaptive associations between irrelevant information in the environment and important events, such as reward or punishment. Pre-exposure effects are believed to prevent the expression of irrelevant associations. For example, learned irrelevance delays the expression of associations between conditioned (CS) and unconditioned (US) stimuli following their uncorrelated presentation. The neuronal substrates of pre-exposure effects in humans are largely unknown because these effects rapidly attenuate when using traditional pre-exposure paradigms. The latter are therefore incompatible with neuroimaging approaches that require many trial repetitions. Moreover, large methodological differences between animal and human research on pre-exposure effects challenge the presumption of shared neurocognitive substrates, and question the prevalent use of pre-exposure effects in animals to model symptoms of human mental disorders. To overcome these limitations, we combined a novel learned irrelevance task with model-based fMRI. We report the results of a model that describes learned irrelevance as a dynamic process, which evolves across trials and integrates the weighting between two state-action values pertaining to 'CS-no US' associations (acquired during pre-exposure) and 'CS-US' associations (acquired during subsequent conditioning). This relative weighting correlated i) positively with the learned irrelevance effect observed in the behavioral task, ii) positively with activity in the entorhinal cortex, and iii) negatively with activity in the nucleus accumbens (NAcc). Furthermore, the model updates the relative weighting of the two state-action values via two separate prediction error (PE) signals that allow the dynamic accumulation of evidence for the CS to predict the 'US' or a 'no US' outcome. One PE signal, designed to increase the relative weight of 'CS-US' associations following 'US' outcomes, correlated with activity in the NAcc, while another PE signal, designed to increase the relative weight of 'CS-no US' associations following 'no US' outcomes, correlated with activity in the basolateral amygdala. By extending previous animal observations to humans, the present study provides a novel approach to foster translational research on pre-exposure effects.

Influence of reward learning on visual attention and eye movements in a naturalistic environment: A virtual reality study

Rewards constitute crucial signals that motivate approach behavior and facilitate the perceptual processing of objects associated with favorable outcomes in past encounters. Reward-related influences on perception and attention have been reliably observed in studies where a reward is paired with a unidimensional low-level visual feature, such as the color or orientation of a line in visual search tasks. However, our environment is drastically different and composed of multidimensional and changing visual features, encountered in complex and dynamic scenes. Here, we designed an immersive virtual reality (VR) experiment using a 4-frame CAVE system to investigate the impact of rewards on attentional orienting and gaze patterns in a naturalistic and ecological environment. Forty-one healthy participants explored a virtual forest and responded to targets appearing on either the left or right side of their path. To test for reward-induced biases in spatial orienting, targets on one side were associated with high reward, whereas those on the opposite side were paired with a low reward. Eye-movements recording showed that left-side high rewards led to subsequent increase of eye gaze fixations towards this side of the path, but no such asymmetry was found after exposure to right-sided high rewards. A milder spatial bias was also observed after left-side high rewards during subsequent exploration of a virtual castle yard, but not during route turn choices along the forest path. Our results indicate that reward-related influences on attention and behavior may be better learned in left than right space, in line with a right hemisphere dominance, and could generalize to another environment to some extent, but not to spatial choices in another decision task, suggesting some domain- or context-specificity. This proof-of-concept study also outlines the advantages and the possible drawbacks of the use of the 3D CAVE immersive platform for VR in neuroscience.

Effects of dopamine on reinforcement learning and consolidation in Parkinson's disease

Emerging evidence suggests that dopamine may modulate learning and memory with important implications for understanding the neurobiology of memory and future therapeutic targeting. An influential hypothesis posits that dopamine biases reinforcement learning. More recent data also suggest an influence during both consolidation and retrieval. Eighteen Parkinson's disease patients learned through feedback ON or OFF medication, with memory tested 24 hr later ON or OFF medication (4 conditions, within-subjects design with matched healthy control group). Patients OFF medication during learning decreased in memory accuracy over the following 24 hr. In contrast to previous studies, however, dopaminergic medication during learning and testing did not affect expression of positive or negative reinforcement. Two further experiments were run without the 24 hr delay, but they too failed to reproduce effects of dopaminergic medication on reinforcement learning. While supportive of a dopaminergic role in consolidation, this study failed to replicate previous findings on reinforcement learning.