Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding

Partner-seeking and limbic dopamine system are enhanced following social loss in male prairie voles (Microtus ochrogaster)

Death of a loved one is recognized as one of life's greatest stresses, and 10%-20% of bereaved individuals will experience a complicated or prolonged grieving period that is characterized by intense yearning for the deceased. The monogamous prairie vole (Microtus ochrogaster) is a rodent species that forms pair bonds between breeding partners and has been used to study the neurobiology of social behaviors and isolation. Male prairie voles do not display distress after isolation from a familiar, same-sex conspecific; however, separation from a bonded female partner increases emotional, stress-related, and proximity-seeking behaviors. Here, we tested the investigatory response of male voles to partner odor during a period of social loss. We found that males who lost their partner spent significantly more time investigating partner odor but not non-partner social odor or food odor. Bachelor males and males in intact pairings did not respond uniquely to any odor. Furthermore, we examined dopamine (DA) receptor mRNA expression in the anterior insula cortex (aIC), nucleus accumbens (NAc), and anterior cingulate (ACC), regions with higher activation in grieving humans. While we found some effects of relationship type on DRD1 and DRD2 expression in some of these regions, loss of a high-quality opposite-sex relationship had a significant effect on DA receptor expression, with pair-bonded/loss males having higher expression in the aIC and ACC compared with pair-bonded/intact and nonbonded/loss males. Together, these data suggest that both relationship type and relationship quality affect reunion-seeking behavior and motivational neurocircuits following social loss of a bonded partner.

Irrelevant Threats Linger and Affect Behavior in High Anxiety

Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.

Average reward rates enable motivational transfer across independent reinforcement learning tasks

Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and was found to interact with dopamine-sensitive cognitive processes, such as vigilance and associative memory encoding. The ARR could therefore provide a bridge between independent tasks when these are performed in temporal proximity, such that the reward rate obtained in one task could influence performance in a second subsequent task. Reinforcement learning depends on the coding of prediction error signals by dopamine neurons and their downstream targets, in particular the nucleus accumbens. Because these brain regions also respond to changes in ARR, reinforcement learning may be vulnerable to changes in ARR. To test this hypothesis, we designed a novel paradigm in which participants (n = 245) performed two probabilistic reinforcement learning tasks presented in interleaved trials. The ARR was controlled by an "induction" task which provided feedback with a low (p = 0.58), a medium (p = 0.75), or a high probability of reward (p = 0.92), while the impact of ARR on reinforcement learning was tested by a second "reference" task with a constant reward probability (p = 0.75). We find that performance was significantly lower in the reference task when the induction task provided low reward probabilities (i.e., during low levels of ARR), as compared to the medium and high ARR conditions. Behavioral modeling further revealed that the influence of ARR is best described by models which accumulates average rewards (rather than average prediction errors), and where the ARR directly modulates the prediction error signal (rather than affecting learning rates or exploration). Our results demonstrate how affective information in one domain may transfer and affect motivated behavior in other domains. These findings are particularly relevant for understanding mood disorders, but may also inform abnormal behaviors attributed to dopamine dysfunction.

Hippocampal convergence during anticipatory midbrain activation promotes subsequent memory formation

The hippocampus has been a focus of memory research since H.M's surgery abolished his ability to form new memories, yet its mechanistic role in memory remains debated. Here, we identify a candidate memory mechanism: an anticipatory hippocampal "convergence state", observed while awaiting valuable information, and which predicts subsequent learning. During fMRI, participants viewed trivia questions eliciting high or low curiosity, followed seconds later by its answer. We reasoned that encoding success requires a confluence of conditions, so that hippocampal states more conducive to memory formation should converge in state space. To operationalize convergence of neural states, we quantified the typicality of multivoxel patterns in the medial temporal lobes during anticipation and encoding of trivia answers. We found that the typicality of anticipatory hippocampal patterns increased during high curiosity. Crucially, anticipatory hippocampal pattern typicality increased with dopaminergic midbrain activation and uniquely accounted for the association between midbrain activation and subsequent recall. We propose that hippocampal convergence states may complete a cascade from motivation and midbrain activation to memory enhancement, and may be a general predictor of memory formation.

The dissociating of reward feedback on familiarity and recollection processing: evidence from event-related potential

OBJECTIVES

Although previous studies have explored the effect of reward feedback on recognition memory, electrophysiological evidence for reward-enhanced memory and its underlying processing mechanisms remains unclear.

METHODS

This study adopts reward-learning and recognition memory tasks. Participants were asked to learn the reward values of two-color images (each color image had either reward or nonreward feedback) in the reward-learning task, and then tested their recognition memory performance with reward and nonreward feedback items.

RESULTS

Results demonstrated that the recognition memory performance of rewarded items was better than that of nonrewarded items. During the reward-learning period, nonreward feedback elicited larger feedback-related negativity (FRN) and P300 amplitudes compared with reward feedback. The findings indicated that participants mainly engaged in prediction error processing in the early stage, followed by comparing and context update of the learned items. During the recognition memory period, reward items elicited larger FN400 amplitude and smaller LPC amplitude compared with nonreward items. This suggests that reward item retrieval has deeper memory traces and can identify items faster, relying mainly on familiarity processing. Conversely, nonreward, as a general or inhibitory item, requires more detail and cognitive resources, that is, relies on recollection processing.

CONCLUSIONS

These findings indicated that participants had different process patterns between reward and nonreward items during recognition retrieval.

Neural Basis of Associative Learning in Trichotillomania and Skin-Picking Disorder

Disorders such as Trichotillomania (TTM) and skin-picking disorder (SPD) are associated with reduced flexibility and increased internally focused attention. While the basal ganglia have been hypothesized to play a key role, the mechanisms underlying learning and flexible accommodation of new information is unclear. Using a Bayesian Learning Model, we evaluated the neural basis of learning and accommodation in individuals with TTM and/or SPD. Participants were 127 individuals with TTM and/or SPD (TTM/SPD) recruited from three sites (age 18-57, 84% female) and 26 healthy controls (HC). During fMRI, participants completed a shape-button associative learning and reversal fMRI task. Above-threshold clusters were identified where the Initial Learning-Reversals BOLD activation contrast differed significantly (p < .05 FDR-corrected) between the two groups. A priori, effects were anticipated in predefined ROIs in bilateral basal ganglia, with exploratory analyses in the hippocampus, dorsolateral prefrontal cortex (dlPFC), and dorsal anterior cingulate cortex (dACC). Relative to HC, individuals with TTM/SPD demonstrated reduced activation during initial learning compared to reversal learning in the right basal ganglia. Similarly, individuals with TTM/SPD demonstrated reduced activation during initial learning compared to reversal learning in several clusters in the dlPFC and dACC compared to HC. Individuals with TTM/SPD may form or reform visual stimulus-motor response associations through different brain mechanisms than healthy controls. The former exhibit altered activation within the basal ganglia, dlPFC, and dACC during an associative learning task compared to controls, reflecting reduced frontal-subcortical activation during initial learning. Future work should determine whether these neural deficits may be restored with targeted treatment.

Towards proteinoid computers. Hypothesis paper

Proteinoids - thermal proteins - are produced by heating amino acids to their melting point and initiation of polymerisation to produce polymeric chains. Proteinoids swell in aqueous solution into hollow microspheres. The proteinoid microspheres produce endogenous burst of electrical potential spikes and change patterns of their electrical activity in response to illumination. The microspheres can interconnect by pores and tubes and form networks with a programmable growth. We speculate on how ensembles of the proteinoid microspheres can be developed into unconventional computing devices.

Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.

Neural substrates of the interplay between cognitive load and emotional involvement in bilingual decision making

Prior work has reported that foreign language influences decision making by either reducing access to emotion or imposing additional cognitive demands. In this fMRI study, we employed a cross-task design to assess at the neural level whether and how the interaction between cognitive load and emotional involvement is affected by language (native L1 vs. foreign L2). Participants completed a Lexico-semantic task where in each trial they were presented with a neutrally or a negatively valenced word either in L1 or L2, either under cognitive load or not. We manipulated cognitive load by varying the difficulty of the task: to increase cognitive demands, we used traditional characters instead of simplified ones in L1 (Chinese), and words with capital letters instead of lowercase letters in L2 (English). After each trial, participants decided whether to take a risky decision in a gambling game. During the Gamling task, left amygdala and right insula were more activated after having processed a negative word under cognitive load in the Lexico-semantic task. However, this was true for L1 but not for L2. In particular, in L1, cognitive load facilitated rather than hindered access to emotion. Further suggesting that cognitive load can enhance emotional sensitivity in L1 but not in L2, we found that functional connectivity between reward-related striatum and right insula increased under cognitive load only in L1. Overall, results suggest that cognitive load in L1 can favor access to emotion and lead to impulsive decision making, whereas cognitive load in L2 can attenuate access to emotion and lead to more rational decisions.

Reward anticipation selectively boosts encoding of gist for visual objects

Reward anticipation at encoding enhances later recognition, but it is unknown to what extent different levels of processing at encoding (gist vs. detail) can benefit from reward-related memory enhancement. In the current study, participants (N = 50) performed an incidental encoding task in which they made gist-related or detail-related judgments about pairs of visual objects while in anticipation of high or low reward. Results of a subsequent old/new recognition test revealed a reward-related memory benefit that was specific to objects from pairs encoded in the attention-to-gist condition. These findings are consistent with the theory of long-axis specialization along the human hippocampus, which localizes gist-based memory processes to the anterior hippocampus, a region highly interconnected with the dopaminergic reward network.

Dopamine Receptor Subtypes, Physiology and Pharmacology: New Ligands and Concepts in Schizophrenia

Dopamine receptors are widely distributed within the brain where they play critical modulator roles on motor functions, motivation and drive, as well as cognition. The identification of five genes coding for different dopamine receptor subtypes, pharmacologically grouped as D1- (D1 and D5) or D2-like (D2S, D2L, D3, and D4) has allowed the demonstration of differential receptor function in specific neurocircuits. Recent observation on dopamine receptor signaling point at dopamine-glutamate-NMDA neurobiology as the most relevant in schizophrenia and for the development of new therapies. Progress in the chemistry of D1- and D2-like receptor ligands (agonists, antagonists, and partial agonists) has provided more selective compounds possibly able to target the dopamine receptors homo and heterodimers and address different schizophrenia symptoms. Moreover, an extensive evaluation of the functional effect of these agents on dopamine receptor coupling and intracellular signaling highlights important differences that could also result in highly differentiated clinical pharmacology. The review summarizes the recent advances in the field, addressing the relevance of emerging new targets in schizophrenia in particular in relation to the dopamine - glutamate NMDA systems interactions.