Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Anticipatory and consummatory neural correlates of monetary and music rewarding stimuli

Most of the literature on the neural bases of human reward and punishment processing has used monetary gains and losses, but less is known about the neurophysiological mechanisms underlying the anticipation and consumption of other types of rewarding stimuli. In the present study, EEG was recorded from 19 participants who completed a modified version of the Monetary Incentive Delay (MID) task. During the task, cues providing information about potential future outcomes were presented to the participants. Then, they had to respond rapidly to a target stimulus to win money or listening to pleasant music, or to avoid losing money or listening to unpleasant music. Results revealed similar responses for monetary and music cues, with increased activity for cues indicating potential gains compared to losses. However, differences emerged in the outcome phase between money and music. Monetary outcomes showed an interaction between the type of the cue and the outcome in the Feedback Related Negativity and Fb-P3 ERPs and increased theta activity increased for negative feedbacks. In contrast, music outcomes showed significant interactions in the Fb-P3 and theta activities. These findings suggest similar neurophysiological mechanisms in processing cues for potential positive or negative outcomes in these two types of stimuli.

Striatal GABA levels correlate with risk sensitivity in monetary loss

Background

Decision-making under risk is a common challenge. It is known that risk-taking behavior varies between contexts of reward and punishment, yet the mechanisms underlying this asymmetry in risk sensitivity remain unclear.

Methods

This study used a monetary task to investigate neurochemical mechanisms and brain dynamics underpinning risk sensitivity. Twenty-eight participants engaged in a task requiring selection of visual stimuli to maximize monetary gains and minimize monetary losses. We modeled participant trial-and-error processes using reinforcement learning.

Results

Participants with higher subjective utility parameters showed risk preference in the gain domain (r = -0.59) and risk avoidance in the loss domain (r = -0.77). Magnetic resonance spectroscopy (MRS) revealed that risk avoidance in the loss domain was associated with γ-aminobutyric acid (GABA) levels in the ventral striatum (r = -0.42), but not in the insula (r = -0.15). Using functional magnetic resonance imaging (fMRI), we tested whether risk-sensitive brain dynamics contribute to participant risky choices. Energy landscape analyses demonstrated that higher switching rates between brain states, including the striatum and insula, were correlated with risk avoidance in the loss domain (r = -0.59), a relationship not observed in the gain domain (r = -0.02).

Conclusions

These findings from MRS and fMRI suggest that distinct mechanisms are involved in gain/loss decision making, mediated by subcortical neurometabolite levels and brain dynamic transitions.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

To lie or to tell the truth? The influence of processing the opponent's feedback on the forthcoming choice

Introduction

The brain mechanisms of deceptive behavior are relatively well studied, and the key brain regions involved in its processing were established. At the same time, the brain mechanisms underlying the processes of preparation for deception are less known.

Methods

We studied BOLD-signal changes during the presentation of the opponent's feedback to a previous deceptive or honest action during the computer game. The goal of the game was to mislead the opponent either by means of deception or by means of telling the truth.

Results

As a result, it was shown that several brain regions that were previously demonstrated as involved in deception execution, such as the left anterior cingulate cortex and anterior insula, also underlie processes related to deception preparation.

Discussion

The results obtained also allowed us to suggest that brain regions responsible for performance monitoring, intention assessment, suppression of non-selected solutions, and reward processing could be involved in shaping future action selection and preparation for deception. By shedding light on the brain mechanisms underlying deception, our study contributes to a deeper understanding of this complex cognitive process. Furthermore, it emphasizes the significance of exploring brain mechanisms governing the choice between deception and truth at various stages of decision-making.

Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex

Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.

Prediction error processing and sharpening of expected information across the face-processing hierarchy

The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.

Contemporary neurocognitive models of memory: A descriptive comparative analysis

The great complexity involved in the study of memory has given rise to numerous hypotheses and models associated with various phenomena at different levels of analysis. This has allowed us to delve deeper in our knowledge about memory but has also made it difficult to synthesize and integrate data from different lines of research. In this context, this work presents a descriptive comparative analysis of contemporary models that address the structure and function of multiple memory systems. The main goal is to outline a panoramic view of the key elements that constitute these models in order to visualize both the current state of research and possible future directions. The elements that stand out from different levels of analysis are distributed neural networks, hierarchical organization, predictive coding, homeostasis, and evolutionary perspective.

Timing along the cardiac cycle modulates neural signals of reward-based learning

Natural fluctuations in cardiac activity modulate brain activity associated with sensory stimuli, as well as perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the relationship between fluctuations in heart activity and other internal representations. Here we investigate whether the cardiac cycle relates to learning-related internal representations - absolute and signed prediction errors. We combined machine learning techniques with electroencephalography with both simple, direct indices of task performance and computational model-derived indices of learning. Our results demonstrate that just as people are more sensitive to low magnitude, near-threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. However, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger differences in their prediction error representations between cardiac cycles exhibited higher learning rates and greater task accuracy.

Increased ventral anterior insular connectivity to sports betting availability indexes problem gambling

With the advent of digital technologies, online sports betting is spurring a fast-growing expansion. In this study, we examined how sports betting availability modulates the brain connectivity of frequent sports bettors with [problem bettors (PB)] or without [non-problem bettors (NPB)] problematic sports betting. We conducted functional connectivity analyses centred on the ventral anterior insular cortex (vAI), a brain region playing a key role in the dynamic interplay between reward-based processes. We re-analysed a dataset on sports betting availability undertaken in PB (n = 30) and NPB (n = 35). Across all participants, we observed that sports betting availability elicited positive vAI coupling with extended clusters of brain activation (encompassing the putamen, cerebellum, occipital, temporal, precentral and central operculum regions) and negative vAI coupling with the orbitofrontal cortex. Between-group analyses showed increased positive vAI coupling in the PB group, as compared with the NPB group, in the left lateral occipital cortex, extending to the left inferior frontal gyrus, the anterior cingulate gyrus and the right frontal pole. Taken together, these results are in line with the central assumptions of triadic models of addictions, which posit that the insular cortex plays a pivotal role in promoting the drive and motivation to get a reward by 'hijacking' goal-oriented processes toward addiction-related cues. Taken together, these findings showed that vAI functional connectivity is sensitive not only to gambling availability but also to the status of problematic sport betting.

Pramipexole Enhances Reward Learning by Preserving Value Estimates

BACKGROUND

Dopamine D2-like agonists show promise as treatments for depression. They are thought to act by enhancing reward learning; however, the mechanisms by which they achieve this are not clear. Reinforcement learning accounts describe 3 distinct candidate mechanisms: increased reward sensitivity, increased inverse decision-temperature, and decreased value decay. As these mechanisms produce equivalent effects on behavior, arbitrating between them requires measurement of how expectations and prediction errors are altered. We characterized the effects of 2 weeks of the D2-like agonist pramipexole on reward learning and used functional magnetic resonance imaging measures of expectation and prediction error to assess which of these 3 mechanistic processes were responsible for the behavioral effects.

METHODS

Forty healthy volunteers (50% female) were randomized to 2 weeks of pramipexole (titrated to 1 mg/day) or placebo in a double-blind, between-subject design. Participants completed a probabilistic instrumental learning task before and after the pharmacological intervention, with functional magnetic resonance imaging data collected at the second visit. Asymptotic choice accuracy and a reinforcement learning model were used to assess reward learning.

RESULTS

Pramipexole increased choice accuracy in the reward condition with no effect on losses. Participants who received pramipexole had increased blood oxygen level-dependent response in the orbital frontal cortex during the expectation of win trials but decreased blood oxygen level-dependent response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.

CONCLUSIONS

The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible mechanism for pramipexole's antidepressant effect.

Mild Deficits in Fear Learning: Evidence from Humans and Mice with Cerebellar Cortical Degeneration

Functional brain imaging studies in humans suggest involvement of the cerebellum in fear conditioning but do not allow conclusions about the functional significance. The main aim of the present study was to examine whether patients with cerebellar degeneration show impaired fear conditioning and whether this is accompanied by alterations in cerebellar cortical activations. To this end, a 2 d differential fear conditioning study was conducted in 20 cerebellar patients and 21 control subjects using a 7 tesla (7 T) MRI system. Fear acquisition and extinction training were performed on day 1, followed by recall on day 2. Cerebellar patients learned to differentiate between the CS+ and CS-. Acquisition and consolidation of learned fear, however, was slowed. Additionally, extinction learning appeared to be delayed. The fMRI signal was reduced in relation to the prediction of the aversive stimulus and altered in relation to its unexpected omission. Similarly, mice with cerebellar cortical degeneration (spinocerebellar ataxia type 6, SCA6) were able to learn the fear association, but retrieval of fear memory was reduced. In sum, cerebellar cortical degeneration led to mild abnormalities in the acquisition of learned fear responses in both humans and mice, particularly manifesting postacquisition training. Future research is warranted to investigate the basis of altered fMRI signals related to fear learning.

Neural correlates of prediction error in patients with schizophrenia: evidence from an fMRI meta-analysis

Abnormal processes of learning from prediction errors, i.e. the discrepancies between expectations and outcomes, are thought to underlie motivational impairments in schizophrenia. Although dopaminergic abnormalities in the mesocorticolimbic reward circuit have been found in patients with schizophrenia, the pathway through which prediction error signals are processed in schizophrenia has yet to be elucidated. To determine the neural correlates of prediction error processing in schizophrenia, we conducted a meta-analysis of whole-brain neuroimaging studies that investigated prediction error signal processing in schizophrenia patients and healthy controls. A total of 14 studies (324 schizophrenia patients and 348 healthy controls) using the reinforcement learning paradigm were included. Our meta-analysis showed that, relative to healthy controls, schizophrenia patients showed increased activity in the precentral gyrus and middle frontal gyrus and reduced activity in the mesolimbic circuit, including the striatum, thalamus, amygdala, hippocampus, anterior cingulate cortex, insula, superior temporal gyrus, and cerebellum, when processing prediction errors. We also found hyperactivity in frontal areas and hypoactivity in mesolimbic areas when encoding prediction error signals in schizophrenia patients, potentially indicating abnormal dopamine signaling of reward prediction error and suggesting failure to represent the value of alternative responses during prediction error learning and decision making.

Dopamine release in human associative striatum during reversal learning

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

Pramipexole for the Treatment of Depression: Efficacy and Mechanisms

Dopaminergic mechanisms are a plausible treatment target for patients with clinical depression but are relatively underexplored in conventional antidepressant medications. There is continuing interest in the potential antidepressant effects of the dopamine receptor agonist, pramipexole, with data from both case series and controlled trials indicating that this agent may produce benefit for patients with difficult-to-treat depression. Pramipexole's therapeutic utility in depression is likely to be expressed through alterations in reward mechanisms which are strongly influenced by dopamine pathways and are known to function abnormally in depressed patients. Our work in healthy participants using brain imaging in conjunction with computational modelling suggests that repeated pramipexole facilitates reward learning by inhibiting value decay. This mechanism needs to be confirmed in larger clinical trials in depressed patients. Such studies will also allow assessment of whether baseline performance in reward learning in depression predicts therapeutic response to pramipexole treatment.

Distinct spatiotemporal brainstem pathways of outcome valence during reward- and punishment-based learning

Learning to seek rewards and avoid punishments, based on positive and negative choice outcomes, is essential for human survival. Yet, the neural underpinnings of outcome valence in the human brainstem and the extent to which they differ in reward and punishment learning contexts remain largely elusive. Here, using simultaneously acquired electroencephalography and functional magnetic resonance imaging data, we show that during reward learning the substantia nigra (SN)/ventral tegmental area (VTA) and locus coeruleus are initially activated following negative outcomes, while the VTA subsequently re-engages exhibiting greater responses for positive than negative outcomes, consistent with an early arousal/avoidance response and a later value-updating process, respectively. During punishment learning, we show that distinct raphe nucleus and SN subregions are activated only by negative outcomes with a sustained post-outcome activity across time, supporting the involvement of these brainstem subregions in avoidance behavior. Finally, we demonstrate that the coupling of these brainstem structures with other subcortical and cortical areas helps to shape participants' serial choice behavior in each context.

A novel technique for delineating the effect of variation in the learning rate on the neural correlates of reward prediction errors in model-based fMRI

Introduction

Computational models play an increasingly important role in describing variation in neural activation in human neuroimaging experiments, including evaluating individual differences in the context of psychiatric neuroimaging. In particular, reinforcement learning (RL) techniques have been widely adopted to examine neural responses to reward prediction errors and stimulus or action values, and how these might vary as a function of clinical status. However, there is a lack of consensus around the importance of the precision of free parameter estimation for these methods, particularly with regard to the learning rate. In the present study, I introduce a novel technique which may be used within a general linear model (GLM) to model the effect of mis-estimation of the learning rate on reward prediction error (RPE)-related neural responses.

Methods

Simulations employed a simple RL algorithm, which was used to generate hypothetical neural activations that would be expected to be observed in functional magnetic resonance imaging (fMRI) studies of RL. Similar RL models were incorporated within a GLM-based analysis method including derivatives, with individual differences in the resulting GLM-derived beta parameters being evaluated with respect to the free parameters of the RL model or being submitted to other validation analyses.

Results

Initial simulations demonstrated that the conventional approach to fitting RL models to RPE responses is more likely to reflect individual differences in a reinforcement efficacy construct (lambda) rather than learning rate (alpha). The proposed method, adding a derivative regressor to the GLM, provides a second regressor which reflects the learning rate. Validation analyses were performed including examining another comparable method which yielded highly similar results, and a demonstration of sensitivity of the method in presence of fMRI-like noise.

Conclusion

Overall, the findings underscore the importance of the lambda parameter for interpreting individual differences in RPE-coupled neural activity, and validate a novel neural metric of the modulation of such activity by individual differences in the learning rate. The method is expected to find application in understanding aberrant reinforcement learning across different psychiatric patient groups including major depression and substance use disorder.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Debates on the dorsomedial prefrontal/dorsal anterior cingulate cortex: insights for future research

The dorsomedial prefrontal cortex/dorsal anterior cingulate cortex (dmPFC/dACC) is a brain area subject to many theories and debates over its function(s). Even its precise anatomical borders are subject to much controversy. In the past decades, the dmPFC/dACC has been associated with more than 15 different cognitive processes, which sometimes appear quite unrelated (e.g. body perception, cognitive conflict). As a result, understanding what the dmPFC/dACC does has become a real challenge for many neuroscientists. Several theories of this brain area's function(s) have been developed, leading to successive and competitive publications bearing different models, which sometimes contradict each other. During the last two decades, the lively scientific exchanges around the dmPFC/dACC have promoted fruitful research in cognitive neuroscience. In this review, we provide an overview of the anatomy of the dmPFC/dACC, summarize the state of the art of functions that have been associated with this brain area and present the main theories aiming at explaining the dmPFC/dACC function(s). We explore the commonalities and the arguments between the different theories. Finally, we explain what can be learned from these debates for future investigations of the dmPFC/dACC and other brain regions' functions.

Reward is not reward: Differential impacts of primary and secondary rewards on expectation, outcome, and prediction error in the human brain's reward processing regions

According to their nature, rewarding stimuli are classified as primary (e.g., food, sex) and secondary (e.g., money) rewards. Neuroimaging studies have provided valuable insights in neural reward processing and its various aspects including reward expectation, outcome and prediction error encoding. However, there is only limited evidence of whether the two different types of rewards are processed in common or distinct brain areas, in particular when considering the different functions of reward processing. We analyzed a sample of 42 healthy, male participants using task-based functional magnetic resonance imaging (fMRI) during a variant of the monetary incentive delay task. We aimed to investigate the effects of three different rewarding stimuli-two primary (food and sex) and one secondary (money)-on the various functions of reward processing. To provide a thorough description, we focused on 12 brain regions of interest and utilized the Bayes factor bound (BFB) to express stimulus-related main effects and pairwise differences at different levels of evidence, ranging from weak to decisive. Our results revealed a dominance of sexually charged stimuli in engaging the brain's reward structures for all investigated aspects of reward processing. Nevertheless, the ventral tegmental area, amygdala, ventral caudate, ventromedial prefrontal cortex, subgenual anterior cingulate cortex, and lateral orbitofrontal cortex were activated by both primary and secondary reward outcomes. For other reward processing functions, i.e., reward expectation and the prediction error, effects of the different stimuli were weaker, and effects from one reward type cannot easily be generalized to the other.

Remembering unexpected beauty: Contributions of the ventral striatum to the processing of reward prediction errors regarding the facial attractiveness in face memory

The COVID-19 pandemic has led people to predict facial attractiveness from partially covered faces. Differences in the predicted and observed facial attractiveness (i.e., masked and unmasked faces, respectively) are defined as reward prediction error (RPE) in a social context. Cognitive neuroscience studies have elucidated the neural mechanisms underlying RPE-induced memory improvements in terms of monetary rewards. However, little is known about the mechanisms underlying RPE-induced memory modulation in terms of social rewards. To elucidate this, the present functional magnetic resonance imaging (fMRI) study investigated activity and functional connectivity during face encoding. In encoding trials, participants rated the predicted attractiveness of faces covered except for around the eyes (prediction phase) and then rated the observed attractiveness of these faces without any cover (outcome phase). The difference in ratings between these phases was defined as RPE in facial attractiveness, and RPE was categorized into positive RPE (increased RPE from the prediction to outcome phases), negative RPE (decreased RPE from the prediction to outcome phases), and non-RPE (no difference in RPE between the prediction and outcome phases). During retrieval, participants were presented with individual faces that had been seen and unseen in the encoding trials, and were required to judge whether or not each face had been seen in the encoding trials. Univariate activity in the ventral striatum (VS) exhibited a linear increase with increased RPE in facial attractiveness. In the multivariate pattern analysis (MVPA), activity patterns in the VS and surrounding areas (extended VS) significantly discriminated between positive/negative RPE and non-RPE. In the functional connectivity analysis, significant functional connectivity between the extended VS and the hippocampus was observed most frequently in positive RPE. Memory improvements by face-based RPE could be involved in functional networks between the extended VS (representing RPE) and the hippocampus, and the interaction could be modulated by RPE values in a social context.

Human thalamic low-frequency oscillations correlate with expected value and outcomes during reinforcement learning

Reinforcement-based adaptive decision-making is believed to recruit fronto-striatal circuits. A critical node of the fronto-striatal circuit is the thalamus. However, direct evidence of its involvement in human reinforcement learning is lacking. We address this gap by analyzing intra-thalamic electrophysiological recordings from eight participants while they performed a reinforcement learning task. We found that in both the anterior thalamus (ATN) and dorsomedial thalamus (DMTN), low frequency oscillations (LFO, 4-12 Hz) correlated positively with expected value estimated from computational modeling during reward-based learning (after outcome delivery) or punishment-based learning (during the choice process). Furthermore, LFO recorded from ATN/DMTN were also negatively correlated with outcomes so that both components of reward prediction errors were signaled in the human thalamus. The observed differences in the prediction signals between rewarding and punishing conditions shed light on the neural mechanisms underlying action inhibition in punishment avoidance learning. Our results provide insight into the role of thalamus in reinforcement-based decision-making in humans.

Neural responses to reward valence and magnitude from pre- to early adolescence

BACKGROUND

Neural activation during reward processing is thought to underlie critical behavioral changes that take place during the transition to adolescence (e.g., learning, risk-taking). Though literature on the neural basis of reward processing in adolescence is booming, important gaps remain. First, more information is needed regarding changes in functional neuroanatomy in early adolescence. Another gap is understanding whether sensitivity to different aspects of the incentive (e.g., magnitude and valence) changes during the transition into adolescence. We used fMRI from a large sample of preadolescent children to characterize neural responses to incentive valence vs. magnitude during anticipation and feedback, and their change over a period of two years.

METHODS

Data were taken from the Adolescent Cognitive and Brain DevelopmentSM (ABCD®) study release 3.0. Children completed the Monetary Incentive Delay task at baseline (ages 9-10) and year 2 follow-up (ages 11-12). Based on data from two sites (N = 491), we identified activation-based Regions of Interest (ROIs; e.g., striatum, prefrontal regions, etc.) that were sensitive to trial type (win $5, win $0.20, neutral, lose $0.20, lose $5) during anticipation and feedback phases. Then, in an independent subsample (N = 1470), we examined whether these ROIs were sensitive to valence and magnitude and whether that sensitivity changed over two years.

RESULTS

Our results show that most ROIs involved in reward processing (including the striatum, prefrontal cortex, and insula) are specialized, i.e., mainly sensitive to either incentive valence or magnitude, and this sensitivity was consistent over a 2-year period. The effect sizes of time and its interactions were significantly smaller (0.002≤η2≤0.02) than the effect size of trial type (0.06≤η2≤0.30). Interestingly, specialization was moderated by reward processing phase but was stable across development. Biological sex and pubertal status differences were few and inconsistent. Developmental changes were mostly evident during success feedback, where neural reactivity increased over time.

CONCLUSIONS

Our results suggest sub-specialization to valence vs. magnitude within many ROIs of the reward circuitry. Additionally, in line with theoretical models of adolescent development, our results suggest that the ability to benefit from success increases from pre- to early adolescence. These findings can inform educators and clinicians and facilitate empirical research of typical and atypical motivational behaviors during a critical time of development.

Altered functional connectivity of the amygdala across variants of callous-unemotional traits: A resting-state fMRI study in children and adolescents

Over the past years, research has shown that primary (high callousness and low anxiety) and secondary (high callousness and anxiety) variants of CU traits may be associated with opposite amygdala activity (hypo- and hyper-reactivity, respectively). However, their differences in amygdala functional connectivity remains largely unexplored. We conducted a Latent Profile Analysis on a large sample of adolescents (n = 1416) to identify homogeneous subgroups with different levels of callousness and anxiety. We then performed a seed-to-voxel connectivity analysis on resting-state fMRI data to compare subgroups on connectivity patterns of the amygdala. We examined the results in relation to conduct problems to identify potential neural risk factors. The Latent Profile Analysis revealed four subgroups, including the primary and secondary variants, anxious, and typically developing adolescents. The seed-to-voxel analyses showed that the primary variant was mainly characterized by increased connectivity between the left amygdala and left thalamus. The secondary variant exhibited deficient connectivity between the amygdala and the dorsomedial prefrontal cortex, temporo-parietal junction, premotor, and postcentral gyrus. Both variants showed increased connectivity between the left amygdala and the right thalamus but exhibited opposite functional connectivity between the left amygdala and the parahippocampal gyrus. Dimensional analyses indicated that conduct problems may play a mediating role between callousness and amygdala-dmPFC functional connectivity across youths with already high levels of callousness. Our study highlights that both variants differ in the functional connectivity of the amygdala. Our results support the importance of disentangling the heterogeneity of adolescents at risk for conduct problems in neuroimaging.

Prediction errors arising from switches between major and minor modes in music: An fMRI study

The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners' neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach's keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener's brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.

The human language system, including its inferior frontal component in "Broca's area," does not support music perception

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

l-DOPA and oxytocin influence the neurocomputational mechanisms of self-benefitting and prosocial reinforcement learning

Humans learn through reinforcement, particularly when outcomes are unexpected. Recent research suggests similar mechanisms drive how we learn to benefit other people, that is, how we learn to be prosocial. Yet the neurochemical mechanisms underlying such prosocial computations remain poorly understood. Here, we investigated whether pharmacological manipulation of oxytocin and dopamine influence the neurocomputational mechanisms underlying self-benefitting and prosocial reinforcement learning. Using a double-blind placebo-controlled cross-over design, we administered intranasal oxytocin (24 IU), dopamine precursor l-DOPA (100 mg + 25 mg carbidopa), or placebo over three sessions. Participants performed a probabilistic reinforcement learning task with potential rewards for themselves, another participant, or no one, during functional magnetic resonance imaging. Computational models of reinforcement learning were used to calculate prediction errors (PEs) and learning rates. Participants behavior was best explained by a model with different learning rates for each recipient, but these were unaffected by either drug. On the neural level, however, both drugs blunted PE signaling in the ventral striatum and led to negative signaling of PEs in the anterior mid-cingulate cortex, dorsolateral prefrontal cortex, inferior parietal gyrus, and precentral gyrus, compared to placebo, and regardless of recipient. Oxytocin (versus placebo) administration was additionally associated with opposing tracking of self-benefitting versus prosocial PEs in dorsal anterior cingulate cortex, insula and superior temporal gyrus. These findings suggest that both l-DOPA and oxytocin induce a context-independent shift from positive towards negative tracking of PEs during learning. Moreover, oxytocin may have opposing effects on PE signaling when learning to benefit oneself versus another.

Neural bases of frustration-aggression theory: A multi-domain meta-analysis of functional neuroimaging studies

BACKGROUND

Early evidence suggests that unexpected non-reward may increase the risk for aggressive behaviors. Despite the growing interest in understanding brain functions that may be implicated in aggressive behaviors, the neural processes underlying such frustrative events remain largely unknown. Furthermore, meta-analytic results have produced discrepant results, potentially due to substantial differences in the definition of anger/aggression constructs.

METHODS

Therefore, we conducted a coordinate-based meta-analysis, using the activation likelihood estimation algorithm, on neuroimaging studies examining reward omission and retaliatory behaviors in healthy subjects. Conjunction analyses were further examined to discover overlapping brain activations across these meta-analytic maps.

RESULTS

Frustrative non-reward deactivated the orbitofrontal cortex, ventral striatum and posterior cingulate cortex, whereas increased activations were observed in midcingulo-insular regions. Retaliatory behaviors recruited the left fronto-insular and anterior midcingulate cortices, the dorsal caudate and the primary somatosensory cortex. Conjunction analyses revealed that both strongly activated midcingulo-insular regions.

LIMITATIONS

Spatial overlap between neural correlates of frustration and retaliatory behaviors was conducted using a conjunction analysis. Therefore, neurobiological markers underlying the temporal sequence of the frustration-aggression theory should be interpreted with caution.

CONCLUSIONS

Nonetheless, our results underscore the role of anterior midcingulate/pre-supplementary motor area and fronto-insular cortex in both frustration and retaliatory behaviors. A neurobiological framework for understanding frustration-based impulsive aggression is provided.

Neuro-computational mechanisms and individual biases in action-outcome learning under moral conflict

Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.

Acute stress alters probabilistic reversal learning in healthy male adults

Behavioural adaptation is a fundamental cognitive ability, ensuring survival by allowing for flexible adjustment to changing environments. In laboratory settings, behavioural adaptation can be measured with reversal learning paradigms requiring agents to adjust reward learning to stimulus-action-outcome contingency changes. Stress is found to alter flexibility of reward learning, but effect directionality is mixed across studies. Here, we used model-based functional MRI (fMRI) in a within-subjects design to investigate the effect of acute psychosocial stress on flexible behavioural adaptation. Healthy male volunteers (n = 28) did a reversal learning task during fMRI in two sessions, once after the Trier Social Stress Test (TSST), a validated psychosocial stress induction method, and once after a control condition. Stress effects on choice behaviour were investigated using multilevel generalized linear models and computational models describing different learning processes that potentially generated the data. Computational models were fitted using a hierarchical Bayesian approach, and model-derived reward prediction errors (RPE) were used as fMRI regressors. We found that acute psychosocial stress slightly increased correct response rates. Model comparison revealed that double-update learning with altered choice temperature under stress best explained the observed behaviour. In the brain, model-derived RPEs were correlated with BOLD signals in striatum and ventromedial prefrontal cortex (vmPFC). Striatal RPE signals for win trials were stronger during stress compared with the control condition. Our study suggests that acute psychosocial stress could enhance reversal learning and RPE brain responses in healthy male participants and provides a starting point to explore these effects further in a more diverse population.

Multivariate functional neuroimaging analyses reveal that strength-dependent face expectations are represented in higher-level face-identity areas

Perception is an active inference in which prior expectations are combined with sensory input. It is still unclear how the strength of prior expectations is represented in the human brain. The strength, or precision, of a prior could be represented with its content, potentially in higher-level sensory areas. We used multivariate analyses of functional resonance imaging data to test whether expectation strength is represented together with the expected face in high-level face-sensitive regions. Participants were trained to associate images of scenes with subsequently presented images of different faces. Each scene predicted three faces, each with either low, intermediate, or high probability. We found that anticipation enhances the similarity of response patterns in the face-sensitive anterior temporal lobe to response patterns specifically associated with the image of the expected face. In contrast, during face presentation, activity increased for unexpected faces in a typical prediction error network, containing areas such as the caudate and the insula. Our findings show that strength-dependent face expectations are represented in higher-level face-identity areas, supporting hierarchical theories of predictive processing according to which higher-level sensory regions represent weighted priors.

Experiential values are underweighted in decisions involving symbolic options

Standard models of decision-making assume each option is associated with subjective value, regardless of whether this value is inferred from experience (experiential) or explicitly instructed probabilistic outcomes (symbolic). In this study, we present results that challenge the assumption of unified representation of experiential and symbolic value. Across nine experiments, we presented participants with hybrid decisions between experiential and symbolic options. Participants' choices exhibited a pattern consistent with a systematic neglect of the experiential values. This normatively irrational decision strategy held after accounting for alternative explanations, and persisted even when it bore an economic cost. Overall, our results demonstrate that experiential and symbolic values are not symmetrically considered in hybrid decisions, suggesting they recruit different representational systems that may be assigned different priority levels in the decision process. These findings challenge the dominant models commonly used in value-based decision-making research.

Large-scale neural network computations and multivariate representations during approach-avoidance conflict decision-making

Many real-world situations require navigating decisions for both reward and threat. While there has been significant progress in understanding mechanisms of decision-making and mediating neurocircuitry separately for reward and threat, there is limited understanding of situations where reward and threat contingencies compete to create approach-avoidance conflict (AAC). Here, we leverage computational learning models, independent component analysis (ICA), and multivariate pattern analysis (MVPA) approaches to understand decision-making during a novel task that embeds concurrent reward and threat learning and manipulates congruency between reward and threat probabilities. Computational modeling supported a modified reinforcement learning model where participants integrated reward and threat value into a combined total value according to an individually varying policy parameter, which was highly predictive of decisions to approach reward vs avoid threat during trials where the highest reward option was also the highest threat option (i.e., approach-avoidance conflict). ICA analyses demonstrated unique roles for salience, frontoparietal, medial prefrontal, and inferior frontal networks in differential encoding of reward vs threat prediction error and value signals. The left frontoparietal network uniquely encoded degree of conflict between reward and threat value at the time of choice. MVPA demonstrated that delivery of reward and threat could accurately be decoded within salience and inferior frontal networks, respectively, and that decisions to approach reward vs avoid threat were predicted by the relative degree to which these reward vs threat representations were active at the time of choice. This latter result suggests that navigating AAC decisions involves generating mental representations for possible decision outcomes, and relative activation of these representations may bias subsequent decision-making towards approaching reward or avoiding threat accordingly.

Neural implementation of computational mechanisms underlying the continuous trade-off between cooperation and competition

Social interactions evolve continuously. Sometimes we cooperate, sometimes we compete, while at other times we strategically position ourselves somewhere in between to account for the ever-changing social contexts around us. Research on social interactions often focuses on a binary dichotomy between competition and cooperation, ignoring people's evolving shifts along a continuum. Here, we develop an economic game - the Space Dilemma - where two players change their degree of cooperativeness over time in cooperative and competitive contexts. Using computational modelling we show how social contexts bias choices and characterise how inferences about others' intentions modulate cooperativeness. Consistent with the modelling predictions, brain regions previously linked to social cognition, including the temporo-parietal junction, dorso-medial prefrontal cortex and the anterior cingulate gyrus, encode social prediction errors and context-dependent signals, correlating with shifts along a cooperation-competition continuum. These results provide a comprehensive account of the computational and neural mechanisms underlying the continuous trade-off between cooperation and competition.

Enhanced endogenous oxytocin signaling in the brain modulates neural responses to social misalignment and promotes conformity in humans: A multi-locus genetic profile approach

The neuropeptide oxytocin (OT) is known to promote social conformity. However, the specific neurocognitive mechanisms underlying OT-induced conformity remain unclear. We aimed to address this gap by examining how genetic variation in the oxytocin receptor gene (OXTR) is linked with behavioral conformity and its underlying neural systems. Specifically, we utilized the genotype-tissue expression database (GTEx) to create a novel multi-locus genetic profile score (MPS) that reflects the level of OXTR expression in the human brain. A total of 194 participants (Neuroimaging N = 50, Behavioral N = 144) performed a novel conformity task in which they viewed a series of word pairs depicting various moral values and virtues widely recognized in the United States. In each trial, participants indicated the relative importance of these words and subsequently learned about the majority opinion. Participants later rated the same word pairs a second time. Changes in participants' ratings between the first and second sessions were measured and analyzed with respect to social feedback, blood oxygen level-dependent (BOLD) functional magnetic resonance imaging (fMRI) signals, and OXTR MPS. We found that participants adjusted their ratings in accordance with the majority opinions. Social misalignment between self and others activated brain areas such as the striatum and the posterior medial frontal cortex (pMFC). However, unlike most findings from previous studies, activation in the pMFC during the inconsistent social feedback negatively, rather than positively, predicted behavioral conformity. Notably, those with higher OXTR MPS had reduced pMFC activation in the face of social misalignment, which led to greater conformity. Our findings suggest that OT may promote conformity by dampening the conflict-related signals in the pMFC. They also show that OXTR MPS may be useful for studying the effect of genes on highly complex human social traits, such as conformity.

Intracerebral mechanisms explaining the impact of incidental feedback on mood state and risky choice

Identifying factors whose fluctuations are associated with choice inconsistency is a major issue for rational decision theory. Here, we investigated the neuro-computational mechanisms through which mood fluctuations may bias human choice behavior. Intracerebral EEG data were collected in a large group of subjects (n=30) while they were performing interleaved quiz and choice tasks that were designed to examine how a series of unrelated feedbacks affect decisions between safe and risky options. Neural baseline activity preceding choice onset was confronted first to mood level, estimated by a computational model integrating the feedbacks received in the quiz task, and then to the weighting of option attributes, in a computational model predicting risk attitude in the choice task. Results showed that (1) elevated broadband gamma activity (BGA) in the ventromedial prefrontal cortex (vmPFC) and dorsal anterior insula (daIns) was respectively signaling periods of high and low mood, (2) increased vmPFC and daIns BGA respectively promoted and tempered risk taking by overweighting gain vs. loss prospects. Thus, incidental feedbacks induce brain states that correspond to different moods and bias the evaluation of risky options. More generally, these findings might explain why people experiencing positive (or negative) outcome in some part of their life tend to expect success (or failure) in any other.

The promise of layer-specific neuroimaging for testing predictive coding theories of psychosis

Predictive coding potentially provides an explanatory model for understanding the neurocognitive mechanisms of psychosis. It proposes that cognitive processes, such as perception and inference, are implemented by a hierarchical system, with the influence of each level being a function of the estimated precision of beliefs at that level. However, predictive coding models of psychosis are insufficiently constrained-any phenomenon can be explained in multiple ways by postulating different changes to precision at different levels of processing. One reason for the lack of constraint in these models is that the core processes are thought to be implemented by the function of specific cortical layers, and the technology to measure layer specific neural activity in humans has until recently been lacking. As a result, our ability to constrain the models with empirical data has been limited. In this review we provide a brief overview of predictive processing models of psychosis and then describe the potential for newly developed, layer specific neuroimaging techniques to test and thus constrain these models. We conclude by discussing the most promising avenues for this research as well as the technical and conceptual challenges which may limit its application.

Meta-analysis of human prediction error for incentives, perception, cognition, and action

Prediction errors (PEs) are a keystone for computational neuroscience. Their association with midbrain neural firing has been confirmed across species and has inspired the construction of artificial intelligence that can outperform humans. However, there is still much to learn. Here, we leverage the wealth of human PE data acquired in the functional neuroimaging setting in service of a deeper understanding, using an MKDA (multi-level kernel-based density) meta-analysis. Studies were identified with Google Scholar, and we included studies with healthy adult participants that reported activation coordinates corresponding to PEs published between 1999-2018. Across 264 PE studies that have focused on reward, punishment, action, cognition, and perception, consistent with domain-general theoretical models of prediction error we found midbrain PE signals during cognitive and reward learning tasks, and an insula PE signal for perceptual, social, cognitive, and reward prediction errors. There was evidence for domain-specific error signals--in the visual hierarchy during visual perception, and the dorsomedial prefrontal cortex during social inference. We assessed bias following prior neuroimaging meta-analyses and used family-wise error correction for multiple comparisons. This organization of computation by region will be invaluable in building and testing mechanistic models of cognitive function and dysfunction in machines, humans, and other animals. Limitations include small sample sizes and ROI masking in some included studies, which we addressed by weighting each study by sample size, and directly comparing whole brain vs. ROI-based results.

Learning under social versus nonsocial uncertainty: A meta-analytic approach

Much of the uncertainty that clouds our understanding of the world springs from the covert values and intentions held by other people. Thus, it is plausible that specialized mechanisms that compute learning signals under uncertainty of exclusively social origin operate in the brain. To test this hypothesis, we scoured academic databases for neuroimaging studies involving learning under uncertainty, and performed a meta‐analysis of brain activation maps that compared learning in the face of social versus nonsocial uncertainty. Although most of the brain activations associated with learning error signals were shared between social and nonsocial conditions, we found some evidence for functional segregation of error signals of exclusively social origin during learning in limited regions of ventrolateral prefrontal cortex and insula. This suggests that most behavioral adaptations to navigate social environments are reused from frontal and subcortical areas processing generic value representation and learning, but that a specialized circuitry might have evolved in prefrontal regions to deal with social context representation and strategic action.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

Exploring gene-culture coevolution in humans by inferring neuroendophenotypes: A case study of the oxytocin receptor gene and cultural tightness

The gene-culture coevolution (GCC) framework has gained increasing prominence in the social and biological sciences. While most studies on human GCC concern the evolution of low-level physiological traits, attempts have also been made to apply GCC to complex human traits, including social behavior and cognition. One major methodological challenge in this endeavor is to reconstruct a specific biological pathway between the implicated genes and their distal phenotypes. Here, we introduce a novel approach that combines data on population genetics and expression quantitative trait loci to infer the specific intermediate phenotypes of genes in the brain. We suggest that such "neuroendophenotypes" will provide more detailed mechanistic insights into the GCC process. We present a case study where we explored a GCC dynamics between the oxytocin receptor gene (OXTR) and cultural tightness-looseness. By combining data from the 1000 Genomes project and the Gene-Tissue-Expression project (GTEx), we estimated and compared OXTR expression in 10 brain regions across five human superpopulations. We found that OXTR expression in the anterior cingulate cortex (ACC) was highly variable across populations, and this variation correlated with cultural tightness and socio-ecological threats worldwide. The mediation models also suggested possible GCC dynamics where the increased OXTR expression in the ACC mediates or emerges from the tight culture and higher socio-ecological threats. Formal selection scans further confirmed that OXTR alleles linked to enhanced receptor expression in the ACC underwent positive selection in East Asian countries with tighter social norms. We discuss the implications of our method in human GCC research.

Linking Salience Signaling With Early Adversity and Affective Distress in Individuals at Clinical High Risk for Psychosis: Results From an Event-Related fMRI Study

Evidence suggests dysregulation of the salience network in individuals with psychosis, but few studies have examined the intersection of stress exposure and affective distress with prediction error (PE) signals among youth at clinical high-risk (CHR). Here, 26 individuals at CHR and 19 healthy volunteers (HVs) completed a monetary incentive delay task in conjunction with fMRI. We compared these groups on the amplitudes of neural responses to surprising outcomes-PEs without respect to their valence-across the whole brain and in two regions of interest, the anterior insula and amygdala. We then examined relations of these signals to the severity of depression, anxiety, and trauma histories in the CHR group. Relative to HV, youth at CHR presented with aberrant PE-evoked activation of the temporoparietal junction and weaker deactivation of the precentral gyrus, posterior insula, and associative striatum. No between-group differences were observed in the amygdala or anterior insula. Among youth at CHR, greater trauma histories were correlated with stronger PE-evoked amygdala activation. No associations were found between affective symptoms and the neural responses to PE. Our results suggest that unvalenced PE signals may provide unique information about the neurobiology of CHR syndromes and that early adversity exposure may contribute to neurobiological heterogeneity in this group. Longitudinal studies of young people with a range of risk syndromes are needed to further disentangle the contributions of distinct aspects of salience signaling to the development of psychopathology.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) entanglement of brain signals of different learning modules, and (iii) a limited number of computational models considered as candidates for explaining behavior. Here, we address these three limitations and (i) introduce a complex sequential decision making task with surprising events that allows us to (ii) dissociate correlates of reward prediction errors from those of surprise in functional magnetic resonance imaging (fMRI); and (iii) we test behavior against a large repertoire of model-free, model-based, and hybrid reinforcement learning algorithms, including a novel surprise-modulated actor-critic algorithm. Surprise, derived from an approximate Bayesian approach for learning the world-model, is extracted in our algorithm from a state prediction error. Surprise is then used to modulate the learning rate of a model-free actor, which itself learns via the reward prediction error from model-free value estimation by the critic. We find that action choices are well explained by pure model-free policy gradient, but reaction times and neural data are not. We identify signatures of both model-free and surprise-based learning signals in blood oxygen level dependent (BOLD) responses, supporting the existence of multiple parallel learning modules in the brain. Our results extend previous fMRI findings to a multi-step setting and emphasize the role of policy gradient and surprise signalling in human learning.

Prediction Errors for Aversive Events Shape Long-Term Memory Formation through a Distinct Neural Mechanism

Prediction errors (PEs) have been known for decades to guide associative learning, but their role in episodic memory formation has been discovered only recently. To identify the neural mechanisms underlying the impact of aversive PEs on long-term memory formation, we used functional magnetic resonance imaging, while participants saw a series of unique stimuli and estimated the probability that an aversive shock would follow. Our behavioral data showed that negative PEs (i.e., omission of an expected outcome) were associated with superior recognition of the predictive stimuli, whereas positive PEs (i.e., presentation of an unexpected outcome) impaired subsequent memory. While medial temporal lobe (MTL) activity during stimulus encoding was overall associated with enhanced memory, memory-enhancing effects of negative PEs were linked to even decreased MTL activation. Additional large-scale network analyses showed PE-related increases in crosstalk between the "salience network" and a frontoparietal network commonly implicated in memory formation for expectancy-congruent events. These effects could not be explained by mere changes in physiological arousal or the prediction itself. Our results suggest that the superior memory for events associated with negative aversive PEs is driven by a potentially distinct neural mechanism that might serve to set these memories apart from those with expected outcomes.

Neural correlates of the inverse base rate effect

The inverse base rate effect (IBRE) is a nonrational behavioral phenomenon in predictive learning. Canonically, participants learn that the AB stimulus compound leads to one outcome and that AC leads to another outcome, with AB being presented three times as often as AC. When subsequently presented with BC, the outcome associated with AC is preferentially selected, in opposition to the underlying base rates of the outcomes. The current leading explanation is based on error‐driven learning. A key component of this account is prediction error, a concept previously linked to a number of brain areas including the anterior cingulate, the striatum, and the dorsolateral prefrontal cortex. The present work is the first fMRI study to directly examine the IBRE. Activations were noted in brain areas linked to prediction error, including the caudate body, the anterior cingulate, the ventromedial prefrontal cortex, and the right dorsolateral prefrontal cortex. Analyzing the difference in activations for singular key stimuli (B and C), as well as frequency matched controls, supports the predictions made by the error‐driven learning account.

Acute stress blunts prediction error signals in the dorsal striatum during reinforcement learning

Acute stress is pervasive in everyday modern life and is thought to affect how people make choices and learn from them. Reinforcement learning, which implicates learning from the unexpected rewarding and punishing outcomes of our choices (i.e., prediction errors), is critical for adjusted behaviour and seems to be affected by acute stress. However, the neural mechanisms by which acute stress disrupts this type of learning are still poorly understood. Here, we investigate whether and how acute stress blunts neural signalling of prediction errors during reinforcement learning using model-based functional magnetic resonance imaging. Male participants completed a well-established reinforcement-learning task involving monetary gains and losses whilst under stress and control conditions. Acute stress impaired participants’ (n = 23) behavioural performance towards obtaining monetary gains (p < 0.001), but not towards avoiding losses (p = 0.57). Importantly, acute stress blunted signalling of prediction errors during gain and loss trials in the dorsal striatum (p = 0.040) — with subsidiary analyses suggesting that acute stress preferentially blunted signalling of positive prediction errors. Our results thus reveal a neurocomputational mechanism by which acute stress may impair reward learning.

Increased brain reactivity to gambling unavailability as a marker of problem gambling

The unprecedented development and ubiquity of sports betting constitute an emerging public health concern. It is crucial to provide markers that could help to better identify people experiencing sports betting-related harms. The current study investigated whether problem gambling status, sports betting passion, and trait-self-control modulate brain reactivity to sports betting cues. Sixty-five frequent sports bettors (35 "nonproblem bettors" and 30 "problem bettors") were exposed to cues representing real upcoming sport events (with varying levels of winning confidence) that were made available or blocked for betting, during functional magnetic resonance imaging (fMRI) recording. Sports betting passion and trait-self-control were assessed using self-report scales. Sport events nonavailable for betting elicited higher insular and striatal activation in problem bettors, as compared with nonproblem bettors. Within a large cluster encompassing the ventral striatum, hippocampus, and amygdala, lower trait-self-control was associated with increased brain reactivity to sport events with high levels of winning confidence that were nonavailable for betting. No significant effect of sports betting passion was observed. These findings suggest that sports bettors' brain reactivity to gambling unavailability might be a relevant marker of sports betting-related harms, as well as of blunted trait-self-control.

Reward-driven attention alters perceived salience

Many studies have revealed that reward-associated features capture attention. Neurophysiological evidence further suggests that this reward-driven attention effect modulates visual processes by enhancing low-level visual salience. However, no behavioral study to date has directly examined whether reward-driven attention changes how people see. Combining the two-phase paradigm with a psychophysical method, the current study found that compared with nonsalient cues associated with lower reward, the nonsalient cues associated with higher reward captured more attention, and increased the perceived contrast of the subsequent stimuli. This is the first direct behavioral evidence of the effect of reward-driven attention on low-level visual perception.

Meta-analytic clustering dissociates brain activity and behavior profiles across reward processing paradigms

Reward learning is a ubiquitous cognitive mechanism guiding adaptive choices and behaviors, and when impaired, can lead to considerable mental health consequences. Reward-related functional neuroimaging studies have begun to implicate networks of brain regions essential for processing various peripheral influences (e.g., risk, subjective preference, delay, social context) involved in the multifaceted reward processing construct. To provide a more complete neurocognitive perspective on reward processing that synthesizes findings across the literature while also appreciating these peripheral influences, we used emerging meta-analytic techniques to elucidate brain regions, and in turn networks, consistently engaged in distinct aspects of reward processing. Using a data-driven, meta-analytic, k-means clustering approach, we dissociated seven meta-analytic groupings (MAGs) of neuroimaging results (i.e., brain activity maps) from 749 experimental contrasts across 176 reward processing studies involving 13,358 healthy participants. We then performed an exploratory functional decoding approach to gain insight into the putative functions associated with each MAG. We identified a seven-MAG clustering solution that represented dissociable patterns of convergent brain activity across reward processing tasks. Additionally, our functional decoding analyses revealed that each of these MAGs mapped onto discrete behavior profiles that suggested specialized roles in predicting value (MAG-1 & MAG-2) and processing a variety of emotional (MAG-3), external (MAG-4 & MAG-5), and internal (MAG-6 & MAG-7) influences across reward processing paradigms. These findings support and extend aspects of well-accepted reward learning theories and highlight large-scale brain network activity associated with distinct aspects of reward processing.