Neural evidence for description dependent reward processing in the framing effect

Valence precedes value in neural encoding of prediction error

Event-related potentials that follow feedback in reinforcement learning tasks have been proposed to reflect neural encoding of prediction errors. Prior research has shown that in the interval of 240-340 ms multiple different prediction error encodings appear to co-occur, including a value signal carrying signed quantitative prediction error and a valence signal merely carrying sign. The effects used to identify these two encoders, respectively a sign main effect and a sign × size interaction, do not reliably discriminate them. A full discrimination is made possible by comparing tasks in which the reinforcer available on a given trial is set to be either appetitive or aversive against tasks where a trial allows the possibility of either. This study presents a meta-analysis of reinforcement learning experiments, the majority of which presented the possibility of winning or losing money. Value and valence encodings were identified by conventional difference wave methodology but additionally by an analysis of their predicted behavior using a Bayesian analysis that incorporated nulls into the evidence for each encoder. The results suggest that a valence encoding, sensitive only to the available outcomes on the trial at hand precedes a later value encoding sensitive to the outcomes available in the wider experimental context. The implications of this for modeling computational processes of reinforcement learning in humans are discussed.

Examining the Neurobiology of Non-Suicidal Self-Injury in Children and Adolescents: The Role of Reward Responsivity

Although prior work has shown heightened response to negative outcomes and reduced response to positive outcomes in youth with a history of non-suicidal self-injury (NSSI), little is known about the neural processes underlying these responses. Thus, this study examined associations between NSSI engagement and functional activation in specific regions of interest (ROIs) and whole-brain connectivity between striatal, frontal, and limbic region seeds during monetary and social reward tasks. To test for specificity of the influence of NSSI, analyses were conducted with and without depressive symptoms as a covariate. We found that NSSI was associated with decreased activation following monetary gains in all ROIs, even after controlling for depressive symptoms. Exploratory connectivity analyses found that NSSI was associated with differential connectivity between regions including the DS, vmPFC, insula, and parietal operculum cortex when controlling for depressive symptoms. Disrupted connectivity between these regions could suggest altered inhibitory control of emotions and pain processing in individuals with NSSI. Findings suggest dysfunctional reward processes in youth with NSSI, even very early in the course of the behavior.

Self-reported daily sleep quality modulates the impact of the framing effect on outcome evaluation in decision-making under uncertainty: An ERP study

A growing body of research has largely confirmed and supported the idea that experimental sleep loss, such as sleep deprivation or sleep restriction, could affect individuals' risk-taking behavior and brain activity. However, whether self-reported sleep quality resulting from daily life modulates how feedback is evaluated during decision-making is still unclear. In this study, we used event-related potentials (ERPs) with a Balloon Analogue Risk Task (BART) to investigate how self-reported daily sleep quality modulates the brain's response to feedback from decision-making in the gain and loss frames. Behavioral data showed an increased aversion to uncertainty in the gain frame relative to the loss frame for individuals with higher sleep quality. However, this was not true for individuals with lower voluntary sleep quality. Similarly, the ERP data demonstrated that individuals with lower self-reported daily sleep quality displayed no changes in feedback-related negativity (FRN) in response to outcomes from decision-making in the gain and loss frames; however, individuals with higher self-reported daily sleep quality showed a greater FRN in response to decision-making in the gain frame than that in the loss frame. A Pearson correlation analysis showed that self-reported daily sleep quality was positively related to the variance of the FRN amplitude in response to the gain and loss frames. These findings suggest that framing effects on decision-making under uncertainty may depend on self-reported daily sleep quality and that the effects disappear when the sleep quality declines.

The Event-Related Potentials Responding to Outcome Valence and Expectancy Violation during Feedback Processing

Feedback-related negativity (FRN) is believed to encode reward prediction error (RPE), a term describing whether the outcome is better or worse than expected. However, some studies suggest that it may reflect unsigned prediction error (UPE) instead. Some disagreement remains as to whether FRN is sensitive to the interaction of outcome valence and prediction error (PE) or merely responsive to the absolute size of PE. Moreover, few studies have compared FRN in appetitive and aversive domains to clarify the valence effect or examine PE's quantitative modulation. To investigate the impact of valence and parametrical PE on FRN, we varied the prediction and feedback magnitudes within a probabilistic learning task in valence (gain and loss domains, Experiment 1) and non-valence contexts (pure digits, Experiment 2). Experiment 3 was identical to Experiment 1 except that some blocks emphasized outcome valence, while others highlighted predictive accuracy. Experiments 1 and 2 revealed a UPE encoder; Experiment 3 found an RPE encoder when valence was emphasized and a UPE encoder when predictive accuracy was highlighted. In this investigation, we demonstrate that FRN is sensitive to outcome valence and expectancy violation, exhibiting a preferential response depending on the dimension that is emphasized.

Altered Neural Processing of Reward and Punishment in Women With Methamphetamine Use Disorder

It has been suggested that the altered function of reward and punishment is an important vulnerability factor leading to the development of drug use disorders. Previous studies have identified evidence of neurophysiological dysfunction in the reward process of individuals with substance use disorders. To date, only a few event-related potential (ERP) studies have examined the neural basis of reward and punishment processing in women with methamphetamine (MA) use disorders. The current ERP research aims to investigate the neurophysiological mechanisms of reward and punishment in women with MA use disorder using a monetary incentive delay task. Nineteen women with MA use disorder (MA group) and 20 healthy controls (HC group) were recruited in this study. The behavioral data showed that the reaction time (RT) was faster and the response accuracy (ACC) was higher for the potential reward and punishment conditions compared to neutral conditions. During the monetary incentive anticipation stage, the Cue-P3, and stimulus-preceding negativity (SPN) were larger in the MA group than in the HC group. The SPN under the potential reward condition was larger than that under the neutral condition in the MA group but not in the HC group. During the monetary incentive consummation stage, the feedback-related negativity and feedback P3 (FB-P3) following positive feedback were significantly larger than negative feedback in the potential reward condition for the HC group, but not for the MA group. However, the FB-P3 following negative feedback was significantly larger than positive feedback in the potential punishment condition for the MA group, but not the HC group. The results suggest that women with MUD have stronger expectations of generic reward and stronger response of generic harm avoidance, which could be targeted in designing interventions for women with MA use disorder.

Comparing the motivational value of rewards and losses in an EEG-pupillometry study

We found earlier that performance-contingent rewards lead to faster performance than equivalent losses [Carsten, Hoofs, Boehler, & Krebs, 2019. Motivation Science, 5(3). http://dx.doi.org/10.1037/mot0000117]. Here, we further tested the hypothesis that motivation to gain rewards is higher than to avoid losses, even when incentive values are matched. As implicit markers of motivation, we assessed electroencephalography (EEG) focusing on the P3 after target and feedback onset, and the Feedback-Related Negativity (FRN), as well as simultaneously recorded pupil size. Comparing only reward and loss prospect trials in Experiment 1, we found no consistent differences in behavior and electrophysiological markers of motivation, although pupil data suggested higher arousal after feedback in potential-loss trials. Including additional no-incentive trials in Experiment 2, we found consistent evidence that motivation to gain rewards was higher than to avoid losses: In line with behavior, the target-P3 was most pronounced for reward-related stimuli, followed by loss and no-incentive ones. This same ranking was found in the P3 and the FRN after positive outcomes (i.e., reward, avoided loss, and correct feedback in no-incentive trials). Negative outcomes featured a different pattern in line with the pupil response, which suggests that losses are emotionally salient events, without invigorating behavior proportionally. In sum, these findings suggest that the motivation to gain rewards is more pronounced than motivation to avoid equivalent losses, at least in tasks promoting transient increases in attention triggered by incentive prospect. These motivational differences may arise as avoided losses are not profitable in the long term, in contrast to gained rewards.

Upward and downward comparisons across monetary and status domains

The ability to accurately infer one's place with respect to others is crucial for social interactions. Individuals tend to evaluate their own actions and outcomes by comparing themselves to others in either an upward or downward direction. We performed two fMRI meta‐analyses on monetary (n = 39; 1,231 participants) and status (n = 23; 572 participants) social comparisons to examine how domain and the direction of comparison can modulate neural correlates of social hierarchy. Overall, both status and monetary downward comparisons activated regions associated with reward processing (striatum) while upward comparisons yielded loss‐related activity. These findings provide partial support for the common currency hypothesis in that downward and upward comparisons from both monetary and status domains resemble gains and losses, respectively. Furthermore, status upward and monetary downward comparisons revealed concordant orbitofrontal cortical activity, an area associated with evaluating the value of goals and decisions implicated in both lesion and empirical fMRI studies investigating social hierarchy. These findings may offer new insight into how people relate to individuals with higher social status and how these social comparisons deviate across monetary and social status domains.

Feedback-related negativity reflects omission of monetary gains: Evidence from ERP gambling study

Feedback processing is an important aspect of learning. In the human brain, feedback processing is often examined by measuring an event-related potential, the feedback-related negativity component. Typically, the feedback-related negativity component is investigated by directly comparing gain with loss feedback randomized across trials; however, this method does not control for confounds associated with unexpected feedback. For this study we used a blocked design gambling task to investigate the sensitivity of feedback-related negativity to positive and negative feedback separately for gains and losses. While there appeared to be no significant feedback-related negativity in the loss domain, results revealed an enlarged feedback-related negativity during the omission of gains compared to the reception of gains. These findings further support the reward positivity hypothesis which declares that the feedback-related negativity is associated with the processing of outcomes in the context of gains as opposed to losses, irrespective of unexpectedness.

The Effect of Apolipoprotein E ε4 (APOE ε4) on Visuospatial Working Memory in Healthy Elderly and Amnestic Mild Cognitive Impairment Patients: An Event-Related Potentials Study

Background: Apolipoprotein E (APOE) ε4 is the only established risk gene for late-onset, sporadic Alzheimer’s disease (AD). Previous studies have provided inconsistent evidence for the effect of APOE ε4 status on the visuospatial working memory (VSWM).

Objective: The aim was to investigate the effect of APOE ε4 on VSWM with an event-related potential (ERP) study in healthy controls (HC) and amnestic mild cognitive impairment (aMCI) patients.

Methods: The study recorded 39 aMCI patients (27 APOE ε4 non-carriers and 12 APOE ε4 carriers) and their 43 matched controls (25 APOE ε4 non-carriers and 18 APOE ε4 carriers) with an 64-channel electroencephalogram. Participants performed an N-back task, a VSWM paradigm that manipulated the number of items to be stored in memory.

Results: The present study detected reduced accuracy and delayed mean correct response time (RT) in aMCI patients compared to HC. P300, a positive component that peaks between 300 and 500 ms, was elicited by the VSWM task. In addition, aMCI patients showed decreased P300 amplitude at the central–parietal (CP1, CPz, and CP2) and parietal (P1, Pz, and P2) electrodes in 0- and 1-back task compared to HC. In both HC and aMCI patients, APOE ε4 carriers showed reduced P300 amplitude with respect to non-carriers, whereas no significant differences in accuracy or RT were detected between APOE ε4 carriers and non-carriers. Additionally, standardized low-resolution brain electromagnetic tomography analysis (s-LORETA) showed enhanced brain activation in the right parahippocampal gyrus (PHG) during P300 time range in APOE ε4 carriers with respect to non-carriers in aMCI patients.

Conclusion: It demonstrated that P300 amplitude could predict VSWM deficits in aMCI patients and contribute to early detection of VSWM deficits in APOE ε4 carriers.

Effects of loss aversion on neural responses to loss outcomes: An event-related potential study

Loss aversion is the tendency to prefer avoiding losses over acquiring gains of the same amount. To shed light on the spatio-temporal processes underlying loss aversion, we analysed the associations between individual loss aversion and electrophysiological responses to loss and gain outcomes in a monetary gamble task. Electroencephalographic feedback-related negativity (FRN) was computed in 29 healthy participants as the difference in electrical potentials between losses and gains. Loss aversion was evaluated using non-linear parametric fitting of choices in a separate gamble task. Loss aversion correlated positively with FRN amplitude (233-263ms) at electrodes covering the lower face. Feedback related potentials were modelled by five equivalent source dipoles. From these dipoles, stronger activity in a source located in the orbitofrontal cortex was associated with loss aversion. The results suggest that loss aversion implemented during risky decision making is related to a valuation process in the orbitofrontal cortex, which manifests during learning choice outcomes.

The Motivational Hierarchy between the Personal Self and Close Others in the Chinese Brain: an ERP Study

People base their decisions not only on their own self-interest but also on the interests of close others. Generally, the personal self has primacy in the motivational hierarchy in the Western culture. A recent study found that friends have the same motivational hierarchy as the personal self in the Eastern collectivist culture. Remaining unknown is whether the motivational hierarchy of the personal self and close others can be manifested in the collectivist brain. In the present study, we asked participants to gamble for the personal self, close others (i.e., mother, father, and close friend), and strangers. The positive-going deflection of event-related potentials (ERPs) in response to positive feedback showed the following pattern: personal self = mother = father > friend > stranger. In the loss condition, no significant beneficiary effect was observed. The present results indicate that the personal self and parents are intertwined in the motivational system in the Chinese undergraduate student brain, supporting the view that the personal self and parents have the same motivational primacy at the electrocortical level.

All framing effects are not created equal: Low convergent validity between two classic measurements of framing

Human risk-taking attitudes can be influenced by two logically equivalent but descriptively different frames, termed the framing effect. The classic hypothetical vignette-based task (Asian disease problem) and a recently developed reward-based gambling task have been widely used to assess individual differences in the framing effect. Previous studies treat framing bias as a stable trait that has genetic basis. However, these two paradigms differ in terms of task domain (loss vs. gain) and task context (vignette-based vs. reward-based) and the convergent validity of these measurements remains unknown. Here, we developed a vignette-based task and a gambling task in both gain and loss domains and tested correlations of the framing effect among these tasks in 159 young adults. Our results revealed no significant correlation between the vignette-based task in the loss domain and the gambling task in the gain domain, indicating low convergent validity. The current findings raise the question of how to measure the framing effect precisely, especially in individual difference studies using large samples and expensive neuroscience methods. Our results suggest that the framing effect is influenced by both task domain and task context and future research should be cautious about the operationalization of the framing effect.

Motivational Hierarchy in the Chinese Brain: Primacy of the Individual Self, Relational Self, or Collective Self?

According to the three-tier hierarchy of motivational potency in the self system, the self can be divided into individual self, relational self, and collective self, and individual self is at the top of the motivational hierarchy in Western culture. However, the motivational primacy of the individual self is challenged in Chinese culture, which raises the question about whether the three-tier hierarchy of motivational potency in the self system can be differentiated in the collectivist brain. The present study recorded the event-related potentials (ERPs) to evaluate brain responses when participants gambled for individual self, for a close friend (relational self), or for the class (collective self). The ERP results showed that when outcome feedback was positive, gambling for individual self evoked a larger reward positivity compared with gambling for a friend or for the class, while there is no difference between the latter two conditions. In contrast, when outcome feedback was negative, no significant effect was found between conditions. The present findings provide direct electrophysiological evidence that individual self is at the top of the three-tier hierarchy of the motivational system in the collectivist brain, which supports the classical pancultural view that individual self has motivational primacy.

Randomised prior feedback modulates neural signals of outcome monitoring

Substantial evidence indicates that decision outcomes are typically evaluated relative to expectations learned from relatively long sequences of previous outcomes. This mechanism is thought to play a key role in general learning and adaptation processes but relatively little is known about the determinants of outcome evaluation when the capacity to learn from series of prior events is difficult or impossible. To investigate this issue, we examined how the feedback-related negativity (FRN) is modulated by information briefly presented before outcome evaluation. The FRN is a brain potential time-locked to the delivery of decision feedback and it is widely thought to be sensitive to prior expectations. We conducted a multi-trial gambling task in which outcomes at each trial were fully randomised to minimise the capacity to learn from long sequences of prior outcomes. Event-related potentials for outcomes (Win/Loss) in the current trial (Outcome_t) were separated according to the type of outcomes that occurred in the preceding two trials (Outcome_t-1 and Outcome_t-2). We found that FRN voltage was more positive during the processing of win feedback when it was preceded by wins at Outcome_t-1 compared to win feedback preceded by losses at Outcome_t-1. However, no influence of preceding outcomes was found on FRN activity relative to the processing of loss feedback. We also found no effects of Outcome_t-2 on FRN amplitude relative to current feedback. Additional analyses indicated that this effect was largest for trials in which participants selected a decision different to the gamble chosen in the previous trial. These findings are inconsistent with models that solely relate the FRN to prediction error computation. Instead, our results suggest that if stable predictions about future events are weak or non-existent, then outcome processing can be determined by affective systems. More specifically, our results indicate that the FRN is likely to reflect the activity of positive affective systems in these contexts. Importantly, our findings indicate that a multifactorial explanation of the nature of the FRN is necessary and such an account must incorporate affective and motivational factors in outcome processing.

•

FRN is sensitive to prior events but does this persist when learning is impossible?

•

Four FRN theories were examined in a task with fully randomised feedback.

•

Previous trial outcome modulated FRN during positive but not negative outcomes.

•

Results consistent with a positive affective modulation interpretation of the FRN.

•

FRN is determined by factors beyond prediction error (e.g. affective/motivational).

Social distance and anonymity modulate fairness consideration: An ERP study

Previous research indicated that fairness consideration can be influenced by social distance. However, it is not clear whether social distance and anonymity have an interactive impact on fairness evaluation during asset distribution and whether these processes can be documented in brain activity. Using a modified ultimatum game combined with measures of event related potential (ERP), we examined how social distance and anonymity modulate brain response to inequality. At the behavior level, we found that acceptance rate and reaction time can be substantially modified by social distance and anonymity. Feedback-related negativity, an ERP component associated with conflict between cognitive and emotion motives, was more negative in response to unfairness than fairness from strangers; however, it showed an opposite trend for unfair offers provided by friends, suggesting that the influence of social distance on fairness perception is relatively fast. The P300 in response to fair offers was more positive when the proposers made offers when uncertain about partner identity than when certain about partner identity. These results suggest that unfairness is evaluated in a fast conflict detection stage and a slower stage that integrates more complex social contextual factors such as anonymity.