Individual differences in reward-prediction-error: extraversion and feedback-related negativity

Feedback-related negativity observed in rodent anterior cingulate cortex

The feedback-related negativity (FRN) refers to a difference in the human event-related potential (ERP) elicited by feedback indicating success versus failure: the difference appears negative when subtracting the success ERP from the failure ERP (Miltner et al., 1997). Although source localization techniques (e.g., BESA) suggest that the FRN is produced in the ACC, the inverse problem (that any given scalp distribution can be produced by an infinite number of possible dipole configurations) limits the certainty of this conclusion. The inverse problem can be circumvented by directly recording from the ACC in animal models. Although a non-human primate homologue of the FRN has been observed in the macaque monkey (e.g. Emeric et al., 2008), a homologue of the FRN has yet to be identified in rodents. We recorded local field potentials (LFPs) directly from the ACC in 6 rodents in a task based on the FRN paradigm. The animals were trained to poke their nose into a lighted port and received a feedback smell indicating whether or not a reward pellet would drop 1.5s later. We observed a FRN-like effect time-locked to the feedback scent whereby the LFP to feedback predicting no-reward was significantly more negative than the LFP to feedback predicting reward. This deflection began on average 130ms before behavioral changes in response to the feedback. Thus, we provide the first evidence of the existence of a rodent homologue of the FRN.

Individual differences in reward prediction error: contrasting relations between feedback-related negativity and trait measures of reward sensitivity, impulsivity and extraversion

Medial-frontal negativity occurring ∼200–300 ms post-stimulus in response to motivationally salient stimuli, usually referred to as feedback-related negativity (FRN), appears to be at least partly modulated by dopaminergic-based reward prediction error (RPE) signaling. Previous research (e.g., Smillie et al., 2011) has shown that higher scores on a putatively dopaminergic-based personality trait, extraversion, were associated with a more pronounced difference wave contrasting unpredicted non-reward and unpredicted reward trials on an associative learning task. In the current study, we sought to extend this research by comparing how trait measures of reward sensitivity, impulsivity and extraversion related to the FRN using the same associative learning task. A sample of healthy adults (N = 38) completed a battery of personality questionnaires, before completing the associative learning task while EEG was recorded. As expected, FRN was most negative following unpredicted non-reward. A difference wave contrasting unpredicted non-reward and unpredicted reward trials was calculated. Extraversion, but not measures of impulsivity, had a significant association with this difference wave. Further, the difference wave was significantly related to a measure of anticipatory pleasure, but not consummatory pleasure. These findings provide support for the existing evidence suggesting that variation in dopaminergic functioning in brain “reward” pathways may partially underpin associations between the FRN and trait measures of extraversion and anticipatory pleasure.

Gene Effects and G × E Interactions in the Differential Prediction of Three Aspects of Impulsiveness

Several polymorphisms relevant to dopamine and serotonin have been identified as potential contributors to individual differences in impulsivity versus self-control. Because impulsivity is a multifaceted construct, a need remains to examine more closely how various genes relate to different aspects of impulsivity. We examined four dopamine-related polymorphisms and the serotonin transporter as predictors of three aspects of impulsivity, two bearing on impulsive reactions to emotions and one on difficulty in completing intended actions. Early adversity was also examined as a potentiator of genetic effects. Undergraduates completed measures of impulsivity and early adversity and were genotyped. COMT, BDNF, DRD4, and 5HTTLPR (the latter two in interaction with early adversity) made independent contributions to prediction of Pervasive Influence of Feelings. BDNF made a contribution to Lack of Follow-Through. ANKK1 and 5HTTLPR (both in interaction with early adversity) made independent contributions to Feelings Trigger Action. Thus, five polymorphisms contributed to predicting impulsivity, but different polymorphisms related to different aspects.

Extraversion differentiates between model-based and model-free strategies in a reinforcement learning task

Prominent computational models describe a neural mechanism for learning from reward prediction errors, and it has been suggested that variations in this mechanism are reflected in personality factors such as trait extraversion. However, although trait extraversion has been linked to improved reward learning, it is not yet known whether this relationship is selective for the particular computational strategy associated with error-driven learning, known as model-free reinforcement learning, vs. another strategy, model-based learning, which the brain is also known to employ. In the present study we test this relationship by examining whether humans' scores on an extraversion scale predict individual differences in the balance between model-based and model-free learning strategies in a sequentially structured decision task designed to distinguish between them. In previous studies with this task, participants have shown a combination of both types of learning, but with substantial individual variation in the balance between them. In the current study, extraversion predicted worse behavior across both sorts of learning. However, the hypothesis that extraverts would be selectively better at model-free reinforcement learning held up among a subset of the more engaged participants, and overall, higher task engagement was associated with a more selective pattern by which extraversion predicted better model-free learning. The findings indicate a relationship between a broad personality orientation and detailed computational learning mechanisms. Results like those in the present study suggest an intriguing and rich relationship between core neuro-computational mechanisms and broader life orientations and outcomes.

Extraversion and Reward Processing

Reward processes have played an increasingly visible role in theories of extraverted personality. Reward processing is usually conceptualized in terms of the brain system responsible for generating incentive motivation and behavioral approach of rewarding stimuli, as in theories by Jeffrey Gray and Richard Depue. Recent increases in the accessibility of neuroscience methods have accelerated our understanding of the relationship between extraversion and neural processing of rewards. An issue that has remained somewhat neglected by this literature concerns the distinctions that have been made between reward desire and reward enjoyment. Higher-level abstractions of reward processing—identifiable in cognitive, social, and narrative approaches—have also received relatively little attention. These promising directions for future research may help further expand knowledge in this area of personality science.

An electrophysiological index of changes in risk decision-making strategies

Human decision-making is significantly modulated by previously experienced outcomes. Using event-related potentials (ERPs), we examined whether ERP components evoked by outcome feedbacks could serve as biomarkers to signal the influence of current outcome evaluation on subsequent decision-making. In this study, 18 adult volunteers participated in a simple monetary gambling task, in which they were asked to choose between two options that differed in risk. Their decisions were immediately followed by outcome presentation. Temporospatial principle component analysis (PCA) was applied to the outcome-onset locked ERPs in the 200-1000 ms time window. The PCA factors that approximated classical ERP components (P2, feedback-related negativity, P3a, and P3b) in terms of time course and scalp distribution were tested for their association with subsequent decision-making strategies. Our results revealed that a fronto-central PCA factor approximating the classical P3a was related to changes of decision-making strategies on subsequent trials. The decision to switch between high- and low-risk options resulted in a larger P3a relative to the decision to retain the same choice. According to the results, we suggest that the amplitude of the fronto-central P3a is an electrophysiological index of the influence of current outcome on subsequent risk decision-making. Furthermore, the ERP source analysis indicated that the activations of the frontopolar cortex and sensorimotor cortex were involved in subsequent changes of strategies, which enriches our understanding of the neural mechanisms of adjusting decision-making strategies based on previous experience.

A neural signature of the creation of social evaluation

Previous research has shown that receiving an unfair monetary offer in economic bargaining elicits also-called feedback negativity (FN). This scalp-recorded brain potential probably reflects a bad-vs-good evaluation in the medial frontal cortex and has been linked to fundamental processes of reinforcement learning. In the present study, we investigated whether the evaluative mechanism indexed by the FN is also involved in learning who is an unfair vs fair bargaining partner. An electroencephalogram was recorded while participants completed a computerized version of the Ultimatum Game, repeatedly receiving fair or unfair monetary offers from alleged other participants. Some of these proposers were either always fair or always unfair in their offers. In each trial, participants first saw a portrait picture of the respective proposer before the monetary offer was presented. Therefore, the faces could be used as predictive cues for the fairness of the pending offers. We found that not only unfair offers themselves induced a FN, but also (over the task) faces of unfair proposers. Thus, when interaction partners repeatedly behave in an unfair way, their faces acquire a negative valence, which manifests in a basal neural mechanism of bad-vs-good evaluation.

Event-related potential studies of outcome processing and feedback-guided learning

In order to control behavior in an adaptive manner the brain has to learn how some situations and actions predict positive or negative outcomes. During the last decade cognitive neuroscientists have shown that the brain is able to evaluate and learn from outcomes within a few hundred milliseconds of their occurrence. This research has been primarily focused on the feedback-related negativity (FRN) and the P3, two event-related potential (ERP) components that are elicited by outcomes. The FRN is a frontally distributed negative-polarity ERP component that typically reaches its maximal amplitude 250 ms after outcome presentation and tends to be larger for negative than for positive outcomes. The FRN has been associated with activity in the anterior cingulate cortex (ACC). The P3 (~300–600 ms) is a parietally distributed positive-polarity ERP component that tends to be larger for large magnitude than for small magnitude outcomes. The neural sources of the P3 are probably distributed over different regions of the cortex. This paper examines the theories that have been proposed to explain the functional role of these two ERP components during outcome processing. Special attention is paid to extant literature addressing how these ERP components are modulated by outcome valence (negative vs. positive), outcome magnitude (large vs. small), outcome probability (unlikely vs. likely), and behavioral adjustment. The literature offers few generalizable conclusions, but is beset with a number of inconsistencies across studies. This paper discusses the potential reasons for these inconsistencies and points out some challenges that probably will shape the field over the next decade.

Attitude toward money modulates outcome processing: an ERP study

Love of money (LOM) is concerned with the attitude toward money, which can be measured by the LOM scale through affective, behavioral, and cognitive dimensions. Research has observed that monetary attitude was tightly related to reward processing and could affect economic behavior. This study examined how monetary attitude modulated risky behavior and the underlying neural mechanisms of reward processing using event-related potential (ERP) technique. We compared both the risk level and brain responses of a high-level LOM (HLOM) group to a low-level LOM (LLOM) group using a simple gambling task. The behavioral results showed that the HLOM group was more risky than the LLOM group, particularly after loss. The feedback-related negativity (FRN) was measured as the difference wave (gain-related ERP was subtracted from loss-related ERP). The FRN difference wave was larger in the HLOM group than that in the LLOM group. The P3 in the HLOM group was more positive than that in the LLOM group. These results suggest that monetary attitude can modulate both the underlying neural mechanisms and behavioral performance in a reward-related task. The HLOM participants are more sensitive to gain/loss than the LLOM participants.

Performance monitoring and the medial prefrontal cortex: a review of individual differences and context effects as a window on self-regulation

The medial prefrontal cortex (MPFC) is central to self-regulation and has been implicated in generating a cluster of event-related potential components, collectively referred to as medial frontal negativities (MFNs). These MFNs are elicited while individuals monitor behavioral and environmental consequences, and include the error-related negativity, Nogo N2, and the feedback-related negativity. A growing cognitive and affective neuroscience literature indicates that the activation of the anterior cingulate cortex (ACC) and surrounding medial prefrontal regions during performance monitoring is not only influenced by task context, but that these patterns of activity also vary as a function of individual differences (e.g., personality, temperament, clinical and non-clinical symptomatology, socio-political orientation, and genetic polymorphisms), as well as interactions between individual differences and task context. In this review we survey the neuroscience literature on the relations between performance monitoring, personality, task context, and brain functioning with a focus on the MPFC. We relate these issues to the role of affect in the paradigms used to elicit performance-monitoring neural responses and highlight some of the theoretical and clinical implications of this research. We conclude with a discussion of the complexity of these issues and how some of the basic assumptions required for their interpretation may be clarified with future research.

Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice

To behave adaptively, we must learn from the consequences of our actions. Studies using event-related potentials (ERPs) have been informative with respect to the question of how such learning occurs. These studies have revealed a frontocentral negativity termed the feedback-related negativity (FRN) that appears after negative feedback. According to one prominent theory, the FRN tracks the difference between the values of actual and expected outcomes, or reward prediction errors. As such, the FRN provides a tool for studying reward valuation and decision making. We begin this review by examining the neural significance of the FRN. We then examine its functional significance. To understand the cognitive processes that occur when the FRN is generated, we explore variables that influence its appearance and amplitude. Specifically, we evaluate four hypotheses: (1) the FRN encodes a quantitative reward prediction error; (2) the FRN is evoked by outcomes and by stimuli that predict outcomes; (3) the FRN and behavior change with experience; and (4) the system that produces the FRN is maximally engaged by volitional actions.