Modulation of feedback-related negativity during trial-and-error exploration and encoding of behavioral shifts

Give it a second try? The influence of feedback and performance in the decision of reattempting

Feedback evaluation can affect behavioural continuation or discontinuation, and is essential for cognitive and motor skill learning. One critical factor that influences feedback evaluation is participants' internal estimation of self-performance. Previous research has shown that two event-related potential components, the Feedback-Related Negativity (FRN) and the P3, are related to feedback evaluation. In the present study, we used a time estimation task and EEG recordings to test the influence of feedback and performance on participants' decisions, and the sensitivity of the FRN and P3 components to those factors. In the experiment, participants were asked to reproduce the total duration of an intermittently presented visual stimulus. Feedback was given after every response, and participants had then to decide whether to retry the same trial and try to earn reward points, or to move on to the next trial. Results showed that both performance and feedback influenced participants' decision on whether to retry the ongoing trial. In line with previous studies, the FRN showed larger amplitude in response to negative than to positive feedback. Moreover, our results were also in agreement with previous works showing the relationship between the amplitude of the FRN and the size of feedback-related prediction error (PE), and provide further insight in how PE size influences participants' decisions on whether or not to retry a task. Specifically, we found that the larger the FRN, the more likely participants were to base their decision on their performance - choosing to retry the current trial after good performance or to move on to the next trial after poor performance, regardless of the feedback received. Conversely, the smaller the FRN, the more likely participants were to base their decision on the feedback received.

The detection of self-group conflicts in exercise behaviors differs with social network centrality: ERP evidence

BACKGROUND

The influence of social norms on exercise behaviors has been explored in studies over the years. However, little is known about whether an individual's role (central or peripheral) in his or her social network, which is associated with social skills, could shift his or her susceptibility to normative effects on exercise behaviors. To that end, event-related potentials (ERPs) were recorded to examine the underlying cognitive mechanism of the effects of network centrality on normative social influence.

METHODS

We manipulated network centrality by assigning participants to exercise support groups, with group members who were their nominated friends (high centrality) or nonnominated classmates (low centrality). Participants were asked to evaluate their willingness to engage in various exercises, after viewing discrepant group ratings (peer influence) or not viewing (no-influence).

RESULTS

Peer influence evoked a larger negative-going feedback-related negativity (FRN) wave, which was linked to automatic social conflict detection, and a larger positive-going P3 wave, which was linked to subsequent conformity behavioral changes. However, effects on the FRN, not the P3, were observed only in the high-centrality group.

CONCLUSION

Our results highlight the important roles of network centrality in encoding self-group exercise attitude discrepancy rather than in decision-making regarding exercise attitude adjustments. Interventions aimed at promoting exercise behaviors should be considered in a broader social environmental framework.

Neural correlates of evaluations of non-binary social feedback: An EEG study

PURPOSE

In complex and diverse social circumstances, decision making is affected by social feedback. Although previous studies have examined the electrophysiological correlates of social feedback with a binary valence, those related to non-binary feedback, or the magnitude of social feedback, remain unclear. This study investigated the electrophysiological correlates of non-binary social feedback and subsequent action selection processing.

METHODS

Participants were asked to complete a Gabor patch direction judgment task in which they were required to make judgments before and after receiving social feedback. They were informed that the feedback stimuli represented the degree to which other participants made the same choice.

RESULTS & CONCLUSION

The results revealed that feedback that was highly concordant with the participant's judgments elicited greater P300 activity, which was associated with the fulfillment of expectations regarding social reward. Moreover, moderately concordant feedback induced stronger theta band power, which may indicate monitoring of subjective conflict. Temporal changes in theta power during feedback phase may also relate to adjustments in prediction error. Additionally, when an initial judgment was maintained following social feedback, we observed a stronger increase in beta power, indicating an association with post-social-feedback action processing.

Processing of performance errors predicts memory formation: Enhanced feedback-related negativities for corrected versus repeated errors in an associative learning paradigm

Learning from errors or negative feedback is crucial for adaptive behavior. FMRI studies have demonstrated enhanced anterior cingulate cortex activity for errors that were later corrected versus repeated errors even when a substantial delay between the error and the opportunity to correct was introduced. We aimed at identifying the electrophysiological correlates of these processes by investigating the feedback‐related negativity (FRN) and stimulus‐locked P3. Participants had to learn and recall the location of 2‐digit targets over consecutive rounds. Feedback was provided in two steps, first a color change indicated a correct or incorrect response (feedback phase) followed by presentation of the correct digit information (re‐encoding phase). Behaviorally, participants improved performance from the first to the third round. FRN amplitudes time‐locked to feedback were enhanced for corrected compared to repeated errors. The P3 in response to re‐encoding did not differ between the two error types. The finding that FRN amplitudes positively predicted memory performance is consistent with the idea that the FRN reflects prediction errors and the need for enhanced cognitive control. Interestingly, this happens early during feedback processing and not at a later time point when re‐encoding of correct information takes place. The prediction error signal reflected in the FRN is usually elicited by performance errors, but may thus also play a role in preparing/optimizing the system for memory formation. This supports the existence of a close link between action control and memory processes even when there is a substantial delay between error feedback and the opportunity to correct the error.

Mood congruent tuning of reward expectation in positive mood: evidence from FRN and theta modulations

Positive mood broadens attention and builds additional mental resources. However, its effect on performance monitoring and reward prediction errors remain unclear. To examine this issue, we used a standard mood induction procedure (based on guided imagery) and asked 45 participants to complete a gambling task suited to study reward prediction errors by means of the feedback-related negativity (FRN) and mid-frontal theta band power. Results showed a larger FRN for negative feedback as well as a lack of reward expectation modulation for positive feedback at the theta level with positive mood, relative to a neutral mood condition. A control analysis showed that this latter result could not be explained by the mere superposition of the event-related brain potential component on the theta oscillations. Moreover, these neurophysiological effects were evidenced in the absence of impairments at the behavioral level or increase in autonomic arousal with positive mood, suggesting that this mood state reliably altered brain mechanisms of reward prediction errors during performance monitoring. We interpret these new results as reflecting a genuine mood congruency effect, whereby reward is anticipated as the default outcome with positive mood and therefore processed as unsurprising (even when it is unlikely), while negative feedback is perceived as unexpected.

Medial frontal cortex response to unexpected motivationally salient outcomes

The medial frontal cortex (MFC) plays a central role allocating resources to process salient information, in part by responding to prediction errors. While there is some recent debate, the feedback-related negativity (FRN) is thought to index a reward prediction error by signaling outcomes that are worse than expected. A recent study utilizing electric shock provided data inconsistent with these accounts and reported that the omission of both appetitive (money) and aversive outcomes (electric shocks) elicited a medial frontal negativity. These data suggest that the ERPs within this time range support a salience prediction error that responds to unexpected events regardless of valence. To compare the reward and salience prediction error models, we employed a design that delivered both appetitive (monetary) and aversive (noise burst) outcomes. Participants completed a passive S1/S2 prediction design where S1 predicted S2 with 80% accuracy and S2 predicted the outcome with 100% accuracy. We compared both earlier and later ERP responses over the medial frontal cortex to compare the salience and reward prediction hypotheses. Considering both time windows, the ERP response to S2 in the early time window was most positive when S2 signaled that an outcome was unexpectedly delivered and in the later time window, was most negative when an outcome was unexpectedly withheld, regardless of outcome valence. Thus, these results are more consistent with a salience prediction error rather than a reward prediction error.

Adaptive coordination of working-memory and reinforcement learning in non-human primates performing a trial-and-error problem solving task

Accumulating evidence suggest that human behavior in trial-and-error learning tasks based on decisions between discrete actions may involve a combination of reinforcement learning (RL) and working-memory (WM). While the understanding of brain activity at stake in this type of tasks often involve the comparison with non-human primate neurophysiological results, it is not clear whether monkeys use similar combined RL and WM processes to solve these tasks. Here we analyzed the behavior of five monkeys with computational models combining RL and WM. Our model-based analysis approach enables to not only fit trial-by-trial choices but also transient slowdowns in reaction times, indicative of WM use. We found that the behavior of the five monkeys was better explained in terms of a combination of RL and WM despite inter-individual differences. The same coordination dynamics we used in a previous study in humans best explained the behavior of some monkeys while the behavior of others showed the opposite pattern, revealing a possible different dynamics of WM process. We further analyzed different variants of the tested models to open a discussion on how the long pretraining in these tasks may have favored particular coordination dynamics between RL and WM. This points towards either inter-species differences or protocol differences which could be further tested in humans.

Choice predicts the feedback negativity

Choosing the appropriate response given the circumstance is integral to all aspects of human behavior. One way of elucidating the mechanisms of choice is to relate behavior to neural correlates. Electrophysiological evidence implicates the ERP feedback-negativity (FN) and the P300 as promising neural correlates of reward processing, an integral component of learning. However, prior research has not adequately addressed how the development of a preference to select one option over another (choice preference) relates to the FN and the P300. We assessed whether variation in choice preference predicted the FN and P300 amplitude within subjects. We used a discrete-trials two-alternative choice procedure, where the reinforcer rate for each option was dependently scheduled by a concurrent variable interval. The reinforcer ratio for selecting each option was varied between sessions. Choice was quantified using both the generalized matching law sensitivity and the log odds of staying on the same versus switching to the other alternative (stay preference). The relationship between stay preference, FN, and P300 amplitudes was assessed using the innovative application of hierarchical Bayesian linear regression. The results demonstrate that stay preference was controlled by the reinforcer ratios and credibly predicted the FN amplitude but not P300 amplitude. The findings are consistent with the view that reinforcers may guide behavior by what they signal about future reinforcement, with the FN related to such a process.

The Feedback-related Negativity Reflects the Combination of Instantaneous and Long-term Values of Decision Outcomes

Hundreds of ERP studies have reported a midfrontal negative-going amplitude shift following negative compared with positive action outcomes. This feedback-related negativity (FRN) effect is typically thought to reflect an early and binary mechanism of action evaluation in the posterior midcingulate cortex. However, in prior research on the FRN effect, the instantaneous value and the long-term value of action outcomes have been perfectly confounded. That is, instantaneously positive outcomes were generally consistent with task goals, whereas instantaneously negative outcomes were inconsistent with task goals. In this study, we disentangled these two outcome aspects in two experiments. Our results reveal an interaction of instantaneous and long-term outcome values. More precisely, our findings strongly suggest that the FRN effect is mainly driven by a reward positivity, which is evoked only by outcomes that possess an instantaneously positive value and also help the organism to reach its long-term goals. These findings add to a recent literature according to which the posterior midcingulate cortex acts as a hierarchical reinforcement learning system and suggest that this system integrates instant and long-term action-outcome values. This, in turn, might be crucial for learning optimal behavioral strategies in a given setting.

Multiple signals in anterior cingulate cortex

•

There are multiple signals in anterior cingulate cortex (ACC).

•

ACC activity reflects value of behavioural change even after controlling for difficulty.

•

ACC activity reflects updating of internal models even after controlling for difficulty.

Activity in anterior cingulate cortex (ACC) has been linked both to commitment to a course of action, even when it is associated with costs, and to exploring or searching for alternative courses of action. Here we review evidence that this is due to the presence of multiple signals in ACC reflecting the updating of beliefs and internal models of the environment and encoding aspects of choice value, including the average value of choices afforded by the environment (‘search value’). We contrast this evidence with the influential view that ACC activity is better described as reflecting task difficulty. A consideration of cortical neural network properties explains why ACC may carry such signals and also exhibit sensitivity to task difficulty.

Mistakes were made: neural mechanisms for the adaptive control of action initiation by the medial prefrontal cortex

Studies in rats, monkeys and humans have established that the medial prefrontal cortex is crucial for the ability to exert adaptive control over behavior. Here, we review studies on the role of the rat medial prefrontal cortex in adaptive control, with a focus on simple reaction time tasks that can be easily used across species and have clinical relevance. The performance of these tasks is associated with neural activity in the medial prefrontal cortex that reflects stimulus detection, action timing, and outcome monitoring. We describe rhythmic neural activity that occurs when animals initiate a temporally extended action. Such rhythmic activity is coterminous with major changes in population spike activity. Testing animals over a series of sessions with varying pre-stimulus intervals showed that the signals adapt to the current temporal demands of the task. Disruptions of rhythmic neural activity occur on error trials (premature responding) and lead to a persistent encoding of the error and a subsequent change in behavioral performance (i.e. post-error slowing). Analysis of simultaneously recorded spike activity suggests that the presence of strong theta rhythms is coterminous with altered network dynamics, and might serve as a mechanism for adaptive control. Computational modeling suggests that these signals may enable learning from errors. Together, our findings contribute to an emerging literature and provide a new perspective on the neuronal mechanisms for the adaptive control of action.

Frontal midline theta reflects anxiety and cognitive control: meta-analytic evidence

Evidence from imaging and anatomical studies suggests that the midcingulate cortex (MCC) is a dynamic hub lying at the interface of affect and cognition. In particular, this neural system appears to integrate information about conflict and punishment in order to optimize behavior in the face of action-outcome uncertainty. In a series of meta-analyses, we show how recent human electrophysiological research provides compelling evidence that frontal-midline theta signals reflecting MCC activity are moderated by anxiety and predict adaptive behavioral adjustments. These findings underscore the importance of frontal theta activity to a broad spectrum of control operations. We argue that frontal-midline theta provides a neurophysiologically plausible mechanism for optimally adjusting behavior to uncertainty, a hallmark of situations that elicit anxiety and demand cognitive control. These observations compel a new perspective on the mechanisms guiding motivated learning and behavior and provide a framework for understanding the role of the MCC in temperament and psychopathology.