Longitudinal development of frontoparietal activity during feedback learning: Contributions of age, performance, working memory and cortical thickness

•

We performed a longitudinal study on feedback learning (N = 208, age 8–27 years).

•

We tested linear and nonlinear patterns in frontoparietal activity during learning.

•

DLPFC and parietal cortex showed a late-adolescent peak in activity.

•

SMA showed a linear increase, and ACC a linear decrease in brain activity with age.

•

Performance predicted DLPFC and parietal activity, thickness predicted SMA activity.

Feedback learning is a crucial skill for cognitive flexibility that continues to develop into adolescence, and is linked to neural activity within a frontoparietal network. Although it is well conceptualized that activity in the frontoparietal network changes during development, there is surprisingly little consensus about the direction of change. Using a longitudinal design (N = 208, 8–27 years, two measurements in two years), we investigated developmental trajectories in frontoparietal activity during feedback learning. Our first aim was to test for linear and nonlinear developmental trajectories in dorsolateral prefrontal cortex (DLPFC), superior parietal cortex (SPC), supplementary motor area (SMA) and anterior cingulate cortex (ACC). Second, we tested which factors (task performance, working memory, cortical thickness) explained additional variance in time-related changes in activity besides age. Developmental patterns for activity in DLPFC and SPC were best characterized by a quadratic age function leveling off/peaking in late adolescence. There was a linear increase in SMA and a linear decrease with age in ACC activity. In addition to age, task performance explained variance in DLPFC and SPC activity, whereas cortical thickness explained variance in SMA activity. Together, these findings provide a novel perspective of linear and nonlinear developmental changes in the frontoparietal network during feedback learning.

Phasic dopamine release induced by positive feedback predicts individual differences in reversal learning

Striatal dopamine (DA) is central to reward-based learning. Less is known about the contribution of DA to the ability to adapt previously learned behavior in response to changes in the environment, such as a reversal of response-reward contingencies. We hypothesized that DA is involved in the rapid updating of response-reward information essential for successful reversal learning. We trained rats to discriminate between two levers, where lever availability was signaled by a non-discriminative cue. Pressing one lever was always rewarded, whereas the other lever was never rewarded. After reaching stable discrimination performance, a reversal was presented, so that the previously non-rewarded lever was now rewarded and vice versa. We used fast-scan cyclic voltammetry to monitor DA release in the ventromedial striatum. During discrimination performance (pre-reversal), cue presentation induced phasic DA release, whereas reward delivery did not. The opposite pattern was observed post-reversal: Striatal DA release emerged after reward delivery, while cue-induced release diminished. Trial-by-trial analysis showed rapid reinstatement of cue-induced DA release on trials immediately following initial correct responses. This effect of positive feedback was observed in animals that learned the reversal, but not in 'non-learners'. In contrast, neither pre-reversal responding and DA signaling, nor post-reversal DA signaling in response to negative feedback differed between learners and non-learners. Together, we show that phasic DA dynamics in the ventromedial striatum encoding reward-predicting cues are associated with positive feedback during reversal learning. Furthermore, these signals predict individual differences in learning that are not present prior to reversal, suggesting a distinct role for dopamine in the adaptation of previously learned behavior.

The role of the cerebellum for feedback processing and behavioral switching in a reversal-learning task

Previous studies have reported cerebellar activations during error and reward processing. The present study investigated if the cerebellum differentially processes feedback depending on changes in response strategy during reversal learning, as is conceivable given its internal models for movement and thought. Negative relative to positive feedback in an fMRI-based reversal learning task was hypothesized to be associated with increased cerebellar activations. Moreover, increased activations were expected for negative feedback followed by a change in response strategy compared to negative feedback not followed by such a change, and for first positive feedback after compared to final negative feedback before a change, due to updating of internal models. As predicted, activation in lobules VI and VIIa/Crus I was increased for negative relative to positive feedback, and for final negative feedback before a change in response strategy relative to negative feedback not associated with a change. Moreover, activation was increased for first positive feedback after relative to final negative feedback before a change. These findings are consistent with updating of cerebellar internal models to accommodate new behavioral strategies. Recruitment of posterior regions in reversal learning is in line with the cerebellar functional topography, with posterior regions involved in complex motor and cognitive functions.

Left-shifting prism adaptation boosts reward-based learning

Visuospatial cognition has an inherent lateralized bias. Individual differences in the direction and magnitude of this bias are associated with asymmetrical D2/3 dopamine binding and dopamine system genotypes. Dopamine level affects feedback-based learning and dopamine signaling asymmetry is related to differential learning from reward and punishment. High D2 binding in the left hemisphere is associated with preference for reward. Prism adaptation (PA) is a simple sensorimotor technique, which modulates visuospatial bias according to the direction of the deviation. Left-deviating prism adaptation (LPA) induces rightward bias in healthy subjects. It is therefore possible that the right side of space increases in saliency along with left hemisphere dopaminergic activity. Right-deviating prism adaptation (RPA) has been used mainly as a control condition because it does not modulate behavior in healthy individuals. Since LPA induces a rightward visuospatial bias as a result of left hemisphere modulation, and higher dopaminergic activity in the left hemisphere is associated with preference for rewarding events we hypothesized that LPA would increase the preference for learning with reward. Healthy volunteers performed a computer-based probabilistic classification task before and after LPA or RPA. Consistent with our predictions, PA altered the preference for rewarded versus punished learning, with the LPA group exhibiting increased learning from reward. These results suggest that PA modulates dopaminergic activity in a lateralized fashion.

Minimal Requirements for the Emergence of Learned Signaling

The emergence of signaling systems has been observed in numerous experimental and real‐world contexts, but there is no consensus on which (if any) shared mechanisms underlie such phenomena. A number of explanatory mechanisms have been proposed within several disciplines, all of which have been instantiated as credible working models. However, they are usually framed as being mutually incompatible. Using an exemplar‐based framework, we replicate these models in a minimal configuration which allows us to directly compare them. This reveals that the development of optimal signaling is driven by similar mechanisms in each model, which leads us to propose three requirements for the emergence of conventional signaling. These are the creation and transmission of referential information, a systemic bias against ambiguity, and finally some form of information loss. Considering this, we then discuss some implications for theoretical and experimental approaches to the emergence of learned communication.

Trait anxiety and probabilistic learning: Behavioral and electrophysiological findings

Anxiety is a negative emotion that affects various aspects of people's daily life. To explain why individuals with high anxiety tend to make suboptimal decisions, we suggest that their learning ability might play an important role. Regarding that anxiety modulates both outcome expectation and attention allocation, it is reasonable to hypothesize that the function of feedback learning should be sensitive to individual level of anxiety. However, previous studies that directly examined this hypothesis were scarce. In this study, forty-two Chinese participants were assigned to a high-trait anxiety (HTA) group or a low-trait anxiety (LTA) group according to their scores in the Trait form of Spielberger's State-Trait Anxiety Inventory (STAI-T). Both groups finished a reward learning task in which two options were associated with different winning probabilities. The event-related potential (ERP) elicited by outcome feedback during the task was recorded and analyzed. Behavioral results revealed that, when the winning probability was 80% for one option and 20% for another, the HTA group chose the 80% winning option less often than the LTA group at the initial stage (i.e., first 20 trials) of the task, but there was no between-group difference in total number of choice. In addition, HTA participants took more time to make decisions in the 80/20 condition than in the 50/50 condition, but this effect was insignificant in the LTA group. ERP results indicated that anxiety affects learning in two ways. First, compared to their LTA counterparts, HTA participants showed a smaller feedback-related negativity (FRN) in response to negative feedback, indicating the impact of anxiety on outcome expectation. Second, HTA participants showed a larger P3 component in the 80/20 condition than in the 50/50 condition, indicating the impact of anxiety on attention allocation. Accordingly, we suggest that individuals' ability of feedback learning could be negatively modulated by anxiety.

Feedback-related negativity reflects omission of monetary gains: Evidence from ERP gambling study

Feedback processing is an important aspect of learning. In the human brain, feedback processing is often examined by measuring an event-related potential, the feedback-related negativity component. Typically, the feedback-related negativity component is investigated by directly comparing gain with loss feedback randomized across trials; however, this method does not control for confounds associated with unexpected feedback. For this study we used a blocked design gambling task to investigate the sensitivity of feedback-related negativity to positive and negative feedback separately for gains and losses. While there appeared to be no significant feedback-related negativity in the loss domain, results revealed an enlarged feedback-related negativity during the omission of gains compared to the reception of gains. These findings further support the reward positivity hypothesis which declares that the feedback-related negativity is associated with the processing of outcomes in the context of gains as opposed to losses, irrespective of unexpectedness.

Processing of Positive and Negative Feedback in Patients with Cerebellar Lesions

It is well accepted that the cerebellum plays a crucial role in the prediction of the sensory consequences of movements. Recent findings of altered error processing in patients with selective cerebellar lesions led to the hypothesis that feedback processing and feedback-based learning might be affected by cerebellar damage as well. Thus, the present study investigated learning from and processing of positive and negative feedback in 12 patients with selective cerebellar lesions and healthy control subjects. Participants performed a monetary feedback learning task. The processing of positive and negative feedback was assessed by means of event-related potentials (ERPs) during the learning task and during a separate task in which the frequencies of positive and negative feedback were balanced. Patients did not show a general learning deficit compared to controls. Relative to the control group, however, patients with cerebellar lesions showed significantly higher ERP difference wave amplitudes (rewards-losses) in a time window between 250 and 450 ms after feedback presentation, possibly indicating impaired outcome prediction. The analysis of the original waveforms suggested that patients and controls primarily differed in their pattern of feedback-related negativity and P300 amplitudes. Our results add to recent findings on altered performance monitoring associated with cerebellar damage and demonstrate, for the first time, alterations of feedback processing in patients with cerebellar damage. Unaffected learning performance appears to suggest that chronic cerebellar lesions can be compensated in behaviour.

Reversing the testing effect by feedback: Behavioral and electrophysiological evidence

The testing effect refers to the finding that retrieval practice of previously studied information enhances its long-term retention more than restudy practice does. Recent work showed that the testing effect can be dramatically reversed when feedback is provided to participants during final recall testing (Storm, Friedman, Murayama, & Bjork, 2014). Following this prior work, in this study, we examined the reversal of the testing effect by investigating oscillatory brain activity during final recall testing. Twenty-six healthy participants learned cue-target word pairs and underwent a practice phase in which half of the items were retrieval practiced and half were restudy practiced. Two days later, two cued recall tests were administered, and immediate feedback was provided to participants in Test 1. Behavioral results replicated the prior work by showing a testing effect in Test 1, but a reversed testing effect in Test 2. Extending the prior work, EEG results revealed a feedback-related effect in alpha/lower-beta and retrieval-related effects in slow and fast theta power, with practice condition modulating the fast theta power effect for items that were not recalled in Test 1. The results indicate that the reversed testing effect can arise without differential strengthening of restudied and retrieval-practiced items via feedback learning. Theoretical implications of the findings, in particular with respect to the distribution-based bifurcation model of testing effects (Kornell, Bjork, & Garcia, 2011), are discussed.

Positive-blank versus negative-blank feedback learning in children and adults

In positive-blank feedback learning, positive feedback is given to a correct response whereas blank feedback is given to an incorrect response. Conversely, in negative-blank feedback learning, blank feedback is given to a correct response and negative feedback to an incorrect response. As blank feedback might be subjectively interpreted as signalling a correct response, negative-blank feedback might be more informative than positive-blank feedback, and thus may result in better performance. However, positive-blank feedback might also be superior as it motivates the learner in lengthy tasks. These "information" and "motivation" accounts were tested in a two-block feedback learning paradigm. In the first block, that is, when the task duration was still short, children but not adults profited more from negative than from positive feedback. The results in children thus support the information account. In the second block, that is, when the task duration had become longer, children and adults profited more from positive feedback, thereby supporting the motivation account. Results are discussed in light of behavioural and neuroscientific theories on feedback learning.

Effects of feedback delay on learning from positive and negative feedback in patients with Parkinson's disease off medication

Phasic dopamine (DA) signals conveyed from the substantia nigra to the striatum and the prefrontal cortex crucially affect learning from feedback, with DA bursts facilitating learning from positive feedback and DA dips facilitating learning from negative feedback. Consequently, diminished nigro-striatal dopamine levels as in unmedicated patients suffering from Parkinson's Disease (PD) have been shown to lead to a negative learning bias. Recent studies suggested a diminished striatal contribution to feedback processing when the outcome of an action is temporally delayed. This study investigated whether the bias towards negative feedback learning induced by a lack of DA in PD patients OFF medication is modulated by feedback delay. To this end, PD patients OFF medication and healthy controls completed a probabilistic selection task, in which feedback was given immediately (after 800 ms) or delayed (after 6800 ms). PD patients were impaired in immediate but not delayed feedback learning. However, differences in the preference for positive/negative learning between patients and controls were seen for both learning from immediate and delayed feedback, with evidence of stronger negative learning in patients than controls. A Bayesian analysis of the data supports the conclusion that feedback timing did not affect the learning bias in the patients. These results hint at reduced, but still relevant nigro-striatal contribution to feedback learning, when feedback is delayed.

Effects of reward and punishment on learning from errors in smokers

BACKGROUND

Punishing errors facilitates adaptation in healthy individuals, while aberrant reward and punishment sensitivity in drug-dependent individuals may change this impact. Many societies have institutions that use the concept of punishing drug use behavior, making it important to understand how drug dependency mediates the effects of negative feedback for influencing adaptive behavior.

METHODS

Using an associative learning task, we investigated differences in error correction rates of dependent smokers, compared with controls. Two versions of the task were administered to different participant samples: One assessed the effect of varying monetary contingencies to task performance, the other, the presence of reward as compared to avoidance of punishment for correct performance.

RESULTS

While smokers recalled associations that were rewarded with a higher value 11% more often than lower rewarded locations, they did not correct higher punished locations more often. Controls exhibited the opposite pattern. The three-way interaction between magnitude, feedback type and group was significant, F(1,48) = 5.288, p =0.026, ɳ²p =0.099. Neither participant group corrected locations offering reward more often than those offering avoidances of punishment. The interaction between group and feedback condition was not significant, F(1,58) = 0.0, p =0.99, ɳ²p =0.001.

CONCLUSIONS

The present results suggest that smokers have poorer learning from errors when receiving negative feedback. Moreover, larger rewards reinforce smokers' behavior stronger than smaller rewards, whereas controls made no distinction. These findings support the hypothesis that dependent smokers may respond to positively framed and rewarded anti-smoking programs when compared to those relying on negative feedback or punishment.

Intuitive visualizations of pitch and loudness in speech

Visualizing acoustic features of speech has proven helpful in speech therapy; however, it is as yet unclear how to create intuitive and fitting visualizations. To better understand the mappings from speech sound aspects to visual space, a large web-based experiment (n = 249) was performed to evaluate spatial parameters that may optimally represent pitch and loudness of speech. To this end, five novel animated visualizations were developed and presented in pairwise comparisons, together with a static visualization. Pitch and loudness of speech were each mapped onto either the vertical (y-axis) or the size (z-axis) dimension, or combined (with size indicating loudness and vertical position indicating pitch height) and visualized as an animation along the horizontal dimension (x-axis) over time. The results indicated that firstly, there is a general preference towards the use of the y-axis for both pitch and loudness, with pitch ranking higher than loudness in terms of fit. Secondly, the data suggest that representing both pitch and loudness combined in a single visualization is preferred over visualization in only one dimension. Finally, the z-axis, although not preferred, was evaluated as corresponding better to loudness than to pitch. This relation between sound and visual space has not been reported previously for speech sounds, and elaborates earlier findings on musical material. In addition to elucidating more general mappings between auditory and visual modalities, the findings provide us with a method of visualizing speech that may be helpful in clinical applications such as computerized speech therapy, or other feedback-based learning paradigms.

Updating false beliefs: The role of misplaced vs. well-placed certainty

People can update their misconceptions or false beliefs by learning from corrective sources. However, research has shown that people vary drastically in the extent to which they learn from feedback and update their false beliefs accordingly. That past work drew attention to cognitive and motivational factors such as cognitive rigidity and closed-mindedness as inhibitors of belief updating. Here we examined a novel epistemic structure, misplaced certainty, a subjective sense of certainty while recognizing uncertainty in oneself or most people (e.g., I feel certain although I recognize X is technically uncertain or it is technically uncertain according to most people), as a unique predictor of lower belief updating. In a preregistered study, we hypothesized that those with high chronic misplaced certainty would be less likely to learn from feedback and revise their misconceptions in a feedback-learning task. In our analyses, we controlled for well-placed certainty-certainty while recognizing no doubt in oneself or most others. We also controlled for variables associated with closed-minded cognition. Consistent with our predictions, those with high misplaced certainty were less likely to revise their false beliefs in response to corrective feedback. In contrast, those with high well-placed certainty were more likely to learn from corrective feedback and revise their false beliefs. By shedding light on the nuances of different forms of subjective certainty, the present work aims to pave the way for further research on epistemic factors in the perseverance and correction of false beliefs.

Give it a second try? The influence of feedback and performance in the decision of reattempting

Feedback evaluation can affect behavioural continuation or discontinuation, and is essential for cognitive and motor skill learning. One critical factor that influences feedback evaluation is participants' internal estimation of self-performance. Previous research has shown that two event-related potential components, the Feedback-Related Negativity (FRN) and the P3, are related to feedback evaluation. In the present study, we used a time estimation task and EEG recordings to test the influence of feedback and performance on participants' decisions, and the sensitivity of the FRN and P3 components to those factors. In the experiment, participants were asked to reproduce the total duration of an intermittently presented visual stimulus. Feedback was given after every response, and participants had then to decide whether to retry the same trial and try to earn reward points, or to move on to the next trial. Results showed that both performance and feedback influenced participants' decision on whether to retry the ongoing trial. In line with previous studies, the FRN showed larger amplitude in response to negative than to positive feedback. Moreover, our results were also in agreement with previous works showing the relationship between the amplitude of the FRN and the size of feedback-related prediction error (PE), and provide further insight in how PE size influences participants' decisions on whether or not to retry a task. Specifically, we found that the larger the FRN, the more likely participants were to base their decision on their performance - choosing to retry the current trial after good performance or to move on to the next trial after poor performance, regardless of the feedback received. Conversely, the smaller the FRN, the more likely participants were to base their decision on the feedback received.