Processing of action- but not stimulus-related prediction errors differs between active and observational feedback learning

The Role of the Human Cerebellum for Learning from and Processing of External Feedback in Non-Motor Learning: A Systematic Review

This review aimed to systematically identify and comprehensively review the role of the cerebellum in performance monitoring, focusing on learning from and on processing of external feedback in non-motor learning. While 1078 articles were screened for eligibility, ultimately 36 studies were included in which external feedback was delivered in cognitive tasks and which referenced the cerebellum. These included studies in patient populations with cerebellar damage and studies in healthy subjects applying neuroimaging. Learning performance in patients with different cerebellar diseases was heterogeneous, with only about half of all patients showing alterations. One patient study using EEG demonstrated that damage to the cerebellum was associated with altered neural processing of external feedback. Studies assessing brain activity with task-based fMRI or PET and one resting-state functional imaging study that investigated connectivity changes following feedback-based learning in healthy participants revealed involvement particularly of lateral and posterior cerebellar regions in processing of and learning from external feedback. Cerebellar involvement was found at different stages, e.g., during feedback anticipation and following the onset of the feedback stimuli, substantiating the cerebellum's relevance for different aspects of performance monitoring such as feedback prediction. Future research will need to further elucidate precisely how, where, and when the cerebellum modulates the prediction and processing of external feedback information, which cerebellar subregions are particularly relevant, and to what extent cerebellar diseases alter these processes.

The human cerebellum in reward anticipation and reward outcome processing: An activation likelihood estimation meta-analysis

The cerebellum generates internal prediction models and actively compares anticipated and actual outcomes in order to reach a desired end state. In this process, reward can serve as a reinforcer that shapes internal prediction models, enabling context-appropriate behavior. While the involvement of the cerebellum in reward processing has been established in animals, there is no detailed account of which cerebellar regions are involved in reward anticipation and reward outcome processing in humans. To this end, an activation likelihood estimation meta-analysis of functional neuroimaging studies was performed to investigate cerebellar functional activity patterns associated with reward anticipation and reward outcome processing in healthy adults. Results showed that reward anticipation (k=31) was associated with regional activity in the bilateral anterior lobe, bilateral lobule VI, left Crus I and the posterior vermis, while reward outcome (k=16) was associated with regional activity in the declive and left lobule VI. The findings of this meta-analysis show distinct involvement of the cerebellum in reward anticipation and reward outcome processing as part of a predictive coding routine.

Cost-benefit trade-offs in decision-making and learning

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes.

Value-based decision-making involves trading off the cost associated with an action–such as physical or mental effort–against its expected reward. Although facing conflicts between competing action alternatives is considered aversive and effortful, it remains unclear whether conflict also constitutes a cost in value-based decisions. We tested this hypothesis by combining a classic conflict (flanker) task with a reinforcement-learning task. Results showed that participants learned to maximise their earnings, but were nevertheless biased to follow irrelevant suggestions. Computational model-based analyses showed a greater choice bias with more uncertainty about the best action to make, supporting the hypothesis that the costs linked to conflict management were traded off against expected rewards. We additionally found that learning rates were reduced when following instructions, relative to when choosing freely what to do. Learning was further reduced by conflict between instructions and subjective action values. In short, we found that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and that different types of conflict may have different effects on learning about action outcomes.

Role of Anterior Cingulate Cortex in Instrumental Learning: Blockade of Dopamine D1 Receptors Suppresses Overt but Not Covert Learning

HIGHLIGHTS

Blockade of dopamine D1 receptors in ACC suppressed instrumental learning when overt responding was required.

Covert learning through observation was not impaired.

After treatment with a dopamine antagonist, instrumental learning recovered but not the rat's pretreatment level of effort tolerance.

ACC dopamine is not necessary for acquisition of task-relevant cues during learning, but regulates energy expenditure and effort based decision.

Dopamine activity in anterior cingulate cortex (ACC) is essential for various aspects of instrumental behavior, including learning and effort based decision making. To dissociate learning from physical effort, we studied both observational (covert) learning, and trial-and-error (overt) learning. If ACC dopamine activity is required for task acquisition, its blockade should impair both overt and covert learning. If dopamine is not required for task acquisition, but solely for regulating the willingness to expend effort for reward, i.e., effort tolerance, blockade should impair overt learning but spare covert learning. Rats learned to push a lever for food rewards either with or without prior observation of an expert conspecific performing the same task. Before daily testing sessions, the rats received bilateral ACC microinfusions of SCH23390, a dopamine D1 receptor antagonist, or saline-control infusions. We found that dopamine blockade suppressed overt responding selectively, leaving covert task acquisition through observational learning intact. In subsequent testing sessions without dopamine blockade, rats recovered their overt-learning capacity but not their pre-treatment level of effort tolerance. These results suggest that ACC dopamine is not required for the acquisition of conditioned behaviors and that apparent learning impairments could instead reflect a reduced level of willingness to expend effort due to cortical dopamine blockade.

Processing of Positive and Negative Feedback in Patients with Cerebellar Lesions

It is well accepted that the cerebellum plays a crucial role in the prediction of the sensory consequences of movements. Recent findings of altered error processing in patients with selective cerebellar lesions led to the hypothesis that feedback processing and feedback-based learning might be affected by cerebellar damage as well. Thus, the present study investigated learning from and processing of positive and negative feedback in 12 patients with selective cerebellar lesions and healthy control subjects. Participants performed a monetary feedback learning task. The processing of positive and negative feedback was assessed by means of event-related potentials (ERPs) during the learning task and during a separate task in which the frequencies of positive and negative feedback were balanced. Patients did not show a general learning deficit compared to controls. Relative to the control group, however, patients with cerebellar lesions showed significantly higher ERP difference wave amplitudes (rewards-losses) in a time window between 250 and 450 ms after feedback presentation, possibly indicating impaired outcome prediction. The analysis of the original waveforms suggested that patients and controls primarily differed in their pattern of feedback-related negativity and P300 amplitudes. Our results add to recent findings on altered performance monitoring associated with cerebellar damage and demonstrate, for the first time, alterations of feedback processing in patients with cerebellar damage. Unaffected learning performance appears to suggest that chronic cerebellar lesions can be compensated in behaviour.

Structural and metabolic differentiation between bipolar disorder with psychosis and substance-induced psychosis: An integrated MRI/PET study

BACKGROUND

Bipolar disorder (BD) may be characterized by the presence of psychotic symptoms and comorbid substance abuse. In this context, structural and metabolic dysfunctions have been reported in both BD with psychosis and addiction, separately. In this study, we aimed at identifying neural substrates differentiating psychotic BD, with or without substance abuse, versus substance-induced psychosis (SIP) by coupling, for the first time, magnetic resonance imaging (MRI) and positron emission tomography (PET).

METHODS

Twenty-seven BD type I psychotic patients with (n=10) or without (n=17) substance abuse, 16 SIP patients and 54 healthy controls were enrolled in this study. 3T MRI and 18-FDG-PET scanning were acquired.

RESULTS

Gray matter (GM) volume and cerebral metabolism reductions in temporal cortices were observed in all patients compared to healthy controls. Moreover, a distinct pattern of fronto-limbic alterations were found in patients with substance abuse. Specifically, BD patients with substance abuse showed volume reductions in ventrolateral prefrontal cortex, anterior cingulate, insula and thalamus, whereas SIP patients in dorsolateral prefrontal cortex and posterior cingulate. Common alterations in cerebellum, parahippocampus and posterior cingulate were found in both BD with substance abuse and SIP. Finally, a unique pattern of GM volumes reduction, with concomitant increased of striatal metabolism, were observed in SIP patients.

CONCLUSIONS

These findings contribute to shed light on the identification of common and distinct neural markers associated with bipolar psychosis and substance abuse. Future longitudinal studies should explore the effect of single substances of abuse in patients at the first-episode of BD and substance-induced psychosis.

Neural substrates and behavioral profiles of romantic jealousy and its temporal dynamics

Jealousy is not only a way of experiencing love but also a stabilizer of romantic relationships, although morbid romantic jealousy is maladaptive. Being engaged in a formal romantic relationship can tune one's romantic jealousy towards a specific target. Little is known about how the human brain processes romantic jealousy by now. Here, by combining scenario-based imagination and functional MRI, we investigated the behavioral and neural correlates of romantic jealousy and their development across stages (before vs. after being in a formal relationship). Romantic jealousy scenarios elicited activations primarily in the basal ganglia (BG) across stages, and were significantly higher after the relationship was established in both the behavioral rating and BG activation. The intensity of romantic jealousy was related to the intensity of romantic happiness, which mainly correlated with ventral medial prefrontal cortex activation. The increase in jealousy across stages was associated with the tendency for interpersonal aggression. These results bridge the gap between the theoretical conceptualization of romantic jealousy and its neural correlates and shed light on the dynamic changes in jealousy.

Temporal dynamics and potential neural sources of goal conduciveness, control, and power appraisal

A major emotion theory, the Component Process Model, predicts that emotion-antecedent appraisal proceeds sequentially (e.g., goal conduciveness>control>power appraisal). In a gambling task, feedback manipulated information about goal conduciveness (outcome: win, loss), control (perceived high and low control), and power appraisals (choice options to change the outcome). Using mean amplitudes of event-related potentials, we examine the sequential prediction of these appraisal criteria. Additionally, we apply source localization analysis to estimate the neural sources of the evoked components of interest. Early ERPs (230-300 ms) show main effects of goal conduciveness and power but no interaction effects suggesting goal obstructiveness assessment of task-relevant feedback information. Late ERPs (350-600 ms) reveal main effects of all appraisals and interaction effects representing the integration of all appraisal information. Source localization analysis suggests distinct neural sources for these appraisal criteria.