Robust valence-induced biases on motor response and confidence in human reinforcement learning

Neural and computational underpinnings of biased confidence in human reinforcement learning

While navigating a fundamentally uncertain world, humans and animals constantly evaluate the probability of their decisions, actions or statements being correct. When explicitly elicited, these confidence estimates typically correlates positively with neural activity in a ventromedial-prefrontal (VMPFC) network and negatively in a dorsolateral and dorsomedial prefrontal network. Here, combining fMRI with a reinforcement-learning paradigm, we leverage the fact that humans are more confident in their choices when seeking gains than avoiding losses to reveal a functional dissociation: whereas the dorsal prefrontal network correlates negatively with a condition-specific confidence signal, the VMPFC network positively encodes task-wide confidence signal incorporating the valence-induced bias. Challenging dominant neuro-computational models, we found that decision-related VMPFC activity better correlates with confidence than with option-values inferred from reinforcement-learning models. Altogether, these results identify the VMPFC as a key node in the neuro-computational architecture that builds global feeling-of-confidence signals from latent decision variables and contextual biases during reinforcement-learning.

Metacognition and the effect of incentive motivation in two compulsive disorders: Gambling disorder and obsessive-compulsive disorder

AIMS

Compulsivity is a common phenotype among psychiatric disorders, such as obsessive-compulsive disorder (OCD) and gambling disorder (GD). Deficiencies in metacognition, such as the inability to estimate one's performance via confidence judgments could contribute to pathological decision-making. Earlier research has shown that patients with OCD exhibit underconfidence, while patients with GD exhibit overconfidence. Moreover, it is known that motivational states (e.g. monetary incentives) influence metacognition, with gain (respectively loss) prospects increasing (respectively decreasing) confidence. Here, we reasoned that OCD and GD symptoms might correspond to an exacerbation of this interaction between metacognition and motivation.

METHODS

We hypothesized GD's overconfidence to be exaggerated during gain prospects, while OCD's underconfidence to be worsened in loss context, which we expected to see represented in ventromedial prefrontal cortex (VMPFC) blood-oxygen-level-dependent activity. We tested those hypotheses in a task-based functional magnetic resonance imaging (fMRI) design (27 patients with GD, 28 patients with OCD, 55 controls). The trial is registered in the Dutch Trial Register (NL6171).

RESULTS

We showed increased confidence for patients with GD versus patients with OCD, which could partly be explained by sex and IQ. Although our primary analyses did not support the hypothesized interaction between incentives and groups, exploratory analyses did show increased confidence in patients with GD specifically in gain context. fMRI analyses confirmed a central role for VMPFC in the processing of confidence and incentives, but no differences between the groups.

CONCLUSION

Patients with OCD and those with GD reside at opposite ends of the confidence spectrum, while no interaction with incentives was found, nor group differences in neuronal processing of confidence.

Motivational signals disrupt metacognitive signals in the human ventromedial prefrontal cortex

A growing body of evidence suggests that, during decision-making, BOLD signal in the ventromedial prefrontal cortex (VMPFC) correlates both with motivational variables – such as incentives and expected values – and metacognitive variables – such as confidence judgments – which reflect the subjective probability of being correct. At the behavioral level, we recently demonstrated that the value of monetary stakes bias confidence judgments, with gain (respectively loss) prospects increasing (respectively decreasing) confidence judgments, even for similar levels of difficulty and performance. If and how this value-confidence interaction is reflected in the VMPFC remains unknown. Here, we used an incentivized perceptual decision-making fMRI task that dissociates key decision-making variables, thereby allowing to test several hypotheses about the role of the VMPFC in the value-confidence interaction. While our initial analyses seemingly indicate that the VMPFC combines incentives and confidence to form an expected value signal, we falsified this conclusion with a meticulous dissection of qualitative activation patterns. Rather, our results show that strong VMPFC confidence signals observed in trials with gain prospects are disrupted in trials with no – or negative (loss) – monetary prospects. Deciphering how decision variables are represented and interact at finer scales seems necessary to better understand biased (meta)cognition.

The human ventromedial prefrontal cortex helps to determine value and confidence in certain decisions, but only in situations when there is a potential for a (monetary) reward.