On-line confidence monitoring during decision making

Examining interpersonal metacognitive monitoring in artificial grammar learning

This study investigates the observers' ability to monitor the ongoing cognitive processes of a partner who is implicitly learning an artificial grammar. Our hypothesis posits that learners experience metacognitive feelings as they attempt to apply their implicit knowledge, and that observers are capable of detecting and interpreting these feelings as cues of the learner's cognitive state. For instance, learners might encounter affective signals linked to cognitive conflicts and errors at different processing stages, which observers can construe as manifestations of the learner's cognitive dissonance. The research involved 126 participants organized into dyads, with one participant acting as a learner, and the other as an observer. The observer's task was to judge whether the learner agrees with the information presented (consonance judgment) and was limited to reading the learner's nonverbal signals to avoid explicit mindreading. The findings suggest that observers possess mindreading abilities, enabling them to detect both learners' confidence and accuracy in stimuli classification. This extends our understanding of non-verbal mindreading capabilities and indicates that observers can effectively interpret early implicit metacognitive information, even in the absence of explicit self-evaluation from the learners. This research offers significant insights into how individuals interpret others' mental states during implicit learning tasks, particularly in the context of utilizing early affective cues within the Artificial Grammar Learning paradigm.

Facial representations of complex affective states combining pain and a negative emotion

Pain is rarely communicated alone, as it is often accompanied by emotions such as anger or sadness. Communicating these affective states involves shared representations. However, how an individual conceptually represents these combined states must first be tested. The objective of this study was to measure the interaction between pain and negative emotions on two types of facial representations of these states, namely visual (i.e., interactive virtual agents; VAs) and sensorimotor (i.e., one's production of facial configurations). Twenty-eight participants (15 women) read short written scenarios involving only pain or a combined experience of pain and a negative emotion (anger, disgust, fear, or sadness). They produced facial configurations representing these experiences on the faces of the VAs and on their face (own production or imitation of VAs). The results suggest that affective states related to a direct threat to the body (i.e., anger, disgust, and pain) share a similar facial representation, while those that present no immediate danger (i.e., fear and sadness) differ. Although visual and sensorimotor representations of these states provide congruent affective information, they are differently influenced by factors associated with the communication cycle. These findings contribute to our understanding of pain communication in different affective contexts.

The asymmetric impact of decision-making confidence on regret and relief

When individuals make uncertain decisions, they often evaluate the correctness of their choices in what is referred to as decision-making confidence. The outcomes of such decision-making can lead to counterfactual thinking wherein alternative possible outcomes are contemplated. This, in turn, can elicit counterfactual emotions including upward and downward counterfactual thinking, which, respectively, refer to regret and relief. Decision-making confidence and counterfactual emotions have key effects on how individuals learn from the past and prepare for the future. However, there has been little understanding of how these experiences are related. For this study, 98 total adults were recruited with the goal of assessing the connections between decision-making confidence and sensations of regret and relief when completing a card-based gambling task. The results of this study suggest that decision-making confidence may reduce the intensity of relief while increasing the degree of regret experienced. These findings thus emphasize the important effect that decision confidence has on emotional processing.

The timing of confidence computations in human prefrontal cortex

Knowing when confidence computations take place is critical for building a mechanistic understanding of the neural and computational bases of metacognition. Yet, even though a substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. To understand when confidence is computed, we delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation while subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating. TMS was delivered to either the right dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to right DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive efficiency. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 msec after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

The timing of confidence computations in human prefrontal cortex

Knowing when confidence computations take place is critical for building mechanistic understanding of the neural and computational bases of metacognition. Yet, even though substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. Subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating regarding the accuracy of their decision. We delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation. TMS was delivered to either dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive ability. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 ms after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

Optimal policy for uncertainty estimation concurrent with decision making

Decision making often depends on vague information that leads to uncertainty, which is a quantity contingent not on choice but on probability distributions of sensory evidence and other cognitive variables. Uncertainty may be computed in parallel and interact with decision making. Here, we adapt the classic random-dot motion direction discrimination task to allow subjects to indicate their uncertainty without having to form a decision first. The subjects' choices and reaction times for perceptual decisions and uncertainty responses are measured, respectively. We then build a value-based model in which decisions are based on optimizing value computed from a drift-diffusion process. The model accounts for key features of subjects' behavior and the variation across the individuals. It explains how the addition of the uncertainty option affects perceptual decision making. Our work establishes a value-based theoretical framework for studying uncertainty and perceptual decisions that can be readily applied in future investigations of the underlying neural mechanism.

The relationship between response dynamics and the formation of confidence varies across the lifespan

Accurate metacognitive judgments, such as forming a confidence judgment, are crucial for goal-directed behavior but decline with older age. Besides changes in the sensory processing of stimulus features, there might also be changes in the motoric aspects of giving responses that account for age-related changes in confidence. In order to assess the association between confidence and response parameters across the adult lifespan, we measured response times and peak forces in a four-choice flanker task with subsequent confidence judgments. In 65 healthy adults from 20 to 76 years of age, we showed divergent associations of each measure with confidence, depending on decision accuracy. Participants indicated higher confidence after faster responses in correct but not incorrect trials. They also indicated higher confidence after less forceful responses in errors but not in correct trials. Notably, these associations were age-dependent as the relationship between confidence and response time was more pronounced in older participants, while the relationship between confidence and response force decayed with age. Our results add to the notion that confidence is related to response parameters and demonstrate noteworthy changes in the observed associations across the adult lifespan. These changes potentially constitute an expression of general age-related deficits in performance monitoring or, alternatively, index a failing mechanism in the computation of confidence in older adults.

Consensus Goals in the Field of Visual Metacognition

Despite the tangible progress in psychological and cognitive sciences over the last several years, these disciplines still trail other more mature sciences in identifying the most important questions that need to be solved. Reaching such consensus could lead to greater synergy across different laboratories, faster progress, and increased focus on solving important problems rather than pursuing isolated, niche efforts. Here, 26 researchers from the field of visual metacognition reached consensus on four long-term and two medium-term common goals. We describe the process that we followed, the goals themselves, and our plans for accomplishing these goals. If this effort proves successful within the next few years, such consensus building around common goals could be adopted more widely in psychological science.

Evidence integration and decision confidence are modulated by stimulus consistency

Evidence integration is a normative algorithm for choosing between alternatives with noisy evidence, which has been successful in accounting for vast amounts of behavioural and neural data. However, this mechanism has been challenged by non-integration heuristics, and tracking decision boundaries has proven elusive. Here we first show that the decision boundaries can be extracted using a model-free behavioural method termed decision classification boundary, which optimizes choice classification based on the accumulated evidence. Using this method, we provide direct support for evidence integration over non-integration heuristics, show that the decision boundaries collapse across time and identify an integration bias whereby incoming evidence is modulated based on its consistency with preceding information. This consistency bias, which is a form of pre-decision confirmation bias, was supported in four cross-domain experiments, showing that choice accuracy and decision confidence are modulated by stimulus consistency. Strikingly, despite its seeming sub-optimality, the consistency bias fosters performance by enhancing robustness to integration noise.

Confidence Modulates the Conformity Behavior of the Investors and Neural Responses of Social Influence in Crowdfunding

The decision about whether to invest can be affected by the choices or opinions of others known as a form of social influence. People make decisions with fluctuating confidence, which plays an important role in the decision process. However, it remains a fair amount of confusion regarding the effect of confidence on the social influence as well as the underlying neural mechanism. The current study applied a willingness-to-invest task with the event-related potentials method to examine the behavioral and neural manifestations of social influence and its interaction with confidence in the context of crowdfunding investment. The behavioral results demonstrate that the conformity tendency of the people increased when their willingness-to-invest deviated far from the group. Besides, when the people felt less confident about their initial judgment, they were more likely to follow the herd. In conjunction with the behavioral findings, the neural results of the social information processing indicate different susceptibilities to small and big conflicts between the own willingness of the people and the group, with small conflict evoked less negative feedback-related negativity (FRN) and more positive late positive potential (LPP). Moreover, confidence only modulated the later neural processing by eliciting larger LPP in the low confidence, implying more reliance on social information. These results corroborate previous findings regarding the conformity effect and its neural mechanism in investment decision and meanwhile extend the existing works of literature through providing behavioral and neural evidence to the effect of confidence on the social influence in the crowdfunding marketplace.

Action planning and control under uncertainty emerge through a desirability-driven competition between parallel encoding motor plans

Living in an uncertain world, nearly all of our decisions are made with some degree of uncertainty about the consequences of actions selected. Although a significant progress has been made in understanding how the sensorimotor system incorporates uncertainty into the decision-making process, the preponderance of studies focus on tasks in which selection and action are two separate processes. First people select among alternative options and then initiate an action to implement the choice. However, we often make decisions during ongoing actions in which the value and availability of the alternatives can change with time and previous actions. The current study aims to decipher how the brain deals with uncertainty in decisions that evolve while acting. To address this question, we trained individuals to perform rapid reaching movements towards two potential targets, where the true target location was revealed only after the movement initiation. We found that reaction time and initial approach direction are correlated, where initial movements towards intermediate locations have longer reaction times than movements that aim directly to the target locations. Interestingly, the association between reaction time and approach direction was independent of the target probability. By modeling the task within a recently proposed neurodynamical framework, we showed that action planning and control under uncertainty emerge through a desirability-driven competition between motor plans that are encoded in parallel.

Confidence guides priority between forthcoming tasks

Humans can estimate confidence in their decisions, and there is increasing interest on how this feeling of confidence regulates future behavior. Here, we investigate whether confidence in a perceptual task affects prioritizing future trials of that task, independently of task performance. To do so, we experimentally dissociated confidence from performance. Participants judged whether an array of differently colored circles was closer to blue or red, and we manipulated the mean and variability of the circles' colors across the array. We first familiarized participants with a low mean low variability condition and a high mean high variability condition, which were matched in performance despite participants being more confident in the former. Then we made participants decide in which order to complete forthcoming trials for both conditions. Crucially, prioritizing one condition was associated with being more confident in that condition compared to the other. This relationship was observed both across participants, by correlating inter-individual heterogeneity in prioritization and in confidence, and within participants, by assessing how changes in confidence with accuracy, condition and response times could predict prioritization choices. Our results suggest that confidence, above and beyond performance, guides prioritization between forthcoming tasks, strengthening the evidence for its role in regulating behavior.

Timing of readiness potentials reflect a decision-making process in the human brain

Decision-making in two-alternative forced choice tasks has several underlying components including stimulus encoding, perceptual categorization, response selection, and response execution. Sequential sampling models of decision-making are based on an evidence accumulation process to a decision boundary. Animal and human studies have focused on perceptual categorization and provide evidence linking brain signals in parietal cortex to the evidence accumulation process. In this exploratory study, we use a task where the dominant contribution to response time is response selection and model the response time data with the drift-diffusion model. EEG measurement during the task show that the Readiness Potential (RP) recorded over motor areas has timing consistent with the evidence accumulation process. The duration of the RP predicts decision-making time, the duration of evidence accumulation, suggesting that the RP partly reflects an evidence accumulation process for response selection in the motor system. Thus, evidence accumulation may be a neural implementation of decision-making processes in both perceptual and motor systems. The contributions of perceptual categorization and response selection to evidence accumulation processes in decision-making tasks can be potentially evaluated by examining the timing of perceptual and motor EEG signals.

Understanding neural signals of post-decisional performance monitoring: An integrative review

Performance monitoring is a key cognitive function, allowing to detect mistakes and adapt future behavior. Post-decisional neural signals have been identified that are sensitive to decision accuracy, decision confidence and subsequent adaptation. Here, we review recent work that supports an understanding of late error/confidence signals in terms of the computational process of post-decisional evidence accumulation. We argue that the error positivity, a positive-going centro-parietal potential measured through scalp electrophysiology, reflects the post-decisional evidence accumulation process itself, which follows a boundary crossing event corresponding to initial decision commitment. This proposal provides a powerful explanation for both the morphological characteristics of the signal and its relation to various expressions of performance monitoring. Moreover, it suggests that the error positivity -a signal with thus far unique properties in cognitive neuroscience - can be leveraged to furnish key new insights into the inputs to, adaptation, and consequences of the post-decisional accumulation process.

Chimpanzees (Pan troglodytes) show subtle signs of uncertainty when choices are more difficult

Humans can tell when they find a task difficult. Subtle uncertainty behaviors like changes in motor speed and muscle tension precede and affect these experiences. Theories of animal metacognition likewise stress the importance of endogenous signals of uncertainty as cues that motivate metacognitive behaviors. However, while researchers have investigated second-order behaviors like information seeking and declining difficult trials in nonhuman animals, they have devoted little attention to the behaviors that express the cognitive conflict that gives rise to such behaviors in the first place. Here we explored whether three chimpanzees would, like humans, show hand wavering more when faced with more difficult choices in a touch screen transitive inference task. While accuracy was very high across all conditions, all chimpanzees wavered more frequently in trials that were objectively more difficult, demonstrating a signature behavior which accompanies experiences of difficulty in humans. This lends plausibility to the idea that feelings of uncertainty, like other emotions, can be studied in nonhuman animals. We propose to routinely assess uncertainty behaviors to inform models of procedural metacognition in nonhuman animals.

Affective Bias Through the Lens of Signal Detection Theory

Affective bias – a propensity to focus on negative information at the expense of positive information – is a core feature of many mental health problems. However, it can be caused by wide range of possible underlying cognitive mechanisms. Here we illustrate this by focusing on one particular behavioural signature of affective bias – increased tendency of anxious/depressed individuals to predict lower rewards – in the context of the Signal Detection Theory (SDT) modelling framework. Specifically, we show how to apply this framework to measure affective bias and compare it to the behaviour of an optimal observer. We also show how to extend the framework to make predictions about bias when the individual holds incorrect assumptions about the decision context. Building on this theoretical foundation, we propose five experiments to test five hypothetical sources of this affective bias: beliefs about prior probabilities, beliefs about performance, subjective value of reward, learning differences, and need for accuracy differences. We argue that greater precision about the mechanisms driving affective bias may eventually enable us to better understand the mechanisms underlying mood and anxiety disorders.

Confidence in visual motion discrimination is preserved in individuals with schizophrenia

BACKGROUND

Metacognition is the set of reflexive processes that allows humans to evaluate the accuracy of their mental operations. Metacognitive deficits have been described in people with schizophrenia using mostly narrative assessment, and they have been linked to several key symptoms.

METHODS

We assessed metacognitive performance objectively by asking people with schizophrenia or schizoaffective disorder (n = 20) and matched healthy participants (n = 21) to perform a visual discrimination task and report their confidence in their performance. Metacognitive performance was defined as the adequacy between visual discrimination performance and confidence.

RESULTS

Bayesian analyses revealed equivalent metacognitive performance in the 2 groups, despite a weaker association between confidence and trajectory tracking during task execution among people with schizophrenia. We reproduced these results using an evidence accumulation model, which showed similar decisional processes in the 2 groups.

LIMITATIONS

These results from a relatively small study sample cannot be generalized to other perceptual and nonperceptual tasks. To meet this purpose, ecological tasks are needed. As well, the role of antipsychotic medication and design deserves greater attention in the future.

CONCLUSION

We found similar decisional and metacognitive capabilities between people with schizophrenia and healthy controls in a visual discrimination task.

Confidence in predicted position error explains saccadic decisions during pursuit

A fundamental problem in motor control is the coordination of complementary movement types to achieve a common goal. As a common example, humans view moving objects through coordinated pursuit and saccadic eye movements. Pursuit is initiated and continuously controlled by retinal image velocity. During pursuit, eye position may lag behind the target. This can be compensated by the discrete execution of a catch-up saccade. The decision to trigger a saccade is influenced by both position and velocity errors, and the timing of saccades can be highly variable. The observed distributions of saccade frequency and trigger time remain poorly understood, and this decision process remains imprecisely quantified. Here, we propose a predictive, probabilistic model explaining the decision to trigger saccades during pursuit to foveate moving targets. In this model, expected position error and its associated uncertainty are predicted through Bayesian inference across noisy, delayed sensory observations (Kalman filtering). This probabilistic prediction is used to estimate the confidence that a saccade is needed (quantified through log-probability ratio), triggering a saccade upon accumulating to a fixed threshold. The model qualitatively explains behavioral observations on the frequency and trigger time distributions of saccades during pursuit over a range of target motion trajectories. Furthermore, this model makes novel predictions that saccade decisions are highly sensitive to uncertainty for small predicted position errors, but this influence diminishes as the magnitude of predicted position error increases. We suggest that this predictive, confidence-based decision-making strategy represents a fundamental principle for the probabilistic neural control of coordinated movements.NEW & NOTEWORTHY This is the first stochastic dynamical systems model of pursuit-saccade coordination accounting for noise and delays in the sensorimotor system. The model uses Bayesian inference to predictively estimate visual motion, triggering saccades when confidence in predicted position error accumulates to a threshold. This model explains saccade frequency and trigger time distributions across target trajectories and makes novel predictions about the influence of sensory uncertainty in saccade decisions during pursuit.

Performance monitoring for sensorimotor confidence: A visuomotor tracking study

To best interact with the external world, humans are often required to consider the quality of their actions. Sometimes the environment furnishes rewards or punishments to signal action efficacy. However, when such feedback is absent or only partial, we must rely on internally generated signals to evaluate our performance (i.e., metacognition). Yet, very little is known about how humans form such judgements of sensorimotor confidence. Do they monitor their actual performance or do they rely on cues to sensorimotor uncertainty? We investigated sensorimotor metacognition in two visuomotor tracking experiments, where participants followed an unpredictably moving dot cloud with a mouse cursor as it followed a random horizontal trajectory. Their goal was to infer the underlying target generating the dots, track it for several seconds, and then report their confidence in their tracking as better or worse than their average. In Experiment 1, we manipulated task difficulty with two methods: varying the size of the dot cloud and varying the stability of the target's velocity. In Experiment 2, the stimulus statistics were fixed and duration of the stimulus presentation was varied. We found similar levels of metacognitive sensitivity in all experiments, which was evidence against the cue-based strategy. The temporal analysis of metacognitive sensitivity revealed a recency effect, where error later in the trial had a greater influence on the sensorimotor confidence, consistent with a performance-monitoring strategy. From these results, we conclude that humans predominantly monitored their tracking performance, albeit inefficiently, to build a sense of sensorimotor confidence.

Single-Neuron Correlates of Decision Confidence in the Human Medial Temporal Lobe

The human medial temporal lobe (MTL) has been suggested to play a role in valuation. However, little is known about its role in binary decisions and metacognition. We performed two decision-making tasks while recording from neurons in the human MTL. During a break, subjects consumed their preferred food item to satiation and subsequently repeated both tasks. We identified both a persistent and a transient modulation of the neural activity. Two independent subpopulations of neurons showed a persistent correlation of their firing rates with either decision confidence or reaction times. Importantly, the changes in confidence and reaction time between experimental sets were accompanied by a correlated change in the neural activity, and this correlation lasted as long as it was relevant for the behavioral task. Previous studies have suggested a transient modulation of the neural activity in the human MTL correlated with subjective value. However, in our study, neither subjective value nor unsigned value could explain this transient activity better than the nutritional features of the stimuli, calling into question the role of the human MTL in valuation.

Robust valence-induced biases on motor response and confidence in human reinforcement learning

In simple instrumental-learning tasks, humans learn to seek gains and to avoid losses equally well. Yet, two effects of valence are observed. First, decisions in loss-contexts are slower. Second, loss contexts decrease individuals’ confidence in their choices. Whether these two effects are two manifestations of a single mechanism or whether they can be partially dissociated is unknown. Across six experiments, we attempted to disrupt the valence-induced motor bias effects by manipulating the mapping between decisions and actions and imposing constraints on response times (RTs). Our goal was to assess the presence of the valence-induced confidence bias in the absence of the RT bias. We observed both motor and confidence biases despite our disruption attempts, establishing that the effects of valence on motor and metacognitive responses are very robust and replicable. Nonetheless, within- and between-individual inferences reveal that the confidence bias resists the disruption of the RT bias. Therefore, although concomitant in most cases, valence-induced motor and confidence biases seem to be partly dissociable. These results highlight new important mechanistic constraints that should be incorporated in learning models to jointly explain choice, reaction times and confidence.

A distributional and dynamic theory of pricing and preference

Theories that describe how people assign prices and make choices are typically based on the idea that both of these responses are derived from a common static, deterministic function used to assign utilities to options. However, preference reversals-where prices assigned to gambles conflict with preference orders elicited through binary choices-indicate that the response processes underlying these different methods of evaluation are more intricate. We address this issue by formulating a new computational model that assumes an initial bias or anchor that depends on type of price task (buying, selling, or certainty equivalents) and a stochastic evaluation accumulation process that depends on gamble attributes. To test this new model, we investigated choices and prices for a wide range of gambles and price tasks, including pricing under time pressure. In line with model predictions, we found that price distributions possessed stark skew that depended on the type of price and the attributes of gambles being considered. Prices were also sensitive to time pressure, indicating a dynamic evaluation process underlying price generation. The model out-performed prospect theory in predicting prices and additionally predicted the response times associated with these prices, which no prior model has accomplished. Finally, we show that the model successfully predicts out-of-sample choices and that its parameters allow us to fit choice response times as well. This price accumulation model therefore provides a superior account of the distributional and dynamic properties of price, leveraging process-level mechanisms to provide a more complete account of the valuation processes common across multiple methods of eliciting preference. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Disentangling the origins of confidence in speeded perceptual judgments through multimodal imaging

The human capacity to compute the likelihood that a decision is correct-known as metacognition-has proven difficult to study in isolation as it usually cooccurs with decision making. Here, we isolated postdecisional from decisional contributions to metacognition by analyzing neural correlates of confidence with multimodal imaging. Healthy volunteers reported their confidence in the accuracy of decisions they made or decisions they observed. We found better metacognitive performance for committed vs. observed decisions, indicating that committing to a decision may improve confidence. Relying on concurrent electroencephalography and hemodynamic recordings, we found a common correlate of confidence following committed and observed decisions in the inferior frontal gyrus and a dissociation in the anterior prefrontal cortex and anterior insula. We discuss these results in light of decisional and postdecisional accounts of confidence and propose a computational model of confidence in which metacognitive performance naturally improves when evidence accumulation is constrained upon committing a decision.

Subjective value then confidence in human ventromedial prefrontal cortex

Two fundamental goals of decision making are to select actions that maximize rewards while minimizing costs and to have strong confidence in the accuracy of a judgment. Neural signatures of these two forms of value: the subjective value (SV) of choice alternatives and the value of the judgment (confidence), have both been observed in ventromedial prefrontal cortex (vmPFC). However, the relationship between these dual value signals and their relative time courses are unknown. Twenty-eight men and women underwent fMRI while performing a two-phase approach-avoidance (Ap-Av) task with mixed-outcomes of monetary rewards paired with painful shock stimuli. Neural responses were measured during offer valuation (offer phase) and choice valuation (commit phase) and analyzed with respect to observed decision outcomes, model-estimated SV and confidence. During the offer phase, vmPFC tracked SV and the decision but not confidence. During the commit phase, vmPFC tracked confidence, computed as the quadratic extension of SV, but not the offer valuation nor the decision. In fact, vmPFC responses from the commit phase were selective for confidence even for reject decisions wherein confidence and SV are inversely related. Conversely, activation of the cognitive control network, including within lateral prefrontal cortex (lPFC) and dorsal anterior cingulate cortex (dACC) was associated with ambivalence, during both the offer and commit phases. Taken together, our results reveal complementary representations in vmPFC during value-based decision making that temporally dissociate such that offer valuation (SV) emerges before decision valuation (confidence).

Mnemonic Introspection in Macaques Is Dependent on Superior Dorsolateral Prefrontal Cortex But Not Orbitofrontal Cortex

The human PFC has been associated more with meta-perceptual as opposed to meta-memory decisions from correlational neuroimaging investigations. Recently, metacognitive abilities have also been shown to be causally dependent upon anterior and dorsal PFC in nonhuman primate lesion studies. Two studies, using postdecision wagering paradigms and reversible inactivation, challenged this meta-perceptual versus meta-memory notion and showed that dorsal and anterior prefrontal areas are associated with metamemory for experienced objects and awareness of ignorance, respectively. Causal investigations are important but scarce; nothing is known, for example, about the causal contributions of prefrontal subregions to spatial metamemory. Here, we investigated the effects of dorsal versus ventral PFC lesions on two-alternative forced-choice spatial discrimination tasks in male macaque monkeys. Importantly, we were rigorous in approach and applied three independent but complementary indices used to quantify individual animals' metacognitive ability (“Type II sensitivity”) by two variants of meta-d′/d′ and phi coefficient (φ). Our results were consistent across indices: while neither lesions to superior dorsolateral PFC nor orbitofrontal cortex impaired spatial recognition performance, only monkeys with superior dorsolateral PFC lesions were impaired in meta-accuracy. Together with the observation that the same orbitofrontal cortex lesioned monkeys were impaired in updating rule value in a Wisconsin Card Sorting Test analog, we therefore document a functional double-dissociation between these two PFC regions. Our study presents important causal evidence that other dimensions, namely, domain-specific processing (e.g., spatial vs nonspatial metamemory), also need considerations in understanding the functional specialization in the neural underpinnings of introspection.

SIGNIFICANCE STATEMENT This study demonstrates macaque monkeys' metacognitive capability of introspecting its own memory success is causally dependent on intact superior dorsolateral prefrontal cortices but not the orbitofrontal cortices. Combining neurosurgical techniques on monkeys and state-of-the-art measures of metacognition, we affirm a critical role of the PFC in supporting spatial meta-recognition memory and delineate functional specificity within PFC for distinct elements of metacognition.

Contextual influence on confidence judgments in human reinforcement learning

The ability to correctly estimate the probability of one’s choices being correct is fundamental to optimally re-evaluate previous choices or to arbitrate between different decision strategies. Experimental evidence nonetheless suggests that this metacognitive process—confidence judgment- is susceptible to numerous biases. Here, we investigate the effect of outcome valence (gains or losses) on confidence while participants learned stimulus-outcome associations by trial-and-error. In two experiments, participants were more confident in their choices when learning to seek gains compared to avoiding losses, despite equal difficulty and performance between those two contexts. Computational modelling revealed that this bias is driven by the context-value, a dynamically updated estimate of the average expected-value of choice options, necessary to explain equal performance in the gain and loss domain. The biasing effect of context-value on confidence, revealed here for the first time in a reinforcement-learning context, is therefore domain-general, with likely important functional consequences. We show that one such consequence emerges in volatile environments, where the (in)flexibility of individuals’ learning strategies differs when outcomes are framed as gains or losses. Despite apparent similar behavior- profound asymmetries might therefore exist between learning to avoid losses and learning to seek gains.

In order to arbitrate between different decision strategies, as well as to inform future choices, a decision maker needs to estimate the probability of her choices being correct as precisely as possible. Surprisingly, this metacognitive operation, known as confidence judgment, has not been systematically investigated in the context of simple instrumental-learning tasks. Here, we assessed how confident individuals are in their choices when learning stimulus-outcome associations by trial-and-errors to maximize gains or to minimize losses. In two experiments, we show that individuals are more confident in their choices when learning to seek gains compared to avoiding losses, despite equal difficulty and performance between those two contexts. To simultaneously account for this pattern of choices and confidence judgments, we propose that individuals learn context-values, which approximate the average expected-value of choice options. We finally show that, in volatile environments, the biasing effect of context-value on confidence induces difference in learning flexibility when outcomes are framed as gains or losses.

Confidence resets reveal hierarchical adaptive learning in humans

Hierarchical processing is pervasive in the brain, but its computational significance for learning under uncertainty is disputed. On the one hand, hierarchical models provide an optimal framework and are becoming increasingly popular to study cognition. On the other hand, non-hierarchical (flat) models remain influential and can learn efficiently, even in uncertain and changing environments. Here, we show that previously proposed hallmarks of hierarchical learning, which relied on reports of learned quantities or choices in simple experiments, are insufficient to categorically distinguish hierarchical from flat models. Instead, we present a novel test which leverages a more complex task, whose hierarchical structure allows generalization between different statistics tracked in parallel. We use reports of confidence to quantitatively and qualitatively arbitrate between the two accounts of learning. Our results support the hierarchical learning framework, and demonstrate how confidence can be a useful metric in learning theory.

The Virtual-Environment-Foraging Task enables rapid training and single-trial metrics of rule acquisition and reversal in head-fixed mice

Behavioural flexibility is an essential survival skill, yet our understanding of its neuronal substrates is still limited. While mouse research offers unique tools to dissect the neuronal circuits involved, the measurement of flexible behaviour in mice often suffers from long training times, poor experimental control, and temporally imprecise binary (hit/miss) performance readouts. Here we present a virtual-environment task for mice that tackles these limitations. It offers fast training of vision-based rule reversals (~100 trials per reversal) with full stimulus control and continuous behavioural readouts. By generating multiple non-binary performance metrics per trial, it provides single-trial estimates not only of response accuracy and speed, but also of underlying processes like choice certainty and alertness (discussed in detail in a companion paper). Based on these metrics, we show that mice can predict new task rules long before they are able to execute them, and that this delay varies across animals. We also provide and validate single-trial estimates of whether an error was committed with or without awareness of the task rule. By tracking in unprecedented detail the cognitive dynamics underlying flexible behaviour, this task enables new investigations into the neuronal interactions that shape behavioural flexibility moment by moment.

Decision and navigation in mouse parietal cortex

Posterior parietal cortex (PPC) has been implicated in navigation, in the control of movement, and in visually-guided decisions. To relate these views, we measured activity in PPC while mice performed a virtual navigation task driven by visual decisions. PPC neurons were selective for specific combinations of the animal's spatial position and heading angle. This selectivity closely predicted both the activity of individual PPC neurons, and the arrangement of their collective firing patterns in choice-selective sequences. These sequences reflected PPC encoding of the animal's navigation trajectory. Using decision as a predictor instead of heading yielded worse fits, and using it in addition to heading only slightly improved the fits. Alternative models based on visual or motor variables were inferior. We conclude that when mice use vision to choose their trajectories, a large fraction of parietal cortex activity can be predicted from simple attributes such as spatial position and heading.

Human VMPFC encodes early signatures of confidence in perceptual decisions

Choice confidence, an individual’s internal estimate of judgment accuracy, plays a critical role in adaptive behaviour, yet its neural representations during decision formation remain underexplored. Here, we recorded simultaneous EEG-fMRI while participants performed a direction discrimination task and rated their confidence on each trial. Using multivariate single-trial discriminant analysis of the EEG, we identified a stimulus-independent component encoding confidence, which appeared prior to subjects’ explicit choice and confidence report, and was consistent with a confidence measure predicted by an accumulation-to-bound model of decision-making. Importantly, trial-to-trial variability in this electrophysiologically-derived confidence signal was uniquely associated with fMRI responses in the ventromedial prefrontal cortex (VMPFC), a region not typically associated with confidence for perceptual decisions. Furthermore, activity in the VMPFC was functionally coupled with regions of the frontal cortex linked to perceptual decision-making and metacognition. Our results suggest that the VMPFC holds an early confidence representation arising from decision dynamics, preceding and potentially informing metacognitive evaluation.

While waiting to cross the road on a foggy morning, you see a shape in the distance that appears to be an approaching car. How do you decide if it is safe to cross? We often have to make important decisions about the world based on imperfect information. What guides our subsequent actions in these situations is a sense of accuracy, or confidence, that we associate with our initial judgments. You would not step off the kerb if you were only 10% confident the car was a safe distance away. But how, when, and where in the brain does such confidence emerge?

Gherman and Philiastides examined how brain activity relates to confidence during the early stages of decision-making, that is, before people have explicitly committed to a particular choice. Healthy volunteers were asked to judge the direction in which dots were moving across a screen. They then had to rate how confident they were in their decision. Two techniques – EEG and fMRI – tracked their brain activity during the task. EEG uses scalp electrodes to reveal when and how electrical activity is changing inside the brain, while fMRI, a type of brain scan, shows where these changes in brain activity occur. Used together, the two techniques provide a greater understanding of brain activity than either used alone.

Activity in multiple regions of the brain correlated with confidence at different stages of the task. Certain brain networks showed confidence-related activity while the volunteers tried to judge the direction of movement, and others were engaged when volunteers made their confidence ratings. However, activity in only one area reliably indicated how confident the volunteers felt before they had made their choice. This area, the ventromedial prefrontal cortex, also helps process rewards. This suggests that feelings of confidence early in the decision-making process could guide our behaviour by virtue of being rewarding.

Many brain disorders – including depression, schizophrenia and Parkinson's disease – compromise decision-making. Patients show changes in accuracy, response times, and in their ability to accurately evaluate their decisions. The methods used in the current study could help reveal the neural changes that cause these impairments. This could lead to new methods to diagnose and predict cognitive deficits, and new ways to treat them at an earlier stage.