Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex

A cognitive process model captures near-optimal confidence-guided waiting in rats

Rational decision-makers invest more time pursuing rewards they are more confident they will eventually receive. A series of studies have therefore used willingness to wait for delayed rewards as a proxy for decision confidence. However, interpretation of waiting behavior is limited because it is unclear how environmental statistics influence optimal waiting, and how sources of internal variability influence subjects' behavior. We trained rats to perform a confidence-guided waiting task, and derived expressions for optimal waiting that make relevant environmental statistics explicit, including travel time incurred traveling from one reward opportunity to another. We found that rats waited longer than fully optimal agents, but that their behavior was closely matched by optimal agents with travel times constrained to match their own. We developed a process model describing the decision to stop waiting as an accumulation to bound process, which allowed us to compare the effects of multiple sources of internal variability on waiting. Surprisingly, although mean wait times grew with confidence, variability did not, inconsistent with scalar invariant timing, and best explained by variability in the stopping bound. Our results describe a tractable process model that can capture the influence of environmental statistics and internal sources of variability on subjects' decision process during confidence-guided waiting.

Localized and global representation of prior value, sensory evidence, and choice in male mouse cerebral cortex

Adaptive behavior requires integrating prior knowledge of action outcomes and sensory evidence for making decisions while maintaining prior knowledge for future actions. As outcome- and sensory-based decisions are often tested separately, it is unclear how these processes are integrated in the brain. In a tone frequency discrimination task with two sound durations and asymmetric reward blocks, we found that neurons in the medial prefrontal cortex of male mice represented the additive combination of prior reward expectations and choices. The sensory inputs and choices were selectively decoded from the auditory cortex irrespective of reward priors and the secondary motor cortex, respectively, suggesting localized computations of task variables are required within single trials. In contrast, all the recorded regions represented prior values that needed to be maintained across trials. We propose localized and global computations of task variables in different time scales in the cerebral cortex.

Metacognition and Confidence: A Review and Synthesis

Determining the psychological, computational, and neural bases of confidence and uncertainty holds promise for understanding foundational aspects of human metacognition. While a neuroscience of confidence has focused on the mechanisms underpinning subpersonal phenomena such as representations of uncertainty in the visual or motor system, metacognition research has been concerned with personal-level beliefs and knowledge about self-performance. I provide a road map for bridging this divide by focusing on a particular class of confidence computation: propositional confidence in one's own (hypothetical) decisions or actions. Propositional confidence is informed by the observer's models of the world and their cognitive system, which may be more or less accurate-thus explaining why metacognitive judgments are inferential and sometimes diverge from task performance. Disparate findings on the neural basis of uncertainty and performance monitoring are integrated into a common framework, and a new understanding of the locus of action of metacognitive interventions is developed.

Functional alterations of the prefrontal circuit underlying cognitive aging in mice

Executive function is susceptible to aging. How aging impacts the circuit-level computations underlying executive function remains unclear. Using calcium imaging and optogenetic manipulation during memory-guided behavior, we show that working-memory coding and the relevant recurrent connectivity in the mouse medial prefrontal cortex (mPFC) are altered as early as middle age. Population activity in the young adult mPFC exhibits dissociable yet overlapping patterns between tactile and auditory modalities, enabling crossmodal memory coding concurrent with modality-dependent coding. In middle age, however, crossmodal coding remarkably diminishes while modality-dependent coding persists, and both types of coding decay in advanced age. Resting-state functional connectivity, especially among memory-coding neurons, decreases already in middle age, suggesting deteriorated recurrent circuits for memory maintenance. Optogenetic inactivation reveals that the middle-aged mPFC exhibits heightened vulnerability to perturbations. These findings elucidate functional alterations of the prefrontal circuit that unfold in middle age and deteriorate further as a hallmark of cognitive aging.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here we provide evidence that the energy landscape around attractor basins in population neural activity in the prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays to reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Studying the neural representations of uncertainty

The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.

Neural and computational underpinnings of biased confidence in human reinforcement learning

While navigating a fundamentally uncertain world, humans and animals constantly evaluate the probability of their decisions, actions or statements being correct. When explicitly elicited, these confidence estimates typically correlates positively with neural activity in a ventromedial-prefrontal (VMPFC) network and negatively in a dorsolateral and dorsomedial prefrontal network. Here, combining fMRI with a reinforcement-learning paradigm, we leverage the fact that humans are more confident in their choices when seeking gains than avoiding losses to reveal a functional dissociation: whereas the dorsal prefrontal network correlates negatively with a condition-specific confidence signal, the VMPFC network positively encodes task-wide confidence signal incorporating the valence-induced bias. Challenging dominant neuro-computational models, we found that decision-related VMPFC activity better correlates with confidence than with option-values inferred from reinforcement-learning models. Altogether, these results identify the VMPFC as a key node in the neuro-computational architecture that builds global feeling-of-confidence signals from latent decision variables and contextual biases during reinforcement-learning.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here, we provide evidence that the energy landscape around attractor basins in population neural activity in prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays-to-reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Natural statistics support a rational account of confidence biases

Previous work has sought to understand decision confidence as a prediction of the probability that a decision will be correct, leading to debate over whether these predictions are optimal, and whether they rely on the same decision variable as decisions themselves. This work has generally relied on idealized, low-dimensional models, necessitating strong assumptions about the representations over which confidence is computed. To address this, we used deep neural networks to develop a model of decision confidence that operates directly over high-dimensional, naturalistic stimuli. The model accounts for a number of puzzling dissociations between decisions and confidence, reveals a rational explanation of these dissociations in terms of optimization for the statistics of sensory inputs, and makes the surprising prediction that, despite these dissociations, decisions and confidence depend on a common decision variable.

Modality independent or modality specific? Common computations underlie confidence judgements in visual and auditory decisions

The mechanisms that enable humans to evaluate their confidence across a range of different decisions remain poorly understood. To bridge this gap in understanding, we used computational modelling to investigate the processes that underlie confidence judgements for perceptual decisions and the extent to which these computations are the same in the visual and auditory modalities. Participants completed two versions of a categorisation task with visual or auditory stimuli and made confidence judgements about their category decisions. In each modality, we varied both evidence strength, (i.e., the strength of the evidence for a particular category) and sensory uncertainty (i.e., the intensity of the sensory signal). We evaluated several classes of computational models which formalise the mapping of evidence strength and sensory uncertainty to confidence in different ways: 1) unscaled evidence strength models, 2) scaled evidence strength models, and 3) Bayesian models. Our model comparison results showed that across tasks and modalities, participants take evidence strength and sensory uncertainty into account in a way that is consistent with the scaled evidence strength class. Notably, the Bayesian class provided a relatively poor account of the data across modalities, particularly in the more complex categorisation task. Our findings suggest that a common process is used for evaluating confidence in perceptual decisions across domains, but that the parameter settings governing the process are tuned differently in each modality. Overall, our results highlight the impact of sensory uncertainty on confidence and the unity of metacognitive processing across sensory modalities.

Feeling lucky? prospective and retrospective cues for sensorimotor confidence

On a daily basis, humans interact with the outside world using judgments of sensorimotor confidence, constantly evaluating our actions for success. We ask, what sensory and motor-execution cues are used in making these judgements and when are they available? Two sources of temporally distinct information are prospective cues, available prior to the action (e.g., knowledge of motor noise and past performance), and retrospective cues specific to the action itself (e.g., proprioceptive measurements). We investigated the use of these two cues in two tasks, a secondary motor-awareness task and a main task in which participants reached toward a visual target with an unseen hand and then made a continuous judgment of confidence about the success of the reach. Confidence was reported by setting the size of a circle centered on the reach-target location, where a larger circle reflects lower confidence. Points were awarded if the confidence circle enclosed the true endpoint, with fewer points returned for larger circles. This incentivized accurate reaches and attentive reporting to maximize the score. We compared three Bayesian-inference models of sensorimotor confidence based on either prospective cues, retrospective cues, or both sources of information to maximize expected gain (i.e., an ideal-performance model). Our findings showed two distinct strategies: participants either performed as ideal observers, using both prospective and retrospective cues to make the confidence judgment, or relied solely on prospective information, ignoring retrospective cues. Thus, participants can make use of retrospective cues, evidenced by the behavior observed in our motor-awareness task, but these cues are not always included in the computation of sensorimotor confidence.

Late ERP correlates of confidence for auditory categorization of complex sounds

Recent research suggests that confidence judgments relate to the quality of early sensory representations and later modality independent processing stages. It is not known whether the nature of this finding might vary based on task and/or stimulus characteristics (e.g., detection vs. categorization). The present study investigated the neural correlates of confidence using electroencephalography (EEG) in an auditory categorization task. This allowed us to examine whether the early event-related potentials (ERPs) related to confidence in detection also apply to a more complex auditory task. Participants listened to frequency-modulated (FM) tonal stimuli going up or down in pitch. The rate of FM tones ranged from slow to fast, making the stimuli harder or easier to categorize. Tone-locked late posterior positivity (LPP) but not N1 or P2 amplitudes were larger for (correct-only) trials rated with high than low confidence. These results replicated for trials presenting stimuli at individually identified threshold levels (rate of change producing ∼71.7% correct performance). This finding suggests that, in this task, neural correlates of confidence do not vary based on difficulty level. We suggest that the LPP is a task general indication of the confidence for an upcoming judgment in a variety of paradigms.

Arc regulates a second-guessing cognitive bias during naturalistic foraging through effects on discrete behavior modules

Foraging in animals relies on innate decision-making heuristics that can result in suboptimal cognitive biases in some contexts. The mechanisms underlying these biases are not well understood, but likely involve strong genetic effects. To explore this, we studied fasted mice using a naturalistic foraging paradigm and discovered an innate cognitive bias called "second-guessing." This involves repeatedly investigating an empty former food patch instead of consuming available food, which hinders the mice from maximizing feeding benefits. The synaptic plasticity gene Arc is revealed to play a role in this bias, as Arc-deficient mice did not exhibit second-guessing and consumed more food. In addition, unsupervised machine learning decompositions of foraging identified specific behavior sequences, or "modules", that are affected by Arc. These findings highlight the genetic basis of cognitive biases in decision making, show links between behavior modules and cognitive bias, and provide insight into the ethological roles of Arc in naturalistic foraging.

Optimal policy for uncertainty estimation concurrent with decision making

Decision making often depends on vague information that leads to uncertainty, which is a quantity contingent not on choice but on probability distributions of sensory evidence and other cognitive variables. Uncertainty may be computed in parallel and interact with decision making. Here, we adapt the classic random-dot motion direction discrimination task to allow subjects to indicate their uncertainty without having to form a decision first. The subjects' choices and reaction times for perceptual decisions and uncertainty responses are measured, respectively. We then build a value-based model in which decisions are based on optimizing value computed from a drift-diffusion process. The model accounts for key features of subjects' behavior and the variation across the individuals. It explains how the addition of the uncertainty option affects perceptual decision making. Our work establishes a value-based theoretical framework for studying uncertainty and perceptual decisions that can be readily applied in future investigations of the underlying neural mechanism.

Consensus Goals in the Field of Visual Metacognition

Despite the tangible progress in psychological and cognitive sciences over the last several years, these disciplines still trail other more mature sciences in identifying the most important questions that need to be solved. Reaching such consensus could lead to greater synergy across different laboratories, faster progress, and increased focus on solving important problems rather than pursuing isolated, niche efforts. Here, 26 researchers from the field of visual metacognition reached consensus on four long-term and two medium-term common goals. We describe the process that we followed, the goals themselves, and our plans for accomplishing these goals. If this effort proves successful within the next few years, such consensus building around common goals could be adopted more widely in psychological science.

Time-sensitive prefrontal involvement in associating confidence with task performance illustrates metacognitive introspection in monkeys

Metacognition refers to the ability to be aware of one's own cognition. Ample evidence indicates that metacognition in the human primate is highly dissociable from cognition, specialized across domains, and subserved by distinct neural substrates. However, these aspects remain relatively understudied in macaque monkeys. In the present study, we investigated the functionality of macaque metacognition by combining a confidence proxy, hierarchical Bayesian meta-d' computational modelling, and a single-pulse transcranial magnetic stimulation technique. We found that Brodmann area 46d (BA46d) played a critical role in supporting metacognition independent of task performance; we also found that the critical role of this region in meta-calculation was time-sensitive. Additionally, we report that macaque metacognition is highly domain-specific with respect to memory and perception decisions. These findings carry implications for our understanding of metacognitive introspection within the primate lineage.

Partially Overlapping Neural Correlates of Metacognitive Monitoring and Metacognitive Control

Metacognition describes the process of monitoring one's own mental states, often for the purpose of cognitive control. Previous research has investigated how metacognitive signals are generated (metacognitive monitoring), for example, when people (both female/male) judge their confidence in their decisions and memories. Research has also investigated how metacognitive signals are used to influence behavior (metacognitive control), for example, setting a reminder (i.e., cognitive offloading) for something you are not confident you will remember. However, the mapping between metacognitive monitoring and metacognitive control needs further study on a neural level. We used fMRI to investigate a delayed-intentions task with a reminder element, allowing human participants to use their metacognitive insight to engage metacognitive control. Using multivariate pattern analysis, we found that we could separately decode both monitoring and control, and, to a lesser extent, cross-classify between them. Therefore, brain patterns associated with monitoring and control are partially, but not fully, overlapping.SIGNIFICANCE STATEMENT Models of metacognition commonly distinguish between monitoring (how metacognition is formed) and control (how metacognition is used for behavioral regulation). Research into these facets of metacognition has often happened in isolation. Here, we provide a study which directly investigates the mapping between metacognitive monitoring and metacognitive control at a neural level. We applied multivariate pattern analysis to fMRI data from a novel task in which participants separately rated their confidence (metacognitive monitoring) and how much they would like to use a reminder (metacognitive control). We find support for the notion that the two aspects of metacognition overlap partially but not fully. We argue that future research should focus on how different metacognitive signals are selected for control.

Metacognition as a Consequence of Competing Evolutionary Time Scales

Evolution is full of coevolving systems characterized by complex spatio-temporal interactions that lead to intertwined processes of adaptation. Yet, how adaptation across multiple levels of temporal scales and biological complexity is achieved remains unclear. Here, we formalize how evolutionary multi-scale processing underlying adaptation constitutes a form of metacognition flowing from definitions of metaprocessing in machine learning. We show (1) how the evolution of metacognitive systems can be expected when fitness landscapes vary on multiple time scales, and (2) how multiple time scales emerge during coevolutionary processes of sufficiently complex interactions. After defining a metaprocessor as a regulator with local memory, we prove that metacognition is more energetically efficient than purely object-level cognition when selection operates at multiple timescales in evolution. Furthermore, we show that existing modeling approaches to coadaptation and coevolution—here active inference networks, predator–prey interactions, coupled genetic algorithms, and generative adversarial networks—lead to multiple emergent timescales underlying forms of metacognition. Lastly, we show how coarse-grained structures emerge naturally in any resource-limited system, providing sufficient evidence for metacognitive systems to be a prevalent and vital component of (co-)evolution. Therefore, multi-scale processing is a necessary requirement for many evolutionary scenarios, leading to de facto metacognitive evolutionary outcomes.

Conversion of concept-specific decision confidence into integrative introspection in primates

Introspection based on the integration of uncertain evidence is critical for acting upon abstract thinking and imagining future scenarios. However, it is unknown how confidence read-outs from multiple sources of different concepts are integrated, especially considering the relationships among the concepts. In this study, monkeys performed wagering based on an estimation of their performance in a preceding mnemonic decision. We found that the longer the response times for post-decision wagering, the more relieved the impairments having been caused by frontal disruption. This suggests the existence of a time-consuming compensatory metacognitive process. We found posterior inferior parietal lobe (pIPL) as its candidate, which was not coding the wagering per se (i.e., just high bet or low bet), but became more active when monkeys successfully chose the optimal bet option based on mnemonic decision performance. Thereafter, the pIPL prompts dorsal anterior cingulate cortex to carry the chosen wagering option. Our findings suggest a role for the pIPL in metacognitive concept integration.

Logistic analysis of choice data: A primer

Logistic regressions were developed in economics to model individual choice behavior. In recent years, they have become an important tool in decision neuroscience. Here, I describe and discuss different logistic models, emphasizing the underlying assumptions and possible interpretations. Logistic models may be used to quantify a variety of behavioral traits, including the relative subjective value of different goods, the choice accuracy, risk attitudes, and choice biases. More complex logistic models can be used for choices between good bundles, in cases of nonlinear value functions, and for choices between multiple options. Finally, logistic models can quantify the explanatory power of neuronal activity on choices, thus providing a valid alternative to receiver operating characteristic (ROC) analyses.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Apparent sunk cost effect in rational agents

Rational decision makers aim to maximize their gains, but humans and other animals often fail to do so, exhibiting biases and distortions in their choice behavior. In a recent study of economic decisions, humans, mice, and rats were reported to succumb to the sunk cost fallacy, making decisions based on irrecoverable past investments to the detriment of expected future returns. We challenge this interpretation because it is subject to a statistical fallacy, a form of attrition bias, and the observed behavior can be explained without invoking a sunk cost-dependent mechanism. Using a computational model, we illustrate how a rational decision maker with a reward-maximizing decision strategy reproduces the reported behavioral pattern and propose an improved task design to dissociate sunk costs from fluctuations in decision valuation. Similar statistical confounds may be common in analyses of cognitive behaviors, highlighting the need to use causal statistical inference and generative models for interpretation.

Subjective confidence reflects representation of Bayesian probability in cortex

What gives rise to the human sense of confidence? Here, we tested the Bayesian hypothesis that confidence is based on a probability distribution represented in neural population activity. We implemented several computational models of confidence, and tested their predictions using psychophysics and fMRI. Using a generative model-based fMRI decoding approach, we extracted probability distributions from neural population activity in human visual cortex. We found that subjective confidence tracks the shape of the decoded distribution. That is, when sensory evidence was more precise, as indicated by the decoded distribution, observers reported higher levels of confidence. We furthermore found that neural activity in the insula, anterior cingulate, and prefrontal cortex was linked to both the shape of the decoded distribution and reported confidence, in ways consistent with the Bayesian model. Altogether, our findings support recent statistical theories of confidence and suggest that probabilistic information guides the computation of one’s sense of confidence.

Corticostriatal Activity Driving Compulsive Reward Seeking

BACKGROUND

Activation of the mesolimbic dopamine system is positively reinforcing. After repeated activation, some individuals develop compulsive reward-seeking behavior, which is a core symptom of addiction. However, the underlying neural mechanism remains elusive.

METHODS

We trained mice in a seek-take chain, rewarded by optogenetic dopamine neuron self-stimulation. After compulsivity was evaluated, AMPA/NMDA ratio was measured at three distinct corticostriatal pathways confirmed by retrograde labeling and anterograde synaptic connectivity. Fiber photometry method and chemogenetics were used to parse the contribution of orbitofrontal cortex afferents onto the dorsal striatum (DS) during the behavioral task. We established a causal link between DS activity and compulsivity using optogenetic inhibition.

RESULTS

Mice that persevered when seeking was punished exhibited an increased AMPA/NMDA ratio selectively at orbitofrontal cortex to DS synapses. In addition, an activity peak of spiny projection neurons in the DS at the moment of signaled reward availability was detected. Chemogenetic inhibition of orbitofrontal cortex neurons curbed the activity peak and reduced punished reward seeking, as did optogenetic hyperpolarization of spiny projection neurons time-locked to the cue predicting reward availability.

CONCLUSIONS

Our results suggest that compulsive individuals display stronger neuronal activity in the DS during the cue predicting reward availability even when at the risk of punishment, nurturing further compulsive reward seeking.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.

The learning of prospective and retrospective cognitive maps within neural circuits

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.

Adaptive neurons compute confidence in a decision network

Humans and many animals have the ability to assess the confidence of their decisions. However, little is known about the underlying neural substrate and mechanism. In this study we propose a computational model consisting of a group of 'confidence neurons' with adaptation, which are able to assess the confidence of decisions by detecting the slope of ramping activities of decision neurons. The simulated activities of 'confidence neurons' in our simple model capture the typical features of confidence observed in humans and animals experiments. Our results indicate that confidence could be online formed along with the decision formation, and the adaptation properties could be used to monitor the formation of confidence during the decision making.

Rats use memory confidence to guide decisions

Memory enables access to past experiences to guide future behavior. Humans can determine which memories to trust (high confidence) and which to doubt (low confidence). How memory retrieval, memory confidence, and memory-guided decisions are related, however, is not understood. In particular, how confidence in memories is used in decision making is unknown. We developed a spatial memory task in which rats were incentivized to gamble their time: betting more following a correct choice yielded greater reward. Rat behavior reflected memory confidence, with higher temporal bets following correct choices. We applied machine learning to identify a memory decision variable and built a generative model of memories evolving over time that accurately predicted both choices and confidence reports. Our results reveal in rats an ability thought to exist exclusively in primates and introduce a unified model of memory dynamics, retrieval, choice, and confidence.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

Statistical neuroscience in the single trial limit

Individual neurons often produce highly variable responses over nominally identical trials, reflecting a mixture of intrinsic 'noise' and systematic changes in the animal's cognitive and behavioral state. Disentangling these sources of variability is of great scientific interest in its own right, but it is also increasingly inescapable as neuroscientists aspire to study more complex and naturalistic animal behaviors. In these settings, behavioral actions never repeat themselves exactly and may rarely do so even approximately. Thus, new statistical methods that extract reliable features of neural activity using few, if any, repeated trials are needed. Accurate statistical modeling in this severely trial-limited regime is challenging, but still possible if simplifying structure in neural data can be exploited. We review recent works that have identified different forms of simplifying structure - including shared gain modulations across neural subpopulations, temporal smoothness in neural firing rates, and correlations in responses across behavioral conditions - and exploited them to reveal novel insights into the trial-by-trial operation of neural circuits.

Metacognitive resources for adaptive learning⋆

Biological organisms display remarkably flexible behaviours. This is an area of active investigation, in particular in the fields of artificial intelligence, computational and cognitive neuroscience. While inductive biases and broader cognitive functions are undoubtedly important, the ability to monitor and evaluate one's performance or oneself -- metacognition -- strikes as a powerful resource for efficient learning. Often measured as decision confidence in neuroscience and psychology experiments, metacognition appears to reflect a broad range of abstraction levels and downstream behavioural effects. Within this context, the formal investigation of how metacognition interacts with learning processes is a recent endeavour. Of special interest are the neural and computational underpinnings of confidence and reinforcement learning modules. This review discusses a general hierarchy of confidence functions and their neuro-computational relevance for adaptive behaviours. It then introduces novel ways to study the formation and use of meta-representations and nonconscious mental representations related to learning and confidence, and concludes with a discussion on outstanding questions and wider perspectives.

Separable neural signatures of confidence during perceptual decisions

Perceptual confidence is an evaluation of the validity of perceptual decisions. While there is behavioural evidence that confidence evaluation differs from perceptual decision-making, disentangling these two processes remains a challenge at the neural level. Here, we examined the electrical brain activity of human participants in a protracted perceptual decision-making task where observers tend to commit to perceptual decisions early whilst continuing to monitor sensory evidence for evaluating confidence. Premature decision commitments were revealed by patterns of spectral power overlying motor cortex, followed by an attenuation of the neural representation of perceptual decision evidence. A distinct neural representation was associated with the computation of confidence, with sources localised in the superior parietal and orbitofrontal cortices. In agreement with a dissociation between perception and confidence, these neural resources were recruited even after observers committed to their perceptual decisions, and thus delineate an integral neural circuit for evaluating perceptual decision confidence.

Integration of sensory evidence and reward expectation in mouse perceptual decision-making task with various sensory uncertainties

In perceptual decision-making, prior knowledge of action outcomes is essential, especially when sensory inputs are insufficient for proper choices. Signal detection theory (SDT) shows that optimal choice bias depends not only on the prior but also the sensory uncertainty; however, it is unclear how animals integrate sensory inputs with various uncertainties and reward expectations to optimize choices. We developed a tone-frequency discrimination task for head-fixed mice in which we randomly presented either a long or short sound stimulus and biased the choice outcomes. The choice was less accurate and more biased toward the large-reward side in short- than in long-stimulus trials. Analysis with SDT found that mice did not use a separate, optimal choice threshold in different sound durations. Instead, mice updated one threshold for short and long stimuli with a simple reinforcement-learning rule. Our task in head-fixed mice helps understanding how the brain integrates sensory inputs and prior.

Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias

The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.

A cortical circuit mechanism for structural knowledge-based flexible sensorimotor decision-making

Making flexible decisions based on prior knowledge about causal environmental structures is a hallmark of goal-directed cognition in mammalian brains. Although several association brain regions, including the orbitofrontal cortex (OFC), have been implicated, the precise neuronal circuit mechanisms underlying knowledge-based decision-making remain elusive. Here, we established an inference-based auditory categorization task where mice performed within-session flexible stimulus re-categorization by inferring the changing task rules. We constructed a reinforcement learning model to recapitulate the inference-based flexible behavior and quantify the hidden variables associated with task structural knowledge. Combining two-photon population imaging and projection-specific optogenetics, we found that auditory cortex (ACx) neurons encoded the hidden task rule variable, which requires feedback input from the OFC. Silencing OFC-ACx input specifically disrupted re-categorization behavior. Direct imaging from OFC axons in the ACx revealed task state-related feedback signals, supporting the knowledge-based updating mechanism. Our data reveal a cortical circuit mechanism underlying structural knowledge-based flexible decision-making.

Different Effects of Alcohol Exposure on Action and Outcome-Related Orbitofrontal Cortex Activity

Alcohol dependence can result in long-lasting deficits to decision-making and action control. Neurobiological investigations have identified orbitofrontal cortex (OFC) as important for outcome-related contributions to goal-directed actions during decision-making. Prior work has shown that alcohol dependence induces long-lasting changes to OFC function that persist into protracted withdrawal and disrupts goal-directed control over actions. However, it is unclear whether these changes in function alter representation of action and outcome-related neural activity in OFC. Here, we used the well-validated chronic intermittent ethanol (CIE) exposure and withdrawal procedure to model alcohol dependence in mice and performed in vivo extracellular recordings during an instrumental task in which lever-press actions made for a food outcome. We found alcohol dependence disrupted goal-directed action control and increased OFC activity associated with lever-pressing but decreased OFC activity during outcome-related epochs. The ability to decode outcome-related information, but not action information, from OFC activity following CIE exposure was reduced. Hence, chronic alcohol exposure induced a long-lasting disruption to OFC function such that activity associated with actions was enhanced, but OFC activity contributions to outcome-related information was diminished. This has important implications for hypotheses regarding compulsive and habitual phenotypes observed in addiction.

Striatal dopamine mediates hallucination-like perception in mice

Hallucinations, a central symptom of psychotic disorders, are attributed to excessive dopamine in the brain. However, the neural circuit mechanisms by which dopamine produces hallucinations remain elusive, largely because hallucinations have been challenging to study in model organisms. We developed a task to quantify hallucination-like perception in mice. Hallucination-like percepts, defined as high-confidence false detections, increased after hallucination-related manipulations in mice and correlated with self-reported hallucinations in humans. Hallucination-like percepts were preceded by elevated striatal dopamine levels, could be induced by optogenetic stimulation of mesostriatal dopamine neurons, and could be reversed by the antipsychotic drug haloperidol. These findings reveal a causal role for dopamine-dependent striatal circuits in hallucination-like perception and open new avenues to develop circuit-based treatments for psychotic disorders.

Processing in Lateral Orbitofrontal Cortex Is Required to Estimate Subjective Preference during Initial, but Not Established, Economic Choice

The orbitofrontal cortex (OFC) is proposed to be critical to economic decision making. Yet one can inactivate OFC without affecting well-practiced choices. One possible explanation of this lack of effect is that well-practiced decisions are codified into habits or configural-based policies not normally thought to require OFC. Here, we tested this idea by training rats to choose between different pellet pairs across a set of standard offers and then inactivating OFC subregions during choices between novel offers of previously experienced pairs or between novel pairs of previously experienced pellets. Contrary to expectations, controls performed as well on novel as experienced offers yet had difficulty initially estimating their subjective preference on novel pairs, difficulty exacerbated by lateral OFC inactivation. This pattern of results indicates that established economic choice reflects the use of an underlying model or goods space and that lateral OFC is only required for normal behavior when the established framework must incorporate new information.

Serotonergic projections to the orbitofrontal and medial prefrontal cortices differentially modulate waiting for future rewards

Optogenetic activation of serotonergic neurons in the dorsal raphe nucleus (DRN) enhances patience when waiting for future rewards, and this effect is maximized by both high probability and high timing uncertainty of reward. Here, we explored which serotonin projection areas contribute to these effects using optogenetic axon terminal stimulation. We found that serotonin stimulation in the orbitofrontal cortex (OFC) is nearly as effective as that in the DRN for promoting waiting, while in the nucleus accumbens, it does not promote waiting. We also found that serotonin stimulation in the medial prefrontal cortex (mPFC) promotes waiting only when the timing of future rewards is uncertain. Our Bayesian decision model of waiting assumed that the OFC and mPFC calculate the posterior probability of reward delivery separately. These results suggest that serotonin in the mPFC affects evaluation of time committed, while serotonin in the OFC is responsible for overall valuation of delayed rewards.