Neural mechanisms for integrating prior knowledge and likelihood in value-based probabilistic inference

Belief inference for hierarchical hidden states in spatial navigation

Uncertainty abounds in the real world, and in environments with multiple layers of unobservable hidden states, decision-making requires resolving uncertainties based on mutual inference. Focusing on a spatial navigation problem, we develop a Tiger maze task that involved simultaneously inferring the local hidden state and the global hidden state from probabilistically uncertain observation. We adopt a Bayesian computational approach by proposing a hierarchical inference model. Applying this to human task behaviour, alongside functional magnetic resonance brain imaging, allows us to separate the neural correlates associated with reinforcement and reassessment of belief in hidden states. The imaging results also suggest that different layers of uncertainty differentially involve the basal ganglia and dorsomedial prefrontal cortex, and that the regions responsible are organised along the rostral axis of these areas according to the type of inference and the level of abstraction of the hidden state, i.e. higher-order state inference involves more anterior parts.

Four Streams Within the Prefrontal Cortex: Integrating Structural and Functional Connectivity

Merging functional evidence derived from studies of autism spectrum disorder and attention-deficit/hyperactivity disorder converges in four neural streams of the prefrontal cortex, hence suggesting a model of information processing through four streams: motor through Brodmann area (BA) 8, emotion through BA 9, memory through BA 10, and emotional-related sensory through BA 11. A growing body of functional data has been supporting this model of information processing. Nevertheless, the underlying structural connectivity was only recently unveiled by a population-based high-definition tractography study with data from 1,065 individuals. This update provides a brief overview of recent evidence supporting the anatomofunctional integration of the four streams of the prefrontal cortex and reviews the white matter fiber tracts subserving the four streams.

The parieto-occipital cortex is a candidate neural substrate for the human ability to approximate Bayesian inference

Adaptive decision-making often requires one to infer unobservable states based on incomplete information. Bayesian logic prescribes that individuals should do so by estimating the posterior probability by integrating the prior probability with new information, but the neural basis of this integration is incompletely understood. We record fMRI during a task in which participants infer the posterior probability of a hidden state while we independently modulate the prior probability and likelihood of evidence regarding the state; the task incentivizes participants to make accurate inferences and dissociates expected value from posterior probability. Here we show that activation in a region of left parieto-occipital cortex independently tracks the subjective posterior probability, combining its subcomponents of prior probability and evidence likelihood, and reflecting the individual participants' systematic deviations from objective probabilities. The parieto-occipital cortex is thus a candidate neural substrate for humans' ability to approximate Bayesian inference by integrating prior beliefs with new information.

A Distinct Neural Code Supports Prospection of Future Probabilities During Instrumental Information-Seeking

To make adaptive decisions, we must actively demand information, but relatively little is known about the mechanisms of active information gathering. An open question is how the brain estimates expected information gains (EIG) when comparing the current decision uncertainty with the uncertainty that is expected after gathering information. We examined this question using fMRI in a task in which people placed bids to obtain information in conditions that varied independently by prior decision uncertainty, information diagnosticity, and the penalty for an erroneous choice. Consistent with value of information theory, bids were sensitive to EIG and its components of prior certainty and expected posterior certainty. Expected posterior certainty was decoded above chance from multivoxel activation patterns in the posterior parietal and extrastriate cortices. This representation was independent of instrumental rewards and overlapped with distinct representations of EIG and prior certainty. Thus, posterior parietal and extrastriate cortices are candidates for mediating the prospection of posterior probabilities as a key step to estimate EIG during active information gathering.

Studying the neural representations of uncertainty

The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.

Conceptualisation of Uncertainty in Decision Neuroscience Research: Do We Really Know What Types of Uncertainties The Measured Neural Correlates Relate To?

In the article "What are neural correlates neural correlates of?" published in the journal BioSocieties, Gabriel Abend points out that neuroscientists cannot avoid philosophical questions concerning the conceptualization and operationalization of social-psychological phenomena they deal with at the physiological level. In this article, we build on Abend's thesis and, through a systematic literature review of decision neuroscience studies, test it with the example of the social-psychological phenomenon of uncertainty in decision making. In this paper, we provide an overview of studies that appropriately attempt to conceptualise uncertainty, and then use these studies to analyse papers looking for neural correlates of uncertainty. Based on a systematic review of studies, we investigate what types of uncertainty authors in the field of decision neuroscience address and define, what criteria they use to distinguish between these types, what problems are associated with their conceptualization, and whether the neural correlates of different types of uncertainty can be accurately identified. The paper concludes that, particularly in the economic context, a collaboration between the natural and social sciences works well, and neuroscience studies use economic conceptualizations of uncertainty that are further developed by sophisticated decision tasks. However, the paper also highlights problematic aspects that obscure the understanding of the phenomena under study. These include the lack of criteria for distinguishing between different types of phenomena, the unclear use of the general concept of uncertainty, and the confusion of phenomena or their erroneous synonymous use.

Uncertainty modulates visual maps during noninstrumental information demand

Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards. We show that valence and uncertainty independently modulated parietal neuronal activity, and uncertainty but not reward-related enhancement consistently correlated with behavioral sensitivity. The findings suggest uncertainty-driven and valence-driven information demand depend on partially distinct pathways, with the former being consistently related to parietal responses and the latter depending on additional mechanisms implemented in downstream structures.

Perceptual inference, accuracy, and precision in temporal reproduction in schizophrenia

Accumulating evidence suggests that deficits in perceptual inference account for symptoms of schizophrenia. One manifestation of perceptual inference is the central bias, i.e., the tendency to put emphasis on prior experiences over actual events in perceiving incoming sensory stimuli. Using an interval reproduction task, this study aimed to determine whether patients with schizophrenia show a stronger central bias than participants without schizophrenia. In the interval reproduction task, participants were shown a cross on a screen. The cross was replaced with a Gaussian patch for a predetermined time interval, and participants were required to reproduce the interval duration by pressing and releasing the space key. We manipulated the uncertainty of prior information using different interval distributions. We found no difference in the influence of prior information on interval reproduction between patients and controls. However, patients with SZ showed a stronger central bias than healthy participants in the intermediate interval range (approximately 450 ms to 900 ms). It is possible that the patients in SZ have non-uniform deficits associated with interval range or uncertainty of prior information in perceptual inference. Further, the severity of avolition and alogia was correlated with the strength of central bias in SZ. This study provides some insights into the mechanisms underlying the association between schizophrenic symptoms and perceptual inference.

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Modulating the Activity of vmPFC Regulates Informational Social Conformity: A tDCS Study

Social conformity has been evaluated in many different contexts, ranging from an emotional contagion in psychology, to speculative episodes in economics, to mass protests concerning politics. Previous neuroscience studies suggest that the ventromedial prefrontal cortex (vmPFC) participates in social conformity, especially when it comes to the value integration process, but the specific mechanism of vmPFC is still unclear. In this study, we aimed to identify a direct link between the vmPFC and conformity tendencies by means of transcranial direct current stimulation (tDCS). Conformity tendencies are measured by the probability that participants change their decisions when they observe the majority responses. In our experiment, subjects could make two decisions in each trial, once without social information and once with social information, which allowed us to directly observe the conformity tendency of subjects in different conditions. We found that cathodal stimulation of the vmPFC significantly increased conformity tendency and decreased response time when the initial decision of participants differs from the majority opinion. Based on the experimental results, our study suggests that the vmPFC mainly inhibits and regulates the informational conformity behavior. These findings complement investigations of the neural mechanism of conformity and the role of the vmPFC in the neural circuit behind conformity behavior.

Base rate neglect and neural computations for subjective weight in decision under uncertainty

Base rate neglect, an important bias in estimating probability of uncertain events, describes humans' tendency to underweight base rate (prior) relative to individuating information (likelihood). However, the neural mechanisms that give rise to this bias remain elusive. In this study, subjects chose between uncertain prospects where estimating reward probability was essential. We found that when the variability of prior and likelihood information about reward probability were systematically manipulated, prior variability significantly affected the degree to which subjects underweight the base rate of reward probability. Activity in the orbitofrontal cortex, medial prefrontal cortex, and putamen represented the relative subjective weight that reflected such bias. Further, sensitivity to likelihood relative to prior variability in the putamen correlated with individuals' overall tendency to underweight base rate. These findings suggest that in combining prior and likelihood, relative sensitivity to information variability and subjective-weight computations critically contribute to the individual heterogeneity in base rate neglect.

Believing in dopamine

Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex-basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

Human representation of multimodal distributions as clusters of samples

Behavioral and neuroimaging evidence shows that human decisions are sensitive to the statistical regularities (mean, variance, skewness, etc.) of reward distributions. However, it is unclear what representations human observers form to approximate reward distributions, or probability distributions in general. When the possible values of a probability distribution are numerous, it is cognitively costly and perhaps unrealistic to maintain in mind the probability of each possible value. Here we propose a Clusters of Samples (CoS) representation model: The samples of the to-be-represented distribution are classified into a small number of clusters and only the centroids and relative weights of the clusters are retained for future use. We tested the behavioral relevance of CoS in four experiments. On each trial, human subjects reported the mean and mode of a sequentially presented multimodal distribution of spatial positions or orientations. By varying the global and local features of the distributions, we observed systematic errors in the reported mean and mode. We found that our CoS representation of probability distributions outperformed alternative models in accounting for subjects’ response patterns. The ostensible influence of positive/negative skewness on the over/under estimation of the reported mean, analogous to the “skewness preference” phenomenon in decisions, could be well explained by models based on CoS.

Life is full of uncertainties: An action may yield multiple possible consequences and a percept may imply multiple possible causes. To survive, humans and animals must compensate for the uncertainty in the environment and in their own perceptual and motor systems. However, how humans represent probability distributions to fulfill probabilistic computations for perception and action remains elusive. The number of possible values in a distribution is vast and grows exponentially with the dimension of the distribution. It would be costly, if not impossible, to maintain the probability of each possible value. Here we propose a sparse representation of probability distributions, which can reduce an arbitrary distribution to a small set of coefficients while still keeping important global and local features of the original distribution. Our experiments provide preliminary evidence for the use of such representations in human cognition.

Transcranial stimulation over right inferior frontal gyrus increases the weight given to private information during sequential decision-making

Decision makers often follow other similarly situated people in making decisions, creating a sequential decision-making context. Although rational behavior is often to make the same choice as previous decision makers, which can result in an information cascade, people may assign inappropriately higher weight to their own private information and discount public information about predecessors' choices. Recent findings suggest that overweighting private information may be associated with increased activities in the inferior frontal gyrus (IFG). In the present study, we employed transcranial direct current stimulation (tDCS) and developed a computational model to examine the causal relationship between right IFG (rIFG) and overweighting private information. Specifically, we applied three types of tDCS over rIFG while participants were completing a sequential decision-making task. Our results showed that anodal stimulation significantly increased the weight given to private information and decreased the response time in making a decision when private information conflicted with public information, but cathodal stimulation did not have such impacts. Importantly, the effect of anodal stimulation was significant in some conditions when information conflict or task difficulty reached a threshold that might trigger cognitive control-related processes. Our findings revealed the important role of rIFG in trade-off between considering private and public information during sequential decision-making.

Electrophysiological correlates of integrating choice and response time during sequential decision making

Response times (RTs) can provide valuable information about a person's underlying decision processes. To investigate electrophysiological correlates of integrating choice and RTs, ERPs elicited by belief updating in long response times condition (Long-RTs) were compared with those in short response times condition (Short-RTs). In both kinds of conditions, three fictitious persons were arranged in random order (P1, P2, P3) and predicted uncertain state of world. P3 took a long time in Long-RTs condition. In Short-RTs condition P3 rapidly made decisions. Participants' task was to infer P3's private signal after observing three fictitious persons' same choice and P3's RTs. ERP results revealed that frontal P200 and N200 distinguished between the two conditions. P200 showed a higher amplitude in Short-RTs condition and might represent early stage valuations of task-relevant perceptual information. N200 showed a more negative amplitude in Long-RTs condition and might reflect conflict between participants' prior knowledge about P3's private signal and P3's long RTs. Our study demonstrates that RTs is an indicator of choice and identifies the temporal process of integrating choice and response time during sequential decision making.

Neurons That Update Representations of the Future

A recent article shows that the brain automatically estimates the probabilities of possible future actions before it has even received all the information necessary to decide what to do next.

A Probability Distribution over Latent Causes, in the Orbitofrontal Cortex

The orbitofrontal cortex (OFC) has been implicated in both the representation of "state," in studies of reinforcement learning and decision making, and also in the representation of "schemas," in studies of episodic memory. Both of these cognitive constructs require a similar inference about the underlying situation or "latent cause" that generates our observations at any given time. The statistically optimal solution to this inference problem is to use Bayes' rule to compute a posterior probability distribution over latent causes. To test whether such a posterior probability distribution is represented in the OFC, we tasked human participants with inferring a probability distribution over four possible latent causes, based on their observations. Using fMRI pattern similarity analyses, we found that BOLD activity in the OFC is best explained as representing the (log-transformed) posterior distribution over latent causes. Furthermore, this pattern explained OFC activity better than other task-relevant alternatives, such as the most probable latent cause, the most recent observation, or the uncertainty over latent causes.

SIGNIFICANCE STATEMENT

Our world is governed by hidden (latent) causes that we cannot observe, but which generate the observations we see. A range of high-level cognitive processes require inference of a probability distribution (or "belief distribution") over the possible latent causes that might be generating our current observations. This is true for reinforcement learning and decision making (where the latent cause comprises the true "state" of the task), and for episodic memory (where memories are believed to be organized by the inferred situation or "schema"). Using fMRI, we show that this belief distribution over latent causes is encoded in patterns of brain activity in the orbitofrontal cortex, an area that has been separately implicated in the representations of both states and schemas.

Neural coding of prior expectations in hierarchical intention inference

The ability to infer other people’s intentions is crucial for successful human social interactions. Such inference relies on an adaptive interplay of sensory evidence and prior expectations. Crucially, this interplay would also depend on the type of intention inferred, i.e., on how abstract the intention is. However, what neural mechanisms adjust the interplay of prior and sensory evidence to the abstractness of the intention remains conjecture. We addressed this question in two separate fMRI experiments, which exploited action scenes depicting different types of intentions (Superordinate vs. Basic; Social vs. Non-social), and manipulated both prior and sensory evidence. We found that participants increasingly relied on priors as sensory evidence became scarcer. Activity in the medial prefrontal cortex (mPFC) reflected this interplay between the two sources of information. Moreover, the more abstract the intention to infer (Superordinate > Basic, Social > Non-Social), the greater the modulation of backward connectivity between the mPFC and the temporo-parietal junction (TPJ), resulting in an increased influence of priors over the intention inference. These results suggest a critical role for the fronto-parietal network in adjusting the relative weight of prior and sensory evidence during hierarchical intention inference.

Sub-optimality in motor planning is retained throughout 9 days practice of 2250 trials

Optimality in motor planning, as well as accuracy in motor execution, is required to maximize expected gain under risk. In this study, we tested whether humans are able to update their motor planning. Participants performed a coincident timing task with an asymmetric gain function, in which optimal response timing to gain the highest total score depends on response variability. Their behaviours were then compared using a Bayesian optimal decision model. After 9 days of practicing 2250 trials, the total score increased, and temporal variance decreased. On the other hand, the participants showed consistent risk-seeking or risk-averse behaviour, preserving suboptimal motor planning. These results suggest that a human's computational ability to calculate an optimal motor plan is limited, and it is difficult to improve it through repeated practice with a score feedback.

Adaptable history biases in human perceptual decisions

When making choices under conditions of perceptual uncertainty, past experience can play a vital role. However, it can also lead to biases that worsen decisions. Consistent with previous observations, we found that human choices are influenced by the success or failure of past choices even in a standard two-alternative detection task, where choice history is irrelevant. The typical bias was one that made the subject switch choices after a failure. These choice history biases led to poorer performance and were similar for observers in different countries. They were well captured by a simple logistic regression model that had been previously applied to describe psychophysical performance in mice. Such irrational biases seem at odds with the principles of reinforcement learning, which would predict exquisite adaptability to choice history. We therefore asked whether subjects could adapt their irrational biases following changes in trial order statistics. Adaptability was strong in the direction that confirmed a subject's default biases, but weaker in the opposite direction, so that existing biases could not be eradicated. We conclude that humans can adapt choice history biases, but cannot easily overcome existing biases even if irrational in the current context: adaptation is more sensitive to confirmatory than contradictory statistics.