What are the neural origins of choice variability?

Frontal circuit specialisations for decision making

There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.

Optimal utility and probability functions for agents with finite computational precision

When making economic choices, such as those between goods or gambles, humans act as if their internal representation of the value and probability of a prospect is distorted away from its true value. These distortions give rise to decisions which apparently fail to maximize reward, and preferences that reverse without reason. Why would humans have evolved to encode value and probability in a distorted fashion, in the face of selective pressure for reward-maximizing choices? Here, we show that under the simple assumption that humans make decisions with finite computational precision--in other words, that decisions are irreducibly corrupted by noise--the distortions of value and probability displayed by humans are approximately optimal in that they maximize reward and minimize uncertainty. In two empirical studies, we manipulate factors that change the reward-maximizing form of distortion, and find that in each case, humans adapt optimally to the manipulation. This work suggests an answer to the longstanding question of why humans make "irrational" economic choices.

Choice perseveration in value-based decision making: The impact of inter-trial interval and mood

In a series of decisions, people tend to show choice perseveration, that is, they repeat their choices. This choice perseveration is assumed to emerge due to residual activity from the previous decision. Here, we use a computational model with attractor dynamics to describe this process and to predict how choice perseveration can be modulated. We derive two qualitative predictions: Choice perseveration should decrease under longer (vs. shorter) inter-trial intervals and positive (vs. negative) mood. We test these predictions in a dynamic decision task where we modulate decisions across trials via sequentially manipulated reward options. Our findings replicate our previous study in showing choice perseveration in value-based decision making. Furthermore, choice perseveration decreased with increasing inter-trial interval as predicted by the model. However, we did not find clear evidence supporting mood effects on choice perseveration. We discuss how integrating decision process dynamics by the means of applying the neural attractor model can increase our understanding of the evolution of decision outcomes and therefore complement the psychophysical perspective on decision making.

Human noise blindness drives suboptimal cognitive inference

Humans typically make near-optimal sensorimotor judgements but show systematic biases when making more cognitive judgements. Here we test the hypothesis that, while humans are sensitive to the noise present during early sensory encoding, the “optimality gap” arises because they are blind to noise introduced by later cognitive integration of variable or discordant pieces of information. In six psychophysical experiments, human observers judged the average orientation of an array of contrast gratings. We varied the stimulus contrast (encoding noise) and orientation variability (integration noise) of the array. Participants adapted near-optimally to changes in encoding noise, but, under increased integration noise, displayed a range of suboptimal behaviours: they ignored stimulus base rates, reported excessive confidence in their choices, and refrained from opting out of objectively difficult trials. These overconfident behaviours were captured by a Bayesian model blind to integration noise. Our study provides a computationally grounded explanation of human suboptimal cognitive inference.

Santiago Herce Castañón and colleagues show that people are blind to mental errors that arise when combining multiple pieces of discordant information. This blindness helps explain why cognitive judgements often are suboptimal.

A distributed, hierarchical and recurrent framework for reward-based choice

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.

Response repetition biases in human perceptual decisions are explained by activity decay in competitive attractor models

Animals and humans have a tendency to repeat recent choices, a phenomenon known as choice hysteresis. The mechanism for this choice bias remains unclear. Using an established, biophysically informed model of a competitive attractor network for decision making, we found that decaying tail activity from the previous trial caused choice hysteresis, especially during difficult trials, and accurately predicted human perceptual choices. In the model, choice variability could be directionally altered through amplification or dampening of post-trial activity decay through simulated depolarizing or hyperpolarizing network stimulation. An analogous intervention using transcranial direct current stimulation (tDCS) over left dorsolateral prefrontal cortex (dlPFC) yielded a close match between model predictions and experimental results: net soma depolarizing currents increased choice hysteresis, while hyperpolarizing currents suppressed it. Residual activity in competitive attractor networks within dlPFC may thus give rise to biases in perceptual choices, which can be directionally controlled through non-invasive brain stimulation.

Selective alteration of human value decisions with medial frontal tDCS is predicted by changes in attractor dynamics

During value-based decision making, ventromedial prefrontal cortex (vmPFC) is thought to support choices by tracking the expected gain from different outcomes via a competition-based process. Using a computational neurostimulation approach we asked how perturbing this region might alter this competition and resulting value decisions. We simulated a perturbation of neural dynamics in a biophysically informed model of decision-making through in silico depolarization at the level of neuronal ensembles. Simulated depolarization increased baseline firing rates of pyramidal neurons, which altered their susceptibility to background noise, and thereby increased choice stochasticity. These behavioural predictions were compared to choice behaviour in healthy participants performing similar value decisions during transcranial direct current stimulation (tDCS), a non-invasive brain stimulation technique. We placed the soma depolarizing electrode over medial frontal PFC. In line with model predictions, this intervention resulted in more random choices. By contrast, no such effect was observed when placing the depolarizing electrode over lateral PFC. Using a causal manipulation of ventromedial and lateral prefrontal function, these results provide support for competition-based choice dynamics in human vmPFC, and introduce computational neurostimulation as a mechanistic assay for neurostimulation studies of cognition.

Economic irrationality is optimal during noisy decision making

According to normative theories, reward-maximizing agents should have consistent preferences. Thus, when faced with alternatives A, B, and C, an individual preferring A to B and B to C should prefer A to C. However, it has been widely argued that humans can incur losses by violating this axiom of transitivity, despite strong evolutionary pressure for reward-maximizing choices. Here, adopting a biologically plausible computational framework, we show that intransitive (and thus economically irrational) choices paradoxically improve accuracy (and subsequent economic rewards) when decision formation is corrupted by internal neural noise. Over three experiments, we show that humans accumulate evidence over time using a "selective integration" policy that discards information about alternatives with momentarily lower value. This policy predicts violations of the axiom of transitivity when three equally valued alternatives differ circularly in their number of winning samples. We confirm this prediction in a fourth experiment reporting significant violations of weak stochastic transitivity in human observers. Crucially, we show that relying on selective integration protects choices against "late" noise that otherwise corrupts decision formation beyond the sensory stage. Indeed, we report that individuals with higher late noise relied more strongly on selective integration. These findings suggest that violations of rational choice theory reflect adaptive computations that have evolved in response to irreducible noise during neural information processing.

Capturing the temporal evolution of choice across prefrontal cortex

Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation. Using primate neurophysiology and human magnetoencephalography, we here recover a single-trial index of PFC internal states from multiple simultaneously recorded PFC subregions. This index can explain the origins of neural representations of economic variables in PFC. It describes the relationship between neural dynamics and behaviour in both human and monkey PFC, directly bridging between human neuroimaging data and underlying neuronal activity. Moreover, it reveals a functionally dissociable interaction between orbitofrontal cortex, anterior cingulate cortex and dorsolateral PFC in guiding cost-benefit decisions. We cast our observations in terms of a recurrent neural network model of choice, providing formal links to mechanistic dynamical accounts of decision-making.

DOI:http://dx.doi.org/10.7554/eLife.11945.001

In 1848, a railroad worker named Phineas Gage suffered an accident that was to secure him a place in neuroscience lore. While constructing a new railway line, a mistimed explosion propelled an iron bar into the base of his skull, where it passed behind his left eye before exiting through the top of his head. Gage survived the accident, but those who knew him reported significant changes in his personality and behaviour.

Gage’s ability to make decisions was particularly impaired by his injury. Decision-making involves weighing up the costs and benefits associated with alternative courses of action. It entails looking into the future to decide whether an anticipated reward will justify the effort or expense necessary to obtain it. This process is dependent on a region of the brain called the prefrontal cortex, the area that sustained the most damage in Phineas Gage.

While many studies have shown correlations between activity in particular parts of prefrontal cortex and the outcome of decisions, little is known about how this activity evolves over time as a decision is made. To explore this process, Hunt et al. trained macaque monkeys to choose between pairs of images that were associated with specific rewards (quantities of fruit juice) and costs (either amounts of work or fixed delays).

Electrode recordings revealed changes in prefrontal activity that varied over time as the monkeys deliberated over each pair of images, choosing for example between a large reward after a long delay versus a smaller reward immediately. This activity was consistent with a mathematical model of decision-making, which also explains data from brain imaging experiments in humans. This provides an important link between human data and electrode recordings in animals.

However, some of the patterns of activity observed in both macaques and humans appeared to reflect the speed at which decisions were made, rather than the outcome of the decisions themselves. By extracting information about decision speed on each decision from each region, it was shown that communication between regions of prefrontal cortex changes when choices are between two different amounts of work, as opposed to two different delays. Further experiments are needed to explore this phenomenon and to determine how other brain regions interact with the prefrontal cortex to support the decision-making process.

DOI:http://dx.doi.org/10.7554/eLife.11945.002