Neuronal variability in orbitofrontal cortex during economic decisions

Neuronal Activity in the Gustatory Cortex during Economic Choice

An economic choice entails computing and comparing the values of individual offers. Offer values are represented in the orbitofrontal cortex (OFC)-an area that participates in value comparison-but it is unknown where offer values are computed in the first place. One possibility is that this computation takes place in OFC. Alternatively, offer values might be computed upstream of OFC. For choices between edible goods, a primary candidate is the gustatory region of the anterior insula (gustatory cortex, GC). Here we recorded from the GC of male rhesus monkeys choosing between different juice types. As a population, neurons in GC represented the flavor, the quantity, and the subjective value of the juice chosen by the animal. These variables were represented by distinct groups of cells and with different time courses. Specifically, chosen value signals emerged shortly after offer presentation, while neurons encoding the chosen juice and the chosen quantity peaked after juice delivery. Surprisingly, neurons in GC did not represent individual offer values in a systematic way. In a computational sense, the variables encoded in GC follow the process of value comparison. Thus our results argue against the hypothesis that offer values are computed in GC. At the same time, signals representing the subjective value of the expected reward indicate that responses in GC are not purely sensory. Thus neuronal responses in GC appear consummatory in nature.

Orbitofrontal high-gamma reflects spike-dissociable value and decision mechanisms

The orbitofrontal cortex (OFC) plays a crucial role in value-based decision-making. While previous research has focused on spiking activity in OFC neurons, the role of OFC local field potentials (LFPs) in decision-making remains unclear. LFPs are important because they can reflect synaptic and subthreshold activity not directly coupled to spiking, and because they are potential targets for less invasive forms of brain-machine interface (BMI). We recorded LFPs and spiking activity using multi-channel vertical probes while monkeys performed a two-option value-based decision-making task. We compared the value- and decision-coding properties of high-gamma range LFPs (HG, 50-150 Hz) to the coding properties of spiking multi-unit activity (MUA) recorded concurrently on the same electrodes. Results show that HG and MUA both represent the values of decision targets, and that their representations have similar temporal profiles in a trial. However, we also identified value-coding properties of HG that were dissociable from the concurrently-measured MUA. On average across channels, HG amplitude increased monotonically with value, whereas the average value encoding in MUA was net neutral. HG also encoded a signal consistent with a comparison between the values of the two targets, a signal which was much weaker in MUA. In individual channels, HG was better able to predict choice outcomes than MUA; however, when simultaneously recorded channels were combined in population-based decoder, MUA provided more accurate predictions than HG. Interestingly, HG value representations were accentuated in channels in or near shallow cortical layers, suggesting a dissociation between neuronal sources of HG and MUA. In summary, we find that HG signals are dissociable from MUA with respect to cognitive variables encoded in prefrontal cortex, evident in the monotonic encoding of value, stronger encoding of value comparisons, and more accurate predictions about behavior. High-frequency LFPs may therefore be a viable - or even preferable - target for BMIs to assist cognitive function, opening the possibility for less invasive access to mental contents that would otherwise be observable only with spike-based measures.

Behavioral read-out from population value signals in primate orbitofrontal cortex

The primate orbitofrontal cortex (OFC) has long been recognized for its role in value-based decisions; however, the exact mechanism linking value representations in the OFC to decision outcomes has remained elusive. Here, to address this question, we show, in non-human primates, that trial-wise variability in choices can be explained by variability in value signals decoded from many simultaneously recorded OFC neurons. Mechanistically, this relationship is consistent with the projection of activity within a low-dimensional value-encoding subspace onto a potentially higher-dimensional, behaviorally potent output subspace. Identifying this neural-behavioral link answers longstanding questions about the role of the OFC in economic decision-making and suggests population-level read-out mechanisms for the OFC similar to those recently identified in sensory and motor cortex.

Action-rule-based cognitive control enables efficient execution of stimulus-response conflict tasks: a model validation of Simon task performance

Introduction

The human brain can flexibly modify behavioral rules to optimize task performance (speed and accuracy) by minimizing cognitive load. To show this flexibility, we propose an action-rule-based cognitive control (ARC) model. The ARC model was based on a stochastic framework consistent with an active inference of the free energy principle, combined with schematic brain network systems regulated by the dorsal anterior cingulate cortex (dACC), to develop several hypotheses for demonstrating the validity of the ARC model.

Methods

A step-motion Simon task was developed involving congruence or incongruence between important symbolic information (illustration of a foot labeled "L" or "R," where "L" requests left and "R" requests right foot movement) and irrelevant spatial information (whether the illustration is actually of a left or right foot). We made predictions for behavioral and brain responses to testify to the theoretical predictions.

Results

Task responses combined with event-related deep-brain activity (ER-DBA) measures demonstrated a key contribution of the dACC in this process and provided evidence for the main prediction that the dACC could reduce the Shannon surprise term in the free energy formula by internally reversing the irrelevant rapid anticipatory postural adaptation. We also found sequential effects with modulated dip depths of ER-DBA waveforms that support the prediction that repeated stimuli with the same congruency can promote remodeling of the internal model through the information gain term while counterbalancing the surprise term.

Discussion

Overall, our results were consistent with experimental predictions, which may support the validity of the ARC model. The sequential effect accompanied by dip modulation of ER-DBA waveforms suggests that cognitive cost is saved while maintaining cognitive performance in accordance with the framework of the ARC based on 1-bit congruency-dependent selective control.

Logistic analysis of choice data: A primer

Logistic regressions were developed in economics to model individual choice behavior. In recent years, they have become an important tool in decision neuroscience. Here, I describe and discuss different logistic models, emphasizing the underlying assumptions and possible interpretations. Logistic models may be used to quantify a variety of behavioral traits, including the relative subjective value of different goods, the choice accuracy, risk attitudes, and choice biases. More complex logistic models can be used for choices between good bundles, in cases of nonlinear value functions, and for choices between multiple options. Finally, logistic models can quantify the explanatory power of neuronal activity on choices, thus providing a valid alternative to receiver operating characteristic (ROC) analyses.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Values encoded in orbitofrontal cortex are causally related to economic choices

In the eighteenth century, Daniel Bernoulli, Adam Smith and Jeremy Bentham proposed that economic choices rely on the computation and comparison of subjective values1. This hypothesis continues to inform modern economic theory2 and research in behavioural economics3, but behavioural measures are ultimately not sufficient to verify the proposal4. Consistent with the hypothesis, when agents make choices, neurons in the orbitofrontal cortex (OFC) encode the subjective value of offered and chosen goods5. Value-encoding cells integrate multiple dimensions6-9, variability in the activity of each cell group correlates with variability in choices10,11 and the population dynamics suggests the formation of a decision12. However, it is unclear whether these neural processes are causally related to choices. More generally, the evidence linking economic choices to value signals in the brain13-15 remains correlational16. Here we show that neuronal activity in the OFC is causal to economic choices. We conducted two experiments using electrical stimulation in rhesus monkeys (Macaca mulatta). Low-current stimulation increased the subjective value of individual offers and thus predictably biased choices. Conversely, high-current stimulation disrupted both the computation and the comparison of subjective values, and thus increased choice variability. These results demonstrate a causal chain linking subjective values encoded in OFC to valuation and choice.

Overt Attention toward Appetitive Cues Enhances Their Subjective Value, Independent of Orbitofrontal Cortex Activity

Neural representations of value underlie many behaviors that are crucial for survival. Previously, we found that value representations in primate orbitofrontal cortex (OFC) are modulated by attention, specifically, by overt shifts of gaze toward or away from reward-associated visual cues (McGinty et al., 2016). Here, we investigate the influence of overt attention on behavior by asking how gaze shifts correlate with reward anticipatory responses and whether activity in OFC mediates this correlation. Macaque monkeys viewed pavlovian conditioned appetitive cues on a visual display, while the fraction of time they spent looking toward or away from the cues was measured using an eye tracker. Also measured during cue presentation were the reward anticipation, indicated by conditioned licking responses (CRs), and single-neuron activity in OFC. In general, gaze allocation predicted subsequent licking responses: the longer the monkeys spent looking at a cue at a given time point in a trial, the more likely they were to produce an anticipatory CR later in that trial, as if the subjective value of the cue were increased. To address neural mechanisms, mediation analysis measured the extent to which the gaze–CR correlation could be statistically explained by the concurrently recorded firing of OFC neurons. The resulting mediation effects were indistinguishable from chance. Therefore, while overt attention may increase the subjective value of reward-associated cues (as revealed by anticipatory behaviors), the underlying mechanism remains unknown, as does the functional significance of gaze-driven modulation of OFC value signals.

Partial Adaptation to the Value Range in the Macaque Orbitofrontal Cortex

Values available for choice in different behavioral contexts can vary immensely. To compensate for this variability, neuronal circuits underlying economic decisions undergo adaptation. In orbitofrontal cortex (OFC), neurons encode the subjective value of offered and chosen goods in a quasilinear way. Previous experiments found that the gain of the encoding is lower when the value range is wider. However, the parameters OFC neurons adapted to remained unclear. Furthermore, previous studies did not examine additive changes in neuronal responses. Computational considerations indicate that these factors can directly impact choice behavior. Here we investigated how OFC neurons adapt to changes in the value range. We recorded from two male rhesus monkeys during a juice choice task. Each session was divided into two blocks of trials. In each block, juices were offered within a set range of values, and ranges changed between blocks. Across blocks, neuronal responses adapted to both the maximum and the minimum value, but only partially. As a result, the minimum neural activity was elevated in some value ranges relative to others. Through simulation of a linear decision model, we showed that increasing the minimum response increases choice variability, lowering the expected payoff. This effect is modulated by the balance between cells with positive and negative encoding. The presence of these two populations induces a non-monotonic relationship between the value range and choice efficacy, such that the expected payoff is highest for decisions in an intermediate value range.SIGNIFICANCE STATEMENT Economic decisions are thought to rely on the orbitofrontal cortex (OFC). The values available for choice vary enormously in different contexts. Previous work showed that neurons in OFC encode values in a linear way, and that the gain of encoding is inversely related to the range of available values. However, the specific parameters driving adaptation remained unclear. Here we show that OFC neurons adapt to both the maximum and minimum value in the current context. However, adaptation is partial, leading to contextual changes in the response offset. Interestingly, increasing the activity offset negatively affects choices in a simulated network. Partial adaptation may allow the circuit to maintain information about context value at the cost of slightly reduced payoff.

Sucrose intensity coding and decision-making in rat gustatory cortices

Sucrose's sweet intensity is one attribute contributing to the overconsumption of high-energy palatable foods. However, it is not known how sucrose intensity is encoded and used to make perceptual decisions by neurons in taste-sensitive cortices. We trained rats in a sucrose intensity discrimination task and found that sucrose evoked a widespread response in neurons recorded in posterior-Insula (pIC), anterior-Insula (aIC), and Orbitofrontal cortex (OFC). Remarkably, only a few Intensity-selective neurons conveyed the most information about sucrose's intensity, indicating that for sweetness the gustatory system uses a compact and distributed code. Sucrose intensity was encoded in both firing-rates and spike-timing. The pIC, aIC, and OFC neurons tracked movement direction, with OFC neurons yielding the most robust response. aIC and OFC neurons encoded the subject's choices, whereas all three regions tracked reward omission. Overall, these multimodal areas provide a neural representation of perceived sucrose intensity, and of task-related information underlying perceptual decision-making.

Covert shift of attention modulates the value encoding in the orbitofrontal cortex

During value-based decision making, we often evaluate the value of each option sequentially by shifting our attention, even when the options are presented simultaneously. The orbitofrontal cortex (OFC) has been suggested to encode value during value-based decision making. Yet it is not known how its activity is modulated by attention shifts. We investigated this question by employing a passive viewing task that allowed us to disentangle effects of attention, value, choice and eye movement. We found that the attention modulated OFC activity through a winner-take-all mechanism. When we attracted the monkeys’ attention covertly, the OFC neuronal activity reflected the reward value of the newly attended cue. The shift of attention could be explained by a normalization model. Our results strongly argue for the hypothesis that the OFC neuronal activity represents the value of the attended item. They provide important insights toward understanding the OFC’s role in value-based decision making.

Orbitofrontal Cortex: A Neural Circuit for Economic Decisions

Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.

Optimal coding and neuronal adaptation in economic decisions

During economic decisions, offer value cells in orbitofrontal cortex (OFC) encode the values of offered goods. Furthermore, their tuning functions adapt to the range of values available in any given context. A fundamental and open question is whether range adaptation is behaviorally advantageous. Here we present a theory of optimal coding for economic decisions. We propose that the representation of offer values is optimal if it ensures maximal expected payoff. In this framework, we examine offer value cells in non-human primates. We show that their responses are quasi-linear even when optimal tuning functions are highly non-linear. Most importantly, we demonstrate that for linear tuning functions range adaptation maximizes the expected payoff. Thus value coding in OFC is functionally rigid (linear tuning) but parametrically plastic (range adaptation with optimal gain). Importantly, the benefit of range adaptation outweighs the cost of functional rigidity. While generally suboptimal, linear tuning may facilitate transitive choices.

Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty

We make choices based on the values of expected outcomes, informed by previous experience in similar settings. When the outcomes of our decisions consistently violate expectations, new learning is needed to maximize rewards. Yet not every surprising event indicates a meaningful change in the environment. Even when conditions are stable overall, outcomes of a single experience can still be unpredictable due to small fluctuations (i.e., expected uncertainty) in reward or costs. In the present work, we investigate causal contributions of the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) in rats to learning under expected outcome uncertainty in a novel delay-based task that incorporates both predictable fluctuations and directional shifts in outcome values. We demonstrate that OFC is required to accurately represent the distribution of wait times to stabilize choice preferences despite trial-by-trial fluctuations in outcomes, whereas BLA is necessary for the facilitation of learning in response to surprising events.

DOI:http://dx.doi.org/10.7554/eLife.27483.001

Nobody likes waiting – we opt for online shopping to avoid standing in lines, grow impatient in traffic, and often prefer restaurants that serve food quickly. When making decisions, humans and other animals try to maximize the benefits by weighing up the costs and rewards associated with a situation. Many regions in the brain help us choose the best options based on quality and size of rewards, and required waiting times. Even before we make decisions, the activity in these brain regions predicts what we will choose.

Sometimes, however, unexpected changes can lead to longer waiting times and our preferences suddenly become less desirable. The brain can detect such changes by comparing the outcomes we anticipate to those we experience. When the outcomes are surprising, specific areas in the brain such as the amygdala and the orbitofrontal cortex help us learn to make better choices. However, as surprising events can occur purely by chance, we need to be able to ignore irrelevant surprises and only learn from meaningful ones. Until now, it was not clear whether the amygdala and orbitofrontal cortex play specific roles in successfully learning under such conditions.

Stolyarova and Izquierdo trained rats to select between two images and rewarded them with sugar pellets after different delays. If rats chose one of these images they received the rewards after a predictable delay that was about 10 seconds, while choosing the other one produced variable delays – sometimes the time intervals were either very short or very long. Then, the waiting times for one of the alternatives changed unexpectedly. Rats with healthy brains quickly learned to choose the option with the shorter waiting time.

Stolyarova and Izquierdo repeated the experiments with rats that had damage in a part of the amygdala. These rats learned more slowly, particularly when the variable option changed for the better. Rats with damage to the orbitofrontal cortex failed to learn at all. Stolyarova and Izquierdo then examined the rats’ behavior during delays. Rats with damage to the orbitofrontal cortex could not distinguish between meaningful and irrelevant surprises and always looked for the food pellet (i.e. anticipated a reward) at the average delay interval.

These findings highlight two brain regions that help us distinguish meaningful surprises from irrelevant ones. A next step will be to examine how the amygdala and orbitofrontal cortex interact during learning and see if changes to the activity of these brain regions may affect responses. Advanced methods to non-invasively manipulate brain activity in humans may help people who find it hard to cope with changes; or individuals suffering from substance use disorders, who often struggle to give up drugs that provide them immediate and predictable rewards.

DOI:http://dx.doi.org/10.7554/eLife.27483.002

Decision Making and Sequential Sampling from Memory

Decisions take time, and as a rule more difficult decisions take more time. But this only raises the question of what consumes the time. For decisions informed by a sequence of samples of evidence, the answer is straightforward: more samples are available with more time. Indeed, the speed and accuracy of such decisions are explained by the accumulation of evidence to a threshold or bound. However, the same framework seems to apply to decisions that are not obviously informed by sequences of evidence samples. Here, we proffer the hypothesis that the sequential character of such tasks involves retrieval of evidence from memory. We explore this hypothesis by focusing on value-based decisions and argue that mnemonic processes can account for regularities in choice and decision time. We speculate on the neural mechanisms that link sampling of evidence from memory to circuits that represent the accumulated evidence bearing on a choice. We propose that memory processes may contribute to a wider class of decisions that conform to the regularities of choice-reaction time predicted by the sequential sampling framework.