Neuronal remapping and circuit persistence in economic decisions

A cellular basis for mapping behavioural structure

To flexibly adapt to new situations, our brains must understand the regularities in the world, as well as those in our own patterns of behaviour. A wealth of findings is beginning to reveal the algorithms that we use to map the outside world1-6. However, the biological algorithms that map the complex structured behaviours that we compose to reach our goals remain unknown. Here we reveal a neuronal implementation of an algorithm for mapping abstract behavioural structure and transferring it to new scenarios. We trained mice on many tasks that shared a common structure (organizing a sequence of goals) but differed in the specific goal locations. The mice discovered the underlying task structure, enabling zero-shot inferences on the first trial of new tasks. The activity of most neurons in the medial frontal cortex tiled progress to goal, akin to how place cells map physical space. These 'goal-progress cells' generalized, stretching and compressing their tiling to accommodate different goal distances. By contrast, progress along the overall sequence of goals was not encoded explicitly. Instead, a subset of goal-progress cells was further tuned such that individual neurons fired with a fixed task lag from a particular behavioural step. Together, these cells acted as task-structured memory buffers, implementing an algorithm that instantaneously encoded the entire sequence of future behavioural steps, and whose dynamics automatically computed the appropriate action at each step. These dynamics mirrored the abstract task structure both on-task and during offline sleep. Our findings suggest that schemata of complex behavioural structures can be generated by sculpting progress-to-goal tuning into task-structured buffers of individual behavioural steps.

The representation of decision variables in orbitofrontal cortex is longitudinally stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the choice input and the choice output, suggesting that the cell groups found in the OFC constitute the building blocks of a decision circuit. Here, we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we record from the OFC of mice engaged in a juice-choice task. Imaging of individual cells continues for up to 40 weeks. For each cell and each session pair, we compare activity profiles using cosine similarity, and we assess whether the neuron encodes the same variable in both sessions. We find a high degree of stability and a modest representational drift. Quantitative estimates indicate that this drift would not randomize the circuit within the animal's lifetime.

The Representation of Decision Variables in Orbitofrontal Cortex is Longitudinally Stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the input and the output of the choice process, suggesting that the cell groups found in OFC constitute the building blocks of a decision circuit. Here we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we recorded from mice choosing between different juice flavors. Recordings of individual cells continued for up to 20 weeks. For each cell and each pair of sessions, we compared the activity profiles using cosine similarity, and we assessed whether the cell encoded the same variable in both sessions. These analyses revealed a high degree of stability and a modest representational drift. A quantitative estimate indicated this drift would not randomize the circuit within the animal's lifetime.

The orbitofrontal cortex: A goal-directed cognitive map framework for social and non-social behaviors

The orbitofrontal cortex (OFC) is regarded as one of the core brain areas in a variety of value-based behaviors. Over the past two decades, tremendous knowledge about the OFC function was gained from studying the behaviors of single subjects. As a result, our previous understanding of the OFC's function of encoding decision variables, such as the value and identity of choices, has evolved to the idea that the OFC encodes a more complex representation of the task space as a cognitive map. Accumulating evidence also indicates that the OFC importantly contributes to behaviors in social contexts, especially those involved in cooperative interactions. However, it remains elusive how exactly OFC neurons contribute to social functions and how non-social and social behaviors are related to one another in the computations performed by OFC neurons. In this review, we aim to provide an integrated view of the OFC function across both social and non-social behavioral contexts. We propose that seemingly complex functions of the OFC may be explained by its role in providing a goal-directed cognitive map to guide a wide array of adaptive reward-based behaviors in both social and non-social domains.

Impact of total sleep deprivation and related mood changes on approach-avoidance decisions to threat-related facial displays

STUDY OBJECTIVES

Total sleep deprivation is known to have significant detrimental effects on cognitive and socio-emotional functioning. Nonetheless, the mechanisms by which total sleep loss disturbs decision-making in social contexts are poorly understood. Here, we investigated the impact of total sleep deprivation on approach/avoidance decisions when faced with threatening individuals, as well as the potential moderating role of sleep-related mood changes.

METHODS

Participants (n = 34) made spontaneous approach/avoidance decisions in the presence of task-irrelevant angry or fearful individuals, while rested or totally sleep deprived (27 h of continuous wakefulness). Sleep-related changes in mood and sustained attention were assessed using the Positive and Negative Affective Scale and the psychomotor vigilance task, respectively.

RESULTS

Rested participants avoided both fearful and angry individuals, with stronger avoidance for angry individuals, in line with previous results. On the contrary, totally sleep deprived participants favored neither approach nor avoidance of fearful individuals, while they still comparably avoided angry individuals. Drift-diffusion models showed that this effect was accounted for by the fact that total sleep deprivation reduced value-based evidence accumulation toward avoidance during decision making. Finally, the reduction of positive mood after total sleep deprivation positively correlated with the reduction of fearful display avoidance. Importantly, this correlation was not mediated by a sleep-related reduction in sustained attention.

CONCLUSIONS

All together, these findings support the underestimated role of positive mood-state alterations caused by total sleep loss on approach/avoidance decisions when facing ambiguous socio-emotional displays, such as fear.

The Neural Instantiation of an Abstract Cognitive Map for Economic Choice

Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.

The learning of prospective and retrospective cognitive maps within neural circuits

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

Neural circuits for inference-based decision-making

In novel situations, where direct experience is lacking or outdated, humans must rely on mental simulations to predict future outcomes. This review discusses recent work on the neural circuits that support such inference-based behavior. We focus on two specific examples: 1) using knowledge about the associative structure of the world to infer outcomes when direct experience is lacking; 2) inferring the current value of options when the desirability of the associated outcome has changed since the original learning experience. These two examples can be studied in the sensory preconditioning and devaluation tasks, respectively. We review results from studies in animals and humans suggesting that the orbitofrontal cortex (OFC), together with the hippocampus and amygdala, is necessary for inference in both of these tasks. Together, these findings suggest that the OFC is a critical hub in the brain network that supports inference-based decision-making.

Frontal circuit specialisations for decision making

There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.

To be specific: The role of orbitofrontal cortex in signaling reward identity

The orbitofrontal cortex (OFC) plays a prominent role in signaling reward expectations. Two important features of rewards are their value (how good they are) and their specific identity (what they are). Whereas research on OFC has traditionally focused on reward value, recent findings point toward a pivotal role of reward identity in understanding OFC signaling and its contribution to behavior. Here, we review work in rodents, nonhuman primates, and humans on how the OFC represents expectations about the identity of rewards, and how these signals contribute to outcome-guided behavior. Moreover, we summarize recent findings suggesting that specific reward expectations in OFC are learned and updated by means of identity errors in the dopaminergic midbrain. We conclude by discussing how OFC encoding of specific rewards complements recent proposals that this region represents a cognitive map of relevant task states, which forms the basis for model-based behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure.

Neural Population Dynamics Underlying Expected Value Computation

Computation of expected values (i.e., probability × magnitude) seems to be a dynamic integrative process performed by the brain for efficient economic behavior. However, neural dynamics underlying this computation is largely unknown. Using lottery tasks in monkeys (Macaca mulatta, male; Macaca fuscata, female), we examined (1) whether four core reward-related brain regions detect and integrate probability and magnitude cued by numerical symbols and (2) whether these brain regions have distinct dynamics in the integrative process. Extraction of the mechanistic structure of neural population signals demonstrated that expected value signals simultaneously arose in the central orbitofrontal cortex (cOFC; medial part of area 13) and ventral striatum (VS). Moreover, these signals were incredibly stable compared with weak and/or fluctuating signals in the dorsal striatum and medial OFC. Temporal dynamics of these stable expected value signals were unambiguously distinct: sharp and gradual signal evolutions in the cOFC and VS, respectively. These intimate dynamics suggest that the cOFC and VS compute the expected values with unique time constants, as distinct, partially overlapping processes.

SIGNIFICANCE STATEMENT Our results differ from those of earlier studies suggesting that many reward-related regions in the brain signal probability and/or magnitude and provide a mechanistic structure for expected value computation employed in multiple neural populations. A central part of the orbitofrontal cortex (cOFC) and ventral striatum (VS) can simultaneously detect and integrate probability and magnitude into an expected value. Our empirical study on these neural population dynamics raises a possibility that the cOFC and VS cooperate on this computation with unique time constants as distinct, partially overlapping processes.

Value and choice as separable and stable representations in orbitofrontal cortex

Value-based decision-making requires different variables-including offer value, choice, expected outcome, and recent history-at different times in the decision process. Orbitofrontal cortex (OFC) is implicated in value-based decision-making, but it is unclear how downstream circuits read out complex OFC responses into separate representations of the relevant variables to support distinct functions at specific times. We recorded from single OFC neurons while macaque monkeys made cost-benefit decisions. Using a novel analysis, we find separable neural dimensions that selectively represent the value, choice, and expected reward of the present and previous offers. The representations are generally stable during periods of behavioral relevance, then transition abruptly at key task events and between trials. Applying new statistical methods, we show that the sensitivity, specificity and stability of the representations are greater than expected from the population's low-level features-dimensionality and temporal smoothness-alone. The separability and stability suggest a mechanism-linear summation over static synaptic weights-by which downstream circuits can select for specific variables at specific times.

Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex

Every decision we make is accompanied by a sense of confidence about its likely outcome. This sense informs subsequent behavior, such as investing more-whether time, effort, or money-when reward is more certain. A neural representation of confidence should originate from a statistical computation and predict confidence-guided behavior. An additional requirement for confidence representations to support metacognition is abstraction: they should emerge irrespective of the source of information and inform multiple confidence-guided behaviors. It is unknown whether neural confidence signals meet these criteria. Here, we show that single orbitofrontal cortex neurons in rats encode statistical decision confidence irrespective of the sensory modality, olfactory or auditory, used to make a choice. The activity of these neurons also predicts two confidence-guided behaviors: trial-by-trial time investment and cross-trial choice strategy updating. Orbitofrontal cortex thus represents decision confidence consistent with a metacognitive process that is useful for mediating confidence-guided economic decisions.

Mechanisms of competitive selection: A canonical neural circuit framework

Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.

Decision Making: How Is Information Represented in Orbitofrontal Cortex?

Classically, specific orbitofrontal cortex (OFC) neurons are thought to represent attributes of specific decision options. A new model proposes instead that OFC neurons represent whichever option is currently attended. A recent study, however, tests these two models and rules out the 'current-focus-of-attention' model.

Neuronal Activity in the Primate Amygdala during Economic Choice

Multiple lines of evidence link economic choices to the orbitofrontal cortex (OFC), but other brain regions may contribute to the computation and comparison of economic values. A particularly strong candidate is the basolateral amygdala (BLA). Amygdala lesions impair performance in reinforcer devaluation tasks, suggesting that the BLA contributes to value computation. Furthermore, previous studies of the BLA have found neuronal activity consistent with a value representation. Here, we recorded from the BLA of two male rhesus macaques choosing between different juices. Offered quantities varied from trial to trial, and relative values were inferred from choices. Approximately one-third of BLA cells were task-related. Our analyses revealed the presence of three groups of neurons encoding variables offer value, chosen value, and chosen juice In this respect, the BLA appeared similar to the OFC. The two areas differed for the proportion of neurons in each group, as the fraction of chosen value cells was significantly higher in the BLA. Importantly, the activity of these neurons reflected the subjective nature of value. Firing rates in the BLA were sustained throughout the trial and maximal after juice delivery. In contrast, firing rates in the OFC were phasic and maximal shortly after offer presentation. Our results suggest that the BLA supports economic choice and reward expectation.SIGNIFICANCE STATEMENT Economic choices rely on the orbitofrontal cortex (OFC), but other brain regions may contribute to this behavior. A strong candidate is the basolateral amygdala (BLA). Previous results are consistent with a neuronal representation of value, but the role of the BLA in economic decisions remains unclear. Here, we recorded from monkeys choosing between juices. Neurons in the BLA encoded three decision variables: offer value, chosen value, and chosen juice These variables were also identified in the OFC. The two areas differed in the proportion of cells encoding each variable and in the activation timing. In the OFC, firing rates peaked shortly after offer presentation; in the BLA, firing rates were sustained and peaked after juice delivery. These results suggest that the BLA supports choices and reward expectation.

Combinations of low-level and high-level neural processes account for distinct patterns of context-dependent choice

Context effects have been explained by either low-level neural adjustments or high-level cognitive processes but not their combination. It is currently unclear how these processes interact to shape individuals’ responses to context. Here, we used a large cohort of human subjects in experiments involving choice between two or three gambles in order to study the dependence of context effects on neural adaptation and individuals’ risk attitudes. Our experiments did not provide any evidence that neural adaptation on long timescales (~100 trials) contributes to context effects. Using post-hoc analyses we identified two groups of subjects with distinct patterns of responses to decoys, both of which depended on individuals’ risk aversion. Subjects in the first group exhibited strong, consistent decoy effects and became more risk averse due to decoy presentation. In contrast, subjects in the second group did not show consistent decoy effects and became more risk seeking. The degree of change in risk aversion due to decoy presentation was positively correlated with the original degrees of risk aversion. To explain these results and reveal underlying neural mechanisms, we developed new models incorporating both low- and high-level processes and used these models to fit individuals’ choice behavior. We found that observed distinct patterns of decoy effects can be explained by a combination of adjustments in neural representations and competitive weighting of reward attributes, both of which depend on risk aversion but in opposite directions. Altogether, our results demonstrate how a combination of low- and high-level processes shapes choice behavior in more naturalistic settings, modulates overall risk preference, and explains distinct behavioral phenotypes.

A large body of experimental work has illustrated that the introduction of a new, and often irrelevant, option can influence preference among the existing options, a phenomenon referred to as context or decoy effects. For example, introducing a new option that is worse than one of the two existing options in all its attributes but better than the alternative option in some attributes (and thus should not ever be selected) can increase the preference for the former option. Context effects have been explained by high-level cognitive processes—such as comparisons and competitions between attributes—or low-level adjustments of neural representations. However, it is unclear how these processes interact to shape individuals’ responses to context. Here, we show that both high-level cognitive processes and low-level neural adjustments shift risk preference during choice between multiple risky options but in opposite directions. Moreover, we demonstrate that combinations of these processes can account for distinct patterns of context effects in human subjects.

Categorical encoding of decision variables in orbitofrontal cortex

A fundamental and recurrent question in systems neuroscience is that of assessing what variables are encoded by a given population of neurons. Such assessments are often challenging because neurons in one brain area may encode multiple variables, and because neuronal representations might be categorical or non-categorical. These issues are particularly pertinent to the representation of decision variables in the orbitofrontal cortex (OFC)-an area implicated in economic choices. Here we present a new algorithm to assess whether a neuronal representation is categorical or non-categorical, and to identify the encoded variables if the representation is indeed categorical. The algorithm is based on two clustering procedures, one variable-independent and the other variable-based. The two partitions are then compared through adjusted mutual information. The present algorithm overcomes limitations of previous approaches and is widely applicable. We tested the algorithm on synthetic data and then used it to examine neuronal data recorded in the primate OFC during economic decisions. Confirming previous assessments, we found the neuronal representation in OFC to be categorical in nature. We also found that neurons in this area encode the value of individual offers, the binary choice outcome and the chosen value. In other words, during economic choice, neurons in the primate OFC encode decision variables in a categorical way.

Neuronal evidence for good-based economic decisions under variable action costs

Previous work showed that economic decisions can be made independently of spatial contingencies. However, when goods available for choice bear different action costs, the decision necessarily reflects aspects of the action. One possibility is that "stimulus values" are combined with the corresponding action costs in a motor representation, and decisions are then made in actions space. Alternatively, action costs could be integrated with other determinants of value in a non-spatial representation. If so, decisions under variable action costs could take place in goods space. Here, we recorded from orbitofrontal cortex while monkeys chose between different juices offered in variable amounts. We manipulated action costs by varying the saccade amplitude, and we dissociated in time and space offer presentation from action planning. Neurons encoding the binary choice outcome did so well before the presentation of saccade targets, indicating that decisions were made in goods space.

Tracking Economic Value of Products in Natural Settings: A Wireless EEG Study

Economic decision making refers to the process of individuals translating their preference into subjective value (SV). Little is known about the dynamics of the neural processes that underpin this form of value-based decision making and no studies have investigated these processes outside of controlled laboratory settings. The current study investigated the spatio-temporal dynamics that accompany economic valuation of products using mobile electroencephalography (EEG) and eye tracking techniques. Participants viewed and rated images of household products in a gallery setting while EEG and eye tracking data were collected wirelessly. A Becker-DeGroot-Marschak (BDM) auction task was subsequently used to quantify the individual's willingness to pay (WTP) for each product. WTP was used to classify products into low, low medium, high medium and high economic value conditions. Eye movement related potentials (EMRP) were examined, and independent component analysis (ICA) was used to separate sources of activity from grand averaged EEG data. Four independent components (ICs) of EMRPs were modulated by WTP (i.e., SV) in the latency range of 150-250 ms. Of the four value-sensitive ICs, one IC displayed enhanced amplitude for all value conditions excluding low value, and another IC presented enhanced amplitude for low value products only. The remaining two value-sensitive ICs resolved inter-mediate levels of SV. Our study quantified, for the first time, the neural processes involved in economic value based decisions in a natural setting. Results suggest that multiple spatio-temporal brain activation patterns mediate the attention and aversion of products which could reflect an early valuation system. The EMRP parietal P200 component could reflect an attention allocation mechanism that separates the lowest-value products (IC7) from products of all other value (IC4), suggesting that low-value items are categorized early on as being aversive. While none of the ICs showed linear amplitude changes that parallel SV's of products, results suggest that a combination of multiple components may sub-serve a fine-grained resolution of the SV of products.

A neuronal population code for resemblance between drug and nondrug reward outcomes in the orbitofrontal cortex

The orbitofrontal cortex (OFC) is implicated in choice and decision-making in both human and non-human animals. We previously identified in the rat OFC a mechanism that influences individual drug choices and preferences between a drug and a nondrug (i.e., sweet) outcome that is common across different types of drugs (cocaine and heroin). Importantly, this research also revealed some intriguing drug-specific differences. Notably, the size of non-selective OFC neurons that indiscriminately encode both the drug and the sweet outcomes varies as a function of the drug outcome available (cocaine or heroin). Here we tested the hypothesis that the relative size of the non-selective OFC population somehow represents the degree of resemblance between the drug and nondrug reward outcomes. We recorded OFC neuronal activity in vivo in the same individual rats while they were choosing between two outcomes with varying degrees of resemblance: high (two concentrations of sweet), intermediate (sweet versus heroin) and low (sweet versus cocaine). We found that the percentage of non-selective OFC neurons dramatically increased with the degree of resemblance between choice outcomes, from 26 to 62%. Overall, these findings reveal the existence of a neuronal population code for resemblance between different kinds of choice outcomes in the OFC.

Computing Value from Quality and Quantity in Human Decision-Making

How organisms learn the value of single stimuli through experience is well described. In many decisions, however, value estimates are computed “on the fly” by combining multiple stimulus attributes. The neural basis of this computation is poorly understood. Here we explore a common scenario in which decision-makers must combine information about quality and quantity to determine the best option. Using fMRI, we examined the neural representation of quality, quantity, and their integration into an integrated subjective value signal in humans of both genders. We found that activity within inferior frontal gyrus (IFG) correlated with offer quality, while activity in the intraparietal sulcus (IPS) specifically correlated with offer quantity. Several brain regions, including the anterior cingulate cortex (ACC), were sensitive to an interaction of quality and quantity. However, the ACC was uniquely activated by quality, quantity, and their interaction, suggesting that this region provides a substrate for flexible computation of value from both quality and quantity. Furthermore, ACC signals across subjects correlated with the strength of quality and quantity signals in IFG and IPS, respectively. ACC tracking of subjective value also correlated with choice predictability. Finally, activity in the ACC was elevated for choice trials, suggesting that ACC provides a nexus for the computation of subjective value in multiattribute decision-making.

SIGNIFICANCE STATEMENT Would you prefer three apples or two oranges? Many choices we make each day require us to weigh up the quality and quantity of different outcomes. Using fMRI, we show that option quality is selectively represented in the inferior frontal gyrus, while option quantity correlates with areas of the intraparietal sulcus that have previously been associated with numerical processing. We show that information about the two is integrated into a value signal in the anterior cingulate cortex, and the fidelity of this integration predicts choice predictability. Our results demonstrate how on-the-fly value estimates are computed from multiple attributes in human value-based decision-making.

Medial orbitofrontal inactivation does not affect economic choice

How are decisions made between different goods? One theory spanning several fields of neuroscience proposes that their values are distilled to a single common neural currency, the calculation of which allows for rational decisions. The orbitofrontal cortex (OFC) is thought to play a critical role in this process, based on the presence of neural correlates of economic value in lateral OFC in monkeys and medial OFC in humans. We previously inactivated lateral OFC in rats without affecting economic choice behavior. Here we inactivated medial OFC in the same task, again without effect. Behavior in the same rats was disrupted by inactivation during progressive ratio responding previously shown to depend on medial OFC, demonstrating the efficacy of the inactivation. These results indicate that medial OFC is not necessary for economic choice, bolstering the proposal that classic economic choice is likely mediated by multiple, overlapping neural circuits.

Linking dynamic patterns of neural activity in orbitofrontal cortex with decision making

Humans and animals demonstrate extraordinary flexibility in choice behavior, particularly when deciding based on subjective preferences. We evaluate options on different scales, deliberate, and often change our minds. Little is known about the neural mechanisms that underlie these dynamic aspects of decision-making, although neural activity in orbitofrontal cortex (OFC) likely plays a central role. Recent evidence from studies in macaques shows that attention modulates value responses in OFC, and that ensembles of OFC neurons dynamically signal different options during choices. When contexts change, these ensembles flexibly remap to encode the new task. Determining how these dynamic patterns emerge and relate to choices will inform models of decision-making and OFC function.

Solving the Credit Assignment Problem With the Prefrontal Cortex

In naturalistic multi-cue and multi-step learning tasks, where outcomes of behavior are delayed in time, discovering which choices are responsible for rewards can present a challenge, known as the credit assignment problem. In this review, I summarize recent work that highlighted a critical role for the prefrontal cortex (PFC) in assigning credit where it is due in tasks where only a few of the multitude of cues or choices are relevant to the final outcome of behavior. Collectively, these investigations have provided compelling support for specialized roles of the orbitofrontal (OFC), anterior cingulate (ACC), and dorsolateral prefrontal (dlPFC) cortices in contingent learning. However, recent work has similarly revealed shared contributions and emphasized rich and heterogeneous response properties of neurons in these brain regions. Such functional overlap is not surprising given the complexity of reciprocal projections spanning the PFC. In the concluding section, I overview the evidence suggesting that the OFC, ACC and dlPFC communicate extensively, sharing the information about presented options, executed decisions and received rewards, which enables them to assign credit for outcomes to choices on which they are contingent. This account suggests that lesion or inactivation/inhibition experiments targeting a localized PFC subregion will be insufficient to gain a fine-grained understanding of credit assignment during learning and instead poses refined questions for future research, shifting the focus from focal manipulations to experimental techniques targeting cortico-cortical projections.

Ensembles in medial and lateral orbitofrontal cortex construct cognitive maps emphasizing different features of the behavioral landscape

The orbitofrontal cortex (OFC) has long been implicated in the ability to use the current value of expected outcomes to guide behavior. More recently, this specific role has been conceptualized as a special case of a more general function that OFC plays in constructing a "cognitive map" of the behavioral task space by labeling the current task state and learning relationships among task states. Here, we have used single unit recording data from 2 prior studies to examine whether and how information relating different states within and across trials is represented in medial versus lateral OFC in rats. Using a hierarchical clustering analysis, we examined how neurons from each area represented information about differently valued trial types, defined by the cue-outcome pairings, versus how those same neurons represented information about similar epochs between these different trial types, such as the stimulus sample, delay, and reward consumption epochs. This analysis revealed that ensembles in the lateral OFC (lOFC) group states according to trial epoch, whereas those in the medial OFC (mOFC) organize the same states by trial type. These results suggest that the lOFC and mOFC construct cognitive maps that emphasize different features of the behavioral landscape, with lOFC tracking events based on local similarities, irrespective of their values and mOFC tracking more distal or higher order relationships relevant to value. (PsycINFO Database Record

Outcome Modulation Across Tasks in the Primate Dorsolateral Prefrontal Cortex

Animals need to learn and to adapt to new and changing environments so that appropriate actions that lead to desirable outcomes are acquired within each context. The prefrontal cortex (PF) is known to underlie such function that directly implies that the outcome of each response must be represented in the brain for behavioral policies update. However, whether such PF signal is context dependent or it is a general representation beyond the specificity of a context is still unclear. Here, we analyzed the activity of neurons in the dorsolateral PF (PFdl) recorded while two monkeys performed two perceptual magnitude discrimination tasks. Both tasks were well known by the monkeys and unexpected changes did not occur but the difficulty of the task varied from trial to trial and thus the monkeys made mistakes in a proportion of trials. We show a context-independent coding of the response outcome with neurons maintaining similar selectivity in both task contexts. Using a classification method of the neural activity, we also show that the trial outcome could be well predicted from the activity of the same neurons in the two contexts. Altogether, our results provide evidence of high degree of outcome generality in PFdl.

Innate visual preferences and behavioral flexibility in Drosophila

Visual decision-making in animals is influenced by innate preferences as well as experience. Interaction between hard-wired responses and changing motivational states determines whether a visual stimulus is attractive, aversive, or neutral. It is however difficult to separate the relative contribution of nature versus nurture in experimental paradigms, especially for more complex visual parameters such as the shape of objects. We used a closed-loop virtual reality paradigm for walking Drosophila flies to uncover innate visual preferences for the shape and size of objects, in a recursive choice scenario allowing the flies to reveal their visual preferences over time. We found that Drosophila flies display a robust attraction / repulsion profile for a range of objects sizes in this paradigm, and that this visual preference profile remains evident under a variety of conditions and persists into old age. We also demonstrate a level of flexibility in this behavior: innate repulsion to certain objects could be transiently overridden if these were novel, although this effect was only evident in younger flies. Finally, we show that a neuromodulatory circuit in the fly brain, Drosophila neuropeptide F (dNPF), can be recruited to guide visual decision-making. Optogenetic activation of dNPF-expressing neurons converted a visually repulsive object into a more attractive object. This suggests that dNPF activity in the Drosophila brain guides ongoing visual choices, to override innate preferences and thereby provide a necessary level of behavioral flexibility in visual decision-making.

Short-term reward experience biases inference despite dissociable neural correlates

Optimal decision-making employs short-term rewards and abstract long-term information based on which of these is deemed relevant. Employing short- vs. long-term information is associated with different learning mechanisms, yet neural evidence showing that these two are dissociable is lacking. Here we demonstrate that long-term, inference-based beliefs are biased by short-term reward experiences and that dissociable brain regions facilitate both types of learning. Long-term inferences are associated with dorsal striatal and frontopolar cortex activity, while short-term rewards engage the ventral striatum. Stronger concurrent representation of reward signals by mediodorsal striatum and frontopolar cortex correlates with less biased, more optimal individual long-term inference. Moreover, dynamic modulation of activity in a cortical cognitive control network and the medial striatum is associated with trial-by-trial control of biases in belief updating. This suggests that counteracting the processing of optimally to-be-ignored short-term rewards and cortical suppression of associated reward-signals, determines long-term learning success and failure.

Making a good decision often requires the weighing of relative short-term rewards against long-term benefits, yet how the brain does this is not understood. Here, authors show that long-term beliefs are biased by reward experience and that dissociable brain regions facilitate both types of learning.

Lateral Orbitofrontal Inactivation Dissociates Devaluation-Sensitive Behavior and Economic Choice

How do we choose between goods that have different subjective values, like apples and oranges? Neuroeconomics proposes that this is done by reducing complex goods to a single unitary value to allow comparison. This value is computed "on the fly" from the underlying model of the goods space, allowing decisions to meet current needs. This is termed "model-based" behavior to distinguish it from pre-determined, habitual, or "model-free" behavior. The lateral orbitofrontal cortex (OFC) supports model-based behavior in rats and primates, but whether the OFC is necessary for economic choice is less clear. Here we tested this question by optogenetically inactivating the lateral OFC in rats in a classic model-based task and during economic choice. Contrary to predictions, inactivation disrupted model-based behavior without affecting economic choice.

Orbitofrontal Cortex: A Neural Circuit for Economic Decisions

Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.

Optimal coding and neuronal adaptation in economic decisions

During economic decisions, offer value cells in orbitofrontal cortex (OFC) encode the values of offered goods. Furthermore, their tuning functions adapt to the range of values available in any given context. A fundamental and open question is whether range adaptation is behaviorally advantageous. Here we present a theory of optimal coding for economic decisions. We propose that the representation of offer values is optimal if it ensures maximal expected payoff. In this framework, we examine offer value cells in non-human primates. We show that their responses are quasi-linear even when optimal tuning functions are highly non-linear. Most importantly, we demonstrate that for linear tuning functions range adaptation maximizes the expected payoff. Thus value coding in OFC is functionally rigid (linear tuning) but parametrically plastic (range adaptation with optimal gain). Importantly, the benefit of range adaptation outweighs the cost of functional rigidity. While generally suboptimal, linear tuning may facilitate transitive choices.

Spatiotemporal dynamics of information encoding revealed in orbitofrontal high-gamma

High-gamma signals mirror the tuning and temporal profiles of neurons near a recording electrode in sensory and motor areas. These frequencies appear to aggregate local neuronal activity, but it is unclear how this relationship affects information encoding in high-gamma activity (HGA) in cortical areas where neurons are heterogeneous in selectivity and temporal responses, and are not functionally clustered. Here we report that populations of neurons and HGA recorded from the orbitofrontal cortex (OFC) encode similar information, although there is little correspondence between signals recorded by the same electrode. HGA appears to aggregate heterogeneous neuron activity, such that the spiking of a single cell corresponds to only small increases in HGA. Interestingly, large-scale spatiotemporal dynamics are revealed in HGA, but less apparent in the population of single neurons. Overall, HGA is closely related to neuron activity in OFC, and provides a unique means of studying large-scale spatiotemporal dynamics of information processing.

High gamma activity (HGA) and local neurons encode similar information, but it’s unclear if this is true when neurons are heterogeneous, as in the orbitofrontal cortex (OFC). Here, Rich & Wallis show that HGA in OFC is closely related to neuron firing, but reveals clearer spatiotemporal dynamics.

Orbitofrontal cortex activity and connectivity predict future depression symptoms in adolescence

BACKGROUND

Major depressive disorder is a leading cause of disability worldwide; however, little is known about pathological mechanisms involved in its development. Research in adolescent depression has focused on reward sensitivity and striatal mechanisms implementing it. The contribution of loss sensitivity to future depression, as well as the orbitofrontal cortex (OFC) mechanisms critical for processing losses and rewards, remain unexplored. Furthermore, it is unclear whether OFC functioning interacts with familial history in predicting future depression.

METHODS

In this longitudinal study we recorded functional magnetic resonance imaging (fMRI) data while 229 adolescent females with or without parental history of depression completed a monetary gambling task. We examined if OFC blood-oxygen-level-dependent (BOLD) response and functional connectivity during loss and win feedback was associated with depression symptoms concurrently and prospectively (9 months later), and whether this relationship was moderated by parental history of depression.

RESULTS

Reduced OFC response during loss was associated with higher depression symptoms concurrently and prospectively, even after controlling for concurrent depression, specifically in adolescents with parental history of depression. Similarly, increased OFC-posterior insula connectivity during loss was associated with future depression symptoms but this relationship was not moderated by parental history of depression.

CONCLUSIONS

This study provides the first evidence for loss-related alterations in OFC functioning and its interaction with familial history of depression as possible mechanisms in the development of depression. While the current fMRI literature has mainly focused on reward, the present findings underscore the need to include prefrontal loss processing in existing developmental models of depression.

Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices

Before making a reward-based choice, we must evaluate each option. Some theories propose that prospective evaluation involves a reactivation of the neural response to the outcome. Others propose that it calls upon a response pattern that is specific to each underlying associative structure. We hypothesize that these views are reconcilable: during prospective evaluation, offers reactivate neural responses to outcomes that are unique to each associative structure; when the outcome occurs, this pattern is activated, simultaneously, with a general response to the reward. We recorded single-units from macaque orbitofrontal cortex (Area 13) in a riskless choice task with interleaved described and experienced offer trials. Here we report that neural activations to offers and their outcomes overlap, as do neural activations to the outcomes on the two trial types. Neural activations to experienced and described offers are unrelated even though they predict the same outcomes. Our reactivation theory parsimoniously explains these results.

How the brain evaluates options to make a reward-based choice is unclear. Here, authors show that, prior to choice, neural activity patterns to the potential outcomes are reactivated in macaque orbitofrontal cortex, in a way that reflects the unique event sequences leading up to the outcomes.

The impact of ultra-high field MRI on cognitive and computational neuroimaging

The ability to measure functional brain responses non-invasively with ultra high field MRI (7 T and above) represents a unique opportunity in advancing our understanding of the human brain. Compared to lower fields (3 T and below), ultra high field MRI has an increased sensitivity, which can be used to acquire functional images with greater spatial resolution, and greater specificity of the blood oxygen level dependent (BOLD) signal to the underlying neuronal responses. Together, increased resolution and specificity enable investigating brain functions at a submillimeter scale, which so far could only be done with invasive techniques. At this mesoscopic spatial scale, perception, cognition and behavior can be probed at the level of fundamental units of neural computations, such as cortical columns, cortical layers, and subcortical nuclei. This represents a unique and distinctive advantage that differentiates ultra high from lower field imaging and that can foster a tighter link between fMRI and computational modeling of neural networks. So far, functional brain mapping at submillimeter scale has focused on the processing of sensory information and on well-known systems for which extensive information is available from invasive recordings in animals. It remains an open challenge to extend this methodology to uniquely human functions and, more generally, to systems for which animal models may be problematic. To succeed, the possibility to acquire high-resolution functional data with large spatial coverage, the availability of computational models of neural processing as well as accurate biophysical modeling of neurovascular coupling at mesoscopic scale all appear necessary.

Computational Architecture of the Parieto-Frontal Network Underlying Cognitive-Motor Control in Monkeys

The statistical structure of intrinsic parietal and parieto-frontal connectivity in monkeys was studied through hierarchical cluster analysis. Based on their inputs, parietal and frontal areas were grouped into different clusters, including a variable number of areas that in most instances occupied contiguous architectonic fields. Connectivity tended to be stronger locally: that is, within areas of the same cluster. Distant frontal and parietal areas were targeted through connections that in most instances were reciprocal and often of different strength. These connections linked parietal and frontal clusters formed by areas sharing basic functional properties. This led to five different medio-laterally oriented pillar domains spanning the entire extent of the parieto-frontal system, in the posterior parietal, anterior parietal, cingulate, frontal, and prefrontal cortex. Different information processing streams could be identified thanks to inter-domain connectivity. These streams encode fast hand reaching and its control, complex visuomotor action spaces, hand grasping, action/intention recognition, oculomotor intention and visual attention, behavioral goals and strategies, and reward and decision value outcome. Most of these streams converge on the cingulate domain, the main hub of the system. All of them are embedded within a larger eye–hand coordination network, from which they can be selectively set in motion by task demands.