Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

Human ventromedial prefrontal cortex is necessary for prosocial motivation

Ventromedial prefrontal cortex (vmPFC) is vital for decision-making. Functional neuroimaging links vmPFC to processing rewards and effort, while parallel work suggests vmPFC involvement in prosocial behaviour. However, the necessity of vmPFC for these functions is unknown. Patients with rare focal vmPFC lesions (n = 25), patients with lesions elsewhere (n = 15) and healthy controls (n = 40) chose between rest and exerting effort to earn rewards for themselves or another person. vmPFC damage decreased prosociality across behavioural and computational measures. vmPFC patients earned less, discounted rewards by effort more, and exerted less force when another person benefited, compared to both control groups. Voxel-based lesion mapping revealed dissociations between vmPFC subregions. While medial damage led to antisocial behaviour, lateral damage increased prosocial behaviour relative to patients with damage elsewhere. vmPFC patients also showed reduced effort sensitivity overall, but reward sensitivity was limited to specific subregions. These results reveal multiple causal contributions of vmPFC to prosocial behaviour, effort and reward.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dissociable representations of decision variables within subdivisions of macaque orbitofrontal and ventrolateral frontal cortex

Ventral frontal cortex (VFC) in macaques is involved in many affective and cognitive processes and has a key role in flexibly guiding reward-based decision-making. VFC is composed of a set of anatomically distinct subdivisions that are within the orbitofrontal cortex, ventrolateral prefrontal cortex, and anterior insula. In part, because prior studies have lacked the resolution to test for differences, it is unclear if neural representations related to decision-making are dissociable across these subdivisions. Here we recorded the activity of thousands of neurons within eight anatomically defined subregions of VFC in macaque monkeys performing a two-choice probabilistic task for different fruit juices outcomes. We found substantial variation in the encoding of decision variables across these eight subdivisions. Notably, ventrolateral subdivision 12l was unique relative to the other areas that we recorded from as the activity of single neurons integrated multiple attributes when monkeys evaluated the different choice options. Activity within 12o, by contrast, more closely represented reward probability and whether reward was received on a given trial. Orbitofrontal area 11m/l contained more specific representations of the quality of the outcome that could be earned later on. We also found that reward delivery encoding was highly distributed across all VFC subregions, while the properties of the reward, such as its flavor, were more strongly represented in areas 11m/l and 13m. Taken together, our work reveals the diversity of encoding within the various anatomically distinct subdivisions of VFC in primates.

Beta and theta oscillations track effort and previous reward in human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we investigate this with chronic intracranial recordings from prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12-20 Hz) oscillations tracking subjective effort on a single trial basis and PFC theta (4-7 Hz) signaling previous trial reward. Stimulation of PFC increased overall acceptance of offers in addition to increasing the impact of reward on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, as well as supporting a causal role of PFC for such choices.

A goal pointer for a cognitive map in the orbitofrontal cortex

Knowing where you are and where you go is a prerequisite for planning a goal-directed journey. The discovery of spatially tuned neurons in the hippocampus and parahippocampal cortices provides a mechanism by which the brain pinpoints an animal’s own position in an environment. By contrast, how the brain encodes a remote navigational goal remained largely obscure until recently. In this review, we discuss algorithmic challenges and requirements for the brain to form a representation of a remote navigational goal at which an animal is not present. We then highlight a line of evidence that neurons in the orbitofrontal cortex (OFC) represent a goal location persistently while an animal navigates to this destination. Finally, we propose a new perspective of navigation research opened by this recently reported brain’s goal map.

Semi-orthogonal subspaces for value mediate a tradeoff between binding and generalization

When choosing between options, we must associate their values with the action needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. To test this hypothesis, we examined neuronal responses in five reward-sensitive regions in macaques performing a risky choice task with sequential offers. Surprisingly, in all areas, the neural population encoded the values of offers presented on the left and right in distinct subspaces. We show that the encoding we observe is sufficient to bind the values of the offers to their respective positions in space while preserving abstract value information, which may be important for rapid learning and generalization to novel contexts. Moreover, after both offers have been presented, all areas encode the value of the first and second offers in orthogonal subspaces. In this case as well, the orthogonalization provides binding. Our binding-by-subspace hypothesis makes two novel predictions borne out by the data. First, behavioral errors should correlate with putative spatial (but not temporal) misbinding in the neural representation. Second, the specific representational geometry that we observe across animals also indicates that behavioral errors should increase when offers have low or high values, compared to when they have medium values, even when controlling for value difference. Together, these results support the idea that the brain makes use of semi-orthogonal subspaces to bind features together.

The orbitofrontal cortex: A goal-directed cognitive map framework for social and non-social behaviors

The orbitofrontal cortex (OFC) is regarded as one of the core brain areas in a variety of value-based behaviors. Over the past two decades, tremendous knowledge about the OFC function was gained from studying the behaviors of single subjects. As a result, our previous understanding of the OFC's function of encoding decision variables, such as the value and identity of choices, has evolved to the idea that the OFC encodes a more complex representation of the task space as a cognitive map. Accumulating evidence also indicates that the OFC importantly contributes to behaviors in social contexts, especially those involved in cooperative interactions. However, it remains elusive how exactly OFC neurons contribute to social functions and how non-social and social behaviors are related to one another in the computations performed by OFC neurons. In this review, we aim to provide an integrated view of the OFC function across both social and non-social behavioral contexts. We propose that seemingly complex functions of the OFC may be explained by its role in providing a goal-directed cognitive map to guide a wide array of adaptive reward-based behaviors in both social and non-social domains.

Value dynamics affect choice preparation during decision-making

During decision-making, neurons in the orbitofrontal cortex (OFC) sequentially represent the value of each option in turn, but it is unclear how these dynamics are translated into a choice response. One brain region that may be implicated in this process is the anterior cingulate cortex (ACC), which strongly connects with OFC and contains many neurons that encode the choice response. We investigated how OFC value signals interacted with ACC neurons encoding the choice response by performing simultaneous high-channel count recordings from the two areas in nonhuman primates. ACC neurons encoding the choice response steadily increased their firing rate throughout the decision-making process, peaking shortly before the time of the choice response. Furthermore, the value dynamics in OFC affected ACC ramping-when OFC represented the more valuable option, ACC ramping accelerated. Because OFC tended to represent the more valuable option more frequently and for a longer duration, this interaction could explain how ACC selects the more valuable response.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

Subspace orthogonalization as a mechanism for binding values to space

When choosing between options, we must solve an important binding problem. The values of the options must be associated with information about the action needed to select them. We hypothesize that the brain solves this binding problem through use of distinct population subspaces. To test this hypothesis, we examined the responses of single neurons in five reward-sensitive regions in rhesus macaques performing a risky choice task. In all areas, neurons encoded the value of the offers presented on both the left and the right side of the display in semi-orthogonal subspaces, which served to bind the values of the two offers to their positions in space. Supporting the idea that this orthogonalization is functionally meaningful, we observed a session-to-session covariation between choice behavior and the orthogonalization of the two value subspaces: trials with less orthogonalized subspaces were associated with greater likelihood of choosing the less valued option. Further inspection revealed that these semi-orthogonal subspaces arose from a combination of linear and nonlinear mixed selectivity in the neural population. We show this combination of selectivity balances reliable binding with an ability to generalize value across different spatial locations. These results support the hypothesis that semi-orthogonal subspaces support reliable binding, which is essential to flexible behavior in the face of multiple options.

Frontal and temporal coding dynamics in successive steps of complex behavior

Ventrolateral prefrontal cortex (vlPFC), dorsolateral prefrontal cortex (dlPFC), and temporal cortex (TE) all contribute to visual decision-making. Accumulating evidence suggests that vlPFC may play a central role in multiple cognitive operations, perhaps resembling domain-general regions of the human frontal lobe. We trained monkeys in a task calling for learning, retrieval, and spatial selection of rewarded target objects. Recordings of neural activity covered large areas of vlPFC, dlPFC, and TE. Results suggested a central role for vlPFC in each cognitive operation with strong coding of each task feature, while only location was strongly coded in dlPFC and current object identity in TE. During target selection, target location was communicated first from vlPFC to dlPFC, followed by extensive mutual support. In vlPFC, stimulus identities were independently coded in different task operations. The results suggest a central role for the inferior frontal convexity in controlling successive operations of a complex, multi-step task.

Integrated neural dynamics of sensorimotor decisions and actions

Recent theoretical models suggest that deciding about actions and executing them are not implemented by completely distinct neural mechanisms but are instead two modes of an integrated dynamical system. Here, we investigate this proposal by examining how neural activity unfolds during a dynamic decision-making task within the high-dimensional space defined by the activity of cells in monkey dorsal premotor (PMd), primary motor (M1), and dorsolateral prefrontal cortex (dlPFC) as well as the external and internal segments of the globus pallidus (GPe, GPi). Dimensionality reduction shows that the four strongest components of neural activity are functionally interpretable, reflecting a state transition between deliberation and commitment, the transformation of sensory evidence into a choice, and the baseline and slope of the rising urgency to decide. Analysis of the contribution of each population to these components shows meaningful differences between regions but no distinct clusters within each region, consistent with an integrated dynamical system. During deliberation, cortical activity unfolds on a two-dimensional "decision manifold" defined by sensory evidence and urgency and falls off this manifold at the moment of commitment into a choice-dependent trajectory leading to movement initiation. The structure of the manifold varies between regions: In PMd, it is curved; in M1, it is nearly perfectly flat; and in dlPFC, it is almost entirely confined to the sensory evidence dimension. In contrast, pallidal activity during deliberation is primarily defined by urgency. We suggest that these findings reveal the distinct functional contributions of different brain regions to an integrated dynamical system governing action selection and execution.

Orbitofrontal cortex populations are differentially recruited to support actions

The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.

Integrated neural dynamics for behavioural decisions and attentional competition in the prefrontal cortex

In the behaving monkey, complex neural dynamics in the prefrontal cortex contribute to context-dependent decisions and attentional competition. We used demixed principal component analysis to track prefrontal activity dynamics in a cued target detection task. In this task, the animal combined identity of a visual object with a prior instruction cue to determine a target/nontarget decision. From population activity, we extracted principal components for each task feature and examined their time course and sensitivity to stimulus and task variations. For displays containing a single choice object in left or right hemifield, object identity, cue identity and decision were all encoded in population activity, with different dynamics and lateralisation. Object information peaked at 100-200 ms from display onset and was largely confined to the contralateral hemisphere. Cue information was weaker and present even prior to display onset. Integrating information from cue and object, decision information arose more slowly and was bilateral. Individual neurons contributed independently to coding of the three task features. The analysis was then extended to displays with a target in one hemifield and a competing distractor in the other. In this case, the data suggest that each hemisphere initially encoded the identity of the contralateral object. The distractor representation was then rapidly suppressed, with the final target decision again encoded bilaterally. The results show how information is coded along task-related dimensions while competition is resolved and suggest how information flows within and across frontal lobes to implement a learned behavioural decision.

Orbitofrontal cortex contributes to the comparison of values underlying economic choices

Economic choices between goods entail the computation and comparison of subjective values. Previous studies examined neuronal activity in the orbitofrontal cortex (OFC) of monkeys choosing between different types of juices. Three groups of neurons were identified: offer value cells encoding the value of individual offers, chosen juice cells encoding the identity of the chosen juice, and chosen value cells encoding the value of the chosen offer. The encoded variables capture both the input (offer value) and the output (chosen juice, chosen value) of the decision process, suggesting that values are compared within OFC. Recent work demonstrates that choices are causally linked to the activity of offer value cells. Conversely, the hypothesis that OFC contributes to value comparison has not been confirmed. Here we show that weak electrical stimulation of OFC specifically disrupts value comparison without altering offer values. This result implies that neuronal populations in OFC participate in value comparison.

Taking stock of value in the orbitofrontal cortex

People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.

A structural and functional subdivision in central orbitofrontal cortex

Economic choice requires many cognitive subprocesses, including stimulus detection, valuation, motor output, and outcome monitoring; many of these subprocesses are associated with the central orbitofrontal cortex (cOFC). Prior work has largely assumed that the cOFC is a single region with a single function. Here, we challenge that unified view with convergent anatomical and physiological results from rhesus macaques. Anatomically, we show that the cOFC can be subdivided according to its much stronger (medial) or weaker (lateral) bidirectional anatomical connectivity with the posterior cingulate cortex (PCC). We call these subregions cOFCm and cOFCl, respectively. These two subregions have notable functional differences. Specifically, cOFCm shows enhanced functional connectivity with PCC, as indicated by both spike-field coherence and mutual information. The cOFCm-PCC circuit, but not the cOFCl-PCC circuit, shows signatures of relaying choice signals from a non-spatial comparison framework to a spatially framed organization and shows a putative bidirectional mutually excitatory pattern.

Medial and orbital frontal cortex in decision-making and flexible behavior

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

Neuronal origins of reduced accuracy and biases in economic choices under sequential offers

Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here, we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in the orbitofrontal cortex revealed that these phenomena emerged at different computational stages. Lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey.

Functional Connectivity Basis and Underlying Cognitive Mechanisms for Gender Differences in Guilt Aversion

Prosocial behavior is pivotal to our society. Guilt aversion, which describes the tendency to reduce the discrepancy between a partner's expectation and his/her actual outcome, drives human prosocial behavior as does well-known inequity aversion. Although women are reported to be more inequity averse than men, gender differences in guilt aversion remain unexplored. Here, we conducted a functional magnetic resonance imaging (fMRI) study (n = 52) and a large-scale online behavioral study (n = 4723) of a trust game designed to investigate guilt and inequity aversions. The fMRI study demonstrated that men exhibited stronger guilt aversion and recruited right dorsolateral prefrontal cortex (DLPFC)-ventromedial PFC (VMPFC) connectivity more for guilt aversion than women, while VMPFC-dorsal medial PFC (DMPFC) connectivity was commonly used in both genders. Furthermore, our regression analysis of the online behavioral data collected with Big Five and demographic factors replicated the gender differences and revealed that Big Five Conscientiousness (rule-based decision) correlated with guilt aversion only in men, but Agreeableness (empathetic consideration) correlated with guilt aversion in both genders. Thus, this study suggests that gender differences in prosocial behavior are heterogeneous depending on underlying motives in the brain and that the consideration of social norms plays a key role in the stronger guilt aversion in men.

The Neural Instantiation of an Abstract Cognitive Map for Economic Choice

Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.

Impaired cognitive flexibility following NMDAR-GluN2B deletion is associated with altered orbitofrontal-striatal function

A common feature across neuropsychiatric disorders is inability to discontinue an action or thought once it has become detrimental. Reversal learning, a hallmark of executive control, requires plasticity within cortical, striatal and limbic circuits and is highly sensitive to disruption of N-methyl-d-aspartate receptor (NMDAR) function. In particular, selective deletion or antagonism of GluN2B containing NMDARs in cortical regions including the orbitofrontal cortex (OFC), promotes maladaptive perseveration. It remains unknown whether GluN2B functions to maintain local cortical activity necessary for reversal learning, or if it exerts a broader influence on the integration of neural activity across cortical and subcortical systems. To address this question, we utilized in vivo electrophysiology to record neuronal activity and local field potentials (LFP) in the orbitofrontal cortex and dorsal striatum (dS) of mice with deletion of GluN2B in neocortical and hippocampal principal cells while they performed touchscreen reversal learning. Reversal impairment produced by corticohippocampal GluN2B deletion was paralleled by an aberrant increase in functional connectivity between the OFC and dS. These alterations in coordination were associated with alterations in local OFC and dS firing activity. These data demonstrate highly dynamic patterns of cortical and striatal activity concomitant with reversal learning, and reveal GluN2B as a molecular mechanism underpinning the timing of these processes.

Neuronal activity in dorsal anterior cingulate cortex during economic choices under variable action costs

The role of the dorsal anterior cingulate cortex (ACCd) in decision making has often been discussed but remains somewhat unclear. On the one hand, numerous studies implicated this area in decisions driven by effort or action cost. On the other hand, work on economic choices between goods (under fixed action costs) found that neurons in ACCd encoded only post-decision variables. To advance our understanding of the role played by this area in decision making, we trained monkeys to choose between different goods (juice types) offered in variable amounts and with different action costs. Importantly, the task design dissociated computation of the action cost from planning of any particular action. Neurons in ACCd encoded the chosen value and the binary choice outcome in several reference frames (chosen juice, chosen cost, chosen action). Thus, this area provided a rich representation of post-decision variables. In contrast to the OFC, neurons in ACCd did not represent pre-decision variables such as individual offer values in any reference frame. Hence, ongoing decisions are unlikely guided by ACCd. Conversely, neuronal activity in this area might inform subsequent actions.

Cortical hemodynamic mechanisms of reversal learning using high-resolution functional near-infrared spectroscopy: A pilot study

OBJECTIVES

Reversal learning is widely used to analyze cognitive flexibility and characterize behavioral abnormalities associated with impulsivity and disinhibition. Recent studies using fMRI have focused on regions involved in reversal learning with negative and positive reinforcers. Although the frontal cortex has been consistently implicated in reversal learning, few studies have focused on whether reward and punishment may have different effects on lateral frontal structures in these tasks.

METHODS

During this pilot study on eight healthy subjects, we used functional near infra-red spectroscopy (fNIRS) to characterize brain activity dynamics and differentiate the involvement of frontal structures in learning driven by reward and punishment.

RESULTS

We observed functional hemispheric asymmetries between punishment and reward processing by fNIRS following reversal of a learned rule. Moreover, the left dorsolateral prefrontal cortex (l-DLPFC) and inferior frontal gyrus (IFG) were activated under the reward condition only, whereas the orbito-frontal cortex (OFC) was significantly activated under the punishment condition, with a tendency towards activation for the right cortical hemisphere (r-DLPFC and r-IFG). Our results are compatible with the suggestion that the DLPFC is involved in the detection of contingency change. We propose a new representation for reward and punishment, with left lateralization for the reward process.

CONCLUSIONS

The results of this pilot study provide insights into the indirect neural mechanisms of reversal learning and behavioral flexibility and confirm the use of fNIRS imaging in reversal-learning tasks as a translational strategy, particularly in subjects who cannot undergo fMRI recordings.

Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex

Choice-relevant brain regions in prefrontal cortex may progressively transform information about options into choices. Here, we examine responses of neurons in four regions of the medial prefrontal cortex as macaques performed two-option risky choices. All four regions encode economic variables in similar proportions and show similar putative signatures of key choice-related computations. We provide evidence to support a gradient of function that proceeds from areas 14 to 25 to 32 to 24. Specifically, we show that decodability of twelve distinct task variables increases along that path, consistent with the idea that regions that are higher in the anatomical hierarchy make choice-relevant variables more separable. We also show progressively longer intrinsic timescales in the same series. Together these results highlight the importance of the medial wall in choice, endorse a specific gradient-based organization, and argue against a modular functional neuroanatomy of choice.

Frontal circuit specialisations for decision making

There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.

Stimulus-dependent relationships between behavioral choice and sensory neural responses

Understanding perceptual decision-making requires linking sensory neural responses to behavioral choices. In two-choice tasks, activity-choice covariations are commonly quantified with a single measure of choice probability (CP), without characterizing their changes across stimulus levels. We provide theoretical conditions for stimulus dependencies of activity-choice covariations. Assuming a general decision-threshold model, which comprises both feedforward and feedback processing and allows for a stimulus-modulated neural population covariance, we analytically predict a very general and previously unreported stimulus dependence of CPs. We develop new tools, including refined analyses of CPs and generalized linear models with stimulus-choice interactions, which accurately assess the stimulus- or choice-driven signals of each neuron, characterizing stimulus-dependent patterns of choice-related signals. With these tools, we analyze CPs of macaque MT neurons during a motion discrimination task. Our analysis provides preliminary empirical evidence for the promise of studying stimulus dependencies of choice-related signals, encouraging further assessment in wider data sets.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Social processing by the primate medial frontal cortex

The primate medial frontal cortex is comprised of several brain regions that are consistently implicated in regulating complex social behaviors. The medial frontal cortex is also critically involved in many non-social behaviors, such as those involved in reward, affective, and decision-making processes, broadly implicating the fundamental role of the medial frontal cortex in internally guided cognition. An essential question therefore is what unique contributions, if any, does the medial frontal cortex make to social behaviors? In this chapter, we outline several neural algorithms necessary for mediating adaptive social interactions and discuss selected evidence from behavioral neurophysiology experiments supporting the role of the medial frontal cortex in implementing these algorithms. By doing so, we primarily focus on research in nonhuman primates and examine several key attributes of the medial frontal cortex. Specifically, we review neuronal substrates in the medial frontal cortex uniquely suitable for enabling social monitoring, observational and vicarious learning, as well as predicting the behaviors of social partners. Moreover, by utilizing the three levels of organization in information processing systems proposed by Marr (1982) and recently adapted by Lockwood, Apps, and Chang (2020) for social information processing, we survey selected social functions of the medial frontal cortex through the lens of socially relevant algorithms and implementations. Overall, this chapter provides a broad overview of the behavioral neurophysiology literature endorsing the importance of socially relevant neural algorithms implemented by the primate medial frontal cortex for regulating social interactions.

Neurons in primate prefrontal cortex signal valuable social information during natural viewing

Information about social partners is innately valuable to primates. Decisions about which sources of information to consume are highly naturalistic but also complex and place unusually strong demands on the brain's decision network. In particular, both the orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC) play key roles in decision making and social behaviour, suggesting a likely role in social information-seeking as well. To test this idea, we developed a 'channel surfing' task in which monkeys were shown a series of 5 s video clips of conspecifics engaged in natural behaviours at a field site. Videos were annotated frame-by-frame using an ethogram of species-typical behaviours, an important source of social information. Between each clip, monkeys were presented with a choice between targets that determined which clip would be seen next. Monkeys' gaze during playback indicated differential engagement depending on what behaviours were presented. Neurons in both OFC and LPFC responded to choice targets and to video, and discriminated a subset of the behaviours in the ethogram during video viewing. These findings suggest that both OFC and LPFC are engaged in processing social information that is used to guide dynamic information-seeking decisions. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Neural Mechanisms of Human Decision-Making

We present a theory and neural network model of the neural mechanisms underlying human decision-making. We propose a detailed model of the interaction between brain regions, under a proposer-predictor-actor-critic​ ​framework. This theory is based on detailed animal data and theories of action-selection. Those theories are adapted to serial operation to bridge levels of analysis and explain human decision-making. Task-relevant areas of cortex propose a candidate plan using fast, model-free, parallel neural computations. Other areas of cortex and medial temporal lobe can then predict​ likely outcomes of that plan in this situation. This optional prediction- (or model-) based computation can produce better accuracy and generalization, at the expense of speed. Next, linked regions of basal ganglia act​ to accept or reject the proposed plan based on its reward history in similar contexts. If that plan is rejected, the process repeats to consider a new option. The reward-prediction system acts as a critic​ to determine the value of the outcome relative to expectations and produce dopamine as a training signal for cortex and basal ganglia. By operating sequentially and hierarchically, the same mechanisms previously proposed for animal action-selection could explain the most complex human plans and decisions. We discuss explanations of model-based decisions, habitization, and risky behavior based on the computational model.

Reward modulates cortical representations of action

People are capable of rapid improvements in performance when they are offered a reward. The neural mechanism by which this performance enhancement occurs remains unclear. We investigated this phenomenon by offering people monetary reward for successful performance in a sequence production task. We found that people performed actions more quickly and accurately when they were offered large reward. Increasing reward magnitude was associated with elevated activity throughout the brain prior to movement. Multivariate patterns of activity in these reward-responsive regions encoded information about the upcoming action. Follow-up analyses provided evidence that action decoding in pre-SMA and other motor planning areas was improved for large reward trials and successful action decoding in SMA was associated with improved performance. These results suggest that reward may enhance performance by enhancing neural representations of action used in motor planning.

Evidence accumulation for value computation in the prefrontal cortex during decision making

A key step of decision making is to determine the value associated with each option. The evaluation process often depends on the accumulation of evidence from multiple sources, which may arrive at different times. How evidence is accumulated for value computation in the brain during decision making has not been well studied. To address this problem, we trained rhesus monkeys to perform a decision-making task in which they had to make eye movement choices between two targets, whose reward probabilities had to be determined with the combined evidence from four sequentially presented visual stimuli. We studied the encoding of the reward probabilities associated with the stimuli and the eye movements in the orbitofrontal (OFC) and the dorsolateral prefrontal (DLPFC) cortices during the decision process. We found that the OFC neurons encoded the reward probability associated with individual pieces of evidence in the stimulus domain. Importantly, the representation of the reward probability in the OFC was transient, and the OFC did not encode the reward probability associated with the combined evidence from multiple stimuli. The computation of the combined reward probabilities was observed only in the DLPFC and only in the action domain. Furthermore, the reward probability encoding in the DLPFC exhibited an asymmetric pattern of mixed selectivity that supported the computation of the stimulus-to-action transition of reward information. Our results reveal that the OFC and the DLPFC play distinct roles in the value computation during evidence accumulation.

Hypothalamic Interactions with Large-Scale Neural Circuits Underlying Reinforcement Learning and Motivated Behavior

Biological agents adapt behavior to support the survival needs of the individual and the species. In this review we outline the anatomical, physiological, and computational processes that support reinforcement learning (RL). We describe two circuits in the primate brain that are linked to specific aspects of learning and goal-directed behavior. The ventral circuit, that includes the amygdala, ventral medial prefrontal cortex, and ventral striatum, has substantial connectivity with the hypothalamus. The dorsal circuit, that includes inferior parietal cortex, dorsal lateral prefrontal cortex, and the dorsal striatum, has minimal connectivity with the hypothalamus. The hypothalamic connectivity suggests distinct roles for these circuits. We propose that the ventral circuit defines behavioral goals, and the dorsal circuit orchestrates behavior to achieve those goals.

A Network Activity Reconfiguration Underlies the Transition from Goal to Action

Neurons in prefrontal cortex (PF) represent mnemonic information about current goals until the action can be selected and executed. However, the neuronal dynamics underlying the transition from goal into specific actions are poorly understood. Here, we show that the goal-coding PF network is dynamically reconfigured from mnemonic to action selection states and that such reconfiguration is mediated by cell assemblies with heterogeneous excitability. We recorded neuronal activity from PF while monkeys selected their actions on the basis of memorized goals. Many PF neurons encoded the goal, but only a minority of them did so across both memory retention and action selection stages. Interestingly, about half of this minority of neurons switched their goal preference across the goal-action transition. Our computational model led us to propose a PF network composed of heterogeneous cell assemblies with single-state and bistable local dynamics able to produce both dynamical stability and input susceptibility simultaneously.

Signal dynamics of midbrain dopamine neurons during economic decision-making in monkeys

When we make economic choices, the brain first evaluates available options and then decides whether to choose them. Midbrain dopamine neurons are known to reinforce economic choices through their signal evoked by outcomes after decisions are made. However, although critical internal processing is executed while decisions are being made, little is known about the role of dopamine neurons during this period. We found that dopamine neurons exhibited dynamically changing signals related to the internal processing while rhesus monkeys were making decisions. These neurons encoded the value of an option immediately after it was offered and then gradually changed their activity to represent the animal's upcoming choice. Similar dynamics were observed in the orbitofrontal cortex, a center for economic decision-making, but the value-to-choice signal transition was completed earlier in dopamine neurons. Our findings suggest that dopamine neurons are a key component of the neural network that makes choices from values during ongoing decision-making processes.

Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models

Schizophrenia is characterized by positive, negative and cognitive symptoms. All current antipsychotic treatments feature dopamine-receptor antagonism that is relatively effective at addressing the psychotic (positive) symptoms of schizophrenia. However, there is no clear evidence that these medications improve the negative or cognitive symptoms, which are the greatest predictors of functional outcomes. One of the most robust pathophysiological observations in patients with schizophrenia is increased subcortical dopamine neurotransmission, primarily in the associative striatum. This brain area has an important role in a range of cognitive processes. Dopamine is also known to play a major part in regulating a number of cognitive functions impaired in schizophrenia but much of this research has been focused on cortical dopamine. Emerging research highlights the strong influence subcortical dopamine has on a range of cognitive domains, including attention, reward learning, goal-directed action and decision-making. Nonetheless, the precise role of the associative striatum in the cognitive impairments observed in schizophrenia remains poorly understood, presenting an opportunity to revisit its contribution to schizophrenia. Without a better understanding of the mechanisms underlying cognitive dysfunction, treatment development remains at a standstill. For this reason, improved preclinical animal models are needed if we are to understand the complex relationship between subcortical dopamine and cognition. A range of new techniques are facillitating the discrete manipulation of dopaminergic neurotransmission and measurements of cognitive performance, which can be investigated using a variety of sensitive translatable tasks. This has the potential to aid the successful incorporation of recent clinical research to address the lack of treatment strategies for cognitive symptoms in schizophrenia. This review will give an overview on the current state of research focused on subcortical dopamine and cognition in the context of schizophrenia research. We also discuss future strategies and approaches aimed at improving the translational outcomes for the treatment of cognitive deficits in schizophrenia.

Turning Touch into Perception

Many brain areas modulate their activity during vibrotactile tasks. The activity from these areas may code the stimulus parameters, stimulus perception, or perceptual reports. Here, we discuss findings obtained in behaving monkeys aimed to understand these processes. In brief, neurons from the somatosensory thalamus and primary somatosensory cortex (S1) only code the stimulus parameters during the stimulation periods. In contrast, areas downstream of S1 code the stimulus parameters during not only the task components but also perception. Surprisingly, the midbrain dopamine system is an actor not considered before in perception. We discuss the evidence that it codes the subjective magnitude of a sensory percept. The findings reviewed here may help us to understand where and how sensation transforms into perception in the brain.

Neuronal Activity in the Primate Amygdala during Economic Choice

Multiple lines of evidence link economic choices to the orbitofrontal cortex (OFC), but other brain regions may contribute to the computation and comparison of economic values. A particularly strong candidate is the basolateral amygdala (BLA). Amygdala lesions impair performance in reinforcer devaluation tasks, suggesting that the BLA contributes to value computation. Furthermore, previous studies of the BLA have found neuronal activity consistent with a value representation. Here, we recorded from the BLA of two male rhesus macaques choosing between different juices. Offered quantities varied from trial to trial, and relative values were inferred from choices. Approximately one-third of BLA cells were task-related. Our analyses revealed the presence of three groups of neurons encoding variables offer value, chosen value, and chosen juice In this respect, the BLA appeared similar to the OFC. The two areas differed for the proportion of neurons in each group, as the fraction of chosen value cells was significantly higher in the BLA. Importantly, the activity of these neurons reflected the subjective nature of value. Firing rates in the BLA were sustained throughout the trial and maximal after juice delivery. In contrast, firing rates in the OFC were phasic and maximal shortly after offer presentation. Our results suggest that the BLA supports economic choice and reward expectation.SIGNIFICANCE STATEMENT Economic choices rely on the orbitofrontal cortex (OFC), but other brain regions may contribute to this behavior. A strong candidate is the basolateral amygdala (BLA). Previous results are consistent with a neuronal representation of value, but the role of the BLA in economic decisions remains unclear. Here, we recorded from monkeys choosing between juices. Neurons in the BLA encoded three decision variables: offer value, chosen value, and chosen juice These variables were also identified in the OFC. The two areas differed in the proportion of cells encoding each variable and in the activation timing. In the OFC, firing rates peaked shortly after offer presentation; in the BLA, firing rates were sustained and peaked after juice delivery. These results suggest that the BLA supports choices and reward expectation.

The contribution of nonhuman primate research to the understanding of emotion and cognition and its clinical relevance

Psychiatric disorders are often conceptualized as arising from dysfunctional interactions between neural systems mediating cognitive and emotional processes. Mechanistic insights into these interactions have been lacking in part because most work in emotions has occurred in rodents, often without concurrent manipulations of cognitive variables. Nonhuman primate (NHP) model systems provide a powerful platform for investigating interactions between cognitive operations and emotions due to NHPs' strong homology with humans in behavioral repertoire and brain anatomy. Recent electrophysiological studies in NHPs have delineated how neural signals in the amygdala, a brain structure linked to emotion, predict impending appetitive and aversive stimuli. In addition, abstract conceptual information has also been shown to be represented in the amygdala and in interconnected brain structures such as the hippocampus and prefrontal cortex. Flexible adjustments of emotional behavior require the ability to apply conceptual knowledge and generalize to different, often novel, situations, a hallmark example of interactions between cognitive and emotional processes. Elucidating the neural mechanisms that explain how the brain processes conceptual information in relation to emotional variables promises to provide important insights into the pathophysiology accounting for symptoms in neuropsychiatric disorders.

Neuronal Representation of Object Choice in the Striatum of the Monkey

According to a widely held view, the decision-making process can be conceptualized as a two-step process: “object choice,” which does not include physical actions, followed by “movement choice,” in which action is executed to obtain the object. Accumulating evidence in the field of decision neuroscience suggests that the cortico-basal ganglia circuits play a crucial role in decision-making. However, the underlying mechanisms of the object and movement choices remain poorly understood, mainly because the two processes occur simultaneously in most experiments. In this study, to uncover the neuronal basis of object choice in the striatum, the main input site of the basal ganglia, we designed a behavioral task in which the processes of object and movement choice were temporally separated, and recorded the single-unit activity of phasically active neurons (PANs) (n = 375) in the striatum of two monkeys. We focused our study mainly on neuronal representation during the object choice period, before movement choice, using a mutual information analysis. Population striatal activities significantly represented the information of the chosen object during the object choice period, which indicated that the monkeys actually made the object choice during the task. For the activity of each individual neuron during the object choice period, we identified offered object- and chosen object-type neurons, corresponding to pre- and post-decision signals, respectively. We also found the movement-type neurons during the movement period after the object choice. Most offered object- or chosen object-type neurons were not overlapped with movement-type neurons. The presence of object choice-related signals independent of movement signal in the striatum indicated that the striatum was part of the site where object choice was made within a cortico-basal ganglia circuit.

Categorical encoding of decision variables in orbitofrontal cortex

A fundamental and recurrent question in systems neuroscience is that of assessing what variables are encoded by a given population of neurons. Such assessments are often challenging because neurons in one brain area may encode multiple variables, and because neuronal representations might be categorical or non-categorical. These issues are particularly pertinent to the representation of decision variables in the orbitofrontal cortex (OFC)-an area implicated in economic choices. Here we present a new algorithm to assess whether a neuronal representation is categorical or non-categorical, and to identify the encoded variables if the representation is indeed categorical. The algorithm is based on two clustering procedures, one variable-independent and the other variable-based. The two partitions are then compared through adjusted mutual information. The present algorithm overcomes limitations of previous approaches and is widely applicable. We tested the algorithm on synthetic data and then used it to examine neuronal data recorded in the primate OFC during economic decisions. Confirming previous assessments, we found the neuronal representation in OFC to be categorical in nature. We also found that neurons in this area encode the value of individual offers, the binary choice outcome and the chosen value. In other words, during economic choice, neurons in the primate OFC encode decision variables in a categorical way.

Specializations for reward-guided decision-making in the primate ventral prefrontal cortex

The estimated values of choices, and therefore decision-making based on those values, are influenced by both the chance that the chosen items or goods can be obtained (availability) and their current worth (desirability) as well as by the ability to link the estimated values to choices (a process sometimes called credit assignment). In primates, the prefrontal cortex (PFC) has been thought to contribute to each of these processes; however, causal relationships between particular subdivisions of the PFC and specific functions have been difficult to establish. Recent lesion-based research studies have defined the roles of two different parts of the primate PFC - the orbitofrontal cortex (OFC) and the ventral lateral frontal cortex (VLFC) - and their subdivisions in evaluating each of these factors and in mediating credit assignment during reward-based decision-making.

Primate prefrontal neurons signal economic risk derived from the statistics of recent reward experience

Risk derives from the variation of rewards and governs economic decisions, yet how the brain calculates risk from the frequency of experienced events, rather than from explicit risk-descriptive cues, remains unclear. Here, we investigated whether neurons in dorsolateral prefrontal cortex process risk derived from reward experience. Monkeys performed in a probabilistic choice task in which the statistical variance of experienced rewards evolved continually. During these choices, prefrontal neurons signaled the reward-variance associated with specific objects ('object risk') or actions ('action risk'). Crucially, risk was not derived from explicit, risk-descriptive cues but calculated internally from the variance of recently experienced rewards. Support-vector-machine decoding demonstrated accurate neuronal risk discrimination. Within trials, neuronal signals transitioned from experienced reward to risk (risk updating) and from risk to upcoming choice (choice computation). Thus, prefrontal neurons encode the statistical variance of recently experienced rewards, complying with formal decision variables of object risk and action risk.

Transforming the Choice Outcome to an Action Plan in Monkey Lateral Prefrontal Cortex: A Neural Circuit Model

In economic decisions, we make a good-based choice first, then we transform the outcome into an action to obtain the good. To elucidate the network mechanisms for such transformation, we constructed a neural circuit model consisting of modules representing choice, integration of choice with target locations, and the final action plan. We examined three scenarios regarding how the final action plan could emerge in the neural circuit and compared their implications with experimental data. Our model with heterogeneous connectivity predicts the coexistence of three types of neurons with distinct functions, confirmed by analyzing the neural activity in the lateral prefrontal cortex (LPFC) of behaving monkeys. We obtained a much more distinct classification of functional neuron types in the ventral than the dorsal region of LPFC, suggesting that the action plan is initially generated in ventral LPFC. Our model offers a biologically plausible neural circuit architecture that implements good-to-action transformation during economic choice.