Robust Encoding of Spatial Information in Orbitofrontal Cortex and Striatum

Toward a neuroscience of natural behavior

One of the most exciting new developments in systems neuroscience is the progress being made toward neurophysiological experiments that move beyond simplified laboratory settings and address the richness of natural behavior. This is enabled by technological advances such as wireless recording in freely moving animals, automated quantification of behavior, and new methods for analyzing large data sets. Beyond new empirical methods and data, however, there is also a need for new theories and concepts to interpret that data. Such theories need to address the particular challenges of natural behavior, which often differ significantly from the scenarios studied in traditional laboratory settings. Here, we discuss some strategies for developing such novel theories and concepts and some example hypotheses being proposed.

Behavioral read-out from population value signals in primate orbitofrontal cortex

The primate orbitofrontal cortex (OFC) has long been recognized for its role in value-based decisions; however, the exact mechanism linking value representations in the OFC to decision outcomes has remained elusive. Here, to address this question, we show, in non-human primates, that trial-wise variability in choices can be explained by variability in value signals decoded from many simultaneously recorded OFC neurons. Mechanistically, this relationship is consistent with the projection of activity within a low-dimensional value-encoding subspace onto a potentially higher-dimensional, behaviorally potent output subspace. Identifying this neural-behavioral link answers longstanding questions about the role of the OFC in economic decision-making and suggests population-level read-out mechanisms for the OFC similar to those recently identified in sensory and motor cortex.

Orbitofrontal cortex control of striatum leads economic decision-making

Animals must continually evaluate stimuli in their environment to decide which opportunities to pursue, and in many cases these decisions can be understood in fundamentally economic terms. Although several brain regions have been individually implicated in these processes, the brain-wide mechanisms relating these regions in decision-making are unclear. Using an economic decision-making task adapted for rats, we find that neural activity in both of two connected brain regions, the ventrolateral orbitofrontal cortex (OFC) and the dorsomedial striatum (DMS), was required for economic decision-making. Relevant neural activity in both brain regions was strikingly similar, dominated by the spatial features of the decision-making process. However, the neural encoding of choice direction in OFC preceded that of DMS, and this temporal relationship was strongly correlated with choice accuracy. Furthermore, activity specifically in the OFC projection to the DMS was required for appropriate economic decision-making. These results demonstrate that choice information in the OFC is relayed to the DMS to lead accurate economic decision-making.

Widespread coding of navigational variables in prefrontal cortex

To navigate effectively, we must represent information about our location in the environment. Traditional research highlights the role of the hippocampal complex in this process. Spurred by recent research highlighting the widespread cortical encoding of cognitive and motor variables previously thought to have localized function, we hypothesized that navigational variables would be likewise encoded widely, especially in the prefrontal cortex, which is associated with volitional behavior. We recorded neural activity from six prefrontal regions while macaques performed a foraging task in an open enclosure. In all regions, we found strong encoding of allocentric position, allocentric head direction, boundary distance, and linear and angular velocity. These encodings were not accounted for by distance, time to reward, or motor factors. The strength of coding of all variables increased along a ventral-to-dorsal gradient. Together, these results argue that encoding of navigational variables is not localized to the hippocampus and support the hypothesis that navigation is continuous with other forms of flexible cognition in the service of action.

A structural and functional subdivision in central orbitofrontal cortex

Economic choice requires many cognitive subprocesses, including stimulus detection, valuation, motor output, and outcome monitoring; many of these subprocesses are associated with the central orbitofrontal cortex (cOFC). Prior work has largely assumed that the cOFC is a single region with a single function. Here, we challenge that unified view with convergent anatomical and physiological results from rhesus macaques. Anatomically, we show that the cOFC can be subdivided according to its much stronger (medial) or weaker (lateral) bidirectional anatomical connectivity with the posterior cingulate cortex (PCC). We call these subregions cOFCm and cOFCl, respectively. These two subregions have notable functional differences. Specifically, cOFCm shows enhanced functional connectivity with PCC, as indicated by both spike-field coherence and mutual information. The cOFCm-PCC circuit, but not the cOFCl-PCC circuit, shows signatures of relaying choice signals from a non-spatial comparison framework to a spatially framed organization and shows a putative bidirectional mutually excitatory pattern.

The whole prefrontal cortex is premotor cortex

We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex

Choice-relevant brain regions in prefrontal cortex may progressively transform information about options into choices. Here, we examine responses of neurons in four regions of the medial prefrontal cortex as macaques performed two-option risky choices. All four regions encode economic variables in similar proportions and show similar putative signatures of key choice-related computations. We provide evidence to support a gradient of function that proceeds from areas 14 to 25 to 32 to 24. Specifically, we show that decodability of twelve distinct task variables increases along that path, consistent with the idea that regions that are higher in the anatomical hierarchy make choice-relevant variables more separable. We also show progressively longer intrinsic timescales in the same series. Together these results highlight the importance of the medial wall in choice, endorse a specific gradient-based organization, and argue against a modular functional neuroanatomy of choice.

Multicentric tracking of multiple agents by anterior cingulate cortex during pursuit and evasion

Successful pursuit and evasion require rapid and precise coordination of navigation with adaptive motor control. We hypothesize that the dorsal anterior cingulate cortex (dACC), which communicates bidirectionally with both the hippocampal complex and premotor/motor areas, would serve a mapping role in this process. We recorded responses of dACC ensembles in two macaques performing a joystick-controlled continuous pursuit/evasion task. We find that dACC carries two sets of signals, (1) world-centric variables that together form a representation of the position and velocity of all relevant agents (self, prey, and predator) in the virtual world, and (2) avatar-centric variables, i.e. self-prey distance and angle. Both sets of variables are multiplexed within an overlapping set of neurons. Our results suggest that dACC may contribute to pursuit and evasion by computing and continuously updating a multicentric representation of the unfolding task state, and support the hypothesis that it plays a high-level abstract role in the control of behavior.

Rules warp feature encoding in decision-making circuits

We have the capacity to follow arbitrary stimulus-response rules, meaning simple policies that guide our behavior. Rule identity is broadly encoded across decision-making circuits, but there are less data on how rules shape the computations that lead to choices. One idea is that rules could simplify these computations. When we follow a rule, there is no need to encode or compute information that is irrelevant to the current rule, which could reduce the metabolic or energetic demands of decision-making. However, it is not clear if the brain can actually take advantage of this computational simplicity. To test this idea, we recorded from neurons in 3 regions linked to decision-making, the orbitofrontal cortex (OFC), ventral striatum (VS), and dorsal striatum (DS), while macaques performed a rule-based decision-making task. Rule-based decisions were identified via modeling rules as the latent causes of decisions. This left us with a set of physically identical choices that maximized reward and information, but could not be explained by simple stimulus-response rules. Contrasting rule-based choices with these residual choices revealed that following rules (1) decreased the energetic cost of decision-making; and (2) expanded rule-relevant coding dimensions and compressed rule-irrelevant ones. Together, these results suggest that we use rules, in part, because they reduce the costs of decision-making through a distributed representational warping in decision-making circuits.

Value Neglect: A Critical Role for Ventromedial Frontal Lobe in Learning the Value of Spatial Locations

Whether you are a gazelle bounding to the richest tract of grassland or a return customer heading to the freshest farm stand at a crowded market, the ability to learn the value of spatial locations is important in adaptive behavior. The ventromedial frontal lobe (VMF) is implicated in value-based decisions between objects and in flexibly learning to choose between objects based on feedback. However, it is unclear if this region plays a material-general role in reward learning. Here, we tested whether VMF is necessary for learning the value of spatial locations. People with VMF damage were compared with healthy participants and a control group with frontal damage sparing VMF in an incentivized spatial search task. Participants chose among spatial targets distributed among distractors, rewarded with an expected value that varied along the right-left axis of the screen. People with VMF damage showed a weaker tendency to reap reward in contralesional hemispace. In some individuals, this impairment could be dissociated from the ability to make value-based decisions between objects, assessed separately. This is the first evidence that the VMF is critically involved in reward-guided spatial search and offers a novel perspective on the relationships between value, spatial attention, and decision-making.

Orbitofrontal cortex is selectively activated in a primate model of attentional bias to cocaine cues

Attentional bias to drug-associated cues correlates with extent of current use, and risk of relapse among those attempting abstinence. Electroencephalogram (EEG) and functional imaging measures in clinical studies have previously investigated the neural basis of attentional bias, but the lack of animal models precluded investigation at the single-unit level. To complement results obtained from clinical studies, we have employed a non-human primate model of attentional bias to cocaine cues while simultaneously recording single-unit activity in cortical and striatal regions implicated in reward processing. Rhesus macaques conditioned to associate particular colors with cocaine or water reward performed an attentional bias task, in which those colors served as irrelevant distractors. Concurrently, multiple electrode arrays for recording single-unit activity were acutely implanted into the orbitofrontal cortex, anterior cingulate cortex, dorsal anterior striatum, and ventral striatum. As in clinical studies, attentional bias was indicated by elongated response times on trials with cocaine-associated distractors compared with trials with water-associated, or control unconditioned distractors. In both animals studied, across an unbiased sample of neurons, the orbitofrontal cortex differentiated distractor condition by the proportion of single-units activated, as well as by population response. In one of the two, the anterior cingulate cortex did as well, but neither striatal region did in either animal. These direct measures of single-unit activity in a primate model complement clinical imaging observations suggesting that cortical mechanisms, especially in orbitofrontal cortex, are likely involved in attentional bias to cocaine-associated environmental cues.

The Transition from Evaluation to Selection Involves Neural Subspace Reorganization in Core Reward Regions

Economic choice proceeds from evaluation, in which we contemplate options, to selection, in which we weigh options and choose one. These stages must be differentiated so that decision makers do not proceed to selection before evaluation is complete. We examined responses of neurons in two core reward regions, orbitofrontal (OFC) and ventromedial prefrontal cortex (vmPFC), during two-option choice with asynchronous offer presentation. Our data suggest that neurons selective during the first (presumed evaluation) and second (presumed comparison and selection) offer epochs come from a single pool. Stage transition is accompanied by a shift toward orthogonality in the low-dimensional population response manifold. Nonetheless, the relative position of each option in driving responses in the population subspace is preserved. The orthogonalization we observe supports the hypothesis that the transition from evaluation to selection leads to reorganization of response subspace and suggests a mechanism by which value-related signals are prevented from prematurely driving choice.

The Nucleus Accumbens Core Is Necessary for Responding to Incentive But Not Instructive Stimuli

An abundant literature has highlighted the importance of the nucleus accumbens core (NAcC) in behavioral tasks dependent on external stimuli. Yet, some studies have also reported the absence of involvement of the NAcC in stimuli processing. We aimed at comparing, in male rats, the underlying neuronal determinants of incentive and instructive stimuli in the same task. We developed a variant of a GO/NOGO task that reveals important differences in these two types of stimuli. The incentive stimulus invites the rat to engage in the task sequence. Once the rat has decided to initiate a trial, it remains engaged in the task until the end of the trial. This task revealed the differential contribution of the NAcC to responding to different types of stimuli: responding to the incentive stimulus depended on NAcC AMPA/NMDA and dopamine D1 receptors, but the retrieval of the response associated with the instructive stimuli (lever pressing on GO, withholding on NOGO) did not. Our electrophysiological study showed that more NAcC neurons responded more strongly to the incentive than the instructive stimuli. Furthermore, when animals did not respond to the incentive stimulus, the induced excitation was suppressed for most projection neurons, whereas interneurons were strongly activated at a latency preceding that found in projection neurons. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.SIGNIFICANCE STATEMENT The nucleus accumbens core (NAcC) is essential to process information carried by reward-predicting stimuli. Yet, stimuli have distinct properties: incentive stimuli orient the attention toward reward-seeking, whereas instructive stimuli inform about the action to perform. Our study shows that, in male rats, NAcC perturbation with glutamate or dopamine antagonists impeded responses to the incentive but not to the instructive stimulus. NAcC neuronal recordings revealed a stronger representation of incentive than instructive stimuli. Furthermore, we found that interneurons are recruited when rats fail to respond to incentive stimuli. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.

Tonic exploration governs both flexibility and lapses

In many cognitive tasks, lapses (spontaneous errors) are tacitly dismissed as the result of nuisance processes like sensorimotor noise, fatigue, or disengagement. However, some lapses could also be caused by exploratory noise: randomness in behavior that facilitates learning in changing environments. If so, then strategic processes would need only up-regulate (rather than generate) exploration to adapt to a changing environment. This view predicts that more frequent lapses should be associated with greater flexibility because these behaviors share a common cause. Here, we report that when rhesus macaques performed a set-shifting task, lapse rates were negatively correlated with perseverative error frequency across sessions, consistent with a common basis in exploration. The results could not be explained by local failures to learn. Furthermore, chronic exposure to cocaine, which is known to impair cognitive flexibility, did increase perseverative errors, but, surprisingly, also improved overall set-shifting task performance by reducing lapse rates. We reconcile these results with a state-switching model in which cocaine decreases exploration by deepening attractor basins corresponding to rule states. These results support the idea that exploratory noise contributes to lapses, affecting rule-based decision-making even when it has no strategic value, and suggest that one key mechanism for regulating exploration may be the depth of rule states.

Ventromedial Prefrontal Cortex Tracks Multiple Environmental Variables during Search

To make efficient foraging decisions, we must combine information about the values of available options with nonvalue information. Some accounts of ventromedial PFC (vmPFC) suggest that it has a narrow role limited to evaluating immediately available options. We examined responses of neurons in area 14 (a putative macaque homolog of human vmPFC) as 2 male macaques performed a novel foraging search task. Although many neurons encoded the values of immediately available offers, they also independently encoded several other variables that influence choice, but that are conceptually distinct from offer value. These variables include average reward rate, number of offers viewed per trial, previous offer values, previous outcome sizes, and the locations of the currently attended offer. We conclude that, rather than serving as specialized economic value center, vmPFC plays a broad role in integrating relevant environmental information to drive foraging decisions.SIGNIFICANCE STATEMENT Decision makers must often choose whether to take an immediately available option or continue to search for a better one. We hypothesized that this process, which is integral to foraging theory, leaves neural signatures in the brain region ventromedial PFC. Subjects performed a novel foraging task in which they searched through differently valued options and attempted to balance their reward threshold with various time costs. We found that neurons not only encode the values of immediately available offers, but multiplexed these with environmental variables, including reward rate, number of offers viewed, previous offer values, and spatial information. We conclude that vmPFC plays a rich role in encoding and integrating multiple foraging-related variables during economic decisions.

Why has evolution not selected for perfect self-control?

Self-control refers to the ability to deliberately reject tempting options and instead select ones that produce greater long-term benefits. Although some apparent failures of self-control are, on closer inspection, reward maximizing, at least some self-control failures are clearly disadvantageous and non-strategic. The existence of poor self-control presents an important evolutionary puzzle because there is no obvious reason why good self-control should be more costly than poor self-control. After all, a rock is infinitely patient. I propose that self-control failures result from cases in which well-learned (and thus routinized) decision-making strategies yield suboptimal choices. These mappings persist in the decision-makers' repertoire because they result from learning processes that are adaptive in the broader context, either on the timescale of learning or of evolution. Self-control, then, is a form of cognitive control and the subjective feeling of effort likely reflects the true costs of cognitive control. Poor self-control, in this view, is ultimately a result of bounded optimality. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications.

Economic Choice as an Untangling of Options into Actions

We propose that economic choice can be understood as a gradual transformation from a domain of options to one of the actions. We draw an analogy with the idea of untangling information in the form vision system and propose that form vision and economic choice may be two aspects of a larger process that sculpts actions based on sensory inputs. From this viewpoint, choice results from the accumulated effect of repetitions of simple computations. These may consist primarily of relative valuations (evaluations relative to the value of rejection, perhaps in a manner akin to divisive normalization) applied to individual offers. With regard to economic choice, cortical brain regions differ primarily in their position and in what information they prioritize, and do not-with a few exceptions-have categorically distinct roles. Each region's specific contribution is determined largely by its inputs; thus, understanding connectivity is crucial for understanding choice. This view suggests that there is no single site of choice, that there is no meaningful distinction between pre- and post-decisionality, and that there is no explicit representation of value in the brain.

A neuronal theory of sequential economic choice

Results of recent studies point towards a new framework for the neural bases of economic choice. The principles of this framework include the idea that evaluation is limited to a single option within the focus of attention and that we accept or reject that option relative to the entire set of alternatives. Rejection leads attention to a new option, although it can later switch back to a previously rejected one. The option to which a neuron's firing rate refers is determined dynamically by attention and not stably by labelled lines. Value is always computed relative to the value of rejection. Comparison results not from explicit competition between discrete populations of neurons, but indirectly, as in a horse race, from the fact that the first option whose value crosses a threshold is selected. Consequently, comparison can occur within a single pool of neurons rather than by competition between two or more neuronal populations. The computations that constitute comparison thus occur at multiple levels, including premotor levels, simultaneously (i.e. the brain uses a distributed consensus), and not in discrete stages. This framework suggests a solution to a set of otherwise unresolved neuronal binding problems that result from the need to link options to values, comparisons to actions, and choices to outcomes.