Orbitofrontal neurons acquire responses to 'valueless' Pavlovian cues during unblocking

Adolescent alcohol exposure persistently alters orbitofrontal cortical encoding of Pavlovian conditional stimulus components in female rats

Exposure to alcohol during adolescence impacts cortical and limbic brain regions undergoing maturation. In rodent models, long-term effects on behavior and neurophysiology have been described after adolescent intermittent ethanol (AIE), especially in males. We hypothesized that AIE in female rats increases conditional approach to a reward-predictive cue and corresponding neuronal activity in the orbitofrontal cortex (OFC) and nucleus accumbens (NAc). We evaluated behavior and neuronal firing after AIE (5 g/kg intragastric) or water (CON) in adult female rats. Both AIE and CON groups expressed a ST phenotype, and AIE marginally increased sign-tracking (ST) and decreased goal-tracking (GT) metrics. NAc neurons exhibited phasic firing patterns to the conditional stimulus (CS), with no differences between groups. In contrast, neuronal firing in the OFC of AIE animals was greater at CS onset and offset than in CON animals. During reward omission, OFC responses to CS offset normalized to CON levels, but enhanced OFC firing to CS onset persisted in AIE. We suggest that the enhanced OFC neural activity observed in AIE rats to the CS could contribute to behavioral inflexibility. Ultimately, AIE persistently impacts the neurocircuitry of reward-motivated behavior in female rats.

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and the mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, OFC leads the process of symbolic reinforcement learning in the ventral frontostriatal circuitry.

Increased ventral anterior insular connectivity to sports betting availability indexes problem gambling

With the advent of digital technologies, online sports betting is spurring a fast-growing expansion. In this study, we examined how sports betting availability modulates the brain connectivity of frequent sports bettors with [problem bettors (PB)] or without [non-problem bettors (NPB)] problematic sports betting. We conducted functional connectivity analyses centred on the ventral anterior insular cortex (vAI), a brain region playing a key role in the dynamic interplay between reward-based processes. We re-analysed a dataset on sports betting availability undertaken in PB (n = 30) and NPB (n = 35). Across all participants, we observed that sports betting availability elicited positive vAI coupling with extended clusters of brain activation (encompassing the putamen, cerebellum, occipital, temporal, precentral and central operculum regions) and negative vAI coupling with the orbitofrontal cortex. Between-group analyses showed increased positive vAI coupling in the PB group, as compared with the NPB group, in the left lateral occipital cortex, extending to the left inferior frontal gyrus, the anterior cingulate gyrus and the right frontal pole. Taken together, these results are in line with the central assumptions of triadic models of addictions, which posit that the insular cortex plays a pivotal role in promoting the drive and motivation to get a reward by 'hijacking' goal-oriented processes toward addiction-related cues. Taken together, these findings showed that vAI functional connectivity is sensitive not only to gambling availability but also to the status of problematic sport betting.

Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex

Orbitofrontal cortex (OFC) is classically linked to inhibitory control, emotion regulation, and reward processing. Recent perspectives propose that the OFC also generates predictions about perceptual events, actions, and their outcomes. We tested the role of the OFC in detecting violations of prediction at two levels of abstraction (i.e., hierarchical predictive processing) by studying the event-related potentials (ERPs) of patients with focal OFC lesions (n = 12) and healthy controls (n = 14) while they detected deviant sequences of tones in a local-global paradigm. The structural regularities of the tones were controlled at two hierarchical levels by rules defined at a local (i.e., between tones within sequences) and at a global (i.e., between sequences) level. In OFC patients, ERPs elicited by standard tones were unaffected at both local and global levels compared to controls. However, patients showed an attenuated mismatch negativity (MMN) and P3a to local prediction violation, as well as a diminished MMN followed by a delayed P3a to the combined local and global level prediction violation. The subsequent P3b component to conditions involving violations of prediction at the level of global rules was preserved in the OFC group. Comparable effects were absent in patients with lesions restricted to the lateral PFC, which lends a degree of anatomical specificity to the altered predictive processing resulting from OFC lesion. Overall, the altered magnitudes and time courses of MMN/P3a responses after lesions to the OFC indicate that the neural correlates of detection of auditory regularity violation are impacted at two hierarchical levels of rule abstraction.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

We aim to differentiate the brain regions involved in the learning and encoding of Pavlovian associations sensitive to changes in outcome value from those that are not sensitive to such changes by combining a learning task with outcome devaluation, eye-tracking, and functional magnetic resonance imaging in humans. Contrary to theoretical expectation, voxels correlating with reward prediction errors in the ventral striatum and subgenual cingulate appear to be sensitive to devaluation. Moreover, regions encoding state prediction errors appear to be devaluation insensitive. We can also distinguish regions encoding predictions about outcome taste identity from predictions about expected spatial location. Regions encoding predictions about taste identity seem devaluation sensitive while those encoding predictions about an outcome's spatial location seem devaluation insensitive. These findings suggest the existence of multiple and distinct associative mechanisms in the brain and help identify putative neural correlates for the parallel expression of both devaluation sensitive and insensitive conditioned behaviors.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

Pavlovian learning depends on multiple and parallel associations leading to distinct classes of conditioned responses that vary in their flexibility following changes in the value of an associated outcome. Here, we aimed to differentiate brain areas involved in learning and encoding associations that are sensitive to changes in the value of an outcome from those that are not sensitive to such changes. To address this question, we combined a Pavlovian learning task with outcome devaluation, eye-tracking and functional magnetic resonance imaging. We used computational modeling to identify brain regions involved in learning stimulus-reward associations and stimulus-stimulus associations, by testing for brain areas correlating with reward-prediction errors and state-prediction errors, respectively. We found that, contrary to theoretical predictions about reward prediction errors being exclusively model-free, voxels correlating with reward prediction errors in the ventral striatum and subgenual anterior cingulate cortex were sensitive to devaluation. On the other hand, brain areas correlating with state prediction errors were found to be devaluation insensitive. In a supplementary analysis, we distinguished brain regions encoding predictions about outcome taste identity from those involved in encoding predictions about its expected spatial location. A subset of regions involved in taste identity predictions were devaluation sensitive while those involved in encoding predictions about spatial location were devaluation insensitive. These findings provide insights into the role of multiple associative mechanisms in the brain in mediating Pavlovian conditioned behavior - illustrating how distinct neural pathways can in parallel produce both devaluation sensitive and devaluation insensitive behaviors.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Neural circuits for inference-based decision-making

In novel situations, where direct experience is lacking or outdated, humans must rely on mental simulations to predict future outcomes. This review discusses recent work on the neural circuits that support such inference-based behavior. We focus on two specific examples: 1) using knowledge about the associative structure of the world to infer outcomes when direct experience is lacking; 2) inferring the current value of options when the desirability of the associated outcome has changed since the original learning experience. These two examples can be studied in the sensory preconditioning and devaluation tasks, respectively. We review results from studies in animals and humans suggesting that the orbitofrontal cortex (OFC), together with the hippocampus and amygdala, is necessary for inference in both of these tasks. Together, these findings suggest that the OFC is a critical hub in the brain network that supports inference-based decision-making.

Increased brain reactivity to gambling unavailability as a marker of problem gambling

The unprecedented development and ubiquity of sports betting constitute an emerging public health concern. It is crucial to provide markers that could help to better identify people experiencing sports betting-related harms. The current study investigated whether problem gambling status, sports betting passion, and trait-self-control modulate brain reactivity to sports betting cues. Sixty-five frequent sports bettors (35 "nonproblem bettors" and 30 "problem bettors") were exposed to cues representing real upcoming sport events (with varying levels of winning confidence) that were made available or blocked for betting, during functional magnetic resonance imaging (fMRI) recording. Sports betting passion and trait-self-control were assessed using self-report scales. Sport events nonavailable for betting elicited higher insular and striatal activation in problem bettors, as compared with nonproblem bettors. Within a large cluster encompassing the ventral striatum, hippocampus, and amygdala, lower trait-self-control was associated with increased brain reactivity to sport events with high levels of winning confidence that were nonavailable for betting. No significant effect of sports betting passion was observed. These findings suggest that sports bettors' brain reactivity to gambling unavailability might be a relevant marker of sports betting-related harms, as well as of blunted trait-self-control.

A bidirectional corticoamygdala circuit for the encoding and retrieval of detailed reward memories

Adaptive reward-related decision making often requires accurate and detailed representation of potential available rewards. Environmental reward-predictive stimuli can facilitate these representations, allowing one to infer which specific rewards might be available and choose accordingly. This process relies on encoded relationships between the cues and the sensory-specific details of the rewards they predict. Here, we interrogated the function of the basolateral amygdala (BLA) and its interaction with the lateral orbitofrontal cortex (lOFC) in the ability to learn such stimulus-outcome associations and use these memories to guide decision making. Using optical recording and inhibition approaches, Pavlovian cue-reward conditioning, and the outcome-selective Pavlovian-to-instrumental transfer (PIT) test in male rats, we found that the BLA is robustly activated at the time of stimulus-outcome learning and that this activity is necessary for sensory-specific stimulus-outcome memories to be encoded, so they can subsequently influence reward choices. Direct input from the lOFC was found to support the BLA in this function. Based on prior work, activity in BLA projections back to the lOFC was known to support the use of stimulus-outcome memories to influence decision making. By multiplexing optogenetic and chemogenetic inhibition we performed a serial circuit disconnection and found that the lOFC→BLA and BLA→lOFC pathways form a functional circuit regulating the encoding (lOFC→BLA) and subsequent use (BLA→lOFC) of the stimulus-dependent, sensory-specific reward memories that are critical for adaptive, appetitive decision making.

Neural substrates of appetitive and aversive prediction error

Prediction error, defined by the discrepancy between real and expected outcomes, lies at the core of associative learning. Behavioural investigations have provided evidence that prediction error up- and down-regulates associative relationships, and allocates attention to stimuli to enable learning. These behavioural advances have recently been followed by investigations into the neurobiological substrates of prediction error. In the present paper, we review neuroscience data obtained using causal and recording neural methods from a variety of key behavioural designs. We explore the neurobiology of both appetitive (reward) and aversive (fear) prediction error with a focus on the mesolimbic dopamine system, the amygdala, ventrolateral periaqueductal gray, hippocampus, cortex and locus coeruleus noradrenaline. New questions and avenues for research are considered.

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

To be specific: The role of orbitofrontal cortex in signaling reward identity

The orbitofrontal cortex (OFC) plays a prominent role in signaling reward expectations. Two important features of rewards are their value (how good they are) and their specific identity (what they are). Whereas research on OFC has traditionally focused on reward value, recent findings point toward a pivotal role of reward identity in understanding OFC signaling and its contribution to behavior. Here, we review work in rodents, nonhuman primates, and humans on how the OFC represents expectations about the identity of rewards, and how these signals contribute to outcome-guided behavior. Moreover, we summarize recent findings suggesting that specific reward expectations in OFC are learned and updated by means of identity errors in the dopaminergic midbrain. We conclude by discussing how OFC encoding of specific rewards complements recent proposals that this region represents a cognitive map of relevant task states, which forms the basis for model-based behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The Role of the Rodent Lateral Orbitofrontal Cortex in Simple Pavlovian Cue-Outcome Learning Depends on Training Experience

The orbitofrontal cortex (OFC) is a critical structure in the flexible control of value-based behaviors. OFC dysfunction is typically only detected when task or environmental contingencies change, against a backdrop of apparently intact initial acquisition and behavior. While intact acquisition following OFC lesions in simple Pavlovian cue-outcome conditioning is often predicted by models of OFC function, this predicted null effect has not been thoroughly investigated. Here, we test the effects of lesions and temporary muscimol inactivation of the rodent lateral OFC on the acquisition of a simple single cue-outcome relationship. Surprisingly, pretraining lesions significantly enhanced acquisition after overtraining, whereas post-training lesions and inactivation significantly impaired acquisition. This impaired acquisition to the cue reflects a disruption of behavioral control and not learning since the cue could also act as an effective blocking stimulus in an associative blocking procedure. These findings suggest that even simple cue-outcome representations acquired in the absence of OFC function are impoverished. Therefore, while OFC function is often associated with flexible behavioral control in complex environments, it is also involved in very simple Pavlovian acquisition where complex cue-outcome relationships are irrelevant to task performance.

Review of Orbitofrontal Cortex in Alcohol Dependence: A Disrupted Cognitive Map?

Alcoholism is a persistent worldwide problem associated with long-lasting impairments to decision making processes. Some aspects of dysfunction are thought to reflect alcohol-induced changes to relevant brain areas such as the orbitofrontal cortex (OFC). In this review, we will examine how chronic alcohol exposure alters OFC function to potentially contribute to maladaptive decision making, and explore experimental behavioral approaches that may be better suited to test whether alcohol dependence disrupts OFC's function. We argue that although past works suggest impairments in aspects of OFC function, more information may be gained by specifically targeting tasks to the broader function of OFC as put forth by the recent hypothesis of OFC as a "cognitive map" of task space. Overall, we suggest that such a focus could provide a better understanding of how OFC function changes in alcohol dependence, and could inform better assessment tools and treatment options for clinicians working with this population.

Orbitofrontal Cortex Encodes Preference for Alcohol

Orbitofrontal cortex (OFC) plays a key role in representation and regulation of reward value, preference, and seeking. OFC function is disrupted in drug use and dependence, but its specific role in alcohol use disorders has not been thoroughly studied. In alcohol-dependent humans OFC activity is increased by alcohol cue presentation. Ethanol (EtOH) also alters OFC neuron excitability in vitro, and OFC manipulation influences EtOH seeking and drinking in rodents.

Orbitofrontal cortex (OFC) plays a key role in representation and regulation of reward value, preference, and seeking. OFC function is disrupted in drug use and dependence, but its specific role in alcohol use disorders has not been thoroughly studied. In alcohol-dependent humans OFC activity is increased by alcohol cue presentation. Ethanol (EtOH) also alters OFC neuron excitability in vitro, and OFC manipulation influences EtOH seeking and drinking in rodents. To understand the relationship between OFC function and individual alcohol use, we recorded OFC neuron activity in rats during EtOH self-administration, characterizing the neural correlates of individual preference for alcohol. After one month of intermittent access to 20% EtOH, male Long–Evans rats were trained to self-administer 20% EtOH, 10% EtOH, and 15% sucrose. OFC neuronal activity was recorded and associated with task performance and EtOH preference. Rats segregated into high and low EtOH drinkers based on homecage consumption and operant seeking of 20% EtOH. Motivation for 10% EtOH and sucrose was equally high in both groups. OFC neuronal activity was robustly increased or decreased during sucrose and EtOH seeking and consumption, and strength of changes in OFC activity was directly associated with individual preference for 20% EtOH. EtOH-associated OFC activity was more similar to sucrose-associated activity in high versus low EtOH drinkers. The results show that OFC neurons are activated during alcohol seeking based on individual preference, supporting this brain region as a potential substrate for alcohol motivation that may be dysregulated in alcohol misuse.

Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats' ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

Neural mechanisms of economic choices in mice

Economic choices entail computing and comparing subjective values. Evidence from primates indicates that this behavior relies on the orbitofrontal cortex. Conversely, previous work in rodents provided conflicting results. Here we present a mouse model of economic choice behavior, and we show that the lateral orbital (LO) area is intimately related to the decision process. In the experiments, mice chose between different juices offered in variable amounts. Choice patterns closely resembled those measured in primates. Optogenetic inactivation of LO dramatically disrupted choices by inducing erratic changes of relative value and by increasing choice variability. Neuronal recordings revealed that different groups of cells encoded the values of individual options, the binary choice outcome and the chosen value. These groups match those previously identified in primates, except that the neuronal representation in mice is spatial (in monkeys it is good-based). Our results lay the foundations for a circuit-level analysis of economic decisions.

The serial blocking effect: a testbed for the neural mechanisms of temporal-difference learning

Temporal-difference (TD) learning models afford the neuroscientist a theory-driven roadmap in the quest for the neural mechanisms of reinforcement learning. The application of these models to understanding the role of phasic midbrain dopaminergic responses in reward prediction learning constitutes one of the greatest success stories in behavioural and cognitive neuroscience. Critically, the classic learning paradigms associated with TD are poorly suited to cast light on its neural implementation, thus hampering progress. Here, we present a serial blocking paradigm in rodents that overcomes these limitations and allows for the simultaneous investigation of two cardinal TD tenets; namely, that learning depends on the computation of a prediction error, and that reinforcing value, whether intrinsic or acquired, propagates back to the onset of the earliest reliable predictor. The implications of this paradigm for the neural exploration of TD mechanisms are highlighted.

Rat Orbitofrontal Ensemble Activity Contains Multiplexed but Dissociable Representations of Value and Task Structure in an Odor Sequence Task

The orbitofrontal cortex (OFC) has long been implicated in signaling information about expected outcomes to facilitate adaptive or flexible behavior. Current proposals focus on signaling of expected value versus the representation of a value-agnostic cognitive map of the task. While often suggested as mutually exclusive, these alternatives may represent extreme ends of a continuum determined by task complexity and experience. As learning proceeds, an initial, detailed cognitive map might be acquired, based largely on external information. With more experience, this hypothesized map can then be tailored to include relevant abstract hidden cognitive constructs. The map would default to an expected value in situations where other attributes are largely irrelevant, but, in richer tasks, a more detailed structure might continue to be represented, at least where relevant to behavior. Here, we examined this by recording single-unit activity from the OFC in rats navigating an odor sequence task analogous to a spatial maze. The odor sequences provided a mappable state space, with 24 unique "positions" defined by sensory information, likelihood of reward, or both. Consistent with the hypothesis that the OFC represents a cognitive map tailored to the subjects' intentions or plans, we found a close correspondence between how subjects were using the sequences and the neural representations of the sequences in OFC ensembles. Multiplexed with this value-invariant representation of the task, we also found a representation of the expected value at each location. Thus, the value and task structure co-existed as dissociable components of the neural code in OFC.

The role of the orbitofrontal cortex in alcohol use, abuse, and dependence

One of the major functions of the orbitofrontal cortex (OFC) is to promote flexible motivated behavior. It is no surprise, therefore, that recent work has demonstrated a prominent impact of chronic drug use on the OFC and a potential role for OFC disruption in drug abuse and addiction. Among drugs of abuse, the use of alcohol is particularly salient with respect to OFC function. Although a number of studies in humans have implicated OFC dysregulation in alcohol use disorders, animal models investigating the association between OFC and alcohol use are only beginning to be developed, and there is still a great deal to be revealed. The goal of this review is to consider what is currently known regarding the role of the OFC in alcohol use and dependence. I will first provide a brief, general overview of current views of OFC function and its contributions to drug seeking and addiction. I will then discuss research to date related to the OFC and alcohol use, both in human clinical populations and in non-human models. Finally I will consider issues and strategies to guide future study that may identify this brain region as a key player in the transition from moderated to problematic alcohol use and dependence.

Medial orbitofrontal inactivation does not affect economic choice

How are decisions made between different goods? One theory spanning several fields of neuroscience proposes that their values are distilled to a single common neural currency, the calculation of which allows for rational decisions. The orbitofrontal cortex (OFC) is thought to play a critical role in this process, based on the presence of neural correlates of economic value in lateral OFC in monkeys and medial OFC in humans. We previously inactivated lateral OFC in rats without affecting economic choice behavior. Here we inactivated medial OFC in the same task, again without effect. Behavior in the same rats was disrupted by inactivation during progressive ratio responding previously shown to depend on medial OFC, demonstrating the efficacy of the inactivation. These results indicate that medial OFC is not necessary for economic choice, bolstering the proposal that classic economic choice is likely mediated by multiple, overlapping neural circuits.

Orbital frontal cortex updates state-induced value change for decision-making

Recent hypotheses have posited that orbital frontal cortex (OFC) is important for using inferred consequences to guide behavior. Less clear is OFC's contribution to goal-directed or model-based behavior, where the decision to act is controlled by previous experience with the consequence or outcome. Investigating OFC's role in learning about changed outcomes separate from decision-making is not trivial and often the two are confounded. Here we adapted an incentive learning task to mice, where we investigated processes controlling experience-based outcome updating independent from inferred action control. We found chemogenetic OFC attenuation did not alter the ability to perceive motivational state-induced changes in outcome value but did prevent the experience-based updating of this change. Optogenetic inhibition of OFC excitatory neuron activity selectively when experiencing an outcome change disrupted the ability to update, leaving mice unable to infer the appropriate behavior. Our findings support a role for OFC in learning that controls decision-making.

Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex

There is general consensus that dopaminergic midbrain neurons signal reward prediction errors, computed as the difference between expected and received reward value. However, recent work in rodents shows that these neurons also respond to errors related to inferred value and sensory features, indicating an expanded role for dopamine beyond learning cached values. Here we utilize a transreinforcer reversal learning task and functional magnetic resonance imaging (fMRI) to test whether prediction error signals in the human midbrain are evoked when the expected identity of an appetitive food odor reward is violated, while leaving value matched. We found that midbrain fMRI responses to identity and value errors are correlated, suggesting a common neural origin for these error signals. Moreover, changes in reward-identity expectations, encoded in the orbitofrontal cortex (OFC), are directly related to midbrain activity, demonstrating that identity-based error signals in the midbrain support the formation of outcome identity expectations in OFC.

Responses in the dopaminergic midbrain are known to signal prediction errors for reward value. Here, the authors show that the human midbrain also encodes errors in predicted reward identity, and that these signals update expectations of reward identity in the orbitofrontal cortex.

Solving the Credit Assignment Problem With the Prefrontal Cortex

In naturalistic multi-cue and multi-step learning tasks, where outcomes of behavior are delayed in time, discovering which choices are responsible for rewards can present a challenge, known as the credit assignment problem. In this review, I summarize recent work that highlighted a critical role for the prefrontal cortex (PFC) in assigning credit where it is due in tasks where only a few of the multitude of cues or choices are relevant to the final outcome of behavior. Collectively, these investigations have provided compelling support for specialized roles of the orbitofrontal (OFC), anterior cingulate (ACC), and dorsolateral prefrontal (dlPFC) cortices in contingent learning. However, recent work has similarly revealed shared contributions and emphasized rich and heterogeneous response properties of neurons in these brain regions. Such functional overlap is not surprising given the complexity of reciprocal projections spanning the PFC. In the concluding section, I overview the evidence suggesting that the OFC, ACC and dlPFC communicate extensively, sharing the information about presented options, executed decisions and received rewards, which enables them to assign credit for outcomes to choices on which they are contingent. This account suggests that lesion or inactivation/inhibition experiments targeting a localized PFC subregion will be insufficient to gain a fine-grained understanding of credit assignment during learning and instead poses refined questions for future research, shifting the focus from focal manipulations to experimental techniques targeting cortico-cortical projections.

Orbitofrontal neurons signal reward predictions, not reward prediction errors

Neurons in the orbitofrontal cortex (OFC) fire in anticipation of and during rewards. Such firing has been suggested to encode reward predictions and to account in some way for the role of this area in adaptive behavior and learning. However, it has also been reported that neural activity in OFC reflects reward prediction errors, which might drive learning directly. Here we tested this question by analyzing the firing of OFC neurons recorded in an odor discrimination task in which rats were trained to sample odor cues and respond left or right on each trial for reward. Neurons were recorded across blocks of trials in which we switched either the number or the flavor of the reward delivered in each well. Previously we have described how neurons in this dataset fired to the predictive cues (Stalnaker et al., 2014); here we focused on the firing in anticipation of and just after delivery of each drop of reward, looking specifically for differences in firing based on whether the reward number or flavor was unexpected or expected. Unlike dopamine neurons recorded in this setting, which exhibited phasic error-like responses after surprising changes in either reward number or reward flavor (Takahashi et al., 2017), OFC neurons showed no such error correlates and instead fired in a way that reflected reward predictions.

Lateral Orbitofrontal Inactivation Dissociates Devaluation-Sensitive Behavior and Economic Choice

How do we choose between goods that have different subjective values, like apples and oranges? Neuroeconomics proposes that this is done by reducing complex goods to a single unitary value to allow comparison. This value is computed "on the fly" from the underlying model of the goods space, allowing decisions to meet current needs. This is termed "model-based" behavior to distinguish it from pre-determined, habitual, or "model-free" behavior. The lateral orbitofrontal cortex (OFC) supports model-based behavior in rats and primates, but whether the OFC is necessary for economic choice is less clear. Here we tested this question by optogenetically inactivating the lateral OFC in rats in a classic model-based task and during economic choice. Contrary to predictions, inactivation disrupted model-based behavior without affecting economic choice.

Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as "OFC," and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Human Orbitofrontal Cortex Represents a Cognitive Map of State Space

Although the orbitofrontal cortex (OFC) has been studied intensely for decades, its precise functions have remained elusive. We recently hypothesized that the OFC contains a "cognitive map" of task space in which the current state of the task is represented, and this representation is especially critical for behavior when states are unobservable from sensory input. To test this idea, we apply pattern-classification techniques to neuroimaging data from humans performing a decision-making task with 16 states. We show that unobservable task states can be decoded from activity in OFC, and decoding accuracy is related to task performance and the occurrence of individual behavioral errors. Moreover, similarity between the neural representations of consecutive states correlates with behavioral accuracy in corresponding state transitions. These results support the idea that OFC represents a cognitive map of task space and establish the feasibility of decoding state representations in humans using non-invasive neuroimaging.

Basolateral Amygdala to Orbitofrontal Cortex Projections Enable Cue-Triggered Reward Expectations

To make an appropriate decision, one must anticipate potential future rewarding events, even when they are not readily observable. These expectations are generated by using observable information (e.g., stimuli or available actions) to retrieve often quite detailed memories of available rewards. The basolateral amygdala (BLA) and orbitofrontal cortex (OFC) are two reciprocally connected key nodes in the circuitry supporting such outcome-guided behaviors. But there is much unknown about the contribution of this circuit to decision making, and almost nothing known about the whether any contribution is via direct, monosynaptic projections, or the direction of information transfer. Therefore, here we used designer receptor-mediated inactivation of OFC→BLA or BLA→OFC projections to evaluate their respective contributions to outcome-guided behaviors in rats. Inactivation of BLA terminals in the OFC, but not OFC terminals in the BLA, disrupted the selective motivating influence of cue-triggered reward representations over reward-seeking decisions as assayed by Pavlovian-to-instrumental transfer. BLA→OFC projections were also required when a cued reward representation was used to modify Pavlovian conditional goal-approach responses according to the reward's current value. These projections were not necessary when actions were guided by reward expectations generated based on learned action-reward contingencies, or when rewards themselves, rather than stored memories, directed action. These data demonstrate that BLA→OFC projections enable the cue-triggered reward expectations that can motivate the execution of specific action plans and allow adaptive conditional responding.SIGNIFICANCE STATEMENT Deficits anticipating potential future rewarding events are associated with many psychiatric diseases. Presently, we know little about the neural circuits supporting such reward expectation. Here we show that basolateral amygdala to orbitofrontal cortex projections are required for expectations of specific available rewards to influence reward seeking and decision making. The necessity of these projections was limited to situations in which expectations were elicited by reward-predictive cues. These projections therefore facilitate adaptive behavior by enabling the orbitofrontal cortex to use environmental stimuli to generate expectations of potential future rewarding events.

Medial Orbitofrontal Neurons Preferentially Signal Cues Predicting Changes in Reward during Unblocking

The orbitofrontal cortex (OFC) has been broadly implicated in the ability to use the current value of expected outcomes to guide behavior. Although value correlates have been prominently reported in lateral OFC, they are more often associated with more medial areas. Further, recent studies in primates have suggested a dissociation in which the lateral OFC is involved in credit assignment and representation of reward identity and more medial areas are critical to representing value. Previously, we used unblocking to test more specifically what information about outcomes is represented by OFC neurons in rats; consistent with the proposed dichotomy between the lateral and medial OFC, we found relatively little linear value coding in the lateral OFC (Lopatina et al., 2015). Here we have repeated this experiment, recording in the medial OFC, to test whether such value signals might be found there. Neurons were recorded in an unblocking task as rats learned about cues that signaled either more, less, or the same amount of reward. We found that medial OFC neurons acquired responses to these cues; however, these responses did not signal different reward values across cues. Surprisingly, we found that cells developed responses to cues predicting a change, particularly a decrease, in reward value. This is consistent with a special role for medial OFC in representing current value to support devaluation/revaluation sensitive changes in behavior.

SIGNIFICANCE STATEMENT

This study uniquely examines encoding in rodent mOFC at the single-unit level in response to cues that predict more, less, or no change in reward in rats during training in a Pavlovian unblocking task, finding more cells responding to change-predictive cues and stronger activity in response to cues predictive of less reward.

A distributed, hierarchical and recurrent framework for reward-based choice

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.

Identity-Specific Reward Representations in Orbitofrontal Cortex Are Modulated by Selective Devaluation

Goal-directed behavior is sensitive to the current value of expected outcomes. This requires independent representations of specific rewards, which have been linked to orbitofrontal cortex (OFC) function. However, the mechanisms by which the human brain updates specific goals on the fly, and translates those updates into choices, have remained unknown. Here we implemented selective devaluation of appetizing food odors in combination with pattern-based neuroimaging and a decision-making task. We found that in a hungry state, participants chose to smell high-intensity versions of two value-matched food odor rewards. After eating a meal corresponding to one of the two odors, participants switched choices toward the low intensity of the sated odor but continued to choose the high intensity of the nonsated odor. This sensory-specific behavioral effect was mirrored by pattern-based changes in fMRI signal in lateral posterior OFC, where specific reward identity representations were altered after the meal for the sated food odor but retained for the nonsated counterpart. In addition, changes in functional connectivity between the OFC and general value coding in ventromedial prefrontal cortex (vmPFC) predicted individual differences in satiety-related choice behavior. These findings demonstrate how flexible representations of specific rewards in the OFC are updated by devaluation, and how functional connections to vmPFC reflect the current value of outcomes and guide goal-directed behavior.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) is critical for goal-directed behavior. A recent proposal is that OFC fulfills this function by representing a variety of state and task variables ("cognitive maps"), including a conjunction of expected reward identity and value. Here we tested how identity-specific representations of food odor reward are updated by satiety. We found that fMRI pattern-based signatures of reward identity in lateral posterior OFC were modulated after selective devaluation, and that connectivity between this region and general value coding ventromedial prefrontal cortex (vmPFC) predicted choice behavior. These results provide evidence for a mechanism by which devaluation modulates a cognitive map of expected reward in OFC and thereby alters general value signals in vmPFC to guide goal-directed behavior.

Thinking Outside the Box: Orbitofrontal Cortex, Imagination, and How We Can Treat Addiction

Addiction involves an inability to control drug-seeking behavior. While this may be thought of as secondary to an overwhelming desire for drugs, it could equally well reflect a failure of the brain mechanisms that allow addicts to learn about and mentally simulate non-drug consequences. Importantly, this process of mental simulation draws upon, but is not normally bound by, our past experiences. Rather we have the ability to think outside the box of our past, integrating knowledge gained from a variety of similar and not-so-similar life experiences to derive estimates or imagine what might happen next. These estimates influence our current behavior directly and also affect future behavior by serving as the background against which outcomes are evaluated to support learning. Here we will review evidence, from our own work using a Pavlovian over-expectation task as well as from other sources, that the orbitofrontal cortex is a critical node in the neural circuit that generates these estimates. Further we will offer the specific hypothesis that degradation of this function secondary to drug-induced changes is a critical and likely addressable part of addiction.

Evidence of a role for orbital prefrontal cortex in preventing over-generalization to moderate predictors of biologically significant events

The mammalian brain is specialized to acquire information about environmental predictors of biologically significant events. However, environments contain an array of stimuli from which animals must ascertain which ones are meaningful in the current situation. This kind of uncertainty is inherent in the discriminative fear conditioning to context task (DFCTC) during which rats are trained to associate one context with foot-shock and another distinct context with no event. Although the contexts differ on several dimensions, they also share similarities making some cues perfect predictors, but others moderate predictors. Appropriate responding requires animals to determine which cues are relevant in the current situation and the ability to constrain their responses only to those perfect predictors. The orbital prefrontal cortex (OPFC) is thought to modulate this function as OPFC lesions result in over-generalization during DFCTC. Two experiments were conducted; the first was intended to dissociate the role of the OPFC in acquisition and expression of DFCTC, and the second intended to determine if the OPFC will also function to constrain responses during an appetitive version of DFCTC. We found that inactivation of the OPFC prior to assessment measures resulted in generalized responses on the appetitive and aversive task, however, these effects may be more prominent during the aversive task. Despite generalization during activity testing, rats were able to discriminate between the two contexts during preference. These results point to a broader role for the OPFC constraining responses to perfect predictors of biologically significant events in uncertain contexts.

Neural Mechanisms of Credit Assignment in a Multicue Environment

In complex environments, many potential cues can guide a decision or be assigned responsibility for the outcome of the decision. We know little, however, about how humans and animals select relevant information sources that should guide behavior. We show that subjects solve this relevance selection and credit assignment problem by selecting one cue and its association with a particular outcome as the main focus of a hypothesis. To do this, we examined learning while using a task design that allowed us to estimate the focus of each subject's hypotheses on a trial-by-trial basis. When a prediction is confirmed by the outcome, then credit for the outcome is assigned to that cue rather than an alternative. Activity in medial frontal cortex is associated with the assignment of credit to the cue that is the main focus of the hypothesis. However, when the outcome disconfirms a prediction, the focus shifts between cues, and the credit for the outcome is assigned to an alternative cue. This process of reselection for credit assignment to an alternative cue is associated with lateral orbitofrontal cortex.

SIGNIFICANCE STATEMENT

Learners should infer which features of environments are predictive of significant events, such as rewards. This "credit assignment" problem is particularly challenging when any of several cues might be predictive. We show that human subjects solve the credit assignment problem by implicitly "hypothesizing" which cue is relevant for predicting subsequent outcomes, and then credit is assigned according to this hypothesis. This process is associated with a distinctive pattern of activity in a part of medial frontal cortex. By contrast, when unexpected outcomes occur, hypotheses are redirected toward alternative cues, and this process is associated with activity in lateral orbitofrontal cortex.

Neuronal remapping and circuit persistence in economic decisions

The orbitofrontal cortex plays a central role in good-based economic decisions. When subjects make choices, neurons in this region represent the identities and values of offered and chosen goods. Notably, choices in different behavioral contexts may involve a potentially infinite variety of goods. Thus a fundamental question concerns the stability versus flexibility of the decision circuit. Here we show in rhesus monkeys that neurons encoding the identity or the subjective value of particular goods in a given context 'remap' and become associated with different goods when the context changes. At the same time, the overall organization of the decision circuit and the function of individual cells remain stable across contexts. In particular, two neurons supporting the same decision in one context also support the same decision in different contexts. These results demonstrate how the same neural circuit can underlie economic decisions involving a large variety of goods.

Back to basics: Making predictions in the orbitofrontal-amygdala circuit

Underlying many complex behaviors are simple learned associations that allow humans and animals to anticipate the consequences of their actions. The orbitofrontal cortex and basolateral amygdala are two regions which are crucial to this process. In this review, we go back to basics and discuss the literature implicating both these regions in simple paradigms requiring the development of associations between stimuli and the motivationally-significant outcomes they predict. Much of the functional research surrounding this ability has suggested that the orbitofrontal cortex and basolateral amygdala play very similar roles in making these predictions. However, electrophysiological data demonstrates critical differences in the way neurons in these regions respond to predictive cues, revealing a difference in their functional role. On the basis of these data and theories that have come before, we propose that the basolateral amygdala is integral to updating information about cue-outcome contingencies whereas the orbitofrontal cortex is critical to forming a wider network of past and present associations that are called upon by the basolateral amygdala to benefit future learning episodes. The tendency for orbitofrontal neurons to encode past and present contingencies in distinct neuronal populations may facilitate its role in the formation of complex, high-dimensional state-specific associations.

Consequences of Adolescent Ethanol Consumption on Risk Preference and Orbitofrontal Cortex Encoding of Reward

Critical development of the prefrontal cortex occurs during adolescence, a period of increased independence marked by decision making that often includes engagement in risky behaviors, such as substance use. Consumption of alcohol during adolescence has been associated with increased impulsivity that persists across the lifespan, an effect which may be caused by long-term disruptions in cortical processing of rewards. To determine if alcohol consumption alters cortical encoding of rewards of different sizes and probabilities, we gave rats limited access to alcohol in gelatin during adolescence only. In adulthood, we recorded the electrophysiological activity of individual neurons of the orbitofrontal cortex while rats performed a risk task that varied the level of risk from day-to-day. Rats that had consumed higher levels of alcohol showed increased risk preference in the task compared with control and low alcohol-consuming rats. Patterns of neuronal responses were identified using principal component analysis. Of the multiple patterns observed, only one was modulated by adolescent alcohol consumption and showed strongest modulation after reward receipt. This subpopulation of neurons showed blunted firing rates following rewards in alcohol-consuming rats, suggesting a mechanism through which adolescent alcohol exposure may have lasting effects on reward processing in the context of decision making. The differences in OFC responses between high alcohol consumers and control animals not given access to alcohol support the idea that, regardless of potential variability in innate alcohol preferences, voluntary consumption of alcohol during adolescence biases choice patterns longitudinally through alterations in cortical function.

Mesolimbic Dopamine Encodes Prediction Errors in a State-Dependent Manner

Mesolimbic dopamine encodes the benefits of a course of action. However, the value of an appetitive reward depends strongly on an animal’s current state. To investigate the relationship between dopamine, value, and physiological state, we monitored sub-second dopamine release in the nucleus accumbens core while rats made choices between food and sucrose solution following selective satiation on one of these reinforcers. Dopamine signals reflected preference for the reinforcers in the new state, decreasing to the devalued reward and, after satiation on food, increasing for the valued sucrose solution. These changes were rapid and selective, with dopamine release returning to pre-satiation patterns when the animals were re-tested in a standard food-restricted state. Such rapid and selective adaptation of dopamine-associated value signals could provide an important signal to promote efficient foraging for a varied diet.

•

Dopamine reward prediction errors are shaped by physiological state

•

Both choices and dopamine signals rapidly update after selective satiation

•

In a new state, dopamine signals mainly only update with experience

•

When returning to a familiar state, dopamine immediately signals stored values

Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation

Prolonged mesolimbic dopamine concentration changes have been detected during spatial navigation, but little is known about the conditions that engender this signaling profile or how it develops with learning. To address this, we monitored dopamine concentration changes in the nucleus accumbens core of rats throughout acquisition and performance of an instrumental action sequence task. Prolonged dopamine concentration changes were detected that ramped up as rats executed each action sequence and declined after earned reward collection. With learning, dopamine concentration began to rise increasingly earlier in the execution of the sequence and ultimately backpropagated away from stereotyped sequence actions, becoming only transiently elevated by the most distal and unexpected reward predictor. Action sequence-related dopamine signaling was reactivated in well-trained rats if they became disengaged in the task and in response to an unexpected change in the value, but not identity of the earned reward. Throughout training and test, dopamine signaling correlated with sequence performance. These results suggest that action sequences can engender a prolonged mode of dopamine signaling in the nucleus accumbens core and that such signaling relates to elements of the motivation underlying sequence execution and is dynamic with learning, overtraining and violations in reward expectation.

Lateral orbitofrontal neurons acquire responses to upshifted, downshifted, or blocked cues during unblocking

The lateral orbitofrontal cortex (lOFC) has been described as signaling either outcome expectancies or value. Previously, we used unblocking to show that lOFC neurons respond to a predictive cue signaling a ‘valueless’ change in outcome features (McDannald, 2014). However, many lOFC neurons also fired to a cue that simply signaled more reward. Here, we recorded lOFC neurons in a variant of this task in which rats learned about cues that signaled either more (upshift), less (downshift) or the same (blocked) amount of reward. We found that neurons acquired responses specifically to one of the three cues and did not fire to the other two. These results show that, at least early in learning, lOFC neurons fire to valued cues in a way that is more consistent with signaling of the predicted outcome’s features than with signaling of a general, abstract or cached value that is independent of the outcome.

DOI:http://dx.doi.org/10.7554/eLife.11299.001

The Origins and Organization of Vertebrate Pavlovian Conditioning

Pavlovian conditioning is the process by which we learn relationships between stimuli and thus constitutes a basic building block for how the brain constructs representations of the world. We first review the major concepts of Pavlovian conditioning and point out many of the pervasive misunderstandings about just what conditioning is. This brings us to a modern redefinition of conditioning as the process whereby experience with a conditional relationship between stimuli bestows these stimuli with the ability to promote adaptive behavior patterns that did not occur before the experience. Working from this framework, we provide an in-depth analysis of two examples, fear conditioning and food-based appetitive conditioning, which include a description of the only partially overlapping neural circuitry of each. We also describe how these circuits promote the basic characteristics that define Pavlovian conditioning, such as error-correction-driven regulation of learning.

Basolateral amygdala rapid glutamate release encodes an outcome-specific representation vital for reward-predictive cues to selectively invigorate reward-seeking actions

Environmental stimuli have the ability to generate specific representations of the rewards they predict and in so doing alter the selection and performance of reward-seeking actions. The basolateral amygdala participates in this process, but precisely how is unknown. To rectify this, we monitored, in near-real time, basolateral amygdala glutamate concentration changes during a test of the ability of reward-predictive cues to influence reward-seeking actions (Pavlovian-instrumental transfer). Glutamate concentration was found to be transiently elevated around instrumental reward seeking. During the Pavlovian-instrumental transfer test these glutamate transients were time-locked to and correlated with only those actions invigorated by outcome-specific motivational information provided by the reward-predictive stimulus (i.e., actions earning the same specific outcome as predicted by the presented CS). In addition, basolateral amygdala AMPA, but not NMDA glutamate receptor inactivation abolished the selective excitatory influence of reward-predictive cues over reward seeking. These data support [corrected] the hypothesis that transient glutamate release in the BLA can encode the outcome-specific motivational information provided by reward-predictive stimuli.

What the orbitofrontal cortex does not do

The number of papers about the orbitofrontal cortex (OFC) has grown from 1 per month in 1987 to a current rate of over 50 per month. This publication stream has implicated the OFC in nearly every function known to cognitive neuroscience and in most neuropsychiatric diseases. However, new ideas about OFC function are typically based on limited data sets and often ignore or minimize competing ideas or contradictory findings. Yet true progress in our understanding of an area's function comes as much from invalidating existing ideas as proposing new ones. Here we consider the proposed roles for OFC, critically examining the level of support for these claims and highlighting the data that call them into question.

Representations of Value in the Brain: An Embarrassment of Riches?

Over the past two decades, neuroscientists have increasingly turned their attention to the question of how the brain implements decisions between differently valued options. This emerging field, called neuroeconomics, has made quick progress in identifying a plethora of brain areas that track or are modulated by reward value. However, it is still unclear how and where in the brain value coding takes place. A primate study by Strait and colleagues in this issue of PLOS Biology finds overlapping signals of value coding in two brain regions central to the valuation process: the ventromedial prefrontal cortex and the ventral striatum. This finding reconciles the primate and rodent literatures, provides valuable insight into the complexity of value computation, and helps set the agenda for future work in this area.

Prospective coding of dorsal raphe reward signals by the orbitofrontal cortex

The orbitofrontal cortex (OFC) is important for the cognitive processes of learning and decision making. Previous recordings have revealed that OFC neurons encode predictions of reward outcomes. The OFC is interconnected with the dorsal raphe nucleus (DRN), which is a major serotonin (5-HT) center of the brain. Recent studies have provided increasing evidence that the DRN encodes reward signals. However, it remains unclear how the activity of DRN neurons affects the prospective reward coding of OFC neurons. By combining single-unit recordings from the OFC and optogenetic activation of the DRN in behaving mice, we found that DRN stimulation is sufficient to organize and modulate the anticipatory responses of OFC neurons. During pavlovian conditioning tasks for mice, odorant cues were associated with the delayed delivery of natural rewards of sucrose solution or DRN stimulation. After training, OFC neurons exhibited prospective responses to the sucrose solution. More importantly, the coupling of an odorant with delayed DRN stimulation resulted in tonic excitation or inhibition of OFC neurons during the delay period. The intensity of the prospective responses was affected by the frequency and duration of DRN stimulation. Additionally, DRN stimulation bidirectionally modulated the prospective responses to natural rewards. These experiments indicate that signals from the DRN are incorporated into the brain reward system to shape the cortical prospective coding of rewards.

Identity-specific coding of future rewards in the human orbitofrontal cortex

Nervous systems must encode information about the identity of expected outcomes to make adaptive decisions. However, the neural mechanisms underlying identity-specific value signaling remain poorly understood. By manipulating the value and identity of appetizing food odors in a pattern-based imaging paradigm of human classical conditioning, we were able to identify dissociable predictive representations of identity-specific reward in orbitofrontal cortex (OFC) and identity-general reward in ventromedial prefrontal cortex (vmPFC). Reward-related functional coupling between OFC and olfactory (piriform) cortex and between vmPFC and amygdala revealed parallel pathways that support identity-specific and -general predictive signaling. The demonstration of identity-specific value representations in OFC highlights a role for this region in model-based behavior and reveals mechanisms by which appetitive behavior can go awry.