Orbitofrontal cortical neurons encode expectation-driven initiation of reward-seeking

Acute taurine reduced alcohol intake and preference in alcohol-experienced, but not in alcohol-näive rats by central mechanisms

Taurine is a non-essential β sulfonated amino acid involved in a plethora of biological functions in the mammalian central nervous system. Taurine is easily accessible in energy drinks for human consumption. Previous preclinical and clinical reports suggest that acute systemic administration of taurine could inhibit some of the behavioral and metabolic effects of alcohol use disorder. Overall, both in rodent and human studies, acute taurine administration reduced voluntary alcohol intake. This study aimed to assess the pharmacological effects of taurine (intracerebroventricular; i.c.v.) on ethanol intake/preference of rats either control (i.e., alcohol naïve) or forced ethanol intake (since juvenile age with a chronic intermittent access model). In addition, to explore anxiety-like behavior (through defensive burying behavior test) as pharmacological control of taurine. We found that acute (i.c.v.) taurine reduced alcohol consumption, i.e., taurine significantly decreased both alcohol intake and preference in adult male Wistar rats. Moreover, taurine elicits an anxiolytic-like effect in all administered groups independently of previous alcohol exposure.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

Pramipexole Enhances Reward Learning by Preserving Value Estimates

BACKGROUND

Dopamine D2-like agonists show promise as treatments for depression. They are thought to act by enhancing reward learning; however, the mechanisms by which they achieve this are not clear. Reinforcement learning accounts describe 3 distinct candidate mechanisms: increased reward sensitivity, increased inverse decision-temperature, and decreased value decay. As these mechanisms produce equivalent effects on behavior, arbitrating between them requires measurement of how expectations and prediction errors are altered. We characterized the effects of 2 weeks of the D2-like agonist pramipexole on reward learning and used functional magnetic resonance imaging measures of expectation and prediction error to assess which of these 3 mechanistic processes were responsible for the behavioral effects.

METHODS

Forty healthy volunteers (50% female) were randomized to 2 weeks of pramipexole (titrated to 1 mg/day) or placebo in a double-blind, between-subject design. Participants completed a probabilistic instrumental learning task before and after the pharmacological intervention, with functional magnetic resonance imaging data collected at the second visit. Asymptotic choice accuracy and a reinforcement learning model were used to assess reward learning.

RESULTS

Pramipexole increased choice accuracy in the reward condition with no effect on losses. Participants who received pramipexole had increased blood oxygen level-dependent response in the orbital frontal cortex during the expectation of win trials but decreased blood oxygen level-dependent response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.

CONCLUSIONS

The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible mechanism for pramipexole's antidepressant effect.

Reinforcer value moderates the effects of prenatal alcohol exposure on learning and reversal

Introduction

Fetal Alcohol Spectrum Disorders (FASD) are the leading cause of preventable developmental disability and are commonly characterized by alterations in executive function. Reversal learning tasks are reliable, cross-species methods for testing a frequently impaired aspect of executive control, behavioral flexibility. Pre-clinical studies commonly require the use of reinforcers to motivate animals to learn and perform the task. While there are several reinforcers available, the most commonly employed are solid (food pellets) and liquid (sweetened milk) rewards. Previous studies have examined the effects of different solid rewards or liquid dietary content on learning in instrumental responding and found that rodents on liquid reward with higher caloric content performed better with increased response and task acquisition rate. The influence of reinforcer type on reversal learning and how this interacts with developmental insults such as prenatal alcohol exposure (PAE) has not been explored.

Methods

We tested whether reinforcer type during learning or reversal would impact an established deficit in PAE mice.

Results

We found that all male and female mice on liquid reward, regardless of prenatal exposure were better motivated to learn task behaviors during pre-training. Consistent with previous findings, both male and female PAE mice and Saccharine control mice were able to learn the initial stimulus reward associations irrespective of the reinforcer type. During the initial reversal phase, male PAE mice that received pellet rewards exhibited maladaptive perseverative responding whereas male mice that received liquid rewards performed comparable to their control counterparts. Female PAE mice that received either reinforcer types did not exhibit any deficits on behavioral flexibility. Female saccharine control mice that received liquid, but not pellet, rewards showed increased perseverative responding during the early reversal phase.

Discussion

These data suggest that reinforcer type can have a major impact on motivation, and therefore performance, during reversal learning. Highly motivating rewards may mask behavioral deficits seen with more moderately sought rewards and gestational exposure to the non-caloric sweetener, saccharine, can impact behavior motivated by those reinforcers in a sex-dependent manner.

The effects of serotonergic psychedelics in synaptic and intrinsic properties of neurons in layer II/III of the orbitofrontal cortex

RATIONALE

Serotonergic psychedelics show promise in the treatment of psychiatric disorders, including obsessive-compulsive disorder. Dysfunction of the orbitofrontal cortex (OFc) has been implicated in the pathophysiology of compulsive behavior, which might be a key region for the efficacy of psychedelics. However, the effects of psychedelics on the neural activities and local excitation/inhibition (E/I) balance in the OFc are unclear.

OBJECTIVES

This study aimed to investigate how 25C-NBOMe, a substituted phenethylamine psychedelic, regulated the synaptic and intrinsic properties of neurons in layer II/III of the OFc.

METHODS

Acute brain slices containing the OFc of adult male Sprague Dawley rats were used for ex vivo whole-cell recording. The synaptic and intrinsic properties of neurons were monitored using voltage and current clamps, respectively. Electrically evoked action potential (eAP) was used to measure synaptic-driven pyramidal activity.

RESULTS

25C-NBOMe enhanced spontaneous neurotransmission at glutamatergic synapses but diminished that in GABAergic synapses through the 5-HT_2A receptor. 25C-NBOMe also increased both evoked excitatory currents and evoked action potentials. Moreover, 25C-NBOMe promoted the excitability of pyramidal neurons but not fast-spiking neurons. Either inhibiting G protein-gated inwardly rectifying potassium channels or activating protein kinase C significantly obstructed the facilitative effect of 25C-NBOMe on the intrinsic excitability of pyramidal neurons.

CONCLUSIONS

This work reveals the multiple roles of 25C-NBOMe in modulating synaptic and neuronal function in the OFc, which collectively promotes local E/I ratios.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Cognitive Networks (Cognits) Process and Maintain Working Memory

Ever since it was discovered in the monkey’s prefrontal cortex, persistent neuronal activity during the delay period of delay tasks has been considered a phenomenon of working memory. Operationally, this interpretation is correct, because during that delay those tasks require the memorization of a sensory cue, commonly visual. What is incorrect is the assumption that the persistent activity during the delay is caused exclusively by the retention of the sensory cue. In this brief review, the author takes the position that the neural substrate of working memory is an array of long-term memory networks, that is, of cognitive networks (cognits), updated and orderly activated for the attainment of a behavioral goal. In the case of a behavioral task, that activated array of cognits has been previously formed in long-term memory (throughout this text, the expression “long-term memory” refers to all experiences acquired after birth, including habits and so-called procedural memory, such as the learning of a behavioral task). The learning of a task is the forming of synaptic associations between neural representations of three cognitive components of the task: perceptual, motor, and reward-related. Thereafter, when needed, the composite cognit of the task is activated in an orderly fashion to serve working memory in the perception-action cycle. To make his points on a complex issue, which has been the focus of his work, and to delineate a frontier for future research, the author refers to several of his own publications and previously published reviews.

Impaired cognitive flexibility following NMDAR-GluN2B deletion is associated with altered orbitofrontal-striatal function

A common feature across neuropsychiatric disorders is inability to discontinue an action or thought once it has become detrimental. Reversal learning, a hallmark of executive control, requires plasticity within cortical, striatal and limbic circuits and is highly sensitive to disruption of N-methyl-d-aspartate receptor (NMDAR) function. In particular, selective deletion or antagonism of GluN2B containing NMDARs in cortical regions including the orbitofrontal cortex (OFC), promotes maladaptive perseveration. It remains unknown whether GluN2B functions to maintain local cortical activity necessary for reversal learning, or if it exerts a broader influence on the integration of neural activity across cortical and subcortical systems. To address this question, we utilized in vivo electrophysiology to record neuronal activity and local field potentials (LFP) in the orbitofrontal cortex and dorsal striatum (dS) of mice with deletion of GluN2B in neocortical and hippocampal principal cells while they performed touchscreen reversal learning. Reversal impairment produced by corticohippocampal GluN2B deletion was paralleled by an aberrant increase in functional connectivity between the OFC and dS. These alterations in coordination were associated with alterations in local OFC and dS firing activity. These data demonstrate highly dynamic patterns of cortical and striatal activity concomitant with reversal learning, and reveal GluN2B as a molecular mechanism underpinning the timing of these processes.

Cortical hemodynamic mechanisms of reversal learning using high-resolution functional near-infrared spectroscopy: A pilot study

OBJECTIVES

Reversal learning is widely used to analyze cognitive flexibility and characterize behavioral abnormalities associated with impulsivity and disinhibition. Recent studies using fMRI have focused on regions involved in reversal learning with negative and positive reinforcers. Although the frontal cortex has been consistently implicated in reversal learning, few studies have focused on whether reward and punishment may have different effects on lateral frontal structures in these tasks.

METHODS

During this pilot study on eight healthy subjects, we used functional near infra-red spectroscopy (fNIRS) to characterize brain activity dynamics and differentiate the involvement of frontal structures in learning driven by reward and punishment.

RESULTS

We observed functional hemispheric asymmetries between punishment and reward processing by fNIRS following reversal of a learned rule. Moreover, the left dorsolateral prefrontal cortex (l-DLPFC) and inferior frontal gyrus (IFG) were activated under the reward condition only, whereas the orbito-frontal cortex (OFC) was significantly activated under the punishment condition, with a tendency towards activation for the right cortical hemisphere (r-DLPFC and r-IFG). Our results are compatible with the suggestion that the DLPFC is involved in the detection of contingency change. We propose a new representation for reward and punishment, with left lateralization for the reward process.

CONCLUSIONS

The results of this pilot study provide insights into the indirect neural mechanisms of reversal learning and behavioral flexibility and confirm the use of fNIRS imaging in reversal-learning tasks as a translational strategy, particularly in subjects who cannot undergo fMRI recordings.

Altered Activity of Lateral Orbitofrontal Cortex Neurons in Mice following Chronic Intermittent Ethanol Exposure

The lateral orbitofrontal cortex (LOFC) is thought to encode information associated with consumption of rewarding substances and is essential for flexible decision-making. Indeed, firing patterns of LOFC neurons are modulated following changes in reward value associated with an action outcome relationship. Damage to the LOFC impairs behavioral flexibility in humans and is associated with suboptimal performance in reward devaluation protocols in rodents. As chronic intermittent ethanol (CIE) exposure also impairs OFC-dependent behaviors, we hypothesized that CIE exposure would alter LOFC neuronal activity during alcohol drinking, especially under conditions when the reward value of ethanol was modulated by aversive or appetitive tastants. To test this hypothesis, we monitored LOFC activity using GCaMP6f fiber photometry in mice receiving acute injections of ethanol and in those trained in operant ethanol self-administration. In naive mice, an acute injection of ethanol caused a dose-dependent decrease in the frequency but not amplitude of GCaMP6f transients. In operant studies, mice were trained on a fixed ratio one schedule of reinforcement and were then separated into CIE or Air groups. Following four cycles of CIE exposure, GCaMP6f activity was recorded during self-administration of alcohol, alcohol+quinine (aversive), or alcohol+sucrose (appetitive) solutions. LOFC neurons showed discrete patterns of activity surrounding lever presses and surrounding drinking bouts. Responding for and consumption of ethanol was greatly enhanced by CIE exposure, was aversion resistant, and was associated with signs of LOFC hyperexcitability. CIE-exposed mice also showed altered patterns of LOFC activity that varied with the ethanol solution consumed.

Alcohol exposure in utero disrupts cortico-striatal coordination required for behavioral flexibility

Deficits in behavioral flexibility are a hallmark of multiple psychiatric, neurological, and substance use disorders. These deficits are often marked by decreased function of the prefrontal cortex (PFC); however, the genesis of such executive deficits remains understudied. Here we report how the most preventable cause of developmental disability, in utero exposure to alcohol, alters cortico-striatal circuit activity leading to impairments in behavioral flexibility in adulthood. We utilized a translational touch-screen task coupled with in vivo electrophysiology in adult mice to examine single unit and coordinated activity of the lateral orbital frontal cortex (OFC) and dorsolateral striatum (DS) during flexible behavior. Prenatal alcohol exposure (PAE) decreased OFC, and increased DS, single unit activity during reversal learning and altered the number of choice responsive neurons in both regions. PAE also decreased coordinated activity within the OFC and DS as measured by oscillatory field activity and altered spike-field coupling. Furthermore, PAE led to sustained connectivity between regions past what was seen in control animals. These findings suggest that PAE causes altered coordination within and between the OFC and DS, promoting maladaptive perseveration. Our model suggests that in optimally functioning mice OFC disengages the DS and updates the newly changed reward contingency, whereas in PAE animals, aberrant and persistent OFC to DS signaling drives behavioral inflexibility during early reversal sessions. Together, these findings demonstrate how developmental exposure alters circuit-level activity leading to behavioral deficits and suggest a critical role for coordination of neural timing during behaviors requiring executive function.

Differential encoding of action selection by orbitofrontal and striatal population dynamics

Survival relies on the ability to flexibly choose between different actions according to varying environmental circumstances. Many lines of evidence indicate that action selection involves signaling in corticostriatal circuits, including the orbitofrontal cortex (OFC) and dorsomedial striatum (DMS). While choice-specific responses have been found in individual neurons from both areas, it is unclear whether populations of OFC or DMS neurons are better at encoding an animal's choice. To address this, we trained head-fixed mice to perform an auditory guided two-alternative choice task, which required moving a joystick forward or backward. We then used silicon microprobes to simultaneously measure the spiking activity of OFC and DMS ensembles, allowing us to directly compare population dynamics between these areas within the same animals. Consistent with previous literature, both areas contained neurons that were selective for specific stimulus-action associations. However, analysis of concurrently recorded ensemble activity revealed that the animal's trial-by-trial behavior could be decoded more accurately from DMS dynamics. These results reveal substantial regional differences in encoding action selection, suggesting that DMS neural dynamics are more specialized than OFC at representing an animal's choice of action.NEW & NOTEWORTHY While previous literature shows that both orbitofrontal cortex (OFC) and dorsomedial striatum (DMS) represent information relevant to selecting specific actions, few studies have directly compared neural signals between these areas. Here we compared OFC and DMS dynamics in mice performing a two-alternative choice task. We found that the animal's choice could be decoded more accurately from DMS population activity. This work provides among the first evidence that OFC and DMS differentially represent information about an animal's selected action.

Orbitofrontal Cortex Encodes Preference for Alcohol

Orbitofrontal cortex (OFC) plays a key role in representation and regulation of reward value, preference, and seeking. OFC function is disrupted in drug use and dependence, but its specific role in alcohol use disorders has not been thoroughly studied. In alcohol-dependent humans OFC activity is increased by alcohol cue presentation. Ethanol (EtOH) also alters OFC neuron excitability in vitro, and OFC manipulation influences EtOH seeking and drinking in rodents.

Orbitofrontal cortex (OFC) plays a key role in representation and regulation of reward value, preference, and seeking. OFC function is disrupted in drug use and dependence, but its specific role in alcohol use disorders has not been thoroughly studied. In alcohol-dependent humans OFC activity is increased by alcohol cue presentation. Ethanol (EtOH) also alters OFC neuron excitability in vitro, and OFC manipulation influences EtOH seeking and drinking in rodents. To understand the relationship between OFC function and individual alcohol use, we recorded OFC neuron activity in rats during EtOH self-administration, characterizing the neural correlates of individual preference for alcohol. After one month of intermittent access to 20% EtOH, male Long–Evans rats were trained to self-administer 20% EtOH, 10% EtOH, and 15% sucrose. OFC neuronal activity was recorded and associated with task performance and EtOH preference. Rats segregated into high and low EtOH drinkers based on homecage consumption and operant seeking of 20% EtOH. Motivation for 10% EtOH and sucrose was equally high in both groups. OFC neuronal activity was robustly increased or decreased during sucrose and EtOH seeking and consumption, and strength of changes in OFC activity was directly associated with individual preference for 20% EtOH. EtOH-associated OFC activity was more similar to sucrose-associated activity in high versus low EtOH drinkers. The results show that OFC neurons are activated during alcohol seeking based on individual preference, supporting this brain region as a potential substrate for alcohol motivation that may be dysregulated in alcohol misuse.

Selective impact of lateral orbitofrontal cortex inactivation on reinstatement of alcohol seeking in male Long-Evans rats

The orbitofrontal cortex (OFC) plays a fundamental role in motivated behavior and decision-making. In humans, OFC structure and function is significantly disrupted in drug using and dependent individuals, including those exhibiting chronic alcohol use and alcoholism. In animal models, the OFC has been shown to significantly influence the seeking of non-alcohol drugs of abuse. However direct investigations of the OFC during alcohol seeking and use have been more limited. In the studies reported here, we inactivated lateral (lOFC) or medial OFC (mOFC) subregions in rats during multiple stages of alcohol seeking. After one month of intermittent access to homecage 20% ethanol (EtOH), rats were trained to self-administer EtOH under an FR3 schedule and implanted with cannulae directed to lOFC or mOFC. We inactivated OFC subregions with baclofen/muscimol during EtOH self-administration, extinction, cue-induced reinstatement, and progressive ratio testing to broadly characterize the influence of these subregions on alcohol seeking. There were no significant effects of mOFC or lOFC inactivation during FR3 self-administration, extinction, or progressive ratio self-administration. However, lOFC, and not mOFC, inactivation significantly decreased cue-induced reinstatement of EtOH seeking. These findings contribute new information to the specific impact of OFC manipulation on operant alcohol seeking, support previous studies investigating the role of OFC in seeking and consumption of alcohol and other drugs of abuse, and indicate a specific role for lOFC vs. mOFC in reinstatement.

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected sequential effects, including increased willingness to take risks following risky wins, and spatial 'win-stay/lose-shift' biases. Recordings from lateral orbitofrontal cortex (lOFC) revealed encoding of reward history and receipt, and optogenetic inhibition of lOFC eliminated rats' increased preference for risk following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and the lOFC's role in adaptive behavior promotes learning of more abstract biases (here, biases for the risky option), but not spatial ones.

Alcohol exposure in utero disrupts cortico-striatal coordination required for behavioral flexibility

Deficits in behavioral flexibility are a hallmark of multiple psychiatric, neurological, and substance use disorders. These deficits are often marked by decreased function of the prefrontal cortex (PFC); however, the genesis of such executive deficits remains understudied. Here we report how the most preventable cause of developmental disability, in utero exposure to alcohol, alters cortico-striatal circuit activity leading to impairments in behavioral flexibility in adulthood. We utilized a translational touch-screen task coupled with in vivo electrophysiology in adult mice to examine single unit and coordinated activity of the lateral orbital frontal cortex (OFC) and dorsolateral striatum (DS) during flexible behavior. Prenatal alcohol exposure (PAE) decreased OFC, and increased DS, single unit activity during reversal learning and altered the number of choice responsive neurons in both regions. PAE also decreased coordinated activity within the OFC and DS as measured by oscillatory field activity and altered spike-field coupling. Furthermore, PAE led to sustained connectivity between regions past what was seen in control animals. These findings suggest that PAE causes altered coordination within and between the OFC and DS, promoting maladaptive perseveration. Our model suggests that in optimally functioning mice OFC disengages the DS and updates the newly changed reward contingency, whereas in PAE animals, aberrant and persistent OFC to DS signaling drives behavioral inflexibility during early reversal sessions. Together, these findings demonstrate how developmental exposure alters circuit-level activity leading to behavioral deficits and suggest a critical role for coordination of neural timing during behaviors requiring executive function.

Single-cell activity tracking reveals that orbitofrontal neurons acquire and maintain a long-term memory to guide behavioral adaptation

Learning to predict rewards based on environmental cues is essential for survival. The orbitofrontal cortex (OFC) contributes to such learning by conveying reward-related information to brain areas such as the ventral tegmental area (VTA). Despite this, how cue-reward memory representations form in individual OFC neurons and are modified based on new information is unknown. To address this, using in vivo two-photon calcium imaging in mice, we tracked the response evolution of thousands of OFC output neurons, including those projecting to VTA, through multiple days and stages of cue-reward learning. Collectively, we show that OFC contains several functional clusters of neurons distinctly encoding cue-reward memory representations, with only select responses routed downstream to VTA. Unexpectedly, these representations were stably maintained by the same neurons even after extinction of the cue-reward pairing, and supported behavioral learning and memory. Thus, OFC neuronal activity represents a long-term cue-reward associative memory to support behavioral adaptation.

Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes

Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.

Preference for Cocaine is Represented in the Orbitofrontal Cortex by an Increased Proportion of Cocaine Use-Coding Neurons

Cocaine addiction is a harmful preference for drug use over and at the expense of other nondrug-related activities. Here we identify in the rat orbitofrontal cortex (OFC) a mechanism that explains individual preferences between cocaine use and an alternative, nondrug action. OFC neuronal activity was recorded while rats performed each of these 2 actions separately or while they chose between them. First, we found that these actions are encoded by 2 nonoverlapping neuronal populations and that the relative size of the cocaine population represented individual preferences. A larger relative size was only observed in cocaine-preferring individuals. Second, OFC neurons encoding a given individual's preferred action progressively fired more than other action-coding neurons few seconds before the preferred action was actually chosen, suggesting a prechoice neuronal competition for action selection. In cocaine-preferring rats, this manifested by a prechoice ramping-up activity in favor of the cocaine population. Finally, pharmacological manipulation of prechoice activity in favor of the cocaine population caused nondrug-preferring rats to shift their choice to cocaine. Overall, this study suggests that an individual preference for cocaine is represented in the OFC by a population size bias that systematically advantages cocaine use-coding neurons during prechoice competition for action selection.

Impaired cognitive flexibility following NMDAR-GluN2B deletion is associated with altered orbitofrontal-striatal function

A common feature across neuropsychiatric disorders is inability to discontinue an action or thought once it has become detrimental. Reversal learning, a hallmark of executive control, requires plasticity within cortical, striatal and limbic circuits and is highly sensitive to disruption of N-methyl-D-aspartate receptor (NMDAR) function. In particular, selective deletion or antagonism of GluN2B containing NMDARs in cortical regions including the orbitofrontal cortex (OFC), promotes maladaptive perseveration. It remains unknown whether GluN2B functions to maintain local cortical activity necessary for reversal learning, or if it exerts a broader influence on the integration of neural activity across cortical and subcortical systems. To address this question, we utilized in vivo electrophysiology to record neuronal activity and local field potentials (LFP) in the orbitofrontal cortex and dorsal striatum (dS) of mice with deletion of GluN2B in neocortical and hippocampal principal cells while they performed touchscreen reversal learning. Reversal impairment produced by corticohippocampal GluN2B deletion was paralleled by an aberrant increase in functional connectivity between the OFC and dS. These alterations in coordination were associated with alterations in local OFC and dS firing activity. These data demonstrate highly dynamic patterns of cortical and striatal activity concomitant with reversal learning, and reveal GluN2B as a molecular mechanism underpinning the timing of these processes.

Increased perseverative errors following high-definition transcranial direct current stimulation over the ventrolateral cortex during probabilistic reversal learning

BACKGROUND

The prefrontal cortex regulates behavioural adaptation in response to feedback. However, the causal role of different prefrontal regions remains unclear, based on indirect evidence derived from functional neuroimaging. Neuroimaging studies show dorsomedial prefrontal activation during feedback monitoring, whereas the ventrolateral prefrontal cortex engages during behavioural adaptation (shifting).

OBJECTIVE

We used high-definition transcranial direct current stimulation (HD-tDCS) to elucidate the roles of the ventrolateral prefrontal cortex (vlPFC) and the dorsomedial prefrontal cortex (dmPFC) in behaviour change, using a probabilistic reversal learning task (PRLT).

METHOD

Fifty-two healthy adults were randomly assigned to receive cathodal HD-tDCS to inhibit the vlPFC or the dmPFC versus sham stimulation, prior to completing the PRLT. The outcome measures were the number of perseverative errors and the electroencephalography (EEG) signals of feedback-related negativity (FRN) in the PRLT. We hypothesised that inhibition of the vlPFC would be specifically associated with more perseverative errors and weaker FRNs.

RESULTS

We found that vlPFC inhibition was associated with higher perseverative errors compared to sham and dmPFC stimulation conditions. Although there were no statistically significant differences in FRN amplitudes, the effect sizes indicate an association between inhibition of the vlPFC and lower FRN amplitudes.

CONCLUSION

Our findings support a causal role of the vlPFC on feedback-based behavioural adaptation, which is critical for adaptive goal-driven behaviour.

The emerging neuroscience of appetitive and drug cue extinction in humans

Fear extinction has been extensively studied in both humans and non-human animals, and this work has contributed greatly to our understanding and treatment of anxiety disorders. Yet other psychopathologies like addiction might be associated with impairments selectively in extinction of non-fear based, appetitive and drug cue associations, and these processes have been underexplored in clinical translational neuroscience. Important questions regarding similarities and differences in the neurobiological mechanisms underlying aversive and appetitive extinction remain unanswered, particularly those pertaining to cross-species evidence for the role of the ventromedial prefrontal cortex and, to some extent, the striatum. Here, we aim to draw attention to the paucity of studies investigating non-fear based extinction in humans, summarize emerging findings from the available literature, and highlight important directions for future research. We argue that closing these gaps in our understanding could help inform the development of more targeted, and perhaps more durable, forms of extinction-based treatments for addiction and related psychopathologies characterized by abnormally persistent appetitive and drug cue associations.

The role of the orbitofrontal cortex in alcohol use, abuse, and dependence

One of the major functions of the orbitofrontal cortex (OFC) is to promote flexible motivated behavior. It is no surprise, therefore, that recent work has demonstrated a prominent impact of chronic drug use on the OFC and a potential role for OFC disruption in drug abuse and addiction. Among drugs of abuse, the use of alcohol is particularly salient with respect to OFC function. Although a number of studies in humans have implicated OFC dysregulation in alcohol use disorders, animal models investigating the association between OFC and alcohol use are only beginning to be developed, and there is still a great deal to be revealed. The goal of this review is to consider what is currently known regarding the role of the OFC in alcohol use and dependence. I will first provide a brief, general overview of current views of OFC function and its contributions to drug seeking and addiction. I will then discuss research to date related to the OFC and alcohol use, both in human clinical populations and in non-human models. Finally I will consider issues and strategies to guide future study that may identify this brain region as a key player in the transition from moderated to problematic alcohol use and dependence.

Functional inactivation of the orbitofrontal cortex disrupts context-induced reinstatement of alcohol seeking in rats

BACKGROUND

The high rate of relapse to drug use remains a central challenge to treating drug addiction. In human and rat models of addiction, environmental stimuli in contexts associated with previous drug use can provoke a relapse of drug seeking. Pre-clinical studies have used the ABA renewal procedure to study context-induced reinstatement of drug seeking. In the current study, we studied the role of the orbitofrontal cortex (OFC) in context-induced reinstatement to alcohol.

METHODS

We trained male and female rats to self-administer alcohol in context A, extinguished drug-reinforced responding in a distinct context B, and assessed context-induced reinstatement in context A or B (control group). Next, we determined the effect of context-induced renewal of alcohol-seeking behavior on the expression of Fos (a neuronal activity marker) in the OFC. Finally, we determined the effect of reversible inactivation by GABAa and GABAb receptor agonists (i.e., muscimol and baclofen, respectively) in the OFC.

RESULTS AND CONCLUSIONS

There were no differences between male and female rats in context-induced reinstatement of alcohol-seeking behavior. Re-exposure to Context A, but not Context B, reinstated alcohol-seeking behavior and increased expression of the neural activity marker Fos in the OFC. Reversible inactivation of the OFC with muscimol and baclofen attenuated context-induced reinstatement. Our data indicated that the OFC mediates context-induced reinstatement of alcohol-seeking behavior.

Neuronal representation of individual heroin choices in the orbitofrontal cortex

Drug addiction is a harmful preference for drug use over and at the expense of other non-drug-related activities. We previously identified in the rat orbitofrontal cortex (OFC) a mechanism that influences individual preferences between cocaine use and an alternative action rewarded by a non-drug reward (i.e. sweet water). Here, we sought to test the generality of this mechanism to a different addictive drug, heroin. OFC neuronal activity was recorded while rats responded for heroin or the alternative non-drug reward separately or while they chose between the two. First, we found that heroin-rewarded and sweet water-rewarded actions were encoded by two non-overlapping OFC neuronal populations and that the relative size of the heroin population represented individual drug choices. Second, OFC neurons encoding the preferred action-which was the non-drug action in the large majority of individuals-progressively fired more than non-preferred action-coding neurons 1 second after the onset of choice trials and around 1 second before the preferred action was actually chosen, suggesting a pre-choice neuronal competition for action selection. Together with a previous study on cocaine choice, the present study on heroin choice reveals important commonalities in how OFC neurons encode individual drug choices and preferences across different classes of drugs. It also reveals some drug-specific differences in OFC encoding activity. Notably, the proportion of neurons that non-selectively encode both the drug and the non-drug reward was higher when the drug was heroin (present study) than when it was cocaine (previous study). We will discuss the potential functional significance of these commonalities and differences in OFC neuronal activity across different drugs for understanding drug choice.

Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as "OFC," and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Neural correlates of appetitive extinction in humans

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS−) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS−. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS− after conditioning. Furthermore, fMRI-results (CS+ − CS−) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Differential Encoding of Time by Prefrontal and Striatal Network Dynamics

Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas.

SIGNIFICANCE STATEMENT

The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs.

Orbitofrontal participation in sign- and goal-tracking conditioned responses: Effects of nicotine

Pavlovian conditioned stimuli can acquire incentive motivational properties, and this phenomenon can be measured in animals using Pavlovian conditioned approach behavior. Drugs of abuse can influence the expression of this behavior, and nicotine in particular exhibits incentive amplifying effects. Both conditioned approach behavior and drug abuse rely on overlapping corticolimbic circuitry. We hypothesize that the orbitofrontal cortex (OFC) regulates conditioned approach, and that one site of nicotine action is in the OFC where it reduces cortical output. To test this, we repeatedly exposed rats to 0.4 mg/kg nicotine (s.c.) during training and then pharmacologically inactivated the lateral OFC or performed in vivo electrophysiological recordings of lateral OFC neurons in the presence or absence of nicotine. In Experiment 1, animals were trained in a Pavlovian conditioning paradigm and behavior was evaluated after inactivation of the OFC by microinfusion of the GABA agonists baclofen and muscimol. In Experiment 2, we monitored phasic firing of OFC neurons during Pavlovian conditioning sessions. Nicotine reliably enhanced conditioned responding to the conditioned cue, and inactivation of the OFC reduced conditioned responding, especially the sign-tracking response. OFC neurons exhibited phasic excitations to cue presentation and during goal tracking, and nicotine acutely blunted this phasic neuronal firing. When nicotine was withheld, both conditioned responding and phasic firing in the OFC returned to the level of controls. These results suggest that the OFC is recruited for the expression of conditioned responses, and that nicotine acutely influences this behavior by reducing phasic firing in the OFC.

Touch-screen visual reversal learning is mediated by value encoding and signal propagation in the orbitofrontal cortex

Behavioral inflexibility is a common symptom of neuropsychiatric disorders which can have a major detrimental impact on quality of life. While the orbitofrontal cortex (OFC) has been strongly implicated in behavioral flexibility in rodents across paradigms, our understanding of how the OFC mediates these behaviors is rapidly adapting. Here we examined neuronal activity during reversal learning by coupling in vivo electrophysiological recording with a mouse touch-screen learning paradigm to further elucidate the role of the OFC in updating reward value. Single unit and oscillatory activity was recorded during well-learned discrimination and 3 distinct phases of reversal (early, chance and well-learned). During touch-screen performance, OFC neuronal firing tracked rewarded responses following a previous rewarded choice when behavior was well learned, but shifted to primarily track repeated errors following a previous error in early reversal. Spike activity tracked rewarded choices independent of previous trial outcome during chance reversal, and returned to the initial pattern of reward response at criterion. Analysis of spike coupling to oscillatory local field potentials showed that less frequently occurring behaviors had significantly fewer neurons locked to any oscillatory frequency. Together, these data support the role of the OFC in tracking the value of individual choices to inform future responses and suggests that oscillatory signaling may be involved in propagating responses to increase or decrease the likelihood that action is taken in the future. They further support the use of touch-screen paradigms in preclinical studies to more closely model clinical approaches to measuring behavioral flexibility.

Changes in Appetitive Associative Strength Modulates Nucleus Accumbens, But Not Orbitofrontal Cortex Neuronal Ensemble Excitability

Cues that predict the availability of food rewards influence motivational states and elicit food-seeking behaviors. If a cue no longer predicts food availability, then animals may adapt accordingly by inhibiting food-seeking responses. Sparsely activated sets of neurons, coined "neuronal ensembles," have been shown to encode the strength of reward-cue associations. Although alterations in intrinsic excitability have been shown to underlie many learning and memory processes, little is known about these properties specifically on cue-activated neuronal ensembles. We examined the activation patterns of cue-activated orbitofrontal cortex (OFC) and nucleus accumbens (NAc) shell ensembles using wild-type and Fos-GFP mice, which express green fluorescent protein (GFP) in activated neurons, after appetitive conditioning with sucrose and extinction learning. We also investigated the neuronal excitability of recently activated, GFP+ neurons in these brain areas using whole-cell electrophysiology in brain slices. Exposure to a sucrose cue elicited activation of neurons in both the NAc shell and OFC. In the NAc shell, but not the OFC, these activated GFP+ neurons were more excitable than surrounding GFP- neurons. After extinction, the number of neurons activated in both areas was reduced and activated ensembles in neither area exhibited altered excitability. These data suggest that learning-induced alterations in the intrinsic excitability of neuronal ensembles is regulated dynamically across different brain areas. Furthermore, we show that changes in associative strength modulate the excitability profile of activated ensembles in the NAc shell.SIGNIFICANCE STATEMENT Sparsely distributed sets of neurons called "neuronal ensembles" encode learned associations about food and cues predictive of its availability. Widespread changes in neuronal excitability have been observed in limbic brain areas after associative learning, but little is known about the excitability changes that occur specifically on neuronal ensembles that encode appetitive associations. Here, we reveal that sucrose cue exposure recruited a more excitable ensemble in the nucleus accumbens, but not orbitofrontal cortex, compared with their surrounding neurons. This excitability difference was not observed when the cue's salience was diminished after extinction learning. These novel data provide evidence that the intrinsic excitability of appetitive memory-encoding ensembles is regulated differentially across brain areas and adapts dynamically to changes in associative strength.

Vagus nerve stimulation reduces cocaine seeking and alters plasticity in the extinction network

Drugs of abuse cause changes in the prefrontal cortex (PFC) and associated regions that impair inhibitory control over drug-seeking. Breaking the contingencies between drug-associated cues and the delivery of the reward during extinction learning reduces rates of relapse. Here we used vagus nerve stimulation (VNS) to induce targeted synaptic plasticity to facilitate extinction of appetitive behaviors and to reduce relapse. Rats self-administered cocaine and were given VNS during extinction. Relapse to drug-seeking was assessed in a cued reinstatement session. We used immunohistochemistry to measure changes in the expression of the phosphorylated transcription factor cAMP response-element binding protein (pCREB) in the PFC and the basolateral amygdala (BLA), which regulate cue learning and extinction. In vivo recordings of evoked field potentials measured drug- and VNS-induced changes in metaplasticity in the pathway from the PFC to the BLA. VNS-treated rats showed improved rates of extinction and reduced reinstatement. Following reinstatement, pCREB levels were reduced in the IL and BLA of VNS-treated rats. Evoked responses in the BLA were greatly reduced in VNS-treated rats, and these rats were also resistant to the induction of LTD. Taken together, these results show that VNS facilitates extinction and reduces reinstatement. Changes in the pathway between the PFC and the amygdala may contribute to these beneficial effects.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Neural and behavioral mechanisms of proactive and reactive inhibition

Response inhibition is an important component of adaptive behavior. Substantial prior research has focused on reactive inhibition, which refers to the cessation of a motor response that is already in progress. More recently, a growing number of studies have begun to examine mechanisms underlying proactive inhibition, whereby preparatory processes result in a response being withheld before it is initiated. It has become apparent that proactive inhibition is an essential component of the overall ability to regulate behavior and has implications for the success of reactive inhibition. Moreover, successful inhibition relies on learning the meaning of specific environmental cues that signal when a behavioral response should be withheld. Proactive inhibitory control is mediated by stopping goals, which reflect the desired outcome of inhibition and include information about how and when inhibition should be implemented. However, little is known about the circuits and cellular processes that encode and represent features in the environment that indicate the necessity for proactive inhibition or how these representations are implemented in response inhibition. In this article, we will review the brain circuits and systems involved in implementing inhibitory control through both reactive and proactive mechanisms. We also comment on possible cellular mechanisms that may contribute to inhibitory control processes, noting that substantial further research is necessary in this regard. Furthermore, we will outline a number of ways in which the temporal dynamics underlying the generation of the proactive inhibitory signal may be particularly important for parsing out the neurobiological correlates that contribute to the learning processes underlying various aspects of inhibitory control.

Ants Can Expect the Time of an Event on Basis of Previous Experiences

Working on three ant species of the genus Myrmica, M. ruginodis, M. rubra, and M. sabuleti, we showed that foragers can expect the subsequent time at which food will be available on the basis of the previous times at which food was present. The ants acquired this expectative ability right after having experienced two time shifts of food delivery. Moreover, the ants' learning score appeared to be a logarithmic function of time (i.e., of the number of training days). This ability to expect subsequent times at which an event will occur may be an advantageous ethological trait.

Vicarious trial and error

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

Effects of stress on behavioral flexibility in rodents

Cognitive flexibility is the ability to switch between different rules or concepts and behavioral flexibility is the overt physical manifestation of these shifts. Behavioral flexibility is essential for adaptive responses and commonly measured by reversal learning and set-shifting performance in rodents. Both tasks have demonstrated vulnerability to stress with effects dependent upon stressor type and number of repetitions. This review compares the effects of stress on reversal learning and set-shifting to provide insight into the differential effect of stress on cognition. Acute and short-term repetition of stress appears to facilitate reversal learning whereas the longer term repetition of stress impairs reversal learning. Stress facilitated intradimensional set-shifting within a single, short-term stress protocol but otherwise generally impaired set-shifting performance in acute and repeated stress paradigms. Chronic unpredictable stress impairs reversal learning and set-shifting whereas repeated cold intermittent stress selectively impairs reversal learning and has no effect on set-shifting. In considering the mechanisms underlying the effects of stress on behavioral flexibility, pharmacological manipulations performed in conjunction with stress are also reviewed. Blocking corticosterone receptors does not affect the facilitation of reversal learning following acute stress but the prevention of corticosterone synthesis rescues repeated stress-induced set-shifting impairment. Enhancing post-synaptic norepinephrine function, serotonin availability, and dopamine receptor activation rescues and/or prevents behavioral flexibility performance following stress. While this review highlights a lack of a standardization of stress paradigms, some consistent effects are apparent. Future studies are necessary to specify the mechanisms underlying the stress-induced impairments of behavioral flexibility, which will aid in alleviating these symptoms in patients with some psychiatric disorders.

The neural basis of reversal learning: An updated perspective

Reversal learning paradigms are among the most widely used tests of cognitive flexibility and have been used as assays, across species, for altered cognitive processes in a host of neuropsychiatric conditions. Based on recent studies in humans, non-human primates, and rodents, the notion that reversal learning tasks primarily measure response inhibition, has been revised. In this review, we describe how cognitive flexibility is measured by reversal learning and discuss new definitions of the construct validity of the task that are serving as a heuristic to guide future research in this field. We also provide an update on the available evidence implicating certain cortical and subcortical brain regions in the mediation of reversal learning, and an overview of the principal neurotransmitter systems involved.

Action-outcome relationships are represented differently by medial prefrontal and orbitofrontal cortex neurons during action execution

Internal representations of action-outcome relationships are necessary for flexible adaptation of motivated behavior in dynamic environments. Prefrontal cortex (PFC) is implicated in flexible planning and execution of goal-directed actions, but little is known about how information about action-outcome relationships is represented across functionally distinct regions of PFC. Here, we observe distinct patterns of action-evoked single unit activity in the medial prefrontal cortex (mPFC) and orbitofrontal cortex (OFC) during a task in which the relationship between outcomes and actions was independently manipulated. The mPFC encoded changes in the number of actions required to earn a reward, but not fluctuations in outcome magnitude. In contrast, OFC neurons decreased firing rates as outcome magnitude was increased, but were insensitive to changes in action requirement. A subset of OFC neurons also tracked outcome availability. Pre-outcome anticipatory activity in both mPFC and OFC was altered when reward expectation was reduced, but did not differ with outcome magnitude. These data provide novel evidence that PFC regions encode distinct information about the relationship between actions and impending outcomes during action execution.

Behavioural and brain responses related to Internet search and memory

The ready availability of data via searches on the Internet has changed how many people seek and perhaps store and recall information, although the brain mechanisms underlying these processes are not well understood. This study investigated brain mechanisms underlying Internet-based vs. non-Internet-based searching. The results showed that Internet searching was associated with lower accuracy in recalling information as compared with traditional book searching. During functional magnetic resonance imaging, Internet searching was associated with less regional brain activation in the left ventral stream, the association area of the temporal-parietal-occipital cortices, and the middle frontal cortex. When comparing novel items with remembered trials, Internet-based searching was associated with higher brain activation in the right orbitofrontal cortex and lower brain activation in the right middle temporal gyrus when facing those novel trials. Brain activations in the middle temporal gyrus were inversely correlated with response times, and brain activations in the orbitofrontal cortex were positively correlated with self-reported search impulses. Taken together, the results suggest that, although Internet-based searching may have facilitated the information-acquisition process, this process may have been performed more hastily and be more prone to difficulties in recollection. In addition, people appear less confident in recalling information learned through Internet searching and that recent Internet searching may promote motivation to use the Internet.

Prefrontal neurons encode context-based response execution and inhibition in reward seeking and extinction

The prefrontal cortex (PFC) guides execution and inhibition of behavior based on contextual demands. In rodents, the dorsal/prelimbic (PL) medial PFC (mPFC) is frequently considered essential for execution of goal-directed behavior ("go") whereas ventral/infralimbic (IL) mPFC is thought to control behavioral suppression ("stop"). This dichotomy is commonly seen for fear-related behaviors, and for some behaviors related to cocaine seeking. Overall, however, data for reward-directed behaviors are ambiguous, and few recordings of PL/IL activity have been performed to demonstrate single-neuron correlates. We recorded neuronal activity in PL and IL during discriminative stimulus driven sucrose seeking followed by multiple days of extinction of the reward-predicting stimulus. Contrary to a generalized PL-go/IL-stop hypothesis, we found cue-evoked activity in PL and IL during reward seeking and extinction. Upon analyzing this activity based on resultant behavior (lever press or withhold), we found that neurons in both areas encoded contextually appropriate behavioral initiation (during reward seeking) and withholding (during extinction), where context was dictated by response-outcome contingencies. Our results demonstrate that PL and IL signal contextual information for regulation of behavior, irrespective of whether that involves initiation or suppression of behavioral responses, rather than topographically encoding go vs. stop behaviors. The use of context to optimize behavior likely plays an important role in maximizing utility-promoting exertion of activity when behaviors are rewarded and conservation of energy when not.

Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models

Over the past 20 years there has been a growing interest in the neural underpinnings of cost/benefit decision-making. Recent studies with animal models have made considerable advances in our understanding of how different prefrontal, striatal, limbic and monoaminergic circuits interact to promote efficient risk/reward decision-making, and how dysfunction in these circuits underlies aberrant decision-making observed in numerous psychiatric disorders. This review will highlight recent findings from studies exploring these questions using a variety of behavioral assays, as well as molecular, pharmacological, neurophysiological, and translational approaches. We begin with a discussion of how neural systems related to decision subcomponents may interact to generate more complex decisions involving risk and uncertainty. This is followed by an overview of interactions between prefrontal-amygdala-dopamine and habenular circuits in regulating choice between certain and uncertain rewards and how different modes of dopamine transmission may contribute to these processes. These data will be compared with results from other studies investigating the contribution of some of these systems to guiding decision-making related to rewards vs. punishment. Lastly, we provide a brief summary of impairments in risk-related decision-making associated with psychiatric disorders, highlighting recent translational studies in laboratory animals.

Executive control signals in orbitofrontal cortex during response inhibition

Orbitofrontal cortex (OFC) lesions produce deficits in response inhibition and imaging studies suggest that activity in OFC is stronger on trials that require suppression of behavior, yet few studies have examined neural correlates at the single-unit level in a behavioral task that probes response inhibition without varying other factors, such as anticipated outcomes. Here we recorded from single neurons in lateral OFC in a task that required animals in the minority of trials to STOP or inhibit an ongoing movement and respond in the opposite direction. We found that population and single-unit firing was modulated primarily by response direction and movement speed, and that very few OFC neurons exhibited a response independent inhibition signal. Remarkably, the strength of the directional signal was not diminished on STOP trials and was actually stronger on STOP trials during conflict adaptation. Finally, directional signals were stronger during sessions in which rats had the most difficulty inhibiting behavior. These results suggest that "inhibition" deficits observed with OFC interference studies reflect deficits unrelated to signaling the need to inhibit behavior, but instead support a role for OFC in executive functions related to dissociating between two perceptually similar actions during response conflict.

Behavioral flexibility in rats and mice: contributions of distinct frontocortical regions

Research examining the contribution of genetics to behavior is increasingly focused on higher order behavioral and cognitive processes including the ability to modify behaviors when environmental demands change. The frontal cortices of mammals, including rodents, subserve a diverse set of behavioral and cognitive functions including motor planning, social behavior, evaluation of expected outcomes and working memory, which may be particularly sensitive to genetic factors and interactions with experience (e.g. stress). Behavioral flexibility is a core attribute of these functions. This review orients readers to the current landscape of the literature on the frontocortical bases of behavioral flexibility in rodent laboratory experiments. Studies are divided into three broad categories: reversal learning, inhibitory learning and set-shifting. Functional dissociations within the broader scope of behavioral flexibility are reviewed, followed by discussion of the associations between specific components of frontal cortex and specific aspects of relevant behavioral processes. Finally, the authors identify open questions that need to be addressed to better establish the constituents of frontal cortex underlying behavioral flexibility.