Insular and Ventrolateral Orbitofrontal Cortices Differentially Contribute to Goal-Directed Behavior in Rodents

Adaptive Responding to Stimulus-Outcome Associations Requires Noradrenergic Transmission in the Medial Prefrontal Cortex

A dynamic environment, such as the one we inhabit, requires organisms to continuously update their knowledge of the setting. While the prefrontal cortex is recognized for its pivotal role in regulating such adaptive behavior, the specific contribution of each prefrontal area remains elusive. In the current work, we investigated the direct involvement of two major prefrontal subregions, the medial prefrontal cortex (mPFC, A32D + A32V) and the orbitofrontal cortex (OFC, VO + LO), in updating pavlovian stimulus-outcome (S-O) associations following contingency degradation in male rats. Specifically, animals had to learn that a particular cue, previously fully predicting the delivery of a specific reward, was no longer a reliable predictor. First, we found that chemogenetic inhibition of mPFC, but not of OFC, neurons altered the rats' ability to adaptively respond to degraded and non-degraded cues. Next, given the growing evidence pointing at noradrenaline (NA) as a main neuromodulator of adaptive behavior, we decided to investigate the possible involvement of NA projections to the two subregions in this higher-order cognitive process. Employing a pair of novel retrograde vectors, we traced NA projections from the locus ceruleus (LC) to both structures and observed an equivalent yet relatively segregated amount of inputs. Then, we showed that chemogenetic inhibition of NA projections to the mPFC, but not to the OFC, also impaired the rats' ability to adaptively respond to the degradation procedure. Altogether, our findings provide important evidence of functional parcellation within the prefrontal cortex and point at mPFC NA as key for updating pavlovian S-O associations.

Impairments in expression of devaluation in a Pavlovian goal-tracking task, but not a free operant devaluation task, after fentanyl exposure in female rats

In laboratory animals, there are numerous demonstrations that past exposure to drugs of abuse can lead to devaluation impairments weeks after the final drug exposure, with the majority of these demonstrations examining effects of exposure to psychostimulants. There has been minimal investigation into whether prior exposure to opiates can lead to devaluation impairments. Here, we first trained female rats that two separate cuelights predicted two different foods and measured Pavlovian goal-tracking responses (Experiment 1) or trained female rats to press two levers to earn two different foods and measured this operant response (Experiment 2). In both experiments, we subsequently gave the rats injections of fentanyl twice daily for 6 days, and then tested rats for conditioned responses after satiation on one of the foods 48-h after the final injection. We found that rats were impaired in the expression of devaluation in the Pavlovian task after fentanyl exposure, but were unimpaired in the expression of devaluation in the operant task. The pattern of results is most consistent with an impairment in lateral orbitofrontal cortex function, but additional research is needed to determine the neurobiological cause of this pattern of results.

A hippocampo-cortical pathway detects changes in the validity of an action as a predictor of reward

Much research has been dedicated to understanding the psychological and neural bases of goal-directed action, yet the relationship between context and goal-directed action is not well understood. Here, we used excitotoxic lesions, chemogenetics, and circuit-specific manipulations to demonstrate the role of the ventral hippocampus (vHPC) in contextual learning that supports sensitivity to action-outcome contingencies, a hallmark of goal-directed action. We found that chemogenetic inhibition of the ventral, but not dorsal, hippocampus attenuated sensitivity to instrumental contingency degradation. We then tested the hypothesis that this deficit was due to an inability to discern the relative validity of the action compared with the context as a predictor of reward. Using latent inhibition and Pavlovian context conditioning, we confirm that degradation of action-outcome contingencies relies on intact context-outcome learning and show that this learning is dependent on vHPC. Finally, we show that chemogenetic inhibition of vHPC terminals in the medial prefrontal cortex also impairs both instrumental contingency degradation and context-outcome learning. These results implicate a hippocampo-cortical pathway in adapting to changes in instrumental contingencies and indicate that the psychological basis of this deficit is an inability to learn the predictive value of the context. Our findings contribute to a broader understanding of the neural bases of goal-directed action and its contextual regulation.

Orbitofrontal cortex control of striatum leads economic decision-making

Animals must continually evaluate stimuli in their environment to decide which opportunities to pursue, and in many cases these decisions can be understood in fundamentally economic terms. Although several brain regions have been individually implicated in these processes, the brain-wide mechanisms relating these regions in decision-making are unclear. Using an economic decision-making task adapted for rats, we find that neural activity in both of two connected brain regions, the ventrolateral orbitofrontal cortex (OFC) and the dorsomedial striatum (DMS), was required for economic decision-making. Relevant neural activity in both brain regions was strikingly similar, dominated by the spatial features of the decision-making process. However, the neural encoding of choice direction in OFC preceded that of DMS, and this temporal relationship was strongly correlated with choice accuracy. Furthermore, activity specifically in the OFC projection to the DMS was required for appropriate economic decision-making. These results demonstrate that choice information in the OFC is relayed to the DMS to lead accurate economic decision-making.

GluN2B inhibition confers resilience against long-term cocaine-induced neurocognitive sequelae

Cocaine self-administration can disrupt the capacity of humans and rodents to flexibly modify familiar behavioral routines, even when they become maladaptive or unbeneficial. However, mechanistic factors, particularly those driving long-term behavioral changes, are still being determined. Here, we capitalized on individual differences in oral cocaine self-administration patterns in adolescent mice and revealed that the post-synaptic protein PSD-95 was reduced in the orbitofrontal cortex (OFC) of escalating, but not stable, responders, which corresponded with later deficits in flexible decision-making behavior. Meanwhile, NMDA receptor GluN2B subunit content was lower in the OFC of mice that were resilient to escalatory oral cocaine seeking. This discovery led us to next co-administer the GluN2B-selective antagonist ifenprodil with cocaine, blocking the later emergence of cocaine-induced decision-making abnormalities. GluN2B inhibition also prevented cocaine-induced dysregulation of neuronal structure and function in the OFC, preserving mature, mushroom-shaped dendritic spine densities on deep-layer pyramidal neurons, which were otherwise lower with cocaine, and safeguarding functional BLA→OFC connections necessary for action flexibility. We posit that cocaine potentiates GluN2B-dependent signaling, which triggers a series of durable adaptations that result in the dysregulation of post-synaptic neuronal structure in the OFC and disruption of BLA→OFC connections, ultimately weakening the capacity for flexible choice. And thus, inhibiting GluN2B-NMDARs promotes resilience to long-term cocaine-related sequelae.

A Novel Insular/Orbital-Prelimbic Circuit That Prevents Persistent Avoidance in a Rodent Model of Compulsive Behavior

BACKGROUND

A common symptom of obsessive-compulsive disorder is the persistent avoidance of cues incorrectly associated with negative outcomes. This maladaptation becomes increasingly evident as subjects fail to respond to extinction-based treatments such as exposure-with-response prevention therapy. While previous studies have highlighted the role of the insular-orbital cortex in fine-tuning avoidance-based decisions, little is known about the projections from this area that might modulate compulsive-like avoidance.

METHODS

Here, we used anatomical tract-tracing, single-unit recording, and optogenetics to characterize the projections from the insular-orbital cortex. To model exposure-with-response prevention and persistent avoidance in rats, we used the platform-mediated avoidance task followed by extinction-with-response prevention training.

RESULTS

Using tract-tracing and unit recording, we found that projections from the agranular insular/lateral orbital (AI/LO) cortex to the prefrontal cortex predominantly target the rostral portion of the prelimbic (rPL) cortex and excite rPL neurons. Photoinhibiting this projection induced persistent avoidance after extinction-with-response prevention training, an effect that was still present 1 week later. Consistent with this, photoexcitation of this projection prevented persistent avoidance in overtrained rats. This projection to rPL appears to be key for AI/LO's effects, considering that there was no effect of photoinhibiting AI/LO projections to the ventral striatum or basolateral amygdala.

CONCLUSIONS

Our findings suggest that projections from the AI/LO to the rPL decreases the likelihood of avoidance behavior following extinction. In humans, this connectivity may share some homology of projections from lateral prefrontal cortices (i.e., ventrolateral prefrontal cortex, orbitofrontal cortex, and insula) to other prefrontal areas and the anterior cingulate cortex, suggesting that reduced activity in these pathways may contribute to obsessive-compulsive disorder.

Whole-brain mapping of long-range inputs to the VIP-expressing inhibitory neurons in the primary motor cortex

The primary motor cortex (MOp) is an important site for motor skill learning. Interestingly, neurons in MOp possess reward-related activity, presumably to facilitate reward-based motor learning. While pyramidal neurons (PNs) and different subtypes of GABAergic inhibitory interneurons (INs) in MOp all undergo cell-type specific plastic changes during motor learning, the vasoactive intestinal peptide-expressing inhibitory interneurons (VIP-INs) in MOp have been shown to preferentially respond to reward and play a critical role in the early phases of motor learning by triggering local circuit plasticity. To understand how VIP-INs might integrate various streams of information, such as sensory, pre-motor, and reward-related inputs, to regulate local plasticity in MOp, we performed monosynaptic rabies tracing experiments and employed an automated cell counting pipeline to generate a comprehensive map of brain-wide inputs to VIP-INs in MOp. We then compared this input profile to the brain-wide inputs to somatostatin-expressing inhibitory interneurons (SST-INs) and parvalbumin-expressing inhibitory interneurons (PV-INs) in MOp. We found that while all cell types received major inputs from sensory, motor, and prefrontal cortical regions, as well as from various thalamic nuclei, VIP-INs received more inputs from the orbital frontal cortex (ORB) - a region associated with reinforcement learning and value predictions. Our findings provide insight on how the brain leverages microcircuit motifs by both integrating and partitioning different streams of long-range input to modulate local circuit activity and plasticity.

Long-term effects of THC exposure on reward learning and motivated behavior in adolescent and adult male rats

RATIONALE

The endocannabinoid system makes critical contributions to reward processing, motivation, and behavioral control. Repeated exposure to THC or other cannabinoid drugs can cause persistent adaptions in the endocannabinoid system and associated neural circuitry. It remains unclear how such treatments affect the way rewards are processed and pursued.

OBJECTIVE AND METHODS

We examined if repeated THC exposure (5 mg/kg/day for 14 days) during adolescence or adulthood led to long-term changes in rats' capacity to flexibly encode and use action-outcome associations for goal-directed decision making. Effects on hedonic feeding and progressive ratio responding were also assessed.

RESULTS

THC exposure had no effect on rats' ability to flexibly select actions following reward devaluation. However, instrumental contingency degradation learning, which involves avoiding an action that is unnecessary for reward delivery, was augmented in rats with a history of adult but not adolescent THC exposure. THC-exposed rats also displayed more vigorous instrumental behavior in this study, suggesting a motivational enhancement. A separate experiment found that while THC exposure had no effect on hedonic feeding behavior, it increased rats' willingness to work for food on a progressive ratio schedule, an effect that was more pronounced when THC was administered to adults. Adolescent and adult THC exposure had opposing effects on the CB1 receptor dependence of progressive ratio performance, decreasing and increasing sensitivity to rimonabant-induced behavioral suppression, respectively.

CONCLUSIONS

Our findings reveal that exposure to a translationally relevant THC exposure regimen induces long-lasting, age-dependent alterations in cognitive and motivational processes that regulate the pursuit of rewards.

Computationally Informed Interventions for Targeting Compulsive Behaviors

Compulsive behaviors are central to addiction and obsessive-compulsive disorder and can be understood as a failure of adaptive decision making. Particularly, they can be conceptualized as an imbalance in behavioral control, such that behavior is guided predominantly by learned rather than inferred outcome expectations. Inference is a computational process required for adaptive behavior, and recent work across species has identified the neural circuitry that supports inference-based decision making. This includes the orbitofrontal cortex, which has long been implicated in disorders of compulsive behavior. Inspired by evidence that modulating orbitofrontal cortex activity can alter inference-based behaviors, here we discuss noninvasive approaches to target these circuits in humans. Specifically, we discuss the potential of network-targeted transcranial magnetic stimulation and real-time neurofeedback to modulate the neural underpinnings of inference. Both interventions leverage recent advances in our understanding of the neurocomputational mechanisms of inference-based behavior and may be used to complement current treatment approaches for behavioral disorders.

Inhibition of noradrenergic signalling in rodent orbitofrontal cortex impairs the updating of goal-directed actions

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.

Disinhibition of the orbitofrontal cortex biases decision-making in obesity

The lateral orbitofrontal cortex (lOFC) receives sensory information about food and integrates these signals with expected outcomes to guide future actions, and thus may play a key role in a distributed network of neural circuits that regulate feeding behavior. Here, we reveal a new role for the lOFC in the cognitive control of behavior in obesity. Food-seeking behavior is biased in obesity such that in male obese mice, behaviors are less flexible to changes in the perceived value of the outcome. Obesity is associated with reduced lOFC inhibitory drive and chemogenetic reduction in GABAergic neurotransmission in the lOFC induces obesity-like impairments in goal-directed behavior. Conversely, pharmacological or optogenetic restoration of inhibitory neurotransmission in the lOFC of obese mice reinstates flexible behavior. Our results indicate that obesity-induced disinhibition of the lOFC leads to a failure to update changes in the value of food with satiety, which in turn may influence how individuals make decisions in an obesogenic environment.

Orbitofrontal cortex populations are differentially recruited to support actions

The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.

Medial orbitofrontal neurotrophin systems integrate hippocampal input into outcome-specific value representations

In everyday life, we mentally represent possible consequences of our behaviors and integrate specific outcome values into existing knowledge to inform decisions. The medial orbitofrontal cortex (MO) is necessary to adapt behaviors when outcomes are not immediately available-when they and their values need to be envisioned. Nevertheless, neurobiological mechanisms remain unclear. We find that the neuroplasticity-associated neurotrophin receptor tropomyosin receptor kinase B (TrkB) is necessary for mice to integrate outcome-specific value information into choice behavior. This function appears attributable to memory updating (and not retrieval) and the stabilization of dendritic spines on excitatory MO neurons, which led us to investigate inputs to the MO. Ventral hippocampal (vHC)-to-MO projections appear conditionally necessary for value updating, involved in long-term aversion-based value memory updating. Furthermore, vHC-MO-mediated control of choice is TrkB dependent. Altogether, we reveal a vHC-MO connection by which specific value memories are updated, and we position TrkB within this functional circuit.

Thalamic regulation of frontal interactions in human cognitive flexibility

Interactions across frontal cortex are critical for cognition. Animal studies suggest a role for mediodorsal thalamus (MD) in these interactions, but the computations performed and direct relevance to human decision making are unclear. Here, inspired by animal work, we extended a neural model of an executive frontal-MD network and trained it on a human decision-making task for which neuroimaging data were collected. Using a biologically-plausible learning rule, we found that the model MD thalamus compressed its cortical inputs (dorsolateral prefrontal cortex, dlPFC) underlying stimulus-response representations. Through direct feedback to dlPFC, this thalamic operation efficiently partitioned cortical activity patterns and enhanced task switching across different contingencies. To account for interactions with other frontal regions, we expanded the model to compute higher-order strategy signals outside dlPFC, and found that the MD offered a more efficient route for such signals to switch dlPFC activity patterns. Human fMRI data provided evidence that the MD engaged in feedback to dlPFC, and had a role in routing orbitofrontal cortex inputs when subjects switched behavioral strategy. Collectively, our findings contribute to the emerging evidence for thalamic regulation of frontal interactions in the human brain.

The expansion of frontal cortex during mammalian evolution suggested a prominent role in intelligent and adaptive behavior, overshadowing earlier parts of the brain. However, recent rodent studies have pointed to a role for the cognitive mediodorsal thalamus (MD) in sustaining and flexibly switching representations in the frontal cortex, but direct relevance to human decision-making are unclear. Here, inspired by animal work, we extended a neural model of an executive frontal-MD network and trained it on a human decision-making task for which human neuroimaging data were collected. We found that the model MD thalamus learned an abstract representation of its cortical inputs and provided direct feedback to frontal cortex leading to flexible computations and enhanced task switching. These abstract MD representations and ability to re-organize frontal computations created an efficient mechanism where MD can integrate input from other regions to select behavioral strategy dynamically. The model predicted an efficient route through the MD for frontal region interactions and we found consistent evidence in human neuroimaging data. Collectively, our findings contribute to the emerging evidence for thalamic regulation of frontal interactions in the human brain.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

A molecularly integrated amygdalo-fronto-striatal network coordinates flexible learning and memory

Behavioral flexibility-that is, the ability to deviate from established behavioral sequences-is critical for navigating dynamic environments and requires the durable encoding and retrieval of new memories to guide future choice. The orbitofrontal cortex (OFC) supports outcome-guided behaviors. However, the coordinated neural circuitry and cellular mechanisms by which OFC connections sustain flexible learning and memory remain elusive. Here we demonstrate in mice that basolateral amygdala (BLA)→OFC projections bidirectionally control memory formation when familiar behaviors are unexpectedly not rewarded, whereas OFC→dorsomedial striatum (DMS) projections facilitate memory retrieval. OFC neuronal ensembles store a memory trace for newly learned information, which appears to be facilitated by circuit-specific dendritic spine plasticity and neurotrophin signaling within defined BLA-OFC-DMS connections and obstructed by cocaine. Thus, we describe the directional transmission of information within an integrated amygdalo-fronto-striatal circuit across time, whereby novel memories are encoded by BLA→OFC inputs, represented within OFC ensembles and retrieved via OFC→DMS outputs during future choice.

Blunted reward prediction error signals in internet gaming disorder

BACKGROUND

Internet gaming disorder (IGD) is a type of behavioural addictions. One of the key features of addiction is the excessive exposure to addictive objectives (e.g. drugs) reduces the sensitivity of the brain reward system to daily rewards (e.g. money). This is thought to be mediated via the signals expressed as dopaminergic reward prediction error (RPE). Emerging evidence highlights blunted RPE signals in drug addictions. However, no study has examined whether IGD also involves alterations in RPE signals that are observed in other types of addictions.

METHODS

To fill this gap, we used functional magnetic resonance imaging data from 45 IGD and 42 healthy controls (HCs) during a reward-related prediction-error task and utilised a psychophysiological interaction (PPI) analysis to characterise the underlying neural correlates of RPE and related functional connectivity.

RESULTS

Relative to HCs, IGD individuals showed impaired reinforcement learning, blunted RPE signals in multiple regions of the brain reward system, including the right caudate, left orbitofrontal cortex (OFC), and right dorsolateral prefrontal cortex (DLPFC). Moreover, the PPI analysis revealed a pattern of hyperconnectivity between the right caudate, right putamen, bilateral DLPFC, and right dorsal anterior cingulate cortex (dACC) in the IGD group. Finally, linear regression suggested that the connection between the right DLPFC and right dACC could significantly predict the variation of RPE signals in the left OFC.

CONCLUSIONS

These results highlight disrupted RPE signalling and hyperconnectivity between regions of the brain reward system in IGD. Reinforcement learning deficits may be crucial underlying characteristics of IGD pathophysiology.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

Neural circuits for inference-based decision-making

In novel situations, where direct experience is lacking or outdated, humans must rely on mental simulations to predict future outcomes. This review discusses recent work on the neural circuits that support such inference-based behavior. We focus on two specific examples: 1) using knowledge about the associative structure of the world to infer outcomes when direct experience is lacking; 2) inferring the current value of options when the desirability of the associated outcome has changed since the original learning experience. These two examples can be studied in the sensory preconditioning and devaluation tasks, respectively. We review results from studies in animals and humans suggesting that the orbitofrontal cortex (OFC), together with the hippocampus and amygdala, is necessary for inference in both of these tasks. Together, these findings suggest that the OFC is a critical hub in the brain network that supports inference-based decision-making.

Pharmacologically induced N-methyl-D-aspartate receptor hypofunction impairs goal-directed food seeking in rats

AIM

Acute N-methyl-D-aspartate (NMDA) receptor antagonism is an important pharmacological animal model of schizophrenia. In previous studies, schizophrenia patients show impaired goal-directed behavior in an outcome-specific devaluation procedure. In this study, we investigated whether the rat model of the NMDA receptor blockade also showed altered goal-directed behavior in a satiety-induced outcome devaluation paradigm.

METHODS

In experiments 1 and 2, we aimed to establish the satiety-induced outcome devaluation test using sucrose and lipid rewards in operant conditioning and free consumption paradigms. In experiment 3, we tested the effect of MK-801 (0.1 mg/kg, i.p.) on outcome-specific devaluation.

RESULTS

Experiments 1 and 2 demonstrated that 1-h ad libitum food consumption is sufficient to induce outcome-specific devaluation in both lever-press and free consumption tests in rats. Experiment 3 showed that the administration of MK-801 impaired satiety-induced devaluation in the lever-press test but not in the subsequent free consumption test.

CONCLUSIONS

Our results suggest that acute pharmacological NMDA receptor antagonism in rats is a useful animal model for impaired goal-directed behavior in schizophrenia.

Controlling one's world: Identification of sub-regions of primate PFC underlying goal-directed behavior

Impaired detection of causal relationships between actions and their outcomes can lead to maladaptive behavior. However, causal roles of specific prefrontal cortex (PFC) sub-regions and the caudate nucleus in mediating such relationships in primates are unclear. We inactivated and overactivated five PFC sub-regions, reversibly and pharmacologically: areas 24 (perigenual anterior cingulate cortex), 32 (medial PFC), 11 (anterior orbitofrontal cortex, OFC), 14 (rostral ventromedial PFC/medial OFC), and 14-25 (caudal ventromedial PFC) and the anteromedial caudate to examine their role in expressing learned action-outcome contingencies using a contingency degradation paradigm in marmoset monkeys. Area 24 or caudate inactivation impaired the response to contingency change, while area 11 inactivation enhanced it, and inactivation of areas 14, 32, or 14-25 had no effect. Overactivation of areas 11 and 24 impaired this response. These findings demonstrate the distinct roles of PFC sub-regions in goal-directed behavior and illuminate the candidate neurobehavioral substrates of psychiatric disorders, including obsessive-compulsive disorder.

•

Monkey pgACC-24 is necessary for detecting causal control of actions over outcomes

•

Its projection target in the caudate nucleus is also implicated

•

Three other subregions of the ventromedial prefrontal cortex are not necessary

•

Anterior OFC-11 may mediate Pavlovian influences on goal-directed behavior

Call for a more balanced approach to understanding orbital frontal cortex function

Orbital frontal cortex (OFC) research has historically emphasized the function of this associative cortical area within top-down theoretical frameworks. This approach has largely focused on mapping OFC activity onto human-defined psychological or cognitive constructs and has often led to OFC circuitry bearing the weight of entire theoretical frameworks. New techniques and tools developed in the last decade have made it possible to revisit long-standing basic science questions in neuroscience and answer them with increasing sophistication. We can now study and specify the genetic, molecular, cellular, and circuit architecture of a brain region in much greater detail, which allows us to piece together how they contribute to emergent circuit functions. For instance, adopting such systematic and unbiased bottom-up approaches to elucidating the function of the visual system has paved the way to building a greater understanding of the spectrum of its computational capabilities. In the same vein, we argue that OFC research would benefit from a more balanced approach that also places focus on novel bottom-up investigations into OFC's computational capabilities. Furthermore, we believe that the knowledge gained by employing a more bottom-up approach to investigating OFC function will ultimately allow us to look at OFC's dysfunction in disease through a more nuanced biological lens. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Parafascicular Thalamic and Orbitofrontal Cortical Inputs to Striatum Represent States for Goal-Directed Action Selection

Several lines of evidence accrued over the last 5–10 years have converged to suggest that the parafascicular nucleus of the thalamus and the lateral orbitofrontal cortex each represent or contribute to internal state/context representations that guide action selection in partially observable task situations. In rodents, inactivations of each structure have been found to selectively impair performance in paradigms testing goal-directed action selection, but only when that action selection relies on state representations. Electrophysiological evidence has suggested that each structure achieves this function via inputs onto cholinergic interneurons (CINs) in the dorsomedial striatum. Here, we briefly review these studies, then point to anatomical evidence regarding the afferents of each structure and what they suggest about the specific features that each contribute to internal state representations. Finally, we speculate as to whether this role might be achieved interdependently through direct PF→OFC projections, or through the convergence of independent direct orbitofrontal cortex (OFC) and parafascicular nucleus of the thalamus (PF) inputs onto striatal targets.

To be specific: The role of orbitofrontal cortex in signaling reward identity

The orbitofrontal cortex (OFC) plays a prominent role in signaling reward expectations. Two important features of rewards are their value (how good they are) and their specific identity (what they are). Whereas research on OFC has traditionally focused on reward value, recent findings point toward a pivotal role of reward identity in understanding OFC signaling and its contribution to behavior. Here, we review work in rodents, nonhuman primates, and humans on how the OFC represents expectations about the identity of rewards, and how these signals contribute to outcome-guided behavior. Moreover, we summarize recent findings suggesting that specific reward expectations in OFC are learned and updated by means of identity errors in the dopaminergic midbrain. We conclude by discussing how OFC encoding of specific rewards complements recent proposals that this region represents a cognitive map of relevant task states, which forms the basis for model-based behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Stimulation in the Rat Anterior Insula and Anterior Cingulate During an Effortful Weightlifting Task

When performing tasks, animals must continually assess how much effort is being expended, and gage this against ever-changing physiological states. As effort costs mount, persisting in the task may be unwise. The anterior cingulate cortex (ACC) and the anterior insular cortex are implicated in this process of cost-benefit decision-making, yet their precise contributions toward driving effortful persistence are not well understood. Here we investigated whether electrical stimulation of the ACC or insular cortex would alter effortful persistence in a novel weightlifting task (WLT). In the WLT an animal is challenged to pull a rope 30 cm to trigger food reward dispensing. To make the action increasingly effortful, 45 g of weight is progressively added to the rope after every 10 successful pulls. The animal can quit the task at any point - with the rope weight at the time of quitting taken as the "break weight." Ten male Sprague-Dawley rats were implanted with stimulating electrodes in either the ACC [cingulate cortex area 1 (Cg1) in rodent] or anterior insula and then assessed in the WLT during stimulation. Low-frequency (10 Hz), high-frequency (130 Hz), and sham stimulations were performed. We predicted that low-frequency stimulation (LFS) of Cg1 in particular would increase persistence in the WLT. Contrary to our predictions, LFS of Cg1 resulted in shorter session duration, lower break weights, and fewer attempts on the break weight. High-frequency stimulation of Cg1 led to an increase in time spent off-task. LFS of the anterior insula was associated with a marginal increase in attempts on the break weight. Taken together our data suggest that stimulation of the rodent Cg1 during an effortful task alters certain aspects of effortful behavior, while insula stimulation has little effect.

Infralimbic cortical glutamate output is necessary for the neural and behavioral consequences of chronic stress

Exposure to prolonged stress is a major risk-factor for psychiatric disorders such as generalized anxiety and major depressive disorder. Human imaging studies have identified structural and functional abnormalities in the prefrontal cortex of subjects with depression and anxiety disorders, particularly Brodmann's area 25 (BA25). Further, deep brain stimulation of BA25 reduces symptoms of treatment-resistant depression. The rat homolog of BA25 is the infralimbic cortex (IL), which is critical for cognitive appraisal, executive function, and physiological stress reactivity. Previous studies indicate that the IL undergoes stress-induced changes in excitatory/inhibitory balance culminating in reduced activity of glutamate output neurons. However, the regulatory role of IL glutamate output in mood-related behaviors after chronic variable stress (CVS) is unknown. Here, we utilized a lentiviral-packaged small-interfering RNA to reduce translation of vesicular glutamate transporter 1 (vGluT1 siRNA), thereby constraining IL glutamate output. This viral-mediated gene transfer was used in conjunction with a quantitative anatomical analysis of cells expressing the stable immediate-early gene product FosB/ΔFosB, which accumulates in response to repeated neural activation. Through assessment of FosB/ΔFosB-expressing neurons across the frontal lobe in adult male rats, we mapped regions altered by chronic stress and determined the coordinating role of the IL in frontal cortical plasticity. Specifically, CVS-exposed rats had increased density of FosB/ΔFosB-expressing cells in the IL and decreased density in the insula. The latter effect was dependent on IL glutamate output. Next, we examined the interaction of CVS and reduced IL glutamate output in behavioral assays examining coping, anxiety-like behavior, associative learning, and nociception. IL glutamate knockdown decreased immobility during the forced swim test compared to GFP controls, both in rats exposed to CVS as well as rats without previous stress exposure. Further, vGluT1 siRNA prevented CVS-induced avoidance behaviors, while also reducing risk aversion and passive coping. Ultimately, this study identifies the necessity of IL glutamatergic output for regulating frontal cortical neural activity and behavior following chronic stress. These findings also highlight how disruption of excitatory/inhibitory balance within specific frontal cortical cell populations may impact neurobehavioral adaptation and lead to stress-related disorders.

Processing in Lateral Orbitofrontal Cortex Is Required to Estimate Subjective Preference during Initial, but Not Established, Economic Choice

The orbitofrontal cortex (OFC) is proposed to be critical to economic decision making. Yet one can inactivate OFC without affecting well-practiced choices. One possible explanation of this lack of effect is that well-practiced decisions are codified into habits or configural-based policies not normally thought to require OFC. Here, we tested this idea by training rats to choose between different pellet pairs across a set of standard offers and then inactivating OFC subregions during choices between novel offers of previously experienced pairs or between novel pairs of previously experienced pellets. Contrary to expectations, controls performed as well on novel as experienced offers yet had difficulty initially estimating their subjective preference on novel pairs, difficulty exacerbated by lateral OFC inactivation. This pattern of results indicates that established economic choice reflects the use of an underlying model or goods space and that lateral OFC is only required for normal behavior when the established framework must incorporate new information.

Targeted Stimulation of an Orbitofrontal Network Disrupts Decisions Based on Inferred, Not Experienced Outcomes

When direct experience is unavailable, animals and humans can imagine or infer the future to guide decisions. Behavior based on direct experience versus inference may recruit partially distinct brain circuits. In rodents, the orbitofrontal cortex (OFC) contains neural signatures of inferred outcomes, and OFC is necessary for behavior that requires inference but not for responding driven by direct experience. In humans, OFC activity is also correlated with inferred outcomes, but it is unclear whether OFC activity is required for inference-based behavior. To test this, we used noninvasive network-based continuous theta burst stimulation (cTBS) in human subjects (male and female) to target lateral OFC networks in the context of a sensory preconditioning task that was designed to isolate inference-based behavior from responding that can be based on direct experience alone. We show that, relative to sham, cTBS targeting this network impairs reward-related behavior in conditions in which outcome expectations have to be mentally inferred. In contrast, OFC-targeted stimulation does not impair behavior that can be based on previously experienced stimulus–outcome associations. These findings suggest that activity in the targeted OFC network supports decision-making when outcomes have to be mentally simulated, providing converging cross-species evidence for a critical role of OFC in model-based but not model-free control of behavior.

SIGNIFICANCE STATEMENT It is widely accepted that the orbitofrontal cortex (OFC) is important for decision-making. However, it is less clear how exactly this region contributes to behavior. Here we test the hypothesis that the human OFC is only required for decision-making when future outcomes have to be mentally simulated, but not when direct experience with stimulus–outcome associations is available. We show that targeting OFC network activity in humans using network-based continuous theta burst stimulation selectively impairs behavior that requires inference but does not affect responding that can be based solely on direct experience. These results are in line with previous findings in animals and suggest a critical role for human OFC in model-based but not model-free behavior.

Adaptive behaviour under conflict: Deconstructing extinction, reversal, and active avoidance learning

In complex environments, organisms must respond adaptively to situations despite conflicting information. Under natural (i.e. non-laboratory) circumstances, it is rare that cues or responses are consistently paired with a single outcome. Inconsistent pairings are more common, as are situations where cues and responses are associated with multiple outcomes. Such inconsistency creates conflict, and a response that is adaptive in one scenario may not be adaptive in another. Learning to adjust responses accordingly is important for species to survive and prosper. Here we review the behavioural and brain mechanisms of responding under conflict by focusing on three popular behavioural procedures: extinction, reversal learning, and active avoidance. Extinction involves adapting from reinforcement to non-reinforcement, reversal learning involves swapping the reinforcement of cues or responses, and active avoidance involves performing a response to avoid an aversive outcome, which may conflict with other defensive strategies. We note that each of these phenomena relies on somewhat overlapping neural circuits, suggesting that such circuits may be critical for the general ability to respond appropriately under conflict.

Review of Orbitofrontal Cortex in Alcohol Dependence: A Disrupted Cognitive Map?

Alcoholism is a persistent worldwide problem associated with long-lasting impairments to decision making processes. Some aspects of dysfunction are thought to reflect alcohol-induced changes to relevant brain areas such as the orbitofrontal cortex (OFC). In this review, we will examine how chronic alcohol exposure alters OFC function to potentially contribute to maladaptive decision making, and explore experimental behavioral approaches that may be better suited to test whether alcohol dependence disrupts OFC's function. We argue that although past works suggest impairments in aspects of OFC function, more information may be gained by specifically targeting tasks to the broader function of OFC as put forth by the recent hypothesis of OFC as a "cognitive map" of task space. Overall, we suggest that such a focus could provide a better understanding of how OFC function changes in alcohol dependence, and could inform better assessment tools and treatment options for clinicians working with this population.

Targeting Catecholaminergic Systems in Transgenic Rats With a CAV-2 Vector Harboring a Cre-Dependent DREADD Cassette

Techniques that allow the manipulation of specific neural circuits have greatly increased in the past few years. DREADDs (Designer receptors exclusively activated by designer drugs) provide an elegant way to manipulate individual brain structures and/or neural circuits, including neuromodulatory pathways. Considerable efforts have been made to increase cell-type specificity of DREADD expression while decreasing possible limitations due to multiple viral vectors injections. In line with this, a retrograde canine adenovirus type 2 (CAV-2) vector carrying a Cre-dependent DREADD cassette has been recently developed. In combination with Cre-driver transgenic animals, the vector allows one to target neuromodulatory pathways with cell-type specificity. In the present study, we specifically targeted catecholaminergic pathways by injecting the vector in knock-in rat line containing Cre recombinase cassette under the control of the tyrosine hydroxylase promoter. We assessed the efficacy of infection of the nigrostriatal pathway and the catecholaminergic pathways ascending to the orbitofrontal cortex (OFC) and found cell-type-specific DREADD expression.

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal-directed action: A brief review

Goal-directed action refers to selecting behaviors based on the expectation that they will be reinforced with desirable outcomes. It is typically conceptualized as opposing habit-based behaviors, which are instead supported by stimulus-response associations and insensitive to consequences. The prelimbic prefrontal cortex (PL) is positioned along the medial wall of the rodent prefrontal cortex. It is indispensable for action-outcome-driven (goal-directed) behavior, consolidating action-outcome relationships and linking contextual information with instrumental behavior. In this brief review, we will discuss the growing list of molecular factors involved in PL function. Ventral to the PL is the medial orbitofrontal cortex (mOFC). We will also summarize emerging evidence from rodents (complementing existing literature describing humans) that it too is involved in action-outcome conditioning. We describe experiments using procedures that quantify responding based on reward value, the likelihood of reinforcement, or effort requirements, touching also on experiments assessing food consumption more generally. We synthesize these findings with the argument that the mOFC is essential to goal-directed action when outcome value information is not immediately observable and must be recalled and inferred.

The transition to compulsion in addiction

Compulsion is a cardinal symptom of drug addiction (severe substance use disorder). However, compulsion is observed in only a small proportion of individuals who repeatedly seek and use addictive substances. Here, we integrate accounts of the neuropharmacological mechanisms that underlie the transition to compulsion with overarching learning theories, to outline how compulsion develops in addiction. Importantly, we emphasize the conceptual distinctions between compulsive drug-seeking behaviour and compulsive drug-taking behaviour (that is, use). In the latter, an individual cannot stop using a drug despite major negative consequences, possibly reflecting an imbalance in frontostriatal circuits that encode reward and aversion. By contrast, an individual may compulsively seek drugs (that is, persist in seeking drugs despite the negative consequences of doing so) when the neural systems that underlie habitual behaviour dominate goal-directed behavioural systems, and when executive control over this maladaptive behaviour is diminished. This distinction between different aspects of addiction may help to identify its neural substrates and new treatment strategies.

Behavioral flexibility is associated with changes in structure and function distributed across a frontal cortical network in macaques

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Action-Outcome Expectancies Require Orbitofrontal Neurotrophin Systems in Naïve and Cocaine-Exposed Mice

Cocaine use during adolescence decreases the likelihood that individuals will seek treatment for recurrent drug use. In rodents, developmental cocaine exposure weakens action-consequence decision-making, causing a deferral to familiar, habit-like behavioral response strategies. Here, we aimed to improve action-outcome decision-making. We found that acute pharmacological stimulation of the tyrosine/tropomyosin receptor kinase B (trkB) via 7,8-dihydroxyflavone (7,8-DHF) or 3,4-methylenedioxymethamphetamine (MDMA) blocked cocaine-induced habit biases by strengthening memory for action-outcome associations. We believe that MDMA acts by stimulating neurotrophin/trkB systems in the orbitofrontal cortex (OFC), a region involved in prospectively evaluating the consequences of one's action, because 1) MDMA also increased brain-derived neurotrophic factor (BDNF) in the OFC, 2) MDMA corrected habit biases due to Bdnf loss in the OFC, and 3) overexpression of a truncated isoform of trkB occluded the memory-enhancing effects of MDMA. Thus, selecting actions based on their consequences requires BDNF-trkB in the OFC, the stimulation of which may improve goal attainment in both drug-naïve and cocaine-exposed individuals. SIGNIFICANCE STATEMENT: Cocaine use during adolescence decreases the likelihood that individuals will seek treatment for recurrent drug use, even as adults. Understanding how early-life cocaine exposure impacts goal-oriented action and prospective decision-making in adulthood is thus important. One key aspect of goal-directed decision-making is anticipating the consequences of one's actions, a process that likely involves the orbitofrontal cortex (OFC). In rodents, developmental cocaine exposure weakens action-consequence decision-making, causing a deferral to familiar, habit-like behavioral response strategies. Here, we report that we can improve memory for action-consequence relationships by stimulating neurotrophic factors, which support cell survival, development, and plasticity in the brain. With strengthened action-consequence associations, cocaine-exposed mice regain the ability to optimally select actions based on their likely outcomes. Brain region-selective manipulations reveal that neurotrophin systems in the OFC are necessary for stable memory of action-consequence relationships.

Real-Time Value Integration during Economic Choice Is Regulated by Orbitofrontal Cortex

Neural correlates implicate the orbitofrontal cortex (OFC) in value-based or economic decision making [1-3]. Yet inactivation of OFC in rats performing a rodent version of the standard economic choice task is without effect [4, 5], a finding more in accord with ideas that the OFC is primarily necessary for behavior when new information must be taken into account [6-9]. Neural activity in the OFC spontaneously updates to reflect new information, particularly about outcomes [10-16], and the OFC is necessary for adjustments to learned behavior only under these conditions [4, 16-26]. Here, we merge these two independent lines of research by inactivating lateral OFC during an economic choice that requires new information about the value of the predicted outcomes to be incorporated into an already established choice. Outcome value was changed by pre-feeding the rats one of two food options before testing. In control rats, this pre-feeding resulted in divergent changes in choice behavior that depended on the rats' prior preference for the pre-fed food. Optogenetic inactivation of the OFC disrupted this bi-directional effect of pre-feeding without affecting other measures that describe the underlying choice behavior. This finding unifies the role of the OFC in economic choice with its role in a host of other behaviors, causally demonstrating that the OFC is not necessary for economic choice per se-unless that choice incorporates new information about the outcomes.

mTORC1 in the orbitofrontal cortex promotes habitual alcohol seeking

The mechanistic target of rapamycin complex 1 (mTORC1) plays an important role in dendritic translation and in learning and memory. We previously showed that heavy alcohol use activates mTORC1 in the orbitofrontal cortex (OFC) of rodents (Laguesse et al., 2017a). Here, we set out to determine the consequences of alcohol-dependent mTORC1 activation in the OFC. We found that inhibition of mTORC1 activity in the OFC attenuates alcohol seeking and restores sensitivity to outcome devaluation in rats that habitually seek alcohol. In contrast, habitual responding for sucrose was unaltered by mTORC1 inhibition, suggesting that mTORC1’s role in habitual behavior is specific to alcohol. We further show that inhibition of GluN2B in the OFC attenuates alcohol-dependent mTORC1 activation, alcohol seeking and habitual responding for alcohol. Together, these data suggest that the GluN2B/mTORC1 axis in the OFC drives alcohol seeking and habit.

Targeting Reciprocally Connected Brain Regions Through CAV-2 Mediated Interventions

An important issue in contemporary neuroscience is to identify functional principles at play within neural circuits. The reciprocity of the connections between two distinct brain areas appears as an intriguing feature of some of these circuits. This organization has been viewed as "re-entry," a process whereby two or more brain regions concurrently stimulate and are stimulated by each other, thus supporting the synchronization of neural firing required for rapid neural integration. However, until relatively recently, it was not possible to provide a comprehensive functional assessment of such reciprocal pathways. In this Brief Research Report, we highlight the use of a chemogenetic strategy to target projection-defined neurons in reciprocally connected areas through CAV-2 mediated interventions in the rat. Specifically, we targeted the bidirectional pathways between the dorsomedial prefrontal cortex (dmPFC) and the mediodorsal thalamus, as well as those connecting the insular cortex (IC) and the basolateral complex of the amygdala (BLA). These data showcase the usefulness of CAV-2-related strategies to address circuit-level issues. Moreover, we illustrate the inherent limitation of Cre-dependent adeno-associated virues (AAVs) with "leaked" expression of the gene of interest in the absence of Cre and highlight the need for appropriate control conditions.

Glucocorticoid-sensitive ventral hippocampal-orbitofrontal cortical connections support goal-directed action - Curt Richter Award Paper 2019

In an ever-changing and often ambiguous environment, organisms must use previously learned associations between antecedents and outcomes to predict future associations and make optimal choices. Chronic stress can impair one's ability to flexibly adjust behaviors when environmental contingencies change, particularly in cases of early-life stress. In mice, exposure to elevated levels of the primary stress hormone, corticosterone (CORT), during early adolescence is sufficient to impair response-outcome decision making later in life, biasing response strategies towards inflexible habits. Nevertheless, neurobiological mechanisms are still being defined. Here, we report that exposure to excess CORT in adolescence causes a loss of dendritic spines on excitatory pyramidal neurons in the lateral, but not medial, orbital prefrontal cortex (loPFC) of mice, and spine loss correlates with the severity of habit biases in adulthood. Excess CORT also reduces the presence of ventral hippocampal (vHC) axon terminals in the loPFC. To identify functional consequences, we inactivated vHC→loPFC projections in typical healthy mice during a period when mice must update response-outcome expectations to optimally acquire food reinforcers. Inactivation impaired the animals' subsequent ability to sustainably choose actions based on likely outcomes, causing them to defer to habit-based response strategies. Thus, vHC→loPFC projections are necessary for response-outcome expectancy updating and a target of excess glucocorticoids during early-life development. Their degradation is likely involved in long-term biases towards habit-based behaviors following glucocorticoid excess in adolescence.

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems

This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.

Pre-training inactivation of basolateral amygdala and mediodorsal thalamus, but not orbitofrontal cortex or prelimbic cortex, impairs devaluation in a multiple-response/multiple-reinforcer cued operant task

Reinforcer devaluation is a task often used to model flexible goal-directed behavior. Here, we inactivated basolateral amygdala (BLA), orbitofrontal cortex (OFC), mediodorsal thalamus (MD) (Exp. 1) and prelimbic cortex (PL) (Exp. 3) in rats during multiple-response/multiple-reinforcer operant training with levers available to earn reinforcers during cued trials. After two training days with each lever-food relationship, a reinforcer was devalued through selective satiety and devaluation was assessed in a choice test with the brain areas non-inactivated. The control and OFC and PL inactivation groups exhibited a devaluation effect, but the BLA or MD groups did not. Since the OFC is proposed to be required in devaluation tasks when a discrete cue signals an outcome and PL is proposed to be required when responses based on lever spatial-location guide behavior, we ran new rats through a cue-switching experiment (Exp. 2) to determine the strategy rats use to complete our task (attending to the discrete light cue or spatial lever location). Both groups (cue switched and cue normal) showed a devaluation effect based on the lever spatial location, suggesting that rats rely on the spatial lever location to guide behavior. Future studies will determine whether OFC and PL can compensate for each other to show intact devaluation when the functioning of one of them is impaired.

Social Isolation in Adolescence Disrupts Cortical Development and Goal-Dependent Decision-Making in Adulthood, Despite Social Reintegration

The social environment influences neurodevelopment. Investigations using rodents to study this phenomenon commonly isolate subjects, then assess neurobehavioral consequences while animals are still isolated. This approach precludes one from dissociating the effects of on-going versus prior isolation, hindering our complete understanding of the consequences of social experience during particular developmental periods. Here, we socially isolated adolescent mice from postnatal day (P)31 to P60, then re-housed them into social groups. We tested their ability to select actions based on expected outcomes using multiple reinforcer devaluation and instrumental contingency degradation techniques. Social isolation in adolescence (but not adulthood) weakened instrumental response updating, causing mice to defer to habit-like behaviors. Habit biases were associated with glucocorticoid insufficiency in adolescence, oligodendrocyte marker loss throughout cortico-striatal regions, and dendritic spine and synaptic marker excess in the adult orbitofrontal cortex (OFC). Artificial, chemogenetic stimulation of the ventrolateral OFC in typical, healthy mice recapitulated response biases following isolation, causing habit-like behaviors. Meanwhile, correcting dendritic architecture by inhibiting the cytoskeletal regulatory protein ROCK remedied instrumental response updating defects in socially isolated mice. Our findings suggest that adolescence is a critical period during which social experience optimizes one's ability to seek and attain goals later in life. Age-typical dendritic spine elimination appears to be an essential factor, and in its absence, organisms may defer to habit-based behaviors.

β1-Integrins in the Developing Orbitofrontal Cortex Are Necessary for Expectancy Updating in Mice

Navigating a changing environment requires associating stimuli and actions with their likely outcomes and modifying these associations when they change. These processes involve the orbitofrontal cortex (OFC). Although some molecular mediators have been identified, developmental factors are virtually unknown. We hypothesized that the cell adhesion factor β1-integrin is essential to OFC function, anticipating developmental windows during which β1-integrins might be more influential than others. We discovered that OFC-selective β1-integrin silencing before adolescence, but not later, impaired the ability of mice to extinguish conditioned fear and select actions based on their likely outcomes. Early-life knock-down also reduced the densities of dendritic spines, the primary sites of excitatory plasticity in the brain, and weakened sensitivity to cortical inputs. Notwithstanding these defects in male mice, females were resilient to OFC (but not hippocampal) β1-integrin loss. Existing literature suggests that resilience may be explained by estradiol-mediated transactivation of β1-integrins and tropomyosin receptor kinase B (trkB). Accordingly, we discovered that a trkB agonist administered during adolescence corrected reward-related decision making in β1-integrin-deficient males. In sum, developmental β1-integrins are indispensable for OFC function later in life.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) is a subregion of the frontal cortex that allows organisms to link behaviors and stimuli with anticipated outcomes, and to make predictions about the consequences of one's behavior. Aspects of OFC development are particularly prolonged, extending well into adolescence, likely optimizing organisms' abilities to prospectively calculate the consequences of their actions and select behaviors appropriately; these decision making strategies improve as young individuals mature into adulthood. Molecular factors are not, however, well understood. Our experiments reveal that a cell adhesion protein termed "β1-integrin" is necessary for OFC neuronal maturation and function. Importantly, β1-integrins operate during a critical period equivalent to early adolescence in humans to optimize the ability of organisms to update expectancies later in life.