Inactivation of prelimbic and infralimbic cortex respectively affects minimally-trained and extensively-trained goal-directed actions

Alcohol, flexible behavior, and the prefrontal cortex: Functional changes underlying impaired cognitive flexibility

Cognitive flexibility enables individuals to alter their behavior in response to changing environmental demands, facilitating optimal behavior in a dynamic world. The inability to do this, called behavioral inflexibility, is a pervasive behavioral phenotype in alcohol use disorder (AUD), driven by disruptions in cognitive flexibility. Research has repeatedly shown that behavioral inflexibility not only results from alcohol exposure across species but can itself be predictive of future drinking. Like many high-level executive functions, flexible behavior requires healthy functioning of the prefrontal cortex (PFC). The scope of this review addresses two primary themes: first, we outline tasks that have been used to investigate flexibility in the context of AUD or AUD models. We characterize these based on the task features and underlying cognitive processes that differentiate them from one another. We highlight the neural basis of flexibility measures, focusing on the PFC, and how acute or chronic alcohol in humans and non-human animal models impacts flexibility. Second, we consolidate findings on the molecular, physiological and functional changes in the PFC elicited by alcohol, that may contribute to cognitive flexibility deficits seen in AUD. Collectively, this approach identifies several key avenues for future research that will facilitate effective treatments to promote flexible behavior in the context of AUD, to reduce the risk of alcohol related harm, and to improve outcomes following AUD. This article is part of the Special Issue on "PFC circuit function in psychiatric disease and relevant models".

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

VTA Excitatory Neurons Control Reward-driven Behavior by Modulating Infralimbic Cortical Firing

The functional dichotomy of anatomical regions of the medial prefrontal cortex (mPFC) has been tested with greater certainty in punishment-driven tasks, and less so in reward-oriented paradigms. In the infralimbic cortex (IL), known for behavioral suppression (STOP), tasks linked with reward or punishment are encoded through firing rate decrease or increase, respectively. Although the ventral tegmental area (VTA) is the brain region governing reward/aversion learning, the link between its excitatory neuron population and IL encoding of reward-linked behavioral expression is unclear. Here, we present evidence that IL ensembles use a population-based mechanism involving broad inhibition of principal cells at intervals when reward is presented or expected. The IL encoding mechanism was consistent across multiple sessions with randomized rewarded target sites. Most IL neurons exhibit FR (Firing Rate) suppression during reward acquisition intervals (T1), and subsequent exploration of previously rewarded targets when the reward is omitted (T2). Furthermore, FR suppression in putative IL ensembles persisted for intervals that followed reward-linked target events. Pairing VTA glutamate inhibition with reward acquisition events reduced the weight of reward-target association expressed as a lower affinity for previously rewarded targets. For these intervals, fewer IL neurons per mouse trial showed FR decrease and were accompanied by an increase in the percentage of units with no change in FR. Together, we conclude that VTA glutamate neurons are likely involved in establishing IL inhibition states that encode reward acquisition, and subsequent reward-target association.

Habit and persistence

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus-response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus-response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus-response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response-outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus-response associations can be used to investigate habits.

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

Early-life seizures alter habit behavior formation and fronto-striatal circuit dynamics

Obsessive compulsive disorder (OCD) can occur comorbidly with epilepsy; both are complex, disruptive disorders that lower quality of life. Both OCD and epilepsy are disorders of hyperexcitable circuits, but it is unclear whether common circuit pathology may underlie the co-occurrence of these two neuropsychiatric disorders. Here, we induced early-life seizures (ELS) in rats to examine habit formation as a model for compulsive behaviors. Compulsive, repetitive behaviors in OCD utilize the same circuitry as habit formation. We hypothesized that rats with ELS could be more susceptible to habit formation than littermate controls, and that altered behavior would correspond to altered signaling in fronto-striatal circuits that underlie decision-making and action initiation. Here, we show instead that rats with ELS were significantly less likely to form habit behaviors compared with control rats. This behavioral difference corresponded with significant alterations to temporal coordination within and between brain regions that underpin the action to habit transition: 1) phase coherence between the lateral orbitofrontal cortex and dorsomedial striatum (DMS) and 2) theta-gamma coupling within DMS. Finally, we used cortical electrical stimulation as a model of transcranial magnetic stimulation (TMS) to show that temporal coordination of fronto-striatal circuits in control and ELS rats are differentially susceptible to potentiating and suppressive stimulation, suggesting that altered underlying circuit physiology may lead to altered response to therapeutic interventions such as TMS.

Cocaine and habit training cause dendritic spine rearrangement in the prelimbic cortex

Successfully navigating dynamic environments requires organisms to learn the consequences of their actions. The prelimbic prefrontal cortex (PL) formulates action-consequence memories and is modulated by addictive drugs like cocaine. We trained mice to obtain food rewards and then unexpectedly withheld reinforcement, triggering new action-consequence memory. New memory was disrupted by cocaine when delivered immediately following non-reinforcement, but not when delayed, suggesting that cocaine disrupted memory consolidation. Cocaine also rapidly inactivated cofilin, a primary regulator of the neuronal actin cytoskeleton. This observation led to the discovery that cocaine also within the time of memory consolidation elevated dendritic spine elimination and blunted spine formation rates on excitatory PL neurons, culminating in thin-type spine attrition. Training drug-naive mice to utilize inflexible response strategies also eliminated thin-type dendritic spines. Thus, cocaine may disrupt action-consequence memory, at least in part, by recapitulating neurobiological sequalae occurring in the formation of inflexible habits.

Prelimbic and infralimbic medial prefrontal cortex neuron activity signals cocaine seeking variables across multiple timescales

RATIONALE AND OBJECTIVES

The prefrontal cortex is critical for execution and inhibition of reward seeking. Neural manipulation of rodent medial prefrontal cortex (mPFC) subregions differentially impacts execution and inhibition of cocaine seeking. Dorsal, or prelimbic (PL), and ventral, or infralimbic (IL) mPFC are implicated in cocaine seeking or extinction of cocaine seeking, respectively. This differentiation is not seen across all studies, indicating that further research is needed to understand specific mPFC contributions to drug seeking.

METHODS

We recorded neuronal activity in mPFC subregions during cocaine self-administration, extinction, and cue- and cocaine-induced reinstatement of cocaine seeking.

RESULTS

Both PL and IL neurons were phasically responsive around lever presses during cocaine self-administration, and activity in both areas was reduced during extinction. During both cue- and, to a greater extent, cocaine-induced reinstatement, PL neurons exhibited significantly elevated responses, in line with previous studies demonstrating a role for the region in relapse. The enhanced PL signaling in cocaine-induced reinstatement was driven by strong excitation and inhibition in different groups of neurons. Both of these response types were stronger in PL vs. IL neurons. Finally, we observed tonic changes in activity in all tasks phases, reflecting both session-long contextual modulation as well as minute-to-minute activity changes that were highly correlated with brain cocaine levels and motivation associated with cocaine seeking.

CONCLUSIONS

Although some differences were observed between PL and IL neuron activity across sessions, we found no evidence of a go/stop dichotomy in PL/IL function. Instead, our results demonstrate temporally heterogeneous prefrontal signaling during cocaine seeking and extinction in both PL and IL, revealing novel and complex functions for both regions during these behaviors. This combination of findings argues that mPFC neurons, in both PL and IL, provide multifaceted contributions to the regulation of drug seeking and addiction.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

Where Actions Meet Outcomes: Medial Prefrontal Cortex, Central Thalamus, and the Basal Ganglia

Medial prefrontal cortex (mPFC) interacts with distributed networks that give rise to goal-directed behavior through afferent and efferent connections with multiple thalamic nuclei and recurrent basal ganglia-thalamocortical circuits. Recent studies have revealed individual roles for different thalamic nuclei: mediodorsal (MD) regulation of signaling properties in mPFC neurons, intralaminar control of cortico-basal ganglia networks, ventral medial facilitation of integrative motor function, and hippocampal functions supported by ventral midline and anterior nuclei. Large scale mapping studies have identified functionally distinct cortico-basal ganglia-thalamocortical subnetworks that provide a structural basis for understanding information processing and functional heterogeneity within the basal ganglia. Behavioral analyses comparing functional deficits produced by lesions or inactivation of specific thalamic nuclei or subregions of mPFC or the basal ganglia have elucidated the interdependent roles of these areas in adaptive goal-directed behavior. Electrophysiological recordings of mPFC neurons in rats performing delayed non-matching-to position (DNMTP) and other complex decision making tasks have revealed populations of neurons with activity related to actions and outcomes that underlie these behaviors. These include responses related to motor preparation, instrumental actions, movement, anticipation and delivery of action outcomes, memory delay, and spatial context. Comparison of results for mPFC, MD, and ventral pallidum (VP) suggest critical roles for mPFC in prospective processes that precede actions, MD for reinforcing task-relevant responses in mPFC, and VP for providing feedback about action outcomes. Synthesis of electrophysiological and behavioral results indicates that different networks connecting mPFC with thalamus and the basal ganglia are organized to support distinct functions that allow organisms to act efficiently to obtain intended outcomes.

Corticotropin-releasing factor receptor 1 in infralimbic cortex modulates social stress-altered decision-making

Chronic stress could lead to a bias in behavioral strategies toward habits. However, it remains unclear which neuronal system modulates stress-induced behavioral abnormality during decision making. The corticotropin-releasing factor (CRF) system in the medial prefrontal cortex (mPFC), which has been implicated in governing strategy choice, is involved in the response to stress. The present study aimed to clarify whether altered function in cortical CRF receptors is linked to abnormal behaviors after chronic stress. In results, mice subjected to a 10-day social defeat preferred to use a habitual strategy. The infralimbic cortex (IL), but not the prelimbic cortex (PL) or anterior cingulate cortex (ACC), showed higher cFos expression in stress-subjected mice than in control mice, which may be associated with habitual behavior choice. Furthermore, CRF receptor 1 (CRFR1) agonist and antagonist infusion in IL during behavioral training mimicked and rescued stress-caused behavioral change in the decision-making assessment, respectively. An electrophysiological approach showed that the frequencies of both spontaneous IPSC and spontaneous EPSC, but not their amplitude, increased after stress and were modulated by CRFR1 agents. Further recordings revealed that an increased ratio of excitation to inhibition (E/I ratio) of IL by stress was rescued under conditions with CRFR1 antagonist. Collectively, these data indicate that CRFR1 plays a critical role in stress-permitted or enhanced glutamatergic and GABAergic presynaptic transmission in direct or indirect ways, as well as the modulation for E/I ratio in the IL. Thus, CRFR1 in the mPFC may be a proper target for treating cases of chronic stress-altered behavior.

Cyclic estrogen and progesterone during instrumental acquisition contributes to habit formation in female rats

Habit formation is thought to involve two parallel processes that are mediated by distinct neural substates: one that suppresses goal-directed behavior, and one that facilitates stimulus-response (S-R) learning, which underscores habitual behavior. In previous studies we showed that habitual responding emerges early during instrumental training in gonadally-intact female, compared to male, rats. The present study aimed to determine the role of ovarian hormones during instrumental acquisition in the transition from goal-directed to habitual behavior in female rats. Ovariectomized (OVX) female rats were given subcutaneous silastic capsules that released low levels of 17-β estradiol (E2) to maintain estrogen receptor availability. Rats were assigned to one of three hormone treatment conditions: no additional hormone replacement (Control group), replacement with high E2 (High E2 group), or replacement with high E2 followed by progesterone (High E2 + P4 group). Hormone replacement occurred twice during acquisition to mimic natural hormone fluctuations. At test, the Control and High E2 groups demonstrated responding that was sensitive to devaluation by lithium chloride-induced illness, indicating goal-directed behavior. In contrast, the High E2 + P4 group exhibited a pattern of devaluation-insensitive, habitual responding, that suggested the suppression of goal-directed processes. In a follow-up experiment, similar procedures were conducted, however during acquisition, OVX rats were given cyclic high E2 plus medroxy-progesterone (MPA), a form of progesterone that does not metabolize to neuroactive metabolites. In this group, goal-directed behavior was observed. These data indicate that habit formation is not facilitated in low estrogen states, nor in the presence of cyclic high E2. However, cyclic high E2, together with progesterone during acquisition, appears to facilitate the early emergence of habitual responding. Furthermore, these data suggest that a neuroactive progesterone metabolite, like allopregnanolone, in combination with high cyclic E2, supports this phenomenon.

Opposing Roles of the Dorsolateral and Dorsomedial Striatum in the Acquisition of Skilled Action Sequencing in Rats

The shift in control from dorsomedial to dorsolateral striatum during skill and habit formation has been well established, but whether striatal subregions orchestrate this shift cooperatively or competitively remains unclear. Cortical inputs have also been implicated in the shift toward automaticity, but it is unknown whether they mirror their downstream striatal targets across this transition. We addressed these questions using a five step heterogeneous action sequencing task in male rats that is optimally performed by automated chains of actions. By optimizing automatic habitual responding, we discovered that loss of function in the dorsomedial striatum accelerated sequence acquisition. In contrast, loss of function in the dorsolateral striatum impeded acquisition of sequencing, demonstrating functional opposition within the striatum. Unexpectedly, the mPFC was not involved; however, the lateral orbitofrontal cortex was critical. These results shift current theories about striatal control of behavior to a model of competitive opposition, where the dorsomedial striatum interferes with the development of dorsolateral-striatum dependent behavior.SIGNIFICANCE STATEMENT We provide the most direct evidence to date that the dorsomedial and dorsolateral striatum compete for control in the acquisition of habitual action sequences. The dorsolateral striatum was critical for sequencing behavior, but loss of dorsomedial striatum function enhanced acquisition. In addition, we found that the mPFC was not required for the formation of automated actions. Using a task that optimizes habitual responding, we demonstrate that the arbitration of dorsomedial and dorsolateral control is not modulated by medial prefrontal cortical activity. However, we find evidence for the role of the lateral orbitofrontal cortex in action sequencing. These results have implications for our understanding of how habits and skills form.

Infralimbic cortex functioning across motivated behaviors: Can the differences be reconciled?

The rodent infralimbic cortex (IL) is implicated in higher order executive functions such as reward seeking and flexible decision making. However, the precise nature of its role in these processes is unclear. Early evidence indicated that the IL promotes the extinction and ongoing inhibition of fear conditioning and cocaine seeking. However, evidence spanning other behavioral domains, such as natural reward seeking and habit-based learning, suggests a more nuanced understanding of IL function. As techniques have advanced and more studies have examined IL function, identifying a unifying explanation for its behavioral function has become increasingly difficult. Here, we discuss evidence of IL function across motivated behaviors, including associative learning, drug seeking, natural reward seeking, and goal-directed versus habit-based behaviors, and emphasize how context-specific encoding and heterogeneous IL neuronal populations may underlie seemingly conflicting findings in the literature. Together, the evidence suggests that a major IL function is to facilitate the encoding and updating of contingencies between cues and behaviors to guide subsequent behaviors.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

2-Heptanone reduces inhibitory control of the amygdala over the prelimbic region in rats

Basolateral amygdala (BLA) nuclei and their reciprocal connections with prelimbic (PL) and infralimbic (IL) regions of the medial prefrontal cortex (mPFC) are involved in the regulation of fear. 2-Heptanone is released in urine in stressed rats, and the olfactory detection of this odor produces immediate avoidance and alarm reactions and modifies neuronal activity in limbic connections in non-stressed rats. If 2-heptanone acts as a danger signal, then long-lasting actions would be expected. The aim of the present study was to investigate whether the forced inhalation of 2-heptanone modifies the response capacity of the BLA-mPFC circuit in the long term (48 h). Single-unit extracellular recordings were obtained from the PL and IL during electrical stimulation of the BLA (square-wave pulses; 1 ms, 20 µA, 0.3 Hz, 110 stimuli over a total duration of 360 s) in three groups of Wistar rats: control group (no sensory stimulation), unpredictable auditory stimulation group, and 2-heptanone stimulation group. A brief-latency (1 ms), short-duration (5 ms) paucisynaptic response followed BLA stimulation and was unaffected by any sensorial stimulation. The paucisynaptic response was followed by a mostly inhibitory and long-lasting (>750 ms) afterdischarge in the control and auditory stimulation groups. In the 2-heptanone group, the inhibitory afterdischarge shifted to an excitatory afterdischarge after ∼250 ms in the PL and after ∼500 ms in the IL. Importantly, the rats that were included in this study were born in local housing facilities. Thus, these animals were never in contact with predators and instead in contact with only conspecifics. These results indicate that the forced inhalation of 2-heptanone is able to modify BLA-mPFC responsivity in the long term. 2-Heptanone decreases inhibitory control of the amygdala over mPFC activity. Disinhibition of the mPFC may lead to the adaptive expression of defensive behaviors, even in animals that are not in the presence of predators.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Dopamine sensitization by methamphetamine treatment prior to instrumental training delays the transition into habit in female rats

Early in instrumental learning, behavior is goal-directed and sensitive to changes in the value of the instrumental outcome. With sufficient repetition, responding becomes insensitive to changes in outcome value, or habitual. We have previously found that females transition into habit over a distinct range of training from 120 to 160 reinforced responses. This low level of instrumental training is markedly less than what has been shown to support habitual responding in male rats. To begin to investigate the early development of habit in females, we conducted a series of experiments in which we pretreated female rats with methamphetamine (METH) with the aim of sensitizing central dopamine, a major modulator of striatal function, prior to instrumental nose-poke training at the beginning and at the endpoint of the transition range in females. Following training, we tested for sensitivity to reinforcer devaluation (RD), which was conducted by repeatedly pairing reinforcers previously earned during training with lithium chloride (LiCl)-induced illness. As a counterpoint, a series of similar experiments was conducted separately in male rats. Additionally, in order to ascertain the validity of using nose-poke as an instrumental response, we compared sensitivity to devaluation between the Pavlovian approach towards the food magazine and the nose-poke response. In females, Vehicle groups responded in a habitual manner at both training levels (120 and 160 reinforced responses), whereas METH groups remained sensitive to devaluation. This suggests that increasing central dopamine delays habit formation in female rats. In male rats, Vehicle groups demonstrated goal-directed responding following training with 120 and 320 reinforced responses, and marginally goal-directed responding,with 160. METH-pretreated males were sensitive to devaluation at the 120 and 160 training levels, however, following more extended training to 320 reinforced responses, METH-pretreated males responded in a habitual manner, indicating that increasing central dopamine can advance habit formation in male rats. Overall, these results suggest that METH pretreatment maintains goal-directed responding in female rats when they are typically transitioning to habitual control of instrumental behavior and can advance habit formation in male rats given sufficient instrumental training. In addition, we found differential RD sensitivity of the nose-poke response used during instrumental training compared to Pavlovian approach towards the food magazine, confirming that there is a distinction between these two behaviors and that nose-poking is a valid instrumental response.

Extended operant training increases infralimbic and prelimbic cortex Fos regardless of fear conditioning experience

Extended fear training can lead to initially low fear expression that grows over time, termed fear incubation. Conversely, a single fear conditioning session typically results in high fear initially that is sustained over time. Fear expression decreases across extended training, suggesting that a fear extinction-like process might be responsible for low fear observed soon after training. Because of the prominent role medial prefrontal cortex (mPFC) plays in fear conditioning and extinction, we decided to examine Fos expression resulting from a cued fear retrieval test to gain insight into possible mechanisms involved in extended training fear incubation. Male Long-Evans rats received 1 or 10 days of tone-shock pairings or tone-only exposure (while lever-pressing for food). Two days after the end of fear training, rats received a cued fear test, with perfusions timed to visualize Fos expression during test. As expected, the limited fear conditioning group exhibited higher fear in the test than any of the other groups (as measured with conditioned suppression of lever-pressing). Interestingly, we found that extended training animals (whether they received tone-shock pairings or tone-only exposure) expressed higher levels of Fos in both prelimbic and infralimbic cortices than limited training animals. There was no association between fear expression and mPFC Fos expression. These results suggest we may have visualized Fos expression related to operant overtraining rather than conditioned fear related processes. Further research is needed to determine the neurobiological basis of extended training fear incubation and to determine processes represented by the pattern of Fos expression we observed.

Hunger or thirst state uncertainty is resolved by outcome evaluation in medial prefrontal cortex to guide decision-making

Physiological need states direct decision-making toward re-establishing homeostasis. Using a two-alternative forced choice task for mice that models elements of human decisions, we found that varying hunger and thirst states caused need-inappropriate choices, such as food seeking when thirsty. These results show limits on interoceptive knowledge of hunger and thirst states to guide decision-making. Instead, need states were identified after food and water consumption by outcome evaluation, which depended on the medial prefrontal cortex.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Prolonged avoidance training exacerbates OCD-like behaviors in a rodent model

Obsessive-compulsive disorder (OCD) is characterized by compulsive behaviors that often resemble avoidance of perceived danger. OCD can be treated with exposure-with-response prevention (ERP) therapy in which patients are exposed to triggers but are encouraged to refrain from compulsions, to extinguish compulsive responses. The compulsions of OCD are strengthened by many repeated exposures to triggers, but little is known about the effects of extended repetition of avoidance behaviors on extinction. Here we assessed the extent to which overtraining of active avoidance affects subsequent extinction-with-response prevention (Ext-RP) as a rodent model of ERP, in which rats are extinguished to triggers, while the avoidance option is prevented. Male rats conditioned for 8d or 20d produced similar avoidance behavior to a tone paired with a shock, however, the 20d group showed a severe impairment of extinction during Ext-RP, as well as heightened anxiety. Furthermore, the majority of overtrained (20d) rats (75%) exhibited persistent avoidance following Ext-RP. In the 8d group, only a minority of rats (37%) exhibited persistent avoidance, and this was associated with elevated activity (c-Fos) in the prelimbic cortex and nucleus accumbens. In the 20d group, the minority of non-persistent rats (25%) showed elevated activity in the insular-orbital cortex and paraventricular thalamus. Lastly, extending the duration of Ext-RP prevented the deleterious effects of overtraining on extinction and avoidance. These rodent findings suggest that repeated expression of compulsion-like behaviors biases individuals toward persistent avoidance and alters avoidance circuits, thereby reducing the effectiveness of current extinction-based therapies.

Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal-directed action: A brief review

Goal-directed action refers to selecting behaviors based on the expectation that they will be reinforced with desirable outcomes. It is typically conceptualized as opposing habit-based behaviors, which are instead supported by stimulus-response associations and insensitive to consequences. The prelimbic prefrontal cortex (PL) is positioned along the medial wall of the rodent prefrontal cortex. It is indispensable for action-outcome-driven (goal-directed) behavior, consolidating action-outcome relationships and linking contextual information with instrumental behavior. In this brief review, we will discuss the growing list of molecular factors involved in PL function. Ventral to the PL is the medial orbitofrontal cortex (mOFC). We will also summarize emerging evidence from rodents (complementing existing literature describing humans) that it too is involved in action-outcome conditioning. We describe experiments using procedures that quantify responding based on reward value, the likelihood of reinforcement, or effort requirements, touching also on experiments assessing food consumption more generally. We synthesize these findings with the argument that the mOFC is essential to goal-directed action when outcome value information is not immediately observable and must be recalled and inferred.

Prelimbic and Infralimbic Prefrontal Regulation of Active and Inhibitory Avoidance and Reward-Seeking

Flexible initiation or suppression of actions to avoid aversive events is crucial for survival. The prelimbic (PL) and infralimbic (IL) regions of the medial prefrontal cortex (mPFC) have been implicated in different aspects of avoidance and reward-seeking, but their respective contribution in instigating versus suppressing actions in aversive contexts remains to be clarified. We examined mPFC involvement in different forms of avoidance in rats well trained on different cued lever-press avoidance tasks. Active/inhibitory avoidance required flexible discrimination between auditory cues signaling foot-shock could be avoided by making or withholding instrumental responses. On a simpler active avoidance task, a single cue signaled when a lever press would avoid shock. PL inactivation disrupted active but not inhibitory avoidance on the discriminative task while having no effect on single-cued avoidance. In comparison, IL inactivation broadly impaired active and inhibitory avoidance. Conversely, on a cued appetitive go/no-go task, both IL and PL inactivation impaired inhibitory but not active reward-seeking, the latter effect being diametrically opposite to that observed on the avoidance task. These findings highlight the complex manner in which different mPFC regions aid in initiating or inhibiting actions in the service of avoiding aversive outcomes or obtaining rewarding ones. IL facilitates active avoidance but suppress inappropriate actions in appetitive and aversive contexts. In contrast, contextual valence plays a critical role in how the PL is recruited in initiating or suppressing actions, which may relate to the degree of cognitive control required to flexibly negotiate response or motivational conflicts and override prepotent behaviors.SIGNIFICANCE STATEMENT Choosing to make or withhold actions in a context-appropriate manner to avoid aversive events or obtain other goals is a critical survival skill. Different medial prefrontal cortex (mPFC) regions have been implicated in certain aspects of avoidance, but their contributions to instigating or suppressing actions remains to be clarified. Here, we show that the dorsal, prelimbic (PL) region of the medial PFC aids active avoidance in situations requiring flexible mitigation of response conflicts, but also aids in withholding responses to obtain rewards. In comparison the ventral infralimbic (IL) cortex plays a broader role in active and inhibitory avoidance as well as suppressing actions to obtain rewards. These findings provide insight into mechanisms underlying normal and maladaptive avoidance behaviors and response inhibition.

Context and renewal of habits and goal-directed actions after extinction

Instrumental behaviors that are goal-directed actions after moderate amounts of training can become habits after more extended training. Little research has asked how actions and habits are affected by retroactive interference treatments like extinction. The present experiments begin to fill this gap in the literature. In Experiments 1a and 1b, lever pressing in rats was minimally trained (1a) or extensively trained (1b) in one context (Context A), extinguished in a second context (Context B), and then tested in the acquisition context (Context A). Exposure to both contexts was equated and controlled throughout, and the status of the behavior as action or habit was determined by reinforcer devaluation methods (taste aversion conditioning). Results confirmed that action (1a) and habit (1b) renewed with action or habit status, respectively, when they were returned to Context A. Experiments 2a and 2b then similarly tested action and habit after extinction in an ABC renewal paradigm. Here, lever pressing that was trained in Context A and extinguished in Context B renewed as action in Context C regardless of whether it had been an action or habit before extinction. The apparent conversion of habit to action during renewal testing in Context C was consistent with other results suggesting that habits converted to action when the context was changed at the start of extinction. Together, the results suggest that extinction in a second context inhibits instrumental behaviors trained as either actions or habits in a context-specific manner. They also expand on prior findings suggesting that actions transfer across contexts, and that habits do not. A change of context may be sufficient to convert a habit to goal-directed action. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Unexpected food outcomes can return a habit to goal-directed action

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Some factors that restore goal-direction to a habitual behavior

Recent findings from our laboratory suggest that an extensively-practiced instrumental behavior can appear to be a goal-directed action (rather than a habit) when a second behavior is added and reinforced during intermixed final sessions (Shipman et al., 2018). The present experiments were designed to explore and understand this finding. All used the taste aversion method of devaluing the reinforcer to distinguish between goal-directed actions and habits. Experiment 1 confirmed that reinforcing a second response in a separate context (but not mere exposure to that context) can return an extensively-trained habit to the status of goal-directed action. Experiment 2 showed that training of the second response needs to be intermixed with training of the first response to produce this effect; training the second response after the first-response training was complete preserved the first response as a habit. Experiment 3 demonstrated that reinforcing the second response with a different reinforcer breaks the habit status of the first response. Experiment 4 found that free reinforcers (that were not response-contingent) were sufficient to restore goal-directed performance. Together, the results suggest that unexpected reinforcer delivery can render a habitual response goal-directed again.

Chemogenetic Silencing of Prelimbic Cortex to Anterior Dorsomedial Striatum Projection Attenuates Operant Responding

Operant (instrumental) conditioning is a laboratory analog for voluntary behavior and involves learning to make a response for a reinforcing outcome. The prelimbic cortex (PL), a region of the rodent medial prefrontal cortex, and the dorsomedial striatum (DMS), have been separately established as important in the acquisition of minimally-trained operant behavior. Despite dense anatomical connections between the two regions, experimenters have only recently linked projections from the PL to the posterior DMS (pDMS) in the acquisition of an operant response. Yet, it is still unknown if these projections mediate behavioral expression, and if more anterior regions of the DMS (aDMS), which receive dense projections from the PL, are also involved. Therefore, we utilized designer receptors exclusively activated by designer drugs (DREADDs) to test whether or not projections from the PL to the aDMS influence the expression of operant behavior. Rats underwent bilateral PL-targeted infusions of either a DREADD virus (AAV8-hSyn-hM4D(Gi)-mCherry) or a control virus (AAV8-hSyn-GFP). In addition, guide cannulae were implanted bilaterally in the aDMS. Rats were tested with both clozapine-N-oxide (CNO) (DREADD ligand) and vehicle infusions into the aDMS. Animals that had received the DREADD virus, but not the control virus, showed attenuated responding when they received CNO microinfusions into the aDMS, compared to vehicle infusions. Patch clamp electrophysiology verified the inhibitory effect of CNO on DREADDs-expressing PL neurons in acute brain slices. GFP-expressing control PL neurons were unaffected by CNO. The results add to the recent literature suggesting that connections between the PL and aDMS are important for the expression of minimally-trained operant responding.

Goal-directed control on interval schedules does not depend on the action-outcome correlation

When an organism's action is based on an anticipation of its consequences, that action is said to be goal-directed. It has long been thought that goal-directed control is made possible by experiencing a strong correlation between response rates and reward rates (Dickinson, 1985). To test this idea, we designed a set of experiments to determine whether the response rate-reward rate correlation is a reliable predictor of goal-directed control on interval schedules. In Experiment 1, rats were trained on random interval (RI) schedules in which the response rate-reward rate correlation was manipulated across groups. In tests of reward devaluation, rats behaved in a goal-directed manner regardless of the experienced correlation. In Experiment 2, rats once again experienced either a strong or weak correlation, but on RI schedules with lower overall reward densities. This time, behavior appeared habitual regardless of the experienced correlation. Experiment 3 confirmed that the density of the RI schedule influences goal-directed control, and also revealed that extensive training on these schedules resulted in goal-directed action. Finally, in Experiment 4 goal-directed responding was greater and emerged sooner on fixed than random interval schedules, but, again, was manifest after extensive training on the RI schedule. Taken together, our data suggest that goal-directed and habitual control are not determined by the correlation between response rates and reward rates. We discuss the importance of temporal uncertainty, action-outcome contiguity, and reinforcement probability in goal-directed control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems

This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.

Responsivity of lateral septum-mPFC connections in alloxan-induced hyperglycemia

The lateral septal nucleus (LSN) is related to the actions of antidepressants, and the prelimbic cortex (PL) and infralimbic cortex (IL) modulate responses to fear. However, unknown is whether experimental diabetes that is induced by alloxan alters the responsivity of these regions. We used a method in which one forebrain region (LSN) was electrically stimulated while single-unit extracellular recordings were performed in another mPFC region (PL and IL). Several experimental groups were tested: (a) animals that were subjected to long-term (42-day) alloxan-hyperglycemia and protected with insulin, (b) healthy animals that received a low dose of insulin that does not produce changes in glycemia, and (c) animals that received long-term treatment with fluoxetine. Additional healthy groups of animals received insulin or fluoxetine and underwent the forced swim test. Biological measurements indicated the induction of diabetes in alloxan-treated rats. In this group, a shift toward an inhibitory response to LSN stimulation was observed in the PL and IL compared with the control group. A low dose of insulin or fluoxetine produced similar changes in LSN-PL and LSN-IL responsivity. Long-term hyperglycemia increased inhibitory responsivity in the LSN-PL and LSN-IL, but this action was less pronounced than the action that was exerted by insulin and fluoxetine, which produced similar actions. Such similar actions were confirmed in the forced swim test, in which the antidepressant-like effects of insulin partially resembled the effects of fluoxetine. The changes that were observed in the alloxan group appeared to be related to neuronal damage, and a low dose of insulin exerted some antidepressant-like actions.

Reduced sensitivity to devaluation for instrumental but not consummatory behaviors in binge eating prone rats

Binge eating is characterized by the consumption of a large amount of palatable food in a short period of time and is a core feature of many eating disorders. Patients with eating disorders are also known to display impairments in inhibitory control, cognition and decision-making, which may promote and maintain binge eating symptomology. In the current study, we examined whether rats that were subsequently characterized as displaying a higher propensity to binge eat would show pre-existing deficits in reinforcer devaluation-a paradigm used to examine decision-making following reductions in the value of a food reinforcer. Female rats were first trained to respond on two levers for the delivery of two food reinforcers (sucrose and maltodextrin solutions). At the test stage, rats were provided 1 h access to one of the two reinforcers to allow for devaluation via sensory specific satiety, immediately followed by an extinction test with both levers. Normal rats typically show reductions in responding on the lever associated with the devalued reinforcer (i.e., intact goal-directed responding). Subsequently, we used intermittent access to palatable food to identify high (BE prone [BEP]; n = 14), intermediate (BE neutral [BEN]; n = 48), and low (BE resistant [BER]; n = 13) phenotypes of binge eating. Prior reinforcer devaluation performance showed BEN and BER rats suppressed responding on the lever associated with the devalued reinforcer while BEP rats did not. This insensitivity to instrumental reinforcer devaluation in BEP rats did not reflect impaired sensory-specific satiety as during a food choice test, BEP rats showed a more robust alteration in food preferences following devaluation. Additionally, across all rats sensory specific satiety was correlated with subsequent intake of palatable food. Collectively, these findings suggest dissociable effects of devaluation procedures on instrumental actions and consummatory behaviors in BEP rats, and may indicate that pre-existing differences in goal-directed behavior and sensory-specific satiety contribute to the propensity to overeat palatable food.

Cerebellum and cognition: Does the rodent cerebellum participate in cognitive functions?

There is a widespread, nearly complete consensus that the human and non-human primate cerebellum is engaged in non-motor, cognitive functions. This body of research has implicated the lateral portions of lobule VII (Crus I and Crus II) and the ventrolateral dentate nucleus. With rodents, however, it is not so clear. We review here approximately 40 years of experiments using a variety of cerebellar manipulations in rats and mice and measuring the effects on executive functions (working memory, inhibition, and cognitive flexibility), spatial navigation, discrimination learning, and goal-directed and stimulus-driven instrumental conditioning. Our conclusion is that there is a solid body of support for engagement of the rodent cerebellum in tests of cognitive flexibility and spatial navigation, and some support for engagement in working memory and certain types of discrimination learning. Future directions will involve determining the relevant cellular mechanisms, cerebellar regions, and precise cognitive functions of the rodent cerebellum.