Pre-surgical training ameliorates orbitofrontal-mediated impairments in spatial reversal learning

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Medial orbitofrontal cortical regulation of different aspects of Pavlovian and instrumental reward seeking

RATIONALE

The medial subregion of the orbitofrontal cortex (mOFC) is thought to play an important role representing the expected outcome of a given course of action, as lesioning or inactivating this cortical region results in the adoption of choice strategies based more on observable (rather than previously learned) information. Despite this, its role in mediating basic associative learning remains to be fully clarified.

OBJECTIVE

The present series of experiments examined the role of the mOFC in (1) Pavlovian conditioned approach, (2) conditioned reinforcement, (3) extinction, and (4) cue-induced reinstatement of food-seeking behavior.

METHODS

Separate cohorts of rats went through Pavlovian or instrumental training. Intra-mOFC infusions of either saline or GABA agonists (to temporarily inactivate neural activity) were given prior to Pavlovian approach, conditioned reinforcement, first or second day of instrumental extinction training, or cue-induced reinstatement test days.

RESULTS

mOFC inactivation increased lever-CS contacts in Pavlovian conditioned approach and (2) had no effect on conditioned reinforcement. These manipulations (3) accelerated within-session instrumental extinction during the initial extinction session, but impaired subsequent extinction learning on drug-free days. (4) mOFC inactivation induced differential effects on reinstatement that depended on baseline performance. mOFC inactivation abolished reinstatement in "Reinstater" rats (who displayed robust responding under control conditions) and robustly increased reinstatement in "Non-Reinstater" rats (who showed little reinstatement under control conditions) suggesting that individual differences in reinstatement may be supported by differences in mOFC mediated representations of expected outcomes.

CONCLUSIONS

These findings have important implications for understanding how the mOFC uses stimulus-outcome and action-outcome expectancies to guide behavior, and how dysfunction within this region may contribute to pathological patterns of reward seeking.

The Role of the Rodent Lateral Orbitofrontal Cortex in Simple Pavlovian Cue-Outcome Learning Depends on Training Experience

The orbitofrontal cortex (OFC) is a critical structure in the flexible control of value-based behaviors. OFC dysfunction is typically only detected when task or environmental contingencies change, against a backdrop of apparently intact initial acquisition and behavior. While intact acquisition following OFC lesions in simple Pavlovian cue-outcome conditioning is often predicted by models of OFC function, this predicted null effect has not been thoroughly investigated. Here, we test the effects of lesions and temporary muscimol inactivation of the rodent lateral OFC on the acquisition of a simple single cue-outcome relationship. Surprisingly, pretraining lesions significantly enhanced acquisition after overtraining, whereas post-training lesions and inactivation significantly impaired acquisition. This impaired acquisition to the cue reflects a disruption of behavioral control and not learning since the cue could also act as an effective blocking stimulus in an associative blocking procedure. These findings suggest that even simple cue-outcome representations acquired in the absence of OFC function are impoverished. Therefore, while OFC function is often associated with flexible behavioral control in complex environments, it is also involved in very simple Pavlovian acquisition where complex cue-outcome relationships are irrelevant to task performance.

Functional heterogeneity within the rodent lateral orbitofrontal cortex dissociates outcome devaluation and reversal learning deficits

The orbitofrontal cortex (OFC) is critical for updating reward-directed behaviours flexibly when outcomes are devalued or when task contingencies are reversed. Failure to update behaviour in outcome devaluation and reversal learning procedures are considered canonical deficits following OFC lesions in non-human primates and rodents. We examined the generality of these findings in rodents using lesions of the rodent lateral OFC (LO) in instrumental action-outcome and Pavlovian cue-outcome devaluation procedures. LO lesions disrupted outcome devaluation in Pavlovian but not instrumental procedures. Furthermore, although both anterior and posterior LO lesions disrupted Pavlovian outcome devaluation, only posterior LO lesions were found to disrupt reversal learning. Posterior but not anterior LO lesions were also found to disrupt the attribution of motivational value to Pavlovian cues in sign-tracking. These novel dissociable task- and subregion-specific effects suggest a way to reconcile contradictory findings between rodent and non-human primate OFC research.

Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning

Different subregions of the prefrontal cortex (PFC) contribute to the ability to respond flexibly to changes in reward contingencies, with the medial versus orbitofrontal cortex (OFC) subregions contributing differentially to processes such as set-shifting and reversal learning. To date, the manner in which these regions may facilitate reversal learning in situations involving reward uncertainty remains relatively unexplored. We investigated the involvement of five distinct regions of the rat OFC (lateral and medial) and medial PFC (prelimbic, infralimbic, and anterior cingulate) on probabilistic reversal learning wherein "correct" versus "incorrect" responses were rewarded on 80% and 20% of trials, respectively. Contingencies were reversed repeatedly within a session. In well trained rats, inactivation of the medial or lateral OFC induced dissociable impairments in performance (indexed by fewer reversals completed) when outcomes were probabilistic, but not when they were assured. Medial OFC inactivation impaired probabilistic learning during the first discrimination, increased perseverative responding and reduced sensitivity to positive and negative feedback, suggestive of a deficit in incorporating information about previous action outcomes to guide subsequent behavior. Lateral OFC inactivation preferentially impaired performance during reversal phases. In contrast, prelimbic inactivation caused an apparent improvement in performance by increasing the number of reversals completed. This was associated with enhanced sensitivity to recently rewarded actions and reduced sensitivity to negative feedback. Infralimbic inactivation had no effect, whereas the anterior cingulate appeared to play a permissive role in this form of reversal learning. These results clarify the dissociable contributions of different regions of the frontal lobes to probabilistic learning.

SIGNIFICANCE STATEMENT

The ability to adjust behavior in response to changes involving uncertain or probabilistic reward contingencies is an essential survival skill that is impaired in a variety of psychiatric disorders. It is well established that different forms of cognitive flexibility are mediated by anatomically distinct regions of the frontal lobes when reinforcement contingencies are assured, however, less is known about the contribution of these regions to probabilistic reinforcement learning. Here we show that different regions of the orbitofrontal and medial prefrontal cortex make distinct contributions to probabilistic reversal learning. These findings provide novel information about the complex interplay between frontal lobe regions in mediating these processes and accordingly provide insight into possible pathophysiology that underlies impairments in cognitive flexibility observed in mental illnesses.

The role of 5-HT2C receptors in touchscreen visual reversal learning in the rat: a cross-site study

RATIONALE

Reversal learning requires associative learning and executive functioning to suppress non-adaptive responding. Reversal-learning deficits are observed in e.g. schizophrenia and obsessive-compulsive disorder and implicate neural circuitry including the orbitofrontal cortex (OFC). Serotonergic function has been strongly linked to visual reversal learning in humans and experimental animals but less is known about which receptor subtypes are involved.

OBJECTIVES

The objectives of the study were to test the effects of systemic and intra-OFC 5-HT2C-receptor antagonism on visual reversal learning in rats and assess the psychological mechanisms underlying these effects within novel touchscreen paradigms.

METHODS

In experiments 1-2, we used a novel 3-stimulus task to investigate the effects of 5-HT2C-receptor antagonism through SB 242084 (0.1, 0.5 and 1.0 mg/kg i.p.) cross-site. Experiment 3 assessed the effects of SB 242084 in 2-choice reversal learning. In experiment 4, we validated a novel touchscreen serial visual reversal task suitable for neuropharmacological microinfusions by baclofen-/muscimol-induced OFC inactivation. In experiment 5, we tested the effect of intra-OFC SB 242084 (1.0 or 3.0 μg/side) on performance in this task.

RESULTS

In experiments 1-3, SB 242084 reduced early errors but increased late errors to criterion. In experiment 5, intra-OFC SB 242084 reduced early errors without increasing late errors in a reversal paradigm validated as OFC dependent (experiment 4).

CONCLUSION

Intra-OFC 5-HT2C-receptor antagonism decreases perseveration in novel touchscreen reversal-learning paradigms for the rat. Systemic 5-HT2C-receptor antagonism additionally impairs late learning-a robust effect observed cross-site and potentially linked to impulsivity. These conclusions are discussed in terms of neural mechanisms underlying reversal learning and their relevance to psychiatric disorders.

Executive function

Components of human executive function, like rule generation and selection in response to stimuli (attention set-shifting) or overcoming a habit (reversal learning), can be reliably modelled in rodents. The rodent paradigms are based upon tasks that assess cognitive flexibility in clinical populations and have been effective in distinguishing the neurobiological substrates and the underlying neurotransmitter systems relevant to executive function. A review of the literature on the attentional set-shifting task highlights a prominent role for the medial region of the prefrontal cortex in the ability to adapt to a new rule (extradimensional shift) while the orbitofrontal cortex has been associated with the reversal learning component of the task. In other paradigms specifically developed to examine reversal learning in rodents, the orbitofrontal cortex also plays a prominent role. Modulation of dopamine, serotonin, and glutamatergic receptors can disrupt executive function, a feature commonly exploited to develop concepts underlying psychiatric disorders. While these paradigms do have excellent translational construct validity, they have been less effective as predictive preclinical models for cognitive enhancers, especially for cognition in health subjects. Accordingly, a more diverse battery of tasks may be necessary to model normal human executive function in the rodent for drug development.

Behavioral flexibility in rats and mice: contributions of distinct frontocortical regions

Research examining the contribution of genetics to behavior is increasingly focused on higher order behavioral and cognitive processes including the ability to modify behaviors when environmental demands change. The frontal cortices of mammals, including rodents, subserve a diverse set of behavioral and cognitive functions including motor planning, social behavior, evaluation of expected outcomes and working memory, which may be particularly sensitive to genetic factors and interactions with experience (e.g. stress). Behavioral flexibility is a core attribute of these functions. This review orients readers to the current landscape of the literature on the frontocortical bases of behavioral flexibility in rodent laboratory experiments. Studies are divided into three broad categories: reversal learning, inhibitory learning and set-shifting. Functional dissociations within the broader scope of behavioral flexibility are reviewed, followed by discussion of the associations between specific components of frontal cortex and specific aspects of relevant behavioral processes. Finally, the authors identify open questions that need to be addressed to better establish the constituents of frontal cortex underlying behavioral flexibility.

Perseveration in the presence of punishment: the effects of chronic cocaine exposure and lesions to the prefrontal cortex

Perseveration is the repetition of a previously appropriate response in a manner, or context, which is detrimental to the individual. Although both cocaine exposure and prefrontal cortex (PFC) dysfunctions have been implicated in perseverative-like behaviours, the underlying nature of the impairments has been debated. The current study tested whether chronic cocaine exposure and PFC lesions induce perseverative-like behaviours by causing insensitivity to punishment. Food-restricted male Sprague-Dawley rats were trained to respond for sucrose on concurrent schedules of reinforcement. After initial training, rats received either a sensitizing regimen of cocaine exposure, or excitotoxic lesions to subregions of the PFC. The test of perseveration involved a choice of responding between two levers associated with fixed ratio and progressive ratio (PR) schedules. Responding on the PR lever was punished by a 1 min timeout period. It was found that, unlike control subjects, those exposed to chronic cocaine, or with lesions to the medial prefrontal cortex, were significantly slower in adapting their responding to avoid punishment. The current study provides evidence that both cocaine exposure and lesions to the prefrontal cortex can increase perseverative-like responding, although the magnitude and permanence of these effects are contingent on the nature of the task.

Orbitofrontal cortex inactivation impairs between- but not within-session Pavlovian extinction: an associative analysis

The orbitofrontal cortex (OFC) is argued to be the neural locus of Pavlovian outcome expectancies. Reinforcement learning theories argue that extinction learning in Pavlovian procedures is caused by the discrepancy between the expected value of the outcome (US) that is elicited by a predictive stimulus (CS), and the lack of experienced US. If the OFC represents Pavlovian outcome expectancies that are necessary for extinction learning, then disrupting OFC function prior to extinction training should impair extinction learning. This was tested. In experiment 1, Long Evans rats received infusions of saline or muscimol targeting the lateral OFC prior to three appetitive Pavlovian extinction sessions. Muscimol infused into the OFC disrupted between-session but not within-session extinction behaviour. This finding was not due to muscimol infusions disrupting the memory consolidation process per se as there was no effect of muscimol infusion when administered immediately post session (experiment 2). These findings support a role for the OFC in representing outcome expectancies that are necessary for learning. A number of ways in which disrupting outcome expectancy information might block learning will be discussed in the context of traditional associative learning theories and the associative structures they depend on.

Consideration of species differences in developing novel molecules as cognition enhancers

The NIH-funded CNTRICS initiative has coordinated efforts to promote the vertical translation of novel procognitive molecules from testing in mice, rats and non-human primates, to clinical efficacy in patients with schizophrenia. CNTRICS highlighted improving construct validation of tasks across species to increase the likelihood that the translation of a candidate molecule to humans will be successful. Other aspects of cross-species behaviors remain important however. This review describes cognitive tasks utilized across species, providing examples of differences and similarities of innate behavior between species, as well as convergent construct and predictive validity. Tests of attention, olfactory discrimination, reversal learning, and paired associate learning are discussed. Moreover, information on the practical implication of species differences in drug development research is also provided. The issues covered here will aid in task development and utilization across species as well as reinforcing the positive role preclinical research can have in developing procognitive treatments for psychiatric disorders.

Dysregulation of D₂-mediated dopamine transmission in monkeys after chronic escalating methamphetamine exposure

Compulsive drug-seeking and drug-taking are important substance-abuse behaviors that have been linked to alterations in dopaminergic neurotransmission and to impaired inhibitory control. Evidence supports the notions that abnormal D₂ receptor-mediated dopamine transmission and inhibitory control may be heritable risk factors for addictions, and that they also reflect drug-induced neuroadaptations. To provide a mechanistic explanation for the drug-induced emergence of inhibitory-control deficits, this study examined how a chronic, escalating-dose regimen of methamphetamine administration affected dopaminergic neurochemistry and cognition in monkeys. Dopamine D₂-like receptor and dopamine transporter (DAT) availability and reversal-learning performance were measured before and after exposure to methamphetamine (or saline), and brain dopamine levels were assayed at the conclusion of the study. Exposure to methamphetamine reduced dopamine D₂-like receptor and DAT availability and produced transient, selective impairments in the reversal of a stimulus-outcome association. Furthermore, individual differences in the change in D₂-like receptor availability in the striatum were related to the change in response to positive feedback. These data provide evidence that chronic, escalating-dose methamphetamine administration alters the dopamine system in a manner similar to that observed in methamphetamine-dependent humans. They also implicate alterations in positive-feedback sensitivity associated with D₂-like receptor dysfunction as the mechanism by which inhibitory control deficits emerge in stimulant-dependent individuals. Finally, a significant degree of neurochemical and behavioral variation in response to methamphetamine was detected, indicating that individual differences affect the degree to which drugs of abuse alter these processes. Identification of these factors ultimately may assist in the development of individualized treatments for substance dependence.

Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats

The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) are important neural regions in responding adaptively to changes in the incentive value of reward. Recent evidence suggests these structures may be differentially engaged in effort and cue-guided choice behavior. In 2 T-maze experiments, we examined the effects of bilateral lesions of either BLA or OFC on (1) effortful choices in which rats could climb a barrier for a high reward or select a low reward with no effort and (2) effortful choices when a visual cue signaled changes in reward magnitude. In both experiments, BLA rats displayed transient work aversion, choosing the effortless low reward option. OFC rats were work averse only in the no cue conditions, displaying a pattern of attenuated recovery from the cue conditions signaling reward unavailability in the effortful arm. Control measures rule out an inability to discriminate the cue in either lesion group.

Excitotoxic lesions to the prefrontal cortex of Sprague-Dawley rats do not impair response matching

Perseveration refers to maladaptive persistence of behavior outside appropriate contexts and despite negative outcomes. In humans, perseveration is a symptom of a variety of psychiatric disorders. In rats, perseveration has been observed in reversal learning tasks following lesions of the prefrontal cortex (PFC). However, the exact nature of the impairment underlying this effect remains unclear. Male Sprague-Dawley rats were trained on a novel reversal task that requires switching between two rewarded options varying in effort (concurrent fixed and progressive ratios) necessary to obtain the reward. Following initial training, bilateral lesions of the dorsal PFC, medial PFC, or orbitofrontal cortex were produced by NMDA infusions. When animals were re-tested post-surgery, no significant impairments were found. These results indicate that, in trained rats, the PFC is not necessary for selecting responses on the basis of favorable effort-to-reward contingencies.

Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task

Risky decision making on the Iowa Gambling Task (IGT) has been observed in several psychiatric disorders, including substance abuse, schizophrenia, and pathological gambling. Such deficits are often attributed to impaired processing within the orbitofrontal cortex (OFC) because patients with damage to this area or to the amygdala, which is strongly interconnected with the OFC, can likewise show enhanced choice of high-risk options. However, whether damage to the OFC or amygdala impairs subjects' ability to learn the task, or actually affects the decision-making process itself, is currently unclear. To address these issues, rats were trained to perform a rodent gambling task (rGT) either before or after bilateral excitotoxic lesions to the basolateral amygdala (BLA) or OFC. Maximum profits in both the rGT and IGT are obtained by favoring smaller rewards associated with lower penalties, and avoiding the tempting, yet ultimately disadvantageous, large reward options. Lesions of the OFC or BLA made before task acquisition initially impaired animals' ability to determine the optimal strategy, but did not disrupt decision making in the long term. In contrast, lesions of the BLA, but not the OFC, made after the task had been acquired increased risky choice. These results suggest that, although both regions contribute to the development of appropriate choice behavior under risk, the BLA maintains a more fundamental role in guiding these decisions. The maladaptive choice pattern observed on the IGT in patients with OFC lesions could therefore partially reflect a learning deficit, whereas amygdala damage may give rise to a more robust decision-making impairment.

Human reversal learning under conditions of certain versus uncertain outcomes

Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.