Selective Modulation of Orbitofrontal Network Activity during Negative Occasion Setting

Disruption of positive- and negative-feature morphine interoceptive occasion setters by dopamine receptor agonism and antagonism in male and female rats

RATIONALE

Internally perceived stimuli evoked by morphine administration can form Pavlovian associations such that they can function as occasion setters (OSs) for externally perceived reward cues in rats, coming to modulate reward-seeking behaviour. Though much research has investigated mechanisms underlying opioid-related reinforcement and analgesia, neurotransmitter systems involved in the functioning of opioids as Pavlovian interoceptive discriminative stimuli remain to be disentangled despite documented differences in the development of tolerance to analgesic versus discriminative stimulus effects.

OBJECTIVES

Dopamine has been implicated in many opioid-related behaviours, so we aimed to investigate the role of this neurotransmitter in expression of morphine occasion setting.

METHODS

Male and female rats were assigned to positive- (FP) or negative-feature (FN) groups and received an injection of morphine or saline before each training session. A 15-s white noise conditioned stimulus (CS) was presented 8 times during every training session; offset of this stimulus was followed by 4-s access to liquid sucrose on morphine, but not saline, sessions for FP rats. FN rats learned the reverse contingency. Following stable discrimination, rats began generalization testing for expression of morphine-guided sucrose seeking after systemic pretreatment with different doses of the non-selective dopamine receptor antagonist, flupenthixol, and the non-selective dopamine receptor agonist, apomorphine, combined with training doses of morphine or saline in a Latin-square design.

RESULTS

The morphine discrimination was acquired under both FP and FN contingencies by males and females. Neither flupenthixol nor apomorphine at any dose substituted for morphine, but both apomorphine and flupenthixol disrupted expression of the morphine OS. This inhibition was specific to sucrose seeking during CS presentations rather than during the period before CS onset and, in the case of apomorphine more so than flupenthixol, to trials on which access to sucrose was anticipated.

CONCLUSIONS

Our findings lend support to a mechanism of occasion setting involving gating of CS-induced dopamine release rather than by direct dopaminergic modulation by the morphine stimulus.

Sex differences in discrimination behavior and orbitofrontal engagement during context-gated reward prediction

Animals, including humans, rely on contextual information to interpret ambiguous stimuli. Impaired context processing is a hallmark of several neuropsychiatric disorders, including schizophrenia, autism spectrum disorders, post-traumatic stress disorder, and addiction. While sex differences in the prevalence and manifestations of these disorders are well established, potential sex differences in context processing remain uncertain. Here, we examined sex differences in the contextual control over cue-evoked reward seeking and its neural correlates, in rats. Male and female rats were trained in a bidirectional occasion-setting preparation in which the validity of two auditory reward-predictive cues was informed by the presence, or absence, of a visual contextual feature (LIGHT: X+/DARK: X-/LIGHT: Y-/DARK: Y+). Females were significantly slower to acquire contextual control over cue-evoked reward seeking. However, once established, the contextual control over behavior was more robust in female rats; it showed less within-session variability (less influence of prior reward) and greater resistance to acute stress. This superior contextual control achieved by females was accompanied by an increased activation of the orbitofrontal cortex (OFC) compared to males. Critically, these behavioral and neural sex differences were specific to the contextual modulation process and not observed in simple, context-independent, reward prediction tasks. These results indicate a sex-biased trade-off between the speed of acquisition and the robustness of performance in the contextual modulation of cued reward seeking. The different distribution of sexes along the fast learning ↔ steady performance continuum might reflect different levels of engagement of the OFC, and might have implications for our understanding of sex differences in psychiatric disorders.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Ambiguity drives higher-order Pavlovian learning

In the natural world, stimulus-outcome associations are often ambiguous, and most associations are highly complex and situation-dependent. Learning to disambiguate these complex associations to identify which specific outcomes will occur in which situations is critical for survival. Pavlovian occasion setters are stimuli that determine whether other stimuli will result in a specific outcome. Occasion setting is a well-established phenomenon, but very little investigation has been conducted on how occasion setters are disambiguated when they themselves are ambiguous (i.e., when they do not consistently signal whether another stimulus will be reinforced). In two preregistered studies, we investigated the role of higher-order Pavlovian occasion setting in humans. We developed and tested the first computational model predicting direct associative learning, traditional occasion setting (i.e., 1st-order occasion setting), and 2nd-order occasion setting. This model operationalizes stimulus ambiguity as a mechanism to engage in higher-order Pavlovian learning. Both behavioral and computational modeling results suggest that 2nd-order occasion setting was learned, as evidenced by lack and presence of transfer of occasion setting properties when expected and the superior fit of our 2nd-order occasion setting model compared to the 1st-order occasion setting or direct associations models. These results provide a controlled investigation into highly complex associative learning and may ultimately lead to improvements in the treatment of Pavlovian-based mental health disorders (e.g., anxiety disorders, substance use).

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the “brake” on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory “drive.” For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the “drive” and “brake” circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

Roles of the prefrontal cortex in learning to time the onset of pre-existing motor programs

The prefrontal cortex (PFC) is involved in cognitive control of motor activities and timing of future intensions. This study investigated the cognitive control of balance recovery in response to unpredictable gait perturbations and the role of PFC subregions in learning by repetition. Bilateral dorsolateral (DLPFC), ventrolateral (VLPFC), frontopolar (FPFC) and orbitofrontal (OFC) cortex hemodynamic changes induced by unpredictable slips were analyzed as a function of successive trials in ten healthy young adults. Slips were induced by the acceleration of one belt as the participant walked on a split-belt treadmill. A portable functional near-infrared spectroscope monitored PFC activities quantified by oxyhemoglobin (ΔO₂Hb) and deoxyhemoglobin (ΔHbR) during the consecutive trial phases: standing, walking, slip-recovery. During the first 3 trials, the average oxyhemoglobin (ΔO₂Hb_avg) in the DLPFC, VLPFC, FPFC, and OFC cortex was significantly higher during slip-recovery than unperturbed walking or the standing baseline. Then, ΔO₂Hb_avg decreased progressively from trial-to-trial in the DLPFC, VLPFC, and FPFC, but increased and then remained constant in the OFC. The average deoxyhemoglobin (ΔHbR_avg) presented mirror patterns. These changes after the third trial were paralleled by the progressive improvement of recovery revealed by kinematic variables. The results corroborate our previous hypothesis that only timing of the onset of a “good enough recovery motor program” is learned with practice. They also strongly support the assumption that the PFC contributes to the recall of pre-existing motor programs whose onset timing is adjusted by the OFC. Hence, learning is clearly divided into two steps delineated by the switch in activity of the OFC. Additionally, motor processes appear to share the working memory as well as decisional and predictive resources of the cognitive system.

Representation of foreseeable choice outcomes in orbitofrontal cortex triplet-wise interactions

Shared neuronal variability has been shown to modulate cognitive processing. However, the relationship between shared variability and behavioral performance is heterogeneous and complex in frontal areas such as the orbitofrontal cortex (OFC). Mounting evidence shows that single-units in OFC encode a detailed cognitive map of task-space events, but the existence of a robust neuronal ensemble coding for the predictability of choice outcome is less established. Here, we hypothesize that the coding of foreseeable outcomes is potentially unclear from the analysis of units activity and their pairwise correlations. However, this code might be established more conclusively when higher-order neuronal interactions are mapped to the choice outcome. As a case study, we investigated the trial-to-trial shared variability of neuronal ensemble activity during a two-choice interval-discrimination task in rodent OFC, specifically designed such that a lose-switch strategy is optimal by repeating the rewarded stimulus in the upcoming trial. Results show that correlations among triplets are higher during correct choices with respect to incorrect ones, and that this is sustained during the entire trial. This effect is not observed for pairwise nor for higher than third-order correlations. This scenario is compatible with constellations of up to three interacting units assembled during trials in which the task is performed correctly. More interestingly, a state-space spanned by such constellations shows that only correct outcome states that can be successfully predicted are robust over 100 trials of the task, and thus they can be accurately decoded. However, both incorrect and unpredictable outcome representations were unstable and thus non-decodeable, due to spurious negative correlations. Our results suggest that predictability of successful outcomes, and hence the optimal behavioral strategy, can be mapped out in OFC ensemble states reliable over trials of the task, and revealed by sufficiency complex neuronal interactions.

Neuronal responses can differ substantially during repetitions of the same tasks; however, they are often coordinated (shared) across multiple neighboring neurons. Such correlation between neurons has been related to the capacity of the brain to take decisions, but specifically how this relation is established is still under study. In this work, we address this question by focusing on an intriguing case study, the orbitofrontal cortex, since this brain area has been found in various studies to be useful for decision-making. Here, we question whether orchestrated groups of neurons encode sufficient information for optimizing their decision strategy; that is, whether the outcome of a choice can be predicted or not on the basis of previous experience. We thus designed a decision-making task for a rat in which some of the correct choices can be predicted. We found that only successful outcomes that can actually be predicted were robustly encoded over time. This finding was shown by analyzing sufficiently complex interactions between three neurons, whilst more complex orchestrations did not add further insights. Thus, we propose that coordinated responses of up to three neurons in the OFC could contribute to the capacity of the animal to take the optimal decision.

Occasion setters determine responses of putative DA neurons to discriminative stimuli

Midbrain dopamine (DA) neurons are involved in the processing of rewards and reward-predicting stimuli, possibly analogous to reinforcement learning reward prediction errors. Here we studied the activity of putative DA neurons (n = 37) recorded in the ventral tegmental area of rats (n = 6) performing a behavioural task involving occasion setting. In this task an occasion setter (OS) indicated that the relationship between a discriminative stimulus (DS) and reinforcement is in effect, so that reinforcement of bar pressing occurred only after the OS (tone or houselight) was followed by the DS (houselight or tone). We found that responses of putative DA cells to the DS were enhanced when preceded by the OS, as were behavioural responses to obtain rewards. Surprisingly though, we did not find a homogeneous increase in the mean activity of the population of putative DA neurons to the OS, contrary to predictions of standard temporal-difference models of DA neurons. Instead, putative DA neurons exhibited a heterogeneous response on a single unit level, so that some units increased and others decreased their activity as a response to the OS. Similarly, putative non-DA cells did not show a homogeneous response to the DS on a population level, but also had heterogeneous responses on a single unit level. The heterogeneity in the responses of neurons in the ventral tegmental area may reflect how DA neurons encode context and point to local differences in DA signalling.

Open source silicon microprobes for high throughput neural recording

OBJECTIVE

Microfabricated multielectrode arrays are widely used for high throughput recording of extracellular neural activity, which is transforming our understanding of brain function in health and disease. Currently there is a plethora of electrode-based tools being developed at higher education and research institutions. However, taking such tools from the initial research and development phase to widespread adoption by the neuroscience community is often hindered by several obstacles. The objective of this work is to describe the development, application, and open dissemination of silicon microprobes for recording neural activity in vivo.

APPROACH

We propose an open source dissemination platform as an alternative to commercialization. This framework promotes recording tools that are openly and inexpensively available to the community. The silicon microprobes are designed in house, but the fabrication and assembly processes are carried out by third party companies. This enables mass production, a key requirement for large-scale dissemination.

MAIN RESULTS

We demonstrate the operation of silicon microprobes containing up to 256 electrodes in conjunction with optical fibers for optogenetic manipulations or fiber photometry. These data provide new insights about the relationship between calcium activity and neural spiking activity. We also describe the current state of dissemination of these tools. A file repository of resources related to designing, using, and sharing these tools is maintained online.

SIGNIFICANCE

This paper is likely to be a valuable resource for both current and prospective users, as well as developers of silicon microprobes. Based on their extensive usage by a number of labs including ours, these tools present a promising alternative to other types of electrode-based technologies aimed at high throughput recording in head-fixed animals. This work also demonstrates the importance of validating fiber photometry measurements with simultaneous electrophysiological recordings.

Occasion setting

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event—an occasion setter (OS)—which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

d-Cycloserine facilitates fear extinction in adolescent rats and differentially affects medial and lateral prefrontal cortex activation

Adolescent humans and rodents are impaired in extinguishing learned fear relative to younger and older groups. This impairment could be due to differences in recruitment of medial prefrontal cortex (PFC), orbitofrontal cortex (OFC), or amygdala during extinction. For example, unlike juveniles and adults, adolescent rats do not express extinction-induced increases in phosphorylated mitogen activated protein kinase (pMAPK), a marker of synaptic plasticity, in the medial PFC. The NMDA receptor partial agonist d-cycloserine (DCS) improves extinction retention in adolescent rats. We investigated whether DCS affected recruitment of the PFC and amygdala during extinction by measuring pMAPK-immunoreactive (IR) neurons. Adolescent rats were trained to fear a conditioned stimulus in one context followed by extinction in a second context or equivalent context exposure only (i.e., no extinction). DCS (15 mg/kg, s.c.) or saline was administered systemically immediately after extinction training or context exposure. DCS enhanced extinction learning and this was associated with increased activation of the MAPK signaling pathway in the OFC after extinction training and increased activation in the medial PFC and amygdala at extinction retention. These findings suggest that DCS improves extinction learning in adolescents because it augments OFC contributions to extinction learning, enabling better medial prefrontal contributions to extinction retention.