Occasion setting

Generalization of a positive-feature interoceptive morphine occasion setter across the rat estrous cycle

INTRODUCTION

Interoceptive stimuli elicited by drug administration acquire conditioned modulatory properties of the induction of conditioned appetitive behaviours by exteroceptive cues. This effect may be modeled using a drug discrimination task in which the drug stimulus is trained as a positive-feature (FP) occasion setter (OS) that disambiguates the relation between an exteroceptive light conditioned stimulus (CS) and a sucrose unconditioned stimulus (US). We previously reported that females are less sensitive to generalization of a FP morphine OS than males, so we investigated the role of endogenous ovarian hormones in this difference.

METHODS

Male and female rats received intermixed injections of 3.2 mg/kg morphine or saline before each daily training session. Training consisted of 8 presentations of the CS, each followed by access to sucrose on morphine, but not saline sessions. Following acquisiton, rats were tested for generalization of the morphine stimulus to 0, 1.0, 3.2, and 5.4 mg/kg morphine. Female rats were monitored for estrous cyclicity using vaginal cytology throughout the study.

RESULTS

Both sexes acquired stable drug discrimination. A gradient of generalization was measured across morphine doses and this behaviour did not differ by sex, nor did it differ across the estrous cycle in females.

CONCLUSIONS

Morphine generalization is independent of fluctuations in levels of sex and endogenous gonadal hormones in females under these experimental conditions.

The effects of a retrieval cue on renewal of conditioned responses in human appetitive conditioning

Contextual renewal of reward anticipation may be one potential mechanism underlying relapse in eating and substance use disorders. We therefore tested retrieval cues, a method derived from an inhibitory retrieval-based model of extinction learning to attenuate contextual renewal using an appetitive conditioning paradigm. A pilot study was carried out in Experiment 1 to validate a differential chocolate conditioning paradigm, in which a specific tray was set up as a conditioned stimulus (CS) for eating chocolate (unconditioned stimulus, US). Using an ABA renewal design in Experiment 2, half of the participants were presented with a retrieval cue in the acquisition phase (group AC) and the other half in the extinction phase (group EC). Presentation of the retrieval cue in the EC was associated with reduced renewal of US-expectancy, while there was a clear renewal effect for US-expectancy in the AC. One limitation was the difference in cue presentations between both groups due to the number of trials in acquisition and extinction. Experiment 3 therefore aimed at replicating the results of Experiment 2, but with fewer cue presentations for the EC to match the AC. No significant group differences were observed indicating no effect of the retrieval cue. Theoretical and clinical implications in light of the differing results are discussed.

Bayesian reinforcement learning: A basic overview

We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

Disruption of positive- and negative-feature morphine interoceptive occasion setters by dopamine receptor agonism and antagonism in male and female rats

RATIONALE

Internally perceived stimuli evoked by morphine administration can form Pavlovian associations such that they can function as occasion setters (OSs) for externally perceived reward cues in rats, coming to modulate reward-seeking behaviour. Though much research has investigated mechanisms underlying opioid-related reinforcement and analgesia, neurotransmitter systems involved in the functioning of opioids as Pavlovian interoceptive discriminative stimuli remain to be disentangled despite documented differences in the development of tolerance to analgesic versus discriminative stimulus effects.

OBJECTIVES

Dopamine has been implicated in many opioid-related behaviours, so we aimed to investigate the role of this neurotransmitter in expression of morphine occasion setting.

METHODS

Male and female rats were assigned to positive- (FP) or negative-feature (FN) groups and received an injection of morphine or saline before each training session. A 15-s white noise conditioned stimulus (CS) was presented 8 times during every training session; offset of this stimulus was followed by 4-s access to liquid sucrose on morphine, but not saline, sessions for FP rats. FN rats learned the reverse contingency. Following stable discrimination, rats began generalization testing for expression of morphine-guided sucrose seeking after systemic pretreatment with different doses of the non-selective dopamine receptor antagonist, flupenthixol, and the non-selective dopamine receptor agonist, apomorphine, combined with training doses of morphine or saline in a Latin-square design.

RESULTS

The morphine discrimination was acquired under both FP and FN contingencies by males and females. Neither flupenthixol nor apomorphine at any dose substituted for morphine, but both apomorphine and flupenthixol disrupted expression of the morphine OS. This inhibition was specific to sucrose seeking during CS presentations rather than during the period before CS onset and, in the case of apomorphine more so than flupenthixol, to trials on which access to sucrose was anticipated.

CONCLUSIONS

Our findings lend support to a mechanism of occasion setting involving gating of CS-induced dopamine release rather than by direct dopaminergic modulation by the morphine stimulus.

Proactive Versus Reactive Control Strategies Differentially Mediate Alcohol Drinking in Male Wistars and P Rats

Problematic alcohol consumption is associated with deficits in decision-making and alterations in prefrontal cortex neural activity likely contribute. We hypothesized that the differences in cognitive control would be evident between male Wistars and a model of genetic risk: alcohol-preferring P rats. Cognitive control is split into proactive and reactive components. Proactive control maintains goal-directed behavior independent of a stimulus, whereas reactive control elicits goal-directed behavior at the time of a stimulus. We hypothesized that Wistars would show proactive control over alcohol seeking whereas P rats would show reactive control over alcohol seeking. Neural activity was recorded from the prefrontal cortex during an alcohol seeking task with two session types. On congruent sessions, the conditioned stimulus (CS+) was on the same side as alcohol access. Incongruent sessions presented alcohol opposite the CS+. Wistars, but not P rats, made more incorrect approaches during incongruent sessions, suggesting that Wistars utilized the previously learned rule. This motivated the hypothesis that neural activity reflecting proactive control would be observable in Wistars but not P rats. While P rats showed differences in neural activity at times of alcohol access, Wistars showed differences prior to approaching the sipper. These results support our hypothesis that Wistars are more likely to engage in proactive cognitive control strategies whereas P rats are more likely to engage in reactive cognitive control strategies. Although P rats were bred to prefer alcohol, the differences in cognitive control may reflect a sequela of behaviors that mirror those in humans at risk for an AUD.

Context and Time Regulate Fear Memory Consolidation and Reconsolidation in the Basolateral Amygdala Complex

It is widely accepted that fear memories are consolidated through protein synthesis-dependent changes in the basolateral amygdala complex (BLA). However, recent studies show that protein synthesis is not required to consolidate the memory of a new dangerous experience when it is similar to a prior experience. Here, we examined whether the protein synthesis requirement for consolidating the new experience varies with its spatial and temporal distance from the prior experience. Female and male rats were conditioned to fear a stimulus (S1, e.g., light) paired with shock in stage 1 and a second stimulus (S2, e.g., tone) that preceded additional S1-shock pairings (S2-S1-shock) in stage 2. The latter stage was followed by a BLA infusion of a protein synthesis inhibitor, cycloheximide, or vehicle. Subsequent testing with S2 revealed that protein synthesis in the BLA was not required to consolidate fear to S2 when the training stages occurred 48 h apart in the same context; was required when they were separated by 14 d or occurred in different contexts; but was again not required if S1 was re-presented after the delay or in the different context. Similarly, protein synthesis in the BLA was not required to reconsolidate fear to S2 when the training stages occurred 48 h apart but was required when they occurred 14 d apart. Thus, the protein synthesis requirement for consolidating/reconsolidating fear memories in the BLA is determined by similarity between present and past experiences, the time and place in which they occur, and reminders of the past experiences.

Dopamine-independent effect of rewards on choices through hidden-state inference

Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Continuous time causal structure induction with prevention and generation

Most research into causal learning has focused on atemporal contingency data settings while fewer studies have examined learning and reasoning about systems exhibiting events that unfold in continuous time. Of these, none have yet explored learning about preventative causal influences. How do people use temporal information to infer which components of a causal system are generating or preventing activity of other components? In what ways do generative and preventative causes interact in shaping the behavior of causal mechanisms and their learnability? We explore human causal structure learning within a space of hypotheses that combine generative and preventative causal relationships. Participants observe the behavior of causal devices as they are perturbed by fixed interventions and subject to either regular or irregular spontaneous activations. We find that participants are capable learners in this setting, successfully identifying the large majority of generative, preventative and non-causal relationships but making certain attribution errors. We lay out a computational-level framework for normative inference in this setting and propose a family of more cognitively plausible algorithmic approximations. We find that participants' judgment patterns can be both qualitatively and quantitatively captured by a model that approximates normative inference via a simulation and summary statistics scheme based on structurally local computation using temporally local evidence.

The systemic effects of the enriched environment on the conditioned fear reaction

In this review, a hypothesis is proposed to explain the beneficial effect of an enriched environment (EE) on the conditioned fear reaction (CFR) from the perspective of a functional system of behavioral control. According to the hypothesis, the EE affects all behavioral act components, including the processing of sensory information, memory, motivational and reinforcing systems, and motor activities, which weakens the CFR. Animals raised in the EE have effects that are comparable to those of context (CTX) and CS pre-exposures at latent inhibition. An abundance of stimuli in the EE and constant contact with them provide the formation of CS-noUS and CTX-noUS connections that later, during CFR learning, slow down and diminish fear. The EE also contributes to faster processing of information and habituation to it. As a result, many stimuli in the context lose their significance, and subjects simply ignore them. And finally, the EE affects the motivational and reinforcing brain mechanisms, induces an impairment of search activity, and worsens memory consolidation, which leads to a reduction of CFR.

The psychological basis of hunger and its dysfunctions

This article describes a new and emerging psychological perspective on hunger, together with the implications of that perspective, which is based upon learning and memory. Hunger is a psychological state characterized by a desire to eat. Historically, conceptions of hunger have largely been expressed in terms of physiology (eg, biological process X causes hunger). However, physiology neither offers a psychological account of hunger nor explains why memory impairment can eliminate hunger. Two forms of hunger are identified - specific and general. Specific hunger is for particular palatable foods. It involves recollecting episodic memories of eating that food, when an associated cue is encountered (eg, an advert). General hunger is a desire to eat triggered by temporal (eg, it is lunchtime) or interoceptive (eg, tummy rumble) cues. It involves semantic memory retrieval, which then augments the expected - remembered - pleasure for any food. Both hungers are supported by the medial temporal lobe memory system. Damage to this system can occur from eating a Western-style diet and, longer-term, from obesity and its consequences. Medial temporal lobe memory damage may cause deficits in specific hunger, but most especially in general hunger, resulting in little motivation to eat foods that the individual considers to be of low-to-moderate palatability, such as fruit and vegetables. The implications of this account for teaching people hunger, for how hunger is affected by diet, for public education, and pharmaceutical intervention, are discussed. Psychological concepts of hunger are widely used in nutritional practice. This article provides a new and emerging perspective on the psychological basis of hunger and its implications.

Fear Learning in Genital Pain: Toward a Biopsychosocial, Ecologically Valid Research and Treatment Model

Although fear learning mechanisms are implicated in the development, maintenance, exacerbation, and reduction of genital pain, systematic research on how fear of genital pain emerges, spreads, persists, and reemerges after treatment is lacking. This paper provides an overview of the literature on pain-related fear, integrates the ideas on learning and sexual arousal responding, and specifies the pathways through which compromised learning may contribute to the development and persistence of genital pain. In order to refine theories of genital pain and optimize treatments, we need to adopt a biopsychosocial framework to pain-related fear learning and uncover potential moderators that shape individual trajectories. This involves examining the role of physiological processes, subjective experiences, as well as partner and relational cues in fear acquisition, excessive generalization and impaired safety learning, extinction of fear, counterconditioning, and return of fear. Recent methodological advances in fear conditioning and sex research are promising to enable more symptom-specific and ecologically valid experimental paradigms.

Proactive Versus Reactive Control Strategies Differentially Mediate Alcohol Drinking in Wistar and P rats

Problematic alcohol consumption is associated with deficits in decision-making, and alterations in prefrontal cortex neural activity likely contributes. We hypothesized that differences in cognitive control would be evident between male Wistar rats and a model for genetic risk for alcohol use disorder (alcohol-preferring P rats). Cognitive control can be split into proactive and reactive components. Proactive control maintains goal-directed behavior independent of a stimulus whereas reactive control elicits goal-directed behavior at the time of a stimulus. We hypothesized that Wistars would show proactive control over alcohol-seeking whereas P rats would show reactive control over alcohol-seeking. Neural ensembles were recorded from prefrontal cortex during an alcohol seeking task that utilized two session types. On congruent sessions the CS+ was on the same side as alcohol access. Incongruent sessions presented alcohol opposite the CS+. Wistars, but not P rats, exhibited an increase in incorrect approaches during incongruent sessions, suggesting that Wistars utilized the previously learned task-rule. This motivated the hypothesis that ensemble activity reflecting proactive control would be observable in Wistars but not P rats. While P rats showed differences in neural activity at times relevant for alcohol delivery, Wistars showed differences prior to approaching the sipper. These results support our hypothesis that Wistars are more likely to engage proactive cognitive-control strategies whereas P rats are more likely to engage reactive cognitive control strategies. Although P rats were bred to prefer alcohol, differences in cognitive control may reflect a sequela of behaviors that mirror those in humans at risk for an AUD.

Learning processes in relapse to alcohol use: lessons from animal models

RATIONALE

Alcohol use is reliably preceded by discrete and contextual stimuli which, through diverse learning processes, acquire the capacity to promote alcohol use and relapse to alcohol use.

OBJECTIVE

We review contemporary extinction, renewal, reinstatement, occasion setting, and sex differences research within a conditioning framework of relapse to alcohol use to inform the development of behavioural and pharmacological therapies.

KEY FINDINGS

Diverse learning processes and corresponding neurobiological substrates contribute to relapse to alcohol use. Results from animal models indicate that cortical, thalamic, accumbal, hypothalamic, mesolimbic, glutamatergic, opioidergic, and dopaminergic circuitries contribute to alcohol relapse through separable learning processes. Behavioural therapies could be improved by increasing the endurance and generalizability of extinction learning and should incorporate whether discrete cues and contexts influence behaviour through direct excitatory conditioning or occasion setting mechanisms. The types of learning processes that most effectively influence responding for alcohol differ in female and male rats.

CONCLUSION

Sophisticated conditioning experiments suggest that diverse learning processes are mediated by distinct neural circuits and contribute to relapse to alcohol use. These experiments also suggest that gender-specific behavioural and pharmacological interventions are a way towards efficacious therapies to prevent relapse to alcohol use.

Learning to stop responding

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

Resource limitations: A taxonomy

Decision making within the context of resource limitations requires balancing the short-term benefits of obtaining a resource and the long-term consequences of depleting those resources. The present manuscript focuses on four types of tasks that share this tradeoff to develop a taxonomy that will encourage a deeper understanding of the psychological processes at play. The four types considered are foraging, common pool traps, deterioration traps, and a novel designation referred to as resource cliffs. All four will be shown to include two opposite processes - depletion of the resource and its replenishment over time. By considering the unique and shared features of these tasks, a taxonomy of features emerges that can be combined to not only create novel tasks but also to shift the research focus to task features rather than specific tasks. The paper closes with a consideration of current theoretical frameworks previously applied to one or more of these resource-limitation tasks as well as the promise of reinforcement learning as a unifying theory.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Hierarchical cue control of cocaine seeking in the face of cost

RATIONALE

Addiction is characterized by intermittent drug seeking despite rising costs. This behavior is heavily influenced by environmental stimuli that signal drug availability and reinforce drug seeking.

OBJECTIVE

To establish the relationship between three key aspects of human drug use in rats: the intermittent, binge nature of drug intake, the motivational conflict of drug seeking in the face of escalating negative costs, and the ability of different drug cues to interact to modulate relapse.

METHODS

Male and female rats were trained to self-administer cocaine on an intermittent access schedule, where brief drug-availability states were signaled by a shift in the ambient lighting of the environment, and cocaine infusions were signaled by a separate proximal discrete cue. Rats then went through a conflict procedure, where foot shock intensity associated with cocaine seeking was escalated until intake was suppressed. We then completed relapse tests where the drug-delivery cue was noncontingently presented alone, or in the context of dynamic drug-availability state transitions.

RESULTS

Intermittent access spurred psychomotor sensitization and binge-like cocaine intake. The intensity of binge-like drug taking during training was predictive of later drug seeking despite escalating costs during conflict. In relapse tests, the ability of a proximal discrete drug cue to trigger relapse was gated by the presence of a global cue signaling drug-availability state transitions.

CONCLUSIONS

Our results suggest that the pattern of drug intake plays a role in many features of addiction, including modifying an individual's willingness to endure high costs associated with drug seeking. Furthermore, our studies indicate that drug-related sensory information can be hierarchically organized to exert a dynamic modulating influence on drug-seeking motivation.

Interoceptive hunger, eating attitudes and beliefs

Interoceptive individual differences have garnered interest because of their relationship with mental health. One type of individual difference that has received little attention is variability in the sensation/s that are understood to mean a particular interoceptive state, something that may be especially relevant for hunger. We examined if interoceptive hunger is multidimensional and idiosyncratic, if it is reliable, and if it is linked to dysfunctional eating and beliefs about the causes of hunger. Participants completed a survey just before a main meal, with most retested around 1 month later. We found that interoceptive hunger has 11 dimensions, and while people differ considerably in their combinations of interoceptive hungers, these represent only 4% of all possible permutations. Hunger reports were reliable. We found relationships between variability in hunger interoception and dysfunctional eating, especially for uncontrolled eating. We also found that hunger beliefs were in some cases strongly related to aspects of hunger interoception. The implications of these findings are discussed.

Inhibitory Learning with Bidirectional Outcomes: Prevention Learning or Causal Learning in the Opposite Direction?

Influential models of causal learning assume that learning about generative and preventive relationships are symmetrical to each other. That is, a preventive cue directly prevents an outcome from occurring (i.e., "direct" prevention) in the same way a generative cue directly causes an outcome to occur. However, previous studies from our lab have shown that many participants do not infer a direct prevention causal structure after feature-negative discrimination (A+/AB-) with a unidirectional outcome (Lee & Lovibond, 2021). Melchers et al. (2006) suggested that the use of a bidirectional outcome that can either increase or decrease from baseline, encourages direct prevention learning. Here we test an alternative possibility that a bidirectional outcome encourages encoding of a generative relationship in the opposite direction, where B directly causes a decrease in the outcome. Thus, previous evidence of direct prevention learning using bidirectional outcomes may instead be explained by some participants inferring an "Opposite Causal" structure. In two experiments, participants did indeed report an opposite causal structure. In Experiment 1, these participants showed the lowest outcome predictions when B was combined with a novel cause in a summation test, and lowest outcome predictions when B was presented alone. In Experiment 2, B successfully blocked learning to a novel cue that was directly paired with a reduction in the outcome, and this effect was strongest among participants who endorsed an Opposite Causal structure. We conclude that previous evidence of direct prevention learning using bidirectional outcomes may be a product of excitatory rather than inhibitory learning.

Hierarchical and configural control in conditional discrimination learning

Considerable discussion has concerned the role of context in conditional discrimination learning. Some authors have proposed that contexts might operate hierarchically on CS-US associations, whereas others have proposed that the context plus the CS might be processed configurally. In the present article, we report the results of two experiments that assessed the role of context on pigeons' conditional discrimination learning. In Experiment 1, we found that our pigeons' responding was inconsistent with hierarchical processing; instead, they may have either relied on local features or on configural compounds comprising the context and the discriminative stimulus presented on each trial. In Experiment 2, we precluded the possibility of using local features by requiring the pigeons to attend to both of the compounds that were simultaneously presented on each trial to solve the task. Methodological and theoretical issues are discussed in light of this work. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Ambiguity drives higher-order Pavlovian learning

In the natural world, stimulus-outcome associations are often ambiguous, and most associations are highly complex and situation-dependent. Learning to disambiguate these complex associations to identify which specific outcomes will occur in which situations is critical for survival. Pavlovian occasion setters are stimuli that determine whether other stimuli will result in a specific outcome. Occasion setting is a well-established phenomenon, but very little investigation has been conducted on how occasion setters are disambiguated when they themselves are ambiguous (i.e., when they do not consistently signal whether another stimulus will be reinforced). In two preregistered studies, we investigated the role of higher-order Pavlovian occasion setting in humans. We developed and tested the first computational model predicting direct associative learning, traditional occasion setting (i.e., 1st-order occasion setting), and 2nd-order occasion setting. This model operationalizes stimulus ambiguity as a mechanism to engage in higher-order Pavlovian learning. Both behavioral and computational modeling results suggest that 2nd-order occasion setting was learned, as evidenced by lack and presence of transfer of occasion setting properties when expected and the superior fit of our 2nd-order occasion setting model compared to the 1st-order occasion setting or direct associations models. These results provide a controlled investigation into highly complex associative learning and may ultimately lead to improvements in the treatment of Pavlovian-based mental health disorders (e.g., anxiety disorders, substance use).

Appetitive interoception, the hippocampus and western-style diet

Obesity, Type 2 diabetes and other metabolic disorders continue to pose serious challenges to human health and well-being. An important source of these challenges is the overconsumption of saturated fats and sugar, main staples of what has been called the Western-style diet (WD). The current paper describes a theoretical model and supporting evidence that links intake of a WD to interference with a specific brain substrate that underlies processing of interoceptive signals of hunger and satiety. We review findings from rats and humans that the capacity of these signals to modulate the strength of appetitive and eating behavior depends on the functional integrity of the hippocampus and the learning memory operations it performs. Important among these operations is the use of contextual information to retrieve memories that are associated with other events. Within our framework, satiety provides an interoceptive context that informs animals that food cues and appetitive behavior will not be followed by rewarding postingestive outcomes. This serves to prevent those cues and responses from retrieving those reward memories. The findings reviewed provide evidence that consuming a WD and the high amounts of saturated fat and sugar it contains (a) is associated with the emergence of pathophysiologies to which the hippocampus appears selectively vulnerable (b) impairs hippocampal-dependent learning and memory (HDLM) and (c) weakens behavioral control by interoceptive hunger and satiety contextual stimuli. It is hypothesized that these consequences of WD intake may establish the conditions for a vicious cycle of further WD intake, obesity, and potentially cognitive decline.

The Medial Prefrontal Cortex, Nucleus Accumbens, Basolateral Amygdala, and Hippocampus Regulate the Amelioration of Environmental Enrichment and Cue in Fear Behavior in the Animal Model of PTSD

A growing body of evidence showed that environmental enrichment (EE) ameliorated footshock-induced fear behavior of posttraumatic stress disorder (PTSD). However, no research comprehensively tested the effect of EE, cue, and the combination of EE and cue in footshock-induced fear behavior of PTSD symptoms. The present study addressed this issue and examined whether the medial prefrontal cortex (mPFC, including the cingulate cortex 1 (Cg1), prelimbic cortex (PrL), and infralimbic cortex (IL)), the nucleus accumbens (NAc), the basolateral amygdala (BLA), and the hippocampus (e.g., CA1, CA3, and dentate gyrus (DG)) regulated the amelioration of the EE, cue, or the combination of EE and cue. The results showed that EE or cue could reduce fear behavior. The combination of EE and cue revealed a stronger decrease in fear behavior. The cue stimulus may play an occasion setting or a conditioned stimulus to modulate the reduction in fear behavior induced by footshock. Regarding the reduction of the EE in fear behavior, the Cg1 and IL of the mPFC and the NAc upregulated the c-Fos expression; however, the BLA downregulated the c-Fos expression. The mPFC (i.e., the Cg1, PrL, and IL) and the hippocampus (i.e., the CA1, CA3, and DG) downregulated the c-Fos expression in the suppression of the cue in fear behavior. The interaction of EE and cue in reduction of fear behavior occurred in the Cg1 and NAc for the c-Fos expression. The data of c-Fos mRNA were similar to the findings of the c-Fos protein expression. These findings related to the EE and cue modulations in fear behavior may develop a novel nonpharmacological treatment in PTSD.

Predictive and motivational factors influencing anticipatory contrast: A comparison of contextual and gustatory predictors in food restricted and free-fed rats

•

Using an anticipatory negative contrast (ANC) paradigm, food restricted animals can act selectively in their eating behavior.

•

Contextual and gustatory predictors in a within-subject design are sufficient for anticipatory negative contrast development.

•

Changes in reward palatability may underlie contextually-driven anticipatory negative contrast.

•

An increase in premature port entries to the unavailable sipper – a second measure of ANC – in all groups reveals a direct influence of response competition on ANC development.

In anticipation of palatable food, rats can learn to restrict consumption of a less rewarding food type resulting in an increased consumption of the preferred food when it is made available. This construct is known as anticipatory negative contrast (ANC) and can help elucidate the processes that underlie binge-like behavior as well as self-control in rodent motivation models. In the current investigation we aimed to shed light on the ability of distinct predictors of a preferred food choice to generate contrast effects and the motivational processes that underlie this behavior. Using a novel set of rewarding solutions, we directly compared contextual and gustatory ANC predictors in both food restricted and free-fed Sprague-Dawley rats. Our results indicate that, despite being food restricted, rats are selective in their eating behavior and show strong contextually-driven ANC similar to free-fed animals. These differences mirrored changes in palatability for the less preferred solution across the different sessions as measured by lick microstructure analysis. In contrast to previous research, predictive cues in both food restricted and free-fed rats were sufficient for ANC to develop although flavor-driven ANC did not relate to a corresponding change in lick patterning. These differences in the lick microstructure between context- and flavor-driven ANC indicate that the motivational processes underlying ANC generated by the two predictor types are distinct. Moreover, an increase in premature port entries to the unavailable sipper – a second measure of ANC – in all groups reveals a direct influence of response competition on ANC development.

Alcohol Seeking Under Risk of Punishment Is Associated With Activation of Cortical and Subcortical Brain Regions

In humans, stimuli associated with alcohol availability can provoke relapse during abstinence. In this study, we investigated the role of discriminative stimuli (DS) in the control of alcohol seeking in two types of behavioral tests. The first test examined the ability of an alcohol-associated DS to promote alcohol seeking (relapse) after punishment-imposed abstinence in the presence of a different DS. Following this, we tested whether the differentially associated DS can promote and suppress alcohol self-administration in a within-session discrimination task. During the within-session discrimination task, we also tested the rate of alcohol self-administration when two DS are presented in a compound. We first trained Long-Evans male rats (n = 24) to self-administer alcohol in the presence of one DS (reward-associated discriminative stimulus, rewDS) and then punished that behavior in the presence of a different DS (punishment-associated discriminative stimulus, punDS). On the test, we found that rats tested with the rewDS showed higher alcohol seeking than rats tested with the punDS. This result shows that a single Cue DS can promote alcohol seeking in a manner comparable to contexts. Subsequently, we trained 16 of these rats in a within-session trial-based discrimination task, comprised of intervening 2-min trials of rewDS, punDS, or conflict with rewDS and punDS in compound and a reduced probability of punishment. We found that alcohol self-administration is bi-directionally regulated by the rewDS and punDS. In conflict trials, alcohol self-administration was at a rate that was intermediate between the rewDS and punDS trials. In a final test, rats were presented with one of the three trial conditions and perfused for Fos immunohistochemistry. We found Fos expression was higher in the rats tested in the conflict condition in three interconnected sub-cortical brain regions. This study demonstrated the important role that alcohol-associated DS plays an important role in promoting relapse to alcohol seeking after punishment-imposed abstinence. We also implemented a within-session discrimination task that allows for the study of alcohol seeking under motivational conflict, which may be relevant for alcohol use despite negative consequences. The results from the Fos data suggest that higher alcohol seeking in approach-avoidance motivational conflict is associated with activation of sub-cortical regions but not cortical regions.

The retrosplenial cortex as a possible "sensory integration" area: A neural network modeling approach of the differential outcomes effect in negative patterning

We explored the hypothesis that learning a Pavlovian negative patterning task would be facilitated when training with differential, as opposed to non-differential, reinforcing outcomes. Two groups of rats received pairings of one visual and one auditory stimulus with food reward when these stimuli were presented on separate training trials, but without reward when both stimuli were presented on simultaneous stimulus compound trials (V+, A+, AV-; similar to an XOR problem). For Group Differential, each stimulus was separately paired with distinctively tasting food rewards, whereas for Group Non-Differential each stimulus was randomly paired with both food reward types across different stimulus element trials. We observed that rats learned the negative patterning task more rapidly and effectively when trained with differential outcomes. These data support a multi-layered connectionist model introduced by Delamater (2012) in which a multi-modal processing structure plays the role of a "sensory integration" area like that hypothesized for the retrosplenial cortex by Dave Bucci and his colleagues (e.g., Todd, Fournier, & Bucci, 2019). We discuss how such a region may develop different "negative occasion setting" and "configural inhibition" mechanisms in solving negative patterning and related tasks.

Conditional Control of Instrumental Avoidance by Context Following Extinction

Using rodents, three training arrangements (i.e., ABB vs. ABA, AAA vs. AAB and ABB vs. ABC) explored whether extinction influences the expression of avoidance in a manner controlled by context. Retention testing following extinction showed that more avoidance responding (i.e., renewal) was observed when extinguished cues were tested outside of the context where they had undergone extinction. In contrast, response rates were significantly lower when stimuli were tested within the context where extinction learning had occurred. These findings add to the emerging literature assessing the role of Pavlovian extinction processes in the development of instrumental avoidance responding by demonstrating conditional control over extinguished responding by context. This study was conducted using a within-subjects approach that minimized the potential for context-outcome associations to bias responding, and thus, reflects hierarchical control over behavior based on the specific associative status of each tested cue in each training context.

Pavlovian occasion setting in human fear and appetitive conditioning: Effects of trait anxiety and trait depression

Contexts and discrete stimuli often hierarchically influence the association between a stimulus and outcome. This phenomenon, called occasion setting, is central to modulation-based Pavlovian learning. We conducted two experiments with humans in fear and appetitive conditioning paradigms, training stimuli in differential conditioning, feature-positive discriminations, and feature-negative discriminations. We also investigated the effects of trait anxiety and trait depression on these forms of learning. Results from both experiments showed that participants were able to successfully learn which stimuli predicted the electric shock and monetary reward outcomes. Additionally, as hypothesized, the stimuli trained as occasion setters had little-to-no effect on simple reinforced or non-reinforced stimuli, suggesting the former were indeed occasion setters. Lastly, in fear conditioning, trait anxiety was associated with increases in fear of occasion setter/conditional stimulus compounds; in appetitive conditioning, trait depression was associated with lower expectations of monetary reward for the trained negative occasion setting compound and transfer of the negative occasion setter to the simple reinforced stimulus. These results suggest that clinically anxious individuals may have enhanced fear of occasion setting compounds, and clinically depressed individuals may expect less reward with compounds involving the negative occasion setter.

Drug-associated cues and drug dosage contribute to increased opioid seeking after abstinence

Patients with opioid use disorder experience high rates of relapse during recovery, despite successful completion of rehabilitation programs. A key factor contributing to this problem is the long-lasting nature of drug-seeking behavior associated with opioid use. We modeled this behavior in a rat drug self-administration paradigm in which drug-seeking is higher after extended abstinence than during the acute abstinence phase. The goal of this study was to determine the contribution of discrete or discriminative drug cues and drug dosage to time-dependent increases in drug-seeking. We examined heroin-seeking after 2 or 21 days of abstinence from two different self-administration cue-context environments using high or low doses of heroin and matched animals for their drug intake history. When lower dosages of heroin are used in discriminative or discrete cue protocols, drug intake history contributed to drug-seeking after abstinence, regardless of abstinence length. Incubation of opioid craving at higher dosages paired with discrete drug cues was not dependent on drug intake. Thus, interactions between drug cues and drug dosage uniquely determined conditions permissible for incubation of heroin craving. Understanding factors that contribute to long-lasting opioid-seeking can provide essential insight into environmental stimuli and drug-taking patterns that promote relapse after periods of successful abstinence.

When gut feelings teach the brain to fear pain: Context-dependent activation of the central fear network in a novel interoceptive conditioning paradigm

The relevance of contextual factors in shaping neural mechanisms underlying visceral pain-related fear learning remains elusive. However, benign interoceptive sensations, which shape patients' clinical reality, may context-dependently become conditioned predictors of impending visceral pain. In a novel context-dependent interoceptive conditioning paradigm, we elucidated the putative role of the central fear network in the acquisition and extinction of pain-related fear induced by interoceptive cues and pain-predictive contexts. In this fMRI study involving rectal distensions as a clinically-relevant model of visceroception, N = 27 healthy men and women underwent differential conditioning. During acquisition training, visceral sensations of low intensity as conditioned stimuli (CS) predicted visceral pain as unconditioned stimulus (US) in one context (Con⁺), or safety from pain in another context (Con^-). During extinction training, interoceptive CS remained unpaired in both contexts, which were operationalized as images of different rooms presented in the MRI scanner. Successful contextual conditioning was supported by increased negative valence of Con⁺ compared to Con^- after acquisition training, which resolved after extinction training. Although interoceptive CS were perceived as comparatively pleasant, they induced significantly greater neural activation of the amygdala, ventromedial PFC, and hippocampus when presented in Con⁺, while contexts alone did not elicit differential responses. During extinction training, a shift from CS to context differentiation was observed, with enhanced responses in the amygdala, ventromedial, and ventrolateral PFC to Con⁺ relative to Con^-, whereas no CS-induced differential activation emerged. Context-dependent interoceptive conditioning can turn benign interoceptive cues into predictors of visceral pain that recruit key regions of the fear network. This first evidence expands knowledge about learning and memory mechanisms underlying interoceptive hypervigilance and maladaptive avoidance behavior, with implications for disorders of the gut-brain axis.

Inhibitory causal structures in serial and simultaneous feature negative learning

We have previously reported that human participants trained with a simultaneous feature negative discrimination (intermixed A+/AB− trials) show only modest transfer of inhibitory properties of feature B to a separately trained excitor in a summation test. Their self-reported causal structure suggested that many participants learned that the effect of feature B was somewhat specific to the excitor it had been trained with (modulation), rather than learning that the feature prevented the outcome (prevention). This pattern is reminiscent of the distinction between negative occasion-setting and conditioned inhibition in the animal conditioning literature. However, in animals, occasion-setting is more commonly seen with a serial procedure, in which the feature (B) precedes the training excitor (A). Accordingly, we ran three experiments to compare serial with simultaneous training in an allergist causal judgement task. Transfer in a summation test was stronger to a previously modulated test excitor compared to a simple excitor after both simultaneous and serial training. There was a numerical trend towards a larger effect in the serial group, but it failed to reach significance and the Bayes Factor indicated support for the null. Serial training had no differential effect on the self-reported causal structure and did not significantly reduce overall transfer. After both simultaneous and serial training, transfer was strongest in participants who reported a prevention structure, replicating and extending our previous results to a previously modulated excitor. These results suggest that serial feature negative training does not promote a qualitatively different inhibitory causal structure compared to simultaneous training in humans.

Behavioral and brain mechanisms mediating conditioned flight behavior in rats

Environmental contexts can inform animals of potential threats, though it is currently unknown how context biases the selection of defensive behavior. Here we investigated context-dependent flight responses with a Pavlovian serial-compound stimulus (SCS) paradigm that evokes freeze-to-flight transitions. Similar to previous work in mice, we show that male and female rats display context-dependent flight-like behavior in the SCS paradigm. Flight behavior was dependent on contextual fear insofar as it was only evoked in a shock-associated context and was reduced in the conditioning context after context extinction. Flight behavior was only expressed to white noise regardless of temporal order within the compound. Nonetheless, rats that received unpaired SCS trials did not show flight-like behavior to the SCS, indicating it is associative. Finally, we show that pharmacological inactivation of two brain regions critical to the expression of contextual fear, the central nucleus of the amygdala (CeA) and bed nucleus of the stria terminalis (BNST), attenuates both contextual fear and flight responses. All of these effects were similar in male and female rats. This work demonstrates that contextual fear can summate with cued and innate fear to drive a high fear state and transition from post-encounter to circa-strike defensive modes.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Pavlovian Learning Processes in Pediatric Anxiety Disorders: A Critical Review

Deficits in associative and Pavlovian learning are thought to lie at the center of anxiety-related disorders. However, the majority of studies have been carried out in adult populations. The aim of this review was to critically examine the behavioral and neuroimaging literature on Pavlovian learning in pediatric anxiety disorders. We conclude that although there is evidence for deficits in Pavlovian processes (e.g., heightened reactivity to safety cues in anxious samples), the extant literature suffers from key methodological and theoretical issues. We conclude with theoretical and methodological recommendations for future research in order to further elucidate the role of Pavlovian learning in the etiology, maintenance, and treatment of pediatric anxiety disorders.

Sex differences in the discriminative stimulus characteristics of a morphine occasion setter in rats

The current study investigated whether the stimulus effects of morphine can function as a positive and negative feature in a Pavlovian occasion setting drug discrimination preparation in male and female rats. Sprague-Dawley rats were assigned to a feature positive (FP) or feature negative (FN) training group and all received intermixed morphine (3.2 mg/kg, IP) or saline injections 15 min before 20-min daily training sessions. For FP rats, on morphine sessions, each of eight 15-s white noise (WN) presentations was followed by 4-s access to sucrose (0.01 ml, 26% w/v); on saline sessions, sucrose was withheld. FN rats learned the reverse contingency. FP discrimination was acquired somewhat sooner than FN discrimination, and females, but not males, became sensitized to the locomotor effects of morphine, which did not influence conditioned responding. Rats then entered dose generalization testing. There was no sex difference in dose generalization for FN groups (ED₅₀ 1.26 for males and 1.57 for females). Yet for FP rats, the dose response curve for females was shifted to the right compared to males (ED₅₀ 0.54 for males and 1.94 for females). FP females exhibited enhanced responding at a dose higher than that of their original training. These findings reveal the need to reassess our notions of drug stimuli that guide appropriate associative behaviours from the perspective of sex differences.

Reward-Related Attentional Capture Moderates the Association between Fear-Driven Motives and Heavy Drinking

BACKGROUND

To date, there has been little investigation on how motivational and cognitive mechanisms interact to influence problematic drinking behaviours. Towards this aim, the current study examined whether reward-related attentional capture is associated with reward, fear (relief), and habit drinking motives, and further, whether it interacts with these motives in relation to problematic drinking patterns.

METHODS

Ninety participants (mean age = 34.8 years, SD = 9.1, 54% male) who reported having consumed alcohol in the past month completed an online visual search task that measured reward-related attentional capture as well as the Habit Reward Fear Scale, a measure of drinking motives. Participants also completed measures of psychological distress, impulsivity, compulsive drinking, and consumption items of Alcohol Use Disorders Identification Test. Regression analyses examined the associations between motives for alcohol consumption and reward-related attentional capture, as well as the associations between reward-related attentional capture, motives, and their interaction, with alcohol consumption and problems.

RESULTS

Greater reward-related attentional capture was associated with greater reward motives. Further, reward-related attentional capture also interacted with fear motives in relation to alcohol consumption. Follow-up analyses showed that this interaction was driven by greater fear motives being associated with heavier drinking among those with lower reward-related attentional capture (i.e., "goal-trackers").

CONCLUSION

These findings have implications for understanding how cognition may interact with motives in association with problematic drinking. Specifically, the findings highlight different potential pathways to problematic drinking according to an individual's cognitive-motivational profile and may inform tailored interventions to target profile-specific mechanisms. Finally, these findings offer support for contemporary models of addiction that view excessive goal-directed behaviour under negative affect as a critical contributor to addictive behaviours.

Individual differences in causal structures inferred during feature negative learning

Traditional associative learning theories predict that training with feature negative (A+/AB-) contingencies leads to the feature B acquiring negative associative strength and becoming a conditioned inhibitor (i.e., prevention learning). However, feature negative training can sometimes result in negative occasion setting, where B modulates the effect of A. Other studies suggest that participants learn about configurations of cues rather than their individual elements. In this study, we administered simultaneous feature negative training to participants in an allergist causal learning task and tested whether evidence for these three types of learning (prevention, modulation, configural) could be captured via self-report in the absence of any procedural manipulation. Across two experiments, we show that only a small subset of participants endorse the prevention option, suggesting that traditional associative models that predict conditioned inhibition do not completely capture how humans learn about negative contingencies. We also show that the degree of transfer in a summation test corresponds to the implied causal structure underlying conditioned inhibition, occasion-setting, and configural learning, and that participants are only partially sensitive to explicit hints about causal structure. We conclude that feature negative training is an ambiguous causal scenario that reveals individual differences in the representation of inhibitory associations, potentially explaining the modest group-level inhibitory effects often found in humans.

Occasion setters determine responses of putative DA neurons to discriminative stimuli

Midbrain dopamine (DA) neurons are involved in the processing of rewards and reward-predicting stimuli, possibly analogous to reinforcement learning reward prediction errors. Here we studied the activity of putative DA neurons (n = 37) recorded in the ventral tegmental area of rats (n = 6) performing a behavioural task involving occasion setting. In this task an occasion setter (OS) indicated that the relationship between a discriminative stimulus (DS) and reinforcement is in effect, so that reinforcement of bar pressing occurred only after the OS (tone or houselight) was followed by the DS (houselight or tone). We found that responses of putative DA cells to the DS were enhanced when preceded by the OS, as were behavioural responses to obtain rewards. Surprisingly though, we did not find a homogeneous increase in the mean activity of the population of putative DA neurons to the OS, contrary to predictions of standard temporal-difference models of DA neurons. Instead, putative DA neurons exhibited a heterogeneous response on a single unit level, so that some units increased and others decreased their activity as a response to the OS. Similarly, putative non-DA cells did not show a homogeneous response to the DS on a population level, but also had heterogeneous responses on a single unit level. The heterogeneity in the responses of neurons in the ventral tegmental area may reflect how DA neurons encode context and point to local differences in DA signalling.

Mapping sign-tracking and goal-tracking onto human behaviors

As evidenced through classic Pavlovian learning mechanisms, environmental cues can become incentivized and influence behavior. These stimulus-outcome associations are relevant in everyday life but may be particularly important for the development of impulse control disorders including addiction. Rodent studies have elucidated specific learning profiles termed 'sign-tracking' and 'goal-tracking' which map onto individual differences in impulsivity and other behaviors associated with impulse control disorders' etiology, course, and relapse. Whereas goal-trackers are biased toward the outcome, sign-trackers fixate on features that are associated with but not necessary for achieving an outcome; a pattern of behavior that often leads to escalation of reward-seeking that can be maladaptive. The vast majority of the sign- and goal-tracking research has been conducted using rodent models and very few have bridged this concept into the domain of human behavior. In this review, we discuss the attributes of sign- and goal-tracking profiles, how these are manifested neurobiologically, and how these distinct learning styles could be an important tool for clinical interventions in human addiction.

Dopaminergic contributions to behavioral control under threat of punishment in rats

RATIONALE

Excessive intake of rewards, such as food and drugs, often has explicit negative consequences, including the development of obesity and addiction, respectively. Thus, choosing not to pursue reward is the result of a cost/benefit decision, proper execution of which requires inhibition of behavior. An extensive body of preclinical and clinical evidence implicates dopamine in certain forms of inhibition of behavior, but it is not fully known how it contributes to behavioral inhibition under threat of explicit punishment.

OBJECTIVES

To assess the involvement of midbrain dopamine neurons and their corticostriatal output regions, the ventral striatum and prefrontal cortex, in control over behavior under threat of explicit (foot shock) punishment in rats.

METHODS

We used a recently developed behavioral inhibition task, which assesses the ability of rats to exert behavioral restraint at the mere sight of food reward, under threat of foot shock punishment. Using in vivo fiber photometry, chemogenetics, c-Fos immunohistochemistry, and behavioral pharmacology, we investigated how dopamine neurons in the ventral tegmental area, as well as its output areas, the ventral striatum and prefrontal cortex, contribute to behavior in this task.

RESULTS

Using this multidisciplinary approach, we found little evidence for a direct involvement of ascending midbrain dopamine neurons in inhibitory control over behavior under threat of punishment. For example, photometry recordings suggested that VTA DA neurons do not directly govern control over behavior in the task, as no differences were observed in neuronal population activity during successful versus unsuccessful behavioral control. In addition, chemogenetic and pharmacological manipulations of the mesocorticolimbic DA system had little or no effect on the animals' ability to exert inhibitory control over behavior. Rather, the dopamine system appeared to have a role in the motivational components of reward pursuit.

CONCLUSIONS

Together, our data provide insight into the mesocorticolimbic mechanisms behind motivated behaviors by showing a modulatory role of dopamine in the expression of cost/benefit decisions. In contrast to our expectations, dopamine did not appear to directly mediate the type of behavioral control that is tested in our task.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event—an occasion setter (OS)—which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.