Occasion setting, inhibition, and the contextual control of extinction in Pavlovian and instrumental (operant) learning

Reward-related attentional capture and cognitive inflexibility interact to determine greater severity of compulsivity-related problems

BACKGROUND AND OBJECTIVES

Neurocognitive processes are key drivers of addictive and compulsive disorders. The current study examined whether reward-related attentional capture and cognitive inflexibility are associated with impulsive and/or compulsive personality traits, and whether these cognitive characteristics interact to predict greater compulsivity-related problems across obsessive-compulsive and drinking behaviors.

METHODS

One-hundred and seventy-three participants (mean age = 34.5 years, S.D = 8.4, 42% female) completed an online visual search task to measure reward-related attentional capture and its persistence following reversal of stimulus-reward contingencies. Participants also completed questionnaires to assess trait impulsivity, compulsivity, alcohol use, and obsessive-compulsive behaviors.

RESULTS

Greater reward-related attentional capture was associated with trait compulsivity, over and above all impulsivity dimensions, while greater cognitive inflexibility was associated with higher negative urgency (distress-elicited impulsivity). Reward-related attentional capture and cognitive inflexibility interacted to predict greater compulsivity-related problems among participants who reported obsessive-compulsive behaviors in the past month (n = 57) as well as current drinkers (n = 88). Follow-up analyses showed that, for OCD behaviors, this interaction was driven by an association between higher reward-related attentional capture and more problematic behaviors among cognitively inflexible participants only. For drinking, the same pattern was seen, albeit at trend level.

LIMITATIONS

This study includes a non-clinical, online sample and is cross-sectional, thus its findings need to be interpreted with these limitations in mind.

CONCLUSIONS

Reward-related attentional capture and cognitive flexibility are related to trait compulsivity and impulsivity (negative urgency) respectively, and interact to determine more problematic behaviors.

Repeated resurgence with and without a context change

Resurgence is the reoccurrence of a target response when reinforcement for a more recently reinforced alternative response is eliminated or reduced. The present study arranged two successive three-phase procedures to assess whether resurgence decreases with repeated assessments. Moreover, we arranged a contextual change from the first to second assessment for some groups. Phase 1 reinforced a target response on a touchscreen computer with typically developing adults as participants according to either variable-ratio or variable-interval schedules of reinforcement. Phase 2 extinguished target responding and reinforced alternative responding. Phase 3 tested for resurgence by extinguishing alternative responding. Resurgence reliably occurred in all tests and decreased from the first to second exposure to the procedures but there were no effects of context change. Therefore, repeated exposures to resurgence tests reduced those effects but contextual changes had no effect.

Cues Associated with Alternative Reinforcement During Extinction Can Attenuate Resurgence of an Extinguished Instrumental Response

In resurgence, a target behavior (R1) is acquired in an initial phase and extinguished in a second phase while an R2 behavior is reinforced. When R2 is extinguished, R1 behavior can return or resurge. Two experiments tested the effectiveness of a potential retrieval cue associated with extinction in attenuating resurgence. Experiment 1 established that a 2-s cue paired with outcome delivery in Phase 2 can attenuate resurgence when presented during testing. This effect depended on the cue being associated with the outcome, and it occurred if the cue was delivered contingently or noncontingently on responding during testing. Pairing the cue with reinforcement might be necessary to maintain attention to it during Phase 2. Experiment 2 demonstrated that the cue must be experienced in sessions that also include R1 extinction and that it does not reduce resurgence through a conditioned reinforcement mechanism. The results suggest that previously neutral stimuli can attenuate resurgence if they are first paired with alternative reinforcement and presented in sessions in which R1 is extinguished. They build on existing literature that suggests enhancing generalization between extinction and testing reduces resurgence. The results may have implications for reducing relapse following interventions in humans such as contingency management (CM), in which participants can earn vouchers contingent upon drug abstinence. A cue associated with CM might help reduce this relapse.

Understanding Addiction Using Animal Models

Drug addiction is a neuropsychiatric disorder with grave personal consequences that has an extraordinary global economic impact. Despite decades of research, the options available to treat addiction are often ineffective because our rudimentary understanding of drug-induced pathology in brain circuits and synaptic physiology inhibits the rational design of successful therapies. This understanding will arise first from animal models of addiction where experimentation at the level of circuits and molecular biology is possible. We will review the most common preclinical models of addictive behavior and discuss the advantages and disadvantages of each. This includes non-contingent models in which animals are passively exposed to rewarding substances, as well as widely used contingent models such as drug self-administration and relapse. For the latter, we elaborate on the different ways of mimicking craving and relapse, which include using acute stress, drug administration or exposure to cues and contexts previously paired with drug self-administration. We further describe paradigms where drug-taking is challenged by alternative rewards, such as appetitive foods or social interaction. In an attempt to better model the individual vulnerability to drug abuse that characterizes human addiction, the field has also established preclinical paradigms in which drug-induced behaviors are ranked by various criteria of drug use in the presence of negative consequences. Separation of more vulnerable animals according to these criteria, along with other innate predispositions including goal- or sign-tracking, sensation-seeking behavior or impulsivity, has established individual genetic susceptibilities to developing drug addiction and relapse vulnerability. We further examine current models of behavioral addictions such as gambling, a disorder included in the DSM-5, and exercise, mentioned in the DSM-5 but not included yet due to insufficient peer-reviewed evidence. Finally, after reviewing the face validity of the aforementioned models, we consider the most common standardized tests used by pharmaceutical companies to assess the addictive potential of a drug during clinical trials.

A review of investigations of operant renewal with human participants: Implications for theory and practice

Operant renewal is the recurrence of a previously eliminated target behavior as a function of changing stimulus contexts. Renewal as a model of treatment relapse in humans suggests that a change in stimulus conditions or context is sufficient to produce relapse of a previously eliminated maladaptive behavior. The extent to which general findings from operant renewal studies involving nonhuman animal subjects are supported by relapse studies involving human participants is unknown. We conducted a systematic review of studies demonstrating or mitigating operant renewal in human participants in peer-reviewed studies found in PsycINFO, ERIC, PubMed, and Scopus between 1980 and 2019. We identified 12 studies involving 61 participants and 93 cases of operant renewal. We coded descriptive data on participant and study characteristics and calculated summary statistics. Results indicated that the renewal effect was a robust phenomenon, supported by demonstrations in both clinical and human-laboratory studies, and across a variety of variables and experimental preparations. However, there were relatively few studies involving human participants that attempted to reduce or eliminate renewal of clinically meaningful behavior. We discuss variables relevant for studying renewal in socially meaningful contexts, practical limitations of observing the renewal effect in real-world settings, implications for theoretical models of renewal, and identify barriers to methodology unique to human participants. We provide directions for future research related to implementing and translating nonhuman animal studies of renewal to applied settings.

The reoccurrence of voluntary behavior in humans is reduced by retrieval cues from extinction

Changes in the temporal as well as the physical context produces the reappearance of extinguished behaviors. Furthermore, combining both kinds of contextual stimuli often causes greater levels of recovery. The current experiment explored the impact of extinction reminders on spontaneous recovery, renewal, and a combination of both effects using an instrumental learning task with humans. All participants learned to shoot at enemies in a videogame. Then, throughout extinction, the instrumental response was eliminated. We found a return of the extinguished behavior by introducing a retention interval of 48 h, by changing the physical background and by testing participants in a spatiotemporal context different from the extinction context. However, we also found that the presentation of a stimulus directly associated with extinction attenuates all three forms of operant reoccurrence. These results are consistent with the perspective that emphasizes that context plays a key role in response-recovery phenomena. Moreover, our findings may be promissory for therapeutic strategies involving relapse treatment.

Occasion setting

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The medial prefrontal cortex is required for responding to alcohol-predictive cues but only in the absence of alcohol delivery

BACKGROUND

The prelimbic medial prefrontal cortex is implicated in promoting drug-seeking in relapse tests. However, drug-seeking behaviour is typically extinguished before a test and tests normally occur without drug delivery.

AIMS

We investigated the involvement of the prelimbic and the infralimbic cortex in responding elicited by a non-extinguished cue for alcohol that was presented without alcohol in an alcohol-associated context or a neutral context, and in responding to the same cue when it was paired with alcohol.

METHODS

Male, Long-Evans rats (220-240 g on arrival) were acclimated to 15% ethanol (v/v; 'alcohol') and then trained to associate a conditioned stimulus (10 s white noise; 15 trials/session) with alcohol delivery into a fluid port (0.2 mL/conditioned stimulus, 3 mL per session) for oral intake. Conditioning sessions occurred in a specific 'alcohol context' and were alternated daily with exposure to a second 'neutral' context that contained neither the conditioned stimulus nor alcohol.

RESULTS

At test, functional prelimbic cortex inactivation using baclofen/muscimol reduced fluid port entries elicited by a non-extinguished conditioned stimulus that was presented without alcohol, but had no subsequent impact on port entries when the conditioned stimulus was paired with alcohol. Similar results were obtained following infralimbic cortex inactivation; however, infralimbic cortex inactivation also non-specifically reduced port entries in the absence of alcohol.

CONCLUSIONS

These data indicate that the prelimbic and infralimbic cortex are involved in responding to cues for alcohol when alcohol delivery is omitted, but suggest that other brain regions are engaged in responding to such cues in the presence of alcohol.

Stable Individual Differences in Occasion Setting

Abstract. In the current investigation, we classified participants as inhibitors or non-inhibitors depending on the extent to which they showed conditioned inhibition in a context that had been used for extinction of a conditioned response. This classification enabled us to predict participant responses in a second experiment which used a different design and a different experimental task. In the second experiment a feature-negative discrimination survived reversal training of the feature to a greater extent in the non-inhibitors than in the inhibitors and this result was supported by Bayesian analyses. We propose that the fundamental distinction between inhibitors and non-inhibitors is based on a tendency to utilize first-order (direct associations) or second-order (occasion-setting) strategies when faced with ambiguous information and that this classification is a stable individual differences attribute.

Resurgence in humans: Reducing relapse by increasing generalization between treatment and testing

Resurgence is the increase in performance of an extinguished instrumental (operant) response that accompanies the extinction of a response that has been reinforced to replace it. Resurgence may involve processes that are relevant for understanding relapse in applied and clinical settings. While resurgence is known to be a robust phenomenon in human operant extinction, the processes that control it remain unclear. Here we asked whether human resurgence is controlled by processes that are similar to those that have been identified in animals by asking whether two methods that reduce resurgence in animals also reduce it in humans. Participants first learned to make an operant response (R1) for a tangible food reinforcer (O1). In a second phase (Phase 2), R1 was extinguished while a second response (R2) was introduced and reinforced with a virtual monetary reward (USD $0.10 coins; O2). In a test phase, extinction was then introduced for R2 and resurgence of R1 was assessed. In Experiment 1, resurgence that occurred after the treatment just described was attenuated if there had been periodic exposure to R2 extinction during the treatment phase (Phase 2). In Experiment 2, resurgence was prevented when O2, but not O1, was presented noncontingently during the test. The results are among the first to suggest a mechanism underlying resurgence in humans, namely, renewal caused by contextual change. They also provide initial evidence to suggest that resurgence may be the result of common processes in animals and humans. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Renewal of extinguished behavior in pigeons (Columba livia) does not require memory consolidation of acquisition or extinction in a free-operant appetitive conditioning paradigm

Extinction learning is a fundamental capacity for adaptive and flexible behavior. As extinguished conditioned responding is prone to relapse under certain conditions, the necessity of memory consolidation for recovery phenomena to occur has been highlighted recently. Several studies have demonstrated that both acquisition and extinction training need to be properly consolidated for a relapse of the original acquired memory trace to occur. Does this imply that extinguished responses cannot relapse before memory consolidation? To answer this question, we investigated the renewal effect subsequent to an immediate or a delayed (24 h) extinction in a discriminative operant conditioning paradigm. In three different experiments, we could show (1) that acquisition learning does not need to be long-term consolidated for the occurrence of renewal, (2) that the offset of extinction training is a reliable marker for extinction recall in a free-operant extinction learning paradigm where organisms undergo consecutive acquisition training, extinction training as well as testing of conditioned responding and (3), that immediate and long-term consolidated renewal do not demonstrate any qualitative difference in terms of the behavioral output. Our results indicate on the behavioral level that the inhibitory nature of extinction is already present in free-operant learning paradigms and that it does not seem to be affected by the absence of long-term memory consolidation.

Role of the Bed Nucleus of the Stria Terminalis in PTSD: Insights From Preclinical Models

Post-traumatic stress disorder (PTSD) afflicts approximately 8% of the United States population and represents a significant public health burden, but the underlying neural mechanisms of this and other anxiety- and stressor-related disorders are largely unknown. Within the last few decades, several preclinical models of PSTD have been developed to help elucidate the mechanisms underlying dysregulated fear states. One brain area that has emerged as a critical mediator of stress-related behavioral processing in both clinical and laboratory settings is the bed nucleus of the stria terminalis (BNST). The BNST is interconnected with essential emotional processing regions, including prefrontal cortex, hippocampus and amygdala. It is activated by stressor exposure and undergoes neurochemical and morphological alterations as a result of stressor exposure. Stress-related neuro-peptides including corticotropin-releasing factor (CRF) and pituitary adenylate cyclase activating peptide (PACAP) are also abundant in the BNST, further implicating an involvement of BNST in stress responses. Behaviorally, the BNST is critical for acquisition and expression of fear and is well positioned to regulate fear relapse after periods of extinction. Here, we consider the role of the BNST in stress and memory processes in the context of preclinical models of PTSD.

Loss of Hierarchical Control by Occasion Setters Following Lesions of the Prelimbic and Infralimbic Medial Prefrontal Cortex in Rats

Recent work suggests complementary roles of the prelimbic and infralimbic regions of the rat medial prefrontal cortex in cognitive control processes, with the prelimbic cortex implicated in top-down modulation of associations and the infralimbic cortex playing a role in the inhibition of inappropriate responses. Following selective lesions made to prelimbic or infralimbic regions (or control sham-surgery) rats received simultaneous training on Pavlovian feature negative (A+, XA-) and feature positive (B-, YB+) discriminations designed to lead to hierarchical occasion-setting control by the features (X, Y) over their respective targets (A, B). Evidence for hierarchical control was assessed in a transfer test in which features and targets were swapped (YA, XB). All groups were able to learn the feature negative and feature positive discriminations. Whilst sham-lesioned animals showed no transfer of control by features to novel targets (a hallmark of hierarchical control), rats with lesions of prelimbic or infralimbic regions showed evidence of transfer from the positive feature (Y) to the negative target (A), and from the negative feature (X) to the positive target (B; although this only achieved significance in infralimbic-lesioned animals). These data indicate that damage to either of these regions disrupts hierarchical occasion-setting control, extending our knowledge of their role in cognitive control to encompass flexible behaviours dictated by discrete cues.

No Time-Dependent Effects of Psychosocial Stress on Fear Contextualization and Generalization: A Randomized-Controlled Study With Healthy Participants

The formation of context-dependent fear memories (fear contextualization) can aid the recognition of danger in new, similar, situations. Overgeneralization of fear is often seen as hallmark of anxiety and trauma-related disorders. In this randomized-controlled study, we investigated whether exposure to a psychosocial stressor influences retention of fear contextualization and generalization in a time-dependent manner. The Trier Social Stress Test was used to induce psychosocial stress. Healthy male participants (n = 117) were randomly divided into three experimental groups that were subjected to the acquisition phase of the Fear Generalization Task: (1) without stress, (2) immediately after acute stress, or (3) 2 h after acute stress. In this task, a male with neutral facial expression (conditioned stimuli) was depicted in two different contexts that modulated the conditioned stimuli-unconditioned stimuli (=shock) association (threat, safe). Salivary alpha-amylase and cortisol levels were measured throughout the experiment. After a 24-h delay, context-dependency of fear memory was investigated with an unannounced memory test consisting of the threat and safe contexts alternated with a novel context (the generalization context). Multilevel analyses revealed that participants showed increased fear-potentiated startle responses to the conditioned stimuli in the threat compared to the safe context, at the end of the acquisition phase, indicating adequate fear contextualization. Directly after acquisition, there were no time-dependent effects of psychosocial stress on fear contextualization. Context-dependency of fear memories was retained 24 h later, as fear-potentiated startle responding was modulated by context (threat > safe or novel). At that time, the context-dependency of fear memories was also not influenced by the early or late effects of the endogenous stress response during acquisition. These results with experimental stress deviate in some aspects from those earlier obtained with exogenous hydrocortisone administration, suggesting a distinct role for stress mediators other than cortisol.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Common neurocircuitry mediating drug and fear relapse in preclinical models

BACKGROUND

Comorbidity of anxiety disorders, stressor- and trauma-related disorders, and substance use disorders is extremely common. Moreover, therapies that reduce pathological fear and anxiety on the one hand, and drug-seeking on the other, often prove short-lived and are susceptible to relapse. Considerable advances have been made in the study of the neurobiology of both aversive and appetitive extinction, and this work reveals shared neural circuits that contribute to both the suppression and relapse of conditioned responses associated with trauma or drug use.

OBJECTIVES

The goal of this review is to identify common neural circuits and mechanisms underlying relapse across domains of addiction biology and aversive learning in preclinical animal models. We focus primarily on neural circuits engaged during the expression of relapse.

KEY FINDINGS

After extinction, brain circuits involving the medial prefrontal cortex and hippocampus come to regulate the expression of conditioned responses by the amygdala, bed nucleus of the stria terminalis, and nucleus accumbens. During relapse, hippocampal projections to the prefrontal cortex inhibit the retrieval of extinction memories resulting in a loss of inhibitory control over fear- and drug-associated conditional responding.

CONCLUSIONS

The overlapping brain systems for both fear and drug memories may explain the co-occurrence of fear and drug-seeking behaviors.

Learning to learn in conditioning and extinction in humans

Learning to Learn (LTL) is the transfer of learning, separate from stimulus generalization, that appears across tasks that share a similar structure. Three experiments examined this phenomenon in both conditioning and extinction learning in humans. The latter effect is of special interest given the failures in the literature to obtain transfer of extinction between stimuli. Conditioning and extinction with one stimulus increased the rate of conditioning and, surprisingly, extinction of a different stimulus (Experiment 1). The effects appeared in the absence of physical generalization. The transfer of extinction was not enhanced by conditions that increased the chances of a mediated extinction effect (Experiment 2). Finally, Experiment 3 ruled out three possible sources for the effect in extinction: a common unconditioned-stimulus representation, a common reinforcement history, and within-stimuli associations. Overall, the findings are consistent with the idea that LTL is an emergent (non-immediate) form of mediated generalization that is dependent upon memory structures retrieved by trial outcomes. The over- or under-prediction of the outcome on the first trial with a new task might retrieve prior episodes associated with similar prediction errors promoting transfer.

d-Cycloserine facilitates fear extinction in adolescent rats and differentially affects medial and lateral prefrontal cortex activation

Adolescent humans and rodents are impaired in extinguishing learned fear relative to younger and older groups. This impairment could be due to differences in recruitment of medial prefrontal cortex (PFC), orbitofrontal cortex (OFC), or amygdala during extinction. For example, unlike juveniles and adults, adolescent rats do not express extinction-induced increases in phosphorylated mitogen activated protein kinase (pMAPK), a marker of synaptic plasticity, in the medial PFC. The NMDA receptor partial agonist d-cycloserine (DCS) improves extinction retention in adolescent rats. We investigated whether DCS affected recruitment of the PFC and amygdala during extinction by measuring pMAPK-immunoreactive (IR) neurons. Adolescent rats were trained to fear a conditioned stimulus in one context followed by extinction in a second context or equivalent context exposure only (i.e., no extinction). DCS (15 mg/kg, s.c.) or saline was administered systemically immediately after extinction training or context exposure. DCS enhanced extinction learning and this was associated with increased activation of the MAPK signaling pathway in the OFC after extinction training and increased activation in the medial PFC and amygdala at extinction retention. These findings suggest that DCS improves extinction learning in adolescents because it augments OFC contributions to extinction learning, enabling better medial prefrontal contributions to extinction retention.

Sex differences in extinction to negative stimuli: Event-related brain potentials

There are controversial observations regarding whether females have a longer time to extinction than men, which may be related to different levels of conditioning acquisition and/or the influence of the menstrual cycle. We explored the electrophysiological evidence of sex differences in extinction.In this study, females in the luteal phase and menstrual phase were examined for event-related potential (ERP) and evidence of attention allocation in the conditioning model using electroencephalogram recordings. A group of male participants was also included and compared.Women in the luteal phase had a higher difference waveform of P3 amplitude to conditioned stimulus (CS) in the extinction phase than women in the menstrual phase and men. There was a shorter latency of P3 to CS+ in men than in women in the extinction phase, suggesting that men react faster than women to unconditioned stimulus (US) expectation. Our study revealed that women in the luteal phase allocated more attentive resources to the expectation of a US. In contrast, men displayed faster expectation of the extinguished US than women. Our results support the superiority of ERP technology in documenting the neural mechanism of the extinction process.

Assessing the blocking of occasion setting

An occasion setter (OS) is a stimulus or context with the capacity to disambiguate an ambiguous conditioned stimulus (CS). Previous research has shown that OSs share some features with regular Pavlovian CSs. Amongst them, research has shown that OSs are subject to blocking; that is, a new OS exerts reduced behavioral control after training in compound with a previously established OS. Of additional interest, in Pavlovian blocking, it has been reported that a blocked CS comes to elicit conditioned responding after the extinction of the blocking CS. This is an example of retrospective revaluation, a family of phenomena in which the response to a specific stimulus is modified by training a related cue. Here, three experiments sought to extend the analogies between OS and Pavlovian conditioning by examining the blocking of OSs and its retrospective revaluation. In all experiments, an OS was established by pairing a CS with food in the presence of the OS, but not in its absence (i.e., positive OS). Blocking was then trained by presenting the OS in compound with a novel OS. Experiment 1 showed blocking of the second OS, but direct exposure to the blocking OS did not enhance responding to the second OS. Experiment 2 replicated the blocking effect but subsequent training of the blocking OS with a reversed contingency showed no retrospective revaluation. Experiment 3 examined whether blocking of the OS occurred with a novel CS during the compound phase. In this experiment blocking was again observed, but only when subjects were tested with the original CS. These results are discussed focusing on the underlying links at work in occasion setting.

The neuroscience of cognitive-motivational styles: Sign- and goal-trackers as animal models

Cognitive-motivational styles describe predominant patterns of processing or biases that broadly influence human cognition and performance. Here we focus on the impact of cognitive-motivational styles on the response to cues predicting the availability of food or addictive drugs. An individual may preferably conduct an analysis of the motivational significance of reward cues, with the result that such cues per se are perceived as rewarding and worth approaching and working for. Alternatively, a propensity for a "cold" analysis of the behavioral utility of a reward cue may yield search behavior for food or drugs but not involve cue approach. Animal models for studying the neuronal mechanisms mediating such styles have originated from research concerning behavioral indices that predict differential vulnerability to addiction-like behaviors. Rats classified as sign- or goal-trackers (STs, GTs) were found to have opposed attentional biases (bottom-up or cue-driven attention vs. top-down or goal-driven attentional control) that are mediated primarily via relatively unresponsive versus elevated levels of cholinergic neuromodulation in the cortex. The capacity for cholinergic neuromodulation in STs is limited by a neuronal choline transporter (CHT) that fails to support increases in cholinergic activity. Moreover, in contrast to STs, the frontal dopamine system in GTs does not respond to the presence of drug cues and, thus, biases against cue-oriented behavior. The opponent cognitive-motivational styles that are indexed by sign- and goal-tracking bestow different cognitive-behavioral vulnerabilities that may contribute to the manifestation of a wide range of neuropsychiatric disorders. (PsycINFO Database Record

The Role of Working Memory for Cognitive Control in Anorexia Nervosa versus Substance Use Disorder

Prefrontal cortex executive functions, such as working memory (WM) interact with limbic processes to foster impulse control. Such an interaction is referred to in a growing body of publications by terms such as cognitive control, cognitive inhibition, affect regulation, self-regulation, top-down control, and cognitive–emotion interaction. The rising trend of research into cognitive control of impulsivity, using various related terms reflects the importance of research into impulse control, as failure to employ cognitions optimally may eventually result in mental disorder. Against this background, we take a novel approach using an impulse control spectrum model – where anorexia nervosa (AN) and substance use disorder (SUD) are at opposite extremes – to examine the role of WM for cognitive control. With this aim, we first summarize WM processes in the healthy brain in order to frame a systematic review of the neuropsychological, neural and genetic findings of AN and SUD. In our systematic review of WM/cognitive control, we found n = 15 studies of AN with a total of n = 582 AN and n = 365 HC participants; and n = 93 studies of SUD with n = 9106 SUD and n = 3028 HC participants. In particular, we consider how WM load/capacity may support the neural process of excessive epistemic foraging (cognitive sampling of the environment to test predictions about the world) in AN that reduces distraction from salient stimuli. We also consider the link between WM and cognitive control in people with SUD who are prone to ‘jumping to conclusions’ and reduced epistemic foraging. Finally, in light of our review, we consider WM training as a novel research tool and an adjunct to enhance treatment that improves cognitive control of impulsivity.

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Diverse Roads to Relapse: A Discriminative Cue Signaling Cocaine Availability Is More Effective in Renewing Cocaine Seeking in Goal Trackers Than Sign Trackers and Depends on Basal Forebrain Cholinergic Activity

Stimuli associated with taking drugs are notorious instigators of relapse. There is, however, considerable variation in the motivational properties of such stimuli, both as a function of the individual and the nature of the stimulus. The behavior of some individuals (sign trackers, STs) is especially influenced by cues paired with reward delivery, perhaps because they are prone to process information via dopamine-dependent, cue-driven, incentive salience systems. Other individuals (goal trackers, GTs) are better able to incorporate higher-order contextual information, perhaps because of better executive/attentional control over behavior, which requires frontal cortical cholinergic activity. We hypothesized, therefore, that a cue that "sets the occasion" for drug taking (a discriminative stimulus, DS) would reinstate cocaine seeking more readily in GTs than STs and that this would require intact cholinergic neurotransmission. To test this, male STs and GTs were trained to self-administer cocaine using an intermittent access schedule with periods of cocaine availability and unavailability signaled by a DS+ and a DS-, respectively. Thereafter, half of the rats received an immunotoxic lesion that destroyed 40-50% of basal forebrain cholinergic neurons and later, after extinction training, were tested for the ability of noncontingent presentations of the DS+ to reinstate cocaine seeking behavior. The DS+ was much more effective in reinstating cocaine seeking in GTs than STs and this effect was abolished by cholinergic losses despite the fact that all rats continued to orient to the DS+ We conclude that vulnerability to relapse involves interactions between individual cognitive-motivational biases and the form of the drug cue encountered.SIGNIFICANCE STATEMENT The most predictable outcome of a diagnosis of addiction is a high chance for relapse. When addicts encounter cues previously associated with drug, their attention may be unduly attracted to such cues and these cues can evoke motivational states that instigate and maintain drug-seeking behavior. Although sign-tracking rats were previously demonstrated to exhibit greater relapse vulnerability to Pavlovian drug cues paired with drug delivery, here, we demonstrate that their counterparts, the goal trackers, are more vulnerable if the drug cue acts to signal drug availability and that the forebrain cholinergic system mediates such vulnerability. Given the importance of contextual cues for triggering relapse and the human cognitive-cholinergic capacity for the processing of such cues, goal trackers model essential aspects of relapse vulnerability.

Coal Is Not Black, Snow Is Not White, Food Is Not a Reinforcer: The Roles of Affordances and Dispositions in the Analysis of Behavior

Reinforcers comprise sequences of actions in context. Just as the white of snow and black of coal depend on the interaction of an organism's visual system and the reflectances in its surrounds, reinforcers depend on an organism's motivational state and the affordances-possibilities for perception and action-in its surrounds. Reinforcers are not intrinsic to things but are a relation between what the thing affords, its context, the organism, and his or her history as capitulated in their current state. Reinforcers and other affordances are potentialities rather than intrinsic features. Realizing those potentialities requires motivational operations and stimulus contexts that change the state of the organism-they change its disposition to make the desired response. An expansion of the three-term contingency is suggested in order to help keep us mindful of the importance of behavioral systems, states, emotions, and dispositions in our research programs.