Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

A parametric manipulation and meta-analysis of target-response punishment on resurgence

Resurgence can be defined as increases in previously reinforced and subsequently extinguished target responding when conditions for an alternative response worsen. Worsening of alternative conditions, such as extinction, has been linked to relapse of clinically relevant behavior. Preclinical researchers have evaluated whether punishing target responses while differentially reinforcing an alternative response could reduce resurgence when conditions are worsened with extinction, with mixed results. In the present investigation, we systematically replicated this line of research with human participants recruited via crowdsourcing, using response cost as punishment. During Phase 1, we reinforced target responses with 100 points per delivery, exchangeable for money. During Phase 2, we reinforced alternative responses, discontinued point reinforcement for target responses, and parametrically manipulated across groups the magnitude of point loss (1, 100, 320, or 1,000 points) contingent on target responses. During Phase 3, we tested for resurgence by extinguishing target and alternative responses. Added punishment systematically decreased target responding during Phase 2 but did not influence resurgence during Phase 3. With a meta-analysis, we compared our findings with existing research examining a range of punishers and species. The results of the meta-analysis comport with the present findings, suggesting that the inclusion of punishment reduces target responding during DRA but, overall, has no systematic effects on resurgence.

Perceived stress and renewal: The effects of long-term stress on the renewal effect

Two online experiments evaluated the relationship between long-term stress, as measured with the Perceived Stress Scale-10, and the Renewal Effect. In the first experiment renewal was assessed with a behavioral suppression task in a science-fiction based video game. Participants learned to suppress mouse clicking during a signal for an upcoming attack to avoid losing points. The signal was first paired with an attack in Context A and extinguished in Context B and tested back in Context A. The contexts were different space galaxies where the gameplay took place. Experiment 2 used a food/illness predictive-learning paradigm. Two food items were paired with stomachache in one restaurant (A) and extinguished in Context B prior to testing in both contexts without feedback. Positive correlations were obtained between renewal and stress in each experiment. Unlike acute stress (Drexler et al., 2017), long term stress was associated with greater renewal. The effects of stress, both chronic and punctual, on renewal are discussed.

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Habit and persistence

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

Neuroplasticity of the extended amygdala in opioid withdrawal and prolonged opioid abstinence

Opioid use disorder is characterized by excessive use of opioids, inability to control its use, a withdrawal syndrome upon discontinuation of opioids, and long-term likelihood of relapse. The behavioral stages of opioid addiction correspond with affective experiences that characterize the opponent process view of motivation. In this framework, active involvement is accompanied by positive affective experiences which gives rise to "reward craving," whereas the opponent process, abstinence, is associated with the negative affective experiences that produce "relief craving." Relief craving develops along with a hypersensitization to the negatively reinforcing aspects of withdrawal during abstinence from opioids. These negative affective experiences are hypothesized to stem from neuroadaptations to a network of affective processing called the "extended amygdala." This negative valence network includes the three core structures of the central nucleus of the amygdala (CeA), the bed nucleus of the stria terminalis (BNST), and the nucleus accumbens shell (NAc shell), in addition to major inputs from the basolateral amygdala (BLA). To better understand the major components of this system, we have reviewed their functions, inputs and outputs, along with the associated neural plasticity in animal models of opioid withdrawal. These models demonstrate the somatic, motivational, affective, and learning related models of opioid withdrawal and abstinence. Neuroadaptations in these stress and motivational systems are accompanied by negative affective and aversive experiences that commonly give rise to relapse. CeA neuroplasticity accounts for many of the aversive and fear-related effects of opioid withdrawal via glutamatergic plasticity and changes to corticotrophin-releasing factor (CRF)-containing neurons. Neuroadaptations in BNST pre-and post-synaptic GABA-containing neurons, as well as their noradrenergic modulation, may be responsible for a variety of aversive affective experiences and maladaptive behaviors. Opioid withdrawal yields a hypodopaminergic and amotivational state and results in neuroadaptive increases in excitability of the NAc shell, both of which are associated with increased vulnerability to relapse. Finally, BLA transmission to hippocampal and cortical regions impacts the perception of conditioned aversive effects of opioid withdrawal by higher executive systems. The prevention or reversal of these varied neuroadaptations in the extended amygdala during opioid withdrawal could lead to promising new interventions for this life-threatening condition.

A comparison of renewal, spontaneous recovery, and reacquisition after punishment and extinction

Punishment and extinction are both effective methods of reducing instrumental responding and may involve similar learning mechanisms. To characterize the similarities and differences between them, we examined three well-established recovery or "relapse" effects -renewal, spontaneous recovery, and reacquisition - following either punishment or extinction of an instrumental response. In Experiment 1a, both punished and extinguished responses renewed to similar degrees following a context change at test (ABA renewal). In Experiment 1b, responding spontaneously recovered to similar degrees following punishment or extinction. In Experiment 2, responding was rapidly reacquired when the response was reinforced again following extinction but not following punishment, as predicted by the idea that the reinforcer delivered in reacquisition is part of the context of punishment, but not extinction. The results collectively suggest that both punishment and extinction produce similar context-dependent retroactive interference effects. More broadly, they also suggest that punished and extinguished responses may be equally likely to return following a change of context despite the intuition that punishment might provide a more extreme and effective means of suppressing behavior. To our knowledge, this is the first direct behavioral comparison of response recovery after punishment and extinction within individual experiments.

Acute stress persistently alters instrumental motivation without affecting appetitive Pavlovian conditioning, extinction, or contextual renewal

In two experiments, we adapted the stress-enhanced fear learning approach to evaluate the persistent effects of acute stress on appetitive learning and motivation in adult male Long Evans rats. In Experiment 1, we found that exposure to a battery of footshocks in one context had no effect on the acquisition, extinction, or contextual renewal of an appetitive Pavlovian discrimination in different contexts. However, when rats were subsequently trained to respond on a progressive ratio instrumental schedule, rats with a history of shock showed lower response rates and progressive ratio break points. Extinction of the shock-associated context had little effect on progressive ratio responding. In Experiment 2, we replicated this instrumental responding deficit with a continuous reinforcement schedule when the Pavlovian phases did not intervene in the time between shock and instrumental testing. Our findings here demonstrate that highly stressful acute experiences produce long-lasting deficits in instrumental motivation for food in male rats.

Prior experience modifies acquisition trajectories via response-strategy sampling

Few studies have considered how signal detection parameters evolve during acquisition periods. We addressed this gap by training mice with differential prior experience in a conditional discrimination, auditory signal detection task. Naïve mice, mice given separate experience with each of the later correct choice options (Correct Choice Response Transfer, CCRT), and mice experienced in conditional discriminations (Conditional Discrimination Transfer, CDT) were trained to detect the presence or absence of a tone in white noise. We analyzed data assuming a two-period model of acquisition: a pre-solution and solution period (Heinemann EG (1983) in The Presolution period and the detection of statistical associations. In: Quantitative analyses of behavior: discrimination processes, vol. 4, pp. 21-36). Ballinger. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.536.1978andrep=rep1andtype=pdf ). The pre-solution period was characterized by a selective sampling of biased response strategies until adoption of a conditional responding strategy in the solution period. Correspondingly, discriminability remained low until the solution period; criterion took excursions reflecting response-strategy sampling. Prior experience affected the length and composition of the pre-solution period. Whereas CCRT and CDT mice had shorter pre-solution periods than naïve mice, CDT and Naïve mice developed substantial criterion biases and acquired asymptotic discriminability faster than CCRT mice. To explain these data, we propose a learning model in which mice selectively sample and test different response-strategies and corresponding task structures until they exit the pre-solution period. Upon exit, mice adopt the conditional responding strategy and task structure, with action values updated via inference and generalization from the other task structures. Simulations of representative mouse data illustrate the viability of this model.

Partial reinforcement extinction and omission effects in the elimination and recovery of discriminated operant behavior

Three experiments explored how training reinforcement schedules and context influence the elimination and recovery of human operant behavior. In Experiment 1, participants learned a discriminated operant response in Context A before the response was eliminated with extinction in Context B. They then received a final test in each context. Groups were trained with a discriminative stimulus that predicted a reinforced response on either every trial (continuous reinforcement [CRF]) or some of the trials (partial reinforcement [PRF]). Extinction was slower following PRF training (a partial reinforcement extinction effect [PREE]) and extinguished responding increased when tested in Context A ("ABA" renewal). Experiment 2 further confirmed the PREE was obtained equally whether extinction occurred in the training context (Context A) or a new context (Context B) which is consistent with trial-based accounts of the PREE. Experiment 3 used the same design as Experiment 1 to evaluate the influence of training reinforcement on response elimination with an omission contingency. Across the omission training phase in Context B, the decrease in responding occurred more slowly in the PRF-trained group in comparison to the CRF-trained group, perhaps the first demonstration of what might be termed a PRF omission effect. Again, ABA renewal was observed in Context A. Training reinforcement schedule therefore had a similar influence on response elimination with extinction and omission. Elimination and recovery of human instrumental behavior, with extinction or omission, are influenced by training reinforcement schedule and context. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Prelimbic cortex inactivation prevents ABA renewal based on satiety state

We have previously shown that the rat prelimbic cortex (PL) is necessary for contexts to promote the performance of instrumental behaviors that have been learned in them, whether the context is physical (operant chamber) or behavioral (recent performance of a behavior that has historically preceded the target in a behavior chain). In the present experiment, we investigated the role of the PL in satiety level as an interoceptive acquisition context. Rats were trained to lever-press for sweet/fat pellets while sated (22 hrs continuous food access) followed by the extinction of the response while hungry (22 hrs food deprived). Pharmacological inactivation of the PL (with baclofen/muscimol infusion) attenuated renewal of the response that occurred upon a return to the sated context. In contrast, animals that received a vehicle (saline) infusion showed renewal of the previously extinguished response. These results support the hypothesis that the PL monitors the relevant contextual elements (physical, behavioral, or satiety state) associated with reinforcement of a response and promotes the subsequent performance of that response in their presence.

The impact of prolonging extinction on the ABC "super renewal" of instrumental responses in rats

Two free operant conditioning experiments with rats examined the impact of conducting a large amount of extinction training on situations that enhance the ABC renewal effect (ABC super renewal). In Experiment 1, ABC renewal was strengthened by conducting acquisition in multiple contexts. All rats were trained to press a lever for food. One group was trained in one context, while the other two groups were trained in three contexts. Then, all rats received extinction in context B. For two groups this phase lasted 4 sessions, whereas it lasted 36 sessions for the other group. In Experiment 2, ABC renewal was strengthened by using a large number of acquisition sessions. Rats were trained to perform an operant response to obtain food in context A. One group received a moderate amount of training, while the rest of the rats received a larger number of acquisition sessions. Responses underwent extinction in context B. Two groups received 4 sessions, while 36 extinction sessions were used for the remaining group. In both experiments, rats were tested in context B (extinction context) and C (renewal context). Greater ABC renewal occurred both when acquisition training was conducted in multiple contexts (Experiment 1) and by increasing the amount of acquisition training (Experiment 2). Nevertheless, we found that conducting a large number of extinction sessions reduced ABC super renewal in Experiment 1 only.

Relapse and its mitigation: Toward behavioral inoculation

Relapse following the successful treatment of problem behavior can increase the likelihood of injury and the need for more intensive care. Current research offers some predictions of how treatment procedures may contribute to relapse, and conversely, how the risk of relapse can be mitigated. This review describes relapse-mitigation procedures with varying levels of support, the quantitative models that have influenced the research on relapse mitigation, different experimental methods for measuring relapse mitigation, and directions for future research. We propose that by viewing the implementation of relapse-mitigation procedures as a means of producing behavioral inoculation, clinicians are placed in the proactive and intentional role of exposing their client's behavior to an array of reinforcement and stimulus conditions during treatment with the goal of decreasing the detrimental impact of future treatment challenges.

Acute food deprivation-induced relapse to heroin seeking after short and long punishment-imposed abstinence in male rats

RATIONAL

Stress is a major trigger for drug relapse in humans and animal models, even after prolonged abstinence. However, animal models for stress-induced relapse were criticized for the lack of predictive and face validity.

OBJECTIVES

Here we investigated the effect of acute food deprivation stress in a novel stress-induced relapse model using voluntary, punishment-imposed abstinence from heroin. We also performed a detailed characterization of the development of punishment-imposed abstinence.

METHODS

Male rats were trained to self-administered heroin (0.1 mg/kg/infusion) for 2 weeks, using the seeking-taking chained schedule. Pressing the 'seeking' lever led to the insertion of the 'taking' lever and pressing the take lever resulted in heroin infusion. Following self-administration training, rats were exposed to 8 or 21 days of heroin-seeking punishment. During punishment, 30% of the completed seek links resulted in a mild escalating footshock instead of take lever presentation. Next, rats were tested for heroin seeking under extinction conditions after 24 h of food deprivation and sated conditions.

RESULTS

Probabilistic punishment of seeking lever responses resulted in gradual suppression of heroin seeking and taking. Exposure to food-deprivation stress induced a robust relapse to heroin seeking after short and long punishment-imposed abstinence periods, without significant effects of time, i.e., no incubation of heroin seeking. Individual differences were observed in the development of punishment-induced abstinence and stress-induced relapse.

CONCLUSIONS

These results suggest that stress is a reliable trigger to relapse even after a prolonged period of punishment-induced, voluntary abstinence.

Learning to stop responding

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

A Comprehensive Systematic Review of Procedures and Analyses Used in Basic and Preclinical Studies of Resurgence, 1970-2020

Resurgence is the return of a previously reinforced response as conditions worsen for an alternative response, such as the introduction of extinction, reductions in reinforcement, or punishment. As a procedure, resurgence has been used to model behavioral treatments and understand behavioral processes contributing both to relapse of problem behavior and flexibility during problem-solving. Identifying existing procedural and analytic methods arranged in basic/preclinical research could be used by basic and preclinical researchers to develop novel approaches to study resurgence, whereas translational and clinical researchers could identify potential approaches to combating relapse during behavioral interventions. Despite the study of resurgence for over half a century, there have been no systematic reviews of the basic/preclinical research on resurgence. To characterize the procedural and analytic methods used in basic/preclinical research on resurgence, we performed a systematic review consistent with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We identified 120 articles consisting of 200 experiments that presented novel empirical research, examined operant behavior, and included standard elements of a resurgence procedure. We reported prevalence and trends in over 60 categories, including participant characteristics (e.g., species, sample size, disability), designs (e.g., single subject, group), procedural characteristics (e.g., responses, reinforcer types, control conditions), criteria defining resurgence (e.g., single test, multiple tests, relative to control), and analytic strategies (e.g., inferential statistics, quantitative analysis, visual inspection). We make some recommendations for future basic, preclinical, and clinical research based on our findings of this expanding literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40614-022-00361-y.

Food Restriction in Anorexia Nervosa in the Light of Modern Learning Theory: A Narrative Review

Improvements in the clinical management of anorexia nervosa (AN) are urgently needed. To do so, the search for innovative approaches continues at laboratory and clinical levels to translate new findings into more effective treatments. In this sense, modern learning theory provides a unifying framework that connects concepts, methodologies and data from preclinical and clinical research to inspire novel interventions in the field of psychopathology in general, and of disordered eating in particular. Indeed, learning is thought to be a crucial factor in the development/regulation of normal and pathological eating behaviour. Thus, the present review not only tries to provide a comprehensive overview of modern learning research in the field of AN, but also follows a transdiagnostic perspective to offer testable explanations for the origin and maintenance of pathological food rejection. This narrative review was informed by a systematic search of research papers in the electronic databases PsycInfo, Scopus and Web of Science following PRISMA methodology. By considering the number and type of associations (Pavlovian, goal-directed or habitual) and the affective nature of conditioning processes (appetitive versus aversive), this approach can explain many features of AN, including why some patients restrict food intake to the point of life-threatening starvation and others restrict calorie intake to lose weight and binge on a regular basis. Nonetheless, it is striking how little impact modern learning theory has had on the current AN research agenda and practice.

Stress effects on memory retrieval of aversive and appetitive instrumental counterconditioning in men

Extinction training creates a second inhibitory memory trace and effectively reduces conditioned responding. However, acute stress inhibits the retrieval of this extinction memory trace. It is not known whether this also applies to other forms of associative learning such as instrumental counterconditioning, where previously learned associations are reversed and paired with the opposite valence. Therefore, the current preregistered study investigates whether stress decreases the retrieval of instrumental counterconditioning memories with aversive and appetitive consequences. Fifty-two healthy men were randomly assigned to either a stress or control group and took part in a two-day instrumental learning paradigm. During a first phase, participants learned that pressing specific buttons in response to the presentation of four neutral stimuli either leads to gaining or losing money. During a second phase, two stimuli reversed their contingencies (counterconditioning). One day later, participants were exposed to acute stress or a control condition prior to the same task, which no longer included feedback about gains or losses. Stressed participants showed more approach behavior towards appetitive and less avoidance behavior towards aversive stimuli as compared to non-stressed participants. Our findings indicate that stress effects on memory retrieval differ depending on the associative learning approach in men. These differences might be related to stress effects on decision making and different motivational systems involved.

Relationship between anatomical characteristics and personality traits in Lipizzan horses

We tested 35 Lipizzan horses older than 5 years, ridden and healthy in three behavioural tests (handling, fear-reaction, and target training test). Physiological (heart rate and heart rate variability) and anatomical measurements (120 head and body distances and angles) were collected to validate parameters that reliably inform on handling/cooperation, fear/exploration and trainability in horses. Utilizing a standard clustering methodology on the behavioural data, we identified four general types of responses and categorised an individual as intermediate, low fearful, horses with low cooperation or low trainability. We additionally analysed the head morphology following Tellington-Jones and Taylor recommendations and correlated the measurements with data from a horse personality questionnaire. Although allocation to a particular personality group was not associated with these two methods, these groups differed in six anatomical characteristics of head and body. Regardless of the group, our results indicated that shorter horses (<75.9 cm) with a wider muzzle (>10.5 cm) are trustworthy, less fearful and easier to handle and train. We also demonstrated that horses with stronger legs and a wider base of the head have a lower heart rate when exposed to the second trial of the handling test.

Relapse after incentivized choice treatment in humans: A laboratory model for studying behavior change

Behavior change interventions that incentivize desired behavior are highly effective for improving personal health, but difficult to maintain long term. Relapse is common and examining the mechanisms that contribute to relapse in experimental settings can identify processes relevant to substance abuse treatment. We developed a laboratory task that parallels a recent operant model of relapse after incentivized choice reported in the rodent laboratory. In two experiments, undergraduate participants first learned to make an operant response (keyboard button; R1) to earn a reinforcer consisting of an image of a preferred snack food (O1). In a second phase (Phase 2), R1 was still reinforced, but a new response (R2) was introduced and reinforced with a different reinforcer (a coin; O2). In a test phase, contingent incentives for R2 were removed (extinction) and relapse of R1 was assessed. Experiment 1 found that the O2 contingency suppressed R1 during Phase 2, and R1 relapsed rapidly in the test. Neither effect was consistently related to O2 value. Experiment 2 examined whether noncontingent presentations of O1 or O2 during the test could weaken relapse. Here, we found that noncontingent reinforcers did little to reduce or slow the increase in R1 responding. The present experiments highlight a laboratory approach to studying variables that may influence relapse after incentivized treatment. We identify and discuss areas for development to address differences between the present results and prior observations from animal and clinical studies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Brief Environmental Enrichment exposure enhances contextual-induced sucrose-seeking with and without memory reactivation in rats

Chronic Environmental Enrichment (EE) has been shown to prevent the relapse to addictive behaviours, such as drug-taking and -seeking. Recently, acute EE was shown to reduce cue-induced sucrose-seeking, but its effects on contextual (Cx)-induced sucrose-seeking is still unknown. Here we report the effects of brief EE exposure on Cx-induced sucrose-seeking with and without prior Cx-memory reactivation. Adult male Sprague-Dawley rats were trained to sucrose self-administration associated to a specific conditioning Cx (CxA), followed by a 7-day extinction in a different Cx (CxB). Afterwards, rats were exposed for 22 h to EE, and 1 h later to either i) Cx-induced sucrose-seeking (1 h, renewal without Cx-memory reactivation), ii) or two different Cx-memory reactivations: short (2-min) and long (15-min) CxA-retrieval session (Cx-Ret). In Cx-Ret experiments, CxA-induced sucrose-seeking test (1 h) was done after a subsequent 3-day extinction phase. The assessment of molecular markers of memory reactivation/reconsolidation, Zif-268 and rpS6P, was performed 2 h after Cx-Ret. Brief EE exposure enhanced Cx-induced sucrose-seeking without and with short but not long Cx-retrieval. Moreover, EE impaired discriminative responding at test prior to long, whereas improved it with or without short Cx-retrieval. Different changes in Zif-268 and rpS6P expression induced by short vs. long Cx-Ret were correlated to behavioural data, suggesting the occurrence of different memory processes affected by EE. Our data show that brief EE exposure may differently affect subsequent appetitive relapse depending on the modality of re-exposure to conditioned context. This finding suggests caution and further studies to understand the proper conditions for the use of EE against appetitive and addiction disorders.

Evaluating effects of context changes on resurgence in humans

Laboratory models of relapse provide methods for evaluating challenges to behavioral treatments with differential reinforcement of an alternative response (DRA). Resurgence occurs with the worsening of conditions of reinforcement for appropriate behavior and renewal occurs when transitioning out of a treatment context. Across five experiments, participants recruited via online crowdsourcing pressed onscreen buttons to earn points exchangeable for money and contexts sometimes changed through changes in the background image. Returning to the training context (ABA, Experiment 1) and transitioning to a novel context (ABC, Experiment 2) produced greater resurgence when removing alternative reinforcement in comparison with remaining in the treatment context (ABB). In contrast, we observed little difference in resurgence among AAA, ABB, and AAC context manipulations (Experiment 3) and ABA, ABC, and AAC context manipulations (Experiment 4). In Experiment 5, we evaluated relative contributions of the presence versus absence of context changes (ABA vs. ABB) in combination with or without the removal of alternative reinforcement. Both changing context and removing alternative reinforcement increased responding in isolation and the combination produced greater-than-additive effects. Overall, the present findings demonstrate a consistent effect of removing alternative reinforcement on relapse that, under certain conditions, can be enhanced by context change.

Infralimbic cortex functioning across motivated behaviors: Can the differences be reconciled?

The rodent infralimbic cortex (IL) is implicated in higher order executive functions such as reward seeking and flexible decision making. However, the precise nature of its role in these processes is unclear. Early evidence indicated that the IL promotes the extinction and ongoing inhibition of fear conditioning and cocaine seeking. However, evidence spanning other behavioral domains, such as natural reward seeking and habit-based learning, suggests a more nuanced understanding of IL function. As techniques have advanced and more studies have examined IL function, identifying a unifying explanation for its behavioral function has become increasingly difficult. Here, we discuss evidence of IL function across motivated behaviors, including associative learning, drug seeking, natural reward seeking, and goal-directed versus habit-based behaviors, and emphasize how context-specific encoding and heterogeneous IL neuronal populations may underlie seemingly conflicting findings in the literature. Together, the evidence suggests that a major IL function is to facilitate the encoding and updating of contingencies between cues and behaviors to guide subsequent behaviors.

Is probabilistic cuing of visual search an inflexible attentional habit? A meta-analytic review

In studies on probabilistic cuing of visual search, participants search for a target among several distractors and report some feature of the target. In a biased stage the target appears more frequently in one specific area of the search display. Eventually, participants become faster at finding the target in that rich region compared to the sparse region. In some experiments, this stage is followed by an unbiased stage, where the target is evenly located across all regions of the display. Despite this change in the spatial distribution of targets, search speed usually remains faster when the target is located in the previously rich region. The persistence of the bias even when it is no longer advantageous has been taken as evidence that this phenomenon is an attentional habit. The aim of this meta-analysis was to test whether the magnitude of probabilistic cuing decreases from the biased to the unbiased stage. A meta-analysis of 42 studies confirmed that probabilistic cuing during the unbiased stage was roughly half the size of cuing during the biased stage, and this decrease persisted even after correcting for publication bias. Thus, the evidence supporting the claim that probabilistic cuing is an attentional habit might not be as compelling as previously thought.

Prevalence of relapse of automatically maintained behavior resulting from context changes

Increases in behavior due to context changes are common and are known as instances of renewal. Clinically relevant examples from the literature highlighting renewal often include socially mediated problem behaviors. This report retrospectively analyzed data during context changes for individuals who engaged in problem behavior maintained by automatic reinforcement, to evaluate the prevalence of relapse. Problem behavior reemerged during changes both in the person implementing treatment (e.g., introducing a caregiver; 36%) and in the setting (e.g., introducing treatment in the home; 26%). Most prevalence studies report greatest relapse immediately following context changes but the highest level of relapse was observed after 5 sessions following person changes and no systematic pattern with setting changes. These patterns of relapse likely reflect differences in the function of settings and people relative to automatically reinforced behavior in the present study. Implications of relapse for treatments of problem behavior maintained by automatic reinforcement are discussed.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Topiramate-chitosan nanoparticles prevent morphine reinstatement with no memory impairment: Dopaminergic and glutamatergic molecular aspects in rats

Besides their clinical application, chronic misuse of opioids has often been associated to drug addiction due to their addictive properties, underlying neuroadaptations of AMPA glutamate-receptor-dependent synaptic plasticity. Topiramate (TPM), an AMPAR antagonist, has been used to treat psychostimulants addiction, despite its harmful effects on memory. This study aimed to evaluate the effects of a novel topiramate nanosystem on molecular changes related to morphine reinstatement. Rats were previously exposed to morphine in conditioned place preference (CPP) paradigm and treated with topiramate-chitosan nanoparticles (TPM-CS-NP) or non-encapsulated topiramate in solution (S-TPM) during CPP extinction; following memory performance evaluation, they were re-exposed to morphine reinstatement. While morphine-CPP extinction was comparable among all experimental groups, TPM-CS-NP treatment prevented morphine reinstatement, preserving memory performance, which was impaired by both morphine-conditioning and S-TPM treatment. In the NAc, morphine increased D1R, D2R, D3R, DAT, GluA1 and MOR immunoreactivity. It also increased D1R, DAT, GluA1 and MOR in the dorsal hippocampus. TPM-CS-NP treatment decreased D1R, D3R and GluA1 and increased DAT in the NAc, decreasing GluA1 and increasing D2 and DAT in the dorsal hippocampus. Taken together, we may infer that TPM-CS-NP treatment was able to prevent the morphine reinstatement without memory impairment. Therefore, TPM-CS-NP may be considered an innovative therapeutic tool due to its property to prevent opioid reinstatement because it acts modifying both dopaminergic and glutamatergic neurotransmission, which are commonly related to morphine addiction.

Altering Perceived Context: Transportation Cues Influence Novelty-Induced Context Exploration

Context is the milieu in which everything occurs. Many research studies consider context, or even explicitly manipulate it; yet it remains challenging to characterize. We know that a context surrounds and influences tasks; however, the boundaries of its influence are difficult to define. In behavioral science, context is often operationalized by the physical environment in which the experiment takes place, and the boundaries of the context are assumed to begin at the entrance to that of the room or apparatus. Experiences during transportation to the testing space have been shown to impact rodent behavior and memory, but transportation's relationship with novelty and physical environment is not fully understood. The current study explored how familiar vs. novel cues, both within a physical environment and preceding it, impact the perception of a context. We manipulated context on three levels: physical testing environment, object cues within that environment, and transportation cues preceding entrance to the testing environment. We found that novel transportation cues can change rats' perception of both familiar and novel contexts. The effects of transportation on perceived context may be affected by the length of the retention interval, testing environment, and behavioral range. These data suggest that context is a broad concept that includes cues across time and is sensitive to small differences in experience.

Habits and Goals in Human Behavior: Separate but Interacting Systems

People automatically repeat behaviors that were frequently rewarded in the past in a given context. Such repetition is commonly attributed to habit, or associations in memory between a context and a response. Once habits form, contexts directly activate the response in mind. An opposing view is that habitual behaviors depend on goals. However, we show that this view is challenged by the goal independence of habits across the fields of social and health psychology, behavioral neuroscience, animal learning, and computational modeling. It also is challenged by direct tests revealing that habits do not depend on implicit goals. Furthermore, we show that two features of habit memory-rapid activation of specific responses and resistance to change-explain the different conditions under which people act on habit versus persuing goals. Finally, we tested these features with a novel secondary analysis of action-slip data. We found that habitual responses are activated regardless of goals, but they can be performed in concert with goal pursuit.

Food reinforcement and habituation to food are processes related to initiation and cessation of eating

An individual bout of eating involves cues to start eating, as well as cues to terminate eating. One process that determines initiation of eating is food reinforcement. Foods with high reinforcing value are also likely to be consumed in greater quantities. Research suggests both cross-sectional and prospective relationships between food reinforcement and obesity, food reinforcement is positively related to energy intake, and energy intake mediates the relationship between food reinforcement and obesity. A process related to cessation of eating is habituation. Habituation is a general behavioral process that describes a reduction in physiological or affective response to a stimulus, or a reduction in the behavioral responding to obtain a stimulus. Repeated exposure to the same food during a meal can result in habituation to that food and a reduction in consumption. Habituation is also cross-sectionally and prospectively related to body weight, as people who habituate slower consume more in a meal and are more overweight. Research from our laboratory has shown that these two processes independently influence eating, as they can account for almost 60% of the variance in ad libitum intake. In addition, habituation phenotypes show reliable relationships with reinforcing value, such that people who habituate faster also find food less reinforcing. Developing a better understanding of cues to start and stop eating is fundamental to understanding how to modify eating behavior. An overview of research on food reinforcement, habituation and food intake for people with a range of weight status and without eating disorders is provided, and ideas about integrating these two processes that are related to initiation and termination of a bout of eating are discussed.

Effects of delay of reinforcement on ABA renewal

ABA renewal was assessed by decreasing response rates by means of extinction or a 12-s delay of reinforcement in one experiment with rats. In a first phase, rats were trained to lever press for water in Context A. Rats were exposed to a second phase in Context B, where lever-pressing decreased for all rats; for one group of rats, lever-pressing decreased by means of extinction, while lever-pressing decreased by means of a 12-s delay for the other group. During a third phase, both groups were once again exposed to Context A with no reinforcer deliveries for lever-pressing. Once rats were exposed to the latter phase, renewal was observed for both groups. Results suggest that delay of reinforcement can be used as an alternative procedure to help reduce behavior and mitigate relapse.

Evaluating Extinction, Renewal, and Resurgence of Operant Behavior in Humans with Amazon Mechanical Turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace providing researchers with the opportunity to collect behavioral data from remote participants at a low cost. Recent research demonstrated reliable extinction effects, as well as renewal and resurgence of button pressing with MTurk participants. To further examine the generality of these findings, we replicated and extended these methods across six experiments arranging reinforcement and extinction of a target button press. In contrast to previous findings, we did not observe as reliable of decreases in button pressing during extinction (1) after training with VR or VI schedules of reinforcement, (2) in the presence or absence of context changes, or (3) with an added response cost for button pressing. However, we found that that a 1-point response cost for all button presses facilitated extinction to a greater extent than the absence of response cost. Nevertheless, we observed ABA renewal of button pressing when changing background contexts across phases and resurgence when extinguishing presses on an alternative button. Our findings suggest that MTurk could be a viable platform from which to ask and address questions about extinction and relapse processes, but further procedural refinements will be necessary to improve the replicability of control by experimental contingencies.

Emergence of complex dynamics of choice due to repeated exposures to extinction learning

Extinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is not only essential for survival in a changing environment, but also plays a fundamental role in the treatment of pathological behaviors. During therapy and other forms of training involving extinction, subjects are typically exposed to several sessions with a similar structure. The effects of this repeated exposure are not well understood. Here, we studied the behavior of pigeons across several sessions of a discrimination-learning task in context A, extinction in context B, and a return to context A to test the context-dependent return of the learned responses (ABA renewal). By focusing on individual learning curves across animals, we uncovered a session-dependent variability of behavior: (1) during extinction, pigeons preferred the unrewarded alternative choice in one-third of the sessions, predominantly during the first one. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerged, and (3) the renewal effect decayed as sessions progressed. We show that the observed results can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the power of “simple” associative learning processes.

Genetic Depletion of BDNF Impairs Extinction Learning of a Spatial Appetitive Task in the Presence or Absence of the Acquisition Context

Brain derived neurotropic factor (BDNF) supports neuronal survival, growth, and differentiation and is involved in forms of hippocampus-dependent and independent learning, as well as hippocampus-dependent learning. Extinction learning comprises active inhibition of no-longer relevant learned information, in conjunction with a decreased response of a previously learned behavior. It is highly dependent on context, and evidence exists that it requires hippocampal activation. The participation of BDNF in memory processing is experience-dependent. For example, BDNF has been associated with synaptic plasticity needed for spatial learning, and it is involved in acquisition and extinction learning of fear conditioning. However, little is known about its role in spatial appetitive extinction learning. In this study, we evaluated to what extent BDNF contributes to spatial appetitive extinction learning in the presence (ABA) or absence (AAA) of exposure to the acquisition context. Daily training, of BDNF^+/--mice or their wildtype (WT) littermates, to reach acquisition criterion in a T-maze, resulted in a similar performance outcome. However, extinction learning was delayed in the AAA, and impaired in the ABA-paradigm compared to performance in WT littermates. Trial-by-trial learning analysis indicated differences in the integration of the context into extinction learning by BDNF^+/--mice compared to WT littermates. Taken together, these results support an important role for BDNF in processes that relate to information updating and retrieval that in turn are crucial for effective extinction learning.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Positive affective states can play the role of context to renew extinguished instrumental behavior in rats

In one experiment with rats, we examined whether positive affective states can serve as contexts in a between-subjects ABA renewal design using appetitive instrumental conditioning. Two groups of rats received training to press a lever for food where each acquisition session was preceded by administration of a tickling procedure (Context A) known to induce positive affective states. Then, lever pressing underwent extinction where rats received a pure handling treatment (Context B) before each session. During a final test session, we found stronger responding when the session was preceded by tickling (Group ABA) compared to handling (Group ABB), indicating an ABA renewal effect. Furthermore, test performance in Group ABB was not different from that in a third group where handling preceded acquisition sessions, and tickling extinction and test sessions (Group BAA), showing that tickling did not elevate instrumental responding during the test if it had been unrelated to initial acquisition. We discuss implications of our results for understanding the role of positive affective states in relapse of problem behavior.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The impact of three weeks of pre-quit varenicline on reinforcing value and craving for cigarettes in a laboratory choice procedure

RATIONALE

Varenicline, a partial nicotinic agonist, is theorized to attenuate pre-quit smoking reinforcement and post-quit withdrawal and craving. However, the mechanisms of action have not been fully characterized, as most studies employ only retrospective self-report measures, hypothetical indices of reinforcing value, and/or nontreatment-seeking samples.

OBJECTIVES

The current research examined the impact of pre-quit varenicline (vs. placebo) on laboratory measures of smoking and food (vs. water) reinforcement and craving.

METHODS

Participants were 162 treatment-seeking smokers enrolled in a randomized controlled trial of smoking cessation ( clinicaltrials.gov ID: NCT03262662). Participants completed two laboratory sessions: a pre-treatment session, ~ 1 week prior to beginning varenicline or placebo, and an active treatment session, after ~ 3 weeks of treatment. At each session, participants completed a laboratory choice procedure; on each of 36 trials, a lit cigarette, food item, or cup of water was randomly presented. Participants reported level of craving and spent $0.01-0.25 to have a corresponding 5-95% chance to sample the cue.

RESULTS

As predicted, spending was significantly higher on cigarette trials than water trials, and varenicline resulted in a greater between-session decline in spending on cigarette trials (but not water) than did placebo. Cigarette craving was enhanced in the presence of smoking cues compared to water, but neither average (tonic) cigarette craving nor cue-specific cigarette craving was significantly influenced by varenicline. Food spending and craving were generally unaffected by varenicline treatment.

CONCLUSIONS

These laboratory data from treatment-seeking smokers provide the strongest evidence to date that varenicline selectively attenuates smoking reinforcement prior to quitting.

Resurgence and repeated within-session progressive-interval thinning of alternative reinforcement

Resurgence of a previously suppressed target behavior is common when reinforcement for a more recently reinforced alternative behavior is thinned. To better characterize such resurgence, these experiments examined repeated within-session alternative reinforcement thinning using a progressive-interval (PI) schedule with rats. In Experiment 1, a transition from a high rate of alternative reinforcement to a within-session PI schedule generated robust resurgence, but subsequent complete removal of alternative reinforcement produced no additional resurgence. Experiment 2 replicated these findings and showed similar effects with a fixed-interval (FI) schedule arranging similarly reduced session-wide rates of alternative reinforcement. Thus, the lack of additional resurgence following repeated exposure to the PI schedule was likely due to the low overall obtained rate of alternative reinforcement provided by the PI schedule, rather than to exposure to within-session reinforcement thinning per se. In both experiments, target responding increased at some point in the session during schedule thinning and continued across the rest of the session. Rats exposed to a PI schedule showed resurgence later in the session and after more cumulative alternative reinforcers than those exposed to an FI schedule. The results suggest the potential importance of further exploring how timing and change-detection mechanisms might be involved in resurgence.

mGlu5: A double-edged sword for aversive learning related therapeutics

Aversive memories underlie many types of anxiety disorders. One area of research to more effectively treat anxiety disorders has therefore been identifying pharmacological targets to affect memory processes. Among these targets, the metabotropic glutamate 5 receptor (mGlu5) has received attention due to the availability of drugs to utilize its role in learning and memory. In this review, we highlight preclinical studies examining the role of mGlu5 at various stages of aversive learning and its inhibition via extinction in order to gain a better understanding of its therapeutic potential. We suggest that mGlu5 has distinct roles at different stages of memory that not only makes it a tricky target, but a double-edged sword as a therapeutic. However, the selective involvement of mGlu5 in different memory stages allows for certain precision that could be harnessed clinically. We therefore suggest potential applications, limitations, and pitfalls when considering use of mGlu5 modulators as therapeutics. In addition, we recommend future studies to address important gaps in this literature, such as sex and age factors in light of anxiety disorders being more prevalent in those demographics.

Decision-making (in)flexibility in gambling disorder

BACKGROUND

Behavioral flexibility -the ability to dynamically readjust our behavior in response to reward contingency changes- is often investigated using probabilistic reversal learning tasks (PRLT). Poor PRLT performance has been proposed as a proxy for compulsivity, and theorized to be related to perseverative gambling. Previous attempts to measure inflexibility with the PRLT in patients with gambling disorder have, however, used a variety of indices that may conflate inflexibility with more general aspects of performance in the task.

METHODS

Trial-by-trial PRLT acquisition and reacquisition curves in 84 treatment-seeking patients with gambling disorder and 64 controls (non-gamblers and non-problem recreational gamblers) were analyzed to distinguish between (a) variability in acquisition learning, and (b) reacquisition learning in reversed contingency phases. Complementarily, stay/switch responses throughout the task were analyzed to identify (c) premature switching, and (d) sensitivity to accumulated negative feedback.

RESULTS AND INTERPRETATION

Even after controlling for differences in acquisition learning, patients were slower to readjust their behavior in reversed contingency phases, and were more prone to maintain their decisions despite accumulated negative feedback. Inflexibility in patients with gambling disorder is thus a robust phenomenon that could predate gambling escalation, or result from massive exposure to gambling activities.

Decision-making inflexibility in a reversal learning task is associated with severity of problem gambling symptoms but not with a diagnosis of substance use disorder

BACKGROUND

Decisions made by individuals with disordered gambling are markedly inflexible. However, whether anomalies in learning from feedback are gambling-specific, or extend beyond gambling contexts, remains an open question. More generally, addictive disorders-including gambling disorder-have been proposed to be facilitated by individual differences in feedback-driven decision-making inflexibility, which has been studied in the lab with the Probabilistic Reversal Learning Task (PRLT). In this task, participants are first asked to learn which of two choice options is more advantageous, on the basis of trial-by-trial feedback, but, once preferences are established, reward contingencies are reversed, so that the advantageous option becomes disadvantageous and vice versa. Inflexibility is revealed by a less effective reacquisition of preferences after reversal, which can be distinguished from more generalized learning deficits.

METHODS

In the present study, we compared PRLT performance across two groups of 25 treatment-seeking patients diagnosed with an addictive disorder and who reported gambling problems, and 25 matched controls [18 Males/7 Females in both groups, Mage(SDage) = 25.24 (8.42) and 24.96 (7.90), for patients and controls, respectively]. Beyond testing for differences in the shape of PRLT learning curves across groups, the specific effect of problematic gambling symptoms' severity was also assessed independently of group assignment. In order to surpass previous methodological problems, full acquisition and reacquisition curves were fitted using generalized mixed-effect models.

RESULTS

Results showed that (1) controls did not significantly differ from patients in global PRLT performance nor showed specific signs of decision-making inflexibility; and (2) regardless of whether group affiliation was controlled for or not, gambling severity was specifically associated with more inefficient learning in phases with reversed contingencies.

CONCLUSION

Decision-making inflexibility, as revealed by difficulty to reacquire decisional preferences based on feedback after contingency reversals, seems to be associated with gambling problems, but not necessarily with a substance-use disorder diagnosis. This result aligns with gambling disorder models in which domain-general compulsivity is linked to vulnerability to develop gambling-specific problems with exposure to gambling opportunities.

Conducting extinction in multiple contexts attenuates relapse of operant behavior in humans

In one experiment with human participants, we investigated the effects of using multiple contexts during extinction on the renewal of operant responses. Undergraduate students played a videogame in which they learned to shoot at enemies in Context A. Then, all participants experienced an extinction training. For half of the participants, extinction trials were conducted in a single context, whereas the other half received extinction in three different contexts. Finally, all participants were tested in Context A. We observed that conducting extinction in multiple contexts attenuated ABA renewal. The present results suggest that conducting extinction in multiple contexts can be used as a behavioral technique to reduce operant renewal.