Facilitation of extinction by an increase or a decrease in trial duration

EXPRESS: Determinants of Extinction in a Streamed Trial Procedure

The strength of an association between a cue and its outcome is influenced by both the probability of the outcome given the cue and the probability of the outcome in the absence of the cue. Once an association has been formed, extinction is the procedure for reducing responding indicative of the association by repeated presentation of the cue without the outcome. The present experiments tested whether cumulative frequency and/or cumulative duration of these events affects associative extinction in a streamed trial extinction procedure with human participants. Experiment 1 assessed the effects of parametric manipulations of the frequency and duration of either the cue by itself or cue-outcome coabsence. In Experiment 1, participants proved relatively insensitive to manipulation of the event's duration. In contrast, judgments of the association by participants decreased when the frequency of cue-alone events was increased, even when the durations of those events were decreased so that cumulative exposure to the cue was equated. No effect of either the duration or the frequency of cue-outcome coabsence was observed. Experiment 2 demonstrated that the effect of cue-alone (i.e., extinction trial) frequency generalizes across a wide range of parameters for initial acquisition achieved by cue-outcome pairings. Experiment 3 tested for an interaction between event duration during initial learning and event duration during extinction. Collectively, these results indicate that the cumulative frequency, and not the cumulative duration, of extinction trials as well as the duration of the cue-outcome coabsences between extinction trials controls the effectiveness of an extinction procedure.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

A systems-neuroscience model of phasic dopamine

We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary rewards (Unconditioned Stimuli [USs]), and a learned value (LV) system for learning about stimuli associated with such rewards (CSs). The LV system represents the amygdala, which drives phasic bursting in midbrain dopamine areas, while the PV system represents the ventral striatum, which drives shunting inhibition of dopamine for expected USs (via direct inhibitory projections) and phasic pausing for expected USs (via the lateral habenula). Our model accounts for data supporting the separability of these systems, including individual differences in CS-based (sign-tracking) versus US-based learning (goal-tracking). Both systems use competing opponent-processing pathways representing evidence for and against specific USs, which can explain data dissociating the processes involved in acquisition versus extinction conditioning. Further, opponent processing proved critical in accounting for the full range of conditioned inhibition phenomena, and the closely related paradigm of second-order conditioning. Finally, we show how additional separable pathways representing aversive USs, largely mirroring those for appetitive USs, also have important differences from the positive valence case, allowing the model to account for several important phenomena in aversive conditioning. Overall, accounting for all of these phenomena strongly constrains the model, thus providing a well-validated framework for understanding phasic dopamine signaling. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Rescaling of temporal expectations during extinction

Previous research suggests that extinction learning is temporally specific. Changing the conditioned stimulus (CS) duration between training and extinction can facilitate the loss of the conditioned response (CR) within the extinction session but impairs long-term retention of extinction. In 2 experiments using conditioned magazine approach with rats, we examined the relation between temporal specificity of extinction and CR timing. In Experiment 1, rats were trained on a 12-s, fixed CS-unconditional stimulus interval and then extinguished with CS presentations that were 6, 12, or 24 s in duration. The design of Experiment 2 was the same except rats were trained using partial rather than continuous reinforcement. In both experiments, extending the CS duration in extinction facilitated the diminution of CRs during the extinction session, but shortening the CS duration failed to slow extinction. In addition, extending (but not shortening) the CS duration caused temporal rescaling of the CR, in that the peak CR rate migrated later into the trial over the course of extinction training. This migration partially accounted for the faster loss of the CR when the CS duration was extended. Results are incompatible with the hypothesis that extinction is driven by cumulative CS exposure and suggest that temporally extended nonreinforced CS exposure reduces conditioned responding via temporal displacement rather than through extinction per se. (PsycINFO Database Record

Experimental extinction in Pavlovian conditioning: Behavioural and neuroscience perspectives

This paper reviews the behavioural and neuroscience literatures on extinction in Pavlovian conditioning with a view towards finding possible points of contact between these two often independent lines of investigation. Recent discoveries at the behavioural level indicate (1) that conditioned stimulus (CS)–unconditioned stimulus (US) associations specific in their sensory content are fully preserved during extinction, (2) that inhibitory stimulus-response associations appear to be learned during extinction, (3) that extinction is influenced by the level of activation of the US representation during nonreinforced trials, (4) that decreases in attention can influence conditioned performance during extinction, and (5) that contexts acquire an ability to modulate learning during both conditioning and extinction. Recent discoveries at the neural systems level suggest (1) that the hippocampus is important in context-specific learning during extinction, (2) that the prefrontal cortex is possibly important in long-term memory for extinction, (3) that the basolateral amygdala may be important in sustaining attention to a CS during extinction, (4) that NMDA receptors are important either in neural plasticity during extinction or by affecting the value of the US representation during extinction, and (5) that the GABAergic system may partially mediate inhibitory learning during extinction. It is concluded that both of these levels of analysis can benefit the other in the pursuit of a more comprehensive understanding of extinction.

Separation of time-based and trial-based accounts of the partial reinforcement extinction effect

Two appetitive conditioning experiments with rats examined time-based and trial-based accounts of the partial reinforcement extinction effect (PREE). In the PREE, the loss of responding that occurs in extinction is slower when the conditioned stimulus (CS) has been paired with a reinforcer on some of its presentations (partially reinforced) instead of every presentation (continuously reinforced). According to a time-based or "time-accumulation" view (e.g., Gallistel and Gibbon, 2000), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger amount of time has accumulated in the CS over trials. In contrast, according to a trial-based view (e.g., Capaldi, 1967), the PREE occurs because the organism has learned in partial reinforcement to expect the reinforcer after a larger number of CS presentations. Experiment 1 used a procedure that equated partially and continuously reinforced groups on their expected times to reinforcement during conditioning. A PREE was still observed. Experiment 2 then used an extinction procedure that allowed time in the CS and the number of trials to accumulate differentially through extinction. The PREE was still evident when responding was examined as a function of expected time units to the reinforcer, but was eliminated when responding was examined as a function of expected trial units to the reinforcer. There was no evidence that the animal responded according to the ratio of time accumulated during the CS in extinction over the time in the CS expected before the reinforcer. The results thus favor a trial-based account over a time-based account of extinction and the PREE. This article is part of a Special Issue entitled: Associative and Temporal Learning.

Human fear conditioning and extinction: timing is everything…or is it?

A differential fear conditioning paradigm was used with 107 healthy undergraduate participants to evaluate the effect of conditioned stimulus (CS) temporal properties on fear acquisition and extinction. Two minute duration CSs were used for Day 1 fear acquisition. Participants were randomized to receive either 1, 2, or 4min CS durations during Day 2 extinction. Extinction re-test was examined on Day 3 using the original acquisition CS duration (2min). Findings indicated that participants who were aware of the CS+/unconditioned stimulus (US) contingency (n=52) develop a temporal expectation about when the unconditioned stimulus will be delivered. Although the shorter duration CS resulted in greater fear reduction during extinction, cessation of fear responding at re-test was the same for CS extinction durations ranging from half the CS acquisition duration to twice the CS acquisition duration. Thus, extinction performance did not predict extinction at re-test, which could have important implications for optimizing exposure therapy for anxiety disorders.

Impacts of forebrain neuronal glycine transporter 1 disruption in the senescent brain: evidence for age-dependent phenotypes in Pavlovian learning

Genetic deletion of glycine transporter 1 (GlyT1) in forebrain neurons gives rise to multiple-procognitive phenotypes, presumably due to enhanced N-methyl-d-aspartate receptor (NMDAR) functions. However, concerns over possible harmful excitotoxic effects under lifelong elevation of synaptic glycine have been raised. Such effects might accelerate the aging process, weakening or even reversing the procognitive phenotypes identified in adulthood. Here, we examined if one of the most robust phenotypes in the mutant mouse line (CamKIIαCre;GlyT1tm1.2fl/fI), namely, enhanced aversive Pavlovian conditioning, might be modified by age. Comparison between 3-month-old (adult) and 22-month-old (aged) mutants confirmed the presence of this phenotype at both ages. However, the temporal expression of the Pavlovian phenotype was modified in senescence; while adult mutants showed a pronounced within-session extinction, aged mutants did not. Expression of NR2B subunits of NMDAR and neural proliferation were examined in the same animals by immunohistochemistry. These were reduced in the aged mice as expected, but not exacerbated by the mutation. Thus, our results do not substantiate the concerns of neurotoxic effects through lifelong GlyT1 disruption in forebrain neurons, but provide evidence for a modification of phenotypic expression as a function of age. The latter points to the need to further investigate other procognitive phenotypes identified at adulthood in this mutant line. In addition, we revealed here for the first time a clear increase in the number of immature neurons in the hippocampus of the mutants, although the behavioral significance of this phenotype remains to be determined.

Time and Associative Learning

In a basic associative learning paradigm, learning is said to have occurred when the conditioned stimulus evokes an anticipatory response. This learning is widely believed to depend on the contiguous presentation of conditioned and unconditioned stimulus. However, what it means to be contiguous has not been rigorously defined. Here we examine the empirical bases for these beliefs and suggest an alternative view based on the hypothesis that learning about the temporal relationships between events determines the speed of emergence, vigor and form of conditioned behavior. This temporal learning occurs very rapidly and prior to the appearance of the anticipatory response. The temporal relations are learned even when no anticipatory response is evoked. The speed with which an anticipatory response emerges is proportional to the informativeness of the predictive cue (CS) regarding the rate of occurrence of the predicted event (US). This analysis gives an account of what we mean by "temporal pairing" and is in accord with the data on speed of acquisition and basic findings in the cue competition literature. In this account, learning depends on perceiving and encoding temporal regularities rather than stimulus contiguities.

Memory is not extinguished along with CS presentation but within a few seconds after CS-offset

Prior work with the crab's contextual memory model showed that CS-US conditioned animals undergoing an unreinforced CS presentation would either reconsolidate or extinguish the CS-US memory, depending on the length of the reexposure to the CS. Either memory process is only triggered once the CS is terminated. Based on these results, the following questions are raised. First, when is extinction memory acquired, if not along extinction training, and how long does it take? Second, can acquisition and consolidation of extinction memory be pharmacologically dissected? Here we address these questions performing three series of experiments: a first one aimed to study systematically the relationship between extinction and increasing periods of unreinforced CS presentations, a second one to determine the time boundaries of the extinction memory acquisition, and the third one to assay the requirement for protein synthesis and NMDA-like receptors of acquisition and consolidation of extinction memory. Our results confirm that it is CS-offset and not the mere retrieval (CS-onset) that triggers acquisition of extinction memory and that it is completed in less than 45 sec after CS-offset. In addition, protein synthesis is required for consolidation but not for acquisition of this memory and, conversely, NMDA-like receptor activity is required for its acquisition but not for its consolidation. Finally, we offer an interpretative scheme of our results and we discuss to what extent it could apply to multitrial extinction.

Timing of fear expression in trace and delay conditioning measured by fear-potentiated startle in rats

In two experiments, the time course of the expression of fear in trace (hippocampus-dependent) versus delay (hippocampus-independent) conditioning was characterized with a high degree of temporal specificity using fear-potentiated startle. In experiment 1, groups of rats were given delay fear conditioning or trace fear conditioning with a 3- or 12-sec trace interval between conditioned stimulus (CS) offset and unconditioned stimulus (US) onset. During test, the delay group showed fear-potentiated startle in the presence of the CS but not after its offset, whereas the trace groups showed fear-potentiated startle both during the CS and after its offset. Experiment 2 compared the time course of fear expression after trace conditioning with the time course in two delay conditioning groups: one matched to the trace conditioning group with respect to CS duration, and the other with respect to ISI. In all groups, fear was expressed until the scheduled occurrence of the US and returned to baseline rapidly thereafter. Thus, in both trace and delay fear conditioning, ISI is a critical determinant of the time course of fear expression. These results are informative as to the possible role of neural structures, such as the hippocampus, in memory processes related to temporal information.

Context and behavioral processes in extinction

This article provides a selective review and integration of the behavioral literature on Pavlovian extinction. The first part reviews evidence that extinction does not destroy the original learning, but instead generates new learning that is especially context-dependent. The second part examines insights provided by research on several related behavioral phenomena (the interference paradigms, conditioned inhibition, and inhibition despite reinforcement). The final part examines four potential causes of extinction: the discrimination of a new reinforcement rate, generalization decrement, response inhibition, and violation of a reinforcer expectation. The data are consistent with behavioral models that emphasize the role of generalization decrement and expectation violation, but would be more so if those models were expanded to better accommodate the finding that extinction involves a context-modulated form of inhibitory learning.

Memory priming and trial spacing effects in Pavlovian learning

Conditioning trials that are massed in time produce less conditioning than those that are spaced in time. Four experiments with rat subjects examined whether a recent conditioning trial interferes with conditioning on the next trial by temporarily "priming" information in short-term memory (e.g., Wagner, 1978, 1981). We used appetitive conditioning procedures in which priming trials preceded target trials by 60 sec. When the priming trials were nonreinforced presentations of a conditioned stimulus (CS), the CS had to be the same CS as the one on the target trial to interfere with conditioning. When priming trials were actual CS-unconditioned stimulus (US) pairings, the CS identity did not matter, the US was the event that interfered with conditioning on the next trial. Reinforced trials reduced performance in a way that did not depend on context blocking. The results suggest that CS and US priming effects do contribute to conditioning deficits observed with massed trial procedures. The results are consistent with Wagner's (1981) "sometimes opponent process," or SOP, model, although a result that is paradoxical for the model suggests that recent USs may have motivational as well as memory effects.

Temporal specificity of extinction in autoshaping

Three experiments investigated the effects of varying the conditioned stimulus (CS) duration between training and extinction. Ring doves (Streptopelia risoria) were autoshaped on a fixed CS-unconditioned stimulus (US) interval and extinguished with CS presentations that were longer, shorter, or the same as the training duration. During a subsequent test session, the training CS duration was reintroduced. Results suggest that the cessation of responding during an extinction session is controlled by generalization of excitation between the training and extinction CSs and by the number of nonreinforced CS presentations. Transfer of extinction to the training CS is controlled by the similarity between the extinction and training CSs. Extinction learning is temporally specific.

A Partial Reinforcement Extinction Effect Despite Equal Rates of Reinforcement During Pavlovian Conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.