Slow reacquisition of a conditioned taste aversion

Partial reinforcement effects on acquisition and extinction of a conditioned taste aversion

Four experiments with rat subjects asked whether a partial reinforcement extinction effect (PREE) occurs in taste aversion learning. The question has received little attention in the literature, and to our knowledge no taste aversion experiment has previously demonstrated a PREE. In each of the present experiments, experimental groups received a taste mixed in drinking water for 20 min; such taste exposures were sometimes paired with a lithium chloride (LiCl) injection and sometimes not. Control groups received only taste-LiCl pairings. There was evidence that each reinforced and non-reinforced trial produced increments and decrements in aversion strength (respectively), and trials mattered more than accumulated time during the conditioned stimulus and during the background (as emphasized in time-accumulation models like those of Gallistel and Gibbon, Psychological Review, 107, 289-344, 2000, and Gibbon and Balsam, Autoshaping and conditioning theory, Academic Press, New York, pp. 219-235, 1981). In addition, a partial reinforcement extinction effect was observed when there was a relatively large number of conditioning trials. The results extend our understanding of extinction in taste aversion learning and provide more evidence that aversion learning might follow rules that are qualitatively similar to those of other forms of learning.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Behavioral and immunohistochemical characterization of rapid reconditioning following extinction of contextual fear

A fundamental property of extinction is that the behavior that is suppressed during extinction can be unmasked through a number of postextinction procedures. Of the commonly studied unmasking procedures (spontaneous recovery, reinstatement, contextual renewal, and rapid reacquisition), rapid reacquisition is the only approach that allows a direct comparison between the impact of a conditioning trial before or after extinction. Thus, it provides an opportunity to evaluate the ways in which extinction changes a subsequent learning experience. In five experiments, we investigate the behavioral and neurobiological mechanisms of postextinction reconditioning. We show that rapid reconditioning of unsignaled contextual fear after extinction in male Long-Evans rats is associative and not affected by the number or duration of extinction sessions that we examined. We then evaluate c-Fos expression and histone acetylation (H4K8) in the hippocampus, amygdala, prefrontal cortex, and bed nucleus of the stria terminalis. We find that in general, initial conditioning has a stronger impact on c-Fos expression and acetylation than does reconditioning after extinction. We discuss implications of these results for theories of extinction and the neurobiology of conditioning and extinction.

Experimental extinction in Pavlovian conditioning: Behavioural and neuroscience perspectives

This paper reviews the behavioural and neuroscience literatures on extinction in Pavlovian conditioning with a view towards finding possible points of contact between these two often independent lines of investigation. Recent discoveries at the behavioural level indicate (1) that conditioned stimulus (CS)–unconditioned stimulus (US) associations specific in their sensory content are fully preserved during extinction, (2) that inhibitory stimulus-response associations appear to be learned during extinction, (3) that extinction is influenced by the level of activation of the US representation during nonreinforced trials, (4) that decreases in attention can influence conditioned performance during extinction, and (5) that contexts acquire an ability to modulate learning during both conditioning and extinction. Recent discoveries at the neural systems level suggest (1) that the hippocampus is important in context-specific learning during extinction, (2) that the prefrontal cortex is possibly important in long-term memory for extinction, (3) that the basolateral amygdala may be important in sustaining attention to a CS during extinction, (4) that NMDA receptors are important either in neural plasticity during extinction or by affecting the value of the US representation during extinction, and (5) that the GABAergic system may partially mediate inhibitory learning during extinction. It is concluded that both of these levels of analysis can benefit the other in the pursuit of a more comprehensive understanding of extinction.

Attenuation and reacquisition of foraging behavior in a patchy environment

The present experiment examined the attenuation and reacquisition of foraging behavior of pigeons on a restricted diet. Using the procedure of Roberts (1988), thirty-two feeders were arranged in four circular patches of eight feeders each. The feeders were baited so that each patch contained a different density of food. Pigeons learned to forage among the patches. During this initial acquisition phase, visits to the patches by the pigeons differed as a function of the food density of the patches. After this initial foraging training phase, each pigeon received one of two response elimination procedures. For half of the subjects, food was not present during the response elimination phase. For the remaining subjects, food was placed exclusively outside the patches during this phase. Foraging behavior decreased quickly and somewhat similarly for both conditions. During a final phase in which the retraining of the original foraging behavior occurred, the group given food outside the patches during response elimination provided evidence of superior foraging with respect to sensitivity to the different patch food densities. The results are discussed with regard to previously published response elimination effects.

Associative Accounts of Recovery-from-Extinction Effects

Recovery-from-extinction effects (e.g., spontaneous recovery, renewal, reinstatement, and facilitated reacquisition) have become the focus of much research in recent years. However, despite a great deal of empirical data, there are few theoretical explanations for these effects. This paucity poses a severe limitation on our understanding of these behavioral effects, impedes advances in uncovering neural mechanisms of response recovery, and reduces our potential to prevent relapse after exposure therapy. Towards correcting this oversight, this review takes prominent models of associative learning that have been used in the past and continue to be used today to explain Pavlovian conditioning and extinction, and assesses how each model can be applied to account for recovery-from-extinction effects. The models include the Rescorla-Wagner (1972) model, Mackintosh's (1975) attentional model, Pearce and Hall's (1980) attentional model, Wagner's (1981) SOP model, Pearce's (1987) configural model, McLaren and Mackintosh's (2002) elemental model, and Stout and Miller's (2007) SOCR (comparator hypothesis) model. Each model is assessed for how well it explains or does not explain the various recovery-from-extinction phenomena. We offer some suggestions for how the models might be modified to account for these effects in those instances in which they initially fail.

Occasional Reinforced Responses During Extinction Can Slow the Rate of Reacquisition of an Operant Response

Three experiments with rats examined reacquisition of an operant response after either extinction or a response-elimination procedure that included occasional reinforced responses during extinction. In each experiment, reacquisition was slower when response elimination had included occasional reinforced responses, although the effect was especially evident when responding was examined immediately following each response-reinforcer pairing during reacquisition (Experiments 2 and 3). An extinction procedure with added noncontingent reinforcers also slowed reacquisition (Experiment 3). The results are consistent with research in classical conditioning (Bouton, Woods, & Pineño, 2004) and suggest that rapid reacquisition after extinction is analogous to a renewal effect that occurs when reinforced responses signal a return to the conditioning context. Clinical implications are also discussed.

Timing of omitted events: an analysis of temporal control of inhibitory behavior

This paper reviews research designed to investigate the temporal control of inhibitory responding using rats as subjects. One area of investigation has focused on the role of temporal variables in conditioned inhibition produced using Pavlov's [Pavlov, I.P., 1927. Conditioned Reflexes. Oxford University Press, London, 430 pp.] procedure. These studies have found that evidence of conditioned inhibition obtained by negative summation testing is strongest when the conditioned inhibitor signals the omission of the unconditioned stimulus (US) at the same temporal location as a transfer excitor signals presentation of the US [e.g., Barnet, R.C., Miller, R.R., 1996. Temporal encoding as a determinant of inhibitory control. Learn. Motiv. 27, 73-91]. Similarly, retardation of acquisition of behavioral control by a previously inhibitory conditioned stimulus (CS) is maximal when the inhibitory CS is paired with the US at the same temporal location as the inhibitor had previously signaled US omission [Burger, D., Denniston, J.C., Miller, R.R., 2001. Temporal coding in condition inhibition: retardation tests. Anim. Learn. Behav. 29, 281-290]. Other lines of research designed to assess the associative structure of temporal control of inhibition [e.g., Denniston, J.C., Blaisdell, A.P., Miller, R.R., 2004. Temporal control in conditioned inhibition: analysis of associative structure of inhibition. J. Exp. Psychol. Anim. Behav. Process. 30, 190-202] are reviewed, as is the assessment of temporal control of inhibition produced through extinction [Denniston, J.C., Miller, R.R., 2003. The role of temporal variables in inhibition produced through extinction. Learn. Behav. 31, 35-48]. These collective observations are discussed in terms of the temporal coding hypothesis [Matzel, L.D., Held, F.P., Miller, R.R., 1988. Reexamination of simultaneous and backward conditioning: Implications for contiguity theory. Learn. Motiv. 19, 317-344].

Primacy and recency effects in extinction and latent inhibition: a selective review with implications for models of learning

In the framework of animal conditioning and human associative learning, primacy and recency effects on acquired stimulus control of behavior refer to the superior influence of first-learned and last-learned associations, respectively. Most contemporary associative models of learning anticipate unwavering recency effects and claim support from numerous published studies. But, for pragmatic reasons, almost all of these studies were conducted under select conditions that favored recency effects. When these conditions are not met, recency effects are far from ubiquitous. We review the literature on primacy and recency effects regarding extinction and latent inhibition (i.e., interference between outcomes), with special emphasis on the impact of certain post-training manipulations and test conditions on conditioned responding. Evidence for recency-to-primacy shifts and for memory integration is examined in light of contemporary models of learning.

Context and behavioral processes in extinction

This article provides a selective review and integration of the behavioral literature on Pavlovian extinction. The first part reviews evidence that extinction does not destroy the original learning, but instead generates new learning that is especially context-dependent. The second part examines insights provided by research on several related behavioral phenomena (the interference paradigms, conditioned inhibition, and inhibition despite reinforcement). The final part examines four potential causes of extinction: the discrimination of a new reinforcement rate, generalization decrement, response inhibition, and violation of a reinforcer expectation. The data are consistent with behavioral models that emphasize the role of generalization decrement and expectation violation, but would be more so if those models were expanded to better accommodate the finding that extinction involves a context-modulated form of inhibitory learning.

The role of temporal variables in inhibition produced through extinction

In two experiments with rats as subjects, the temporal characteristics of inhibition produced through extinction were investigated. Each experiment established two independent signals for unconditioned stimulus presentation, one trace and one delay. Following initial training, either the trace or the delay conditioned stimulus (CS) was massively extinguished. In Experiment 1, a summation test established that an extinguished delay CS (but not a neutral CS) passed a summation test with a delay, but not with a trace, transfer excitor, and an extinguished trace CS (but not a neutral CS) passed a summation test with a trace, but not with a delay, transfer excitor. In Experiment 2, a retardation test showed retarded behavioral control by an extinguished delay CS when the CS was retrained as a delay CS, but not as a trace CS, and by an extinguished trace CS when the CS was retrained as a trace CS, but not as a delay CS. The results are discussed in terms of contemporary theories of extinction.

Savings in classical conditioning in the rabbit as a function of extended extinction

In the present experiments, savings phenomena following a limited amount of initial acquisition and extended extinction were examined. Experiments 1 and 2 compared rates of reacquisition following brief acquisition and various amounts of extinction in conditioning of the rabbit's nictitating membrane and heart rate response, respectively. Experiment 3 compared rates of acquisition to a novel stimulus (e.g., light) following brief acquisition and various amounts of extinction to another stimulus (e.g., tone). In addition, in Experiment 3 recovery of responding to the extinguished stimulus during acquisition to the novel, cross-modal stimulus was examined. Experiments 1, 2, and 3 demonstrated that with a limited number of acquisition trials (1) there was a graded reduction in the rate of reacquisition as a function of the number of extinction trials in both conditioning preparations, (2) there was a graded reduction in the rate of cross-modal acquisition as a function of the number of extinction trials, but (3), in Experiment 3, recovery of responding to the extinguished stimulus during cross-modal training of the novel stimulus appeared uniformly robust even in the face of extended extinction.

Impact of brief or extended extinction of a taste aversion on inhibitory associations: evidence from summation, retardation, and preference tests

In five conditioned taste aversion experiments with rats, summation, retardation, and preference tests were used to assess the effects of extinguishing a conditioned saccharin aversion for three or nine trials. In Experiment 1, a summation test showed that saccharin aversion extinguished over nine trials reduced the aversion to a merely conditioned flavor (vinegar), whereas three saccharin extinction trials did not subsequently influence the vinegar aversion. Experiment 2 clarified that result, with unpaired controls equated on flavor exposure prior to testing; the results with those controls suggested that the flavor extinguished for nine trials produced generalization decrement during testing. In Experiment 3, the saccharin aversion reconditioned slowly after nine extinction trials, but not after three. Those results suggested the development of latent inhibition after more than three extinction trials. Preference tests comparing saccharin consumption with a concurrently available fluid (water in Experiment 4, saline in Experiment 5) showed that the preference for saccharin was greater after nine extinction trials than after three. However, saccharin preference after nine extinction trials was not greater, as compared with that for either latent inhibition controls (Experiments 4 and 5) or a control given equated exposures to saccharin and trained to drink saline at a high rate prior to testing (Experiment 5). Concerns about whether conditioned inhibition has been demonstrated in any flavor aversion procedure are discussed. Our findings help explain both successes and failures in demonstrating post-extinction conditioned response recovery effects reported in the conditioned taste aversion literature, and they can be explained using a memory interference account.

Context, ambiguity, and unlearning: sources of relapse after behavioral extinction

There is now ample evidence that extinction, the loss of learned performance that occurs when a Pavlovian signal or an instrumental action is repeatedly presented without its reinforcer, does not reflect a destruction of the original learning. This article summarizes the evidence and extends and updates earlier reviews. The main alternative to "unlearning" is the idea that extinction (as well as other retroactive interference processes, including counterconditioning) involves new learning that is stored along with the old. One consequence is that the Pavlovian signal or instrumental action has two available "meanings" and thus has the properties of an ambiguous word: its current meaning (and the resulting behavioral output) depends on what the current context retrieves. Contexts can be provided by a variety of background stimuli, including the physical environment, internal drug state, and time. The second thing learned (e.g., extinction, counterconditioning) seems especially dependent on the context for retrieval. A variety of evidence is consistent with this analysis, which highlights several important sources of relapse after extinction. The article concludes with several issues for future research, among them the question of how we can optimize extinction and other putative "unlearning" treatments so as to prevent the various forms of relapse discussed here.

Common processes may contribute to extinction and habituation

Psychologists routinely attribute the characteristics of conditioned behavior to complicated cognitive processes. For example, many of the characteristics of behavior undergoing extinction have been attributed to retrieval from memory. The authors argue that these characteristics may result from the simpler process of habituation. In particular, conditioned responding may decrease during extinction partially because habituation occurs to the stimuli that control responding when those stimuli are presented repeatedly or for a prolonged time (e.g., the experimental context, the conditioned stimulus in classical conditioning). This idea is parsimonious, has face validity, and evokes only processes that are well established by other evidence. In addition, behavior undergoing extinction shows 12 of the fundamental properties of behavior undergoing habituation. However, this model probably cannot provide a complete theory of extinction. It provides no obvious explanation for some of the other characteristics of extinguished behavior.

Extinction revisited: similarities between extinction and reductions in US intensity in classical conditioning of the rabbit's nictitating membrane response

The mechanisms of extinction were examined by reducing the intensity of the unconditioned stimulus (US) after acquisition training to determine whether such reductions lie on a continuum with CS-alone extinction. The experiments revealed that reductions in US intensity yielded extinction-like effects. Specifically, there were proportional reductions in the daily mean level of responding across sessions. There were also persistent within-session declines and between-session increases of responding analogous to spontaneous recovery. Surprisingly, even when US intensity was held constant, within-session declines and between-session increases were apparent. The results are discussed with respect to possible contributions from unlearning, new learning, generalization decrement, and nonassociative loss, especially CS-specific attentional changes and CR-specific reactive inhibition.

Extinction of a conditioned taste aversion in rats: I. Behavioral effects

The literature is divided over whether a conditioned taste aversion (CTA) can be fully extinguished. In Experiment 1, we created a powerful aversion in 54 rats by pairing the taste of 0.0025 M NaSaccharin (CS) with intraperitoneal injections of 127 mg/kg LiCl (US) on 3 occasions. We then offered 23-h deprived rats NaSaccharin for 10 min/day to observe the course of recovery. Extinction occurred in three phases: static, dynamic, and asymptotic. During the static phase (mean = 9.6 days), rats consumed the CS at < 10% of their preconditioned rate. With dynamic recovery (6.0 days), they increased acceptance to > 80% of preconditioning levels. Finally, they achieved asymptote (3.1 days) at 100% acceptance. In Experiment 2, we used 8 additional conditioned rats and 8 unconditioned controls. We followed the same 1-bottle extinction procedure and, again, obtained 100% acceptance. Then we offered both NaSaccharin and water for 8 days at 23 h/day and monitored lick patterns every 6 s to determine taste preferences. The conditioned animals consumed less NaSaccharin than controls on Day 1, and less NaSaccharin as a percentage of total fluid as late as Day 3. For the last 5 days of 2-bottle preference testing, there were no significant differences between the groups with regard to 1. volume of NaSaccharin or water consumed, 2. percentage of total fluid taken as NaSaccharin, 3. consumption of each fluid associated with a meal or taken spontaneously, 4. intake during the light or dark periods, or 5. the characteristics of ingestion, including number of drinking bouts, duration of bouts, number of licks/bout, and rate of licking. Therefore, a robust CTA is subject to complete behavioral extinction.

What's elementary about associative learning?

The scientific study of associative learning began nearly 100 years ago with the pioneering studies of Thorndike and Pavlov, and it continues today as an active area of research and theory. Associative learning should be the foundation for our understanding of other forms of behavior and cognition in human and nonhuman animals. The laws of associative learning are complex, and many modern theorists posit the involvement of attention, memory, and information processing in such basic conditioning phenomena as overshadowing and blocking, and the effects of stimulus preexposure on later conditioning. An unresolved problem for learning theory is distinguishing the formation of associations from their behavioral expression. This and other problems will occupy future generations of behavioral scientists interested in the experimental investigation of associative learning. Neuroscientists and cognitive scientists will both contribute to and benefit from that effort in the next 100 years of inquiry.