Ethanol-maintained responding of rats is more resistant to change in a context with added non-drug reinforcement

Resistance to change, of behavior and of theory

The persistence of operant behavior when disrupted tends to be positively related to how often reinforcers were delivered in the past. Behavioral momentum theory describes this finding as the outcome of Pavlovian processes. That is, the relation between discriminative stimuli and reinforcers that were delivered in their presence strengthens behavior, thereby making it more likely to persist. If only the story were that simple. A growing number of findings challenge the basic tenets of behavioral momentum theory. Some even call into question whether Pavlovian relations contribute to persistence in the first place. In this paper, I will review behavioral momentum theory and some of the data that have been problematic for the theory. I will argue that despite these very real challenges, the theory provides important utility not only to basic analyses of response persistence but also to clinical interventions directed at long-term reductions in problem behavior. It, for example, has set the stage for the development of alternative conceptual analyses of resistance to change, two of which will be highlighted for readers. Moreover, behavioral momentum theory may tell us something important about the reasons it continues to have an influence on the field, despite the challenging data that deter it.

Post-weaning social isolation increases the incentive value of nicotine-related contexts and decreases the accumulation of ΔFosB in nucleus accumbens in adolescent rats

Adolescent social conditions profoundly affect vulnerability to drug abuse. Preclinical studies have shown that preventing social interactions during adolescence increases the rewarding effects of drugs like alcohol, cocaine, or amphetamines, however, little data exist regarding the impact of social isolation on nicotine effects. The current study evaluated the effects of differential rearing conditions during adolescence (isolation or group rearing) on (1) conditioned place preference induced by low nicotine doses (0.1 or 0.3 mg/kg) and (2) sensitization to the locomotor effects of nicotine after sub-chronic administration (3) and accumulation of ΔFosB in nucleus accumbens (NAc). Results showed that nicotine induced place preference in isolated and grouped rats, but the effect was more persistent for the rats reared in isolation. Isolated reared rats also exhibited lower levels of ΔFosB accumulation in NAc. No differences were found in the behavioral sensitization to nicotine effects between rearing conditions. The results suggest that isolation engenders a more robust incentive value of nicotine-related contexts. This effect could be related to the basal expression of ΔFosB: lower levels of this transcription factor seem to impair the motivation of isolated reared rats and increase their vulnerability to the effects of drugs like nicotine.

Nondrug reinforcers contingent on alternative behavior or abstinence increase resistance to extinction and reinstatement of ethanol-maintained behavior

The effects of delivering nondrug alternative reinforcement on resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing were evaluated in two experiments. In both, rats self-administered ethanol by lever pressing in a two-component multiple schedule during baseline. In the Rich component, alternative food reinforcement was made available for performing an alternative response (Experiment 1) or according to a differential-reinforcement-of-other-behavior schedule for lever pressing (Experiment 2). In the Lean component, only ethanol was available. Moreover, the frequency of alternative reinforcement was manipulated across conditions in Experiment 1. Following baseline, lever pressing was extinguished in both components by suspending ethanol reinforcement, and alternative food reinforcers were discontinued. Finally, to test for reinstatement, ethanol reinforcers were delivered independently of lever pressing in both components. In both experiments, proportion-of-baseline response rates were higher during extinction and reinstatement testing in the Rich component than in the Lean component (although differentiation was not observed at the lowest frequency of alternative reinforcement in Experiment 1). Thus, alternative nondrug reinforcers increased resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing, even when those reinforcers were delivered contingently on an alternative response or on abstinence from lever pressing.

Role of the discriminative properties of the reinforcer in resurgence

In three experiments with rat subjects, we examined the effects of the discriminative effects of reinforcers that were presented during or after operant extinction. Experiments 1 and 2 examined resurgence, in which an extinguished operant response (R1) recovers when a second behavior (R2) that has been reinforced to replace it is also placed in extinction. The results of Experiment 1 suggest that the amount of R1's resurgence is a decreasing linear function of the interreinforcement interval used during the reinforcement of R2. In Experiment 2, R1 was reinforced with one outcome (O1), and R2 was then reinforced with a second outcome (O2) while R1 was extinguished. In resurgence tests, response-independent (noncontingent) presentations of O2 prevented resurgence of R1, which otherwise occurred when testing was conducted with either no reinforcers or noncontingent presentations of O1. In Experiment 3, we then examined the effects of noncontingent O1 and O2 presentations after simple extinction in either the presence or the absence of noncontingent presentations of O2. Overall, the results are consistent with a role for the discriminative properties of the reinforcer in controlling operant behavior. In resurgence, the reinforcer used during response elimination provides a distinct context that controls the inhibition of R1. The results are less consistent with an alternative view emphasizing the disrupting effects of alternative reinforcement.

Additional free reinforcers increase persistence of problem behavior in a clinical context: A partial replication of laboratory findings

Behavioral momentum theory is a quantitative framework used to characterize the persistence of behavior during response disruptors as a function of baseline stimulus-reinforcer relations. Results of several investigations have shown that alternative reinforcement can increase the resistance to change of a target response during extinction. In the present study, concomitant variable-interval fixed-time schedules of reinforcement for problem behavior were employed to simulate naturalistic situations involving the superimposition of response-independent reinforcers on a baseline schedule of reinforcement for problem behavior, as in the common use of noncontingent reinforcement treatments. Resistance to change of problem behavior was assessed during postsession periods of extinction by comparing response rates in extinction following sessions with and without additional reinforcer deliveries arranged by fixed-time schedules. For 2 out of 3 participants, problem behavior tended to be more resistant to extinction following periods in which additional fixed-time reinforcers were delivered. These results are discussed in terms of potential effects of noncontingent reinforcement on problem behavior when the intervention is discontinued or implemented without good treatment integrity.

Resurgence as Choice

Resurgence is typically defined as an increase in a previously extinguished target behavior when a more recently reinforced alternative behavior is later extinguished. Some treatments of the phenomenon have suggested that it might also extend to circumstances where either the historic or more recently reinforced behavior is reduced by other non-extinction related means (e.g., punishment, decreases in reinforcement rate, satiation, etc.). Here we present a theory of resurgence suggesting that the phenomenon results from the same basic processes governing choice. In its most general form, the theory suggests that resurgence results from changes in the allocation of target behavior driven by changes in the values of the target and alternative options across time. Specifically, resurgence occurs when there is an increase in the relative value of an historically effective target option as a result of a subsequent devaluation of a more recently effective alternative option. We develop a more specific quantitative model of how extinction of the target and alternative responses in a typical resurgence paradigm might produce such changes in relative value across time using a temporal weighting rule. The example model does a good job in accounting for the effects of reinforcement rate and related manipulations on resurgence in simple schedules where Behavioral Momentum Theory has failed. We also discuss how the general theory might be extended to other parameters of reinforcement (e.g., magnitude, quality), other means to suppress target or alternative behavior (e.g., satiation, punishment, differential reinforcement of other behavior), and other factors (e.g., non- contingent versus contingent alternative reinforcement, serial alternative reinforcement, and multiple schedules).

Behavioral momentum and resurgence: Effects of time in extinction and repeated resurgence tests

Resurgence is an increase in a previously extinguished operant response that occurs if an alternative reinforcement introduced during extinction is removed. Shahan and Sweeney (2011) developed a quantitative model of resurgence based on behavioral momentum theory that captures existing data well and predicts that resurgence should decrease as time in extinction and exposure to the alternative reinforcement increases. Two experiments tested this prediction. The data from Experiment 1 suggested that without a return to baseline, resurgence decreases with increased exposure to alternative reinforcement and to extinction of the target response. Experiment 2 tested the predictions of the model across two conditions, one with constant alternative reinforcement for five sessions, and the other with alternative reinforcement removed three times. In both conditions, the alternative reinforcement was removed for the final test session. Experiment 2 again demonstrated a decrease in relapse across repeated resurgence tests. Furthermore, comparably little resurgence was observed at the same time point in extinction in the final test, despite dissimilar previous exposures to alternative reinforcement removal. The quantitative model provided a good description of the observed data in both experiments. More broadly, these data suggest that increased exposure to extinction may be a successful strategy to reduce resurgence. The relationship between these data and existing tests of the effect of time in extinction on resurgence is discussed.

Behavioral momentum and accumulation of mass in multiple schedules

Behavioral momentum theory suggests that the relation between a discriminative-stimulus situation and reinforcers obtained in that context (i.e., the Pavlovian stimulus-reinforcer relation) governs persistence of operant behavior. Within the theory, a mass-like aspect of behavior has been shown to be a power function of predisruption reinforcement rates. Previous investigations of resistance to change in multiple schedules, however, have been restricted to examining response persistence following protracted periods of stability in reinforcer rates within a discriminative situation. Thus, it is unclear how long a stimulus-reinforcer relation must be in effect prior to disruption in order to affect resistance to change. The present experiment examined resistance to change of pigeon's key pecking following baseline conditions where reinforcer rates that were correlated with discriminative-stimulus situations changed. Across conditions, one multiple-schedule component arranged either relatively higher rates or lower rates of variable-interval food delivery, while the other component arranged the opposite rate. These schedules alternated between multiple-schedule components across blocks of sessions such that reinforcer rates in the components were held constant for 20, 5, 3, 2, or 1 session(s) between alternations. Resistance to extinction was higher in the component that most recently was associated with higher rates of food delivery in all conditions except when schedules alternated daily or every other day. These data suggest that resistance to change in multiple schedules is related to recently experienced reinforcer rates but only when multiple-schedule components are associated with specific reinforcer rates for several sessions.

Comparison of dehydroepiandrosterone (DHEA) and pregnanolone with existing pharmacotherapies for alcohol abuse on ethanol- and food-maintained responding in male rats

The present study compared two putative pharmacotherapies for alcohol abuse and dependence, dehydroepiandrosterone (DHEA) and pregnanolone, with two Food and Drug Administration (FDA)-approved pharmacotherapies, naltrexone and acamprosate. Experiment 1 assessed the effects of different doses of DHEA, pregnanolone, naltrexone, and acamprosate on both ethanol- and food-maintained responding under a multiple fixed-ratio (FR)-10 FR-20 schedule, respectively. Experiment 2 assessed the effects of different mean intervals of food presentation on responding for ethanol under a FR-10 variable-interval (VI) schedule, whereas Experiment 3 assessed the effects of a single dose of each drug under a FR-10 VI-80 schedule. In Experiment 1, all four drugs dose-dependently decreased response rate for both food and ethanol, although differences in the rate-decreasing effects were apparent among the drugs. DHEA and pregnanolone decreased ethanol-maintained responding more potently than food-maintained responding, whereas the reverse was true for naltrexone. Acamprosate decreased responding for both reinforcers with equal potency. In Experiment 2, different mean intervals of food presentation significantly affected the number of food reinforcers obtained per session; however, changes in the number of food reinforcements did not significantly affect responding for ethanol. Under the FR-10 VI-80 schedule in Experiment 3, only naltrexone significantly decreased both the dose of alcohol presented and blood ethanol concentration (BEC). Acamprosate and pregnanolone had no significant effects on any of the dependent measures, whereas DHEA significantly decreased BEC, but did not significantly decrease response rate or the dose presented. In summary, DHEA and pregnanolone decreased ethanol-maintained responding more potently than food-maintained responding under a multiple FR-10 FR-20 schedule, and were more selective for decreasing ethanol self-administration than either naltrexone or acamprosate under that schedule. Experiment 2 showed that ethanol intake was relatively independent of the interval of reinforcement in the food-maintained component, and Experiment 3 showed that naltrexone was the most effective drug at the doses tested when the interval for food reinforcement was low and maintained under a variable-interval schedule.

Modeling the effects of sensory reinforcers on behavioral persistence with alternative reinforcement

Problem behavior often has sensory consequences that cannot be separated from the target response, even if external, social reinforcers are removed during treatment. Because sensory reinforcers that accompany socially mediated problem behavior may contribute to persistence and relapse, research must develop analog sensory reinforcers that can be experimentally manipulated. In this research, we devised analogs to sensory reinforcers in order to control for their presence and determine how sensory reinforcers may impact treatment efficacy. Experiments 1 and 2 compared the efficacy of differential reinforcement of alternative behavior (DRA) versus noncontingent reinforcement (NCR) with and without analog sensory reinforcers in a multiple schedule. Experiment 1 measured the persistence of key pecking in pigeons, whereas Experiment 2 measured the persistence of touchscreen responses in children with intellectual and developmental disabilities. Across both experiments, the presence of analog sensory reinforcers increased the levels, persistence, and variability of responding relative to when analog sensory reinforcers were absent. Also in both experiments, target responding was less persistent under conditions of DRA compared to NCR regardless of the presence or absence of analog sensory reinforcers.

Reinforcer magnitude and resistance to disruption of forgetting functions and response rates

The present experiment investigated the effects of reinforcer magnitude on resistance to disruption of remembering and response rates. Pigeons were exposed to a variable-interval (VI), delayed-matching-to-sample (DMTS) procedure with two components (rich and lean, distinguished by differing discriminative stimuli and hopper presentation duration). Completion of a VI 20 s schedule resulted in DMTS trials. In a DMTS trial, a choice of one of two comparison stimuli resulted in food if the choice matched the color of the previously presented sample stimulus. Separable aspects of the forgetting functions (initial discrimination and rate of forgetting) were examined by determining accuracy across a range of delays. Response rates and accuracy were higher in the rich relative to the lean component during baseline, and were more persistent during disruptors (extinction and prefeeding). During DMTS trials, extinction decreased initial discrimination more in the lean than the rich component, but had no systematic effect on rate of forgetting. During prefeeding, the rate of forgetting increased more in the lean than the rich component, but initial discrimination was not systematically affected. These results show persistence of response rates and remembering are positively related to reinforcer magnitude. The type of disruptor also influences the way in which remembering is disrupted.

Effects of naltrexone on alcohol drinking patterns and extinction of alcohol seeking in baboons

RATIONALE

Understanding naltrexone's effect on motivation to drink and pattern of drinking is important for better treatment outcomes and for comparison with novel medications.

OBJECTIVES

Naltrexone's effects on number and pattern of seeking, self-administration, and extinction responses were evaluated in two groups of baboons trained under a three-component chained schedule of reinforcement (CSR).

METHODS

Alcohol (4 % w/v; n = 4; alcohol group) or a preferred nonalcoholic beverage (n = 4; control group) was available for self-administration only in component 3 of the CSR. Responses in component 2 provided indices of motivation to drink (seeking). Naltrexone (0.32-3.2 mg/kg) and saline were administered before drinking and component 2 extinction sessions.

RESULTS

Acute doses of naltrexone significantly decreased total self-administration responses (p < 0.01), intake volume (p < 0.001), and grams per kilogram of alcohol (p < 0.01) in the alcohol group only. Pattern of drinking did not change, but the number of drinks during the initial drinking bout was decreased significantly by naltrexone for both groups (p < 0.05). During within-session extinction tests, acute naltrexone significantly decreased time to reach extinction (p < 0.01) and number of seeking responses (p < 0.05), particularly early in the extinction period in the alcohol group only. When administered chronically, naltrexone did not decrease progressive ratio breaking points to gain access to alcohol, but dose dependently reduced alcohol self-administration (p < 0.05) by decreasing the magnitude of the initial drinking bout.

CONCLUSIONS

The results support clinical observations that naltrexone may be most effective at reducing self-administration in the context of ongoing alcohol availability and may reduce motivation to drink in the presence of alcohol-related cues.

Ethanol seeking by Long Evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.

Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers

An extensive body of research using concurrent-chains schedules of reinforcement has shown that choice for one of two differentially valued food-associated stimuli is dependent upon the overall temporal context in which those stimuli are embedded. The present experiments examined whether the concurrent chains procedure was useful for the study of behavior maintained by alcohol and alcohol-associated stimuli. In Experiment 1, rats responded on concurrent-chains schedules with equal variable-interval (VI) 10-s schedules in the initial links. Across conditions, fixed-interval schedules in the terminal links were varied to yield 1∶1, 9∶1, and 1∶9 ratios of alcohol delivery. Initial-link response rates reflected changes in terminal-link schedules, with greater relative responding in the rich terminal link. In Experiment 2, terminal-link schedules remained constant with a 9∶1 ratio of alcohol delivery rates while the length of two equal duration initial-link schedules was varied. Preference for the rich terminal link was less extreme when initial links were longer (i.e., the initial-link effect), as has been previously reported with food reinforcers. This result suggests that the conditioned reinforcing value of an alcohol-associated stimulus depends on the temporal context in which it is embedded. The concurrent-chains procedure and quantitative models of concurrent chains performance may provide a useful framework within which to study how contextual variables modulate preference for drug-associated conditioned reinforcers.

The potency of fluvoxamine to reduce ethanol self-administration decreases with concurrent availability of food

The selective serotonin reuptake inhibitor fluvoxamine reduces responding for ethanol at lower doses than responding for food when each is available in separate components or separate groups of rats. However, when both are available concurrently and deliveries earned per session are equal, this apparent selectivity inverts and food-maintained behavior is more sensitive than ethanol-maintained behavior to rate-decreasing effects of fluvoxamine. Here, we investigated further the impact that concurrent access to both food and ethanol has on the potency of fluvoxamine. Fluvoxamine (5.6-17.8 mg/kg) potency was assessed under conditions in which food and ethanol were available concurrently and response rates were equal [average variable intervals (VIs) 405 and 14 s for food and ethanol, respectively], as well as when density of food delivery was increased (average VI 60 s for food and VI 14 s for ethanol). The potency of fluvoxamine was also determined when only ethanol was available (food extinction and average VI 14 s for ethanol) and under multiple VIs (VI 30 s for food and ethanol) wherein either food or ethanol was the only programmed reinforcement available during each component. Fluvoxamine was less potent at decreasing ethanol self-administration when food was available concurrently {ED50 [95% confidence limit (CL): 8.2 (6.5-10.3) and 10.7 (7.9-14.4)]} versus when ethanol was available in isolation [ED50: 4.0 (2.7-5.9) and 5.1 (4.3-6.0)]. Effects on food were similar under each condition in which food was available. The results demonstrate that the potency of fluvoxamine in reducing ethanol-maintained behavior depends on whether ethanol is available in isolation or in the context of concurrently scheduled food reinforcement.

Behavioral momentum theory: equations and applications

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.

Resistance to extinction and behavioral momentum

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.

A model of resurgence based on behavioral momentum theory

Resurgence is the reappearance of an extinguished behavior when an alternative behavior reinforced during extinction is subsequently placed on extinction. Resurgence is of particular interest because it may be a source of relapse to problem behavior following treatments involving alternative reinforcement. In this article we develop a quantitative model of resurgence based on the augmented model of extinction provided by behavioral momentum theory. The model suggests that alternative reinforcement during extinction of a target response acts as both an additional source of disruption during extinction and as a source of reinforcement in the context that increases the future strength of the target response. The model does a good job accounting for existing data in the resurgence literature and makes novel and testable predictions. Thus, the model appears to provide a framework for understanding resurgence and serves to integrate the phenomenon into the existing theoretical account of persistence provided by behavioral momentum theory. In addition, we discuss some potential implications of the model for further development of behavioral momentum theory.

Resistance to change and frequency of response-dependent stimuli uncorrelated with reinforcement

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response-dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable-interval 60-s schedules of food reinforcement in two components of a multiple schedule, and brief response-dependent keylight-color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable-interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.

Extinction, relapse, and behavioral momentum

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a discriminative stimulus depends upon the baseline rate or magnitude of reinforcement associated with that stimulus (i.e., the Pavlovian stimulus-reinforcer relation). Recently, we have shown that relapse of operant behavior in reinstatement, resurgence, and context renewal preparations also is a function of baseline stimulus-reinforcer relations. In this paper we present new data examining the role of baseline stimulus-reinforcer relations on resistance to extinction and relapse using a variety of baseline training conditions and relapse operations. Furthermore, we evaluate the adequacy of a behavioral momentum based model in accounting for the results. The model suggests that relapse occurs as a result of a decrease in the disruptive impact of extinction precipitated by a change in circumstances associated with extinction, and that the degree of relapse is a function of the pre-extinction baseline Pavlovian stimulus-reinforcer relation. Across experiments, relative resistance to extinction and relapse were greater in the presence of stimuli associated with more favorable conditions of reinforcement and were positively related to one another. In addition, the model did a good job in accounting for these effects. Thus, behavioral momentum theory may provide a useful quantitative approach for characterizing how differential reinforcement conditions contribute to relapse of operant behavior.

Translational research in behavior analysis: historical traditions and imperative for the future

"Pure basic" science can become detached from the natural world that it is supposed to explain. "Pure applied" work can become detached from fundamental processes that shape the world it is supposed to improve. Neither demands the intellectual support of a broad scholarly community or the material support of society. Translational research can do better by seeking innovation in theory or practice through the synthesis of basic and applied questions, literatures, and methods. Although translational thinking has always occurred in behavior analysis, progress often has been constrained by a functional separation of basic and applied communities. A review of translational traditions in behavior analysis suggests that innovation is most likely when individuals with basic and applied expertise collaborate. Such innovation may have to accelerate for behavior analysis to be taken seriously as a general-purpose science of behavior. We discuss the need for better coordination between the basic and applied sectors, and argue that such coordination compromises neither while benefiting both.

Conditioned reinforcement and response strength

Stimuli associated with primary reinforcers appear themselves to acquire the capacity to strengthen behavior. This paper reviews research on the strengthening effects of conditioned reinforcers within the context of contemporary quantitative choice theories and behavioral momentum theory. Based partially on the finding that variations in parameters of conditioned reinforcement appear not to affect response strength as measured by resistance to change, long-standing assertions that conditioned reinforcers do not strengthen behavior in a reinforcement-like fashion are considered. A signposts or means-to-an-end account is explored and appears to provide a plausible alternative interpretation of the effects of stimuli associated with primary reinforcers. Related suggestions that primary reinforcers also might not have their effects via a strengthening process are explored and found to be worthy of serious consideration.

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.

Behavioral momentum of cocaine self-administration: effects of frequency of reinforcement on resistance to extinction

Persistent drug seeking is a defining property of substance abuse and is generally thought to depend, in part, on exposure to drug-associated contexts. Behavioral momentum theory provides a set of methods and a theoretical framework for understanding how stimulus contexts contribute to the persistence of operant behavior. Earlier research has extended behavioral momentum theory to alcohol self-administration, but not to intravenous drug self-administration. This experiment extended behavioral momentum theory to cocaine self-administration by examining the effects of frequency of cocaine reinforcement in a context on resistance to extinction. Rats self-administered 0.32 mg/kg infusions of cocaine in a multiple schedule of reinforcement arranging two distinct contexts. Responding in a Rich context was reinforced by cocaine infusions at a higher frequency (i.e. variable interval 120 s) and in a Lean context at a lower frequency (variable interval 360 s). After establishment of responding in the two contexts, resistance to extinction was examined. Preextinction response rates for cocaine were similar in the Rich and Lean contexts. Nonetheless, relative resistance to extinction was greater in the Rich context than in the Lean context. The difference in resistance to extinction in the two contexts was well described by a quantitative model of behavioral momentum. These results suggest that the frequency of drug reinforcement in a context contributes to the persistence of drug seeking in that context, and that behavioral momentum theory might be useful for understanding how drug-associated contexts contribute to the persistence of drug seeking.

Examination of reinforcement magnitude on the pharmacological disruption of fixed-ratio performance

Behavioral momentum theory proposes that operant behavior is the product of two separable processes: its rate of occurrence and its resistance to change. Generally speaking, operant situations providing more densely spaced or greater magnitudes of reinforcement should be more resistant to disruption. Attempts to disrupt ongoing behavior by manipulating the availability of food or deprivation level typically have supported the predictions of behavioral momentum. Tests with pharmacological disruptors, however, have yielded mixed results. Most investigations of pharmacological disruption of operant behavior have evaluated momentum across situations that differ in rate of reinforcement. The present experiment was an attempt to systematically replicate prior work, but under conditions of differing reinforcement magnitudes. Pigeons were trained to key peck on a multiple fixed-ratio 30 schedule of food presentation, where different components programmed 2-, 4-, or 8-s access to grain. Resistance to rate-decreasing effects of drugs was evaluated with several compounds drawn from distinct pharmacological classes: chlordiazepoxide, cocaine, clonidine, haloperidol, morphine, and ethanol were tested. Additionally, disruption by prefeeding and extinction was examined. Generally, resistance to change by drug administration was not modulated by reinforcement magnitude. Prefeeding and extinction tests, however, replicated previous work, indicating that our procedure was sensitive to more common disruptors. The results give additional support to the notion that pharmacological disruptors may not behave in the manner predicted by behavioral momentum theory.

Latent structure of facets of alcohol reinforcement from a behavioral economic demand curve

RATIONALE

Behavioral economic demand curves are quantitative representations of the relationship between consumption of a drug and its cost. Demand curves provide a multidimensional assessment of reinforcement, but the relationships among the various indices of reinforcement have been largely unstudied.

OBJECTIVES

The objective of the study is to use exploratory factor analysis to examine the underlying factor structure of the facets of alcohol reinforcement generated from an alcohol demand curve.

MATERIALS AND METHODS

Participants were 267 weekly drinkers [76% female; age M = 20.11 (SD = .1.51); drinks/week M = 14.33 (SD = 11.82)] who underwent a single group assessment session. Alcohol demand curves were generated via an alcohol purchase task, which assessed consumption at 14 levels of prices from $0 to $9. Five facets of demand were generated from the measure [intensity, elasticity, P (max) (maximum inelastic price), O (max) (maximum alcohol expenditure), and breakpoint], using both observed and derived calculations. Principal components analysis was used to examine the latent structure among the variables.

RESULTS

The results revealed a clear two-factor solution, which were interpreted as "Persistence," reflecting sensitivity to escalating price, and "Amplitude," reflecting the amount consumed and spent. The two factors were generally quantitatively distinct, although O (max) loaded on both.

CONCLUSIONS

These findings suggest that alcohol reinforcement as measured via a demand curve is binary in nature, with separate dimensions of price-sensitivity and volumetric consumption. If supported, these findings may contribute theoretically and experimentally to a reinforcement-based approach to alcohol use and misuse.

Stimuli, reinforcers, and the persistence of behavior

This article reviews evidence from basic and translational research with pigeons and humans suggesting that the persistence of operant behavior depends on the contingency between stimuli and reinforcers, and considers some implications for clinical interventions.

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.

Differential effects of nucleus accumbens core, shell, or dorsal striatal inactivations on the persistence, reacquisition, or reinstatement of responding for a drug-paired conditioned reinforcer

Drug-paired conditioned reinforcers can maintain persistent instrumental responding, thus providing a model of some aspects of long-term drug addiction. The purpose of the present study was to investigate the effects of inactivating the dorsal striatum (DStr), nucleus accumbens (NAcc) core, or NAcc shell on different types of responding, each maintained by drug-paired conditioned reinforcers. Inactivations were achieved by infusing a combination of baclofen and muscimol prior to (1) persistent responding for a drug-paired conditioned reinforcer, (2) reacquisition of this instrumental response after extinction by omission of the contingent conditioned stimulus (CS), or (3) CS (cue)-induced reinstatement of the original (and different) instrumental response that had previously delivered cocaine. Inactivation of the DStr attenuated persistent responding for a cocaine-paired conditioned reinforcer, as well as its reacquisition after extinction of this response, while the only effect of inactivation of the NAcc shell was to increase CS (cue)-induced reinstatement of the extinguished instrumental response that had previously delivered cocaine. Inactivation of the NAcc core affected all measures of responding maintained by drug-paired conditioned reinforcers. These results are discussed with reference to the neural systems involved in different aspects of responding maintained by drug-paired conditioned reinforcers.

Facilitated acquisition but not persistence of responding for a cocaine-paired conditioned reinforcer following sensitization with cocaine

Sensitization has been hypothesized to increase the incentive salience of drug-paired conditioned stimuli and in the present study the ability of a sensitizing pretreatment with cocaine to increase responding for a drug-paired conditioned reinforcer was tested. In support of the incentive-sensitization hypothesis, sensitized rats earned more presentations of a drug-paired conditioned reinforcer during acquisition of a new response for this stimulus. By comparison, sensitization had no effect on the number of CSs earned during reversal of the contingency or following pretreatment with d-amphetamine. During reversal learning, however, sensitized rats were impaired in the extinction of the inappropriate response once the contingent CS was no longer available following presses on that lever. The results are discussed with reference to a possible role of increased incentive salience of a drug-paired CS to the formation of a habit.

Quantitative analyses of observing and attending

We review recent experiments examining whether simple models of the allocation and persistence of operant behavior are applicable to attending. In one series of experiments, observing responses of pigeons were used as an analog of attending. Maintenance of observing is often attributed to the conditioned reinforcing effects of a food-correlated stimulus (i.e., S+), so these experiments also may inform our understanding of conditioned reinforcement. Rates and allocations of observing were governed by rates of food or S+ delivery in a manner consistent with the matching law. Resistance to change of observing was well described by behavioral momentum theory only when rates of primary reinforcement in the context were considered. Rate and value of S+ deliveries did not affect resistance to change. Thus, persistence of attending to stimuli appears to be governed by primary reinforcement rates in the training context rather than conditioned reinforcing effects of the stimuli. An additional implication of these findings is that conditioned "reinforcers" may affect response rates through some mechanism other than response-strengthening. In a second series of experiments, we examined the applicability of the matching law to the allocation of attending to the elements of compound stimuli in a divided-attention task. The generalized matching law described performance well, and sensitivity to relative reinforcement varied with sample duration. The bias and sensitivity terms of the generalized matching law may provide measures of stimulus-driven and goal-driven control of divided attention. Further application of theories of operant behavior to performance on attention tasks may provide insights into what is referred to variously as endogenous, top-down, or goal-directed control of attention.

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.

Resistance to change of alcohol self-administration: effects of alcohol-delivery rate on disruption by extinction and naltrexone

A common finding in resistance to change research with food-maintained operant behavior is that the persistence of behavior depends on the rate of reinforcement delivered in the context in which the behavior occurs. The present experiment evaluated the effects of rate of response-dependent alcohol delivery on the resistance to change of rats' alcohol self-administration in the face of disruption produced by extinction and a range of doses of naltrexone (1.0, 3.0, 10.0 mg/kg, subcutaneous). Rats responded for a 10% alcohol solution in a multiple schedule of reinforcement arranging a higher rate of alcohol delivery (variable interval 15 s) in the presence of one stimulus and a lower rate of alcohol delivery (variable interval 45 s) in the presence of another stimulus. Baseline response rates and resistance to extinction were higher in the presence of the stimulus associated with higher rates of alcohol delivery. This finding is consistent with studies of the resistance to change of food-maintained behavior. The rate of alcohol delivered in the components, however, did not systematically affect resistance to disruption by naltrexone. One interpretation of this finding from the perspective of behavioral momentum theory is that naltrexone may decrease the impact of alcohol-associated stimuli on the persistence of drinking by reducing sensitivity to the relative reinforcement conditions arranged in the presence of different stimuli.

Interoceptive Pavlovian conditioning with nicotine as the conditional stimulus varies as a function of the number of conditioning trials and unpaired sucrose deliveries

In rats, the pharmacological (interoceptive) effects of nicotine can serve as a signal (conditional stimulus) in a Pavlovian (classical) conditioning task. In this task, nicotine administration (0.4 mg base/kg, subcutaneous) is typically paired with intermittent access to a liquid sucrose unconditional stimulus; sucrose is withheld on saline sessions. An increase in sucrose receptacle entries (goal tracking) on nicotine sessions indicates conditioning. Given our limited understanding of the functional relationships controlling conditioned responding to a nicotine conditional stimulus, the present research examined nicotine's sensitivity to several manipulations shown to affect the conditioned responding in more widely studied Pavlovian conditioning tasks that use exteroceptive conditional stimuli: number of nicotine conditional stimulus-sucrose unconditional stimulus pairings per session (0, 3, 9, 18, or 36) and the impact of sucrose deliveries in saline sessions. Differential goal tracking developed in fewer sessions and asymptotic conditioned responding magnitude was greater with more nicotine-sucrose pairings. Further, goal tracking was more resistant to extinction (unconditional stimulus withheld) with more conditional-unconditional stimulus pairings during the acquisition phase. The discrimination was not acquired when sucrose presentations (9 or 18) also occurred during saline sessions. Furthermore, expression of the discrimination was disrupted when sucrose was presented in saline sessions; this disruption resulted from goal tracking in saline sessions. These results are consistent with the notion that nicotine-evoked goal tracking results from interoceptive conditioning processes.

Fluvoxamine effects on concurrent ethanol- and food-maintained behaviors

In previous studies, the selective serotonin reuptake inhibitor fluvoxamine preferentially reduced responding for ethanol compared with responding for food under conditions in which each was available alone in separate groups or in the same subjects under a multiple schedule in which baseline response rates were matched. The impact of providing concurrent access to food on pharmacological effects on ethanol self-administration remains largely unexplored. In this study, acute doses of fluvoxamine (3.0-17.8 mg/kg) were administered 30 min before the experimental session to Lewis rats responding under a concurrent fixed-ratio, fixed-ratio schedule of ethanol and food presentation. Ratios for food were adjusted for each subject to provide matched rates of food and ethanol reinforcement across the 30-min session. Although the number of ethanol and food deliveries did not significantly differ under baseline conditions, response rates did differ. Following fluvoxamine administration, responding for food was decreased more than responding for ethanol. This differential effect did not appear to be related to response rate or fixed-ratio size. Thus, the selectivity of fluvoxamine on ethanol- versus food-maintained responding depends on the context in which the behavior occurs. Such results may help explain inconsistencies between preclinical results and those in humans, and could provide insight into the behavioral determinants of pharmacological effects on ethanol self-administration.

A theory of attending and reinforcement in conditional discriminations

A model of conditional discrimination performance (Davison & Nevin, 1999) is combined with the notion that unmeasured attending to the sample and comparison stimuli, in the steady state and during disruption, depends on reinforcement in the same way as predicted for overt free-operant responding by behavioral momentum theory (Nevin & Grace, 2000). The rate of observing behavior, a measurable accompaniment of attending, is well described by an equation for steady-state responding derived from momentum theory, and the resistance to change of observing conforms to predictions of momentum theory, supporting a key assumption of the model. When probabilities of attending are less than 1.0, the model accounts for some aspects of conditional-discrimination performance that posed problems for the Davison-Nevin model: (a) the effects of differential reinforcement on the allocation of responses to the comparison stimuli and on accuracy in several matching-to-sample and signal-detection tasks where the differences between the stimuli or responses were varied across conditions, (b) the effects of overall reinforcer rate on the asymptotic level and resistance to change of both response rate and accuracy of matching to sample in multiple schedules, and (c) the effects of fixed-ratio reinforcement on accuracy. Some tests and extensions of the model are suggested, and the role of unmeasured events in behavior theory is considered.