Measuring behavioral momentum

Resistance to change, of behavior and of theory

The persistence of operant behavior when disrupted tends to be positively related to how often reinforcers were delivered in the past. Behavioral momentum theory describes this finding as the outcome of Pavlovian processes. That is, the relation between discriminative stimuli and reinforcers that were delivered in their presence strengthens behavior, thereby making it more likely to persist. If only the story were that simple. A growing number of findings challenge the basic tenets of behavioral momentum theory. Some even call into question whether Pavlovian relations contribute to persistence in the first place. In this paper, I will review behavioral momentum theory and some of the data that have been problematic for the theory. I will argue that despite these very real challenges, the theory provides important utility not only to basic analyses of response persistence but also to clinical interventions directed at long-term reductions in problem behavior. It, for example, has set the stage for the development of alternative conceptual analyses of resistance to change, two of which will be highlighted for readers. Moreover, behavioral momentum theory may tell us something important about the reasons it continues to have an influence on the field, despite the challenging data that deter it.

Mechanisms of within-session sequential behavior in pigeons

Correctly and efficiently selecting among options is critical to the organization of behavior across different time scales (minutes, days, seasons). As a result, understanding the mechanisms underlying the sequential behavior of animals has been a long-standing aim. In three experiments, four pigeons were tested in a four-choice simultaneous color discrimination. Across a session, they had to sequentially select a colored stimulus, and the correct color changed over four 24-trial phases (A→B→C→D). After learning this ABCD within-session sequence, tests identified that both timing and outcome feedback mechanisms contributed to the organization of pigeons' behavior. Different representational mechanisms are considered as accounts for the pigeons' observed sequential behavior.

Nondrug reinforcers contingent on alternative behavior or abstinence increase resistance to extinction and reinstatement of ethanol-maintained behavior

The effects of delivering nondrug alternative reinforcement on resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing were evaluated in two experiments. In both, rats self-administered ethanol by lever pressing in a two-component multiple schedule during baseline. In the Rich component, alternative food reinforcement was made available for performing an alternative response (Experiment 1) or according to a differential-reinforcement-of-other-behavior schedule for lever pressing (Experiment 2). In the Lean component, only ethanol was available. Moreover, the frequency of alternative reinforcement was manipulated across conditions in Experiment 1. Following baseline, lever pressing was extinguished in both components by suspending ethanol reinforcement, and alternative food reinforcers were discontinued. Finally, to test for reinstatement, ethanol reinforcers were delivered independently of lever pressing in both components. In both experiments, proportion-of-baseline response rates were higher during extinction and reinstatement testing in the Rich component than in the Lean component (although differentiation was not observed at the lowest frequency of alternative reinforcement in Experiment 1). Thus, alternative nondrug reinforcers increased resistance to extinction and reinstatement of rats' ethanol-maintained lever pressing, even when those reinforcers were delivered contingently on an alternative response or on abstinence from lever pressing.

Effects of dose on acquisition and persistence of a new response for a remifentanil-associated stimulus

Previous research demonstrated that a remifentanil-associated stimulus facilitated the acquisition of a previously unlearned response; however, it is unclear how long a remifentanil-associated stimulus maintains conditioned reinforcing properties under conditions of daily testing. To address this gap, we exposed adult male rats to response-independent stimulus presentations and deliveries of remifentanil (1.0, 3.2, or 10.0 μg/kg/infusion). Rats either received the stimulus presentations and remifentanil deliveries together (Paired Pavlovian conditioning) or according to separate clocks (Random control group). In the sessions following Pavlovian conditioning, we allowed rats to emit nose-poke responses for the presentation of the stimulus alone and measured the extent to which the stimulus facilitated and maintained a previously unlearned response. We tested responding for the stimulus presentations across 28 daily sessions to assess the Pavlovian extinction (degradation of the drug-stimulus association) of the conditioned reinforcing properties of the remifentanil-associated stimulus. We observed the highest and most persistent levels of responding in rats with a Paired Pavlovian conditioning history at 3.2 and 10.0 μg/kg/infusion. In addition, we included analyses of the variability in responding for each group, which revealed individual differences in the susceptibility of the remifentanil-associated stimulus acting as a conditioned reinforcer. These findings demonstrate that a remifentanil-associated stimulus has the ability to sustain drug-seeking behavior and underscores the importance of Pavlovian conditioning in promoting drug abuse.

Relational Density Theory: Nonlinearity of Equivalence Relating Examined through Higher-Order Volumetric-Mass-Density

We propose relational density theory, as an integration of stimulus equivalence and behavioral momentum theory, to predict the nonlinearity of equivalence responding of verbal humans. Consistent with Newtonian classical mechanics, the theory posits that equivalence networks will demonstrate the higher order properties of density, volume, and mass. That is, networks containing more relations (volume) that are stronger (density) will be more resistant to change (i.e., contain greater mass; mass = volume * density). Data from several equivalence experiments that are not easily interpreted through existing accounts are described in terms of the theory, generating predictable results in most cases. In addition, we put forward the higher-order properties of relational acceleration and gravity, which follow directly from the theory and may inspire future researchers to evaluate the seemingly self-organizing nature of human cognition. Finally, we conclude by describing avenues for real-world translation, considering past research interpreted through relational density theory, and call for basic experimental research to validate and extend core theoretical assumptions.

Effects of 5-HT2C, 5-HT1A receptor challenges and modafinil on the initiation and persistence of gambling behaviours

RATIONALE

Problematic patterns of gambling are characterised by loss of control and persistent gambling often to recover losses. However, little is known about the mechanisms that mediate initial choices to begin gambling and then continue to gamble in the face of losing outcomes.

OBJECTIVES

These experiments first assessed gambling and loss-chasing performance under different win/lose probabilities in C57Bl/6 mice, and then investigated the effects of antagonism of 5-HT_2CR with SB242084, 5-HT_1AR agonism with 8-OH-DPAT and modafinil, a putative cognitive enhancer.

RESULTS

As seen in humans and other species, mice demonstrated the expected patterns of behaviour as the odds for winning were altered increasing gambling and loss-chasing when winning was more likely. SB242084 decreased the likelihood to initially gamble, but had no effects on subsequent gambling choices in the face of repeated losses. In contrast, 8-OH-DPAT had no effects on choosing to gamble in the first place, but once started 8-OH-DPAT increased gambling choices in a dose-sensitive manner. Modafinil effects were different to the serotonergic drugs in both decreasing the propensity to initiate gambling and chase losses.

CONCLUSIONS

We present evidence for dissociable effects of systemic drug administration on different aspects of gambling behaviour. These data extend and reinforce the importance of serotonergic mechanisms in mediating discrete components of gambling behaviour. They further demonstrate the ability of modafinil to reduce gambling behaviour. Our work using a novel mouse paradigm may be of utility in modelling the complex psychological and neurobiological underpinnings of gambling problems, including the analysis of genetic and environmental factors.

Re-exposure to reinforcement in context a during treatment in context b reduces ABC renewal

Previous work from our laboratory showed that intermittently re-exposing rats to reinforcement for lever pressing in a training (A) context, while eliminating lever pressing in a second (B) context, increased ABA renewal of lever pressing relative to rats that experienced only Context B during response elimination. In the current study, we replicated these procedures while assessing renewal in the presence of a novel context (i.e., ABC renewal). Unlike the findings described above, renewal was reduced in the group that experienced re-exposure to Context A during lever-press elimination relative to rats that experienced only Context B. These findings suggest that alternating between contexts associated with reinforcement and extinction during treatment reduces the probability that organisms will respond in novel contexts. These outcomes may be the result of discrimination and/or generalization processes. Moreover, this training procedure may offer a potential mitigation strategy for ABC renewal.

Behavioral momentum and resistance to extinction across repeated extinction tests

The present experiments assessed whether resistance to extinction of pigeons' key pecking decreased across repeated extinction tests. An additional impetus for this research was to determine how the quantitative framework provided by behavioral momentum theory might be used to describe any such changes across tests. Pigeons pecked keys in two-component multiple schedules (one component associated with a higher reinforcer rate and the other with a lower rate) in which baseline and extinction conditions alternated. In Experiment 1, baseline and extinction conditions alternated every session, and, in Experiment 2, these conditions lasted for 10 and 7 sessions, respectively. Resistance to extinction decreased across successive extinction conditions in both experiments. Fits of the behavioral-momentum based model of extinction to the data returned uncertain results in Experiment 1 but implicated both generalization decrement and response-reinforcer contingency termination as the possible mechanisms responsible for behavior change in Experiment 2. Thus, these data suggest that experimental manipulations that affect discrimination of changes in reinforcement contingencies may influence resistance to extinction by modulating the disruptive impacts of removing reinforcers from the experimental context and of suspending response-reinforcer contingencies.

Any reward will do: Effects of a reverse-reward contingency on size preference with pet dogs (Canis lupus familiaris)

The reverse-reward contingency (RRC) task involves presenting subjects with a choice between one plate containing a large amount of food and a second plate containing a small amount of food. Subjects are then required to select the smaller of the two options in order to receive the larger-magnitude reward. The RRC task is a commonly used paradigm for assessing complex cognition, such as inhibitory control, in subjects. To date, the RRC task has not been tested with pet dogs as subjects, and it may provide insights to their ability to perceive quantities of differing magnitudes. Nine dogs were tested in an RRC task involving three conditions. In Condition 1, plates of food were presented, and the dogs were allowed to consume their choice. In Condition 2, plates with different-sized symbols resembling the quantities of food in Condition 1 were presented, and dogs received food quantities of the same size as their choice (e.g., a larger-magnitude reward for selecting the plate with the larger shape). In Condition 3, the same plates were presented, but dogs received a reverse-sized quantity of food, relative to their choice (e.g., a smaller-magnitude reward for selecting the plate with the larger shape). A novel addition here to the traditional RRC task was the inclusion of a third, empty (control) plate that was present throughout all conditions, and no programmed consequences were provided when that plate was selected. Our results were consistent with the previous RRC literature: All dogs developed and maintained a preference for the larger stimulus option across conditions. The use of symbolic representations did not ameliorate performance on the RRC task. Applied implications are discussed.

Optimisation of cognitive performance in rodent operant (touchscreen) testing: Evaluation and effects of reinforcer strength

Operant testing is a widely used and highly effective method of studying cognition in rodents. Performance on such tasks is sensitive to reinforcer strength. It is therefore advantageous to select effective reinforcers to minimize training times and maximize experimental throughput. To quantitatively investigate the control of behavior by different reinforcers, performance of mice was tested with either strawberry milkshake or a known powerful reinforcer, super saccharin (1.5% or 2% (w/v) saccharin/1.5% (w/v) glucose/water mixture). Mice were tested on fixed (FR)- and progressive-ratio (PR) schedules in the touchscreen-operant testing system. Under an FR schedule, both the rate of responding and number of trials completed were higher in animals responding for strawberry milkshake versus super saccharin. Under a PR schedule, mice were willing to emit similar numbers of responses for strawberry milkshake and super saccharin; however, analysis of the rate of responding revealed a significantly higher rate of responding by animals reinforced with milkshake versus super saccharin. To determine the impact of reinforcer strength on cognitive performance, strawberry milkshake and super saccharin-reinforced animals were compared on a touchscreen visual discrimination task. Animals reinforced by strawberry milkshake were significantly faster to acquire the discrimination than animals reinforced by super saccharin. Taken together, these results suggest that strawberry milkshake is superior to super saccharin for operant behavioral testing and further confirms that the application of response rate analysis to multiple ratio tasks is a highly sensitive method for the detection of behavioral differences relevant to learning and motivation.

Piloting the addition of contingency management to best practice counselling as an adjunct treatment for rural and remote disordered gamblers: study protocol

INTRODUCTION

Problematic gambling is a significant Canadian public health concern that causes harm to the gambler, their families, and society. However, a significant minority of gambling treatment seekers drop out prior to the issue being resolved; those with higher impulsivity scores have the highest drop-out rates. Consequently, retention is a major concern for treatment providers. The aim of this study is to investigate the efficacy of internet-delivered cognitive behavioural therapy (CBT) and internet-delivered CBT and contingency management (CM+) as treatments for gambling disorder in rural Albertan populations. Contingency management (CM) is a successful treatment approach for substance dependence that uses small incentives to reinforce abstinence. This approach may be suitable for the treatment of gambling disorder. Furthermore, internet-delivered CM may hold particular promise in rural contexts, as these communities typically struggle to access traditional clinic-based counselling opportunities.

METHODS AND ANALYSIS

54 adults with gambling disorder will be randomised into one of two conditions: CM and CBT (CM+) or CBT alone (CBT). Gambling will be assessed at intake, every treatment session, post-treatment, and follow-up. The primary outcome measures are treatment attendance, gambling abstinence, gambling, gambling symptomatology, and gambling urge. In addition, qualitative interviews assessing study experiences will be conducted with the supervising counsellor, graduate student counsellors, study affiliates, and a subset of treatment seekers. This is the first study to use CM as a treatment for gambling disorder in rural and remote populations.

ETHICS AND DISSEMINATION

This study was approved by the University of Lethbridge's Human Subject Research Committee (#2016-080). The investigators plan to publish the results from this study in academic peer-reviewed journals. Summary information will be provided to the funder.

TRIAL REGISTRATION NUMBER

NCT02953899; Pre-results.

Multiple schedules, off-baseline reinforcement shifts, and resistance to extinction

Resistance to extinction in a target multiple-schedule component varies inversely with the rate of reinforcement arranged in an alternative component during baseline. The present experiment asked whether changing the reinforcer rate in an alternative component would impact extinction of target component responding if those changes occurred in an off-baseline phase during which the target component was never experienced. Pigeons' key pecking was studied in three types of conditions, and each condition consisted of three phases. In Phase 1, pecking produced food in the target and alternative components of a multiple schedule according to variable-interval 60-s schedules. In Phase 2, the alternative-component stimulus was presented alone in a single schedule. Pecking during this phase produced the same reinforcer rate as in baseline in the Control condition, a higher rate of food (variable-interval 15 s) in the High-Rate condition, or was extinguished in the Extinction condition. Extinction of target- and alternative-component key pecking then was assessed in a multiple schedule during the final phase of each condition. Resistance to extinction of target-component key pecking was the same between the Control and High-Rate conditions but lower in the Extinction condition. These findings are discussed in terms of discrimination and generalization processes.

Response-reinforcer dependency and resistance to change

The effects of the response-reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two-component multiple schedule. Across conditions, in the fixed component, all reinforcers were response-dependent; in the alternative component, the percentage of response-dependent reinforcers was 100, 50 (i.e., 50% response-dependent and 50% response-independent) or 10% (i.e., 10% response-dependent and 90% response-independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three-component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response-dependent or independent reinforcers. In the other component, response-dependent and -independent reinforcers were programmed by superimposing a variable-time schedule on an independent variable-interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple-schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response-reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.

Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: Fits and misfits

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence

The behavioral-momentum model of resurgence predicts reinforcer rates within a resurgence preparation should have three effects on target behavior. First, higher reinforcer rates in baseline (Phase 1) produce more persistent target behavior during extinction plus alternative reinforcement. Second, higher rate alternative reinforcement during Phase 2 generates greater disruption of target responding during extinction. Finally, higher rates of either reinforcement source should produce greater responding when alternative reinforcement is suspended in Phase 3. Recent empirical reports have produced mixed results in terms of these predictions. Thus, the present experiment further examined reinforcer-rate effects on persistence and resurgence. Rats pressed target levers for high-rate or low-rate variable-interval food during Phase 1. In Phase 2, target-lever pressing was extinguished, an alternative nose-poke became available, and nose-poking produced either high-rate variable-interval, low-rate variable-interval, or no (an extinction control) alternative reinforcement. Alternative reinforcement was suspended in Phase 3. For groups that received no alternative reinforcement, target-lever pressing was less persistent following high-rate than low-rate Phase-1 reinforcement. Target behavior was more persistent with low-rate alternative reinforcement than with high-rate alternative reinforcement or extinction alone. Finally, no differences in Phase-3 responding were observed for groups that received either high-rate or low-rate alternative reinforcement, and resurgence occurred only following high-rate alternative reinforcement. These findings are inconsistent with the momentum-based model of resurgence. We conclude this model mischaracterizes the effects of reinforcer rates on persistence and resurgence of operant behavior.

Experience with dynamic reinforcement rates decreases resistance to extinction

The ability of organisms to detect reinforcer-rate changes in choice preparations is positively related to two factors: the magnitude of the change in rate and the frequency with which rates change. Gallistel (2012) suggested similar rate-detection processes are responsible for decreases in responding during operant extinction. Although effects of magnitude of change in reinforcer rate on resistance to extinction are well known (e.g., the partial-reinforcement-extinction effect), effects of frequency of changes in rate prior to extinction are unknown. Thus, the present experiments examined whether frequency of changes in baseline reinforcer rates impacts resistance to extinction. Pigeons pecked keys for variable-interval food under conditions where reinforcer rates were stable and where they changed within and between sessions. Overall reinforcer rates between conditions were controlled. In Experiment 1, resistance to extinction was lower following exposure to dynamic reinforcement schedules than to static schedules. Experiment 2 showed that resistance to presession feeding, a disruptor that should not involve change-detection processes, was unaffected by baseline-schedule dynamics. These findings are consistent with the suggestion that change detection contributes to extinction. We discuss implications of change-detection processes for extinction of simple and discriminated operant behavior and relate these processes to the behavioral-momentum based approach to understanding extinction.

Behavioral momentum and accumulation of mass in multiple schedules

Behavioral momentum theory suggests that the relation between a discriminative-stimulus situation and reinforcers obtained in that context (i.e., the Pavlovian stimulus-reinforcer relation) governs persistence of operant behavior. Within the theory, a mass-like aspect of behavior has been shown to be a power function of predisruption reinforcement rates. Previous investigations of resistance to change in multiple schedules, however, have been restricted to examining response persistence following protracted periods of stability in reinforcer rates within a discriminative situation. Thus, it is unclear how long a stimulus-reinforcer relation must be in effect prior to disruption in order to affect resistance to change. The present experiment examined resistance to change of pigeon's key pecking following baseline conditions where reinforcer rates that were correlated with discriminative-stimulus situations changed. Across conditions, one multiple-schedule component arranged either relatively higher rates or lower rates of variable-interval food delivery, while the other component arranged the opposite rate. These schedules alternated between multiple-schedule components across blocks of sessions such that reinforcer rates in the components were held constant for 20, 5, 3, 2, or 1 session(s) between alternations. Resistance to extinction was higher in the component that most recently was associated with higher rates of food delivery in all conditions except when schedules alternated daily or every other day. These data suggest that resistance to change in multiple schedules is related to recently experienced reinforcer rates but only when multiple-schedule components are associated with specific reinforcer rates for several sessions.

Response strength in extreme multiple schedules

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.

Behavioral momentum theory: equations and applications

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.

Resistance to extinction and behavioral momentum

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.

An evaluation of persistence of treatment effects during long-term treatment of destructive behavior

Eight young children who displayed destructive behavior maintained, at least in part, by negative reinforcement received long-term functional communication training (FCT). During FCT, the children completed a portion of a task and then touched a communication card attached to a microswitch to obtain brief breaks. Prior to and intermittently throughout FCT, extinction probes were conducted within a withdrawal design in which task completion, manding, and destructive behavior were placed on extinction to evaluate the relative persistence of appropriate and destructive behavior over the course of treatment. FCT continued until appropriate behavior persisted and destructive behavior failed to recur at baseline levels during extinction probes. The completion of FCT was followed by four challenges to the persistence of treatment effects conducted within mixed- or multiple-schedule designs: (a) extended extinction sessions (from 5 to 15 min), (b) introduction of a novel task, (c) removal of the microswitch and communication card, and (d) a mixed schedule of reinforcement in which both appropriate and destructive behavior produced reinforcement. The results showed that although FCT often resulted in quick reductions in destructive behavior and increases in appropriate behavior, destructive behavior often recurred during the extinction probes conducted during the initial treatment. When the effects of treatment persisted during the extinction probes, the remaining challenges to treatment effects resulted in only mild to moderate disruptions in behavior. These results are consistent with the quantitative predictions of behavioral momentum theory and may provide an alternative definition of maintenance as constituting behavioral persistence.

Temporal context, preference, and resistance to change

According to behavioral momentum theory, preference and relative resistance to change in concurrent-chains schedules are correlated and reflect the relative conditioned value of discriminative stimuli. In the present study, we explore the generality of this relation by manipulating the temporal context within a concurrent-chains procedure through changes in the duration of the initial links. Consistent with previous findings, preference for a richer terminal link was less extreme with longer initial links across three experiments with pigeons. In Experiment 1, relative resistance to change and preference were related inversely when responding was disrupted with response-independent food presentations during initial links, replicating a previous finding with rats. However, more food was presented with longer initial links, confounding the disrupter and initial-link duration. In Experiment 2, presession feeding was used instead and eliminated the negative relation between relative resistance to change and preference, but relative resistance to change was not sensitive to relative terminal-link reinforcement rates. In Experiment 3, with more extreme relative terminal-link reinforcement rates, increasing initial-link duration similarly decreased preference and relative resistance to change for the richer terminal link. Thus, when conditions of disruption are equal and assessed under the appropriate reinforcement conditions, changes in temporal context impact relative resistance to change and preference similarly.

Reinforcement magnitude modulation of rate dependent effects in pigeons and rats

Response rate can influence the behavioral effects of many drugs. Reinforcement magnitude may also influence drug effects. Further, reinforcement magnitude can influence rate-dependent effects. For example, in an earlier report, we showed that rate-dependent effects of two antidepressants depended on reinforcement magnitude. The ability of reinforcement magnitude to interact with rate-dependency has not been well characterized. It is not known whether our previous results are specific to antidepressants or generalize to other drug classes. Here, we further examine rate-magnitude interactions by studying effects of two stimulants (d-amphetamine [0.32-5.6 mg/kg] and cocaine [0.32-10 mg/kg]) and two sedatives (chlordiazepoxide [1.78-32 mg/kg] and pentobarbital [1.0-17.8 mg/kg]) in pigeons responding under a 3-component multiple fixed-interval (FI) 300-s schedule maintained by 2-, 4-, or 8-s of food access. We also examine the effects of d-amphetamine [0.32-3.2 mg/kg] and pentobarbital [1.8-10 mg/kg] in rats responding under a similar multiple FI300-s schedule maintained by 2- or 10- food pellet (45 mg) delivery. In pigeons, cocaine and, to a lesser extent, chlordiazepoxide exerted rate-dependent effects that were diminished by increasing durations of food access. The relationship was less apparent for pentobarbital, and not present for d-amphetamine. In rats, rate-dependent effects of pentobarbital and d-amphetamine were not modulated by reinforcement magnitude. In conclusion, some drugs appear to exert rate-dependent effect which are diminished when reinforcement magnitude is relatively high. Subsequent analysis of the rate-dependency data suggest the effects of reinforcement magnitude may be due to a diminution of drug-induced increases in low-rate behavior that occurs early in the fixed-interval.

A model of resurgence based on behavioral momentum theory

Resurgence is the reappearance of an extinguished behavior when an alternative behavior reinforced during extinction is subsequently placed on extinction. Resurgence is of particular interest because it may be a source of relapse to problem behavior following treatments involving alternative reinforcement. In this article we develop a quantitative model of resurgence based on the augmented model of extinction provided by behavioral momentum theory. The model suggests that alternative reinforcement during extinction of a target response acts as both an additional source of disruption during extinction and as a source of reinforcement in the context that increases the future strength of the target response. The model does a good job accounting for existing data in the resurgence literature and makes novel and testable predictions. Thus, the model appears to provide a framework for understanding resurgence and serves to integrate the phenomenon into the existing theoretical account of persistence provided by behavioral momentum theory. In addition, we discuss some potential implications of the model for further development of behavioral momentum theory.

Extinction, relapse, and behavioral momentum

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a discriminative stimulus depends upon the baseline rate or magnitude of reinforcement associated with that stimulus (i.e., the Pavlovian stimulus-reinforcer relation). Recently, we have shown that relapse of operant behavior in reinstatement, resurgence, and context renewal preparations also is a function of baseline stimulus-reinforcer relations. In this paper we present new data examining the role of baseline stimulus-reinforcer relations on resistance to extinction and relapse using a variety of baseline training conditions and relapse operations. Furthermore, we evaluate the adequacy of a behavioral momentum based model in accounting for the results. The model suggests that relapse occurs as a result of a decrease in the disruptive impact of extinction precipitated by a change in circumstances associated with extinction, and that the degree of relapse is a function of the pre-extinction baseline Pavlovian stimulus-reinforcer relation. Across experiments, relative resistance to extinction and relapse were greater in the presence of stimuli associated with more favorable conditions of reinforcement and were positively related to one another. In addition, the model did a good job in accounting for these effects. Thus, behavioral momentum theory may provide a useful quantitative approach for characterizing how differential reinforcement conditions contribute to relapse of operant behavior.

Conditioned reinforcement and response strength

Stimuli associated with primary reinforcers appear themselves to acquire the capacity to strengthen behavior. This paper reviews research on the strengthening effects of conditioned reinforcers within the context of contemporary quantitative choice theories and behavioral momentum theory. Based partially on the finding that variations in parameters of conditioned reinforcement appear not to affect response strength as measured by resistance to change, long-standing assertions that conditioned reinforcers do not strengthen behavior in a reinforcement-like fashion are considered. A signposts or means-to-an-end account is explored and appears to provide a plausible alternative interpretation of the effects of stimuli associated with primary reinforcers. Related suggestions that primary reinforcers also might not have their effects via a strengthening process are explored and found to be worthy of serious consideration.

Behavioral momentum of cocaine self-administration: effects of frequency of reinforcement on resistance to extinction

Persistent drug seeking is a defining property of substance abuse and is generally thought to depend, in part, on exposure to drug-associated contexts. Behavioral momentum theory provides a set of methods and a theoretical framework for understanding how stimulus contexts contribute to the persistence of operant behavior. Earlier research has extended behavioral momentum theory to alcohol self-administration, but not to intravenous drug self-administration. This experiment extended behavioral momentum theory to cocaine self-administration by examining the effects of frequency of cocaine reinforcement in a context on resistance to extinction. Rats self-administered 0.32 mg/kg infusions of cocaine in a multiple schedule of reinforcement arranging two distinct contexts. Responding in a Rich context was reinforced by cocaine infusions at a higher frequency (i.e. variable interval 120 s) and in a Lean context at a lower frequency (variable interval 360 s). After establishment of responding in the two contexts, resistance to extinction was examined. Preextinction response rates for cocaine were similar in the Rich and Lean contexts. Nonetheless, relative resistance to extinction was greater in the Rich context than in the Lean context. The difference in resistance to extinction in the two contexts was well described by a quantitative model of behavioral momentum. These results suggest that the frequency of drug reinforcement in a context contributes to the persistence of drug seeking in that context, and that behavioral momentum theory might be useful for understanding how drug-associated contexts contribute to the persistence of drug seeking.

Cocaine and automaintained responding in pigeons: rate-reducing effects and tolerance thereto with different durations of food delivery

Pigeons were exposed to an automaintenance procedure in which 6-s key illuminations in one color (red or white) were immediately followed by 3-s food deliveries and key illuminations in the other color were followed by 9-s food deliveries. Both conditions engendered consistent responding. With both durations of food delivery, acute and chronic cocaine administrations (1.0-17.8 mg/kg) produced dose-dependent decreases in mean percent trials (key illuminations) with a response and mean total response per session. Tolerance developed to the disruptive effects of cocaine on both response measures. Food duration did not significantly affect either response measure or significantly interact with cocaine dose or drug regimen. The orderliness of the present findings, like those of a related study examining whether probability of food delivery modulated the effects of cocaine on automaintained responding [Porritt, M., Arnold, M., Poling, A., Cocaine and automaintained responding in pigeons: rate-reducing effects and tolerance thereto with different CS-US pairing probabilities. Pharmacol Biochem Behav 2007; 87:405-411.], suggests that the automaintenance procedure is a useful assay for examining tolerance to drug effects on classically-conditioned responding. Unlike the results of that study, however, the present findings are inconsistent with a behavioral momentum analysis of drug effects on such responding.

Progressive ratio schedules of reinforcement

Pigeons' pecks produced grain under progressive ratio (PR) schedules, whose response requirements increased systematically within sessions. Experiment 1 compared arithmetic (AP) and geometric (GP) progressions. Response rates increased as a function of the component ratio requirement, then decreased linearly (AP) or asymptotically (GP). Experiment 2 found the linear decrease in AP rates to be relatively independent of step size. Experiment 3 showed pausing to be controlled by the prior component length, which predicted the differences between PR and regressive ratio schedules found in Experiment 4. When the longest component ratios were signaled by different key colors, rates at moderate ratios increased, demonstrating control by forthcoming context. Models for response rate and pause duration based on Bizo and Killeen (1997) described performance on AP schedules; GP schedules required an additional parameter representing the contextual reinforcement.

Quantitative analyses of observing and attending

We review recent experiments examining whether simple models of the allocation and persistence of operant behavior are applicable to attending. In one series of experiments, observing responses of pigeons were used as an analog of attending. Maintenance of observing is often attributed to the conditioned reinforcing effects of a food-correlated stimulus (i.e., S+), so these experiments also may inform our understanding of conditioned reinforcement. Rates and allocations of observing were governed by rates of food or S+ delivery in a manner consistent with the matching law. Resistance to change of observing was well described by behavioral momentum theory only when rates of primary reinforcement in the context were considered. Rate and value of S+ deliveries did not affect resistance to change. Thus, persistence of attending to stimuli appears to be governed by primary reinforcement rates in the training context rather than conditioned reinforcing effects of the stimuli. An additional implication of these findings is that conditioned "reinforcers" may affect response rates through some mechanism other than response-strengthening. In a second series of experiments, we examined the applicability of the matching law to the allocation of attending to the elements of compound stimuli in a divided-attention task. The generalized matching law described performance well, and sensitivity to relative reinforcement varied with sample duration. The bias and sensitivity terms of the generalized matching law may provide measures of stimulus-driven and goal-driven control of divided attention. Further application of theories of operant behavior to performance on attention tasks may provide insights into what is referred to variously as endogenous, top-down, or goal-directed control of attention.

A theory of attending, remembering, and reinforcement in delayed matching to sample

A theory of attending and reinforcement in conditional discriminations is extended to working memory in delayed matching to sample by adding terms for disruption of attending during the retention interval. Like its predecessor, the theory assumes that reinforcers and disruptors affect the independent probabilities of attending to sample and comparison stimuli in the same way as the rate of overt free-operant responding as suggested by Nevin and Grace, and that attending is translated into discriminative performance by the model of Davison and Nevin. The theory accounts for the effects of sample-stimulus discriminability and retention-interval disruption on the levels and slopes of forgetting functions, and for the diverse relations between accuracy and sensitivity to reinforcement reported in the literature. It also accounts for the effects of reinforcer probability in multiple schedules on the levels and resistance to change of forgetting functions; for the effects of reinforcer probabilities signaled within delayed-matching trials; and for the effects of reinforcer delay, sample duration, and intertrial-interval duration. The model accounts for some data that have been problematic for previous theories, and makes testably different predictions of the effects of reinforcer probabilities and disruptors on forgetting functions in multiple schedules and signaled trials.

Measuring resistance to change at the within-session level

Resistance to change is often studied by measuring response rate in various components of a multiple schedule. Response rate in each component is normalized (that is, divided by its baseline level) and then log-transformed. Differential resistance to change is demonstrated if the normalized, log-transformed response rate in one component decreases more slowly than in another component. A problem with normalization, however, is that it can produce artifactual results if the relation between baseline level and disruption is not multiplicative. One way to address this issue is to fit specific models of disruption to untransformed response rates and evaluate whether or not a multiplicative model accounts for the data. Here we present such a test of resistance to change, using within-session response patterns in rats as a data base for fitting models of disruption. By analyzing response rate at a within-session level, we were able to confirm a central prediction of the resistance-to-change framework while discarding normalization artifacts as a plausible explanation of our results.

Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis

Previous experiments have shown that unsignaled delayed reinforcement decreases response rates and resistance to change. However, the effects of different delays to reinforcement on underlying response structure have not been investigated in conjunction with tests of resistance to change. In the present experiment, pigeons responded on a three-component multiple variable-interval schedule for food presented immediately, following brief (0.5 s), or following long (3 s) unsignaled delays of reinforcement. Baseline response rates were lowest in the component with the longest delay; they were about equal with immediate and briefly delayed reinforcers. Resistance to disruption by presession feeding, response-independent food during the intercomponent interval, and extinction was slightly but consistently lower as delays increased. Because log survivor functions of interresponse times (IRTs) deviated from simple modes of bout initiations and within-bout responding, an IRT-cutoff method was used to examine underlying response structure. These analyses suggested that baseline rates of initiating bouts of responding decreased as scheduled delays increased, and within-bout response rates tended to be lower in the component with immediate reinforcers. The number of responses per bout was not reliably affected by reinforcer delay, but tended to be highest with brief delays when total response rates were higher in that component. Consistent with previous findings, resistance to change of overall response rate was highly correlated with resistance to change of bout-initiation rates but not with within-bout responding. These results suggest that unsignaled delays to reinforcement affect resistance to change through changes in the probability of initiating a response bout rather than through changes in the underlying response structure.

Short-term item memory in successive same-different discriminations

Pigeons were tested in a successive same-different (S/D) discrimination procedure to examine the short-term memory for individual items in sequences of different or identical pictures. Item-by-item analyses of pecking behavior within single trials revealed this S/D discrimination emerged at the earliest possible point in the sequence--the presentation of the second item. Further, by comparing peck rates at points where different types of sequences diverged (e.g. ABA versus ABC), we determined that the pigeons remembered the first item for at least 4-8s and across one to two intervening items. These results indicate that this S/D discrimination was controlled by relational comparisons of pictorial content across memories of specific items, rather than the detection of low-level perceptual "transients" between items. A second experiment supported this conclusion by showing increased discrimination with longer first item viewing times, consistent with encoding of details about individual pictures. These findings further support a qualitative similarity among birds and primates in possessing a general capacity to judge certain types of stimulus relations, such as stimulus identity and difference. Implications for the temporal continuity of experience in animals are also considered.

A theory of attending and reinforcement in conditional discriminations

A model of conditional discrimination performance (Davison & Nevin, 1999) is combined with the notion that unmeasured attending to the sample and comparison stimuli, in the steady state and during disruption, depends on reinforcement in the same way as predicted for overt free-operant responding by behavioral momentum theory (Nevin & Grace, 2000). The rate of observing behavior, a measurable accompaniment of attending, is well described by an equation for steady-state responding derived from momentum theory, and the resistance to change of observing conforms to predictions of momentum theory, supporting a key assumption of the model. When probabilities of attending are less than 1.0, the model accounts for some aspects of conditional-discrimination performance that posed problems for the Davison-Nevin model: (a) the effects of differential reinforcement on the allocation of responses to the comparison stimuli and on accuracy in several matching-to-sample and signal-detection tasks where the differences between the stimuli or responses were varied across conditions, (b) the effects of overall reinforcer rate on the asymptotic level and resistance to change of both response rate and accuracy of matching to sample in multiple schedules, and (c) the effects of fixed-ratio reinforcement on accuracy. Some tests and extensions of the model are suggested, and the role of unmeasured events in behavior theory is considered.

Resistance to extinction in the steady state and in transition

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data.

Mathematical principles of reinforcement and resistance to change

Although Killeen's mathematical principles of reinforcement (MPR) apply to the asymptotic rate of a free operant after extended exposure to a single schedule of reinforcement, they can be extended to resistance to change in multiple schedules via alterations in the parameter representing the activating effects of reinforcers. MPR's predictions of resistance to change in relation to reinforcer rate on variable-interval (VI) schedules are empirically correct and agree with behavioral momentum theory (BMT). However, both MPR and BMT encounter problems in accounting for the effects of delayed reinforcement on resistance to change, relative to immediate reinforcement at the same rate. Further problems are raised by differences in resistance to change between variable-ratio (VR) and variable-interval performances maintained by the same reinforcer rate. With both delayed versus immediate reinforcement and variable-ratio versus variable-interval reinforcement, differential resistance to change is negatively correlated with the log ratios of baseline response rates when reinforcer rates are equated. Cases where resistance to change varies despite equated reinforcer rates, and the correlations among behavioral measures, provide challenges and opportunities for both MPR and BMT.

MPR

Mathematical Principles of Reinforcement (MPR) is a theory of reinforcement schedules. This paper reviews the origin of the principles constituting MPR: arousal, association and constraint. Incentives invigorate responses, in particular those preceding and predicting the incentive. The process that generates an associative bond between stimuli, responses and incentives is called coupling. The combination of arousal and coupling constitutes reinforcement. Models of coupling play a central role in the evolution of the theory. The time required to respond constrains the maximum response rates, and generates a hyperbolic relation between rate of responding and rate of reinforcement. Models of control by ratio schedules are developed to illustrate the interaction of the principles. Correlations among parameters are incorporated into the structure of the models, and assumptions that were made in the original theory are refined in light of current data.