Rats don’t always respond faster for more food: The paradoxical incentive effect

Resistance to change, of behavior and of theory

The persistence of operant behavior when disrupted tends to be positively related to how often reinforcers were delivered in the past. Behavioral momentum theory describes this finding as the outcome of Pavlovian processes. That is, the relation between discriminative stimuli and reinforcers that were delivered in their presence strengthens behavior, thereby making it more likely to persist. If only the story were that simple. A growing number of findings challenge the basic tenets of behavioral momentum theory. Some even call into question whether Pavlovian relations contribute to persistence in the first place. In this paper, I will review behavioral momentum theory and some of the data that have been problematic for the theory. I will argue that despite these very real challenges, the theory provides important utility not only to basic analyses of response persistence but also to clinical interventions directed at long-term reductions in problem behavior. It, for example, has set the stage for the development of alternative conceptual analyses of resistance to change, two of which will be highlighted for readers. Moreover, behavioral momentum theory may tell us something important about the reasons it continues to have an influence on the field, despite the challenging data that deter it.

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

The aversiveness of timeout from response-dependent and response-independent food deliveries as a function of delivery rate

Seven experiments with rats assessed the aversiveness of timeout using punishment and avoidance procedures. Experiments 1 and 2 considered the contributions of stimulus change, suspending the response-reinforcer contingency, response prevention, the general disruption in the reinforcement schedule during time-in, and overall decreases in reinforcement. Results support the conclusion that response-contingent timeouts punish behavior because they are signaled periods during which an ongoing schedule of positive reinforcement is suspended. Experiments 3, 4, and 5 assessed effects of the reinforcement rate during time-in on the punitive efficacy of timeout and, for comparison, electric shock. Evidence for a direct relation between reinforcement rate and punitive efficacy was equivocal. In Experiments 6 and 7, responding avoided timeout from response-independent food deliveries. Responding was acquired rapidly when it avoided timeouts from free deliveries of pellets or a sucrose solution, but not when it avoided free deliveries of water. At steady-state, avoidance rates and proficiency were directly related to the rate of pellet or sucrose deliveries. The relation between the nature of the time-in environment and the aversiveness of timeout was clear in our avoidance experiments, but not in our punishment experiments. We discuss interpretive problems in evaluating the aversiveness of timeout in the punishment paradigm.

Reinforcer quality matters: A test of the Mathematical Principles of Reinforcement with domestic hens (Gallus gallus domesticus)

This study evaluated the ability of Killeen's (1994) Mathematical Principles of Reinforcement to account for the effects of changes in reinforcer quality on hens' rates of responding on fixed-ratio schedules. Hens were trained to peck a key on a fixed-ratio schedule of reinforcement and then experienced an ascending series of ratio values in two separate conditions. In different conditions, the food reinforcer was either wheat or puffed wheat. Response rates initially increased with increases in ratio requirement before eventually decreasing at larger ratios. Quantitative fits of the model accounted for the data well. The fits revealed that different foods were systematically associated with changes in the specific activation parameter, a, and these were consistent with previous reports of preference for those food items.

Concurrent progressive-ratio and fixed-ratio schedule performance under geometric and arithmetic progressions by brushtail possums

The aim of this study was to compare the demand for food under concurrent progressive- and fixed-ratio schedules. Twelve brushtail possums participated in 16 conditions where schedule, progression and food type were varied. An incrementing schedule increased the fixed-ratio requirement within and across sessions and was arranged as either a geometric sequence (base 2), or an arithmetic sequence (step 5). Two foods were tested: a flaked barley and coco-pop(®) mix versus rolled oats. Overall, performance was similar for most possums in the within- and across-session incrementing schedules. An analysis of the estimates of essential value and break point produced the same account of demand for foods under the geometric or arithmetic progressions and within- and across-session procedures for 8 of 12 possums. Six possums showed higher demand for rolled oats compared to flaked barley, and two possums showed higher demand for flaked barley compared to rolled oats. Incrementing ratios within, rather than between sessions using an arithmetic progression was demonstrated to be a time efficient procedure for investigating demand for different food types without affecting conclusions about the relative demand for those foods.

Evidence for a role of 5-HT2C receptors in the motor aspects of performance, but not the efficacy of food reinforcers, in a progressive ratio schedule

RATIONALE

5-Hydroxytryptamine2C (5-HT2C) receptor agonists reduce the breakpoint in progressive ratio schedules of reinforcement, an effect that has been attributed to a decrease of the efficacy of positive reinforcers. However, a reduction of the breakpoint may also reflect motor impairment. Mathematical models can help to differentiate between these processes.

OBJECTIVE

The effects of the 5-HT2C receptor agonist Ro-600175 ((αS)-6-chloro-5-fluoro-α-methyl-1H-indole-1-ethanamine) and the non-selective 5-HT receptor agonist 1-(m-chlorophenyl)piperazine (mCPP) on rats' performance on a progressive ratio schedule maintained by food pellet reinforcers were assessed using a model derived from Killeen's Behav Brain Sci 17:105-172, 1994 general theory of schedule-controlled behaviour, 'mathematical principles of reinforcement'.

METHOD

Rats were trained under the progressive ratio schedule, and running and overall response rates in successive ratios were analysed using the model. The effects of the agonists on estimates of the model's parameters, and the sensitivity of these effects to selective antagonists, were examined.

RESULTS

Ro-600175 and mCPP reduced the breakpoint. Neither agonist significantly affected a (the parameter expressing incentive value), but both agonists increased δ (the parameter expressing minimum response time). The effects of both agonists could be attenuated by the selective 5-HT2C receptor antagonist SB-242084 (6-chloro-5-methyl-N-{6-[(2-methylpyridin-3-yl)oxy]pyridin-3-yl}indoline-1-carboxamide). The effect of mCPP was not altered by isamoltane, a selective 5-HT1B receptor antagonist, or MDL-100907 ((±)2,3-dimethoxyphenyl-1-(2-(4-piperidine)methanol)), a selective 5-HT2A receptor antagonist.

CONCLUSIONS

The results are consistent with the hypothesis that the effect of the 5-HT2C receptor agonists on progressive ratio schedule performance is mediated by an impairment of motor capacity rather than by a reduction of the incentive value of the food reinforcer.

Quantitative analysis of performance on a progressive-ratio schedule: effects of reinforcer type, food deprivation and acute treatment with Δ⁹-tetrahydrocannabinol (THC)

Rats' performance on a progressive-ratio schedule maintained by sucrose (0.6M, 50 μl) and corn oil (100%, 25 μl) reinforcers was assessed using a model derived from Killeen's (1994) theory of schedule-controlled behaviour, 'Mathematical Principles of Reinforcement'. When the rats were maintained at 80% of their free-feeding body weights, the parameter expressing incentive value, a, was greater for the corn oil than for the sucrose reinforcer; the response-time parameter, δ, did not differ between the reinforcer types, but a parameter derived from the linear waiting principle (T0), indicated that the minimum post-reinforcement pause was longer for corn oil than for sucrose. When the rats were maintained under free-feeding conditions, a was reduced, indicating a reduction of incentive value, but δ was unaltered. Under the food-deprived condition, the CB1 cannabinoid receptor agonist Δ(9)-tetrahydrocannabinol (THC: 0.3, 1 and 3 mg kg(-1)) increased the value of a for sucrose but not for corn oil, suggesting a selective enhancement of the incentive value of sucrose; none of the other parameters was affected by THC. The results provide new information about the sensitivity of the model's parameters to deprivation and reinforcer quality, and suggest that THC selectively enhances the incentive value of sucrose.

Effect of streptozotocin-induced diabetes on performance on a progressive ratio schedule

RATIONALE

It has been suggested that streptozotocin (STZ)-induced diabetes causes a motivational deficit in rodents. However, some of the evidence adduced in support of this suggestion may be interpreted in terms of a motor impairment rather than a motivational deficit.

OBJECTIVE

This experiment examined the effect of STZ-induced diabetes on performance on a progressive ratio schedule. The data were analysed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement model which enables the effects of interventions on motivation or incentive value to be separated from effects on motor function.

METHOD

Animals were trained under a progressive ratio schedule using food-pellet reinforcement. Then they received a single intraperitoneal injection of 50 mg/kg of STZ or the vehicle. Training continued for 30 sessions after treatment. Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between groups.

RESULTS

The parameter expressing incentive value was reduced in the group treated with STZ, whereas the parameters expressing motor capacity and post-reinforcement pausing were not affected by the treatment. Blood glucose concentration was significantly elevated in the STZ-treated group compared to the vehicle-treated group.

CONCLUSIONS

The results are consistent with the suggestion that STZ-induced diabetes is associated with a reduction of the incentive value of food.

A quantitative analysis of the effects of qualitatively different reinforcers on fixed ratio responding in inbred strains of mice

Previous studies of inbred mouse strains have shown reinforcer-strain interactions that may potentially mask differences among strains in memory performance. The present research examined the effects of two qualitatively different reinforcers (heterogeneous mix of flavored pellets and sweetened-condensed milk) on responding maintained by fixed-ratio schedules of reinforcement in three inbred strains of mice (BALB/c, C57BL/6, and DBA/2). Responses rates for all strains were a bitonic (inverted U) function of the size of the fixed-ratio schedule and were generally higher when responding was maintained by milk. For the DBA/2 and C57BL/6 and to a lesser extent the BALB/c, milk primarily increased response rates at moderate fixed ratios, but not at the largest fixed ratios tested. A formal model of ratio-schedule performance, Mathematical Principles of Reinforcement (MPR), was applied to the response rate functions of individual mice. According to MPR, the differences in response rates maintained by pellets and milk were mostly due to changes in motoric processes as indicated by changes in the minimum response time (δ) produced by each reinforcer type and not specific activation (a), a model term that represents value and is correlated with reinforcer magnitude and the break point obtained under progressive ratio schedules. MPR also revealed that, although affected by reinforcer type, a parameter interpreted as the rate of saturation of working memory (λ), differed among the strains.

Effects of SKF-83566 and haloperidol on performance on progressive ratio schedules maintained by sucrose and corn oil reinforcement: quantitative analysis using a new model derived from the Mathematical Principles of Reinforcement (MPR)

RATIONALE

Mathematical models can assist the interpretation of the effects of interventions on schedule-controlled behaviour and help to differentiate between processes that may be confounded in traditional performance measures such as response rate and the breakpoint in progressive ratio (PR) schedules.

OBJECTIVE

The effects of a D1-like dopamine receptor antagonist, 8-bromo-2,3,4,5-tetrahydro-3-methyl-5-phenyl-1H-3-benzazepin-7-ol hydrobromide (SKF-83566), and a D2-like receptor antagonist, haloperidol, on rats' performance on PR schedules maintained by sucrose and corn oil reinforcers were assessed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement.

METHOD

Separate groups of rats were trained under a PR schedule using sucrose or corn oil reinforcers. SKF-83566 (0.015 and 0.03 mg kg(-1)) and haloperidol (0.05 and 0.1 mg kg(-1)) were administered intraperitoneally (five administrations of each treatment). Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between treatments.

RESULTS

Haloperidol reduced a (the parameter expressing incentive value) in the case of both reinforcers, but did not affect the parameters related to response time and post-reinforcement pausing. SKF-83566 reduced a and k (the parameter expressing sensitivity of post-reinforcement pausing to the prior inter-reinforcement interval) in the case of sucrose, but did not affect any of the parameters in the case of corn oil.

CONCLUSIONS

The results are consistent with the hypothesis that blockade of both D1-like and D2-like receptors reduces the incentive value of sucrose, whereas the incentive value of corn oil is more sensitive to blockade of D2-like than D1-like receptors.

A clozapine-like effect of cyproheptadine on progressive ratio schedule performance

The atypical antipsychotic drug clozapine has multiple pharmacological actions, some of which, including 5-hydroxytryptamine (5-HT₂) and histamine (H₁) receptor antagonist effects, are shared by the non-selective 5-HT receptor antagonist cyproheptadine. Atypical antipsychotics have a characteristic profile of action on operant behaviour maintained by progressive ratio schedules, as revealed by Killeen's (1994) mathematical model of schedule controlled behaviour. These drugs increase the values of a parameter that expresses the 'incentive value' of the reinforcer (a) and a parameter that is inversely related to the 'motor capacity' of the organism (δ). This experiment examined the effects of acute treatment with cyproheptadine and clozapine on performance on a progressive ratio schedule of food reinforcement in rats; the effects of a conventional antipsychotic, haloperidol, and two drugs with food intake-enhancing effects, chlordiazepoxide and Δ⁹-tetrahydrocannabinol (THC), were also examined. Cyproheptadine (1, 5 mg kg⁻¹) and clozapine (3.75, 7.5 mg kg⁻¹) increased a and δ. Haloperidol (0.05, 0.1 mg kg⁻¹) reduced a and increased δ. Chlordiazepoxide (3, 10 mg kg⁻¹) increased a but reduced δ. THC (1, 3 mg kg⁻¹) had no effect. Interpretation based on Killeen's (1994) model suggests that cyproheptadine and clozapine enhanced the incentive value of the reinforcer and impaired motor performance. Motor impairment may be due to sedation (possibly reflecting H₁ receptor blockade). Enhancement of incentive value may reflect simultaneous blockade of H₁ and 5-HT₂ receptors, which has been proposed as the mechanism underlying the food intake-enhancing effect of cyproheptadine. In agreement with previous findings, haloperidol impaired motor performance and reduced the incentive value of the reinforcer. Chlordiazepoxide's effect on a is consistent with its food intake-enhancing effect.

Effect of orexin-B-saporin-induced lesions of the lateral hypothalamus on performance on a progressive ratio schedule

It has been suggested that a sub-population of orexinergic neurones whose somata lie in the lateral hypothalamic area (LHA) play an important role in regulating the reinforcing value of both food and drugs. This experiment examined the effect of disruption of orexinergic mechanisms in the LHA on performance on the progressive ratio schedule of reinforcement, in which the response requirement increases progressively for successive reinforcers. The data were analysed using a mathematical model which yields a quantitative index of reinforcer value and dissociates effects of interventions on motor and motivational processes. Rats were trained under a progressive ratio schedule using food-pellet reinforcement. They received bilateral injections of conjugated orexin-B-saporin (OxSap) into the LHA or sham lesions. Training continued for a further 40 sessions after surgery. Equations were fitted to the response rate data from each rat, and the parameters of the model were derived for successive blocks of 10 sessions. The OxSap lesion reduced the number of orexin-containing neurones in the LHA by approximately 50% compared with the sham-lesioned group. The parameter expressing the incentive value of the reinforcer was not significantly altered by the lesion. However, the parameter related to the maximum response rate was significantly affected, suggesting that motor capacity was diminished in the OxSap-lesioned group. The results indicate that OxSap lesions of the LHA disrupted food-reinforced responding on the progressive ratio schedule. It is suggested that this disruption was brought about by a change in non-motivational (motor) processes.

Effect of reinforcer magnitude on performance maintained by progressive-ratio schedules

This experiment examined the relationship between reinforcer magnitude and quantitative measures of performance on progressive-ratio schedules. Fifteen rats were trained under a progressive-ratio schedule in seven phases of the experiment in which the volume of a 0.6-M sucrose solution reinforcer was varied within the range 6-300 microl. Overall response rates in successive ratios conformed to a bitonic equation derived from Killeen's (1994) Mathematical Principles of Reinforcement. The "specific activation" parameter, a, which is presumed to reflect the incentive value of the reinforcer, was a monotonically increasing function of reinforcer volume; the "response time" parameter, delta, which defines the minimum response time, increased as a function of reinforcer volume; the "currency" parameter, beta, which is presumed to reflect the coupling of responses to the reinforcer, declined as a function of volume. Running response rate (response rate calculated after exclusion of the postreinforcement pause) decayed monotonically as a function of ratio size; the index of curvature of this function increased as a function of reinforcer volume. Postreinforcement pause increased as a function of ratio size. Estimates of a derived from overall response rates and postreinforcement pauses showed a modest positive correlation across conditions and between animals. Implications of the results for the quantification of reinforcer value and for the use of progressive-ratio schedules in behavioral neuroscience are discussed.

Effect of quinolinic acid-induced lesions of the subthalamic nucleus on performance on a progressive-ratio schedule of reinforcement: a quantitative analysis

The subthalamic nucleus (STN), a major relay in the indirect striatofugal pathway, plays an important role in extrapyramidal motor control. Recent evidence indicates that it may also be involved in regulating the incentive value of food reinforcers.

OBJECTIVE

To examine the effect of lesions of the STN on performance on a progressive-ratio schedule using a quantitative model that dissociates effects of interventions on motor and motivational processes [Killeen PR. Mathematical principles of reinforcement. Behav Brain Sci 1994;17:105-72]. Rats with bilateral quinolinic acid-induced lesions of the STN (n=14) or sham lesions (n=14) were trained to press a lever for food-pellet reinforcers under a progressive-ratio schedule. In Phase 1 (90 sessions) the reinforcer was one pellet; in Phase 2 (30 sessions) it was two pellets; in Phase 3 (30 sessions) it was again one pellet.

RESULTS

The performance of both groups conformed to the model of progressive-ratio schedule performance. The motor parameter, delta, was significantly higher in the STN-lesioned than the sham-lesioned group, reflecting lower overall response rates in the lesioned group. The motivational parameter, a, was significantly higher in the STN-lesioned group than in the sham-lesioned group, consistent with enhanced reinforcer value in the STN-lesioned group compared to the sham-lesioned group. In both groups, a was sensitive to changes in reinforcer size, being significantly greater under the two-pellet condition (Phase 2) than under the one-pellet condition (Phases 1 and 3). The results suggest that destruction of the STN impairs response capacity and enhances the incentive value of food reinforcers.

Modeling operant behavior in the Parkinsonian rat

Mathematical principles of reinforcement (MPR; Killeen, 1994) is a quantitative model of operant behavior that contains three parameters representing motor capacity (delta), motivation (a), and short term memory (lambda). The present study applied MPR to characterize the effects of bilateral infusions of 6-OHDA into the substantia nigra pars compacta in the rat, a model of Parkinson's disease. Rats were trained to lever press under a 5-component fixed-ratio (5, 15, 30, 60, and 100) schedule of food reinforcement. Rats were tested for 15 days prior to dopamine lesions and again for 15 days post-lesion. To characterize functional loss relative to lesion size, rats were grouped according to the extent and the degree of lateralization of their dopamine loss. Response rates decreased as a function of dopamine depletion, primarily at intermediate ratios. MPR accounted for 98% of variance in pre- and post-lesion response rates. Consistent with reported disruptions in motor behavior induced by dopaminergic lesions, estimates of delta increased when dopamine was severely depleted. There was no support for different estimates of a based on pre- and post-lesion performance of any lesion group, suggesting that dopamine loss has negligible effects on incentive motivation. The present study demonstrates the usefulness of combining operant techniques with a theoretical model to better understand the effects of a neurochemical manipulation.

Avoidance of timeout from response-independent food: effects of delivery rate and quality

In three experiments, a rat's lever presses could postpone timeouts from food pellets delivered on response-independent schedules. In Experiment 1, the pellets were delivered at variable-time (VT) rates ranging from VT 0.5 to VT 8 min. Experiment 2 replicated the VT 1 min and VT 8 min conditions of Experiment 1 with new subjects. Finally, subjects in Experiment 3 could postpone timeouts from delivery of pellets that differed in quality rather than quantity (unsweetened versus sweetened pellets). In general, response rates and success in avoiding increased as a function of the rate and quality of the pellets. Also, performance efficiency increased as the experiments progressed, that is, the avoidance response occurred later and later in the response-timeout interval. The results support the conclusion that timeout from reinforcement has functional properties similar to those of more commonly studied aversive stimuli (e.g., shock).

Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice

RATIONALE

The nucleus accumbens core (AcbC) is believed to contribute to the control of operant behaviour by reinforcers. Recent evidence suggests that it is not crucial for determining the incentive value of immediately available reinforcers, but is important for maintaining the values of delayed reinforcers.

OBJECTIVE

This study aims to examine the effect of AcbC lesions on performance on a progressive-ratio schedule using a quantitative model that dissociates effects of interventions on motor and motivational processes (Killeen 1994 Mathematical principles of reinforcement. Behav Brain Sci 17:105-172).

MATERIALS AND METHODS

Rats with bilateral quinolinic acid-induced lesions of the AcbC (n = 15) or sham lesions (n = 14) were trained to lever-press for food-pellet reinforcers under a progressive-ratio schedule. In Phase 1 (90 sessions) the reinforcer was one pellet; in Phase 2 (30 sessions), it was two pellets; in Phase 3, (30 sessions) it was one pellet.

RESULTS

The performance of both groups conformed to the model of progressive-ratio performance (group mean data: r2 > 0.92). The motor parameter, delta, was significantly higher in the AcbC-lesioned than the sham-lesioned group, reflecting lower overall response rates in the lesioned group. The motivational parameter, a, was sensitive to changes in reinforcer size, but did not differ significantly between the two groups. The AcbC-lesioned group showed longer post-reinforcement pauses and lower running response rates than the sham-lesioned group.

CONCLUSIONS

The results suggest that destruction of the AcbC impairs response capacity but does not alter the efficacy of food reinforcers. The results are consistent with recent findings that AcbC lesions do not alter sensitivity to reinforcer size in inter-temporal choice schedules.

Does satiation close the open economy?

Pigeons responded on fixed-interval and fixed-ratio food schedules during sessions of extended duration. Pause lengths from the beginning of the session, when the subjects were hungry, resembled those found in open economies, whereas pause lengths from the end of the sessions, when the subjects were close to satiation, resembled those from closed economies. A model of motivation captured key features of the data, suggesting that a changing level of hunger is a causal factor in the behavioral differences observed between open and closed economies. Behavioral theories may provide a parsimonious alternative to economic theories in accounting for such effects.

The operant reserve: A computer simulation in (accelerated) real time

In Skinner's Reflex Reserve theory, reinforced responses added to a reserve depleted by responding. It could not handle the finding that partial reinforcement generated more responding than continuous reinforcement, but it would have worked if its growth had depended not just on the last response but also on earlier responses preceding a reinforcer, each weighted by delay. In that case, partial reinforcement generates steady states in which reserve decrements produced by responding balance increments produced when reinforcers follow responding. A computer simulation arranged schedules for responses produced with probabilities proportional to reserve size. Each response subtracted a fixed amount from the reserve and added an amount weighted by the reciprocal of the time to the next reinforcer. Simulated cumulative records and quantitative data for extinction, random-ratio, random-interval, and other schedules were consistent with those of real performances, including some effects of history. The model also simulated rapid performance transitions with changed contingencies that did not depend on molar variables or on differential reinforcement of inter-response times. The simulation can be extended to inhomogeneous contingencies by way of continua of reserves arrayed along response and time dimensions, and to concurrent performances and stimulus control by way of different reserves created for different response classes.

On the relationship between anticipatory behaviour in a Pavlovian paradigm and Pavlovian-to-Instrumental Transfer in rats (Rattus norvegicus)

The present rat study assessed the relationship between, and the sensitivity of, two different tests for appetitive conditioned responding to differences in the contingency between a conditioned stimulus (CS) and an unconditioned stimulus (US), and to differences in US magnitude. The first test used a Pavlovian-to-Instrumental Transfer (PIT) paradigm, assessing the capacity of the CS to enhance instrumental responding for food. The second test employed a Pavlovian conditioning paradigm with an extended CS-US interval, and total number of behavioural elements in this interval as a dependent measure. The PIT test proved to be sensitive to contingency but not reward magnitude differences, whereas the reverse was true for the Pavlovian test. Although there was a significant correlation between tests in the magnitude of the CS-induced increase of food-magazine entries, the main dependent measure from PIT (number of lever presses) and that from the Pavlovian test (total number of behavioural elements) did not correlate. It is suggested that in the PIT procedure, the CS induces a chain of behavioural responses of which lever pressing is just a single element and that the Pavlovian test, in principle, is more sensitive.

Extending mathematical principles of reinforcement into the domain of behavioral pharmacology

Mathematical principles of reinforcement (MPR) attempts to integrate the empirical laws of reinforcement schedules that have accumulated over the decades. MPR is based on three principles: incentives excite behavior; there are temporal constraints on responding; and coupling of responses to reinforcers strengthens behavior [Behav. Brain Sci. 17 (1994) 105]. In the present paper MPR is extended into the domain of behavioral pharmacology, specifically to model the effects of D-amphetamine on operant behavior. In Experiment 1a, a five-component multiple fixed-ratio schedule was designed to generate behavioral baselines that were subsequently used to assess drug effect. In Experiments 1b and 1c, the quality and quantity of reinforcer were manipulated. The data generated by the three experiments were consistent with MPR. In Experiment 2, MPR was used to model the effects of D-amphetamine on rats responding under the five-component multiple fixed-ratio schedule. According to the model, the rate-decreasing effects of D-amphetamine were due primarily to motor disruption and secondarily to increased impulsivity; at the highest dosages, D-amphetamine also may have decreased the incentive value of food.

MPR

Mathematical Principles of Reinforcement (MPR) is a theory of reinforcement schedules. This paper reviews the origin of the principles constituting MPR: arousal, association and constraint. Incentives invigorate responses, in particular those preceding and predicting the incentive. The process that generates an associative bond between stimuli, responses and incentives is called coupling. The combination of arousal and coupling constitutes reinforcement. Models of coupling play a central role in the evolution of the theory. The time required to respond constrains the maximum response rates, and generates a hyperbolic relation between rate of responding and rate of reinforcement. Models of control by ratio schedules are developed to illustrate the interaction of the principles. Correlations among parameters are incorporated into the structure of the models, and assumptions that were made in the original theory are refined in light of current data.

Writing and overwriting short-term memory

An integrative account of short-term memory is based on data from pigeons trained to report the majority color in a sequence of lights. Performance showed strong recency effects, was invariant over changes in the interstimulus interval, and improved with increases in the intertrial interval. A compound model of binomial variance around geometrically decreasing memory described the data; a logit transformation rendered it isomorphic with other memory models. The model was generalized for variance in the parameters, where it was shown that averaging exponential and power functions from individuals or items with different decay rates generates new functions that are hyperbolic in time and in log time, respectively. The compound model provides a unified treatment of both the accrual and the dissipation of memory and is consistent with data from various experiments, including the choose-short bias in delayed recall, multielement stimuli, and Rubin and Wenzel's (1996) meta-analyses of forgetting.