Choice and reinforcement delay

A replication of extended implementation and social interaction during the Step it UP! Game for adults with disabilities

The purpose of the present experiments was to systematically replicate Step it UP! Game interventions with adults with disabilities. Participants were divided into two competing teams, and the team with the highest step count participated in a prize drawing. Experiment 1 (N = 9) evaluated the efficacy of an extended version of the Step it UP! Game that included additional and longer sessions. Experiment 2 (N = 8) evaluated the addition of contingent experimenter attention during the Step it UP! Game. Baseline, Step it UP! Game, and Step it UP! Game plus interaction sessions were evaluated in a multielement design. The Step it UP! Game was effective with additional and longer sessions, and all participants took more steps during Step it UP! Game sessions. Adding experimenter interaction to the Step it UP! Game did not increase the efficacy of the intervention.

Effects of schedule type and terminal-link duration on choice in concurrent chains

Four pigeons responded in a three-component concurrent chains procedure in which the terminal-link schedules were either both fixed-interval (FI FI), both variable-interval (VI VI), both mixed-interval (MI MI) or variable-interval fixed-interval (VI FI). Across components within sessions, overall terminal-link duration was varied while schedule types were varied across conditions. For the conditions with homogeneous schedules, the strongest preference was obtained with FI FI, intermediate with MI MI, and weakest with VI VI. Preference increased with overall terminal-link duration for all schedules, but the increase was more rapid for VI FI. The hyperbolic value-added model (HVA) and cumulative decision model (CDM) were fitted to the choice data and accounted for 83.9% and 76.1% of the variance, respectively. However, deviations from both models' predictions were systematic. A modification of the CDM, which assumed that reinforcers delivered after variable delays were more effective, improved the fit of the model and substantially reduced the systematic deviations in residuals. Comparable modifications of HVA produced only limited improvement. Results show that preference in concurrent chains with homogeneous terminal links depends on the degree of variability in reinforcer delays, and that the CDM can provide an excellent account of results across different terminal-link schedules.

Delay of reinforcement versus rate of reinforcement in Pavlovian conditioning

Conditioned stimulus (CS) duration is a determinant of conditioned responding, with increases in duration leading to reductions in response rates. The CS duration effect has been proposed to reflect sensitivity to the reinforcement rate across cumulative exposure to the CS, suggesting that the delay of reinforcement from the onset of the cue is not crucial. Here, we compared the effects of delay and rate of reinforcement on Pavlovian appetitive conditioning in mice. In Experiment 1, the influence of reinforcement delay on the timing of responding was removed by making the duration of cues variable across trials. Mice trained with variable duration cues were sensitive to differences in the rate of reinforcement to a similar extent as mice trained with fixed duration cues. Experiments 2 and 3 tested the independent effects of delay and reinforcement rate. In Experiment 2, food was presented at either the termination of the CS or during the CS. In Experiment 3, food occurred during the CS for all cues. The latter experiment demonstrated an effect of delay, but not reinforcement rate. Experiment 4 ruled out the possibility that the lack of effect of reinforcement rate in Experiment 3 was due to mice failing to learn about the nonreinforced CS exposure after the presentation of food within a trial. These results demonstrate that although the CS duration effect is not simply a consequence of timing of conditioned responses, it is dependent on the delay of reinforcement. The results provide a challenge to current associative and nonassociative, time-accumulation models of learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Sensitivity to pre- and post-reinforcer delays in self-control choice

Rats were exposed to a concurrent-chains schedule in which reinforcer amounts, pre-reinforcer and post-reinforcer delays were varied between alternatives. Two independent variable-interval 60-s schedules were in effect during the initial links. Pre- and post-reinforcer delays were defined by fixed-time schedules. In one condition, rats chose between a larger reinforcer delayed by 0, 40, 80, or 120-s and an immediate smaller reinforcer. In the other condition, rats chose between an immediate larger reinforcer terminated with 0, 40, 80, or 120-s post-reinforcer delays and an immediate smaller reinforcer terminated with no post-reinforcer delay. In both conditions, choice was predicted from overall reinforcement density (i.e. reinforcer amount divided by total trial time). The results indicate that the preference for the larger reinforcer systematically decreased as both pre- and post-reinforcer delays increased. Although the effect of pre-reinforcer delay was greater than that of post-reinforcer delay, post-reinforcer delay itself influenced rats' choices in the present self-control situation. In conclusion, the present study demonstrated the limitation of local reinforcement density model of the matching law and all time components that constitute a choice alternative need to be weighted and incorporated into the general model of choice behavior.

Preference for sequences of rewards: further tests of a parallel discounting model

Brunner and Gibbon [Brunner, D., Gibbon, J., 1995. Anim. Behav. 50, 1627-1634] studied rats' choices between sequences of equal numbers of rewards that differed in their temporal arrangement. Rats' preferences were well accounted for by a parallel discounting model of aggregate food value in which the sequence value is the simple sum of the hyperbolically discounted value of the individual rewards [Mazur, J.E., 1984. J. Exp. Anal. Behav. 46, 67-77]. Three experiments reported here extend their work to sequences of unequal number of rewards and test the specific prediction that rats prefer sequences that get worse with time over sequences that improve, given that the total reward rate is the same. Choice functions from this and other two experiments that offered choice between unequal reward numbers were S-shaped and supported a signal detection-like model in which the value of each option obeys the parallel discounting model.

A decision model for steady-state choice in concurrent chains

Grace and McLean (2006) proposed a decision model for acquisition of choice in concurrent chains which assumes that after reinforcement in a terminal link, subjects make a discrimination whether the preceding reinforcer delay was short or long relative to a criterion. Their model was subsequently extended by Christensen and Grace (2008, 2009a, 2009b) to include effects of initial- and terminal-link duration on choice. We show that an expression for steady-state responding can be derived from the decision model, which enables a model for choice that provides an account of archival data that is equal or superior to the contextual choice model (Grace, 1994) and hyperbolic value-added model (Mazur, 2001) in terms of goodness of fit, parsimony, and parameter invariance. The success of the steady-state decision model validates the strategy of understanding acquisition phenomena as a bridge toward explaining choice at the molar level.

Delay of reinforcers in a concurrent-chain schedule: An extension of the hyperbolic-decay model

Six pigeons were trained in concurrent-chain schedules with equal aperiodic initial links and delays to reinforcers in the terminal links. The terminal links always lasted 30 s. In Experiment 1, two reinforcers were delivered in each terminal link, with the first reinforcer delivered either 1 s (Experiment 1A) or 5 s (Experiment 1B) after choice. In these experiments, the delay between the first and second reinforcers in one terminal link was 10 s, and the delay between the first and second reinforcers on the other key was varied. This variation produced little change in preference. In Experiment 1C, the first and second delays on one key were 10 s, and on the other key they were varied within the restriction that the sum of delays was 20 s. Preference for the varied terminal link increased as the first delay was decreased. A hyperbolic model of the value of reinforcer delay provided a good description of the data from Experiment 1. In Experiment 2, a single reinforcer was delivered in each terminal link after a delay of either 0.2 or 19.8 s, and these delays were reversed between conditions. The initial-link schedule providing terminal-link access was varied from means of 5 s to 480 s. As the initial-link duration was increased, preference for the shorter delay became less extreme. An extension of the hyperbolic-decay model, in which the decay constant was a hyperbolic function of the initial-link duration, described the results well. Differences between the procedure used here (constant-duration terminal links) and that used in conventional concurrent-chain research precludes use of the model as a general account of concurrent-chain performance.

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.

Concurrent-chain performance: Effects of absolute and relative terminal-link entry frequency

Six pigeons were trained in a concurrent-chain procedure with constant variable-interval 6-s variable-interval 12-s terminal links. Five groups of conditions were arranged. Within a group of conditions, the duration of one initial-link schedule was held constant and the duration of the other initial link was varied. The duration of the varied initial link was always longer than, or equal to, the constant initial-link duration. The duration of the shorter initial link was varied across groups of conditions from 5 s to 70 s. The data from each group were well described by the generalized matching law. Sensitivity (a) to the terminal-link entry ratio increased as the shorter initial-link duration increased, but appeared to reach an asymptote at shorter initial-link durations greater than 32 s. Terminal-link bias did not change with changes in shorter initial-link duration for the response-allocation data, but showed a small increase with increasing shorter initial-link duration for the time-allocation data.

Choice and the relative immediacy of reinforcement

The relative immediacy of reinforcement in concurrent-chain schedules was varied while the relative reduction in the overall average time to reinforcement associated with terminal-link entry was held constant. For each of four pigeons, choice did not vary with relative immediacy of reinforcement. Subsequently, choice by the same subjects was shown to be sensitive to relative reduction in average time to reinforcement.

Incentive theory: II. Models for choice

Incentive theory is extended to account for concurrent chained schedules of reinforcement. The basic model consists of additive contributions from the primary and secondary effects of reinforcers, which serve to direct the behavior activated by reinforcement. The activation is proportional to the rate of reinforcement and interacts multiplicatively with the directive effects. The two free parameters are q, the slope of the delay of reinforcement gradient, whose value is constant across many experiments, and b, a bias parameter. The model is shown to provide an excellent description of all results from studies that have varied the terminal-link schedules, and of many of the results from studies that have varied initial-link schedules. The model is extended to diverse modifications of the terminal links, such as varied amount of reinforcement, varied signaling of the terminal-link schedules, and segmentation of the terminal-link schedules. It is demonstrated that incentive theory provides an accurate and integrated account of many of the phenomena of choice.

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).

A contextual model of concurrent-chains choice

An extension of the generalized matching law incorporating context effects on terminal-link sensitivity is proposed as a quantitative model of behavior under concurrent chains. The contextual choice model makes many of the same qualitative predictions as the delay-reduction hypothesis, and assumes that the crucial contextual variable in concurrent chains is the ratio of average times spent, per reinforcement, in the terminal and initial links; this ratio controls differential effectiveness of terminal-link stimuli as conditioned reinforcers. Ninety-two concurrent-chains data sets from 19 published studies were fitted to the model. Averaged across all studies, the model accounted for 90% of the variance in pigeons' relative initial-link responding. The model therefore demonstrates that a matching law analysis of concurrent chains-the assumption that relative initial-link responding equals relative terminal-link value-remains quantitatively viable. Because the model reduces to the generalized matching law when terminal-link duration is zero, it provides a quantitative integration of concurrent schedules and concurrent chains.

Delay and amount of reward in a concurrent chain

Eight pigeons responded under a concurrent-chain schedule for rewards differing in both delay and amount, the larger reward being associated with the longer delay. Preference was examined as the absolute durations of the terminal-link delays were increased at four different delay ratios. Difficulties with other experiments of this type were controlled for by the use of (a) a single-tape initial link to equalize terminal-link entries, (b) a blackout following the more immediate reward to equalize terminal-link length, and (c) a photocell to measure reinforcer duration more accurately. Preference for the larger reward changed systematically as delays increased in all conditions, decreasing for the 6:1, 3:1, and 3:2 ratios, and increasing for the 1:1 ratio. These results were similar to, but significantly different from, those of previous investigations. The implications of these results for various models of concurrent-chain behavior are discussed.

Choice between rewards differing in amount and delay: Toward a choice model of self control

A concurrent-chain procedure was used to study pigeons' choices between rewards differing in both amount and delay. The shorter delay terminated with a 2-second access to grain whereas the longer delay terminated with a 6-second access to grain. The ratio of the delays was constant within a given condition while their absolute values were varied. Over conditions, ratios of 6:1, 3:1, and 3:2 were studied. As the absolute values of the delays to reinforcement increased, preference for the longer-delayed but larger reward decreased under both the 6:1 and 3:1 ratios, but increased under the 3:2 ratio. These results are inconsistent with choice models predicting no change in preference when the ratios of delays and amounts are held constant. In addition, the change in preference under the 3:1 ratio is inconsistent with a simple multiplicative interaction of the trade off between reinforcer amount and delay, and suggests that delay is a more potent determinant of choice than is amount. These results have implications for models that view choice between small immediate rewards and large but delayed rewards as underlying the behavior commonly called self control.

Effect of rate of delivery of intravenous cocaine on self-administration in rats

Many studies of drug self-administration in primates have shown that faster infusions of a drug are more reinforcing than slower infusions. Similar effects have not been shown in rats. We assessed the influence of delivery rate by allowing rats to choose between the same doses of intravenous cocaine delivered over two different infusion speeds. Rats were trained in chambers containing two nose-poke response devices. In Experiment 1, responses in one nose-poke delivered 0.3 mg/kg/injection of cocaine over 10 s, and responses in the other delivered the same dose over 100 s. In Experiment 2, the same procedure was used, but with 1.0 mg/kg/injection dose delivered over 1.7 versus 100 s. During acquisition, most rats preferred the faster infusion. When the delivery rates associated with the nose pokes were reversed, rats trained with 0.3 mg/kg/injection failed to switch nose-poke preference, but half the rats trained with 1.0 mg/kg/injection did switch. In Experiment 3, the choice was between 1 mg/kg cocaine delivered over 1.7 s and no reinforcement. Here, rats quickly learned to respond in the nose-poke associated with cocaine and quickly switched their choice during reversal. In Experiment 4, two groups of rats were allowed to choose between food delivered with a delay of 1 versus 5 s or 1 versus 10 s, respectively. Rats preferred the shorter delay during initial training. In reversal, some rats in the 1 vs 5 s group failed to reverse, while all the rats in the 1 vs 10 s group reversed. These results show that faster infusions of cocaine are clearly more reinforcing during acquisition, but delivery rate may not be as important to the maintenance of self-administration once it has been established. The results with food suggest that these findings represent general principles of behavior and are not unique to drug self-administration.

Response allocation in concurrent chains when terminal-link delays follow an ascending and descending series

Eight pigeons were trained in a concurrent-chains procedure in which the terminal-link immediacy ratio followed an ascending or descending series. Across sessions, one terminal-link delay changed from 2 s to 32 s to 2 s or from 32 s to 2 s to 32 s, while the other was always 8 s. For all pigeons, response allocation tracked changes in delay and was biased towards the 8-s alternative on the descending series, indicating a hysteresis effect, and was more sensitive to changes in the terminal-link delay ratio for relatively long (> 8 s) than short (< 8 s) delays. Both the hysteresis and effect of delay duration were predicted by an extended version of Grace and McLean's (2006) decision model. The extended decision model provided an overall better account of the results than a simple linear-operator model (Grace, 2002), and holds promise for an integrated account of choice in concurrent chains for both acquisition and steady-state conditions.

Temporal context in concurrent chains: I. Terminal-link duration

Two experiments are reported in which the ratio of the average times spent in the terminal and initial links (Tt/Ti) in concurrent chains was varied. In Experiment 1, pigeons responded in a three-component procedure in which terminal-link variable-interval schedules were in constant ratio, but their average duration increased across components by a factor of two. The log initial-link response ratio was a negatively accelerated function of Tt/Ti. Overall, the data were well described by Grace's (1994) contextual choice model (CCM) with temporal context represented as (Tt/Ti)k or 2Tt/(Tt + Ti), and by Mazur's (2001) hyperbolic value-added model (HVA), with each model accounting for approximately 93% of the variance. In Experiment 2, fixed-parameter predictions for each model were generated, based on the data from Experiment 1, for conditions in which Tt/Ti was varied over a more extreme range. Data were consistent with the predictions of CCM with temporal context represented as 2Tt/(Tt + Ti) and to a lesser extent as (Tt/Ti)k, but not with HVA. Overall, these results suggest that preference increases as a hyperbolic function of Tt/Ti when terminal-link duration is increased relative to initial-link duration, with the terminal-link schedule ratio held constant.

Effects of dose and infusion delay on cocaine self-administration choice in rhesus monkeys

RATIONALE

Drug abuse is often considered a problem related to impulse-control disorders, but little is known about the factors that determine the choice to self-administer a drug in a self-control/impulsivity paradigm.

OBJECTIVE

The objective of the present study was to evaluate choice between a low dose of cocaine administered after a relatively short delay (impulsive option) and a high dose of cocaine following a relatively longer delay (self-control option).

METHODS

Rhesus monkeys self-administered intravenous cocaine in a discrete-trials choice procedure. First, choice was between different 3:1 doses (0.3/0.1 and 0.1/0.03 mg/kg per injection) following equal 30-s delays to infusion. Second, choice was between equal doses (0.1 mg/kg per injection) following 3:1 delays (30 s/10 s, 90 s/30 s, 270 s/90 s, 810 s/270 s). Third, choice was between 0.1 or 0.03 mg/kg per injection after the same 3:1 delays with the larger dose following the longer delay and the smaller dose following the shorter delay. Fourth, the same 3:1 delays were used to study choice between 0.3 and 0.1 mg/kg per injection.

RESULTS

With equal delays, the larger dose of cocaine was chosen almost exclusively, and with equal doses, the shorter delay was chosen almost exclusively. When both dose and delay were manipulated, mean large-dose (0.1 mg/kg per injection) choices for three of four subjects was 98% when the delays were the shortest (30 s/10 s), but this preference reversed as the delays increased, so that 74% of choices were for the smaller dose (0.03 mg/kg per injection) at the longest delays (810 s/270 s). This systematic decrease in large-dose choices as the absolute, but not relative, values of the delays were increased, was also observed with the higher dose combination.

CONCLUSION

Delay discounting was supported by the present findings in that the value of a large reinforcer (higher cocaine dose) was decreased as its delay to presentation was increased. The importance of not only relative, but absolute, values of delays to drug reinforcement in determining drug choice was also demonstrated. Thus, a self-control/impulsivity paradigm can be extended to conditions with non-human subjects and drug reinforcers.

Effects of signaled versus unsignaled delay of reinforcement on choice

Pigeons chose between 5-s and 15-s delay-of-reinforcement alternatives. The first key peck to satisfy the choice schedule began a delay timer, and food was delivered at the end of the interval. Key pecks during the delay interval were measured, but had no scheduled effect. In Experiment 1, signal conditions and choice schedules were varied across conditions. During unsignaled conditions, no stimulus change signaled the beginning of a delay interval. During differential and nondifferential signal conditions, offset of the choice stimuli and onset of a delay stimulus signaled the beginning of a delay interval. During differential signal conditions, different stimuli were correlated with the 5-s and 15-s delays, whereas the same stimulus appeared during both delay durations during nondifferential signal conditions. Pigeons showed similar, extreme levels of preference for the 5-s delay alternative during unsignaled and differentially signaled conditions. Preference levels were reliably lower with nondifferential signals. Experiment 2 assessed preference with two pairs of unsignaled delays in which the ratio of delays was held constant but the absolute duration was increased fourfold. No effect of absolute duration was found. The results highlight the importance of delayed primary reinforcement effects and challenge models of choice that focus solely on conditioned reinforcement.

Hyperbolic value addition and general models of animal choice

Three mathematical models of choice--the contextual-choice model (R. Grace, 1994), delay-reduction theory (N. Squires & E. Fantino, 1971), and a new model called the hyperbolic value-added model--were compared in their ability to predict the results from a wide variety of experiments with animal subjects. When supplied with 2 or 3 free parameters, all 3 models made fairly accurate predictions for a large set of experiments that used concurrent-chain procedures. One advantage of the hyperbolic value-added model is that it is derived from a simpler model that makes accurate predictions for many experiments using discrete-trial adjusting-delay procedures. Some results favor the hyperbolic value-added model and delay-reduction theory over the contextual-choice model, but more data are needed from choice situations for which the models make distinctly different predictions.

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.

Relative sensitivity to reinforcer amount and delay in a self-control choice situation

Rats were exposed to concurrent-chains schedules in which a single variable-interval schedule arranged entry into one of two terminal-link delay periods (fixed-interval schedules). The shorter delay ended with the delivery of a single food pellet; the longer day ended with a larger number of food pellets (two under some conditions and six under others). In Experiment 1, the terminal-link delays were selected so that under all conditions the ratio of delays would exactly equal the ratio of the number of pellets. But the absolute duration of the delays differed across conditions. In one condition, for example, rats chose between one pellet delayed 5 s and six pellets delayed 30 s; in another condition rats chose between one pellet delayed 10 s and six pellets delayed 60 s. The generalized matching law predicts indifference between the two alternatives, assuming that the sensitivity parameters for amount and delay of reinforcement are equal. The rats' choices were, in fact, close to indifference except when the choice was between one pellet delayed 5 s and six pellets delayed 30 s. That deviation from indifference suggests that the sensitivities to amount and delay differ from each other depending on the durations of the delays. In Experiment 2, rats chose between one pellet following a 5-s delay and six pellets following a delay that was systematically increased over sessions to find a point of indifference. Indifference was achieved when the delay to the six pellets was approximately 55 s. These results are consistent with the possibility that the relative sensitivities to amount and delay differ as a function of the delays.

A quantitative analysis of sensitivity to the conditioned reinforcing value of terminal-link stimuli in a concurrent-chains schedule

Pigeons were exposed to a concurrent-chains schedule in which a single variable-interval 30-s schedule was used in the initial links and fixed-time schedules were used in the terminal links. Three types of keylight conditions were used in the terminal links. In the first condition, different delays were associated with different keylight stimuli (cued condition). In the second condition, different delays were associated with the same stimulus, either a blackout (uncued blackout condition) or a white key (uncued white condition). Paired values of terminal-link fixed-time schedules differed by a constant ratio of 3:1, while the absolute value of delays was varied from 3 s to 54 s. The results showed that choice proportions for the shorter of two delays increased when the absolute size of the delays was increased for all keylight conditions. Further, the choice proportions for the shorter delay increased from the uncued blackout condition, to the uncued white condition, to the cued condition. A modified version of Fantino's (1969) delay-reduction model (expressed as a function relating the response ratio to the delay-reduction ratio) can be applied to these data by showing that sensitivity to delay reduction increased from the uncued blackout condition, to the uncued white condition, to the cued condition. Thus, the present study demonstrated that a modified version of the delay-reduction model can be used to assess quantitative differences in the terminal-link keylight condition in terms of sensitivity to delay reduction (i.e., the conditioned reinforcing value of the terminal-link keylight stimuli).

Unification of models for choice between delayed reinforcers

Two models for choice between delayed reinforcers, Fantino's delay-reduction theory and Killeen's incentive theory, are reviewed. Incentive theory is amended to incorporate the effects of arousal on alternate types of behavior that might block the reinforcement of the target behavior. This amended version is shown to differ from the delay-reduction theory in a term that is an exponential in incentive theory and a difference in delay-reduction theory. A power series approximation to the exponential generates a model that is formally identical with delay-reduction theory. Correlations between delay-reduction theory and the amended incentive theory show excellent congruence over a range of experimental conditions. Although the assumptions that gave rise to delay-reduction theory and incentive theory remain different and testable, the models deriving from the theories are unlikely to be discriminable by parametric experimental tests. This congruence of the models is recognized by naming the common model the delayed reinforcement model, which is then compared with other models of choice such as Killeen and Fetterman's (1988) behavioral theory of timing, Mazur's (1984) equivalence rule, and Vaughan's (1985) melioration theory.

A molecular analysis of choice on concurrent-chains schedules

Six pigeons responded on concurrent-chains schedules with either independent or interdependent equal variable-interval schedules in the initial links and unequal variable-interval schedules, always in a 2:1 ratio, in the terminal links. Relative response rates in the initial links increased across conditions as initial-link duration was shortened and decreased across conditions as terminal-link duration was shortened, replicating previous findings. Responses in the initial links were recorded in 5-s bins, and local or molecular relative response rates were calculated in order to ascertain how relative response rate varied as a function of time since the onset of the initial links. Two distinct molecular patterns were found. With interdependent initial links, relative response rates for the preferred key were elevated for the first 10 or 20 s of the initial links and then declined to an asymptotic value. With independent initial links, a negative recency effect was found similar to that reported by Killeen (1970). These two molecular patterns were related to the different momentary reinforcement probabilities resulting from independent and interdependent scheduling.