Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value

Briefly delayed reinforcement effects on variable-ratio and yoked-interval schedule performance

Most investigations of briefly delayed reinforcement have involved schedules that arrange a time-plus-response requirement. The present experiment examined whether briefly delaying reinforcement on schedules that have a ratio requirement differs from results with schedules that have a time-plus-response requirement. Four pigeons responded on a two-component multiple schedule. One component arranged a variable-ratio (VR) 50 and the other a variable-interval (VI) schedule in which the distribution of reinforcers was yoked to the preceding VR schedule. Across a series of conditions, delays were imposed in both schedules. These delays were brief (0.25- or 0.5-s) unsignaled delays and, as control conditions, a 5-s unsignaled delay and a 0.5-s delay signaled by a blackout of the chamber. In the yoked-VI component, the brief unsignaled delay increased response rates in six of nine opportunities and increased the proportion of short interresponse times (IRTs) (<0.4 s) in eight of nine opportunities. In the VR component, the brief unsignaled delay increased response rates and the proportion of short IRTs in only two of nine opportunities. For two of the three pigeons that were exposed to the 5-s unsignaled delay, response rates and the proportion of short IRTs decreased in both of the components. The 0.5-s signaled delay did not systematically change response rates nor did it change the distribution of short IRTs relative to the immediate reinforcement condition. The results replicate effects reported with time-based schedules and extend these observations by showing that changes commonly observed in VI performance with briefly delayed reinforcement are not characteristic of VR responding.

A comparison of the effects of brief rules, a timer, and preferred toys on self-control

Some children make impulsive choices (i.e., choose a small but immediate reinforcer over a large but delayed reinforcer). Previous research has shown that delay fading, providing an alternative activity during the delay, teaching participants to repeat a rule during the delay, combining delay fading with an alternative activity, and combining delay fading with a countdown timer are effective for increasing self-control (i.e., choosing the large but delayed reinforcer over the small but immediate reinforcer). The purpose of the current study was to compare the effects of various interventions in the absence of delay fading (i.e., providing brief rules, providing a countdown timer during the delay, or providing preferred toys during the delay) on self-control. Results suggested that providing brief rules or a countdown timer during the delay was ineffective for enhancing self-control. However, providing preferred toys during the delay effectively enhanced self-control.

Operant variability when reinforcement is delayed

Operant responses are often weakened when delays are imposed between the responses and reinforcers. We examined what happens when delayed reinforcers were contingent upon operant response variability. Three groups of rats were rewardedfor varying their response sequences, with onegroup rewarded for high variability, another for middle, and the third for low levels. Consistent with many reports in the literature, responding slowed significantly in all groups as delays were lengthened. Consistent with other reports, large differences in variability were maintained across the three groups despite the delays. Reinforced variability appears to be relatively immune to disruption by such things as delays, response slowing, prefeeding, and noncontingent reinforcement. Furthermore, the small effects on variability depended on baseline levels: As delays lengthened, variability increased in the low group, was statistically unchanged in the middle group, and decreased in the high group, an interaction similar to that reported previously when reinforcement frequencies were lowered. Thus, variable operant responding is controlled by reinforcement contingencies, but sometimes differently than more commonly studied repetitive responding.

Unsignaled delay of reinforcement, relative time, and resistance to change

Two experiments with pigeons examined the effects of unsignaled, nonresetting delays of reinforcement on responding maintained by different reinforcement rates. In Experiment 1, 3-s unsignaled delays were introduced into each component of a multiple variable-interval (VI) 15-s VI 90-s VI 540-s schedule. When considered as a proportion of the preceding immediate reinforcement baseline, responding was decreased similarly for the three multiple-schedule components in both the first six and last six sessions of exposure to the delay. In addition, the relation between response rates and reinforcement rates was altered such that both parameters of the single-response version of the matching law (i.e., k and Re) were decreased. Experiment 2 examined the effects of unsignaled delays ranging from 0.5 s to 8.0 s on responding maintained by a multiple VI 20-s VI 120-s schedule of reinforcement. Response rates in both components increased with brief unsignaled delays and decreased with longer delays. As in Experiment 1, response rates as a proportion of baseline were affected similarly for the two components in both the first six and last six sessions of exposure to the delay. Unlike delays imposed between two stimulus events, the effects of delays between responses and reinforcers do not appear to be attenuated when the average time between reinforcers is longer. In addition, the disruptions produced by unsignaled delays appear to be inconsistent with the general finding that responding maintained by higher rates of reinforcement is less resistant to change.

Effects of delayed reinforcers on the behavior of an animal model of attention-deficit/hyperactivity disorder (ADHD)

Attention-deficit/hyperactivity disorder (ADHD), affecting 3-5% of grade-school children, is a behavioral disorder characterized by developmentally inappropriate levels of inattention, hyperactivity, and impulsivity. It has been suggested that the symptoms are caused by altered reinforcement and extinction processes, behaviorally described as an abnormally short and steep delay-of-reinforcement gradient in ADHD. The present study tested predictions from the suggested shortened and steepened delay gradient in ADHD in an animal model, the spontaneously hypertensive rats (SHRs). It was predicted that SHR responding during baseline would mainly consist of responses with short inter-response times, and that responding would be more rapidly reduced in the SHR than in the controls by the introduction of a time interval between the response and reinforcer delivery. Effects of a resetting delay of reinforcement procedure with water as the reinforcer were tested on two baseline reinforcement schedules: variable interval 30 s (VI 30 s) and conjoint variable interval 60 s differential reinforcement of high rate 1s (VI 60 s DRH 1 s). The results showed a higher rate of responses in the SHR than in the controls during baseline, mainly consisting of responses with short inter-response times. The statistical analyses showed that response rates decreased more rapidly as a function of reinforcer delay in the SHR than in the controls. The analyses of the estimates of the reinforcer decay parameter showed no strain differences during the VI 30 s schedule but showed a significant strain difference at the end, but not at the start, of the sessions during the VI 60 s DRH 1 s schedule. In general, the results support predictions from the suggested steepened delay gradient in SHR. However, the predictions were only partly confirmed by the analyses of the decay parameter.

Reinforcement schedule thinning following treatment with functional communication training

We evaluated four methods for increasing the practicality of functional communication training (FCT) by decreasing the frequency of reinforcement for alternative behavior. Three participants whose problem behaviors were maintained by positive reinforcement were treated successfully with FCT in which reinforcement for alternative behavior was initially delivered on fixed-ratio (FR) 1 schedules. One participant was then exposed to increasing delays to reinforcement under FR 1, a graduated fixed-interval (FI) schedule, and a graduated multiple-schedule arrangement in which signaled periods of reinforcement and extinction were alternated. Results showed that (a) increasing delays resulted in extinction of the alternative behavior, (b) the FI schedule produced undesirably high rates of the alternative behavior, and (c) the multiple schedule resulted in moderate and stable levels of the alternative behavior as the duration of the extinction component was increased. The other 2 participants were exposed to graduated mixed-schedule (unsignaled alternation between reinforcement and extinction components) and multiple-schedule (signaled alternation between reinforcement and extinction components) arrangements in which the durations of the reinforcement and extinction components were modified. Results obtained for these 2 participants indicated that the use of discriminative stimuli in the multiple schedule facilitated reinforcement schedule thinning. Upon completion of treatment, problem behavior remained low (or at zero), whereas alternative behavior was maintained as well as differentiated during a multiple-schedule arrangement consisting of a 4-min extinction period followed by a 1-min reinforcement period.

Pigeons may not remember the stimuli that reinforced their recent behavior

In two experiments the conditioned reinforcing and delayed discriminative stimulus functions of stimuli that signal delays to reinforcement were studied. Pigeons' pecks to a center key produced delayed-matching-to-sample trials according to a variable-interval 60-s (or 30-s in 1 pigeon) schedule (Experiment 1) or a multiple variable-interval 20-s variable-interval 120-s schedule (Experiment 2). The trials consisted of a 2-s illumination of one of two sample key colors followed by delays ranging across phases from 0.1 to 27.0 s followed in turn by the presentation of matching and nonmatching comparison stimuli on the side keys. Pecks to the key color that matched the sample were reinforced with 4-s access to grain. Under some conditions of Experiment 1, pecks to nonmatching comparison stimuli produced a 4-s blackout and the start of the next interval. Under other conditions of Experiment 1 and each condition of Experiment 2, pecks to nonmatching stimuli had no effect and trials ended only when pigeons pecked the other, matching stimulus and received food. The functions relating pretrial response rates to delays differed markedly from those relating matching-to-sample accuracy to delays. Specifically, response rates remained relatively high until the longest delays (15.0 to 27.0 s) were arranged, at which point they fell to low levels. Matching accuracy was high at short delays, but fell to chance at delays between 3.0 and 9.0 s. In Experiment 2, both matching accuracy and response rates remained high over a wider range of delays in the variable-interval 120-s component relative to the variable-interval 20-s component. The difference in matching accuracy between the components was not due to an increased tendency in the variable-interval 20-s component toward proactive interference following short intervals. Thus, under these experimental conditions the conditioned reinforcing and the delayed discriminative functions of the sample stimulus depended on the same variables (delay and variable-interval value), but were nevertheless dissociated.

Evaluating self-control and impulsivity in children with severe behavior disorders

Impulsivity and self-control involve a choice between a smaller, more immediate reinforcer and a larger, more delayed reinforcer. Impulsive behavior occurs when responding produces the more immediate, relatively smaller reinforcers at the expense of delayed larger reinforcers. Self-control occurs when responding produces delayed larger reinforcers at the expense of immediate smaller reinforcers. Recently, researchers in applied behavior analysis have suggested that evaluations of self-control and impulsivity are relevant to socially important behaviors. Further, common behavioral treatments such as differential reinforcement may be influenced by variables such as reinforcer delay. In this study, we showed that aggression, reinforced by access to food, could be maintained as impulsive behavior. The participants were 2 young boys with severe developmental disabilities. For both participants, descriptive observations, care provider report, and functional analyses suggested that aggression was reinforced by food access (and television access for 1 participant). Next, we introduced a differential reinforcement procedure in which appropriate mands were reinforced. After various manipulations, we showed that aggression occurred when it produced immediate but small reinforcers even though mands produced larger, more delayed reinforcers. However, both participants displayed self-control when the delay to reinforcement was signaled (with a hand gesture or a timer).

Discrimination of methadone and cocaine by pigeons without explicit discrimination training

Pigeons were trained to peck a key on a variable-interval 2-min schedule of food reinforcement. Prior to each session, either 2.0 mg/kg methadone (n = 3), 3.0 mg/kg cocaine (n = 4), or 5.6 mg/kg cocaine (n = 2) was administered. When each pigeon's rate of pecking was stable, a range of doses of the training drug and saline were administered prior to 20-min extinction sessions separated by at least four training sessions. Rate of pecking during these extinction tests was generally an increasing function of dose, with the lowest rates obtained following saline and low doses and the highest rates obtained following doses near the training doses. Dose functions from pigeons trained with 5.6 mg/kg cocaine were steeper than those from pigeons trained with 3.0 mg/kg cocaine. Pigeons trained with methadone or 3.0 mg/kg cocaine were then given discrimination training, in which food reinforcement followed drug administration and 20-min extinction sessions followed saline administration. Rates of pecking under these conditions quickly diverged until near-zero rates were obtained following saline and high rates were obtained following drug. Discrimination training steepened dose functions for the training drugs, and the effects of several other substituted drugs depended on the pharmacology of the training drug. The pigeons trained with 5.6 mg/kg cocaine were tested with d-amphetamine, methadone, and morphine prior to discrimination training. d-Amphetamine increased rates dose dependently, and methadone and morphine did not. The results suggest that discriminative control by methadone and cocaine was established without explicit discrimination training.