Interactions in multiple schedules: the role of the stimulus-reinforcer contingency

Behaviour of Rats in Multiple Schedules of Response-Contingent and Response-Independent Food Presentation

During Phase I, three rats were exposed to two-component multiple schedules of response-independent food presentation. Low rates of lever-pressing were observed, and response rates in one component did not increase when food presentation was withheld in the other component. During Phase II, the same rats were exposed to two-component multiple schedules of response-contingent reinforcement. Much higher rates of lever-pressing were observed. Moreover, when reinforcement was withheld in one component, response rates in the other component increased (positive contrast), and when reinforcement was reinstated in the changed component, response rates in the other component declined (negative contrast). During Phase III, when food was again delivered independently of responding, the response rates declined again to low levels. These results indicate that the occurrence of non-instrumental lever-pressing is not a prerequisite for the occurrence of behavioural contrast in the rat, and thus cast doubt on the general applicability of the autoshaping theory of behavioural contrast

Interactions in multiple schedules: negative induction with squirrel monkeys

In Experiment I, lever pressing by squirrel monkeys was maintained under a sequence of variable-interval, multiple variable-interval variable-interval, and multiple variable-interval extinction schedules of food presentation. Negative induction (decreased responding in the unchanged component) occurred when one component of the multiple variable-interval variable-interval schedule was changed to extinction. Negative induction was transient over sessions; responding in the unchanged component usually recovered to a rate similar to that under the multiple variable-interval variable-interval schedule. Negative induction was not accompanied by consistent changes in the patterns of local responding within the unchanged component, and did not depend on whether component schedules were associated with localized (lever lights) or diffuse visual stimuli (houselights), or on whether the unchanged component was a 60- or 180-sec variable-interval schedule. In Experiment II, responding was maintained under a sequence of variable-interval and multiple variable-interval timeout schedules of food presentation. Negative induction occurred when responding declined gradually in the timeout component but not when responding declined abruptly. The nature of interactions in multiple schedules may depend on the species; negative induction was observed with squirrel monkeys under conditions similar to those that produce positive contrast with pigeons.

Determinants of contrast in the signal-key procedure: Evidence against additivity theory

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that "elicited" behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.

Positive and negative contrast as a function of component duration for key pecking and treadle pressing

Pigeons responded on several multiple schedules for food reinforcers. The duration of the components varied from four seconds to 16 minutes. The absolute size of positive (Experiment 1) and negative (Experiment 2) behavioral contrast varied inversely with component duration when key pecks produced the reinforcers. The absolute size of negative contrast varied directly with component duration, when treadle presses produced the reinforcers (Experiment 3). These results conform to theories that suggest that positive and negative contrast are symmetrical when pigeons peck keys. They also conform to theories that suggest that the same principles do not govern contrast when pigeons peck keys as when they press treadles. Finally, the results support the measurement of behavioral contrast by the differences between baseline rates of responding and the rates emitted when contrast is present.

Three versions of the additive theories of behavioral contrast

The additive theories of behavioral contrast state that contrast will occur only when two types of responses interact during multiple schedules. Three more specific versions of the theories may be defined according to how they distinguish these two types of responses. A strong version physically distinguishes them. A second version distinguishes them according to the theoretical processes which control them. A weak version distinguishes them on the basis of the environmental relations which control them. Only the weak version of the theories is currently testable. The weak theory should be tested by establishing each of the two environmental relations independently and then combining them to assess their effect on behavior. Because this test is not usually performed, many of the results which have been taken to support or contradict the additive theories are actually ambiguous.

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the "additivity theory" of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.

Behavioral contrast as differential time allocation

In Experiment I, hooded rats were exposed to multiple variable-interval schedules of reinforcement in which manipulanda and reinforcement magazines at opposite ends of the experimental chamber were associated with the different components. Time allocated to each component was measured by recording the time spent by the subject in the appropriate half of the chamber. Positive behavioral contrast was observed for the comparison between multiple variable-interval 30-second variable-interval 30-second and multiple variable-interval 30-second variable-interval 90-second conditions for both response frequency and time allocation measures, but not for mean local response rate (response frequency per time allocated to a component). In Experiment II, rats were exposed to multiple variable-time schedules in which reinforcement was response independent. Time allocated to each component was measured for two conditions, multiple variable-time 30-second variable-time 30-second and multiple variable-time 30-second variable-time 90-second. Positive behavioral contrast of time allocation was exhibited. The results indicated that time allocation was differentially sensitive to changes in reinforcement probability, and that behavioral contrast may result from the differential allocation of time to the different components of the multiple schedule.

Some parameters of behavioral contrast and allocation of interim behavior in rats

Two experiments examined the effects of baseline reinforcement rate and component duration on behavioral contrast and on re-allocation of interim behavior in rats. Positive behavioral contrast occurred during multiple variable-interval 10-second extinction (VI 10 EXT) after a multiple VI 10 VI 10 baseline condition, but not during multiple VI 60 EXT following multiple VI 60 VI 60 baseline. Component duration had no significant effect on contrast. These results differed from those found in studies of pigeons' key pecking. Contrast was accompanied by an increased rate of drinking in the changed component, but drinking in the constant component did not decrease. These results are not consistent with the competition theory of contrast, but are consistent with the predictions based on the matching law. However, no current theory seems to account for all instances of behavioral contrast.

Associative interaction: joint control of key pecking by stimulus-reinforcer and response-reinforcer relationships

The joint control of rate of key pecking in pigeons by stimulus-reinforcer and response-reinforcer relationships was studied in the context of a two-component multiple schedule of reinforcement. Food presentation was always associated with one component and extinction with the other. The stimulus-reinforcer relationship was manipulated by varying the relative durations of the two components. In the food-presentation component, a fixed rate of reinforcement, independent of rate of responding, was generated by a schedule referred to as "T*". One aspect of the response-reinforcer relationship, contiguity, was manipulated by varying the percentage of delayed reinforcers. With the multiple T* extinction schedule, stimulus-reinforcer and response-reinforcer relationships could be varied independently of one another. Rate of key pecking was sensitive to manipulations of both relationships. However, significant differential effects due to either the stimulus-reinforcer or response-reinforcer relationship were obtained only when the other relationship was weak: stimulus-reinforcer and response-reinforcer relationships interacted in the joint control of responding.

Contrast and autoshaping in multiple schedules varying reinforcer rate and duration

Thirteen master pigeons were exposed to multiple schedules in which reinforcement frequency (Experiment I) or duration (Experiment II) was varied. In Phases 1 and 3 of Experiment I, the values of the first and second components' random-interval schedules were 33 and 99 seconds, respectively. In Phase 2, these values were 99 seconds for both components. In Experiment II, a random-interval 33-second schedule was associated with each component. During Phases 1 and 3, the first and second components had hopper durations of 7.5 and 2.5 seconds respectively. During Phase 2, both components' hopper durations were 2.5 seconds. In each experiment, positive contrast obtained for about half the master subjects. The rest showed a rate increase in both components (positive induction). Each master subject's key colors and reinforcers were synchronously presented on a response-independent basis to a yoked control. Richer component key-pecking occurred during each experiment's Phases 1 and 3 among half these subjects. However, none responded during the contrast condition (unchanged component of each experiment's Phase 2). From this it is inferred that autoshaping did not contribute to the contrast and induction findings among master birds. Little evidence of local contrast (highest rate at beginning of richer component) was found in any subject. These data show that (a) contrast can occur independently from autoshaping, (b) contrast assays during equal-valued components may produce induction, (c) local contrast in multiple schedules often does not occur, and (d) differential hopper durations can produce autoshaping and contrast.

The following schedule of reinforcement as a fundamental determinant of steady state contrast in multiple schedules

Two experiments investigated whether steady-state interactions in multiple schedules depend exclusively on the following schedule of reinforcement. Experiment 1 used a four-component multiple schedule in which two components were associated with the same constant schedule of reinforcement, and where rate of reinforcement was varied in the component that followed one of these. Contrast effects were reliable only in the component that preceded the point of reinforcement variation, although some contrast did occur otherwise. In those instances where contrast other than the following-schedule effect did occur, it was accounted for by the effect of the preceding schedule, an effect for which there were consistent individual differences among subjects, and which varied with component duration. Experiment 2 used a three-component schedule, in which reinforcement rate was varied in the middle component. The results were consistent with Experiment 1, as the following-schedule effect was the only consistent effect that occurred, although an effect of the preceding schedule did occur for some subjects under some conditions, and was especially evident early in training. The conclusion from both experiments is that there is no general effect of relative rate of reinforcement apart from the sum of the effects of the preceding and following schedules, and that the following-schedule effect is the fundamental cause of steady-state interactions.

Another look at contrast in multiple schedules

Recent research on multiple schedule interactions is reviewed. Contrary to formulations that view contrast as the result of elicited behavior controlled by the stimulus-reinforcer contingency (e.g., additivity theory), the major controlling variable is the relative rate of reinforcement, which cannot be reduced to some combination of stimulus-reinforcer and response-reinforcer effects. Other recent theoretical formulations are also reviewed and all are found to face serious counterevidence. The best description of the available data continues to be in terms of the "context of reinforcement," but Herrnstein's (1970) formulation of the basis of such context effects appears to be inadequate. An alternative conception is provided by Catania's concept of "inhibition by reinforcement," by which rate of responding is inversely related to the average rate of reinforcement in the situation. Such a conception is related to Gibbon's recent scalar-expectancy account of autoshaping and Fantino's delay-reduction model of conditioned reinforcement, suggesting that a common set of principles determines several diverse conditioning phenomena. However, the empirical status of such a description remains uncertain, because recent evidence shows that schedule interactions are temporally asymmetric, depending primarily upon the conditions of reinforcement that follow a schedule component.

Locating reward cue at response manipulandum (CAM) induces symptoms of drug abuse

Appetitive instrumental discrimination learning procedures provide for CAM (cue and manipulandum) when the reward cue (discriminative stimulus positively correlated with positive reinforcement) is located at the response manipulandum (object that when contacted or manipulated defines the performance of the instrumental response). Evidence reviewed shows that CAM induces excessive and compulsive instrumental responding relative to otherwise comparable non-CAM control procedures. In humans, symptoms of drug abuse are particularly likely when the drug-taking implement (response manipulandum at which instrumental drug-taking is directed) is also predictive of the drug's rewarding effects (reward cue). Evidence that the predictive relationship between a drug-taking implement and drug reward relates to drug abuse is reviewed, and implications for treatment and prevention are considered. CAM is related to neurobiological models of drug abuse that emphasize the role of the neurotransmitter dopamine (DA). CAM produces convergence of DA-mediated responding for conditioned reinforcement with DA mediation of psychomotor activation and incentive-motivational processes to yield reflexive cue-directed responding not observed in non-CAM controls.

Behavioral contrast: Pavlovian effects and anticipatory contrast

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.

Real-time detection of orientation during negative behavioral contrast with key pecking and a turning response

We developed a video system for real-time detection of a pigeon's orientation and for reinforcement of a "turning response." Using this system, negative behavioral contrast was found across key-peck and turning responses. In addition, turning away from the pecking key was detected by the system just after presentation of the negative discriminative stimulus on the key. The results suggest that avoidance of the discriminative stimulus in the constant component, which has been regarded as a causal factor for negative contrast (additivity theory), is not the primary factor for negative behavioral contrast of pigeons' key pecking, but may account for negative local contrast.

The effects of component duration on multiple-schedule performance in closed and open economies

Pigeons responded on multiple variable-interval variable-interval schedules of reinforcement in an open and a closed economy. Equal duration components were increased in duration while the component rates of reinforcement were held constant, the component schedules were reversed, and component duration was decreased. In the open economy, daily sessions were limited to 1 hr, and subjects were maintained at 80% of their free-feeding weights through supplemental feeding when necessary in their home cages. In the closed economy, subjects were housed in their experimental chambers and no deprivation regimen was enforced. Relative response rate decreased as components were lengthened in the open economy, whereas in the closed economy relative rate increased as components were lengthened. Response proportions overmatched reinforcer proportions to a greater extent at long component durations in the closed economy, but there was no systematic effect of component duration on responding in the open economy.

Behavioral contrast in fixed-interval components: effects of extinction-component duration

Seven albino rats were exposed to a multiple schedule of reinforcement in which the two components (fixed interval and extinction) alternated such that a presentation of the extinction component followed each fixed-interval reinforcement. In baseline sessions, the duration of the extinction component was constant and always one-third of the fixed-interval value. Probe sessions contained a probe segment in which the duration of the extinction component was increased; the response rate in fixed-interval components during the probe segment was compared with the response rate in the segments preceding and following the probe. The effect of increasing the duration of the extinction component was studied under three values of fixed interval: 30 s, 120 s, and 18 s, in three successive conditions. Response rate within fixed intervals was a direct function of duration of the extinction component. Pausing at the beginning of the fixed interval decreased as extinction duration increased. These effects were larger and more consistent for the shorter fixed-interval values (18 s and 30 s). These results indicate a functional relation between relative component duration and responding. For the component providing more frequent reinforcement, this could be stated as an inverse relationship between relative component duration and response rate. This relation is similar to findings regarding the ratio of trial and intertrial duration in Pavlovian conditioning procedures, and suggests that behavioral contrast may be related to Pavlovian contingencies underlying the multiple schedule.

Changeover delay effects on topographically tagged discriminative behavior

In a multiple variable-interval extinction schedule, pigeons' responses on an operant key were differentially reinforced in the presence of discriminative stimuli located on a signal key. Changeover delays of zero, one, two, or four seconds specified the time following a signal-key response within which an operant-key response was not reinforced. Systematic reduction of signal-key response rates with increasing changeover-delay duration indicated that signal-key responding was largely maintained by reinforcement of adventitious signal-key/operant-key response chains.

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.

Behavioral contrast and the automaintained key peck

In two experiments behavioral contrast was demonstrated during discrimination training in a positive automaintenance procedure. During the baseline condition in each experiment, a key was transilluminated for eight seconds by one of two colors (CS) following a variable intertrial interval signaled by a dark key. Keylight transillumination terminated with a response-independent food presentation. In the first experiment, food was eliminated during one CS for up to fifty sessions. After reinstatement of food following each CS, the discrimination was reversed. Six of the eight subjects showed positive behavioral contrast, i.e., response rates increased during the CS associated with food as they decreased during the CS associated with no food. The effect was replicated in Experiment II, but it did not occur when both the food and its associated CS were eliminated. These results were comparable to those obtained with operant discrimination training procedures (behavioral contrast) and with Pavlovian discrimination training. The results suggest that additivity theories of behavioral contrast may be insufficient to account for these data.

The effects of the stimulus-reinforcer correlation in a discrete-trials IRT>t procedure

The correlation between a keylight and food in a discrete-trials, interresponse-time-greater-than 6-sec (IRT>6-sec) procedure was varied by manipulating the rate of response-independent food presentation in the intertrial interval. When the correlation was positive, the rates of pecking in the IRT>6-sec condition were high and food was obtained on only about 5% of the trials. Likewise, responding was maintained at a high rate in yoked birds that received the same presentations of the light and food as the birds in the IRT>6-sec condition. When the rate of reinforcement between trials was equated to or made greater than the rate of reinforcement within trials, the response rate decreased for all birds, and those decreases were considerably larger for the yoked birds. However, the percentage of trials in which reinforced responses occurred under the IRT>6-sec procedure did not increase substantially when the light and food were either uncorrelated or negatively correlated. The percentage of trials in which a reinforcer was obtained increased when the keylight was left on continuously and the discriminative stimulus was not presented on the key. The results show that the stimulus-reinforcer correlation affects responding in the discrete-trials IRT>6-sec procedure, but that the effects of the stimulus-reinforcer correlation vary as a function of whether reinforcement is response-dependent or response-independent. The differences between the effects of response-independent and response-dependent pairings and nonpairings of the light and food are best accounted for in terms of differences in the control of responding by background stimuli.

Drug effects on responding maintained by stimulus-reinforcer and response-reinforcer contingencies

The effects of pentobarbital and d-amphetamine were assessed on key pecking by pigeons under conventional single-key multiple schedules and under two-key multiple schedules in which discriminative stimuli appeared on one key (stimulus key) while pecks on a second key (constant key) produced food. Pecks on the stimulus key had no scheduled consequences. A 60-second variable-interval schedule operated in one component of each multiple schedule: either extinction or a 60-second variable-time schedule operated in the alternate component. When the alternate-component schedule was extinction, a high rate of responding was maintained in the variable-interval component of the single-key schedule; responding on both keys was maintained in the variable-interval component of the two-key schedule. Pentobarbital increased responding in the variable-interval component of the single-key schedule and increased stimulus-key, but not constant-key responding in that component of the two-key schedule. When the alternate-component schedule was changed to variable time, responding declined in the variable-interval component of the single-key schedule; stimulus-key responding was no longer maintained under the two-key schedule. Pentobarbital decreased responding in the variable-interval component of both schedules. With an exception, d-amphetamine only decreased responding in the variable-interval component of the single- and two-key schedules both when the alternate-component schedule was extinction and when it was variable time. The results suggest that the effects of pentobarbital, but not d-amphetamine, depend on the nature of the contingency (stimulus-reinforcer, response-reinforcer) that maintains responding.