Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Choosing a future from a murky past: A generalization-based model of behavior

Remembering the past appears critical in allowing organisms to detect order in an environment, and hence to behave in accordance with likely future events. Yet the shortcomings of remembering and perceiving typically mean that the remembered past differs from the actual past, and hence that behavior does not perfectly track the structure of the environment. Here, we outline how the process of generalization might be used to understand differences between what an organism does, and the structure of the past and potential structure of the environment. We explore how different sources of generalization - both from within the same stimulus situation, and from different stimulus situations - might be modeled quantitatively, and how predictions made by this modeling approach are supported by research. Finally, we discuss how generalization from multiple stimulus situations, longer-term experience, and from stimulus situations in the past that are not identical to the stimulus situation in the present, might contribute to our understanding of how an organism's experience translates into behavior.

A model for discriminating reinforcers in time and space

Both the response-reinforcer and stimulus-reinforcer relation are important in discrimination learning; differential responding requires a minimum of two discriminably-different stimuli and two discriminably-different associated contingencies of reinforcement. When elapsed time is a discriminative stimulus for the likely availability of a reinforcer, choice over time may be modeled by an extension of the Davison and Nevin (1999) model that assumes that local choice strictly matches the effective local reinforcer ratio. The effective local reinforcer ratio may differ from the obtained local reinforcer ratio for two reasons: Because the animal inaccurately estimates times associated with obtained reinforcers, and thus incorrectly discriminates the stimulus-reinforcer relation across time; and because of error in discriminating the response-reinforcer relation. In choice-based timing tasks, the two responses are usually highly discriminable, and so the larger contributor to differences between the effective and obtained reinforcer ratio is error in discriminating the stimulus-reinforcer relation. Such error may be modeled either by redistributing the numbers of reinforcers obtained at each time across surrounding times, or by redistributing the ratio of reinforcers obtained at each time in the same way. We assessed the extent to which these two approaches to modeling discrimination of the stimulus-reinforcer relation could account for choice in a range of temporal-discrimination procedures. The version of the model that redistributed numbers of reinforcers accounted for more variance in the data. Further, this version provides an explanation for shifts in the point of subjective equality that occur as a result of changes in the local reinforcer rate. The inclusion of a parameter reflecting error in discriminating the response-reinforcer relation enhanced the ability of each version of the model to describe data. The ability of this class of model to account for a range of data suggests that timing, like other conditional discriminations, is choice under the joint discriminative control of elapsed time and differential reinforcement. Understanding the role of differential reinforcement is therefore critical to understanding control by elapsed time.

The generalized matching law as a description of multiple-schedule responding

The literature was examined to determine how well the generalized matching law (Baum, 1974) describes multiple-schedule responding. In general, it describes the data well, accounting for a median of 91% of the variance. The median size of the undermatching parameter was 0.46; the median bias parameter was 1.00. The size of the undermatching parameter, and the proportion of the variance accounted for by the equation, varied inversely with the number of schedules conducted, with the number of sessions conducted per schedule, and with the time within a component. The undermatching parameter also varied with the operanda used to produce reinforcers and with the reinforcer used. The undermatching parameter did not vary consistently with component duration or with several other variables. Bias was greater when fewer rather than more schedules were conducted, when two rather than one operanda were used, and when White Carneaux rather than homing pigeons served as subjects. These results imply that the generalized matching law may describe both concurrent and multiple-schedule responding, but that the same variables do not always influence the bias and undermatching parameters in the same way for the two types of schedules.

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.

Quantitative analysis

Quantitative analysis permits the isolation of invariant relations in the study of behavior. The parameters of these relations can serve as higher-order dependent variables in more extensive analyses. These points are illustrated by reference to quantitative descriptions of performance maintained by concurrent schedules, multiple schedules, and signal-detection procedures. Such quantitative descriptions of empirical data may be derived from mathematical theories, which in turn can lead to novel empirical analyses so long as their terms refer to behavioral and environmental events. Thus, quantitative analysis is an integral aspect of the experimental analysis of behavior.

Choosing among natural stimuli

Pigeons worked on concurrent variable-interval, variable-interval schedules with the alternatives signaled by slides either containing trees or not. The schedules were designed to hold both overall and relative rates of reinforcement within narrowly constrained limits, and slides were quasi-randomly ordered each day. Responding to the two alternatives was well described by the generalized matching equation with substantial undermatching. Using an adaptation of the matching law, we estimated that the subjects were correctly classifying 82% to 95% of exemplars. The matching performance transferred to new exemplars of trees and nontrees with only slight generalization decrement. The pigeons appeared to be discriminating among exemplars even when the alternatives provided equal rates of reinforcement and the average relative performances were close to 50%.

Behavioral contrast redux

Behavioral contrast is defined as a change in response rate during a stimulus associated with a constant reinforcement schedule, in inverse relation to the rates of reinforcement in the surrounding stimulus conditions. Contrast has at least two functionally separable components: local contrast, which occurs after component transition, and molar contrast. Local contrast contributes to molar contrast under some conditions, but not generally. Molar contrast is due primarily to anticipatory contrast. However, anticipatory contrast with respect to response rate has been shown to be inversely related to stimulus preference, which challenges the widely held view that contrast effects reflect changes in stimulus value owing to the reinforcement context. More recent data demonstrate that the inverse relation between response rate and preference with respect to anticipatory contrast is due to Pavlovian contingencies embedded in anticipatory contrast procedures. When those contingencies are weakened, anticipatory contrast and stimulus preference are positively related, thus reaffirming the view that the reinforcing effectiveness of a constant schedule is inversely related to the value of the context of reinforcement in which it occurs. The underlying basis of how the context of reinforcement controls reinforcement value remains uncertain, although clear parallels exist between contrast and the effects of contingency in both Pavlovian and operant conditioning.

Action at a temporal distance: Component transition as the relational basis for successive discrimination

In a successive discrimination, red and green hues signaled component variable-interval schedules. The exponent of the power function relating ratios of responses in the red and green components to ratios of reinforcers provided a reinforcement-free measure of discrimination or stimulus control. Responses were recorded in successive 10-s subintervals of the 50-s components. The power-function exponent decreased systematically with increasing time since component transition in most conditions of five experiments. This reduction was not influenced by the absolute rate of reinforcement, consistent with the interpretation of the exponent as a measure of stimulus control. A reduction in the overall level of stimulus control by increasing the duration of response-produced keylight offset did not influence the decrease in discrimination with increasing time since component transition. The results support the conclusion that discriminative responding in successive discriminations is governed by several sources of stimulus control including delayed control by component transition.

Delayed and current stimulus control in successive discriminations

In a successive discrimination in which successively alternating red and green hues signaled component variable-interval schedules, sensitivity of the ratio of responses in the two components to variation in the component reinforcer ratio decreased systematically during the course of the component. This decrease in stimulus control or discrimination over the course of the component was shown to be the result of delayed control of responding during the component by the stimulus transition between components. When the red-green stimulus transition was altered by interpolating a white stimulus at the end of each 60-s component, discrimination at the beginning of the component (measured by the power-function exponent for sensitivity to reinforcement) was reduced. Conditions with the white stimulus inserted in other quarters of the component indicated that the current discriminative stimulus exerts control over responding throughout the component, whereas during about the first half of the component, response differentials are influenced by the transition between discriminative stimuli.

Discrete and continuous measures of dimensional stimulus control

In two sets of experiments, we examined dimensional stimulus control of pigeons' responses to a visual flicker-rate continuum. In the first experiment, responses to a single key were reinforced periodically during stimuli from one half of the stimulus continuum, and responses during other stimuli were extinguished. In the second experiment, two response keys were simultaneously available, with reinforcement for each response alternative associated with different halves of the stimulus continuum. Conditions of the second experiment involved either free-operant or discrete-trial stimulus presentations. Results from these experiments show that positive dimensional contrast appeared in discrimination tasks with one or two response alternatives, but only with free-operant procedures. In addition, discrimination between stimulus classes established by differential reinforcement was assessed as accurately by continuous rate measures as by discrete response choice in the two-alternative situation. The general implication of these experiments is that response rate measures, when properly applied, may reveal sources of variation within stimulus classes, such as dimensional contrast, that are not evident with discrete measures.

Conjoint control of performance in conditional discriminations by successive and simultaneous stimuli

In a conditional discrimination, reinforcement of pigeons' responses to pairs of simultaneously presented wavelength stimuli depended on the orientation of white lines superimposed on the wavelengths. Over different conditions in Experiment 1, three wavelength differences were combined with two differences between successively presented line orientations. Measures of stimulus discriminability increased with increases in the difference between both orientation and wavelength stimuli. Conditional-discrimination performance was thus conjointly determined by stimulus disparity in the successive and simultaneous discriminations. In Experiment 2, ratios of rates of reinforcement contingent upon the two categories of correct responses were varied over several conditions for difficult and easy discriminations. Ratios of responses to wavelength pairs were sensitive to variations in the reinforcement ratio to a greater extent for the more difficult orientation discrimination than for the easier orientation discrimination. Performance in the conditional discrimination was therefore determined by the interacting effects of stimulus disparity and the relative rates of reinforcement contingent upon the two correct choices. It was concluded that the effect of temporally distant reinforcement on behavior in a prevailing schedule component is attenuated to an extent that depends on similarity of stimuli that delineate the successive components.

An equation for behavioral contrast

Pigeons were trained on a three-component multiple schedule in which the rates of reinforcement in the various components were systematically varied. Response rates were described by an equation that posits that the response-strengthening effects of reinforcement are inversely related to the context of reinforcement in which it occurs, and that the context is calculated as the weighted average of the various sources of reinforcement in the situation. The quality of fits was comparable to that found with previous quantitative analyses of concurrent schedules, especially for relative response rates, with over 90% of the variance accounted for in every case. As with previous research, reinforcements in the component that was to follow received greater weights in determining the context than did reinforcements in the preceding component.

Temporal proximity and reinforcement sensitivity in multiple schedules

Distributions of reinforcers between two components of multiple variable-interval schedules were varied over a number of conditions. Sensitivity to reinforcement, measured by the exponent of the power function relating ratios of responses in the two components to ratios of reinforcers obtained in the components, did not differ between conditions with 15-s or 60-s component durations. The failure to demonstrate the "short-component effect," where sensitivity is high for short components, was consistent with reanalysis of previous data. With 60-s components, sensitivity to reinforcement decreased systematically with time since component alternation, and was higher in the first 15-s subinterval of the 60-s component than for the component whose total duration was 15 s. Varying component duration and sampling behavior at different times since component transition may not be equivalent ways of examining the effects of average temporal distance between components.