Choice between response rates

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.

Time allocation and response rate

Food reinforcement for key pecking by three pigeons was arranged by a variable-interval schedule and a device that assigned each reinforcement to one of 10 component response rates corresponding to 10 classes of equally reinforced interresponse times ranging from 1.0 to 6.0 sec in 0.5-sec classes. The overall number of reinforcements per hour was varied from one to more than 60. Overall response rate was a monotonically increasing, negatively accelerated function of the overall number of reinforcements per hour. This function was decomposed into two time-allocation functions: (1) the time allocated to all of the reinforced component response rates as a function of the total reinforcement rate, and, (2) the time allocated to a particular reinforced component response rate as a function of the reinforcement rate for that component. Asymptotic response rate was predicted by combining the asymptotes of the two separate time-allocation functions: virtually all of the time was spent responding, and the percentage of the time spent responding that was allocated to a particular reinforced component response rate roughly equalled the relative reinforcements per hour for that component.

Time-allocation, matching, and contrast

A variable-interval schedule arranged food reinforcement for key pecking by pigeons on a single operandum at two rates, corresponding to two classes of reinforced interresponse times ranging from 1.5 to 2.5 sec and from 3.5 to 4.5 sec. The scheduled reinforcement rate for the higher component response rate was constant and equivalent to that of a variable-interval 4-min schedule. The scheduled reinforcement rate for the lower component response rate varied from zero to over 100 per hour. The number of occurrences of the constant component response rate varied inversely with the reinforcement rate for the variable component. This result, by definition a concurrent reinforcement interaction, or contrast, was the combined effect of two time-allocation functions, which together determine mean response rate: the time allocated to both component rates as a function of the total reinforcement rate, and the time allocated to a particular component rate as a function of the percentage of reinforcements for that component. The present experiment reveals a further parallel between the controlling relations for free responding on a single operandum and those for choice between two operanda; in each case, a concurrent reinforcement interaction can be found that corresponds to matching.

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.

Organization in memory and behavior

SOME COMMON REINFORCEMENT CONTINGENCIES MAKE THE DELIVERY OF A REINFORCER DEPEND ON THE OCCURRENCE OF BEHAVIOR LACKING SIGNIFICANT TEMPORAL STRUCTURE: a reinforcer may be contingent on nearly instantaneous responses such as a pigeon's key peck, a rat's lever press, a human's button press or brief verbal utterance, and so on. Such a reinforcement contingency conforms much more closely to the functionalist tradition in experimental psychology than to the structuralist tradition. Until recently, the functionalist tradition, in the form of a kind of associationism, typified most research on human learning and memory. Recently, however, research on human memory has focused more on structural issues: now the basic unit of analysis often involves an organized temporal pattern of behavior. A focus on the interrelations between the function and structure of behavior identifies a set of independent and dependent variables different from those identified by certain common kinds of "molar" behavioral analyses. In so doing, such a focus redefines some of the significant issues in the experimental analysis of behavior.

Conservation, choice, and the concurrent fixed-ratio schedule

Five rats got all of their water in daily 60-minute sessions. Two levers and a water spout were freely available throughout baseline sessions. Contingency sessions offered a choice between two alternative fixed-ratio components, in the form of a choice between the two levers. Each component required a specified number of lever presses for access to the spout, and then a specified number of licks for another choice between components. Given the observed relative frequency, the absolute frequency of selecting each component was predicted accurately by assuming that the subject conserved between baseline and contingency the total amount of a dimension attributable to lever pressing and licking. Several quantitative models for predicting relative frequency were examined. The best of these assumed that the subject would show a nonexclusive preference for the component requiring fewer lever presses.

Choice and reinforcement delay

Previous studies of choice between two delayed reinforcers have indicated that the relative immediacy of the reinforcer is a major determinant of the relative frequency of responding. Parallel studies of choice between two interresponse times have found exceptions to this generality. The present study looked at the choice by pigeons between two delays, one of which was always four times longer than the other, but whose absolute durations were varied across conditions. The results indicated that choice is not uniquely determined by the relative immediacy of reinforcement, but that absolute delays are also involved. Models for concurrent chained schedules appear to be more applicable to the present data than the matching relation; however, these too failed to predict choice for long delays.

Concurrent second-order schedules: some effects of variations in response number and duration

To examine the effects on concurrent performance of independent manipulations of response-unit duration and number, 6 hens were exposed to concurrent second-order schedules of reinforcement. Each first-order operant unit required completion of a fixed-ratio schedule within the time specified by a fixed-interval schedule, with one further response completing the fixed-interval schedule. The fixed-ratio and fixed-interval requirements comprising the first-order operant units were systematically and independently varied under three pairs of concurrent variable-interval schedules to produce differences in the first-order response and duration requirements (response and duration differentials). These manipulations produced consistent changes in response, time, and operant-unit biases. A 1:4 response differential biased the time and operant-unit measures towards the smaller fixed ratio, but to a degree less than the imposed response differential. The response-based biases favored the larger fixed ratio. Duration differentials of 4:1 and 8:1 biased the response and operant-unit measures towards the shorter fixed interval, again less than the imposed duration differential, but the time biases remained close to zero. Both sorts of differentials acted to bias operant-unit completions more systematically than the other measures, but undermatching to the differentials occurred. The undermatching appears to have arisen from a pattern of fix and sample (in which visits to the less preferred alternative involved only a single completed operant unit) under combinations of unequal operant-unit requirements and reinforcer rates. The response and time bias measures appeared to arise as by-products of the changes in operant-unit completions.

Interresponse-time sensitivity during discrete-trial and free-operant concurrent variable-interval schedules

Two experiments investigated the sensitivity of pigeons' choice to elapsed time since the last response (i.e., to inter-response time [IRT]) during concurrent variable-interval variable-interval schedules. Experiment 1 used a two-key discrete-trial procedure with variable intertrial intervals. Experiment 2 employed a three-key free-operant procedure. In both experiments choice was found to be a function of the active-schedule IRT, defined as the time since the most recent response. Monte Carlo simulations show how this finding permits the joining of several seemingly incompatible data sets held to both support and contradict a kind of choice strategy, termed momentary maximizing, which attempts to maximize momentary reinforcement probabilities. The studies suggest that only two variables are needed to describe the static molecular structure of concurrent variable-interval choice: active-schedule IRTs and "response states" consisting of the last one or two schedule choices.

Preference for starting and finishing behavior patterns

Pigeon's key pecking was reinforced with food in two experiments in which the correspondence between preference for starting one of two reinforced behavior patterns and the likelihood of finishing it subsequently was examined. Reinforcers were scheduled according to concurrent schedules for two classes of interresponse times, modified such that reinforcers followed a center-key peck terminating either a shorter interresponse time started by a left-key peck or a longer interresponse time started by a right-key peck. In Experiment 1, the times when reinforcers potentially were available were not discriminated, whereas in Experiment 2 they were. Absolute reinforced pattern durations were varied. The relative frequency of starting a particular pattern was highly correlated with relative frequency of that completed pattern in both experiments. Other relations between starting and finishing a pattern depended on whether reinforced interresponse times were discriminated. For instance, preference for starting a pattern sometimes correlated negatively with the likelihood of subsequently completing it. The present experiments are described as capturing part of the ordinary language meaning of "intention," according to which an organism's behavior at one moment sets the occasion for an observer to say that the organism "intends" in the future to engage in one behavior rather than another.

Discriminated interresponse times: role of autoshaped responses

When discriminated interresponse-time (IRT) procedures have been used to assess preference relations among temporally extended operants, deviations from matching have been obtained. Using a yoked-control procedure, the present study found that key pecking in a discriminated IRT procedure has two sources of strength--that arising from the response-reinforcer contingency that is explicitly arranged, and that arising from a stimulus-reinforcer contingency that is a by-product of the explicitly arranged contingency. The key pecking of all lead birds, and that of 3 of the 4 birds exposed to a yoked autoshaping procedure, was controlled by the keylight that signaled the lead birds' criterion IRTs. Because stimulus control of key pecking by the keylight, whether autoshaped or discriminative, fosters deviations from matching, the discriminated IRT procedure does not provide an appropriate basis for conclusions about preference relations among IRTs.

Timing, learning, and forgetting

A computer-simulation model (AL) was developed for molecular and molar operant conditioning data. AL assumes a simple forgetting rule and all-or-nothing associations between memory representations of responses and reinforcers. AL describes acquisition and steady-state behavior in both concurrent interresponse time (IRT) schedules and in ordinary concurrent schedules. It describes functions relating the relative frequency of an IRT to IRT duration and to reinforcement frequency. AL accurately predicts for ordinary concurrent variable-interval schedules that the log ratio of responses to the two alternatives is a linear function of the log ratio of the reinforcers they deliver, and that the slope of this straight line approximates the degree of undermatching that characterizes the behavior of real animals. In addition, AL successfully describes the fact that in real data of Nevin, Heyman, and others, the probability of a changeover from one response to the other is roughly independent of the number of consecutive responses to the same alternative preceding the changeover. A previous version of AL assimilated data on temporal psychophysics, such as that animals bisect two temporal intervals at the geometric mean, and that the Weber fraction in DRL schedules is approximately constant. AL therefore describes transient and steady-state molecular and molar performances as well as data on temporal psychophysics.

The local organization of behavior: dissociations between a pigeon's behavior and self-reports of that behavior

The purpose of the experiment was to study the relation between what an organism does in a setting that demands temporal patterning of behavior and what it reports it has done. More specifically, a pigeon produced two classes, shorter and longer, of temporal patterns of key pecks (interresponse times) on a center key. Occasionally, a symbolic matching-to-sample probe arranged on side keys asked the pigeon whether its most recent pattern was a shorter or longer one. The longer reinforced pattern was always three times as long as the shorter one and the two patterns were reinforced equally often. Absolute duration of reinforced patterns was varied. In some conditions, interresponse-time distributions on the center key were bimodal, indicating a clear behavioral adaptation to the contingency, yet a bird did not report very well by appropriate side-key responding what its most recent interresponse time had been. In other conditions, the interresponse-time distributions were less clearly bimodal, yet a bird reported more accurately its previous interresponse time as shorter or longer. Thus, there was a dissociation between how well behavior on the center key conformed to the schedule requirement and how well a bird reported what it was doing on the center key. In addition, as absolute duration of the reinforced patterns was increased, a bird categorized its most recent pattern less well even as its preference for the shorter pattern increased dramatically. These results were interpreted as an example of the phenomenon of dissociation between tacit knowledge and knowledge.

The local organization of behavior: discrimination of and memory for simple behavioral patterns

A procedure was developed to enable nonverbal organisms to report what they remember of the temporal organization of their recent behavior. A baseline behavior with known temporal structure was established by a concurrent variable-interval variable-interval schedule for two temporal patterns of behavior (two different classes of reinforced inter-response times). The five pigeon subjects emitted these two temporal patterns on a center key and were occasionally given a short-term memory probe for their most-recently-emitted pattern. The probes consisted of symbolic delayed matching-to-sample tests, in which a response on a green side key was reinforced if the most recent pattern belonged to the shorter reinforced class, and a response to a red side key was reinforced if the most recent pattern belonged to the longer reinforced class. All subjects could report with over ninety percent accuracy what their most recently emitted behavioral pattern was when a retention interval separating the pattern from the memory probe was only .1 seconds. The retention interval was then manipulated, and it was found that recall for a pattern was frequently above chance after a delay of as much as eight seconds. Thus, pigeons can remember their most recent interresponse time not only right after it is emitted, but for several seconds thereafter. In other conditions, the patterns themselves were manipulated. It was found that as the patterns became more similar, discrimination became poorer. These results agree with the view that reinforcement tends to organize and integrate the local structure of behavior to the extent to which that structure is remembered.