Changeover behavior and preference in concurrent schedules

The active time model of concurrent choice

The following paper describes a steady-state model of concurrent choice, termed the active time model (ATM). ATM is derived from maximization principles and is characterized by a semi-Markov process. The model proposes that the controlling stimulus in concurrent variable-interval (VI) VI schedules of reinforcement is the time interval since the most recent response, termed here "the active interresponse time" or simply "active time." In the model after a response is generated, it is categorized by a function that relates active times to switch/stay probabilities. In the paper the output of ATM is compared with predictions made by three other models of operant conditioning: melioration, a version of scalar expectancy theory (SET), and momentary maximization. Data sets considered include preferences in multiple-concurrent VI VI schedules, molecular choice patterns, correlations between switching and perseveration, and molar choice proportions. It is shown that ATM can account for all of these data sets, while the other models produce more limited fits. However, rather than argue that ATM is the singular model for concurrent VI VI choice, a consideration of its concept space leads to the conclusion that operant choice is multiply-determined, and that an adaptive viewpoint-one that considers experimental procedures both as selecting mechanisms for animal choice as well as tests of the controlling variables of that choice-is warranted.

Failure to find a distance effect in pigeon choice

Primates take longer to choose between alternatives with smaller differences in value. This effect-a particular instance of the distance effect in symbolic comparisons-has not been replicated in birds. Instead, birds appear to respond independently to each alternative, such that the latency to choose depends primarily on the alternative of highest value. Three experiments tested for the distance effect in pigeons under conditions not previously considered. Experiment 1 presented pigeons with forced- and binary free-choice trials, where each alternative was one of three possible delays to reinforcement (4, 8, and 16 s). Pigeons were exposed to the choice stimuli for different amounts of time and with different sample response requirements prior to the choice response. Experiment 2 added a fourth (0-s delay) alternative. Experiment 3 substituted the 16-s delay with a second 4-s delay. In all experiments, pigeons systematically chose the shortest delay to reinforcement. Latency to choose the 4-s delay did not vary when choosing against the 8-s or 16-s delay, regardless of whether choice stimuli were exposed for the duration of nine pecks (Experiment 1), or whether a 0-s delay alternative was sometimes present (Experiment 2). Latency to choose the preferred of two identical alternatives (4-s vs. 4-s) was shorter than the latency to choose between different alternatives (4-s vs. 8-s; Experiment 3); this is the opposite of a distance effect. These results show no evidence of a distance effect in pigeon choice, consistent with the hypothesis that pigeons respond independently to each choice alternative.

Pragmatism, mathematical models, and the scientific ideal of prediction and control

Mathematical models are often held to be valuable, if not necessary, for theories and explanations in the quantitative analysis of behavior. The present review suggests that mathematical models primarily derived from the observation of functional relations do indeed contribute to the scientific value of theories and explanations, even though the final form of the models appears to be highly abstract. However, mathematical models not primarily so derived risk being essentialist in character, based on a particular view of formal causation. Such models invite less effective and frequently mentalistic theories and explanations of behavior. Models may be evaluated in terms of both (a) the verbal processes responsible for their origin and development and (b) the prediction and control engendered by the theories and explanations that incorporate the models, however indirect or abstract that prediction and control may be. Overall, the present review suggests that technological application and theoretical contemplation may be usefully viewed as continuous and overlapping forms of scientific activity, rather than dichotomous and mutually exclusive.

Effects of changeover delay on response allocation during probe tests

The present study assessed whether a pattern of responding that develops when choosing between two alternatives generalizes to novel choice tests when alternatives are presented in new combinations. Pigeons were trained on a two-component multiple schedule. In both components, a concurrent variable-interval (VI) 40-s VI 80-s schedule was used. The COD was 1 s in one component and 10 s in the other. The long COD produced consistently longer dwell times than the short COD did. Following training, subjects were presented with four types of probe-test components in which one alternative was drawn from the component with the short COD and one alternative was drawn from the component with the long COD. When the schedule values of the two alternatives were identical (VI 40 vs. VI 40 and VI 80 vs. VI 80), subjects preferred the alternative trained with the long COD (Ms = .78 and .61, respectively). Additionally, subjects preferred the VI 40-s alternative trained with the long COD to the VI 80-s alternative that was trained with the short COD (M = .85). Systematic preference was not observed when subjects were given a choice between the VI 40-s alternative that was trained with the short COD and the VI 80-s alternative that was trained with the long COD. These results demonstrate that a stimulus associated with a longer COD, and thus longer dwell times in baseline training, may be more preferred during probe tests than expected on the basis of the rate of primary reinforcement associated with that stimulus.

Preference as a function of active interresponse times: a test of the active time model

In this article, we describe a test of the active time model for concurrent variable interval (VI) choice. The active time model (ATM) suggests that the time since the most recent response is one of the variables controlling choice in concurrent VI VI schedules of reinforcement. In our experiment, pigeons were trained in a multiple concurrent similar to that employed by Belke (1992), with VI 20-s and VI 40-s schedules in one component, and VI 40-s and VI 80-s schedules in the other component. However, rather than use a free-operant design, we used a discrete-trial procedure that restricted interresponse times to a range of 0.5-9.0 s. After 45 sessions of training, unreinforced probe periods were mixed with reinforced training periods. These probes paired the two stimuli associated with the VI 40-s schedules. Further, the probes were defined such that during their occurrence, interresponse times were either "short" (0.5-3.0 s) or "long" (7.5-9.0 s). All pigeons showed a preference for the stimulus associated with the relatively rich VI 40-s schedule--a result mirroring that of Belke. We also observed, though, that this preference was more extreme during long probes than during short probes--a result predicted by ATM.

A further application of the active time model to multiple concurrent variable-interval schedules

In this experiment we show that the active time model (ATM) accurately predicts probe data from multiple concurrent VI VI schedules. Subjects were trained under a concurrent VI 30-s VI 60-s and a concurrent VI 60-s VI 120-s schedule. Two types of unreinforced probes were then conducted. The first paired the two VI 60-s stimuli. These stimuli, while equivalent in their associated absolute rates of reinforcement, differed in their relative rates of reinforcement. The second probe paired the VI 30-s stimulus with the relatively rich VI 60-s stimulus. In contrast with the first probe, these stimuli differed in their absolute rates of reinforcement, while being similar in their relative rates. During the first set of probes, birds preferred the VI 60-s stimulus trained with the VI 120-s schedule. During the second set of probes, birds were indifferent to the two stimuli. These results are less extreme than others reported in the literature. Nonetheless, we found that ATM accurately fit individual subject data in both sets of probes. In contrast a variant of scalar expectancy theory did not fit the data at either the individual or group level.

An application of the active time model to multiple concurrent variable-interval schedules

The current experiment investigates whether an active time model can account for anomalous results that have emerged from multiple schedule, concurrent variable-interval (VI) VI experiments. The model assumes that (1) during concurrent VI VI training pigeons learn a function that relates time since the most immediate response, i.e., active time, to changeover probabilities and (2) that molar preference is the result of an interaction between inter-response time frequencies and the learned active time changeover functions. Pigeons were trained under a concurrent VI 30-s VI 30-s schedule and a concurrent VI 60-s VI 60-s schedule. Probes were conducted in which VI 30-s and VI 60-s stimuli were paired. During these probes, birds allocated choices equally to the stimuli. The active time model accurately fit individual subject data. In contrast data were not fit by a variant of scalar expectancy theory proposed by Gibbon [Gibbon, J., 1995. Dynamics of time matching: arousal makes better seem worse. Psychon. Bull. Rev. 2, 208-215].

Discrete-trial vs. continuous free-operant procedures in assessing whether reinforcement context affects reinforcement value

Pigeons were trained to respond to two alternating concurrent reinforcement schedules. The reinforcement probabilities were .05 and .10 in one component, and .10 and .20 in the other. In one condition, the pigeons received training on a discrete-trial procedure in which the keylights remained illuminated for 5s or until a response occurred. In another condition, pigeons received training on a procedure in which the reinforcement contingencies were the same as in the discrete-trial procedure, but the stimuli were not turned off after 5s or after a response. Following training in each condition, probe tests were presented. In both conditions, the .20 alternative was, overall, preferred to the .05 alternative during probe tests. Following discrete-trial training, there was no reliable preference between the two .10 alternatives. However, when the stimuli remained illuminated during the intertrial interval periods during training, probe tests results showed preference for the .10 alternative that had been presented in the leaner context during training. The pattern of results is consistent with the notion that probe preference can be influenced both by the absolute reinforcement schedules associated with each alternative, as well as changeover behavior developed during training.

Influence of temporal context on value in the multiple-chains and successive-encounters procedures

This set of studies explored the influence of temporal context across multiple-chain and multiple-successive-encounters procedures. Following training with different temporal contexts, the value of stimuli sharing similar reinforcement schedules was assessed by presenting these stimuli in concurrent probes. The results for the multiple-chain schedule indicate that temporal context does impact the value of a conditioned reinforcer consistent with delay-reduction theory, such that a stimulus signaling a greater reduction in delay until reinforcement has greater value. Further, nonreinforced stimuli that are concurrently presented with the preferred terminal link also have greater value, consistent with value transfer. The effects of context on value for conditions with the multiple-successive-encounters procedure, however, appear to depend on whether the search schedule or alternate handling schedule was manipulated, as well as on whether the tested stimuli were the rich or lean schedules in their components. Overall, the results help delineate the conditions under which temporal context affects conditioned-reinforcement value (acting as a learning variable) and the conditions under which it does not (acting as a performance variable), an issue of relevance to theories of choice.

Matching: its acquisition and generalization

Choice typically is studied by exposing organisms to concurrent variable-interval schedules in which not only responses controlled by stimuli on the key are acquired but also switching responses and likely other operants as well. In the present research, discriminated key-pecking responses in pigeons were first acquired using a multiple schedule that minimized the reinforcement of switching operants. Then, choice was assessed during concurrent-probe periods in which pairs of discriminative stimuli were presented concurrently. Upon initial exposure to concurrently presented stimuli, choice approximated exclusive preference for the alternative associated with the higher reinforcement frequency. Concurrent schedules were then implemented that gave increasingly greater opportunities for switching operants to be conditioned. As these operants were acquired, the relation of relative response frequency to relative reinforcement frequency converged toward a matching relation. An account of matching with concurrent schedules is proposed in which responding exclusively to the discriminative stimulus associated with the higher reinforcement frequency declines as the concurrent stimuli become more similar and other operants-notably switching-are acquired and generalize to stimuli from both alternatives. The concerted effect of these processes fosters an approximate matching relation in commonly used concurrent procedures.

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variable-interval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variable-interval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variable-interval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variable-interval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.

Choice, changing over, and reinforcement delays

In three experiments, pigeons were used to examine the independent effects of two normally confounded delays to reinforcement associated with changing between concurrently available variable-interval schedules of reinforcement. In Experiments 1 and 2, combinations of changeover-delay durations and fixed-interval travel requirements were arranged in a changeover-key procedure. The delay from a changeover-produced stimulus change to a reinforcer was varied while the delay between the last response on one alternative and a reinforcer on the other (the total obtained delay) was held constant. Changeover rates decreased as a negative power function of the total obtained delay. The delay between a changeover-produced stimulus change had a small and inconsistent effect on changeover rates. In Experiment 3, changeover delays and fixed-interval travel requirements were arranged independently. Changeover rates decreased as a negative power function of the total obtained delay despite variations in the delay from a change in stimulus conditions to a reinforcer. Periods of high-rate responding following a changeover, however, were higher near the end of the delay from a change in stimulus conditions to a reinforcer. The results of these experiments suggest that the effects of changeover delays and travel requirements primarily result from changes in the delay between a response at one alternative and a reinforcer at the other, but the pattern of responding immediately after a changeover depends on the delay from a changeover-produced change in stimulus conditions to a reinforcer.