Time allocation and response rate

The active time model of concurrent choice

The following paper describes a steady-state model of concurrent choice, termed the active time model (ATM). ATM is derived from maximization principles and is characterized by a semi-Markov process. The model proposes that the controlling stimulus in concurrent variable-interval (VI) VI schedules of reinforcement is the time interval since the most recent response, termed here "the active interresponse time" or simply "active time." In the model after a response is generated, it is categorized by a function that relates active times to switch/stay probabilities. In the paper the output of ATM is compared with predictions made by three other models of operant conditioning: melioration, a version of scalar expectancy theory (SET), and momentary maximization. Data sets considered include preferences in multiple-concurrent VI VI schedules, molecular choice patterns, correlations between switching and perseveration, and molar choice proportions. It is shown that ATM can account for all of these data sets, while the other models produce more limited fits. However, rather than argue that ATM is the singular model for concurrent VI VI choice, a consideration of its concept space leads to the conclusion that operant choice is multiply-determined, and that an adaptive viewpoint-one that considers experimental procedures both as selecting mechanisms for animal choice as well as tests of the controlling variables of that choice-is warranted.

Toward the Unification of Molecular and Molar Analyses

Three categories of behavior analysis may be called molecular, molar, and unified. Molecular analyses focus on how manual shaping segments moment-to-moment behaving into new, unified, hierarchically organized patterns. Manual shaping is largely atheoretical, qualitative, and practical. Molar analyses aggregate behaviors and then compute a numerical average for the aggregate. Typical molar analyses involve average rate of, or average time allocated to, the aggregated behaviors. Some molar analyses have no known relation to any behavior stream. Molar analyses are usually quantitative and often theoretical. Unified analyses combine automated shaping of moment-to-moment behaving and molar aggregates of the shaped patterns. Unified controlling relations suggest that molar controlling relations like matching confound shaping and strengthening effects of reinforcement. If a molecular analysis is about how reinforcement organizes individual behavior moment by moment, and a molar analysis is about how reinforcement encourages more or less of an activity aggregated over time, then a unified analysis handles both kinds of analyses. Only theories engendered by computer simulation appear to be able to unify all three categories of behavior analysis.

Inter-response-time reinforcement and relative reinforcer frequency control choice

In Conditions 1 and 3 of our Experiment 1, rats pressed levers for food in a two-component multiple schedule. The first component was concurrent variable-ratio (VR) 20 variable-interval (VI) 90 s, and the second was concurrent yoked VI (its reinforcement rate equaled that of the prior component's VR) VI 90 s. In Condition 2, the VR was changed to tandem VR 20, differential reinforcement of low rates (DRL) 0.8 s. Local response rates were higher in the VR than in the yoked VI schedule, and this difference disappeared between tandem VR DRL and yoked VI. The relative time allocations to VR and yoked VI, as well as to tandem VR DRL and yoked VI, were approximately the same across conditions. In Experiment 2, rats chose in a single session between five different VI pairs, each lasting for 12 reinforcer presentations (variable-environment procedure). The across-schedule hourly reinforcement rates were 120 and 40, respectively, in Conditions 1-3 and 4-6. During Conditions 2 and 5, one lever's VI was changed to tandem VI, DRL 2 s. High covariation between relative time allocations and relative reinforcer frequencies, as well as invariance in local response rates to the schedules, was evident in all conditions. In addition, the relative local response rates were biased toward the unchanged VI in Conditions 2 and 5. These results demonstrate two-process control of choice: Inter-response-time reinforcement controls the local response rate, and relative reinforcer frequency controls relative time allocations.

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.

Short-term memory in the pigeon: stimulus-response associations

THREE PIGEONS PECKED FOR FOOD IN TWO EXPERIMENTS IN WHICH EACH TRIAL CONSISTED OF TWO PHASES: a study and a test phase. The study phase in Experiment I consisted of two stimulus-response pairs presented successively. Each pair consisted of the illumination of a left or right key (the stimulus) and a peck on the lighted side key (the response). The study phase in Experiment II consisted of three such pairs presented successively. A retention interval, varied between 0.1 and 4.0 sec, separated the study phase from the test phase. The test phase of a trial began with the illumination of the center key by one of two (Experiment I) or three (Experiment II) colors. This color was the same as the stimulus element of one of the pairs in the study phase. A reinforcer was presented if a subject then emitted the response element of the indicated stimulus-response pair. The results provide information on the conditions that enable a pigeon to remember the responses most recently emitted in the presence of various stimuli. The results suggest an account of the maintenance of behavior that is temporally noncontiguous with reinforcement.

Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement

The spatiotemporal patterns of behavior exhibited by two pigeons during a variable-interval 15-second schedule of food reinforcement, a variable-interval 5-minute schedule, and then extinction of key pecking were recorded using an apparatus that continuously tracked the position of the bird in the experimental chamber. The variable-interval 15-second schedule produced a close-to-key pattern between reinforcements with two types of regular excursions from the region of the key frequently occurring after reinforcement. Subsequent exposure to the variable-interval 5-minute schedule produced more extended and extremely regular patterns between responses. Reinstatement of the variable-interval 15-second schedule reestablished the close-to-key pattern with regular excursions frequently occurring after reinforcement. During extinction the spatiotemporal patterns that had developed during the variable-interval 5-minute schedule reappeared and gradually dissipated. These patterns may have been a form of superstitious behavior.

Time-allocation, matching, and contrast

A variable-interval schedule arranged food reinforcement for key pecking by pigeons on a single operandum at two rates, corresponding to two classes of reinforced interresponse times ranging from 1.5 to 2.5 sec and from 3.5 to 4.5 sec. The scheduled reinforcement rate for the higher component response rate was constant and equivalent to that of a variable-interval 4-min schedule. The scheduled reinforcement rate for the lower component response rate varied from zero to over 100 per hour. The number of occurrences of the constant component response rate varied inversely with the reinforcement rate for the variable component. This result, by definition a concurrent reinforcement interaction, or contrast, was the combined effect of two time-allocation functions, which together determine mean response rate: the time allocated to both component rates as a function of the total reinforcement rate, and the time allocated to a particular component rate as a function of the percentage of reinforcements for that component. The present experiment reveals a further parallel between the controlling relations for free responding on a single operandum and those for choice between two operanda; in each case, a concurrent reinforcement interaction can be found that corresponds to matching.

Reinforcement rate and interresponse time differentiation

Reinforcement rate and differential reinforcement of IRTs were independently manipulated to assess their relative contribution to the control of interresponse times (IRTs). Modified percentile reinforcement schedules (Platt, 1973) allowed control of reinforcement rate while longest or shortest IRTs were selectively reinforced. In the absence of differential IRT reinforcement, mean IRT decreased with increasing reinforcement rate. Compared to this small effect of reinforcement rate, reinforcement of long IRTs produced large changes in mean IRT at constant reinforcement rates. No interaction of reinforcement rate and IRT reinforcement was detected. The demonstration of large IRT changes in the absence of reinforcement-rate changes indicates the precedence of IRT reinforcement over molar reinforcement-rate correlations in the determination of IRTs in these procedures.

Short-term memory in the pigeon: the previously reinforced response

Eighteen pigeons served in a discrete-trials short-term memory experiment in which the reinforcement probability for a peck on one of two keys depended on the response reinforced on the previous trial: either the probability of reinforcement on a trial was 0.8 for the same response reinforced on the previous trial and was 0.2 for the other response (Group A), or, it was 0 or 0.2 for the same response and 1.0 or 0.8 for the other response (Group B). A correction procedure ensured that over all trials reinforcement was distributed equally across the left and right keys. The optimal strategy was either a winstay, lose-shift strategy (Group A) or a win-shift, lose-stay strategy (Group B). The retention interval, that is the intertrial interval, was varied. The average probability of choosing the optimal alternative reinforced 80% of the time was 0.96, 0.84, and 0.74 after delays of 2.5, 4.0, and 6.0 sec, respectively for Group A, and was 0.87, 0.81, and 0.55 after delays of 2.5, 4.0, and 6.0 sec, respectively, for Group B. This outcome is consistent with the view that behavior approximated the optimal response strategy but only to an extent permitted by a subject's short-term memory for the cue correlated with reinforcement, that is, its own most-recently reinforced response. More generally, this result is consistent with "molecular" analyses of operant behavior, but is inconsistent with traditional "molar" analyses holding that fundamental controlling relations may be discovered by routinely averaging over different local reinforcement contingencies. In the present experiment, the molar results were byproducts of local reinforcement contingencies involving an organism's own recent behavior.

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.

Organization in memory and behavior

SOME COMMON REINFORCEMENT CONTINGENCIES MAKE THE DELIVERY OF A REINFORCER DEPEND ON THE OCCURRENCE OF BEHAVIOR LACKING SIGNIFICANT TEMPORAL STRUCTURE: a reinforcer may be contingent on nearly instantaneous responses such as a pigeon's key peck, a rat's lever press, a human's button press or brief verbal utterance, and so on. Such a reinforcement contingency conforms much more closely to the functionalist tradition in experimental psychology than to the structuralist tradition. Until recently, the functionalist tradition, in the form of a kind of associationism, typified most research on human learning and memory. Recently, however, research on human memory has focused more on structural issues: now the basic unit of analysis often involves an organized temporal pattern of behavior. A focus on the interrelations between the function and structure of behavior identifies a set of independent and dependent variables different from those identified by certain common kinds of "molar" behavioral analyses. In so doing, such a focus redefines some of the significant issues in the experimental analysis of behavior.

Interresponse-time shaping by variable-interval-like interresponse-time reinforcement contingencies

The interresponse-time reinforcement contingencies and distributions of interreinforcement intervals characteristic of certain variable-interval schedules were mimicked by reinforcing each key peck with a probability equal to the duration of the interresponse time it terminated, divided by the scheduled mean interreinforcement interval. The interresponse-time reinforcement contingency was then eliminated by basing the probability of reinforcement on the fifth interresponse time preceding the key peck. Even though distributions of interreinforcement intervals were unaffected by this manipulation, response rates consistently increased. A second experiment replicated this effect and showed it to combine additively with that of mean reinforcement rate. These results provide strong support for the contention that current analyses of variable-interval response rates that ignore the inherent interresponse-time reinforcement contingency may be seriously in error.

Molecular contingencies: reinforcement probability

Pigeons obtained food by responding in a discrete-trials two-choice probability-learning experiment involving temporal stimuli. A given response alternative, a left- or right-key peck, had 11 associated reinforcement probabilities within each session. Reinforcement probability for a choice was an increasing or a decreasing function of the time interval immediately preceding the choice. The 11 equiprobable temporal stimuli ranged from 1 to 11 sec in 1-sec classes. Preference tended to deviate from probability matching in the direction of maximizing; i.e., the percentage of choices of the preferred response alternative tended to exceed the probability of reinforcement for that alternative. This result was qualitatively consistent with probability-learning experiments using visual stimuli. The result is consistent with a molecular analysis of operant behavior and poses a difficulty for molar theories holding that local variations in reinforcement probability may safely be disregarded in the analysis of behavior maintained by operant paradigms.

Short-term memory in the pigeon: relative recency

Three pigeons pecked for food in an experiment in which each trial consisted of two phases. The first phase consisted of a pattern of three successively illuminated, randomly selected left or right keys. A subject was required to peck each of the lighted keys as they appeared. Thus, in the first phase, a subject emitted a pattern of three left- or right-key pecks. Over trials, all eight possible patterns appeared. A time interval separated the first phase from the second phase, which began with presentation of a randomly selected one of three cues. A reinforcer was delivered in the second phase if a subject pecked the side key that had appeared in the first phase in an ordinal position corresponding to the cue presented in the second phase. That is, the three cues probed a pigeon's memory for the side key it had pecked first, second, or third, in the first phase of a trial. The results show that a pigeon can remember for more than 4 sec the order in which it has just seen and pecked two lighted keys: a pigeon can remember the temporal organization or pattern of events in its recent environment. Consequently, the functional stimulus present when a reinforcer is delivered may include a subject's short-term memory for the temporal organization of recent events, such as the pattern of its own recent behavior. This possibility is consistent with a molecular analysis of operant behavior focusing on local patterns of behavior.

Response rate, latency, and resistance to change

Pigeons were trained on a multiple variable-interval/variable-interval schedule with pacing contingencies that generated high response rates in one component and low response rates in the other. Timeout periods separated the schedule components. During resistance-to-change tests, response-independent food was presented during the timeout periods, and the duration of that food presentation was varied among test sessions. Response rates in the schedule components decreased and latencies to the first response increased as a function of the duration of food presentations during the timeout. Both dependent measures changed about the same amount relative to their own baseline levels. The conclusions are that baseline response rates controlled by pacing contingencies are equally resistant to change, given equal reinforcement densities, and latency is a sensitive measure of resistance to change.

Reinforcement of behavioral patterns: shaping a scallop

Temporal patterns of key pecking by pigeons were shaped by a schedule in which the delivery of food was contingent upon a measure of the overall extent to which the temporal pattern of behavior within a 5-sec trial conformed to a required pattern. This pattern approximated a constant rate of change in the rate of key pecking throughout the 5-sec trial. In comparison with behavior maintained by a classical fixed-interval 5-sec schedule, the new schedule controlled a better approximation to a "scallop" within individual trials and greatly reduced intersubject variability. These results are consistent with the view that the delivery of a reinforcer after a behavioral pattern a few seconds in duration may strengthen the entire pattern as a unit, or operant. The response topography contiguous with reinforcement may be a negligible fraction of the strengthened operant. One implication of this view is that mean response rate for such brief responses as key pecks and lever presses is a byproduct of whatever patterns are strengthened, and generally will not reveal fundamental controlling relationships, whenever a reinforcer is not contiguous with all the behavior on which it is contingent.

Perception d'une annonce créative par le consommateur: construction et validation d'une échelle de mesure

L'objet de cet article est de clarifier la perception de la créativité publicitaire. Alors que la créativité en publicité est actuellement mesurée par les professionnels du secteur, notre démarche s'appuie sur les jugements des consommateurs (créativité perçue). Notre première étude est constituée d'entretiens qualitatifs exploratoires permettant de mieux cerner la vision implicite des consommateurs en matière de créativité publicitaire. La seconde étude nous a permis de développer une échelle de mesure de la créativité publicitaire perçue. Les publicités perçues comme créatives par le consommateur comporteraient trois dimensions: la nouveauté, la complexité et l'esthétique. La nouveauté correspond à l'association de stimuli à la fois inattendus et pertinents par rapport au message publicitaire. La complexité est associée à un message publicitaire riche de significations multiples. L'esthétique a trait au design spécifique des publicités créatives. Notre recherche permet une meilleure compréhension de la perception de la créativité par les consommateurs, et offre la possibilité pour les praticiens d'améliorer la créativité de leurs campagnes.

Systems-level integration of interval timing and reaction time

Reaction time (RT) procedures are a prominent tool for the study of information processing by humans and other animals. The interpretation of how RT changes after manipulating the appropriate experimental variables has contributed to the contemporary understanding of a variety of cognitive constructs, including attention and memory. With the use of properly designed tasks, evaluating how RT is modified in response to various neural perturbations has become common within the realms of behavioral and cognitive neuroscience. One interesting observation made during both human and animal RT experiments is that the RT to a signal often speeds-up as more time is allotted to prepare for the signal's onset-referred to as the preparatory interval (PI) effect. In the human RT literature, the PI effect has been used as evidence for time estimation playing a fundamental role in the determination of RT. On the other hand, our theoretical understanding of time estimation remains largely divorced from the RT findings in the animal cognition literature. In order to bridge these different perspectives, we provide here a review of the behavioral parallels between RT and interval-timing experiments. Moreover, both the PI effect and interval timing are shown to be jointly influenced by neuropathologies such as Parkinson's disease in humans or dopamine-depleting brain lesions in experimental animals. The primary goal of this review is to consider human and animal RT experiments within the broader context of interval timing. This is accomplished by first integrating human RT theory with scalar timing theory-the leading model of interval timing. Following this, both RT and interval timing are discussed at a brain systems level insofar as these two processes share common neural substrates. Our conclusion is that interval timing and RT processes are in fact two sides of the same coin.

Interresponse-time sensitivity during discrete-trial and free-operant concurrent variable-interval schedules

Two experiments investigated the sensitivity of pigeons' choice to elapsed time since the last response (i.e., to inter-response time [IRT]) during concurrent variable-interval variable-interval schedules. Experiment 1 used a two-key discrete-trial procedure with variable intertrial intervals. Experiment 2 employed a three-key free-operant procedure. In both experiments choice was found to be a function of the active-schedule IRT, defined as the time since the most recent response. Monte Carlo simulations show how this finding permits the joining of several seemingly incompatible data sets held to both support and contradict a kind of choice strategy, termed momentary maximizing, which attempts to maximize momentary reinforcement probabilities. The studies suggest that only two variables are needed to describe the static molecular structure of concurrent variable-interval choice: active-schedule IRTs and "response states" consisting of the last one or two schedule choices.

The S-R issue: its status in behavior analysis and in Donahoe and Palmer's learning and complex behavior

The central focus of this essay is whether the effect of reinforcement is best viewed as the strengthenng of responding or the strengthening of the environmental control of responding. We make the argument that adherence to Skinner's goal of achieving a moment-to-moment analysis of behavior compels acceptance of the latter view. Moreover, a thoroughgoing commitment to a moment-to-moment analysis undermines the fundamental distinction between the conditioning process instantiated by operant and respondent contingencies while buttressing the crucially important differences in their cumulative outcomes. Computer simulations informed by experimental analyses of behavior and neuroscience are used to illustrate these points.