The local organization of behavior: discrimination of and memory for simple behavioral patterns

The active time model of concurrent choice

The following paper describes a steady-state model of concurrent choice, termed the active time model (ATM). ATM is derived from maximization principles and is characterized by a semi-Markov process. The model proposes that the controlling stimulus in concurrent variable-interval (VI) VI schedules of reinforcement is the time interval since the most recent response, termed here "the active interresponse time" or simply "active time." In the model after a response is generated, it is categorized by a function that relates active times to switch/stay probabilities. In the paper the output of ATM is compared with predictions made by three other models of operant conditioning: melioration, a version of scalar expectancy theory (SET), and momentary maximization. Data sets considered include preferences in multiple-concurrent VI VI schedules, molecular choice patterns, correlations between switching and perseveration, and molar choice proportions. It is shown that ATM can account for all of these data sets, while the other models produce more limited fits. However, rather than argue that ATM is the singular model for concurrent VI VI choice, a consideration of its concept space leads to the conclusion that operant choice is multiply-determined, and that an adaptive viewpoint-one that considers experimental procedures both as selecting mechanisms for animal choice as well as tests of the controlling variables of that choice-is warranted.

Why it is crucial to understand thinking and feeling: An analysis and application to drug abuse

Behavior analysis has long accepted the legitimacy of the analysis of private events in a natural science of behavior. However, the topic has languished as a focus of empirical research in either applied or basic arenas. We argue that recent empirical work examining the bidirectional nature of verbal relations may shed light on the role of private events in complex human behavior. Skinner argued that although it would be possible to analyze private events, we need not, because thoughts and feelings were viewed as co-occuring products of the same contingencies that are responsible for changes in overt responses. However, the bidirectional transformation of stimulus function inherent in verbal behavior changes the way that private events participate in complex behavioral episodes for verbal organisms. We examine why we have reached such a conclusion, with special emphasis on the role of self-awareness. Finally, we conclude with an application of our analysis to the problem of substance abuse.

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.

Verbal self-reports of delayed matching to sample by humans

Undergraduates participated in two experiments to develop methods for the experimental analysis of self-reports about behavior. The target behavior was the choice response in a delayed-matching-to-sample task in which monetary reinforcement was contingent upon both speed and accuracy of the choice. In Experiment 1, the temporal portion of the contingency was manipulated within each session, and the presence and absence of feedback about reinforcement was manipulated across sessions. As the time limits became stricter, target response speeds increased, but accuracy and reinforcement rates decreased. When feedback was withheld, further reductions in speed and reinforcement occurred, but only at the strictest time limit. Thus, the procedures were successful in producing systematic variation in the speed, accuracy, and reinforcement of the target behavior. Experiment 2 was designed to assess the influence of these characteristics on self-reports. In self-report conditions, each target response was followed by a computer-generated query: "Did you earn points?" The subject reported by pressing "Yes" or "No" buttons, with the sole consequence of advancing the session. In some cases, feedback about reinforcement of the target response followed the reports; in other cases it was withheld. Self-reports were less accurate when the target responses occurred under greater time pressure. When feedback was withheld, the speed of the target response influenced reports, in that the probability of a "Yes" report increased directly with the speed of accurate target responses. In addition, imposing the self-report procedure disrupted target performance by reducing response speeds at the strictest time limit. These results allow investigation of issues in both behavioral and cognitive psychology. More important, the overall order in the data suggests promise for the experimental analysis of self-reports by human subjects.

Molecular maximizing characterizes choice on Vaughan's (1981) procedure

Pigeons keypecked on a two-key procedure in which their choice ratios during one time period determined the reinforcement rates assigned to each key during the next period (Vaughan, 1981). During each of four phases, which differed in the reinforcement rates they provided for different choice ratios, the duration of these periods was four minutes, duplicating one condition from Vaughan's study. During the other four phases, these periods lasted six seconds. When these periods were long, the results were similar to Vaughan's and appeared compatible with melioration theory. But when these periods were short, the data were consistent with molecular maximizing (see Silberberg & Ziriax, 1982) and were incompatible with melioration, molar maximizing, and matching. In a simulation, stat birds following a molecular-maximizing algorithm responded on the short- and long-period conditions of this experiment. When the time periods lasted four minutes, the results were similar to Vaughan's and to the results of the four-minute conditions of this study; when the time periods lasted six seconds, the choice data were similar to the data from real subjects for the six-second conditions. Thus, a molecular-maximizing response rule generated choice data comparable to those from the short- and long-period conditions of this experiment. These data show that, among extant accounts, choice on the Vaughan procedure is most compatible with molecular maximizing.

Local temporal pattering of operant behavior in humans

Button pressing by 44 college students intermittently produced points and the words "GOOD" or "POOR" on a computer screen. The events were arranged according to a paced random-interval 10-s schedule in which the target interresponse-time categories were 1 to 3, 3 to 5, or 6 to 12 s. The degree to which instructions specified certain aspects of the contingency (e.g., whether response spacing was critical) was also varied, and in some conditions the experimenter prompted specifically paced responses during the first 2 min of the session. The procedures shaped the local patterning of behavior of some subjects in less than 30 min of exposure to the contingencies. Most subjects who, in a postexperimental questionnaire, accurately identified the schedule contingencies also responded more accurately than those whose verbal descriptions were inaccurate or imprecise.

Single-sample discrimination of different schedules' reinforced interresponse times

Food-deprived rats in Experiment 1 responded to one of two tandem schedules that were, with equal probability, associated with a sample lever. The tandem schedules' initial links were different random-interval schedules. Their values were adjusted to approximate equality in time to completing each tandem schedule's response requirements. The tandem schedules differed in their terminal links: One reinforced short interresponse times; the other reinforced long ones. Tandem-schedule completion presented two comparison levers, one of which was associated with each tandem schedule. Pressing the lever associated with the sample-lever tandem schedule produced a food pellet. Pressing the other produced a blackout. The difference between terminal-link reinforced interresponse times varied across 10-trial blocks within a session. Conditional-discrimination accuracy increased with the size of the temporal difference between terminal-link reinforced interresponse times. In Experiment 2, one tandem schedule was replaced by a random ratio, while the comparison schedule was either a tandem schedule that only reinforced long interresponse times or a random-interval schedule. Again, conditional-discrimination accuracy increased with the temporal difference between the two schedules' reinforced interresponse times. Most rats mastered the discrimination between random ratio and random interval, showing that the interresponse times reinforced by these schedules can serve to discriminate between these schedules.

Empirical validation of a procedure to correct position and stimulus biases in matching-to-sample

The development of position and stimulus biases often occurs during initial training on matching-to-sample tasks. Furthermore, without intervention, these biases can be maintained via intermittent reinforcement provided by matching-to-sample contingencies. The present study evaluated the effectiveness of a correction procedure designed to eliminate both position and stimulus biases. Following key-peck training, a group of 6 pigeons had extended exposure to matching-to-sample contingencies without a correction procedure, a group of 4 pigeons was briefly exposed to a simultaneous matching-to-sample procedure to assess biases prior to exposure to the correction procedure, and a group of 5 pigeons was exposed directly to the correction procedure. The correction procedure arranged that every time an incorrect match was made, the trial configuration was repeated on the subsequent trial until a correct match was made. Extended exposure to matching-to-sample contingencies without a correction procedure was associated with reduced biases eventually for most subjects, but rapid development of near-perfect accuracy and bias-free performance was observed upon the implementation of the correction procedure regardless of the type of bias. Bias-free performance was maintained following subsequent exposure to a zero-delay MTS procedure.

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.

Correspondence as conditional stimulus control: insights from experiments with pigeons

Correspondence between saying and doing, typically studied in young children and individuals with developmental disabilities, was examined as an instance of conditional stimulus control. In Experiment 1, 3 pigeons were exposed to a two-component repeated-trials procedure. In the first-sample or say-component, two response keys transilluminated by different colored lights were presented and the pigeon pecked one of the keys. After 1 s of darkness in the chamber, the second-choice or do-component was presented, in which the two keys again were transilluminated, one by the color selected in the first component and the second by another color. Selecting the color that matched that selected in the say component resulted in access to food. Selecting the other color produced a blackout of the chamber. After an intertrial interval (ITI), the next say component was programmed, and the procedure was repeated. Correspondence remained at chance levels through several manipulations of ITI duration and sample response requirement. When a correction procedure was added such that only the originally selected sample stimulus was re-presented until a correct choice response occurred, reliable correspondence developed in 2 pigeons. This correspondence was eliminated by making reinforcement independent of correspondence and subsequently was reestablished when reinforcement again depended on correspondence. In Experiment 2, 3 other pigeons rapidly acquired correspondence under the final procedure used in Experiment 1. Increasing the time interval between the say and do components diminished correspondence. The results of the two experiments suggest how correspondence may be considered an instance of conditional stimulus control and that it is possible to construct a homologue of human say-do correspondence with pigeons.

Interresponse-time sensitivity during discrete-trial and free-operant concurrent variable-interval schedules

Two experiments investigated the sensitivity of pigeons' choice to elapsed time since the last response (i.e., to inter-response time [IRT]) during concurrent variable-interval variable-interval schedules. Experiment 1 used a two-key discrete-trial procedure with variable intertrial intervals. Experiment 2 employed a three-key free-operant procedure. In both experiments choice was found to be a function of the active-schedule IRT, defined as the time since the most recent response. Monte Carlo simulations show how this finding permits the joining of several seemingly incompatible data sets held to both support and contradict a kind of choice strategy, termed momentary maximizing, which attempts to maximize momentary reinforcement probabilities. The studies suggest that only two variables are needed to describe the static molecular structure of concurrent variable-interval choice: active-schedule IRTs and "response states" consisting of the last one or two schedule choices.

Signal-detection properties of verbal self-reports

The bias (B'H) and discriminability (A') of college students' self-reports about choices made in a delayed identity matching-to-sample task were studied as a function of characteristics of the response about which they reported. Each matching-to-sample trial consisted of two, three, or four simultaneously presented sample stimuli, a 1-s retention interval, and two, three, or four comparison stimuli. One sample stimulus was always reproduced among the comparisons, and choice of the matching comparison in less than 800 ms produced points worth chances in a drawing for money. After each choice, subjects pressed either a "yes" or a "no" button to answer a computer-generated query about whether the choice met the point contingency. The number of sample and comparison stimuli was manipulated across experimental conditions. Rates of successful matching-to-sample choices were negatively correlated with the number of matching-to-sample stimuli, regardless of whether samples or comparisons were manipulated. As in previous studies, subjects exhibited a pronounced bias for reporting successful responses. Self-report bias tended to become less pronounced as matching-to-sample success became less frequent, an outcome consistent with signal-frequency effects in psychophysical research. The bias was also resistant to change, suggesting influences other than signal frequency that remain to be identified. Self-report discriminability tended to decrease with the number of sample stimuli and increase with the number of comparison stimuli, an effect not attributable to differential effects of the two manipulations on matching-to-sample performance. Overall, bias and discriminability indices revealed effects that were not evident in self-report accuracy scores. The results indicate that analyses based on signal-detection theory can improve the description of correspondence between self-reports and their referents and thus contribute to the identification of environmental sources of control over verbal self-reports.

Discriminated interresponse times: role of autoshaped responses

When discriminated interresponse-time (IRT) procedures have been used to assess preference relations among temporally extended operants, deviations from matching have been obtained. Using a yoked-control procedure, the present study found that key pecking in a discriminated IRT procedure has two sources of strength--that arising from the response-reinforcer contingency that is explicitly arranged, and that arising from a stimulus-reinforcer contingency that is a by-product of the explicitly arranged contingency. The key pecking of all lead birds, and that of 3 of the 4 birds exposed to a yoked autoshaping procedure, was controlled by the keylight that signaled the lead birds' criterion IRTs. Because stimulus control of key pecking by the keylight, whether autoshaped or discriminative, fosters deviations from matching, the discriminated IRT procedure does not provide an appropriate basis for conclusions about preference relations among IRTs.

Delayed signal detection, differential reinforcement, and short-term memory in the pigeon

In two discrete-trial delayed-detection experiments, six pigeons were trained on dependent concurrent variable-interval schedules. Pecking a red side key was reinforced when the brighter of two white lights (S1) had been presented on the center key, and pecking a green side key was reinforced when the duller of two white lights (S2) had been presented on the center key. Incorrect responses were red side-key pecks following S2 presentations and green side-key pecks following S1 presentations; these resulted in three-second blackouts. In Experiment 1, the time between presentation of S1 or S2 on the center key and the onset of the red and green side keys was varied nonsystematically from 0.06 seconds to 19.69 seconds across experimental conditions. Stimulus discriminability decreased as the stimulus-choice delay increased. A rectangular-hyperbolic function better described this decrease in discriminability over time than did a negative-exponential function. In Experiment 2, at each of three stimulus-choice delays (0.06, 3.85, and 10.36 seconds), relative reinforcer frequency for correct responses to the red and green side keys was varied by changing the values of the dependent concurrent variable-interval schedules. The sensitivity of choice to relative reinforcer frequency was independent of the decrease in stimulus discriminability with increasing stimulus-choice delay.

Comments on Shimp's (1983) double dissociation between knowledge and tacit knowledge

Shimp (1983) found in five pigeons "double dissociation" between the distribution of pairs of long and short reinforced interresponse times and "self-reports," obtained by symbolic matching to sample, concerning these interresponse times. This result can be explained by assuming that the birds used a fixed temporal interval as matching criterion, independently of which pattern of interresponse times was reinforced.

Knowing before doing: discrimination by rats of a brief interruption of a tone

Eight rats' lever presses were reinforced after an interruption in a tone, provided the lever had not been pressed before the tone interruption. After a few sessions, long before the animals reliably refrained from lever pressing before the interruption, the latencies of postinterruption presses (time from the termination of the interruption to the moment of the lever press) dissociated into two classes: short ones for to-be-rewarded presses, and long ones for presses in the other trials, which contained no reward because one or more lever presses had occurred before the interruption. Thus discrimination of impending reinforcement in the current trial occurred before there was evidence of sensitivity to reinforcement in the reinforcement-producing aspect of behavior. This finding is related to Shimp's (1981) contention that the temporal properties of recent behavior are reinforceable, if remembered. The present finding shows that learning to discriminate whether one's behavior has met a contingency, and learning to carry out this behavior, need not go together, implying that memory of temporal properties is probably a necessary but not a sufficient condition for learning the latter.

Reproduction memory of two-event sequences in pigeons

Six pigeons were trained to reproduce two-event sequences in an experiment that employed a discrete-trial procedure that required subjects to peck one of four possible sample sequences (left-left, left-right, right-right, right-left) signaled on a given trial by the successive illumination of response keys. Following a retention interval (0.1 to 30 seconds), a reinforcer was delivered if a subject reproduced the prior sample sequence during a test condition in which both left and right keys were illuminated. The pigeons readily reproduced the orders in which they had just seen and pecked two illuminated keys. Reproduction accuracy declined as the retention interval was increased. Homogeneous sequences (left-left, right-right) were reproduced with greater accuracy than heterogeneous sequences.

The local organization of behavior: dissociations between a pigeon's behavior and self-reports of that behavior

The purpose of the experiment was to study the relation between what an organism does in a setting that demands temporal patterning of behavior and what it reports it has done. More specifically, a pigeon produced two classes, shorter and longer, of temporal patterns of key pecks (interresponse times) on a center key. Occasionally, a symbolic matching-to-sample probe arranged on side keys asked the pigeon whether its most recent pattern was a shorter or longer one. The longer reinforced pattern was always three times as long as the shorter one and the two patterns were reinforced equally often. Absolute duration of reinforced patterns was varied. In some conditions, interresponse-time distributions on the center key were bimodal, indicating a clear behavioral adaptation to the contingency, yet a bird did not report very well by appropriate side-key responding what its most recent interresponse time had been. In other conditions, the interresponse-time distributions were less clearly bimodal, yet a bird reported more accurately its previous interresponse time as shorter or longer. Thus, there was a dissociation between how well behavior on the center key conformed to the schedule requirement and how well a bird reported what it was doing on the center key. In addition, as absolute duration of the reinforced patterns was increased, a bird categorized its most recent pattern less well even as its preference for the shorter pattern increased dramatically. These results were interpreted as an example of the phenomenon of dissociation between tacit knowledge and knowledge.