Effects of a conditioned reinforcer upon accuracy of match-to-sample behavior in pigeons

Cleaner wrasse Labroides dimidiatus perform above chance in a "matching-to-sample" experiment

Concept learning have been studied widely in non-human animal species within or not an ecological context. Here we tested whether cleaner fish Labroides dimidiatus, which show generalised rule learning in an ecologically relevant context; they generalise that any predator may provide protection from being chased by other fish; can also learn a general concept when presented with abstract cues. We tested for this ability in the matching-to-sample task. In this task, a sample is shown first, and then the subject needs to choose the matching sample over a simultaneously presented different one in order to obtain a food reward. We used the most general form of the task, using each stimulus only once in a total of 200 trials. As a group, the six subjects performed above chance, and four individuals eventually reached learning criteria. However, individual performance was rather unstable, yielding overall only 57% correct choices. These results add to the growing literature that ectotherms show the ability of abstract concept learning, though the lack of stable high performance may indicate quantitative performance differences to endotherms.

Accuracy of performance on a matching-to-sample procedure under interval schedules

Correct matches on a matching-to-sample procedure were reinforced under fixed-interval, chained fixed-interval, and fixed-interval schedules with exteroceptive stimulus changes correlated with time since the last reinforcer (an added clock). For all four pigeons, accuracy changed within the fixed-interval and fixed-interval schedules with added clock, decreasing from the beginning of the interval to some point in the middle. The performance then became increasingly more accurate until the end of the interval. Under the chained schedules, accuracy also changed within the components. During the initial component, accuracy decreased from the beginning of the fixed interval to some point in the middle or at the end. During the middle component, the performance usually remained at an intermediate level of accuracy. During the terminal component, the initially inaccurate performance became increasingly more accurate throughout the interval. Systematic relationships between response rate and per cent error showed that all four pigeons performed most accurately at high rates. The accuracy of the performance at low rates was also quite high. These relationships held for all three types of schedules through an eight-fold variation in scheduled interreinforcement time.

Complex learning and information processing by pigeons: a critical analysis

THREE MODELS OF CONDITIONAL DISCRIMINATION LEARNING BY PIGEONS ARE DESCRIBED: stimulus configuration learning, the multiple-rule model, and concept learning. A review of the literature reveals that true concept learning is not characteristic of the behavior of pigeons in matching-to-sample, oddity-from-sample, or symbolic matching studies. Instead, pigeons learn a set of sample-specific S(D) rules. Transfer of the discrimination to novel stimuli, at least along the hue dimension, is predicted by a "coding hypothesis", which holds that pigeons make a unique, but usually unobserved response, R(1), to each sample, and that the comparison stimulus chosen depends on which R(1) was emitted in the presence of the sample. Convincing evidence is found that pigeons do code sample hues, but there is little evidence that allows one to infer that the "coding event" must have behavioral properties. Parameters of the conditional discrimination paradigm are identified, and it is shown that by appropriate parametric manipulation, a variety of analogous tasks may be generated for both human and animal subjects. The tasks make possible the comparative study of complex learning, attention, memory, and information processing, with the added advantage that behavior processes may be compared systematically across tasks.

Second-order schedules: comparison of different procedures for scheduling paired and nonpaired brief stimuli

Pigeons performed on a second-order schedule in which fixed-interval components were maintained under a variable-interval schedule. Completion of each fixed-interval component resulted in a brief-stimulus presentation and/or food. The relation of the brief stimulus and food was varied across conditions. Under some conditions, the brief stimulus was never paired with food. Under other conditions, the brief stimulus was paired with food; three different pairing procedures were used: (a) a response produced the simultaneous onset of the stimulus and food; (b) a response produced the stimulus before food with the stimulus remaining on during food presentation; (c) a response produced the stimulus and the offset of the stimulus was simultaneous with the onset of the food cycle. The various pairing and nonpairing operations all produced similar effects on performance. Under all conditions, response rates were positively accelerated within fixed-interval components. Total response rates and Index of Curvature measures were similar across conditions. In one condition, a blackout was paired with food; with this different stimulus in effect, less curvature resulted. The results suggest that pairing of a stimulus is not a necessary condition for within-component patterning under some second-order schedules.

Fixed-ratio and variable-ratio schedules of brief stimuli in second-order schedules of matching to sample

Eight pigeons matched to sample under second-order schedules of food reinforcement. Under fixed-interval unit schedules, the first correct match to occur after a given period of time was followed by the presentation of a brief stimulus. The termination of the last fixed-interval unit schedule was followed by food according to second-order fixed-ratio and variable-ratio schedules. In Experiment 1, as the number of fixed-interval unit schedules increased, long pauses occurred under the second-order fixed-ratio schedules, but not under the variable-ratio schedules. The similarity of performance measures such as local rate and accuracy indicated that the differences engendered by these two types of schedule are in the duration of the periods of not-responding. In Experiment 2, the addition of a brief stimulus at the end of each unit schedule in chained schedules that had different discriminative stimuli present for the duration of each unit did not substantially affect the performance, and long pauses continued to occur. However, few long pauses occurred under schedules with brief stimulus presentations alone. The most inaccurate performances were engendered by chained schedules without brief stimuli.

Discriminative properties of briefly presented stimuli

In Experiment I, pigeons' responses produced food according to a fixed-interval schedule while responses on the key also produced brief stimuli according to a variable-interval schedule. Each brief stimulus reset the fixed interval. Thus, a brief stimulus occurred irregularly but a fixed minimum time separated the occurrence of food from a brief stimulus. Pauses followed brief stimuli and were followed by an accelerated response rate until another brief stimulus or food occurred. In Experiment II, four control procedures were examined. (1) Brief-stimulus presentations were omitted, producing a loss of response patterning. (2) A second-order schedule was studied with fixed-interval components. This schedule produced patterning following brief stimuli similar in kind and degree to that found in Experiment I. (3) A conjoint schedule was arranged in which food was no longer separated from the stimulus by a fixed time; pauses following the stimulus no longer resulted. (4) A brief food reinforcer replaced the brief visual stimulus, resulting in a constant response rate with no pausing following the brief food stimulus. The results suggest that the brief-stimulus effects were due to discriminative functions produced by the fixed temporal relation separating the stimulus from food.

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.

Fixed-interval matching-to-sample: intermatching time and intermatching error runs

Four pigeons were trained on a matching-to-sample task in which reinforcers followed either the first matching response (fixed interval) or the fifth matching response (tandem fixed-interval fixed-ratio) that occurred 80 seconds or longer after the last reinforcement. Relative frequency distributions of the matching-to-sample responses that concluded intermatching times and runs of mismatches (intermatching error runs) were computed for the final matching responses directly followed by grain access and also for the three matching responses immediately preceding the final match. Comparison of these two distributions showed that the fixed-interval schedule arranged for the preferential reinforcement of matches concluding relatively extended intermatching times and runs of mismatches. Differences in matching accuracy and rate during the fixed interval, compared to the tandem fixed-interval fixed-ratio, suggested that reinforcers following matches concluding various intermatching times and runs of mismatches influenced the rate and accuracy of the last few matches before grain access, but did not control rate and accuracy throughout the entire fixed-interval period.

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.

Substitutability between conditioned and primary reinforcers in discrimination acquisition

Rats and pigeons were trained on a series of reversals of a conditional simultaneous discrimination. The percentage of reinforcement for correct trials was varied across reversals. When nonreinforced correct trials produced the same feedback as incorrect trials, the number of errors to reach an acquisition criterion was greater for smaller percentages of reinforcement, but the number of reinforcers required was either approximately constant or smaller for the smaller percentages. When a stimulus paired with food (the conditioned reinforcer) was added on nonreinforced correct trials, both measures were substantially decreased. When the same stimulus was presented, but without a history of food pairing, learning rate was similar to when no stimulus was presented on nonreinforced trials. The results provide direct evidence that conditioned reinforcers may substitute, although imperfectly, for a primary reinforcer, and that pairing with the primary reinforcer is a necessary condition for such substitutability to occur.

The effects of chlorpromazine and imipramine on rate and stimulus control of matching to sample

Pigeons were trained to perform simultaneous, two-color matching to sample under a multiple fixed-ratio fixed-interval schedule of food presentation. The sequence terminating with a peck on the matching key (a "match") was treated as a unit, analogous to a single key peck in conventional schedules. Except for intermittent reinforcement of matches, no consequent stimulus distinguished matches from mismatches (sequences terminating with pecks on the nonmatching key). The pattern of matches during nondrug sessions resembled that of simpler operants maintained by similar schedules. Matches increased in rate toward the end of both components; mismatch rates increased more slowly. Imipramine increased the rate of mismatches, disrupted schedule patterning, and lowered accuracy in a dose-dependent fashion. Chlorpromazine lowered the overall rate of matches but affected schedule patterns and accuracy less than imipramine. The types of errors during drug sessions were not systematically related to the types of errors that appeared during nondrug sessions. Stimulus control was evaluated for each of the four possible color configurations and was found to be by the entire configuration of colors, not simply by the color of the sample.

Preferences among stimulus matches in the pigeon

Three pigeons were trained to peck two to five illuminated response keys. A peck to any of the keys changed the stimulus on the key. When all keys showed the same stimulus (i.e., a stimulus match), an additional key was illuminated with white light. A peck on this key produced three-second access to grain, a three-second intertrial interval, and the next trial. For most sessions, no particular stimulus match was required. Although there were often several stimuli available, each bird preferred a particular stimulus match. With up to 12 stimuli available, birds matched a particular stimulus 60% to 100% of the time.

Matching-to-sample accuracy on fixed-ratio schedules

Pigeons performing on a matching-to-sample procedure were exposed to six fixed-ratio (FR) schedules (FR 1, 5, 10, 20, 40, and 60) of food reinforcement for correct matching responses. During both a correction and a noncorrection procedure without an intertrial interval (ITI), matching accuracy was lower on FR 1 and FR 60 than at intermediate ratios. With the FR 1 schedule, both a 5-sec and a 25-sec ITI resulted in higher matching accuracy than without an ITI; accuracy, with an ITI, was fairly constant for ratios of 1 to 20 but declined at higher ratios. The results suggest that the presence or absence of an ITI in matching to sample may account for inconsistencies obtained in earlier studies of the relationship of matching accuracy to ratio size.

Fixed-ratio discrimination: effects of intermittent reinforcement

Four pigeons had discrimination training that required the choice of a left side-key after completing a fixed-ratio 10 on the center key, and a right side-key choice after fixed-ratio 20. Correct choices were reinforced on various fixed-interval, fixed-ratio, random-interval, and random-ratio schedules. When performance was examined across successive 15-second intervals (fixed-interval and fixed-ratio schedules) accuracy was high in the first 15-second interval, decreased in one or several of the next 15-second intervals, and then increased again as reinforcement was approached. When performance was examined across correct trials on fixed-interval and fixed-ratio schedules, accuracy was lowest immediately after reinforcement, followed by a systematic increase in accuracy as the number of correct choices increased. These patterns were due primarily to errors on fixed-ratio 20 trials. Systematic accuracy patterns did not occur on random-interval or random-ratio schedules. The results indicate that when choice patterns differed on fixed-interval and fixed-ratio schedules, the differences were due to the method of data analysis.

Fixed-ratio and fixed-interval schedule control of matching-to-sample errors by children

Nine children, ages 4 through 7 yr, matched-to-sample on fixed-ratio, fixed-interval, variable-ratio, and variable-interval schedules of reinforcement. Simultaneous, zero-delay, and 2-sec delay matching were employed. Distributions of errors, in which the greatest number of errors occurred at the ordinal position immediately after reinforcement with fewer errors occurring at subsequent positions in the ratio, were produced by six of six children on fixed-ratio schedules for zero-delay and both of two children for 2-sec delay matching. Only two children of seven produced similar error distributions on simultaneous matching for fixed-ratio reinforcement. Variable-ratio schedules produced slightly lower accuracy for most subjects and no systematic error patterns for any subject. Error distributions occurred for all of the five children who experienced fixed-interval schedules for zero-delay matching. Peak error production occurred in the second fourth of the interval. Similar patterns were not produced on variable-interval schedules of equal reinforcement density. Schedule control of complex discriminated operants in children resembles control over similar responses of nonhuman animals.

Control of preference in children by conditioned positive reinforcement

A preference measure was employed with children to evaluate the conditioned positive reinforcing function of a stimulus that preceded reinforcement. A match-to-sample procedure was arranged in which subjects could respond to either the form or color dimension of a compound sample stimulus. Intermittent token reinforcement was provided equally for color and form matches. Two stimuli were employed (Stimulus A and Stimulus B), each consisting of a distinctive tone and colored light. One of these stimuli (the paired stimulus) preceded each token delivery, and the other did not (nonpaired stimulus). The paired stimulus was dependent upon each response to one match dimension, and the nonpaired stimulus followed each response to the other dimension. Three of the five subjects responded primarily to the dimension that was followed by the paired stimulus. This effect was obtained regardless of which stimulus (A or B) was paired and on which match dimension (color or form) the paired stimulus was dependent. These results were unaltered by discontinuing the nonpaired stimulus. The other two subjects demonstrated consistent preferences for the form dimension and Stimulus A, respectively.