Perspectives on observational learning in animals

Observational learning is presumed to have occurred when an organism copies an improbable action or action outcome that it has observed and the matching behavior cannot be explained by an alternative mechanism. Psychologists have been particularly interested in the form of observational learning known as imitation and in how to distinguish imitation from other processes. To successfully make this distinction, one must disentangle the degree to which behavioral similarity results from (a) predisposed behavior, (b) increased motivation resulting from the presence of another animal, (c) attention drawn to a place or object, (d) learning about the way the environment works, as distinguished from what we think of as (e) imitation (the copying of the demonstrated behavior). Several of the processes that may be involved in observational learning are reviewed, including social facilitation, stimulus enhancement, several kinds of emulation, and various forms of imitation.

Preference for the Outcome That Follows a Relatively Aversive Event: Contrast or Delay Reduction?

Pigeons prefer a positive discriminative (S+) stimulus that follows a less preferred event (a large number of required responses, a longer delay, or the absence of food) over a different S+ with a similar history of reinforcement that follows a more preferred event (a single required response, no delay, or food). We proposed that this phenomenon results from contrast (referred to as within-trial contrast) between the less preferred initial event and the signal for reinforcement. Delay reduction theory (Fantino, 1969) can account for these results by proposing that the less preferred initial event lengthens the duration of the trial, thereby allowing the S+ stimulus to occur later in the trial and thus become a better predictor of reinforcement. In the present experiments, we further explored this effect. In Experiment 1, we controlled for trial duration by using a fixed ratio response (30 pecks) as one initial event and the absence of pecking for the same duration as the other initial event (0 pecks). The pigeons showed a reliable preference for the positive stimulus that followed the least preferred initial event. In Experiment 2, we controlled for trial duration by using 30 pecks as one initial event and 1 peck followed by a delay that matched the duration of the preceding 30-peck trial. (Group Time Same). For Group Time Different, there was no delay following the 1-peck initial event. For Group Time Same, preference for the initial event negatively predicted the pigeons' preference for the S+ stimulus that followed, supporting the contrast account. A somewhat greater preference for the discriminative stimulus that followed the least preferred initial event was found for Group Time Different suggesting that in addition to contrast, delay reduction also may play a small role. However, the greater initial-event preference found for Group Time Different suggests that contrast can account for the group difference as well.

Sub-Optimal Choice by Pigeons: Failure to Support The Allais Paradox

Pigeons show a preference for an alternative that provides them with discriminative stimuli (sometimes a stimulus that predicts reinforcement and at other times a stimulus that predicts the absence of reinforcement) over an alternative that provides them with non discriminative stimuli, even if the non discriminative stimulus alternative is associated with 2.5 times as much reinforcement (Stagner & Zentall, 1910). In Experiment 1 we found that the delay to reinforcement associated with the non discriminative stimuli could be reduced by almost one half before the pigeons were indifferent between the two alternatives. In Experiment 2 we tested the hypothesis that the preference for the discriminative stimulus alternative resulted from the fact that, like humans, the pigeons were attracted by the stimulus that consistently predicted reinforcement (the Allais paradox). When the probability of reinforcement associated with the discriminative stimulus that predicted reinforcement was reduced from 100% to 80% the pigeons still showed a strong preference for the discriminative stimulus alternative. Thus, under these conditions, the Allais paradox cannot account for the sub-optimal choice behavior shown by pigeons. Instead we propose that sub-optimal choice results from positive contrast between the low expectation of reinforcement associated with the discriminative stimulus alternative and the much higher obtained reinforcement when the stimulus associated with reinforcement appears. We propose that similar processes can account for sub-optimal gambling behavior by humans.

Subjective Time: Cognitive and Physical Secondary Tasks Affect Timing Differently

Humans were trained on a temporal discrimination to make one response when the stimulus duration was short (2 s) and a different response when the stimulus duration was long (8 s). They were then tested with stimulus durations in between to determine the bisection point. In Experiment 1, we examined the effect of a secondary cognitive task (counting backwards by threes) on the bisection point when participants were trained without a cognitive load and were tested with a cognitive load or the reverse (relative to appropriate controls). When the cognitive load increased from training, the psychophysical function plotting long responses against the increase in stimulus duration shifted to the right (as if the internal clock slowed down), and when the cognitive load decreased from training the psychophysical function shifted to the left (as if the internal clock speeded up). In Experiment 2, when the secondary task consisted of exerting continuous force on a transducer (a physically effortful task), it had the opposite effect. When the required force increased from training, the psychophysical function shifted to the left (as if the internal clock speeded up), and when the required force decreased from training, the psychophysical function shifted to the right (as if the internal clock slowed down). The results support an attentional view of the subjective passage of time. A cognitive secondary task appears to decrease attention to temporal cues, resulting in the underestimation of the passage of time, whereas a force requirement appears to increase attention to temporal cues, resulting in the overestimation of the passage of time.

Sunk cost: pigeons (Columba livia), too, show bias to complete a task rather than shift to another

The sunk cost effect involves the bias to stay with an alternative because one has already invested resources, even when there is a better alternative available. In a series of experiments, at various points during a 30-peck requirement, pigeons (Columba livia) could choose between completing the response requirement (at a different location in Experiment 1 or the same location in Experiments 3 and 4) and switching to a constant number of pecks. In three experiments, the pigeons showed a bias to complete the pecks already started, even when that required more pecking. We also demonstrated that the bias depended on the initial investment and was not produced merely because the pigeons preferred a variable alternative over a fixed alternative. The deviation from optimal choice suggests that pigeons show a bias similar to the sunk cost effect in humans.

Maladaptive "gambling" by pigeons

When humans buy a lottery ticket or gamble at a casino they are engaging in an activity that on average leads to a loss of money. Although animals are purported to engage in optimal foraging behavior, similar sub-optimal behavior can be found in pigeons. They show a preference for an alternative that is associated with a low probability of reinforcement (e.g., one that is followed by a red hue on 20% of the trials and then reinforcement or by a green hue on 80% of the trials and then the absence of reinforcement) over an alternative that is associated with a higher probability of reinforcement (e.g., blue or yellow each of which is followed by reinforcement 50% of the time). This effect appears to result from the strong conditioned reinforcement associated with the stimulus that is always followed by reinforcement. Surprisingly, although it is experienced four times as much, the stimulus that is never followed by reinforcement does not appear to result in significant conditioned inhibition (perhaps due to the absence of observing behavior). Similarly, human gamblers tend to overvalue wins and undervalue losses. Thus, this animal model may provide a useful analog to human gambling behavior, one that is free from the influence of human culture, language, social reinforcement, and other experiential biases that may influence human gambling behavior.

Maladaptive choice behaviour by pigeons: an animal analogue and possible mechanism for gambling (sub-optimal human decision-making behaviour)

Consistent with human gambling behaviour but contrary to optimal foraging theory, pigeons showed maladaptive choice behaviour in experiment 1 by choosing an alternative that provided on average two food pellets over an alternative that provided a certain three food pellets. On 20 per cent of the trials, choice of the two-pellet alternative resulted in a stimulus that always predicted ten food pellets; on the remaining 80 per cent of the trials, the two-pellet alternative resulted in a different stimulus that always predicted zero food pellets. Choice of the three-pellet alternative always resulted in three food pellets. This choice behaviour mimics human monetary gambling in which the infrequent occurrence of a stimulus signalling the winning event (10 pellets) is overemphasized and the more frequent occurrence of a stimulus signalling the losing event (zero pellets) is underemphasized, compared with the certain outcome associated with not gambling (the signal for three pellets). In experiment 2, choice of the two-pellet alternative resulted in ten pellets with a probability of 20 per cent following presentation of either stimulus. Choice of the three-pellet alternative continued to result in three food pellets. In this case, the pigeons reliably chose the alternative that provided a certain three pellets over the alternative that provided an average of two pellets. Thus, in experiment 1, the pigeons were responding to obtain the discriminative stimuli signalling reinforcement and the absence of reinforcement, rather than to obtain the variability in reinforcement.

Justification of Effort by Humans and Pigeons

Justification of effort by humans is a form of reducing cognitive dissonance by enhancing the value of rewards when they are more difficult to obtain. Presumably, assigning greater value to rewards provides justification for the greater effort needed to obtain them. We have found such effects in adult humans and children with a highly controlled laboratory task. More importantly, under various conditions we have found similar effects in pigeons, animals not typically thought to need to justify their behavior to themselves or others. To account for these results, I propose a simpler mechanism than justification of effort—a mechanism based on contrast between the end of the effort and the reinforcement (or signal for reinforcement) that follows. This model predicts that any relatively aversive event can serve to enhance the value of the reward that follows it simply through the contrast between those two events. In support of this general model, my colleagues and I have found this effect in pigeons when the prior event consists of (a) more rather than less effort (pecking), (b) a long rather than a short delay, and (c) the absence of food rather than food. We have also found that a pigeon’s preference for food at one location can shift toward a different location if acquiring food at the new location requires that the pigeon work harder to obtain the food. Contrast may also play a role in other social psychological phenomena that have been interpreted in terms of cognitive dissonance.

Key pecking in pigeons produced by pairing keylight with inaccessible grain

In Experiment I, keylight was paired with inaccessible grain delivery (under two conditions of keylight intensity) to determine if autoshaping would occur in the absence of primary reinforcement. In Experiment II, the procedure was repeated with accessible grain, for comparison. In Experiment III, the procedures were repeated with explicitly unpaired presentations of keylight and either inaccessible or accessible grain. The results indicated that key pecking occurred as quickly in the presence of keylight pairings with inaccessible grain as with accessible grain, though (except for one bird) key pecking was not maintained with inaccessible grain. Furthermore, compared to the dim keylight, the bright keylight greatly suppressed key pecking when paired with inaccessible grain, and reduced the rate of key pecking when paired with accessible grain. Little key pecking occurred in groups exposed to explicitly unpaired presentations of keylight (whether bright or dim) and grain (whether accessible or inaccessible). When the birds in Experiment III were retested with explicitly paired presentations of keylight and grain, little key pecking was observed, suggesting suppressive effects of prior explicitly unpaired presentations. It is suggested that the effects of key-brightness manipulation were produced by the association of grain with cues other than the response key, or by distraction produced by partial illumination of the grain hopper.

Same/different concept learning in the pigeon: the effect of negative instances and prior adaptation to transfer stimuli

Pigeons were trained on a matching-to-sample or oddity-from-sample task with shapes (circle and plus). Half of each group was exposed to "negative instance" trials i.e., for matching birds, neither comparison key matched the sample, and for oddity birds both comparison keys matched the sample. When all birds were transferred to a new task involving colors (red and green), nonshifted birds (transferred from matching to matching, or oddity to oddity) performed significantly better than shifted birds (transferred from matching to oddity, or oddity to matching), but only if they had experienced negative instances of the training concept. When all birds were exposed to negative instances of the transfer task and then transferred to a new color task (yellow and blue), dramatic transfer effects were observed. The effect of pre-exposure to the yellow and blue colors, in order to reduce transfer-stimulus novelty, had a minor effect on transfer.

Self-control without a "self"?: common self-control processes in humans and dogs

Self-control constitutes a fundamental aspect of human nature. Yet there is reason to believe that human and nonhuman self-control processes rely on the same biological mechanism--the availability of glucose in the bloodstream. Two experiments tested this hypothesis by examining the effect of available blood glucose on the ability of dogs to exert self-control. Experiment 1 showed that dogs that were required to exert self-control on an initial task persisted for a shorter time on a subsequent unsolvable task than did dogs that were not previously required to exert self-control. Experiment 2 demonstrated that providing dogs with a boost of glucose eliminated the negative effects of prior exertion of self-control on persistence; this finding parallels a similar effect in humans. These findings provide the first evidence that self-control relies on the same limited energy resource among humans and nonhumans. Our results have broad implications for the study of self-control processes in human and nonhuman species.

A relational differential outcomes effect: pigeons can classify outcomes as "good" and "better"

In a conditional discrimination each of two sample stimuli indicates which of two comparison stimuli is correct. When correct choice following each conditional stimulus is followed by a different outcome (one kind of food following one, a different kind of food following the other) it often facilitates acquisition and improves memory. In transfer designs, in which two different conditional discriminations are followed by the same two differential outcomes, outcome expectation can be shown to be sufficient for comparison choice. That is, the samples from one conditional discrimination are matched to comparisons from the other conditional discrimination based on the common outcomes alone. In the present study we asked if for pigeons the relative value of the differential outcomes (higher versus lower value) can serve as the basis for comparison choice, independent of other characteristics of the outcomes and of differential sample responding. That is, would different outcomes that could be described as "good" and "better" form two stimulus classes. For one conditional discrimination, the differential outcomes involved differential probability of reinforcement for choice of the correct comparison stimulus (0.80 vs. 0.20 for correct choice of the two comparisons, respectively). For the other conditional discrimination, the differential outcomes involved differential responding to the two comparison stimuli (5 pecks vs. 20 pecks to the correct comparisons, respectively). On test trials, when conditional stimuli from the two conditional discriminations were interchanged and the relative value of the differential outcomes could serve as the only basis for comparison choice, we found positive transfer. The results indicate that relational attributes of outcomes can serve as effective cues for comparison choice.

Within-trial contrast: The effect of probability of reinforcement in training

There is evidence that pigeons prefer conditioned reinforcers that are preceded by greater effort over those that are preceded by less effort (an effect that has been attributed to within-trial contrast). In past research the probability of reinforcement for correct choice of the conditioned reinforcer has been 100%, however, the high level of reinforcement for both alternatives in training may result in a performance ceiling when choice between those alternatives is provided on test trials. In the present study we tested this hypothesis by including a group for which the probability of reinforcement in training was only 50%. Pigeons were trained on two simultaneous discriminations, one that was preceded by a 30 peck requirement the other by a single peck requirement. On test trials, we found a significant preference for the S+ that required the greater effort in training for pigeons trained with 100% and a small but nonsignificant effect for pigeons trained with 50% reinforcement. Although the hypothesis that the within-trial contrast effect was constrained by a performance ceiling was not confirmed, we did find a reliable within-trial contrast effect with 100% reinforcement.

Radial Maze Analog for Pigeons: Evidence for Flexible Coding Strategies May Result from Faulty Assumptions

Previous research with the radial maze has found evidence that rats can remember both places that they have already been (retrospective coding) and places they have yet to visit (prospective coding; Cook, Brown, & Riley, 1985). Such dual coding also has been found in pigeons using a radial maze analog in which insertion of a delay at different points during a trial affects performance differentially depending on where in the trial it is inserted. When a delay is interpolated either early or late in a trial, there is minimal disruption of performance compared with when it is interpolated in the middle of the trial. However, the analysis required with this procedure requires the assumption that if errors made on control trials are subtracted from errors made in delay trials, the remaining errors can be directly attributed to the delay. But errors may also be attributed to the changing criterion for making a response as the trial proceeds. Furthermore, the animal's tendency to choose alternatives in a systematic order may also affect its need to remember the sequence of choices made (and yet to be made) on each trial. In the present research, we avoided having to make this assumption by giving the pigeons a two-alternative choice at the time of testing and by randomly determining for the pigeon the order of predelay choices on each trial. This change in procedure resulted in comparable performance as a function of where in the trial the test occurred on both control and delay trials. The effect of the delay was to produce a general decrement in performance independent of where it occurred in the trial.

Matching-to-sample by pigeons: the dissociation of comparison choice frequency from the probability of reinforcement

It has been proposed that comparison choice in matching-to-sample should depend on two factors, the relative probability of reinforcement associated with each of the comparison stimuli and the conditional probability of each comparison stimulus being correct given presentation of one of the samples. DiGian and Zentall [DiGian, K.A., Zentall, T.R., 2007. Matching-to-sample in pigeons: in the absence of sample memory, sample frequency is a better predictor of comparison choice than the probability of reinforcement for comparison choice. Learn Behav. 35, 242-261] have shown that sample frequency together with the probability of choosing each of the comparison stimuli in training can influence comparison choice when delays are introduced, when the number of reinforcements associated with each of the comparison stimuli is equated. Furthermore, Zentall and Clement [Zentall, T.R., Clement, T.S., 2002. Memory mechanisms in pigeons: Evidence of base-rate neglect. J. Exp. Psych.: Anim. Behav. Proc. 28, 111-115] have found that sample frequency can affect comparison choice when delays are introduced independently of the number of choices of each of the comparison stimuli in training and the number of reinforcements associated with each of the comparison stimuli is equated. In the present experiment we found that the probability of choosing each of the comparison stimuli in training can affect comparison choice when delays are introduced, independently of sample frequency and when the number of reinforcements associated with each of the comparison stimuli is equated. Together, these experiments suggest that when the sample is not available, there is a partial dissociation between comparison choice and the probability of reinforcement associated with each of the comparison stimuli.

Pigeons may not use dual coding in the radial maze analog task

Using a radial maze analog task, T. R. Zentall, J. N. Steirn, and P. Jackson-Smith (1990) found evidence that when a delay was interpolated early in a trial, pigeons coded locations retrospectively, but when the delay was interpolated late in the trial, they coded locations prospectively (support for a dual coding hypothesis). In Experiment 1 of the present study, the authors replicated the original finding of dual coding. In Experiments 2 and 3, they used a 2-alternative test procedure that does not require the assumption that pigeons' choice criterion, which changes over the course of the trial, is the same on delay and control trials. Under these conditions, the pigeons no longer showed evidence for dual coding. Instead, there was some evidence that they showed prospective coding, but a more parsimonious account of the results may be that the delay produced a relatively constant decrement in performance at all points of delay interpolation. The original finding of dual coding by Zentall et al. might have been biased by more impulsive choices early in control trials but not in delay trials and by a more stringent choice criterion late in delay trials.

Within-trial contrast: when is a failure to replicate not a type I error?

Vasconcelos, Urcuioli, and Lionello-DeNolf (2007) report the results of five experiments that fail to replicate the results of our within-trial contrast study (Clement, Feltus, Kaiser, & Zentall, 2000) and suggest that our results may represent a Type I Error. We believe that this conclusion is not warranted because (a) there is considerable evidence in support of the effect and (b) the amount of training that they gave to their pigeons prior to test may not have been sufficient to observe the effect reliably. We suggest that when sufficient training is provided, reliable contrast can be found.

Within-trial contrast: pigeons prefer conditioned reinforcers that follow a relatively more rather than a less aversive event

When behavior suggests that the value of a reinforcer depends inversely on the value of the events that precede or follow it, the behavior has been described as a contrast effect. Three major forms of contrast have been studied: incentive contrast, in which a downward (or upward) shift in the magnitude of reinforcement produces a relatively stronger downward (or upward) shift in the vigor of a response; anticipatory contrast, in which a forthcoming improvement in reinforcement results in a relative reduction in consummatory response; and behavioral contrast, in which a decrease in the probability of reinforcement in one component of a multiple schedule results in an increase in responding in an unchanged component of the schedule. Here we discuss a possible fourth kind of contrast that we call within-trial contrast because within a discrete trial, the relative value of an event has an inverse effect on the relative value of the reinforcer that follows. We show that greater effort, longer delay to reinforcement, or the absence of food all result in an increase in the preference for positive discriminative stimuli that follow (relative to less effort, shorter delay, or the presence of food). We further distinguish this within-trial contrast effect from the effects of delay reduction. A general model of this form of contrast is proposed in which the value of a primary or conditioned reinforcer depends on the change in value from the value of the event that precedes it.

Use of a single-code/default strategy by pigeons to acquire duration sample discriminations

Past evidence that pigeons may adopt a single-code/default strategy to solve duration sample discriminations may be attributable to the similarity between the intertrial interval (ITI) and the retention interval. The present experiments tested whether pigeons would adopt a single-code/default strategy when possible ITI-retention-interval ambiguity was eliminated and sample salience was increased. Previous studies of duration sample discriminations that have purported to show evidence for the use of a single-code/default coding strategy have used durations of 0, 2, and 10 sec (Zentall, Klein, and Singer, 2004). However, the results of Experiment 1 suggest that the use of a 0-sec sample may produce an artifact resulting in inadvertent present/absent sample matching. In Experiment 2, when pigeons were trained with three nonzero duration samples (2, 8, and 32 sec), clear evidence for the use of a single-code/default strategy was found.

Preference for a stimulus that follows a relatively aversive event: contrast or delay reduction?

Several types of contrast effects have been identified including incentive contrast, anticipatory contrast, and behavioral contrast. Clement, Feltus, Kaiser, and Zentall (2000) proposed a type of contrast that appears to be different from these others and called it within-trial contrast. In this form of contrast the relative value of a reinforcer depends on the events that occur immediately prior to the reinforcer. Reinforcers that follow relatively aversive events are preferred over those that follow less aversive events. In many cases the delay reduction hypothesis proposed by Fantino (1969) also can account for such effects. The current experiments provide a direct test of the delay reduction and contrast hypotheses by manipulating the schedule of reinforcement while holding trial duration constant. In Experiment 1, preference for fixed-interval (FI) versus differential-reinforcement-of-other-behavior (DRO) schedules of reinforcement was assessed. Some pigeons preferred one schedule over the other while others demonstrated a position (side) preference. Thus, no systematic preference was found. In Experiment 2, a simultaneous color discrimination followed the FI or DRO schedule, and following training, preference was assessed by presenting the two positive stimuli simultaneously. Consistent with the contrast hypothesis, pigeons showed a significant preference for the positive stimulus that in training had followed their less preferred schedule.

Repeated Cocaine Experience Facilitates Sucrose-Reinforced Operant Responding in Enriched and Isolated Rats

The purpose of the present experiment was to determine whether repeated cocaine exposure differentially affects sucrose-reinforced operant responding in rats raised in an enriched condition (EC) or an isolated condition (IC). Specifically, the performance of EC and IC rats pressing a lever for sucrose under a high fixed-ratio schedule (FR 30) prior to and after 10 days of exposure to cocaine (15 mg/kg, i.p.) or saline was compared. Regardless of rearing condition, rats repeatedly exposed to cocaine had shorter reacquisition latencies to complete a sucrose-reinforced FR 30 task than saline controls. The results suggest that cocaine exposure may have cross-sensitized both EC and IC rats to the reinforcing effects of sucrose or sucrose-associated cues, thus facilitating reacquisition of operant responding.