Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making

Stimuli that are more predictive of subsequent reward also function as better conditioned reinforcers. Moreover, stimuli attributed with incentive salience function as more robust conditioned reinforcers. Some theories have suggested that conditioned reinforcement plays an important role in promoting suboptimal choice behavior, like gambling. The present experiments examined how different stimuli, those attributed with incentive salience versus those without, can function in tandem with stimulus-reward predictive utility to promote maladaptive decision-making in rats. One group of rats had lights associated with goal-tracking as the reward-predictive stimuli and another had levers associated with sign-tracking as the reward-predictive stimuli. All rats were first trained on a choice procedure in which the expected value across both alternatives was equivalent but differed in their stimulus-reward predictive utility. Next, the expected value across both alternatives was systematically changed so that the alternative with greater stimulus-reward predictive utility was suboptimal in regard to primary reinforcement. The results demonstrate that in order to obtain suboptimal choice behavior, incentive salience alongside strong stimulus-reward predictive utility may be necessary; thus, maladaptive decision-making can be driven more by the value attributed to stimuli imbued with incentive salience that reliably predict a reward rather than the reward itself.

Mechanisms of midsession reversal accuracy: Memory for preceding events and timing

The midsession reversal task involves a simultaneous discrimination between 2 stimuli (S1 and S2) in which, for the first half of each session, choice of S1 is reinforced and, for the last half, choice of S2 is reinforced. On this task, pigeons appear to time the occurrence of the reversal rather than using feedback from previous trials, resulting in increased numbers of errors. In the present experiments, we tested the hypothesis that pigeons make so many errors because they fail to remember the last response made and/or the consequence of making that response both of which are needed ideally as cues to respond on the next trial. To facilitate memory, during the 5-s intertrial interval, we differentially lit a houselight correlated with the prior response to S1 or S2 and maintained the hopper light when that response was correct. A control group received uncorrelated houselights and no maintained hopper light. To test for continued use of temporal information, both groups received probe sessions in which the intertrial interval was either halved or doubled. Providing relevant reminder cues of the stimulus chosen and its consequence resulted in improved reversal accuracy and reduced disruption from probe sessions compared with irrelevant cues. Nevertheless, despite the reminder cues, the pigeons in both groups appeared to continue to time the point in the session at which the reversal occurred. (PsycINFO Database Record

Now You See It, Now You Don’t

The cognitive ability of dogs can be assessed using tasks from the human developmental literature. A task that appears to have ecological relevance is the object-permanence task, in which performance hinges on understanding that an object continues to exist once it can no longer be seen. Although dogs are good at visible displacement tasks, in which an object disappears into a container, they can also understand an invisible displacement, in which the container holding the object is moved. Furthermore, we have found that dogs are able to show considerable memory for the invisibly displaced object. We have also found evidence for object permanence in dogs using the violation-of-expectancy procedure, in which subjects look longer at a stimulus that violates expectations (a screen that appears to pass through an object that has been placed behind the screen) than one that does not. Similarly, we have found that dogs look longer at an object that appears to have changed color or size after being placed behind a screen compared to an object that has not changed. Object-permanence tasks provide an ecologically relevant means of evaluating the cognitive development of dogs.

I can time with a little help from my friends: effect of social enrichment on timing processes in Pigeons (Columba livia)

There is evidence that impulsive decision-making is associated with errors in timing. However, there has been little attempt to identify the putative mechanism responsible for impulsive animals' timing errors. One means of manipulating impulsivity in non-human animals is providing different levels of access to conspecifics. These preclinical models have revealed that social isolation increases impulsive responding across a wide range of tasks. The goal of the present study was to determine whether social isolation modulates time perception in pigeons by inducing more variability or a bias to underestimate the passage of time in temporal judgments. A temporal bisection task was used to characterize time perception. One group of pigeons performed the bisection following social enrichment, and the remaining half of the pigeons were tested following social isolation. Results revealed pigeons in the social isolation condition categorized a temporal stimulus sample as "long" at shorter durations than pigeons in the social enrichment condition. These data highlight the mechanism(s) thought to underlie timing-based interventions aimed at reducing impulsivity in humans. Future work should consider whether impulsivity is produced by misperceptions of time or a reduced threshold for a response.

Suboptimal Choice in Pigeons: Stimulus Value Predicts Choice over Frequencies

Pigeons have shown suboptimal gambling-like behavior when preferring a stimulus that infrequently signals reliable reinforcement over alternatives that provide greater reinforcement overall. As a mechanism for this behavior, recent research proposed that the stimulus value of alternatives with more reliable signals for reinforcement will be preferred relatively independently of their frequencies. The present study tested this hypothesis using a simplified design of a Discriminative alternative that, 50% of the time, led to either a signal for 100% reinforcement or a blackout period indicative of 0% reinforcement against a Nondiscriminative alternative that always led to a signal that predicted 50% reinforcement. Pigeons showed a strong preference for the Discriminative alternative that remained despite reducing the frequency of the signal for reinforcement in subsequent phases to 25% and then 12.5%. In Experiment 2, using the original design of Experiment 1, the stimulus following choice of the Nondiscriminative alternative was increased to 75% and then to 100%. Results showed that preference for the Discriminative alternative decreased only when the signals for reinforcement for the two alternatives predicted the same probability of reinforcement. The ability of several models to predict this behavior are discussed, but the terminal link stimulus value offers the most parsimonious account of this suboptimal behavior.

An Animal Model of Human Gambling

Human gambling generally involves taking a risk on a low probability high outcome alternative over the more economically optimal high probability low outcome alternative (not gambling). Surprisingly, although optimal foraging theory suggests that animals should be sensitive to the overall probability of reinforcement, the results of many experiments suggest otherwise. For example, they do not prefer an alternative that 100% of the time provides them with a stimulus that always predicts reinforcement over an alternative that provides them with a stimulus that predicts reinforcement 50% of the time. This line of research leads to the conclusion that preference depends on the predictive value of the stimulus that follows and surprisingly, not on its frequency. A similar mechanism likely accounts for the suboptimal choice that humans have to engage in commercial gambling.

Symbolic Representation by Pigeons

The capacity for symbolic representation is a prerequisite for the development of human language because words, the basic units of language, are symbols that represent things. But symbolic representation may also serve a nonlinguistic role of organizing events into categories having the same meaning, and such a capacity could have considerable survival value for many species. In a number of experiments, my co-workers and I have found that pigeons that are trained to treat two different stimuli similarly also learn that those stimuli are commonly represented and, thus, that they have the same meaning. We have demonstrated evidence for such common representations in a number of ways, but perhaps the most convincing is when pigeons learn a new association involving one of the presumed commonly represented stimuli, and without further training demonstrate that they have learned a similar association involving the other stimulus. Furthermore, we have found that when pigeons are trained to treat two stimuli similarly, one of those stimuli is represented in terms of the other. These results have implications not only for the generality of cognitive processes across species, but also for the generality of symbolic representation beyond language use.

Imitation by Animals

Imitation is of psychological interest in part because it has cognitive implications for how organisms view the behavior of others, relative to their own behavior. It implies the ability to take the perspective of another. For this reason, researchers have tried to distinguish imitation from other kinds of social learning and influence. In the two-action procedure, one of two response topographies is demonstrated, and the correlation between the topography demonstrated and the topography later used by the observer is a measure of imitation. Both pigeons and Japanese quail show response matching, despite the fact that from their perspective, their own behavior appears quite different from that demonstrated. Although imitation has been demonstrated in birds and several species of primates, researchers are still not certain what mechanisms underlie this ability.

True Imitative Learning in Pigeons

Providing evidence for imitative learning in animals has been made difficult by the need to control for a number of possible nonimitation accounts (e.g., mere presence of another animal, attention drawn to a location, attention drawn to an object being manipulated) that often have not been recognized in previous research In the present experiment we used a version of the two-action method in which a treadle could be operated by a pigeon in one of two distinctive ways with its beak by pecking or with its foot by stepping What is unique in this experiment is not only the distinct response topographies, but also that both responses have the same effect on the environment (depression of the treadle followed by food reward) When pigeons that had observed one of the two response topographies were given access to the treadle, a significant correspondence was found between the topography of the observers responses and that of their respective demonstrators' responses

Can Imitation in Pigeons be Explained by Local Enhancement Together with Trial-and-Error Learning?

Zentall Sutton and Sherburne (1996) reported that pigeons observing a conspecific demonstrator either step on or peck at a treadle to obtain food subsequently showed a significant tendency to manipulate the treadle as had their demonstrator Zentall et al suggested this finding showed observer pigeons had learned by imitation to peck at or step on the treadle However, the same result might have been obtained if pigeons had learned to step on the treadle by trial and error, and pigeons exposed to a treadle-pecking demonstrator had come to peck at the treadle as a result of nonimitative social-learning processes such as local enhancement or contagion Here we report the results for two control groups showing that pigeons do not learn to step on or peck at a treadle for food reward unless they observe a relevant demonstrator These results considerably strengthen the original conclusion Future research using the two action method to demonstrate imitative learning should include similar controls

Suboptimal choice in pigeons: Choice is primarily based on the value of the conditioned reinforcer rather than overall reinforcement rate

Pigeons have sometimes shown a preference for a signaled 50% reinforcement alternative (leading half of the time to a stimulus that signaled 100% reinforcement and otherwise to a stimulus that signaled 0% reinforcement) over a 100% reinforcement alternative. We hypothesized that pigeons may actually be indifferent between the 2 alternatives with previous inconsistent preferences resulting in part from an artifact of the use of a spatial discrimination. In the present experiments, we tested the hypothesis that pigeons would be indifferent between alternatives that provide conditioned reinforcers of equal value. In Experiment 1, we used the signaled 50% reinforcement versus 100% reinforcement procedure, but cued the alternatives with shapes that varied in their spatial location from trial to trial. Consistent with the stimulus value hypothesis, the pigeons showed indifference between the alternatives. In Experiment 2, to confirm that the pigeons could discriminate between the shapes, we removed the discriminative function from the 50% reinforcement alternative and found a clear preference for the 100% reinforcement alternative. Finally, in Experiment 3, when we returned the discriminative function to the 50% reinforcement alternative and reduced the 100% reinforcement alternative to 50% reinforcement, we found a clear preference for the discriminative stimulus alternative. These results support the hypothesis that pigeons prefer the alternative with the conditioned reinforcer that best predicts reinforcement, whereas its frequency may be relatively unimportant.

The relative value of two options for pigeons depends on their context

It is well known that when humans are given a choice between two options, their preference is affected by the presence of a third. Generally, there is an increase in preference for the option closer to the third. We show that a shift in preference in the direction away from the third option can occur in animals. We gave pigeons a choice between A, reinforcement following 10 pecks and a 0.5-s delay, and B, reinforcement following 5 pecks and a 1.5-s delay. Once stable preferences were established, we introduced a third stimulus, less preferred than the other two: C, reinforcement following 20 pecks and a 0.5-s delay or D, reinforcement following 5 pecks and a 4.5-s delay. We found that pigeons presented with C showed an increased preference for B, whereas pigeons presented with D showed an increased preference for A. Our results were consistent with the similarity or attentional hypothesis, which suggests that the third option should interfere with the option more similar to it or would draw attention to the less preferred component of the third option and generalize to the option more similar to it. Possible accounts for the differences in outcomes are suggested.

Resolving the paradox of suboptimal choice

When humans engage in commercial (totally probabilistic) gambling they are making suboptimal choices because the return is generally less than the investment. This review (a) examines the literature on pigeon suboptimal choice, (b) describes the conditions under which it occurs, (c) identifies the mechanisms that appear to be responsible for the effect, and (d) suggests that similar processes may be able to account for analogous suboptimal choice when humans engage in commercial gambling. Pigeons show suboptimal choice when they choose between 1 alternative that 20% of the time provides them with a signal that they will always get fed or 80% of the time with a signal that they will not get fed (overall 20% reinforcement) and a second alternative that 100% of the time provides them with a signal that they will get fed 50% of the time (overall 50% reinforcement). The pigeons' strong preference for the suboptimal choice was investigated in a series of experiments that found the preference for the suboptimal alternative was determined by the value of the signal that predicted reinforcement, rather its frequency and that the frequency of the signal that predicted nonreinforcement had little effect on the suboptimal choice. Paradoxically, this account makes the prediction that pigeons will be indifferent between an alternative that 50% of the time provides a fully predictive stimulus for reinforcement and an alternative that 100% of the time provides a fully predictive stimulus for reinforcement. The similarities and differences of this suboptimal choice task to human gambling are discussed.

Who are the real bird brains? Qualitative differences in behavioral flexibility between dogs (Canis familiaris) and pigeons (Columba livia)

Pigeons given a simultaneous spatial discrimination reversal, in which a single reversal occurs at the midpoint of each session, consistently show anticipation prior to the reversal as well as perseveration after the reversal, suggesting that they use a less effective cue (time or trial number into the session) than what would be optimal to maximize reinforcement (local feedback from the most recent trials). In contrast, rats (Rattus norvegicus) and humans show near-optimal reversal learning on this task. To determine whether this is a general characteristic of mammals, in the present research, pigeons (Columba livia) and dogs (Canis familiaris) were tested with a simultaneous spatial discrimination mid-session reversal. Overall, dogs performed the task more poorly than pigeons. Interestingly, both pigeons and dogs employed what resembled a timing strategy. However, dogs showed greater perseverative errors, suggesting that they may have relatively poorer working memory and inhibitory control with this task. The greater efficiency shown by pigeons with this task suggests they are better able to time and use the feedback from their preceding choice as the basis of their future choice, highlighting what may be a qualitative difference between the species.

Do pigeons prefer alternatives that include near-hit outcomes?

Pigeons show suboptimal choice on a gambling-like task similar to that shown by humans. Humans also show a preference for gambles in which there are near hits (losses that come close to winning). In the present research, we asked if pigeons would show a preference for alternatives with near-hit-like trials. In Experiment 1, we included an alternative that presented a near hit, in which a stimulus associated with reinforcement (a presumed conditioned reinforcer) changed to a stimulus associated with the absence of reinforcement (a presumed conditioned inhibitor). The pigeons tended to avoid this alternative. In Experiment 2, we varied the duration of the presumed conditioned reinforcer (2 vs. 8 s) that changed to a presumed conditioned inhibitor (8 vs. 2 s) and found that the longer the conditioned reinforcer was presented, the more the pigeons avoided it. In Experiment 3, the near-hit alternative involved an ambiguous stimulus for 8 s that changed to a presumed conditioned reinforcer (or a presumed conditioned inhibitor) for 2 s, but the pigeons still avoided it. In Experiment 4, we controlled for the duration of the conditioned reinforcer by presenting it first for 2 s followed by the ambiguous stimulus for 8 s. Once again, the pigeons avoided the alternative with the near-hit trials. In all 4 experiments, the pigeons tended to avoid alternatives that provided near-hit-like trials. We concluded that humans may be attracted to near-hit trials because near-hit trials give them the illusion of control, whereas this does not appear to be a factor for pigeons.