Within-trial contrast or Wagner's SOP model: Can they both account for two presumed complex cognitive phenomena?

When humans make biased or suboptimal choices, they are often attributed to complex cognitive processes that are viewed as being uniquely human. Alternatively, several phenomena, such as suboptimal gambling behavior and cognitive dissonance (justification of effort) may be explained more simply as examples of the contrast between what is expected and what occurs as well as Wagner's Standard Operating Procedure model based on reward prediction error. For example, when pigeons are attracted to choices involving a suboptimal, low probability of a high payoff, as in unskilled gambling behavior, it may be attributed to reward prediction error or the contrast between the low probability of reward expected and the sometimes high probability of reward obtained (when one wins). Similarly, justification of effort, the tendency to attribute greater value to rewards that are difficult to obtain, is typically explained in terms of the tendency to inflate the value of a reward to justify the effort required to obtain it. When pigeons prefer outcomes that require more effort to obtain, however, it is more likely to be explained in terms of contrast between the effort and the reward that follows. We readily attribute the behavior of animals to contrast-like effects or reward prediction error, however, when similar behavior occurs in humans, we also should be prepared to explain it in terms of simpler learning mechanisms. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Midsession Reversal Task with Pigeons Does a Brief Delay Between Choice and Reinforcement Facilitate Reversal Learning?

In a midsession reversal task, the session begins with a simple simultaneous discrimination in which one stimulus (S1) is correct and the other stimulus (S2) is incorrect (S1+/S2-). At the midpoint of the session, the discrimination reverses and S2 becomes the correct choice (S2+/S1-). When choosing optimally, a pigeon should choose S1 until the first trial in which its choice is not reinforced and then it should shift to S2 (win-stay/lose-shift). With this task, pigeons have been shown to respond suboptimally by anticipating the reversal (making anticipatory errors) and continuing to choose S1 after the reversal (making perseverative errors). This suboptimal behavior may result from a pigeon's relative impulsivity due to the immediacy of reinforcement following choice. In other choice tasks, there is evidence that the introduction of a short delay between choice and reinforcement may decrease pigeons' impulsivity. In the present experiment, a delay was introduced between stimulus selection and reinforcement to assess whether it results in a decrease in anticipatory and perseverative errors. Pigeons that had a delay between choice and reinforcement were a bit slower in acquiring the midsession reversal task compared to those without a delay, but showed no decrease in either anticipatory or perseverative errors. It is likely that the pigeons' natural tendency to use time from the start of the session to the reversal as a cue to reverse prevented the delay from increasing accuracy on this task.

Midsession reversal learning: Pigeons learn what stimulus to avoid

The midsession reversal task involves a simple simultaneous discrimination in which, each session, choice of 1 stimulus (S1) is correct for the first 40 trials of each session, and choice of the other stimulus (S2) is correct for the remaining 40 trials. After considerable training with this task, pigeons typically continue to choose S2 too early (making anticipatory errors) and continue choosing S1 for following the reversal (making perseverative errors). Errors can be reduced, however, by decreasing the probability of reinforcement for correct S2 choices or by increasing the response requirement for S2 choices. Increasing the number of S2 stimuli (over trials, 1 S2 stimulus on each trial), however, does not reduce errors. Instead, it results in an increase in anticipatory errors but no change in perseverative errors. In the present experiment, we increased the number of S1 stimuli (over trials, 1 S1 stimulus on each trial) and found an increase in the number of perseverative errors but no change in anticipatory errors. The results suggest that the pigeons acquire this task by learning which stimuli to avoid, rather than which stimuli to choose, although it is also possible that these effects result from attention drawn to the variable stimuli when they are incorrect. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Pigeons can learn a difficult discrimination if reinforcement is delayed following choice

Delaying reinforcement typically has been thought to retard the rate of acquisition of an association, but there is evidence that it may facilitate acquisition of some difficult simultaneous discriminations. After describing several cases in which delaying reinforcement can facilitate acquisition, we suggest that under conditions in which the magnitude of reinforcement is difficult to discriminate, the introduction of a delay between choice and reinforcement can facilitate the discrimination. In the present experiment, we tested the hypothesis that the discrimination between one pellet of food for choice of one alternative and two pellets of food for choice of another may be a difficult discrimination when choice consists of a single peck. If a 10-s delay occurs between choice and reinforcement, however, the discrimination is significantly easier. It is suggested that when discrimination between the outcomes of a choice is difficult and impulsive choice leads to immediate reinforcement, acquisition may be retarded. Under these conditions, the introduction of a brief delay may facilitate acquisition.

Less information results in better midsession reversal accuracy by pigeons

The midsession reversal task involves a simultaneous discrimination between Stimulus 1 (S1) and Stimulus 2 (S2) in which, for the first half of each session, choice of S1 is reinforced and S2 is not, and for the last half of each session, choice of S2 is reinforced and S1 is not. With this task, even after considerable training, pigeons tend to make anticipatory errors as they approach the reversal and they continue to make perseverative errors following the reversal. In the present research, we tested the hypothesis that reversal accuracy would improve by devaluing choice of S2 relative to S1. In Experiment 1, correct choice of S1 was reinforced 100% of the time, whereas correct choice of S2 was reinforced only 20% of the time. This manipulation reduced anticipatory errors but did not increase perseverative errors. In Experiment 2, choice of S1 required a single peck, whereas choice of S2 was devalued by requiring 10 pecks. A similar result was found. In Experiment 3 we devalued S1 by requiring 10 pecks and found decreased accuracy in the form of increased anticipatory errors. Paradoxically, in Experiments 1 and 2, by encouraging the pigeons to avoid using the feedback from choice of S2, and rely solely on feedback from choice of S1, discrimination reversal errors were reduced. The results have implications for attentional theories of learning and theories of behavior change. They also have implications for the conditions responsible for pigeons' tendency to time the occurrence of the change in reinforcement contingencies. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Transitive inference in pigeons may result from differential tendencies to reject the test stimuli acquired during training

In the five-term, transitive inference task used with animals, pigeons are trained on four simultaneous discrimination premise pairs: A + B -, B + C -, C + D -, D + E -. Typically, when tested with the BD pair, most pigeons show a transitive inference effect, choosing B over D. Two non-inferential hypotheses have been proposed to account for this effect but neither has been reliably supported by research. Here we test a third non-inferential hypothesis that the preference for B arises because the animals have not had as much experience with B - in the A + B - discrimination as they have had with the D - in the C + D - discrimination. To test this hypothesis we trained the Experimental Group with the A + B - discrimination in which, over trials, there were four possible A + stimuli that could appear. This was done to encourage the pigeons to learn to reject the B - stimulus. For the Control Group there was only one A + stimulus over trials, as is typically the case. We also varied the nature of the stimuli between groups, such that colors served as the stimuli for half of the pigeons, whereas flags of different counties served as stimuli for the remaining pigeons. In both stimulus conditions, for the Experiment Group, we found little preference for stimulus B over stimulus D, whereas for the Control Group we found the typical preference for stimulus B. Thus, we propose that it is not necessary to attribute the transitive inference effect to an inferential process.

Sooner Rather Than Later: Precrastination Rather Than Procrastination

Putting things off as long as possible (procrastination) is a well-known tendency. Less well known is the tendency to attempt to get things done as soon as possible, even if that involves extra effort ( precrastination). Since its discovery in 2014, precrastination has been demonstrated in humans and animals and has recently been revealed in an analogous tendency called the mere-urgency effect. Trying to get things done as soon as one can may reflect optimal foraging, but another less obvious factor may also contribute—reducing cognitive demands associated with having to remember what to do when. Individual differences may also play a role. Understanding precrastination will have important implications for explaining why hurrying happens as often as it does and may help reduce the chance that haste makes waste.

Object permanence in the pigeon (Columba livia): Insertion of a delay prior to choice facilitates visible- and invisible-displacement accuracy

Object permanence, often viewed as a measure of human cognitive development, has also been used to assess animals' cognitive abilities. Tests of object permanence have distinguished between visible displacement, in which an object may be placed into one of two (or more) containers to be retrieved, and invisible displacement, in which after the object is placed into the container, the container is moved before retrieval is attempted. We tested pigeons' accuracy on both visible and invisible displacement using a rotational beam with a container at either end. In Experiment 1, the pigeons showed some evidence of object permanence on an initial visible displacement test, but they did not maintain accurate choice. With training, their accuracy improved but only to about 70% correct. When tested on a 90° invisible displacement (rotation), accuracy transferred but once again dropped with further training. In Experiment 2, a 5-s delay was inserted between container baiting and choice. Once again, the pigeons showed some evidence of object permanence on an initial visible displacement test, although on the first test session, choice accuracy was not much better than in Experiment 1. With training, choice accuracy improved greatly. Furthermore, pigeons showed good transfer when they were tested on the 90° invisible displacement. Finally, and importantly, they also transferred well to a 180° invisible displacement, a displacement on which dogs failed. The results of these experiments suggest that under the right conditions, pigeons can show a moderate degree of object permanence. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Suboptimal choice in pigeons: Does the predictive value of the conditioned reinforcer alone determine choice?

Prior research has found that pigeons are indifferent between an option that always provides a signal for reinforcement and an alternative that provides a signal for reinforcement only 50% of the time (and a signal for the absence of reinforcement 50% of the time). This suboptimal choice suggests that the frequency of the signal for reinforcement plays virtually no role and choice depends only on the predictive value of the signal for reinforcement associated with each alternative. In the present research we tested the hypothesis that if there are two or three signals for reinforcement associated with the suboptimal alternative but each occurs only 25% or 17% of the time, respectively, pigeons would show a greater preference for the suboptimal alternative. Although we found that increasing the number of signals for reinforcement associated with the suboptimal alternative did not increase the preference for the suboptimal alternative (relative to a single signal for reinforcement) extended training on this task resulted in a significant preference for the suboptimal alternative by both groups. This result suggests that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is also responsible for choice of the suboptimal alternative.

Sameness May Be a Natural Concept That Does Not Require Learning

It has been assumed that when pigeons learn how to match to sample, they learn simple stimulus-response chains but not the concept of sameness. However, transfer to novel stimuli has been influenced by pigeons' tendency to be neophobic. We trained pigeons on matching ( n = 7) and mismatching ( n = 8) with colors as samples and, with each sample, one color as the nonmatching comparison. We then replaced either the matching or the nonmatching stimulus with a familiar stimulus never presented with that sample. Results suggest that for both matching and mismatching, pigeons locate the stimulus that matches the sample: If the task involves matching, they chose it; if it involves mismatching, they avoid it. Thus, the concept of sameness is the basis for correct choice with both tasks. This finding suggests that sameness is a basic concept that does not have to be learned and may have evolved in many species, including humans.

The role of 'jackpot' stimuli in maladaptive decision-making: dissociable effects of D1/D2 receptor agonists and antagonists

RATIONALE

Laboratory experiments often model risk through a choice between a large, uncertain (LU) reward against a small, certain (SC) reward as an index of an individual's risk tolerance. An important factor generally lacking from these procedures are reward-associated cues that may modulate risk preferences.

OBJECTIVE

We tested whether the addition of cues signaling 'jackpot' wins to LU choices would modulate risk preferences and if these cue effects were mediated by dopaminergic signaling.

METHODS

Three groups of rats chose between LU and SC rewards for which the LU probability of reward decreased across blocks. The unsignaled group received a non-informative stimulus of trial outcome. The signaled group received a jackpot signal prior to reward delivery and blackout on losses. The signaled-light group received a similar jackpot for wins, but a salient loss signal distinct from the win signal.

RESULTS

Presenting win signals decreased the discounting of LU value for both signaled groups regardless of loss signal, while the unsignaled group showed discounting similar to previous research without cues. Pharmacological challenges with D1/D2 agonists and antagonists revealed that D1 antagonism increased and decreased sensitives to the relative probability of reward for unsignaled and signaled groups, respectively, while D2 agonists decreased sensitivities to the relative magnitude of reward.

CONCLUSION

The results highlight how signals predictive of wins can promote maladaptive risk taking in individuals, while loss signals have reduced effect. Additionally, the presence of reward-predictive cues may change the underlying neurobehavioral mechanisms mediating decision-making under risk.

The Value of Research in Comparative Cognition

Most research of comparative cognition has focused on the degree to which cognitive phenomena that have been reported in humans, especially children, can also be demonstrated in other animals. The value of such comparative research has not only been the finding that other animals show behavior that is qualitatively similar to that of humans but because the comparative approach calls for the careful control of variables often confounded with the mechanisms being tested, the comparative approach has identified procedures that could also improve the design of research with humans. The comparative approach has also been used to study the degree to which other animals demonstrate human biases and suboptimal behavior (e.g., commercial gambling). When applied to this field of research, the comparative approach has generally taken the position that human biases generally thought to be established by complex social and societal mechanisms (e.g., social reinforcement and entertainment) may be more parsimoniously accounted for by simpler mechanisms (i.e., conditioned reinforcement and positive contrast). When explained in terms of these mechanisms, the results have implications for explaining in simpler and more general terms the results of similar research with humans. Thus, comparative psychology tells us not only about the similarities and possible differences in behavior among species but it also may have implications for our understanding of similar behavior in humans.

Prior commitment: Its effect on suboptimal choice in a gambling-like task

Animals choose suboptimally when provided with cues that signal whether reinforcement is coming or not. For example, pigeons do not prefer an alternative that always provides them with a signal for reinforcement over an alternative that provides them with a signal for reinforcement only half of the time and a signal for the absence of reinforcement the rest of the time. In the present research, we tested the hypothesis that if the results of the choice are delayed, pigeons will choose less suboptimally. We tested this hypothesis by forcing pigeons to wait following their choice, requiring them to complete a fixed-interval 20-s schedule prior to receiving the signals for reinforcement. In Experiment 1, we gave the pigeons a choice between (a) a 50% chance of receiving a signal for reinforcement or a 50% chance of receiving a signal for the absence of reinforcement and (b) a 100% chance of receiving a signal for reinforcement. When the signal for reinforcement was delayed, most of the pigeons chose optimally. When it was not delayed, most of the pigeons chose suboptimally. In Experiment 2, we gave the pigeons a choice between (a) a 25% chance of receiving a signal for reinforcement or a 75% chance of receiving a signal for nonreinforcement and (b) a 100% chance of receiving an unreliable signal for reinforcement (predicting reinforcement 75% of the time). When the signal was not delayed, the pigeons showed a strong tendency to choose suboptimally but they chose suboptimally much less when the signal was delayed.