Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement

On the value of advanced information about delayed rewards

In a variety of laboratory preparations, several animal species prefer signaled over unsignaled outcomes. Here we examine whether pigeons prefer options that signal the delay to reward over options that do not and how this preference changes with the ratio of the delays. We offered pigeons repeated choices between two alternatives leading to a short or a long delay to reward. For one alternative (informative), the short and long delays were reliably signaled by different stimuli (e.g., SS for short delays, SL for long delays). For the other (non-informative), the delays were not reliably signaled by the stimuli presented (S1 and S2). Across conditions, we varied the durations of the short and long delays, hence their ratio, while keeping the average delay to reward constant. Pigeons preferred the informative over the non-informative option and this preference became stronger as the ratio of the long to the short delay increased. A modified version of the Δ-Σ hypothesis (González et al., J Exp Anal Behav 113(3):591-608. https://doi.org/10.1002/jeab.595 , 2020a) incorporating a contrast-like process between the immediacies to reward signaled by each stimulus accounted well for our findings. Functionally, we argue that a preference for signaled delays hinges on the potential instrumental advantage typically conveyed by information.

Inhibition and paradoxical choice

The present study evaluated the role of inhibition in paradoxical choice in pigeons. In a paradoxical choice procedure, pigeons receive a choice between two alternatives. Choosing the "suboptimal" alternative is followed 20% of the time by one cue (the S+) that is always reinforced, and 80% of the time by another cue (S-) that is never reinforced. Thus, this alternative leads to an overall reinforcement rate of 20%. Choosing the "optimal" alternative, however, is followed by one of two cues (S3 or S4), each reinforced 50% of the time. Thus, this alternative leads to an overall reinforcement rate of 50%. González and Blaisdell (2021) reported that development of paradoxical choice was positively correlated to the development of inhibition to the S- (signal that no food will be delivered on that trial) post-choice stimulus. The current experiment tested the hypothesis that inhibition to a post-choice stimulus is causally related to suboptimal preference. Following acquisition of suboptimal preference, pigeons received two manipulations: in one condition one of the cues in the optimal alternative (S4) was extinguished and, in another condition, the S- cue was partially reinforced. When tested on the choice task afterward, both manipulations resulted in a decrement in suboptimal preference. This result is paradoxical given that both manipulations made the suboptimal alternative the richer option. We discuss the implications of our results, arguing that inhibition of a post-choice cue increases attraction to or value of that choice.

Paradoxical choice and the reinforcing value of information

Signals that reduce uncertainty can be valuable because well-informed decision-makers can better align their preferences to opportunities. However, some birds and mammals display an appetite for informative signals that cannot be used to increase returns. We explore the role that reward-predictive stimuli have in fostering such preferences, aiming at distinguishing between two putative underlying mechanisms. The 'information hypothesis' proposes that reducing uncertainty is reinforcing per se, somewhat consistently with the concept of curiosity: a motivation to know in the absence of tractable extrinsic benefits. In contrast, the 'conditioned reinforcement hypothesis', an associative account, proposes asymmetries in secondarily acquired reinforcement: post-choice stimuli announcing forthcoming rewards (S⁺) reinforce responses more than stimuli signalling no rewards (S^-) inhibit responses. In three treatments, rats faced two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), food or no food was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred Info when (1) both outcomes were explicitly signalled by salient auditory cues, (2) only forthcoming food delivery was explicitly signalled, and (3) only the absence of forthcoming reward was explicitly signalled. Acquisition was slower in (3), when food was not explicitly signalled, showing that signals for positive outcomes have a greater influence on the development of preference than signals for negative ones. Our results are consistent with an elaborated conditioned reinforcement account, and with the conjecture that both uncertainty reduction and conditioned reinforcement jointly act to generate preference.

Forced-exposure trials increase suboptimal choice

The influence of single option or forced-exposure (FE) trials was studied in the suboptimal choice task. Pigeons chose between an optimal alternative that led to food half of the time and a suboptimal alternative that led to food 20% of the time. Choice of the suboptimal alternative was compared across groups of subjects that received different numbers of FE trials during training. In Experiment 1, subjects received 100% FE trials, 67% FE trials, or only choice trials. Pigeons in the two groups that had FE trials developed extreme preference for the signaled suboptimal alternative over the unsignaled optimal alternative, while pigeons that had no FE trials showed pronounced individual differences. Experiment 2 compared 10% and 90% FE trials. When neither alternative signaled trial outcomes, both groups of subjects strongly preferred the optimal alternative. When the suboptimal alternative provided differential signals, the subjects in the 90% FE group developed strong preference for the suboptimal alternative and subjects in the 10% FE group maintained preference for the optimal alternative. The results of both experiments demonstrate that FE trials can have substantial effects on the development of preference in the suboptimal choice task.

The Role of the Lateral Habenula in Inhibitory Learning from Reward Omission

The lateral habenula (LHb) is a phylogenetically primitive brain structure that plays a key role in learning to inhibit distinct responses to specific stimuli. This structure is activated by primary aversive stimuli, cues predicting an imminent aversive event, unexpected reward omissions, and cues associated with the omission of an expected reward. The most widely described physiological effect of LHb activation is acutely suppressing midbrain dopaminergic signaling. However, recent studies have identified multiple means by which the LHb promotes this effect as well as other mechanisms of action. These findings reveal the complex nature of LHb circuitry. The present paper reviews the role of this structure in learning from reward omission. We approach this topic from the perspective of computational models of behavioral change that account for inhibitory learning to frame key findings. Such findings are drawn from recent behavioral neuroscience studies that use novel brain imaging, stimulation, ablation, and reversible inactivation techniques. Further research and conceptual work are needed to clarify the nature of the mechanisms related to updating motivated behavior in which the LHb is involved. As yet, there is little understanding of whether such mechanisms are parallel or complementary to the well-known modulatory function of the more recently evolved prefrontal cortex.

Effect of Environmental Enrichment on the Brain and on Learning and Cognition by Animals

Simple Summary

Most people consider the environment in which animals are kept to be an ethical matter, separate from the research that we conduct with them. Those of us who do research on the cognitive behavior of animals try to consider their welfare, but what we often fail to recognize is that the welfare of the animals we study can affect the results of experiments that we investigate. We have but scratched the surface of the question, how do enriched environments affect the cognitive behavior of animals, in our case pigeons. We have found that pigeons with experience in an enriched environment are less impulsive. The reduction in impulsivity results in a reduced tendency to make the suboptimal choice. It also has been claimed to make animals more optimistic, as assessed by their tendency to make choices of more favorable alternatives, under ambiguous conditions.

Abstract

The humane treatment of animals suggests that they should be housed in an environment that is rich in stimulation and allows for varied activities. However, even if one’s main concern is an accurate assessment of their learning and cognitive abilities, housing them in an enriched environment can have an important effect on the assessment of those abilities. Research has found that the development of the brain of animals is significantly affected by the environment in which they live. Not surprisingly, their ability to learn both simple and complex tasks is affected by even modest time spent in an enriched environment. In particular, animals that are housed in an enriched environment are less impulsive and make more optimal choices than animals housed in isolation. Even the way that they judge the passage of time is affected by their housing conditions. Some researchers have even suggested that exposing animals to an enriched environment can make them more “optimistic” in how they treat ambiguous stimuli. Whether that behavioral effect reflects the subtlety of differences in optimism/pessimism or something simpler, like differences in motivation, incentive, discriminability, or neophobia, it is clear that the conditions of housing can have an important effect on the learning and cognition of animals.

Revisited: Pigeons Have Much Cognitive Behavior in Common With Humans

The hypothesis proposed by Macphail (1987) is that differences in intelligent behavior thought to distinguish different species were likely attributed to differences in the context of the tasks being used. Once one corrects for differences in sensory input, motor output, and incentive, it is likely that all vertebrate animals have comparable intellectual abilities. In the present article I suggest a number of tests of this hypothesis with pigeons. In each case, the evidence suggests that either there is evidence for the cognitive behavior, or the pigeons suffer from biases similar to those of humans. Thus, Macphail’s hypothesis offers a challenge to researchers to find the appropriate conditions to bring out in the animal the cognitive ability being tested.

Outcome expectancy and suboptimal risky choice in nonhuman primates

Animals will favor a risky option when a stimulus signaling reward bridges the choice and the outcome. The present experiments investigated signal-induced risky choices and reward-outcome expectations in rhesus and capuchin monkeys. Risky choice was assessed by preference for a large-probabilistic reward over a modest-certain reward. Outcome expectancy was assessed by providing a truncation-response to shorten the delay period. In Experiment 1 both species generally favored the risky option compared to a safe option when the outcomes were signaled and generally shortened the delays except when a signaled-loss stimulus was presented. The use of the delay-truncation response suggested that the monkeys were sensitive to the information conveyed by the stimulus. Experiments 2 and 3 were designed to investigate whether the delay-truncation response used by capuchin monkeys was strategically used reflecting explicit decision-making versus a conditioned response to reward stimuli. A perceptual judgment task was included and the selective use of the delay-truncation response on unsignaled correct trials may suggest the involvement of metacognitive processes. The capuchin monkeys generally truncated the delays except under conditions where reward would not be expected (risky-loss or incorrect-judgment). When the outcomes were unsignaled during the delay some capuchin monkeys were less likely to truncate the delay following an incorrect task response. Overall, the monkeys: (1) made more risky choices when the outcomes were signaled - consistent with gambling-like behavior. (2) selectively truncated the unsignaled delays when rewards could be anticipated (even when metacognitive-like awareness guided anticipation) - suggesting that delay truncation responses reflect explicit outcome expectancy.

Human Choice Predicted by Obtained Reinforcers, Not by Reinforcement Predictors

Macphail (1985) proposed that “intelligence” should not vary across vertebrate species when contextual variables are accounted for. Focusing on research involving choice behavior, the propensity for choosing an option that produces stimuli that predict the presence or absence of reinforcement but that also results in less food over time can be examined. This choice preference has been found multiple times in pigeons (Stagner and Zentall, 2010; Zentall and Stagner, 2011; Laude et al., 2014) and has been likened to gambling behavior demonstrated by humans (Zentall, 2014, 2016). The present experiments used a similarly structured task to examine adult human preferences for reinforcement predictors and compared findings to choice behavior demonstrated by children (Lalli et al., 2000), monkeys (Smith et al., 2017; Smith and Beran, 2020), dogs (Jackson et al., 2020), rats (Chow et al., 2017; Cunningham and Shahan, 2019; Jackson et al., 2020), and pigeons (Roper and Zentall, 1999; Stagner and Zentall, 2010). In Experiment 1, adult human participants showed no preference for reinforcement predictors. Results from Experiment 2 suggest that not only were reinforcement predictors not preferred, but that perhaps reinforcement predictors had no effect at all on choice behavior. Results from Experiments 1 and 2 were further assessed using a generalized matching equation, the findings from which support that adult human choice behavior in the present research was largely determined by reinforcement history. Overall, the present results obtained from human adult participants are different than those found from pigeons in particular, suggesting that further examination of Macphail (1985) hypothesis is warranted.

The functional equivalence of two variants of the suboptimal choice task: choice proportion and response latency as measures of value

In the suboptimal-choice task, birds systematically choose the leaner but informative option (suboptimal) over the richer but non-informative option (optimal). The task has two variations. In the standard task, the optimal option includes two terminal link stimuli. In the original task, it includes a single terminal link stimulus. Two models, the temporal information account (Cunningham and Shahan, J Exp Psychol Anim Learn Cogn 44:1-22, 2018) and the ∆-∑ hypothesis (González et al., J Exp Anal Behav 113:591-608, 2020), presuppose that these procedures are equivalent, but no formal comparison is available. Here we test whether or not these procedures are functionally equivalent. One group of pigeons was trained with the standard procedure, another group with the original procedure, and a third group was trained with a hybrid of the other two (i.e., the two options were the optimal links of the standard and original procedures). Our findings indicate that the number of terminal link stimuli in the optimal option is inconsequential vis-à-vis choice. Moreover, our findings also indicate that latencies to respond are a sensitive metric of value and choice. As predicted by the Sequential Choice Model, we were able to predict simultaneous choices from the latencies of sequential choices and observed a substantial shortening of latencies during simultaneous choices.

Suboptimal choice by pigeons is eliminated when key-pecking behavior is replaced by treadle-pressing

In the study of suboptimal choice, a reliable result is that pigeons strongly prefer an alternative that signals whether a reinforcer will be delivered or not over another alternative without that information even if the first provides a lower probability of reinforcement. In the aforementioned research, key pecking has been the operant response and illuminated keys the discriminative stimuli. In the present study we modified both of these aspects of the procedure in order to analyze the generality of suboptimal preferences of pigeons and to investigate the effect of changes in the incentive salience of the discriminative stimuli. To accomplish this, we presented pigeons a choice situation with the same parameters of reinforcement than previous research, but with treadle pressing as the choice response and ambient lights as discriminative stimuli. Under these conditions, most of the pigeons showed optimal behavior and a high degree of discrimination of the stimuli associated with the discriminative alternative. A control condition with key pecking as choice response and keylights as discriminative stimuli showed that the same pigeons turned to be suboptimal, a result that discards the possibility that the optimality found in the main condition was a consequence of a particular characteristic of our sample of subjects or of our procedure. We discuss the influence that the attribution of incentive salience to the discriminative stimuli has on suboptimal choice in both pigeons and rats.

Enhancing "self-control": The paradoxical effect of delay of reinforcement

Delay of reinforcement is generally thought to be inversely correlated with speed of acquisition. However, in the case of simultaneous discrimination learning, in which choice results in immediate reinforcement, delay of reinforcement can improve acquisition. For example, in the ephemeral reward task, animals are given a choice between two alternatives, A and B. Choice of A provides reinforcement, and the trial is over. Choice of B provides reinforcement and access to alternative A (thus, two reinforcements). Many animals appear unable to learn to choose B consistently, but inserting a 20-s delay between choice and outcome has been shown to facilitate optimal choice. Similarly, pigeons given a choice between a signal for one pellet and a signal for two pellets (each occurring without a delay) have difficulty learning to choose the two-pellet alternative, unless the reinforcement is delayed. In a version of object permanence, food is placed in one of two containers, and the pigeon must choose the container with the food. Pigeons have difficulty reliably choosing the correct container unless a brief delay is inserted between baiting and choice. Finally, pigeons have been shown to prefer a suboptimal alternative (a 20% chance of getting a cue for reinforcement) over an optimal alternative (a 100% chance of getting a cue for 50% reinforcement). However, if pigeons are forced to wait 20 s following their choice to receive the cues, no preference for the suboptimal alternative is found. Thus, impulsive choice may be reduced by delaying the consequence of that choice.

Decision-Making Processes Underlying Pedestrian Behaviors at Signalized Crossing: Part 1. The First to Step off the Kerb

Pedestrians are ideal subjects for the study of decision-making, due to the inter-individual variation in risk taking. Many studies have attempted to understand which environmental factors influence the number of times pedestrians broke the rules at road-crossings, very few focused on the decision-making process of pedestrians according to the different conditions of these variables, that is to say their perception and interpretation of the information they receive. We used survival analyses and modeling to highlight the decision-making process of pedestrians crossing the road at signalized crossings in France and in Japan. For the first pedestrians to step off the kerb, we showed that the probability to cross the road follows three different processes: one at the red signal, one just before the pedestrian signal turns green, and one after the signal has turned green. Globally, the decision of the first pedestrian to cross, whether he or she does so at the green or at the red signal, is influenced by their country of residence. We identify the use of cognitive processes such as risk sensitivity and temporal discounting, and propose new concepts based on the results of this study to decrease the incidence of rule-breaking by pedestrians.

Rats' optimal choice behavior in a gambling-like task

Among the different procedures that model gambling behavior in non-human animals, the "suboptimal choice procedure" has been extensively employed for analyzing the impact of environmental cues on choice behavior. It has been repeatedly demonstrated that pigeons prefer an alternative that infrequently presents a stimulus that signals a larger amount of reinforcement, than another alternative that always presents a stimulus associated with a smaller amount of reinforcement, even though the net rate of reinforcement is lower in the former. In the present study, we tested rats in the magnitude version of the suboptimal choice procedure. Eight rats were given a choice between two alternatives: a) one in which a stimulus predicting the delivery of ten pellets was presented with probability (p) = 0.2 and a stimulus predicting zero pellets was presented with p = 0.8, and b) one in which either of two stimuli predicted the delivery of three pellets with p = 1.0. Contrary to the consistent and robust suboptimal behavior of pigeons, rats preferred the optimal alternative. This effect occurred despite the high index of discrimination of the stimuli associated with the different outcomes shown by the rats. The relevance of this result to the development of animal models of gambling behavior is discussed.

Suboptimal choice in pigeons: Does the predictive value of the conditioned reinforcer alone determine choice?

Prior research has found that pigeons are indifferent between an option that always provides a signal for reinforcement and an alternative that provides a signal for reinforcement only 50% of the time (and a signal for the absence of reinforcement 50% of the time). This suboptimal choice suggests that the frequency of the signal for reinforcement plays virtually no role and choice depends only on the predictive value of the signal for reinforcement associated with each alternative. In the present research we tested the hypothesis that if there are two or three signals for reinforcement associated with the suboptimal alternative but each occurs only 25% or 17% of the time, respectively, pigeons would show a greater preference for the suboptimal alternative. Although we found that increasing the number of signals for reinforcement associated with the suboptimal alternative did not increase the preference for the suboptimal alternative (relative to a single signal for reinforcement) extended training on this task resulted in a significant preference for the suboptimal alternative by both groups. This result suggests that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is also responsible for choice of the suboptimal alternative.

The role of 'jackpot' stimuli in maladaptive decision-making: dissociable effects of D1/D2 receptor agonists and antagonists

RATIONALE

Laboratory experiments often model risk through a choice between a large, uncertain (LU) reward against a small, certain (SC) reward as an index of an individual's risk tolerance. An important factor generally lacking from these procedures are reward-associated cues that may modulate risk preferences.

OBJECTIVE

We tested whether the addition of cues signaling 'jackpot' wins to LU choices would modulate risk preferences and if these cue effects were mediated by dopaminergic signaling.

METHODS

Three groups of rats chose between LU and SC rewards for which the LU probability of reward decreased across blocks. The unsignaled group received a non-informative stimulus of trial outcome. The signaled group received a jackpot signal prior to reward delivery and blackout on losses. The signaled-light group received a similar jackpot for wins, but a salient loss signal distinct from the win signal.

RESULTS

Presenting win signals decreased the discounting of LU value for both signaled groups regardless of loss signal, while the unsignaled group showed discounting similar to previous research without cues. Pharmacological challenges with D1/D2 agonists and antagonists revealed that D1 antagonism increased and decreased sensitives to the relative probability of reward for unsignaled and signaled groups, respectively, while D2 agonists decreased sensitivities to the relative magnitude of reward.

CONCLUSION

The results highlight how signals predictive of wins can promote maladaptive risk taking in individuals, while loss signals have reduced effect. Additionally, the presence of reward-predictive cues may change the underlying neurobehavioral mechanisms mediating decision-making under risk.

Rats' preferences in the suboptimal choice procedure: Evaluating the impact of reinforcement probability and conditioned inhibitors

Previous research has shown that pigeons and other birds display a strong and consistent preference for an alternative of reinforcement that presents stimuli that allow to discriminate whether a reinforcer will be delivered or not, even when its probability of reinforcement is lower than that of another alternative without those stimuli. In contrast, most of the studies performed with rats report that they show the opposite preference, choosing the alternative with higher probability of reinforcement. To explain these opposite preferences, it has been proposed that rats and pigeons have a differential sensitivity to the conditioned inhibition that emerges from the stimulus that predicts non-reinforcement: While it does not have an impact in pigeons, it strongly influences rats´ preferences. Alternatively, it was recently proposed that there is not a fundamental difference in the behavior of rats and pigeons, but that the procedure employed to evaluate each of these species has generated the difference; in particular, it was proposed that both species prefer the discriminative alternative when the discriminative stimuli have incentive salience. Two recent studies provide support for each of these hypotheses, so that the available evidence does not allow to distinguish between them. In the present report, we present three studies that systematically explore the influence of the procedural differences between the studies with discrepant results. The obtained results provide support for the following ideas: a) there is a fundamental difference between pigeons and rats in their choice behavior in the "suboptimal choice procedure", b) considering the incentive salience of the discriminative stimuli does not resolve it, and c) rats' optimality is a consistent phenomenon, which resists manipulations in reinforcement probabilities and the absence of conditioned inhibitors in the discriminative alternative.

Individual differences in incentive salience attribution are not related to suboptimal choice in rats

Previous research has shown great variation in the extent to which individual rats attribute incentive salience to stimuli that are predictors of reinforcement. When using the Pavlovian Conditioned Approach procedure, in which a discrete stimulus is presented contingently before the delivery of reinforcement, the attribution of incentive salience is demonstrated by sign-tracking behavior (responses directed toward the stimulus predictor of reinforcement), while an absence of this attribution is reflected by goal-tracking behavior (responses directed toward the source of reinforcement). It has been reported that sign-tracking subjects have a higher tendency to perform some maladaptive behaviors than goal-tracking subjects, and that in non-classified rats, increasing the incentive salience of the stimuli promotes suboptimal choice in the "suboptimal choice procedure". In this task, subjects are presented with two alternatives, one of them better in terms of the information provided by the discriminative stimuli, but worse in terms of probability of reinforcement (suboptimal alternative). Integrating these ideas, we hypothesized that sign-trackers would behave suboptimally, in contrast to goal-trackers. In the present study, 45 rats were classified according to their performance in the Pavlovian Conditioned Approach procedure and subjects with extreme values (sign-trackers, and goal-trackers) were evaluated in the suboptimal choice procedure. Both groups were found to behave optimally, with no differences between them. The difference between groups in capacity of attribution of incentive salience was preserved during the entire experiment, suggesting that this variable is not related to choice performance in the suboptimal choice procedure.

Toward a Unifying Account of Impulsivity and the Development of Self-Control

Impulsivity has traditionally been thought to involve various behavioral traits that can be measured using different laboratory protocols. Whereas some authors regard different measures of impulsivity as reflecting fundamentally distinct and unrelated behavioral tendencies (fragmentation approach), others regard those different indexes as analogue forms of the same behavioral tendency, only superficially different (unification approach). Unifying accounts range from mere intuitions to more sophisticated theoretical systems. Some of the more complete attempts at unifying are intriguing but have validity weaknesses. We propose a new unifying attempt based on theoretical points posed by other authors and supplemented by theory and research on associative learning. We then apply these assumptions to characterize the paradigms used to study impulsivity in laboratory settings and evaluate their scope as an attempt at unification. We argue that our approach possesses a good balance of parsimony and empirical and theoretical grounding, as well as a more encompassing scope, and is more suitable for experimental testing than previous theoretical frameworks. In addition, the proposed approach is capable of generating a new definition of impulsivity and outlines a hypothesis of how self-control can be developed. Finally, we examine the fragmentation approach from a different perspective, emphasizing the importance of finding similarities among seemingly different phenomena.

Uncertainty processing in bees exposed to free choices: Lessons from vertebrates

Bees tend to avoid or to show indifference to uncertain ("risky") relative to certain ("safe") food rewards, whether in nectar volume or in nectar concentration. The unattractiveness of uncertain food rewards is also sometimes independent of the energy budget of bees. This pattern of responses seems to differ from that observed in mammals and birds, which may exhibit a strong preference for the uncertainty over the certainty of food delivery on a given trial in dual-choice tasks. Upon analysis of the conditions that determine preference and aversion for uncertain food rewards in "higher" vertebrates, I attempt to demonstrate that bees react to uncertainty in a similar way. It is argued that, because of their social organization and of the type of resources they seek, bees are essentially exposed to situations in which "higher" vertebrates find reward uncertainty unattractive as well. The nature of their representation of food distribution is discussed, and it is suggested that scout bees may differ from recruits with respect to uncertainty processing.

Gambling in rhesus macaques (Macaca mulatta): The effect of cues signaling risky choice outcomes

Preference for a larger-variable "risky" option over a smaller-reliable "safe" option often depends upon the likelihood that the risky option will deliver a sufficiently sized reward to have an equivalent or superior expected value. However, preference for the risky option has been shown to increase under conditions where informative stimuli signaling the outcome of a risky choice is included between the choice and the outcome and this risk-prone preference persists even when the risky option has a lower expected value than the alternative safe option. In the present study, rhesus macaques chose between a risky option and a safe option across two experimental phases to determine whether the outcome signal affected the degree of preference for the risky option. Overall, six out of seven macaques showed a greater preference for the risky option in the signaled condition than in the unsignaled condition. The macaques' risky choices were sensitive to the expected value of the risky option and the signaled condition produced a general increase in risky choices independently of the expected value of the risky outcome. Overall, these results are consistent with those obtained with other animals, and this may relate to a process where animals show a biased preference for "good news." This process may model some of the relevant factors that explain the psychology of gambling in humans.

Gambling-like behavior in pigeons: 'jackpot' signals promote maladaptive risky choice

Individuals often face choices that have uncertain outcomes and have important consequences. As a model of this environment, laboratory experiments often offer a choice between an uncertain, large reward that varies in its probability of delivery against a certain but smaller reward as a measure of an individual’s risk aversion. An important factor generally lacking from these procedures are gambling related cues that may moderate risk preferences. The present experiment offered pigeons choices between unreliable and certain rewards but, for the Signaled group on winning choices, presented a ‘jackpot’ signal prior to reward delivery. The Unsignaled group received an ambiguous stimulus not informative of choice outcomes. For the Signaled group, presenting win signals effectively blocked value discounting for the large, uncertain outcome as the probability of a loss increased, whereas the Unsignaled group showed regular preference changes similar to previous research lacking gambling related cues. These maladaptive choices were further shown to be unaffected by more salient loss signals and resistant to response cost increases. The results suggest an important role of an individual’s sensitivity to outcome-correlated cues in influencing risky choices that may moderate gambling behaviors in humans, particularly in casino and other gambling-specific environments.

How suboptimal is suboptimal choice?

In a frequently used suboptimal-choice procedure pigeons choose between an alternative that delivers three food pellets with p = 1.0 and an alternative that delivers ten pellets with p = 0.2. Because pigeons reliably choose the probabilistic (suboptimal) alternative, the procedure has been proposed as a nonhuman analog of human gambling. The present experiments were conducted to evaluate two potential threats to the validity of this procedure. Experiments 1 and 2 evaluated if pigeons obtained food at a lower unit price (i.e., pecks per pellet) on the suboptimal alternative than on the optimal alternative. When pigeons worked under this suboptimal procedure they all preferred the suboptimal alternative despite some pigeons paying a higher price for food on that alternative. In Experiment 2, when the unit price ratio more closely approximated the inverse of the expected value ratio, pigeons continued to prefer the suboptimal alternative despite its economic suboptimality. Experiment 3 evaluated if, in accord with the string-theory of gambling, the valuation of the suboptimal alternative was increased when pigeons misattributed a subset of the suboptimal no-food trials to the optimal alternative. When trial sequences were arranged to minimize these possible attribution errors, pigeons still preferred the suboptimal alternative. These data remove two threats to the validity of the suboptimal choice procedure; threats that would have suggested that suboptimal choice reflects economic maximization.

I can time with a little help from my friends: effect of social enrichment on timing processes in Pigeons (Columba livia)

There is evidence that impulsive decision-making is associated with errors in timing. However, there has been little attempt to identify the putative mechanism responsible for impulsive animals' timing errors. One means of manipulating impulsivity in non-human animals is providing different levels of access to conspecifics. These preclinical models have revealed that social isolation increases impulsive responding across a wide range of tasks. The goal of the present study was to determine whether social isolation modulates time perception in pigeons by inducing more variability or a bias to underestimate the passage of time in temporal judgments. A temporal bisection task was used to characterize time perception. One group of pigeons performed the bisection following social enrichment, and the remaining half of the pigeons were tested following social isolation. Results revealed pigeons in the social isolation condition categorized a temporal stimulus sample as "long" at shorter durations than pigeons in the social enrichment condition. These data highlight the mechanism(s) thought to underlie timing-based interventions aimed at reducing impulsivity in humans. Future work should consider whether impulsivity is produced by misperceptions of time or a reduced threshold for a response.

Suboptimal Choice in Pigeons: Stimulus Value Predicts Choice over Frequencies

Pigeons have shown suboptimal gambling-like behavior when preferring a stimulus that infrequently signals reliable reinforcement over alternatives that provide greater reinforcement overall. As a mechanism for this behavior, recent research proposed that the stimulus value of alternatives with more reliable signals for reinforcement will be preferred relatively independently of their frequencies. The present study tested this hypothesis using a simplified design of a Discriminative alternative that, 50% of the time, led to either a signal for 100% reinforcement or a blackout period indicative of 0% reinforcement against a Nondiscriminative alternative that always led to a signal that predicted 50% reinforcement. Pigeons showed a strong preference for the Discriminative alternative that remained despite reducing the frequency of the signal for reinforcement in subsequent phases to 25% and then 12.5%. In Experiment 2, using the original design of Experiment 1, the stimulus following choice of the Nondiscriminative alternative was increased to 75% and then to 100%. Results showed that preference for the Discriminative alternative decreased only when the signals for reinforcement for the two alternatives predicted the same probability of reinforcement. The ability of several models to predict this behavior are discussed, but the terminal link stimulus value offers the most parsimonious account of this suboptimal behavior.

Optimal behavior by rats in a choice task is associated to a persistent conditioned inhibition effect

When given a choice between an alternative with a low probability of reinforcement and discriminative stimuli, and another with a higher probability of reinforcement and non-discriminative stimuli, pigeons show a clear preference for the former but rats clearly prefer the later. It has been reported that pigeon's suboptimal choice is associated to a diminishing effect of the stimulus correlated with non-reinforcement. In the present paper, we explored the possibility that rats' optimal choice is more strongly influenced than pigeons' by the stimulus associated to non-reinforcement and that the effects of it do not dissipate during training. We trained rats to choose between an alternative with 0.50 probability of reinforcement and discriminative stimuli, and an alternative with 0.75 probability of reinforcement and non-discriminative stimuli. We replicated the strong preference for the optimal alternative. Then, after several sessions of training, we presented summation trials in which both the stimulus associated to reinforcement and the stimulus associated to non-reinforcement were simultaneously presented. The results showed that the stimulus associated to non-reinforcement exerted a strong effect on choice, and, more importantly, that it did not seem to dissipate across training. These results suggest that the strong difference found between pigeons and rats in the suboptimal choice procedure is potentially related to differences in the impact of conditioned inhibitors.

Rats are optimal in a choice task in which pigeons are not

In an extensive list of studies, it has been found that pigeons prefer an alternative associated with discriminative stimuli over another associated with non-discriminative stimuli, even when the probability of reinforcement is higher in the latter. This behavior has been named "suboptimal choice". In the present experiment, we evaluated whether rats, another widely studied species within the Experimental Analysis of Behavior, also shows this behavior. We systematically replicated the procedure employed with pigeons, and found that rats are not suboptimal, i.e., they prefer the non-discriminative alternative associated with .5 probability of reinforcement, over the discriminative alternative associated with .2 probability of reinforcement. This effect occurred even though rats discriminated the contingencies of reinforcement associated with each stimulus, suggesting that rats' optimal choice was driven by the overall probability of reinforcement of each alternative. Different procedural details are offered as possibilities for explaining this apparent inter-species difference.

Risk should be objectively defined: reply to Zentall and Smith

Zentall and Smith (2014) have published a comment on Pelé and Sueur (Anim Cogn 16:543-556, 2013) in which they raise two issues, one about the definition of risk and a second concerning the optimality of decisions. When making a choice, subjects are faced not only with several possible alternatives but also with the risk of opting for an unsuitable choice which depends on several variables (context, internal state, knowledge and perception). Although it is true that animals might learn about their environment and adapt their decisions to the context and to their experience, strong constraints make some behavioural traits stable over individual lifetime and even generations. We therefore consider that experience has limited impact on the variability of temporal discounting. These behavioural traits make the difference between perceived and actual risk. If the perceived risk strongly differs from the actual risk, a decision should be considered as suboptimal. If we want to lead individual and collective cognition to a common decision science, it is crucial to use the same definitions for terms implied in decision-making.

Incentive salience attribution under reward uncertainty: A Pavlovian model

There is a vast literature on the behavioural effects of partial reinforcement in Pavlovian conditioning. Compared with animals receiving continuous reinforcement, partially rewarded animals typically show (a) a slower development of the conditioned response (CR) early in training and (b) a higher asymptotic level of the CR later in training. This phenomenon is known as the partial reinforcement acquisition effect (PRAE). Learning models of Pavlovian conditioning fail to account for it. In accordance with the incentive salience hypothesis, it is here argued that incentive motivation (or 'wanting') plays a more direct role in controlling behaviour than does learning, and reward uncertainty is shown to have an excitatory effect on incentive motivation. The psychological origin of that effect is discussed and a computational model integrating this new interpretation is developed. Many features of CRs under partial reinforcement emerge from this model.

Suboptimal choice by pigeons: an analog of human gambling behavior

Human gambling often involves the choice of a low probability but high valued outcome over a high probability (certain) low valued outcome (not gambling) that is economically more optimal. We have developed an analog of gambling in which pigeons prefer a suboptimal alternative that infrequently provides a signal for a high probability (or high magnitude) of reinforcement over an optimal alternative that always provides a signal for a lower probability (or lower magnitude) of reinforcement. We have identified two mechanisms that may be responsible for this suboptimal behavior. First, the effect of nonreinforcement results in considerably less inhibition of choice than ideally it should. Second, the frequency of the occurrence of the signal for a high probability or high magnitude of reinforcement is less important than ideally it should. Also analogous to human gambling is the finding that pigeons that are normally food restricted choose suboptimally, whereas those that are minimally food restricted choose optimally. In addition, pigeons that are singly housed choose suboptimally, whereas those that are exposed to a more enriched environment choose less suboptimally. We believe that these findings have implications for the understanding and treatment of problem gambling behavior.