Discounting of delayed food rewards in pigeons and rats: is there a magnitude effect?

Corvids can decide if a future exchange is worth waiting for

Evidence for time-dependent calculations about future rewards is scarce in non-human animals. In non-human primates, only great apes are comparable with humans. Still, some species wait for several minutes to obtain a better reward in delayed exchange tasks. Corvids have been shown to match with non-human primates in some time-related tasks. Here, we investigate a delay of gratification in two corvid species, the carrion crow (Corvus corone) and the common raven (Corvus corax), in an exchange task. Results show that corvids success decreases quickly as delay increases, with a maximal delay of up to 320 s (more than 5 min). The decision to wait rests both on the quality of the prospective reward and the time required to obtain it. Corvids also apply tactics (placing the reward on the ground or caching it) that probably alleviate costs of waiting and distract their attention during waiting. These findings contrast previous results on delayed gratification in birds and indicate that some species may perform comparably to primates.

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 µl of a 0.6 M sucrose solution than when they were 25 and 100 µl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 µl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.

Discounting in Pigeons When the Choice is between Two Delayed Rewards: Implications for Species Comparisons

Studies of delay discounting typically have involved choices between smaller, immediate outcomes and larger, delayed outcomes. In a study of delay discounting in humans, Green et al. (2005) added a period of time prior to both outcomes, creating a delay common to both. They found that the subjective value of the more delayed reward was well described by a hyperboloid discounting function and that the degree to which that outcome was discounted decreased as the common delay increased. In two experiments, we examined the effect of adding a common delay on the discounting of food rewards in pigeons. In Experiment 1, an adjusting-amount procedure was used to establish discounting functions when the common delay was 0, 3, 5, and 10 s, and different stimuli signaled time to the smaller, sooner and larger, later rewards. In contrast to humans, the pigeons showed increases in the degree of discounting when a common delay was added. In Experiment 2, the delay common to both rewards and the delay unique to the larger, later reward were each specifically signaled. With this procedure, the degree of discounting decreased as the common delay increased, a result consistent with that obtained with humans (Green et al., 2005). These findings reveal fundamental similarities between pigeons' and humans' choice behavior, and provide strong interspecies support for the hypothesis that choice between delayed outcomes is based on comparison of their hyperbolically discounted present subjective values.

Spatial discounting of food and social rewards in guppies (poecilia reticulata)

In temporal discounting, animals trade off the time to obtain a reward against the quality of a reward, choosing between a smaller reward available sooner versus a larger reward available later. Similar discounting can apply over space, when animals choose between smaller and closer versus larger and more distant rewards. Most studies of temporal and spatial discounting in non-human animals use food as the reward, and it is not established whether animals trade off other preferred stimuli in similar ways. Here, we offered female guppies (Poecilia reticulata) a spatial discounting task in which we measured preferences for a larger reward as the distance to it increased relative to a closer but smaller reward. We tested whether the fish discounted reward types differently by offering subjects either food items or same-sex conspecifics as rewards. Before beginning the discounting tasks, we conducted validation tests to ensure that subjects equally valued the food and social stimuli in the quantities provided. In the discounting task, subjects switched their preferences from the larger to the smaller reward as the distance to the larger reward increased (spatial discounting), but the pattern and magnitude of discounting did not differ across the two reward types. These findings indicate that guppies show similar patterns of discounting for food and social rewards in a spatial task. In an examination of travel times, however, the fish swam faster to food rewards than to shoaling partners. Analysis of travel times suggests that fish temporally discounted social rewards less steeply than food rewards. Thus, reward type influences temporal discounting, suggesting a dissociation between temporal and spatial discounting. Our results illustrate how animals adjust choices and travel times depending on both the type of cost (time, distance) and benefits (food, social partners).

Delay discounting of qualitatively different reinforcers in rats

Humans discount larger delayed rewards less steeply than smaller rewards, whereas no such magnitude effect has been observed in rats (and pigeons). It remains possible that rats' discounting is sensitive to differences in the quality of the delayed reinforcer even though it is not sensitive to amount. To evaluate this possibility, Experiment 1 examined discounting of qualitatively different food reinforcers: highly preferred versus nonpreferred food pellets. Similarly, Experiment 2 examined discounting of highly preferred versus nonpreferred liquid reinforcers. In both experiments, an adjusting-amount procedure was used to determine the amount of immediate reinforcer that was judged to be of equal subjective value to the delayed reinforcer. The amount and quality of the delayed reinforcer were varied across conditions. Discounting was well described by a hyperbolic function, but no systematic effects of the quantity or the quality of the delayed reinforcer were observed.

Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid

Behavior such as depression of a lever or perception of a stimulus may be strengthened by consequent behaviorally significant events (BSEs), such as reinforcers. This is the Law of Effect. As time passes since its emission, the ability for the behavior to be reinforced decreases. This is trace decay. It is upon decayed traces that subsequent BSEs operate. If the trace comes from a response, it constitutes primary reinforcement; if from perception of an extended stimulus, it is classical conditioning. This paper develops simple models of these processes. It premises exponentially decaying traces related to the richness of the environment, and conditioned reinforcement as the average of such traces over the extended stimulus, yielding an almost-hyperbolic function of duration. The models account for some data, and reinforce the theories of other analysts by providing a sufficient account of the provenance of these effects. It leads to a linear relation between sooner and later isopreference delays whose slope depends on sensitivity to reinforcement, and intercept on that and the steepness of the delay gradient. Unlike human prospective judgments, all control is vested in either primary or secondary reinforcement processes; therefore the use of the term discounting, appropriate for humans, may be less descriptive of the behavior of nonverbal organisms.

Effects of time between trials on rats' and pigeons' choices with probabilistic delayed reinforcers

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.

Explicit information reduces discounting behavior in monkeys

Animals are notoriously impulsive in common laboratory experiments, preferring smaller, sooner rewards to larger, delayed rewards even when this reduces average reward rates. By contrast, the same animals often engage in natural behaviors that require extreme patience, such as food caching, stalking prey, and traveling long distances to high-quality food sites. One possible explanation for this discrepancy is that standard laboratory delay discounting tasks artificially inflate impulsivity by subverting animals' common learning strategies. To test this idea, we examined choices made by rhesus macaques in two variants of a standard delay discounting task. In the conventional variant, post-reward delays were uncued and adjusted to render total trial length constant; in the second, all delays were cued explicitly. We found that measured discounting was significantly reduced in the cued task, with discount parameters well below those reported in studies using the standard uncued design. When monkeys had complete information, their decisions were more consistent with a strategy of reward rate maximization. These results indicate that monkeys, and perhaps other animals, are more patient than is normally assumed, and that laboratory measures of delay discounting may overstate impulsivity.

The ecological rationality of delay tolerance: insights from capuchin monkeys

Both human and non-human animals often face decisions between options available at different times, and the capacity of delaying gratification has usually been considered one of the features distinguishing humans from other animals. However, this characteristic can widely vary across individuals, species, and types of task and it is still unclear whether it is accounted for by phylogenetic relatedness, feeding ecology, social structure, or metabolic rate. To disentangle these hypotheses, we evaluated temporal preferences in capuchin monkeys, South-American primates that, despite splitting off from human lineage approximately 35 million years ago, show striking behavioural analogies with the great apes. Then, we compared capuchins' performance with that of the other primate species tested so far with the same procedure. Overall, capuchins showed a delay tolerance significantly higher than closely related species, such as marmosets and tamarins, and comparable to that shown by great apes. Capuchins' tool use abilities might explain their comparatively high preference for delayed options in inter-temporal choices. Moreover, as in humans, capuchin females showed a greater delay tolerance than males, possibly because of their less opportunistic foraging style. Thus, our results shed light on the evolutionary origins of self-control supporting explanations of delay tolerance in terms of feeding ecology.

Pigeons' discounting of probabilistic and delayed reinforcers

Pigeons' discounting of probabilistic and delayed food reinforcers was studied using adjusting-amount procedures. In the probability discounting conditions, pigeons chose between an adjusting number of food pellets contingent on a single key peck and a larger, fixed number of pellets contingent on completion of a variable-ratio schedule. In the delay discounting conditions, pigeons chose between an adjusting number of pellets delivered immediately and a larger, fixed number of pellets delivered after a delay. Probability discounting (i.e., subjective value as a function of the odds against reinforcement) was as well described by a hyperboloid function as delay discounting was (i.e., subjective value as a function of the time until reinforcement). As in humans, the exponents of the hyperboloid function when it was fitted to the probability discounting data were lower than the exponents of the hyperboloid function when it was fitted to the delay discounting data. The subjective values of probabilistic reinforcers were strongly correlated with predictions based on simply substituting the average delay to their receipt in each probabilistic reinforcement condition into the hyperboloid discounting function. However, the subjective values were systematically underestimated using this approach. Using the discounting function proposed by Mazur (1989), which takes into account the variability in the delay to the probabilistic reinforcers, the accuracy with which their subjective values could be predicted was increased. Taken together, the present findings are consistent with Rachlin's (Rachlin, 1990; Rachlin, Logue, Gibbon, & Frankel, 1986) hypothesis that choice involving repeated gambles may be interpreted in terms of the delays to the probabilistic reinforcers.

Temporal discounting of reward and the cost of time in motor control

Why do movements take a characteristic amount of time, and why do diseases that affect the reward system alter control of movements? Suppose that the purpose of any movement is to position our body in a more rewarding state. People and other animals discount future reward as a hyperbolic function of time. Here, we show that across populations of people and monkeys there is a correlation between discounting of reward and control of movements. We consider saccadic eye movements and hypothesize that duration of a movement is equivalent to a delay of reward. The hyperbolic cost of this delay not only accounts for kinematics of saccades in adults, it also accounts for the faster saccades of children, who temporally discount reward more steeply. Our theory explains why saccade velocities increase when reward is elevated, and why disorders in the encoding of reward, for example in Parkinson's disease and schizophrenia, produce changes in saccade. We show that delay of reward elevates the cost of saccades, reducing velocities. Finally, we consider coordinated movements that include motion of eyes and head and find that their kinematics is also consistent with a hyperbolic, reward-dependent cost of time. Therefore, each voluntary movement carries a cost because its duration delays acquisition of reward. The cost depends on the value that the brain assigns to stimuli, and the rate at which it discounts this value in time. The motor commands that move our eyes reflect this cost of time.

Are people really more patient than other animals? Evidence from human discounting of real liquid rewards

In previous studies, researchers have found that humans discount delayed rewards orders of magnitude less steeply than do other animals. Humans also discount smaller delayed reward amounts more steeply than larger amounts, whereas animals apparently do not. These differences between humans and animals might reflect differences in the types of rewards studied and/or the fact that animals actually had to wait for their rewards. In the present article, we report the results of three experiments in which people made choices involving liquid rewards delivered and consumed after actual delays, thereby bridging the gap between animal and human studies. Under these circumstances, humans, like animals, discounted the value of rewards delayed by seconds; however, unlike animals, they still showed an effect of reward amount. Human discounting was well described by the same hyperboloid function that has previously been shown to describe animal discounting of delayed food and water rewards, as well as human discounting of real and hypothetical monetary rewards.

Extended access to amphetamine self-administration increases impulsive choice in a delay discounting task in rats

BACKGROUND

d-Amphetamine (AMPH) is a widely prescribed attention deficit hyperactivity disorder medication, but little is known about its effects on impulsive choice with escalated use.

OBJECTIVE

The current study examined the effects of short and long access to AMPH self-administration on impulsive choice in a delay discounting task in which rats chose between a small immediate reward (one sucrose pellet immediately) and a larger delayed reward (three sucrose pellets after an adjusting delay).

METHODS

Following choice stability in delay discounting, all rats received 15 1-h sessions of AMPH self-administration (0.1 or 0.03 mg/kg/infusion); self-administration sessions began 45 min after each delay discounting session. Rats were then either maintained on the short access (ShA) self-administration session or were switched to a long access (LgA) 6-h session for 21 days, followed by a 7-day withdrawal phase in which only the delay discounting task continued.

RESULTS

LgA rats in the 0.03 mg/kg/infusion dose group escalated in total number of infusions across sessions, although rats in the 0.1 mg/kg/infusion dose group did not. LgA groups at both unit doses showed decreased mean adjusted delays across sessions compared to the ShA groups, indicating that long access to AMPH increases impulsive choice. During the AMPH withdrawal phase, LgA groups returned back to baseline mean adjusted delays, indicating that the effect on impulsive choice was reversible.

CONCLUSION

These results show that extended AMPH self-administration produces a transient loss of inhibitory control, which may play a role in the escalating pattern of drug intake that characterizes the addiction process.

Cognitive impulsivity in animal models: role of response time and reinforcing rate in delay intolerance with two-choice operant tasks

Impulsivity, a key symptom of ADHD (attention-deficit hyperactivity disorder), is also common in obsessive-compulsive and addictive disorders. There is rising interest in animal models of inhibitory-control impairment. Adolescent rats were tested daily in the intolerance-to-delay (ID) task (session 25 min, timeout 20 s), involving choice between either immediate small amount of food (SS), or larger amount of food after a delay (LL). The mixed 5-HT(1A/7) agonist (8-OH-DPAT, 0 or 0.060 mg/kg i.p.) was administered acutely just before the last three sessions at highest delays. In addition to the classical choice parameter (percent LL preference), the spontaneous waiting (termed response time, RT) occurring between end of a timeout (TO) and next nose-poke was calculated. The pace between consecutive reinforcer deliveries is given by the mean inter-trial interval (mITI, i.e. TO + RT). Hence, the impact of any given delay may be proportional to this pace and be expressed as delay-equivalent odds, i.e. the extent by which delays are multiples of the mITI. Data revealed that RT/mITI increased sharply from around 15 s/35 s to around 30 s/50 s when imposed delay changed from 30 s to 45 s (i.e. odds from 0.91 to 1.06). This suggests that rats adopted a strategy allowing them to keep in pace with perceived reinforcing rate. The increasing delay constraint directly influenced the length of rats' spontaneous waiting (RT) before next decision. For higher delays, with odds >1, rats shifted to a clear-cut SS preference, which is devoid of any exogenous temporal constraint. A challenge with 8-OH-DPAT (0 or 0.060 mg/kg i.p.) decreased impulsive choice but also increased RT. Thus, tapping onto 5-HT(1A/7) receptors slightly enhanced RT/mITI values, possibly reflecting ability of rats to cope with slower reinforcing rates and/or with delay-cancelled reward paces. In summary, delay-induced states of aversion may arise from the innate tendency to rely on a regular rate of reinforcement. Conversely, a drug-enhanced capacity to cope with delay may involve an internal ability to adjust expectancy about such a reinforcing rate.

The effects of real versus hypothetical reward on delay and probability discounting

Delay discounting and probability discounting tasks providing hypothetical rewards have commonly been used to measure levels of impulsivity and risk taking. Results have been inconsistent as to whether real, compared with hypothetical, rewards influence choice behaviour in delay discounting tasks. This experiment examined choice behaviour in 38 healthy individuals using delay and probability discounting tasks involving both real and hypothetical rewards (pound sterling 0.10/pound sterling 0.20), real delays, and probabilistic outcomes. It was hypothesized that choice behaviour when receiving real rewards would be less impulsive and less risky than when receiving hypothetical rewards. The tasks were designed so that a mathematical model (the multiplicative hyperbolic model of choice) could be applied to assess effects on discounting parameters. The provision of real rewards was associated with significantly decreased impulsive, but not risky, choice although the size of effect was small and was influenced by the order of presentation. Repeated presentation of both tasks increased quantity discounting but not delay or probability discounting parameters. These results indicate that the nature of the reward provided, and repetition effects, need to be taken into account when designing discounting studies.

Perception of food amounts by chimpanzees (Pan troglodytes): the role of magnitude, contiguity, and wholeness

The authors investigated choice behavior by chimpanzees in five experiments involving choices between different amounts of food. Chimpanzees did not maximize the amount of food they obtained when choosing between a single 20-g banana piece and another option containing a 20-g piece and a 5-g piece. This was true even though they successfully discriminated between 20-g and 25-g banana pieces in other trials. When items in the mixed option were stacked, however, the chimpanzees chose the larger amount. Later experiments indicated that changing the magnitude of the two amounts did not change performance if the difference in magnitude between the two options remained the same (e.g., 40 g plus 10 g vs. 40 g). However, chimpanzees did improve when the two-item option was increased in its magnitude relative to the single slice (e.g., 20 g plus 10 g vs. 20 g). These results indicated that chimpanzees undervalued the total amount of food in sets when items differed in size and did not appear to be whole. Another experiment confirmed that it was this notion of wholeness that evoked suboptimal responding because chimpanzees were successful in the same comparisons with a different type of food that appeared less fractionated when presented as two pieces. These results provide evidence of suboptimal responding in some natural choice situations that prevents chimpanzees from maximizing food intake.

Percent body fat is related to delay and probability discounting for food in humans

This study describes delay and probability discounting patterns for hypothetical food and money in relation to percent body fat (PBF). Sixty university students completed four computerized discounting tasks in which they were asked to make a series of hypothetical decisions between (a) 10 dollars after one of several different delays (1, 2, 30, 180, and 365 days) or a smaller amount of money available immediately; (b) 10 bites of food after one of several delays (1, 2, 5, 10, and 20h) or a smaller number of bites available immediately; (c) $10 at one of several probabilities (0.9, 0.75, 0.5, 0.25, 0.1) or a smaller amount of money to be received for sure; and (d) 10 bites of food at one of several probabilities (0.9, 0.75, 0.5, 0.25, 0.1) or a smaller number of bites to be received for sure. Median indifference points for all participants across each task were well described using the hyperbolic discounting function. Results suggest that percent body fat predicted discounting for hypothetical food, but not money, using regression analyses with the entire sample and when comparing individuals in the high and low quartiles for PBF. None of the other dietary variables (body mass index, subjective hunger, and time since last meal or snack) were related to discounting patterns. This suggests that individuals with high PBF may exhibit heightened sensitivities to delay and probability when making decisions about food.

Delay discounting of saccharin in rhesus monkeys

The value of a reinforcer decreases as the time until its receipt increases, a phenomenon referred to as delay discounting. Although delay discounting of non-drug reinforcers has been studied extensively in a number of species, our knowledge of discounting in non-human primates is limited. In the present study, rhesus monkeys were allowed to choose in discrete trials between 0.05% saccharin delivered in different amounts and with different delays. Indifference points were calculated and discounting functions were established. Discounting functions for saccharin were well described by a hyperbolic function. Moreover, the discounting rates for saccharin in all six monkeys were comparable to those of other non-human animals responding for non-drug reinforcers. Also consistent with other studies of non-human animals, changing the amount of a saccharin reinforcer available after a 10-s delay did not affect its relative subjective value. Discounting functions for saccharin were steeper than we found in a previous study with cocaine, raising the possibility that drugs such as cocaine may be discounted less steeply than non-drug reinforcers.

Temporal discounting and inter-temporal choice in rhesus monkeys

Humans and animals are more likely to take an action leading to an immediate reward than actions with delayed rewards of similar magnitudes. Although such devaluation of delayed rewards has been almost universally described by hyperbolic discount functions, the rate of this temporal discounting varies substantially among different animal species. This might be in part due to the differences in how the information about reward is presented to decision makers. In previous animal studies, reward delays or magnitudes were gradually adjusted across trials, so the animals learned the properties of future rewards from the rewards they waited for and consumed previously. In contrast, verbal cues have been used commonly in human studies. In the present study, rhesus monkeys were trained in a novel inter-temporal choice task in which the magnitude and delay of reward were indicated symbolically using visual cues and varied randomly across trials. We found that monkeys could extract the information about reward delays from visual symbols regardless of the number of symbols used to indicate the delay. The rate of temporal discounting observed in the present study was comparable to the previous estimates in other mammals, and the animal's choice behavior was largely consistent with hyperbolic discounting. Our results also suggest that the rate of temporal discounting might be influenced by contextual factors, such as the novelty of the task. The flexibility furnished by this new inter-temporal choice task might be useful for future neurobiological investigations on inter-temporal choice in non-human primates.

Dopamine modulates effort-based decision making in rats

Previous research has implicated dopamine as a modulating factor in choice behavior based on effort. The purpose of the present study was to determine the individual contribution of different dopamine receptors to effort-based decision making in rats. Rats were trained in a T-maze to choose a large-reward arm that contained 8 pellets of food over a small-reward arm that contained 2 pellets of food. The rats then were trained to climb progressively higher barriers to obtain the food from the large-reward arm. Using a discounting procedure on each test day, it was found that rats were more likely to choose the small-reward arm after treatment with the D1 antagonist, SCH-23390, or the D2 antagonist, haloperidol. The dopamine agonist, D-amphetamine, biased the rats toward choosing the large-reward arm and blunted the effects of SCH-23390 or haloperidol. Treatment with the D3 receptor antagonist, U99194, or the D3 receptor agonist, 7-OH-DPAT, did not alter choice behavior. These data indicate that D1 and D2 receptors are required for decisions based on effort.

Delay-amount tradeoffs in choices by pigeons and rats: hyperbolic versus exponential discounting

An adjusting-delay procedure was used to study the choices of pigeons and rats when both delay and amount of reinforcement were varied. In different conditions, the choice alternatives included one versus two reinforcers, one versus three reinforcers, and three versus two reinforcers. The delay to one alternative (the standard alternative) was kept constant in a condition, and the delay to the other (the adjusting alternative) was increased or decreased many times a session so as to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Indifference functions were constructed by plotting the adjusting delay as a function of the standard delay for each pair of reinforcer amounts. The experiments were designed to test the prediction of a hyperbolic decay equation that the slopes of the indifference functions should increase as the ratio of the two reinforcer amounts increased. Consistent with the hyperbolic equation, the slopes of the indifference functions depended on the ratios of the two reinforcer amounts for both pigeons and rats. These results were not compatible with an exponential decay equation, which predicts slopes of 1 regardless of the reinforcer amounts. Combined with other data, these findings provide further evidence that delay discounting is well described by a hyperbolic equation for both species, but not by an exponential equation. Quantitative differences in the y-intercepts of the indifference functions from the two species suggested that the rate at which reinforcer strength decreases with increasing delay may be four or five times slower for rats than for pigeons.

Effects of altering reinforcer magnitude and reinforcement schedule on phencyclidine (PCP) self-administration in monkeys using an adjusting delay task

Compared to nondrug reinforcers, few studies have examined delay discounting for drug reinforcers. The purpose of the present study was to examine delay discounting in rhesus monkeys using orally delivered phencyclidine (PCP) as the reinforcer and to examine the effects of manipulating reinforcer magnitude and cost on delay discounting for PCP using an adjusting delay task. Monkeys could choose between a single delivery of PCP available immediately or a bundle of PCP deliveries available following a titrated delay. The average of the delays, or the mean adjusted delay (MAD), served as the quantitative measure of delay discounting. In Experiment 1, reinforcer magnitude was manipulated by varying the PCP concentration and the size of the delayed reinforcer (6 or 12 deliveries). The concentration-effect curve for PCP deliveries assumed an inverted U-shaped function, but varying PCP concentration had little effect on MAD values or the choice between immediate and delayed reinforcers. Increasing the size of the delayed reinforcer produced an upward and leftward shift in the concentration-effect curve. In Experiment 2, the cost of reinforcers was manipulated by increasing the fixed ratio (FR) requirement for each choice. Increasing the FR led to increased MAD values and decreased PCP self-administration.

The role of impulsive behavior in drug abuse

BACKGROUND

Impulsivity is a multifaceted construct that has recently been recognized as a factor contributing to enhanced vulnerability to drug abuse.

OBJECTIVES

In the present review, we focus on two facets of impulsivity (and tasks that measure them): (1) impulsive choice (delay discounting task) and (2) inhibitory failure (go/no-go, stop signal reaction time, and five-choice serial reaction time tasks). We also describe how performance on each of these tasks is associated with drug-related behavior during phases of drug abuse that capture the essential features of addiction (acquisition, escalation, and reinstatement of drug-seeking after drug access has terminated). Three hypotheses (H) regarding the relationship between impulsivity and drug abuse are discussed: (1) increased levels of impulsivity lead to drug abuse (H1), (2) drugs of abuse increase impulsivity (H2), and (3) impulsivity and drug abuse are associated through a common third factor (H3).

CONCLUSION

Impulsivity expressed as impulsive choice or inhibitory failure plays a role in several key transition phases of drug abuse. There is evidence to support all three nonexclusive hypotheses. Increased levels of impulsivity lead to acquisition of drug abuse (H1) and subsequent escalation or dysregulation of drug intake. Drugs of abuse may increase impulsivity (H2), which is an additional contributor to escalation/dysregulation. Abstinence, relapse, and treatment may be influenced by both H1 and H2. In addition, there is a relationship between impulsivity and other drug abuse vulnerability factors, such as sex, hormonal status, reactivity to nondrug rewards, and early environmental experiences that may impact drug intake during all phases of addiction (H3). Relating drug abuse and impulsivity in phases of addiction via these three hypotheses provides a heuristic model from which future experimental questions can be addressed.

Jeab and the Skinnerian interpretation of behavior

At one time, JEAB editorial policy was perceived by some to consist mainly of dogmatically enforcing a Skinnerian interpretation of all findings reported in the journal. Partly in response to that undeserved reputation, the journal explicitly defined itself as the place to publish research on the behavior of individual organisms, and not as a place that encourages any particular theoretical orientation. My own view is that this should not be the journal's identity; instead, JEAB should be the one journal that seriously considers Skinnerian (and related environment based) interpretations of empirical results, whether or not the study involved an analysis of individual subject behavior and whether or not the Skinnerian analysis is ultimately endorsed by the author.

The Journal of the Experimental Analysis of Behavior at zero, fifty, and one hundred

The experimental content areas represented in JEAB in its first volume (1958) and 50 years later in Volume 87 are in many ways similar with regard to research on schedules of reinforcement, research with human subjects, and several other topics. Experimental analysis has not been displaced by quantitative analysis. Much less research on aversive control has been published in recent than in earlier years. Wishes for progress in the next 50 years include experiments on verbal behavior, the sources of novel behavior, and observing responses based on stimuli correlated with escape or avoidance.

Economic principles motivating social attention in humans

We know little about the processes by which we evaluate the opportunity to look at another person. We propose that behavioural economics provides a powerful approach to understanding this basic aspect of social attention. We hypothesized that the decision process culminating in attention to another person follows the same economic principles that govern choices about rewards such as food, drinks and money. Specifically, such rewards are discounted as a function of time, are tradable for other rewards, and reinforce work. Behavioural and neurobiological evidence suggests that looking at other people can also be described as rewarding, but to what extent these economic principles apply to social orienting remains unknown. Here, we show that the opportunity to view pictures of the opposite sex is discounted by delay to viewing, substitutes for money and reinforces work. The reward value of photos of the opposite sex varied with physical attractiveness and was greater in men, suggesting differential utility of acquiring visual information about the opposite sex in men and women. Together, these results demonstrate that choosing whom to look at follows a general set of economic principles, implicating shared neural mechanisms in both social and non-social decision making.

Do adjusting-amount and adjusting-delay procedures produce equivalent estimates of subjective value in pigeons?

The current experiment examined whether adjusting-amount and adjusting-delay procedures provide equivalent measures of discounting. Pigeons' discounting on the two procedures was compared using a within-subject yoking technique in which the indifference point (number of pellets or time until reinforcement) obtained with one procedure determined the value of the corresponding variable in the yoked condition with the other procedure. Behavior on each procedure was well described by a hyperbolic discounting function. Results revealed no systematic differences in the degree of discounting as measured by the discounting rate parameter of the hyperbola in Experiment 1, which used 20-mg pellets. These results were replicated in Experiment 2 using smaller, 14-mg pellets, which potentially yield more precise measurement of indifference points on the adjusting-amount procedure. The finding that estimates of the k parameter in the hyperbolic discounting function obtained with one procedure did not differ systematically from estimates obtained from the same subjects with the other procedure represents strong support for the hypothesis that the same process underlies the discounting of delayed rewards on both adjusting-amount and adjusting-delay procedures.

Delay discounting of cocaine by rhesus monkeys

The present, subjective value of a reinforcer typically decreases as a function of the delay to its receipt, a phenomenon termed delay discounting. Delay discounting, which is assumed to reflect impulsivity, is hypothesized to play an important role in drug abuse. The present study examined delay discounting of cocaine injections by rhesus monkeys. Subjects were studied on a discrete-trials task in which they chose between 2 doses of cocaine: a smaller, immediate dose and a larger, delayed dose. The immediate dose varied between 0.012 and 0.4 mg/kg/injection, whereas the delayed dose was always 0.2 mg/kg/injection and was delivered after a delay that varied between 0 and 300 s in different conditions. At each delay, the point at which a monkey chose the immediate and delayed doses equally often (i.e., the ED50) provided a measure of the present, subjective value of the delayed dose. Dose-response functions for the immediate dose shifted to the left as delay increased. The amount of the immediate dose predicted to be equal in subjective value to the delayed dose decreased as a function of the delay, and hyperbolic discounting functions provided good fits to the data (median R(2)=.86). The current approach may provide the basis for an animal model of the effect of delay on the subjective value of drugs of abuse.

Experimental evidence for route integration and strategic planning in wild capuchin monkeys

Both in captivity and the wild, primates are found to travel mostly to the nearest available resource, but they may skip over the closest resource and travel to more distant resources, which are often found to be more productive. This study examines the tradeoff between distance and reward in the foraging choices of one group of wild capuchin monkeys (Cebus apella nigritus) using feeding platforms in large-scale foraging experiments conducted over four years. Three feeding sites were arrayed in an oblique triangle, such that once the monkey group had chosen one site to feed, they had a choice between two remaining sites, a close one with less food and the other up to 2.3 times as far away but with more food. Sites were provisioned once per day. The capuchins generally chose the closer feeding site, even when the more distant site offered up to 12 times as much food. The distances to, rewards of, or various profitability measures applied to each alternative site individually did not explain the group's choices in ways consistent with foraging theory or principles of operant psychology. The group's site choices were predicted only by comparing efficiency measures of entire foraging pathways: (1) direct travel to the more rewarding distant site, versus (2) the longer 'detour' through the closer site on the way to the more distant one. The group chose the detour more often when the reward was larger and the added detour distance shorter. They appeared to be more sensitive to the absolute increase in detour distance than to the relative increase compared to the straight route. The qualitative and quantitative results agree with a simple rule: do not use the detour unless the energy gain from extra food outweighs the energy cost of extra travel. These results suggest that members of this group integrate information on spatial location, reward, and perhaps potential food competition in their choice of multi-site foraging routes, with important implications for social foraging.

Impulsivity (delay discounting) for food and cocaine in male and female rats selectively bred for high and low saccharin intake

Previous research in rats indicates that delay discounting for food, a model of impulsivity, predicted the rate of acquisition of cocaine self-administration. In other studies, rats bred for high saccharin intake (HiS) acquired cocaine self-administration at higher rates than those with low saccharin intake (LoS), and female (F) rats acquired cocaine self-administration more rapidly than males (M). The purpose of this study was to examine a possible connection between impulsivity, saccharin intake, and sex by comparing M and F rats from the HiS and LoS selectively bred lines on measures of impulsivity; i.e., their rate of delay discounting for food or i.v. cocaine infusions. The adjusting delay procedure allowed rats access to 2 response levers, and a pellet dispenser or an i.v. drug infusion pump. In 4 groups (HiS M, HiS F, LoS M, LoS F) responses under a fixed-ratio (FR) 1 schedule on one lever resulted in one 45 mg pellet immediately, and responses on the other lever resulted in 3 or 6 pellets after a delay. Four additional groups received either a small cocaine (0.2, 0.4, or 0.8 mg/kg) infusion immediately or a delayed larger infusion (3x the amount of the small infusions). The delay to the larger reinforcer began at 6 s and increased or decreased by 1 s following responses on the delay or immediate levers, respectively. A mean adjusted delay (MAD) was calculated over 30 choice trials during each daily 3-hour session, and it was used as a quantitative measure of impulsivity. In groups maintained by food, HiS rats were more impulsive (lower MADs) than LoS rats, and LoS females were more impulsive than LoS males. There were no phenotype or sex differences in delay discounting for cocaine. Understanding the relationship between impulsivity and other predictors of drug abuse (e.g., sex, saccharin intake) is important in developing prevention and treatment strategies.

Temporal development of asymmetric reward-induced bias in macaques

Time and expected outcome are two ubiquitous factors contributing to decision-making. However, it is unclear how they interact to influence motor responses. When two differential reward outcomes are expected at the end of a waiting period, behavioral bias is consistently induced, manifested as shorter latencies for motor responses associated with the preferred reward. To examine how this bias develops in time during the waiting period, we manipulated the duration of the waiting period in an asymmetric reward saccade task in monkeys. We found that the bias increases with the duration of waiting period. Surprisingly, the bias resulted from gradual suppression of saccades to nonpreferred targets rather than from facilitation of saccades to preferred targets. These results have important implications on the neural correlates of reward-induced bias.

Integrated versus segregated accounting and the magnitude effect in temporal discounting

Temporal discounting rates in humans generally decrease as the amount of reward increases, a phenomenon known as the magnitude effect. In the present study, we examined whether temporal discounting and the magnitude effect are related to segregation of choices in terms of gains or losses for waiting for or expediting receipt of a reward. Subjects (N = 24) responded to a series of hypothetical choices about amounts of money available either immediately or after a delay. The immediate and delayed amounts either were presented as integrated amounts in the baseline condition or were segregated as differential gains or losses for choosing delayed or expedited consumption (delay and speedup conditions, respectively). Temporal discounting rates decreased in the segregated conditions, in accord with the standard discounted utility model but contrary to the hypothesis that the subjects were choosing on the basis of reward differentials in the baseline condition. The size of the magnitude effect was comparable in the baseline and the delay conditions but decreased in the speed-up condition. These results challenge explanations of the magnitude effect in terms of an increasing proportional sensitivity property of the utility function (Loewenstein & Prelec, 1992) and the hypothesis that subjects choose on the basis of differentials even when the rewards are presented as integrated amounts.

Mathematical models and the experimental analysis of behavior

The use of mathematical models in the experimental analysis of behavior has increased over the years, and they offer several advantages. Mathematical models require theorists to be precise and unambiguous, often allowing comparisons of competing theories that sound similar when stated in words. Sometimes different mathematical models may make equally accurate predictions for a large body of data. In such cases, it is important to find and investigate situations for which the competing models make different predictions because, unless two models are actually mathematically equivalent, they are based on different assumptions about the psychological processes that underlie an observed behavior. Mathematical models developed in basic behavioral research have been used to predict and control behavior in applied settings, and they have guided research in other areas of psychology. A good mathematical model can provide a common framework for understanding what might otherwise appear to be diverse and unrelated behavioral phenomena. Because psychologists vary in their quantitative skills and in their tolerance for mathematical equations, it is important for those who develop mathematical models of behavior to find ways (such as verbal analogies, pictorial representations, or concrete examples) to communicate the key premises of their models to nonspecialists.

Effects of methylphenidate and morphine on delay-discount functions obtained within sessions

Four rats responded under a "self-control" procedure designed to obtain delay-discount functions within sessions. Each session consisted of seven blocks, with seven trials within each block. Each block consisted of two initial forced-choice trials followed by five free-choice trials. On choice trials, the rats could press either of two retractable levers. A press on one lever was followed by presentation of a smaller reinforcer (a single dipper presentation of a sucrose solution); a press on the other lever was followed by presentation of a larger reinforcer (four consecutive dipper presentations). The delay associated with the smaller reinforcer always was 0 s, whereas the signaled delay associated with the larger reinforcer increased across blocks (from 0 to 50 s). Under these conditions, the percentage of choices of the larger reinforcer decreased across blocks, and relatively reliable delay-discount functions were obtained within sessions. Doses of methylphenidate (1.0 to 17.0 mg/kg) and morphine (0.3 to 17.0 mg/kg) were then administered prior to selected sessions. Typically, intermediate doses of methylphenidate shifted the discount functions to the right (increased choices of the larger reinforcer). For 2 of the rats, this effect was pronounced; for the other 2 rats, this effect occurred after the range of delays for the larger reinforcer was decreased (0 to 20 s). On the other hand, in most cases morphine produced a slight leftward shift in the discount function (decreased choices of the larger reinforcer). The present procedure appears to be a useful and efficient method to characterize drug effects on an entire delay-discount function. As with many procedures used to study self-control choices, however, sources of control other than reinforcement delay and amount may have been operating in the present study, and these sources must be considered when interpreting drug effects.

Will travel for food: spatial discounting in two new world monkeys

Nonhuman animals steeply discount the future, showing a preference for small, immediate over large, delayed rewards. Currently unclear is whether discounting functions depend on context. Here, we examine the effects of spatial context on discounting in cotton-top tamarins (Saguinus oedipus) and common marmosets (Callithrix jacchus), species known to differ in temporal discounting. We presented subjects with a choice between small, nearby rewards and large, distant rewards. Tamarins traveled farther for the large reward than marmosets, attending to the ratio of reward differences rather than their absolute values. This species difference contrasts with performance on a temporal task in which marmosets waited longer than tamarins for the large reward. These comparative data indicate that context influences choice behavior, with the strongest effect seen in marmosets who discounted more steeply over space than over time. These findings parallel details of each species' feeding ecology. Tamarins range over large distances and feed primarily on insects, which requires using quick, impulsive action. Marmosets range over shorter distances than tamarins and feed primarily on tree exudates, a clumped resource that requires patience to wait for sap to exude. These results show that discounting functions are context specific, shaped by a history of ecological pressures.

A discounting framework for choice with delayed and probabilistic rewards

When choosing between delayed or uncertain outcomes, individuals discount the value of such outcomes on the basis of the expected time to or the likelihood of their occurrence. In an integrative review of the expanding experimental literature on discounting, the authors show that although the same form of hyperbola-like function describes discounting of both delayed and probabilistic outcomes, a variety of recent findings are inconsistent with a single-process account. The authors also review studies that compare discounting in different populations and discuss the theoretical and practical implications of the findings. The present effort illustrates the value of studying choice involving both delayed and probabilistic outcomes within a general discounting framework that uses similar experimental procedures and a common analytical approach.