Discounting of delayed food rewards in pigeons and rats: is there a magnitude effect?

Rats are optimal in a choice task in which pigeons are not

In an extensive list of studies, it has been found that pigeons prefer an alternative associated with discriminative stimuli over another associated with non-discriminative stimuli, even when the probability of reinforcement is higher in the latter. This behavior has been named "suboptimal choice". In the present experiment, we evaluated whether rats, another widely studied species within the Experimental Analysis of Behavior, also shows this behavior. We systematically replicated the procedure employed with pigeons, and found that rats are not suboptimal, i.e., they prefer the non-discriminative alternative associated with .5 probability of reinforcement, over the discriminative alternative associated with .2 probability of reinforcement. This effect occurred even though rats discriminated the contingencies of reinforcement associated with each stimulus, suggesting that rats' optimal choice was driven by the overall probability of reinforcement of each alternative. Different procedural details are offered as possibilities for explaining this apparent inter-species difference.

Delay discounting of food by rhesus monkeys: Cocaine and food choice in isomorphic and allomorphic situations

Research on delay discounting has focused largely on nondrug reinforcers in an isomorphic context in which choice is between alternatives that involve the same type of reinforcer. Less often, delay discounting has been studied with drug reinforcers in a more ecologically valid allomorphic context where choice is between alternatives involving different types of reinforcers. The present experiment is the first to examine discounting of drug and nondrug reinforcers in both isomorphic and allomorphic situations using a theoretical model (i.e., the hyperbolic discounting function) that allows for comparisons of discounting rates between reinforcer types and amounts. The goal of the current experiment was to examine discounting of a delayed, nondrug reinforcer (food) by male rhesus monkeys when the immediate alternative was either food (isomorphic situation) or cocaine (allomorphic situation). In addition, we sought to determine whether there was a magnitude effect with delayed food in the allomorphic situation. Choice of immediate food and immediate cocaine increased with amount and dose, respectively. Choice functions for immediate food and cocaine generally shifted leftward as delay increased. Compared to isomorphic situations in which food was the immediate alternative, delayed food was discounted more steeply in allomorphic situations where cocaine was the immediate alternative. Notably, discounting was not affected by the magnitude of the delayed reinforcer. These data indicate that how steeply a delayed nondrug reinforcer is discounted may depend more on the qualitative characteristics of the immediate reinforcer and less on the magnitude of the delayed one.

Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation

There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity.

One of the main functions of the brain is to select sequences of actions that lead to rewarding outcomes (e.g., food). However, such rewards are often not readily available; instead physical effort may be required to obtain them, or their arrival may be delayed. The ability to integrate the costs and benefits of potential courses of action is severely impaired in several common disorders, such as depression and schizophrenia. Mathematical models can describe how individuals depreciate rewards based on the costs associated with them. For example, models of how a reward loses appeal with increasing temporal delays can provide individual impulsivity scores, and can serve as a predictor of financial mismanagement. To date, there is no established model to describe accurately how humans depreciate rewards when obtaining them requires physical effort. This is surprising given the prevalence of disorders related to a diminished willingness to exert effort. Here we derive a biologically plausible mathematical model that can describe how healthy humans make decisions tied to physical efforts. We show that effort and delay influence reward valuation in different ways, contrary to common assumptions. Our model will be important for characterizing decision-making deficits in clinical disorders characterized by behavioral inactivity.

Sex difference in the relationship between salivary testosterone and inter-temporal choice

Humans often prefer a small immediate reward to large reward in the future. This myopic tendency in inter-temporal choice is termed delay discounting, and has been the focus of intensive research in the past decades. Recent studies indicate that the neural regions underlying delay discounting are influenced by the gonadal steroids. However, the specific relationship between the testosterone levels and delay discounting is unclear at this point, especially in females. The present study investigated the relationship between salivary testosterone concentrations and discounting rates in delay- and probability-discounting tasks with healthy males and females. The results revealed a positive correlation between testosterone concentrations and delay-discounting rates in females and a negative correlation in males. Testosterone concentrations were unrelated to probability-discounting rates. Although causal effects of testosterone cannot be certain in this correlational study, if testosterone directly influenced this behavior, observed sex differences in delay discounting may be evidence of a curvilinear effect of testosterone. Alternatively, the findings may reflect inverse pattern of responsiveness to testosterone between male and female neural systems, or basic sex-difference in the neural mechanism underlying delay-discounting independent of testosterone itself.

Models of ADHD: Five ways smaller sooner is better

BACKGROUND

In the delay discounting paradigm choices are made between a good of small utility available soon (SS) vs. a good of greater utility available later (LL). Versions of the task may be used with human and non-human subjects, they generate characteristic data, and are of relevance the topic of this special issue, the imputed impulsivity of organisms categorized as ADHD, and animal models of ADHD.

METHOD

Mathematical models of judgment were formulated. The first model adds the utility of a good to the utility of the delay. It provides distinct information about future perspective, the marginal utility of the goods offered, and the relative weight on delay. It grounds hyperbolic discounting. It predicts slower rates of discount for more preferred goods. It can reduce to an exponential function on future perspective time. When choice is mediated by conditioning, not judgment, a model of conditioned reinforcement strength is written. For studies where only the degree of preference for the LL is reported, a Thurstonian model of discrimination provides a shell that embraces all of the earlier models, providing additional information about the precision and bias of those preferential judgments.

COMPARISON WITH OTHER MODELS

All other prominent models are nested within, or approximations to, the models described here.

RESULTS

The models are fit to exemplary data. They provide some support for theories of delay aversion, and explanations for some inconsistencies in the literature. They involve parameters of general utility and provide connection to economic models.

The evolutionary roots of human decision making

Humans exhibit a suite of biases when making economic decisions. We review recent research on the origins of human decision making by examining whether similar choice biases are seen in nonhuman primates, our closest phylogenetic relatives. We propose that comparative studies can provide insight into four major questions about the nature of human choice biases that cannot be addressed by studies of our species alone. First, research with other primates can address the evolution of human choice biases and identify shared versus human-unique tendencies in decision making. Second, primate studies can constrain hypotheses about the psychological mechanisms underlying such biases. Third, comparisons of closely related species can identify when distinct mechanisms underlie related biases by examining evolutionary dissociations in choice strategies. Finally, comparative work can provide insight into the biological rationality of economically irrational preferences.

Level of deprivation does not affect degree of discounting in pigeons

Two experiments tested the effects of food deprivation on discounting in pigeons. An adjusting-amount procedure was used to estimate the subjective value of food at delays ranging from 1 to 24 s. Experiment 1 compared pigeons' discounting of delayed food reinforcers at 75 %-80 % and 90 %-95 % of free-feeding weight. Experiment 2 compared discounting under 1- and 23-h food deprivation. In both experiments at both deprivation levels, discounting was well described by the hyperboloid discounting function. No systematic effect of level of deprivation on degree of discounting was observed in either experiment. This finding is consistent with the view that pigeons' choices are controlled by the relative, rather than the absolute, value of reinforcers.

Increased delay discounting tracks with a high ethanol-seeking phenotype and subsequent ethanol seeking but not consumption

BACKGROUND

Increased levels of delay discounting have been associated with alcoholism and problematic levels of drinking. Attempts to assess the directionality of this relationship by studying individuals with a family history of alcoholism as well as rodent lines selectively bred for high home cage alcohol preference have yielded discordant results. One possible reason for this discordance is that increased levels of delay discounting may only track with specific processes that lead to addiction vulnerability. This study investigated this possibility by assessing 3 strains of rats previously identified to exhibit heritable differences in ethanol (EtOH) seeking and consumption.

METHODS

In an adjusting amount delay discounting task, alcohol-preferring (P) rats who display high levels of both EtOH seeking and consumption were compared to high alcohol-drinking (HAD2) rats who only exhibit moderate EtOH seeking despite high levels of consumption, and Long Evans (LE) rats who display moderate seeking and consumption. EtOH-seeking and consumption phenotypes were subsequently confirmed in an operant self-administration task with a procedural separation between EtOH seeking and drinking.

RESULTS

P rats discounted delayed rewards to a greater extent than both HAD2s and LE who did not show differences in discounting. Moreover, the EtOH-seeking and drinking phenotypes were replicated with P rats displaying greater EtOH seeking compared to both the HAD2s and LE, and both the HAD2s and P rats consuming more EtOH than LEs.

CONCLUSIONS

Only the high-seeking strain, the P rats, exhibited increased levels of delay discounting. This suggests that this measure of behavioral under-control is specifically associated with alcohol-related appetitive, but not consummatory, processes as the moderate seeking/high drinking line did not show increased levels of impulsivity. This finding supports the hypothesis that delay discounting is specifically associated with only certain processes which are sufficient but not necessary to confer addiction vulnerability and therefore also supports increased levels of delay discounting as a predisposing risk factor for alcoholism.

Rats' choices with token stimuli in concurrent variable-interval schedules

Four rats responded on concurrent variable-interval schedules that delivered token stimuli (stimulus lights arranged vertically above each of two side levers). During exchange periods, each token could be exchanged for one food pellet by responding on a center lever, with one response required for each pellet delivery. In different conditions, the exchange requirements (number of tokens that had to be earned before they could be exchanged for food) varied between one and four for the two response levers. The experiments were closely patterned after research with pigeons by Mazur and Biondi (2013), and the results from the rats in the present experiment were similar. Response percentages on the two levers changed as each additional token was earned, and these patterns indicated that choice was controlled by both the time to the exchange periods and the number of food pellets that were delivered in the exchange period. In some conditions, the exchange requirement was three tokens for each lever, but the token lights were not turned on as they were earned for one of the two keys. The rats showed a slight preference for the lever without the token lights, which may indicate that the token lights were not serving as conditioned reinforcers (a result also found by Mazur and Biondi with pigeons). Overall, these results suggest that, in this choice procedure, the token stimuli served primarily as discriminative stimuli that signaled the temporal proximity and quantity of the primary reinforcer, food.

Relatively high motivation for context-evoked reward produces the magnitude effect in rats

Using a concurrent-chain schedule, we demonstrated the effect of absolute reinforcement (i.e., the magnitude effect) on choice behavior in rats. In general, animals' simultaneous choices conform to a relative reinforcement ratio between alternatives. However, studies in pigeons and rats have found that on a concurrent-chain schedule, the overall reinforcement ratio, or absolute amount, also influences choice. The effect of reinforcement amount has also been studied in inter-temporal choice situations, and this effect has been referred to as the magnitude effect. The magnitude effect has been observed in humans under various conditions, but little research has assessed it in animals (e.g., pigeons and rats). The present study confirmed the effect of reinforcement amount in rats during simultaneous and inter-temporal choice situations. We used a concurrent-chain procedure to examine the cause of the magnitude effect during inter-temporal choice. Our results suggest that rats can use differences in reinforcement amount as a contextual cue during choice, and the direction of the magnitude effect in rats might be similar to humans when using the present procedure. Furthermore, our results indicate that the magnitude effect was caused by the initial-link effect when the reinforcement amount was relatively small, while a loss aversion tendency was observed when the reinforcement amount changed within a session. The emergence of the initial-link effect and loss aversion suggests that rats make choices through cognitive processes predicted by prospect theory.

Pigeons' delay discounting functions established using a concurrent-chains procedure

Several studies have examined discounting by pigeons and rats using concurrent-chains procedures, but the results have been inconsistent. None of these studies, however, has established that discounting functions derived from estimates of indifference points can be obtained with a concurrent-chains procedure, so their validity remains in doubt. The present study used a concurrent-chains procedure within sessions combined with an adjusting-amount procedure across sessions to determine the present, subjective values of food reinforcers to be obtained after a delay. Discounting was well described by the hyperbolic discounting function, suggesting that the concurrent-chains procedure and the more typical adjusting-amount procedure are measuring the same process. Consistent with previous studies with rats and pigeons using adjusting-amount procedures, no significant effect of the amount of the delayed reinforcer on the degree of discounting was observed, suggesting that the amount effect may be unique to humans although consistent with the view that animals' choices are controlled by the relative, rather than the absolute, value of reinforcers.

No evidence of temporal preferences in caching by Western scrub-jays (Aphelocoma californica)

•

We ask whether scrub-jays prefer earlier cache-retrieval when caching.

•

Caching appears similar for trays that come back after 1 h, 25 h, and 49 h.

•

Caching appears similar for delays of 7 min and 26 h when no choice of trays is provided.

•

We find no evidence of a preference to cache in trays that are more accessible for recovery.

Humans and other animals often favour immediate gratification over long-term gain. Primates, including humans, appear more willing to wait for rewards than other animals, such as rats or pigeons. Another group displaying impressive patience are the corvids, which possess large brains and show sophisticated cognitive abilities. Here, we assess intertemporal choice in one corvid species, the Western scrub-jay (Aphelocoma californica). These birds cache food for future consumption and respond flexibly to future needs. Cache-theft and cache-degradation are time-dependent processes in scrub-jay ecology that might necessitate sensitivity to delays between caching and retrieval. We adopt a caching paradigm with delays of up to 49 h. Across two experiments we find no evidence of a preference for earlier recovery. We highlight the possibility that, although scrub-jays can discriminate between the present and the future, they may not understand how far into the future an event will occur.

Do pigeons prefer information in the absence of differential reinforcement?

Prior research has indicated that pigeons do not prefer an alternative that provides a sample (for matching to sample) over an alternative that does not provide a sample (i.e., there is no indication of which comparison stimulus is correct). However, Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) showed that when delay of reinforcement was controlled, pigeons had a strong preference for matching over pseudomatching (i.e., there was a sample, but it did not indicate which comparison stimulus was correct). Experiment 1 of the present study replicated and extended the results of the Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) study by including an identity relation between the sample and one of the comparison stimuli in both the matching and pseudomatching tasks. In Experiment 2, in which we asked whether the pigeons would still prefer matching if we equated the two tasks for probability of reinforcement, we found no systematic preference for matching over pseudomatching. Thus, it appears that in the absence of differential reinforcement, the information provided by a sample that signals which of the two comparison stimuli is correct is insufficient to produce a preference for that alternative.

A critical role for the hippocampus in the valuation of imagined outcomes

Converging evidence from neuroimaging and clinical data demonstrates the important involvement of the hippocampus in finding the motivation to pursue goals that we need to imagine because they are not within sight.

Many choice situations require imagining potential outcomes, a capacity that was shown to involve memory brain regions such as the hippocampus. We reasoned that the quality of hippocampus-mediated simulation might therefore condition the subjective value assigned to imagined outcomes. We developed a novel paradigm to assess the impact of hippocampus structure and function on the propensity to favor imagined outcomes in the context of intertemporal choices. The ecological condition opposed immediate options presented as pictures (hence directly observable) to delayed options presented as texts (hence requiring mental stimulation). To avoid confounding simulation process with delay discounting, we compared this ecological condition to control conditions using the same temporal labels while keeping constant the presentation mode. Behavioral data showed that participants who imagined future options with greater details rated them as more likeable. Functional MRI data confirmed that hippocampus activity could account for subjects assigning higher values to simulated options. Structural MRI data suggested that grey matter density was a significant predictor of hippocampus activation, and therefore of the propensity to favor simulated options. Conversely, patients with hippocampus atrophy due to Alzheimer's disease, but not patients with Fronto-Temporal Dementia, were less inclined to favor options that required mental simulation. We conclude that hippocampus-mediated simulation plays a critical role in providing the motivation to pursue goals that are not present to our senses.

Economic theory assumes that we assign some sort of value to options that are presented to us in order to choose between them. In neuroscience, evidence suggests that memory brain regions, such as the hippocampus, are involved in imagining novel situations. We therefore hypothesized that the hippocampus might be critical for evaluating outcomes that we need to imagine. This is typically the case in intertemporal choices, where immediate rewards are considered against future gratifications (e.g., a beer now or a bottle of champagne a week from now). Previous investigations have implicated the dorsal prefrontal cortex brain region in resisting immediate rewards. Here we manipulated the mode of presentation (text or picture), such that options were represented either in simulation or in perception systems. Functional neuroimaging data confirmed that hippocampal activity lends a preference to choosing simulated options (irrespective of time), whereas dorsal prefrontal cortex brain activity supports the preference for delayed options (irrespective of presentation mode). Structural neuroimaging in healthy subjects and in patients with brain atrophy, due to Alzheimer's disease (with hippocampal damage) or Fronto-Temporal Dementia (with damage to the prefrontal cortex), further demonstrated the critical implication of the hippocampus. Individuals with higher neuronal density in the hippocampus, but not in the dorsal prefrontal cortex, were more likely to choose future rewards that have to be mentally simulated.

Adaptive intertemporal preferences in foraging-style environments

Decision makers often face choices between smaller more immediate rewards and larger more delayed rewards. For example, when foraging for food, animals must choose between actions that have varying costs (e.g., effort, duration, energy expenditure) and varying benefits (e.g., amount of food intake). The combination of these costs and benefits determine what optimal behavior is. In the present study, we employ a foraging-style task to study how humans make reward-based choices in response to the real-time constraints of a dynamic environment. On each trial participants were presented with two rewards that differed in magnitude and in the delay until their receipt. Because the experiment was of a fixed duration, maximizing earnings required decision makers to determine how to trade off the magnitude and the delay associated with the two rewards on each trial. To evaluate the extent to which participants could adapt to the decision environment, specific task characteristics were manipulated, including reward magnitudes (Experiment 1) and the delay between trials (Experiment 2). Each of these manipulations was designed to alter the pattern of choices made by an optimal decision maker. Several findings are of note. First, different choice strategies were observed with the manipulated environmental constraints. Second, despite contextually-appropriate shifts in behavior between conditions in each experiment, choice patterns deviated from theoretical optimality. In particular, the delays associated with the rewards did not exert a consistent influence on choices as required by exponential discounting. Third, decision makers nevertheless performed surprisingly well in all task environments with any deviations from strict optimality not having particularly deleterious effects on earnings. Taken together, these results suggest that human decision makers are capable of exhibiting intertemporal preferences that reflect a variety of environmental constraints.

Response factors in delay discounting: evidence for Pavlovian influences on delay discounting in pigeons

Pigeons completed a delay-discounting task where in different conditions the required response was either key pecking or treadle pressing. Because of stimulus-reinforcer relations that are known to form between localized visual cues and the delivery of food (e.g., autoshaping), we predicted that there would be steeper rates of discounting with key pecking than treadle pressing. To account for possible effort differences between key pecking and treadle pressing, pigeons also completed a discounting task where multiple key pecks were required to gain access to the food. The rates of discounting for the key peck and effort-equivalence discounting procedures were similar, and both were steeper than the rate of discounting for the treadle-pressing procedure. While it is tacitly assumed that behavior in choice situations is largely under the control of operant contingencies, the present results suggest that when developing animal analogs to study discounting in a discrete-trial choice procedure, the stimulus-reinforcer relations (Pavlovian conditioning) may need to be taken into consideration.

Magnitude effects for experienced rewards at short delays in the escalating interest task

A first-person shooter video game was adapted for the study of choice between smaller sooner and larger later rewards. Participants chose when to fire a weapon that increased in damage potential over a short interval. When the delay to maximum damage was shorter (5-8 s), people showed greater sensitivity to the consequences of their choices than when the delay was longer (17-20 s). Participants also evidenced a magnitude effect by waiting proportionally longer when the damage magnitudes were doubled for all rewards. The experiment replicated the standard magnitude effect with this new video game preparation over time scales similar to those typically used in nonhuman animal studies and without complications due to satiation or cost.

A concurrent-choice analysis of amount-dependent temporal discounting

Amount-dependent temporal discounting refers to the differential rates at which the values of large and small rewards are discounted over time. A lower rate of discounting of larger rewards is known as the magnitude effect. The present study aimed to establish a magnitude effect in humans using a concurrent-choice procedure. Participants indicated their strength of preference between hypothetical outcomes that differed in monetary value, and in the delay at which they were available. Most studies of temporal discounting measure preference using indifference points estimated from a titration procedure. The present study measured preference using a concurrent choice procedure. The main analysis demonstrated temporal discounting and a magnitude effect. Further analysis showed that the result was consistent with the matching law.

Preference reversal between impulsive and self-control choice

In a concurrent-chains procedure, pigeons chose between reinforcers varying in delay and amount. Reinforcer amount was determined by duration of access to grain, and delay was determined by fixed-interval schedules in the terminal links. Preference was measured by the ratio of responses in initial links. Dependent scheduling of variable-interval schedules in initial links ensured that delay and amount were not confounded with frequency of outcomes, which remained equal for the two choices. In Experiment 1, in components signaled by red keys in the initial links, small and large reinforcers were delivered after delays of 1 s and 10 s respectively. In components signaled by green, additional time was added to both delays. Smaller-sooner reinforcers were preferred in red components. In green components, smaller-sooner reinforcers were preferred at short delays, and choices for the larger-later reinforcer generally increased with increasing duration of the added delay. At longer delays, up to 15 s, the larger-later reinforcer was preferred. That is, the pigeons showed within-session preference reversal, with impulsive choice at short delays in red components and self-control choice at long delays in green components. In Experiment 2, added delay to both reinforcement and reinforcer amount were varied. Sensitivity of initial-link response ratios to ratios of amount increased with increasing duration of the added delay. This interaction between delay and amount was predicted if the temporal discounting functions assumed the magnitude effect in which discounting rate was inversely proportional to amount. It was also predicted by the contextual choice model of performance in concurrent-chains procedures.

Effect of orbitofrontal cortex lesions on temporal discounting in rats

Although choices of both humans and animals are more strongly influenced by immediate than delayed rewards, methodological limitations have made it difficult to estimate the precise form of temporal discounting in animals. In the present study, we sought to characterize temporal discounting in rats and to test the role of the orbitofrontal cortex (OFC) in this process. Rats were trained in a novel intertemporal choice task in which the sequence of delay durations was randomized across trials. The animals tended to choose a small immediate reward more frequently as the delay for a large reward increased, and, consistent with previous findings in other species, their choice behavior was better accounted for by hyperbolic than exponential discount functions. In addition, model comparisons showed that the animal's choice behavior was better accounted for by more complex discount functions with an additional parameter than a hyperbolic discount function. Following bilateral OFC lesions, rats extensively trained in this task showed no significant change in their intertemporal choice behavior. Our results suggest that the rodent OFC may not always play a role in temporal discounting when delays are randomized and/or after extensive training.

A quantitative analysis of the effects of qualitatively different reinforcers on fixed ratio responding in inbred strains of mice

Previous studies of inbred mouse strains have shown reinforcer-strain interactions that may potentially mask differences among strains in memory performance. The present research examined the effects of two qualitatively different reinforcers (heterogeneous mix of flavored pellets and sweetened-condensed milk) on responding maintained by fixed-ratio schedules of reinforcement in three inbred strains of mice (BALB/c, C57BL/6, and DBA/2). Responses rates for all strains were a bitonic (inverted U) function of the size of the fixed-ratio schedule and were generally higher when responding was maintained by milk. For the DBA/2 and C57BL/6 and to a lesser extent the BALB/c, milk primarily increased response rates at moderate fixed ratios, but not at the largest fixed ratios tested. A formal model of ratio-schedule performance, Mathematical Principles of Reinforcement (MPR), was applied to the response rate functions of individual mice. According to MPR, the differences in response rates maintained by pellets and milk were mostly due to changes in motoric processes as indicated by changes in the minimum response time (δ) produced by each reinforcer type and not specific activation (a), a model term that represents value and is correlated with reinforcer magnitude and the break point obtained under progressive ratio schedules. MPR also revealed that, although affected by reinforcer type, a parameter interpreted as the rate of saturation of working memory (λ), differed among the strains.

Linking behavioural syndromes and cognition: a behavioural ecology perspective

With the exception of a few model species, individual differences in cognition remain relatively unstudied in non-human animals. One intriguing possibility is that variation in cognition is functionally related to variation in personality. Here, we review some examples and present hypotheses on relationships between personality (or behavioural syndromes) and individual differences in cognitive style. Our hypotheses are based largely on a connection between fast-slow behavioural types (BTs; e.g. boldness, aggressiveness, exploration tendency) and cognitive speed-accuracy trade-offs. We also discuss connections between BTs, cognition and ecologically important aspects of decision-making, including sampling, impulsivity, risk sensitivity and choosiness. Finally, we introduce the notion of cognition syndromes, and apply ideas from theories on adaptive behavioural syndromes to generate predictions on cognition syndromes.

Rats (Rattus norvegicus) and pigeons (Columbia livia) are sensitive to the distance to food, but only rats request more food when distance increases

Three experiments investigated foraging by rats and pigeons. In Experiment 1, each response on a manipulandum delivered food to a cup, with the distance between the manipulandum and the cup varying across conditions. The number of responses made before traveling to collect and eat the food increased with distance for rats, but not for pigeons. In Experiment 2, two manipulanda were placed at different distances from a fixed food source; both pigeons and rats preferentially used the manipulandum closest to the food source. Experiment 3 was a systematic replication of Experiment 1 with pigeons. In different conditions, each peck on the left key increased the upcoming hopper duration by 0.5, 1.5 or 2.5s. Completing a ratio requirement on the right key of 1, 4, 8, 16 or 32 pecks, depending on the condition, then produced the food hopper for a duration that depended on the number of prior left pecks. As the ratio requirement increased on the right key, pigeons responded more on the left key and earned more food. Overall, the results replicate previous research, underlining similarities and differences between these species. The results are discussed in terms of optimal foraging, reinforcer sensitivity and delay discounting.

How costs influence decision values for mixed outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.

Delay discounting decreases in those completing treatment for opioid dependence

Several studies examining both control and substance-dependent populations have found delay discounting to remain stable over time. In this report, we examine whether delay discounting changes in opioid-dependent individuals who complete a 12-week treatment. The 159 subjects who completed discounting assessments at baseline and treatment-end come from two separate clinical trials: 56 from Chopra et al. (2009) and 103 from Christensen et al. (2012). Mean discounting at 12 weeks significantly decreased to less than half (44.8%) of the baseline level (95% CIs (27.5, 73.2)). Analyzing each subject's discounting data individually, over 3 times (95% CIs (1.9, 5.5)) as many subjects statistically decreased their discounting from their own baseline levels than those who exhibited a statistical increase. Though we failed to find any relationship among discounting measures and abstinence outcomes, the results from this large study suggest that treatment for substance dependence promotes decreases in delay discounting.

Victims' time discounting 2.5 years after the Wenchuan earthquake: an ERP study

BACKGROUND

Time discounting refers to the fact that the subjective value of a reward decreases as the delay until its occurrence increases. The present study investigated how time discounting has been affected in survivors of the magnitude-8.0 Wenchuan earthquake that occurred in China in 2008.

METHODOLOGY

Nineteen earthquake survivors and 22 controls, all school teachers, participated in the study. Event-related brain potentials (ERPs) for time discounting tasks involving gains and losses were acquired in both the victims and controls.

FINDINGS

The behavioral data replicated our previous findings that delayed gains were discounted more steeply after a disaster. ERP results revealed that the P200 and P300 amplitudes were increased in earthquake survivors. There was a significant group (earthquake vs. non-earthquake) × task (gain vs. loss) interaction for the N300 amplitude, with a marginally significantly reduced N300 for gain tasks in the experimental group, which may suggest a deficiency in inhibitory control for gains among victims.

CONCLUSIONS

The results suggest that post-disaster decisions might involve more emotional (System 1) and less rational thinking (System 2) in terms of a dual-process model of decision making. The implications for post-disaster intervention and management are also discussed.

Delay discounting in rhesus monkeys: equivalent discounting of more and less preferred sucrose concentrations

Humans discount larger amounts of a delayed reinforcer less steeply than smaller amounts, but studies with pigeons and rats have yet to reveal such a magnitude effect, suggesting that the effect may be unique to humans. The present study examined whether the magnitude effect is observed in a species phylogenetically closer to humans, by comparing the rates at which rhesus monkeys discounted 10% and 20% concentrations of sucrose. There were no systematic differences in the rates at which the monkeys discounted the two sucrose concentrations, despite the fact that they strongly preferred the 20% concentration. Interestingly, the monkeys discounted delayed sucrose at a rate higher than was observed with delayed cocaine, and lower than was observed with delayed saccharin in previous studies (Freeman et al. Behavioural Processes, 82, 214-218, 2009; Woolverton et al. Experimental and Clinical Psychopharmacology, 15, 238-244, 2007). Taken together, these findings suggest that although both quantitative and qualitative differences can affect monkeys' preferences between immediate reinforcers, qualitative differences between types of reinforcers (e.g., sucrose vs. cocaine) can affect monkeys' discounting rates in a way that quantitative differences within a reinforcer (e.g., 10% vs. 20% sucrose) do not.

Carrion crows cannot overcome impulsive choice in a quantitative exchange task

The ability to control an immediate impulse in return for a more desirable – though delayed – outcome has long been thought to be a uniquely human feature. However, studies on non-human primates revealed that some species are capable of enduring delays in order to get food of higher quality or quantity. Recently two corvid species, common raven (Corvus corax) and carrion crow (Corvus corone corone), exchanged food for a higher quality reward though seemed less capable of enduring delays when exchanging for the same food type in a higher quantity. In the present study, we specifically investigated the ability of carrion crows to overcome an impulsive choice in a quantitative exchange task. After a short delay, individuals were asked to give back an initial reward (cheese) to the human experimenter in order to receive a higher amount of the same reward (two, four, or eight pieces). We tested six captive crows – three individuals never exchanged the initial reward for a higher quantity; the other three birds did exchange though at very low rates. We performed a preference test between one or more pieces of cheese in order to address whether crow poor performance could be due to an inability to discriminate between different quantities or not attributing a higher value to the higher quantities. All birds chose the higher quantities significantly more often, indicating that they can discriminate between quantities and that higher quantities are more desirable. Taken together, these results suggest that, although crows may possess the cognitive abilities to judge quantities and to overcome an impulsive choice, they do so only in order to optimize the qualitative but not quantitative output in the exchange paradigm.

Suppression of cocaine self-administration in monkeys: effects of delayed punishment

RATIONALE

Delaying presentation of a drug can decrease its effectiveness as a reinforcer, but the effect of delaying punishment of drug self-administration is unknown.

OBJECTIVE

This study examined whether a histamine injection could punish cocaine self-administration in a drug-drug choice, whether delaying histamine would decrease its effectiveness, and whether the effects of delay could be described within a delay discounting framework.

METHODS

Monkeys were implanted with double-lumen catheters to allow separate injection of cocaine and histamine. In discrete trials, subjects first chose between cocaine (50 or 100 μg/kg/inj) alone and an injection of the same dose of cocaine followed immediately by an injection of histamine (0.37-50 μg/kg). Next, they chose between cocaine followed immediately by histamine and cocaine followed by an equal but delayed dose of histamine.

RESULTS

When choosing between cocaine alone and cocaine followed immediately by histamine, preference increased with histamine dose from indifference to >80% choice of cocaine alone. When choosing between cocaine followed by immediate histamine and cocaine followed by delayed histamine, monkeys showed strong position preferences. When delayed histamine was associated with the nonpreferred position, preference for that option increased with delay from ≤30% to >85%. The corresponding decrease in choice of the preferred position was well described by a hyperboloid discounting function.

CONCLUSIONS

Histamine can function as a punisher in the choice between injections of cocaine and delay can decrease its effectiveness as a punisher. The effects of delaying punishment of drug self-administration can be conceptualized within the delay discounting framework.

Effects of pre-trial response requirements on self-control choices by rats and pigeons

Parallel experiments with rats and pigeons examined whether the size of a pre-trial ratio requirement would affect choices in a self-control situation. In different conditions, either 1 response or 40 responses were required before each trial. In the first half of each experiment, an adjusting-ratio schedule was used, in which subjects could choose a fixed-ratio schedule leading to a small reinforcer, or an adjusting-ratio schedule leading to a larger reinforcer. The size of the adjusting ratio requirement was increased and decreased over trials based on the subject's responses, in order to estimate an indifference point-a ratio at which the two alternatives were chosen about equally often. The second half of each experiment used an adjusting-delay procedure-fixed and adjusting delays to the small and large reinforcers were used instead of ratio requirements. In some conditions, particularly with the reinforcer delays, the rats had consistently longer adjusting delays with the larger pre-trial ratios, reflecting a greater tendency to choose the larger, delayed reinforcer when more responding was required to reach the choice point. No consistent effects of the pre-trial ratio were found for the pigeons in any of the conditions. These results may indicate that rats are more sensitive to the long-term reinforcement rates of the two alternatives, or they may result from a shallower temporal discounting rate for rats than for pigeons, a difference that has been observed in previous studies.

Behavioral toxicology of cognition: extrapolation from experimental animal models to humans: behavioral toxicology symposium overview

A variety of behavioral instruments are available for assessing important aspects of cognition in both animals and humans and, in many cases, the same instruments can be used in both. While nonhuman primates are phylogenetically closest to humans, rodents, pigeons and other animals also offer behaviors worthy of note. Delay Discounting procedures are as useful as any in studies of impulsivity and may have utility in shedding light on processes associated with drug abuse. Specific memory tests such as Visual Paired Comparisons tasks (similar to the Fagan test of infant intelligence) can be modified to allow for assessment of different aspects of memory such as spatial memory. Use of these and other specific memory tasks can be used to directly monitor aspects of cognitive development in infant animals, particularly in nonhuman primates such as monkeys, and children and to draw inferences with respect to possible neuroanatomical substrates sub-serving their functions. Tasks for assessing working memory such as Variable Delayed Response (VDR), modified VDR and Spatial Working Memory tasks are now known to be affected in Parkinson's disease (PD). These and other cognitive function tasks are being used in a monkey model of PD to assess the ability of anti-Parkinson's disease therapies to ameliorate these cognitive deficits without diminishing their therapeutic effects on motor dysfunction. Similarly, in a rat model of the cognitive deficits associated with perinatal exposure to polychlorinated biphenyls (PCBs), clear parallels with children can be seen in at least two areas of executive function: cognitive flexibility and response inhibition. In the rat model, discrimination reversal tasks were utilized to assess cognitive flexibility, a function often assessed in humans using the Wisconsin Card Sorting Task. Response inhibition was assessed using performance in a Differential Reinforcement of Low Response Rates (DRL) task. As the data continue to accumulate, it becomes more clear that our attempts to adapt animal-appropriate tasks for the study of important aspects of human cognition have proven to be very fruitful.

Characterization of a semi-rapid method for assessing delay discounting in rodents

Delay discounting is a key component of many psychiatric disorders, including drug addiction, compulsive gambling, ADHD, and obesity. However, its underlying mechanisms are not yet fully characterized. One impediment to full characterization of such mechanisms is the fact that rodent models of the task are often complicated and involve extended training of subjects, often requiring more than a month before a stable baseline is obtained. We have therefore characterized a version of the rodent delay discounting task which generates data more quickly than most other published versions. In this version of the task, learning of the operant response is established prior to introduction of the delay component, and delay is tested across subsequent daily sessions with a single delay length per day. We demonstrate here that this version generates a delay discounting curve similar to many published tasks, and is sensitive to changes in reward magnitude and to chronic treatment with cocaine. Furthermore, we present a detailed description of the within-session patterns of behavior in the task, which provides evidence of within-session learning and establishment of stable response patterns. This faster version of the delay discounting task will facilitate future studies involving pharmacological, electrophysiological, and other mechanistic studies of the underlying basis of this important disease process.

Some observations from behavioral economics for consideration in promoting money management among those with substance use disorders

BACKGROUND

Behavioral economics research has revealed systematic biases in decision making that merit consideration in efforts to promote money management skills among those with substance use disorders (SUDs).

OBJECTIVES

The objective of this article was to briefly review the literature on five of those biases (i.e., hyperbolic delay discounting, defaults and preference for the status quo, loss aversion, mental accounting, and failure to account for opportunity cost) that may have particular relevance to the topic of money management.

METHODS

Selected studies are reviewed to illustrate these biases and how they may relate to efforts to promote money management skills among those with substance use disorders. Studies were identified by searching PubMed using the terms "behavioral economics" and "substance use disorders", reviewing bibliographies of published articles, and discussions with colleagues.

RESULTS

Only one of these biases (i.e., hyperbolic delay discounting) has been investigated extensively among those with SUDs. Indeed, it has been found to be sufficiently prevalent among those with SUDs to be considered as a potential risk factor for those disorders and certainly merits careful consideration in efforts to improve money management skills in that population. There has been relatively little empirical research reported regarding the other biases among those with SUDs, although they appear to be sufficiently fundamental to human behavior and relevant to the topic of money management (e.g., loss aversion) to also merit consideration. There is precedent of effective leveraging of behavioral economics principles in treatment development for SUDs (e.g., contingency management), including at least one intervention that explicitly focuses on money management (i.e., advisor-teller money management therapy).

CONCLUSIONS AND SCIENTIFIC SIGNIFICANCE

The consideration of the systematic biases in human decision making that have been revealed in behavioral economics research has the potential to enhance efforts to devise effective strategies for improving money management skills among those with SUDs.

Intertemporal choice in lemurs

Different species vary in their ability to wait for delayed rewards in intertemporal choice tasks. Models of rate maximization account for part of this variation, but other factors such as social structure and feeding ecology seem to underly some species differences. Though studies have evaluated intertemporal choice in several primate species, including Old World monkeys, New World monkeys, and apes, prosimians have not been tested. This study investigated intertemporal choices in three species of lemur (black-and-white ruffed lemurs, Varecia variegata, red ruffed lemurs, Varecia rubra, and black lemurs, Eulemur macaco) to assess how they compare to other primate species and whether their choices are consistent with rate maximization. We offered lemurs a choice between two food items available immediately and six food items available after a delay. We found that by adjusting the delay to the larger reward, the lemurs were indifferent between the two options at a mean delay of 17 s, ranging from 9 to 25 s. These data are comparable to data collected from common marmosets (Callithrix jacchus). The lemur data were not consistent with models of rate maximization. The addition of lemurs to the list of species tested in these tasks will help uncover the role of life history and socio-ecological factors influencing intertemporal choices.

A fruit in hand is worth many more in the bush: steep spatial discounting by free-ranging rhesus macaques (Macaca mulatta)

Decision making is one of the principal cognitive processes underlying goal-directed behaviour and thus there is justifiably strong interest in modeling it. However, many of these models have yet to be tested outside of the laboratory. At the same time, field work would benefit from the use of experimental methods developed in the laboratory to determine the causal relationships between environmental variables and behaviour. We therefore adapted a laboratory-derived experimental paradigm to test decision making in the wild. The experiment used an indifference-point procedure to determine the influence of both the amount and distance of food on choice behaviour. Free-ranging rhesus monkeys were given the choice between a smaller amount of food at a closer distance and a larger amount farther away. In four conditions, we held the closer amount constant across trials and varied the farther amount to determine the point at which the monkeys were indifferent to the choice alternatives. For example, in condition one, we used one piece of food at the closer location, and determined how many pieces would be equivalent in the farther location. Four different closer amounts were tested to obtain an indifference point curve, with the indifference amounts at the farther location plotted against the closer amounts. The slope of the obtained linear indifference curve was surprisingly high, suggesting that rhesus monkeys significantly discount food that is farther away. Possible reasons for this steep spatial discounting are discussed.

Future decision-making without episodic mental time travel

Deficits in episodic memory are associated with deficits in the ability to imagine future experiences (i.e., mental time travel). We show that K.C., a person with episodic amnesia and an inability to imagine future experiences, nonetheless systematically discounts the value of future rewards, and his discounting is within the range of controls in terms of both rate and consistency. Because K.C. is neither able to imagine personal uses for the rewards nor provide a rationale for selecting larger future rewards over smaller current rewards, this study demonstrates a dissociation between imagining and making decisions involving the future. Thus, although those capable of mental time travel may use it in making decisions about future rewards, these results demonstrate that it is not required for such decisions.

Acute nicotine increases both impulsive choice and behavioural disinhibition in rats

RATIONALE

Heavy smokers exhibit greater levels of impulsive choice and behavioural disinhibition than non-smokers. To date, however, the relationship between nicotine use and differing dimensions of impulsivity has not been systematically assessed.

OBJECTIVES

A series of studies was designed to assess the acute dose-response effects of nicotine and the nicotinic receptor antagonist mecamylamine alone, and in combination with nicotine, on impulsive choice and behavioural disinhibition in rats.

METHODS

Separate groups of rats were trained on a symmetrically reinforced go/no-go task to measure levels of disinhibition and a systematic delayed reward task to measure levels of impulsive choice. Once trained, all animals in each task were treated acutely with nicotine (0.125, 0.25, 0.5 and 1.0 mg/kg), mecamylamine (0.1, 0.3 and 1.0 mg/kg) and varying doses of mecamylamine (0.1, 0.3 and 1.0 mg/kg) prior to nicotine (0.5 mg/kg). An additional experiment assessed the effects of alterations in primary motivation (presatiation and fasting) on performance in both tasks.

RESULTS

Acute nicotine increased both impulsive choice and behavioural disinhibition, effects that were blocked by pre-treatment with mecamylamine. Mecamylamine when administered alone did not alter impulsive behaviour. The lack of effect of presatiation on performance measures suggests that the observed nicotine-induced impulsivity cannot be attributed to the anorectic activity of the compound.

CONCLUSIONS

Present findings support the hypothesis that heightened impulsivity in smokers may in part be a consequence of the direct acute effects of nicotine. As such, drug-induced changes in impulsivity may play a critical role in the transition to and maintenance of nicotine dependence.

Corvids can decide if a future exchange is worth waiting for

Evidence for time-dependent calculations about future rewards is scarce in non-human animals. In non-human primates, only great apes are comparable with humans. Still, some species wait for several minutes to obtain a better reward in delayed exchange tasks. Corvids have been shown to match with non-human primates in some time-related tasks. Here, we investigate a delay of gratification in two corvid species, the carrion crow (Corvus corone) and the common raven (Corvus corax), in an exchange task. Results show that corvids success decreases quickly as delay increases, with a maximal delay of up to 320 s (more than 5 min). The decision to wait rests both on the quality of the prospective reward and the time required to obtain it. Corvids also apply tactics (placing the reward on the ground or caching it) that probably alleviate costs of waiting and distract their attention during waiting. These findings contrast previous results on delayed gratification in birds and indicate that some species may perform comparably to primates.