Delay discounting of saccharin in rhesus monkeys

Neural dynamics underlying self-control in the primate subthalamic nucleus

The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost-benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.

Self-Controlled Choice Arises from Dynamic Prefrontal Signals That Enable Future Anticipation

Self-control allows humans the patience necessary to maximize reward attainment in the future. Yet it remains elusive when and how the preference to self-controlled choice is formed. We measured brain activity while female and male humans performed an intertemporal choice task in which they first received delayed real liquid rewards (forced-choice trial), and then made a choice between the reward options based on the experiences (free-choice trial). We found that, while subjects were awaiting an upcoming reward in the forced-choice trial, the anterior prefrontal cortex (aPFC) tracked a dynamic signal reflecting the pleasure of anticipating the future reward. Importantly, this prefrontal signal was specifically observed in self-controlled individuals, and moreover, interregional negative coupling between the prefrontal region and the ventral striatum (VS) became stronger in those individuals. During consumption of the liquid rewards, reduced ventral striatal activity predicted self-controlled choices in the subsequent free-choice trials. These results suggest that a well-coordinated prefrontal-striatal mechanism during the reward experience shapes preferences regarding the future self-controlled choice.SIGNIFICANCE STATEMENT Anticipating future desirable events is a critical mental function that guides self-controlled behavior in humans. When and how are the self-controlled choices formed in the brain? We monitored brain activity while humans awaited a real liquid reward that became available in tens of seconds. We found that the frontal polar cortex tracked temporally evolving signals reflecting the pleasure of anticipating the future reward, which was enhanced in self-controlled individuals. Our results highlight the contribution of the fronto-polar cortex to the formation of self-controlled preferences, and further suggest that future prospect in the prefrontal cortex (PFC) plays an important role in shaping future choice behavior.

Analyzing the magnitude effect in spontaneously hypertensive (SHR) and wistar Kyoto (WKY) rats

We analyzed the magnitude effect in Spontaneously Hypertensive (SHR) and Wistar Kyoto (WKY) rats choosing between a smaller-sooner (SSF) and a larger-later food (LLF) in the initial link of a concurrent-chains procedure. The SSF was delivered immediately in one terminal link and the LLF delayed 0.01, 5, 10, 20, 40, or 80 s in the other terminal link. An ABABA design varied food amount, 1 vs. 4 and 3 vs. 8 food-pellets in conditions A and B, respectively. The SHRs made more impulsive choices than the WKYs. The hyperbolic-decay model and the Generalized Matching Law fitted the data well. Discounting rate (k) and the area under the discounting curve (AUC) for the choices made by the SHRs in conditions A, were like those in conditions B. For the choices that the individual WKYs made, k was slightly higher and AUC smaller in conditions B than in conditions A. For both strains sensitivity to the immediacy of the LLF (s) was slightly higher in conditions A than in conditions B. Thus, we found no conclusive, compelling evidence either supporting or discarding the magnitude effect in the SHRs and scarce evidence supporting an effect opposite to the magnitude effect in the WKYs.

Stimulus control depends on the subjective value of the outcome

Stimuli that provide information about likely future reinforcers tend to shift behavior, provided a reliable relation between the stimulus and the reinforcer can be discriminated. Stimuli that are apparently more reliable exert greater control over behavior. We asked how the subjective value (measured in terms of preference) of reinforcers associated with stimuli influences stimulus control. Five pigeons worked on a concurrent chains procedure in which half of all trials ended in a smaller reinforcer sooner, and the other half in a larger reinforcer later. In Signaled trials, the color and flash duration on the keys in the initial link signaled the outcome of the trial. In Conflicting probe trials, the color and the flash duration signaled conflicting information about the outcome of the trial. Choice in Signaled trials shifted toward the signaled outcome, but was never exclusive. In Conflicting probe trials, control was divided idiosyncratically between the 2 stimulus dimensions, but still favored the outcome with the higher subjective value. Thus, stimulus control depends not only on the perceived reliability of stimuli, but also on the subjective value of the outcome.

The effect of economy type on demand and preference for cocaine and saccharin in rats

BACKGROUND

Economy type is an important determinant of reinforcer value. This study investigated the effect of open and closed economies on demand and preference for cocaine and saccharin in rats.

METHODS

In the first phase, rats were trained to lever press for cocaine infusions or saccharin. The number of presses required for each reinforcer increased across sessions. Cocaine and saccharin economy type was manipulated over groups by varying post-session availability of these reinforcers. One group of rats had three hours' post-session access to unlimited cocaine (open economy). A second group had three hours' post-session access to unlimited saccharin. A third group had no post-session access to either reinforcer (closed economy). In a second phase, rats in the three conditions could make mutually exclusive choices for cocaine or saccharin.

RESULTS

Post-session access to saccharin caused saccharin demand to become more elastic. Post-session access to cocaine had no effect on demand for cocaine but made demand for saccharin more elastic. Results from the choice phase generally paralleled those from the demand phase, the main finding being that post-session saccharin access caused an increase in cocaine preference.

CONCLUSIONS

These results show that manipulating economy type can affect cocaine and non-drug reinforcers differently. Opening the saccharin economy decreased saccharin's value. Opening the cocaine economy did not decrease cocaine's value, but instead led to a devaluation of saccharin. These results suggest that cocaine choice may be determined not only by the reinforcers immediately available, but also by those reinforcers' broader contexts of availability.

Pharmacotherapies for decreasing maladaptive choice in drug addiction: Targeting the behavior and the drug

Drug addiction can be conceptualized as a disorder of maladaptive decision making in which drugs are chosen at the expense of pro-social, nondrug alternatives. The study of decision making in drug addiction has focused largely on the role of impulsivity as a facilitator of addiction, in particular the tendency for drug abusers to choose small, immediate gains over larger but delayed outcomes (i.e., delay discounting). A parallel line of work, also focused on decision making in drug addiction, has focused on identifying the determinants underlying the choice to take drugs over nondrug alternatives (i.e., drug vs. nondrug choice). Both tracks of research have been valuable tools in the development of pharmacotherapies for treating maladaptive decision making in drug addiction, and a number of common drugs have been studied in both designs. However, we have observed that there is little uniformity in the administration regimens of potential treatments between the designs, which hinders congruence in the development of single treatment strategies to reduce both impulsive behavior and drug choice. The current review provides an overview of the drugs that have been tested in both delay-discounting and drug-choice designs, and focuses on drugs that reduced the maladaptive choice in both designs. Suggestions to enhance congruence between the findings in future studies are provided. Finally, we propose the use of a hybridized, experimental approach that may enable researchers to test the effectiveness of therapeutics at decreasing impulsive and drug choice in a single design.

Remifentanil maintains lower initial delayed nonmatching-to-sample accuracy compared to food pellets in male rhesus monkeys

Emerging human laboratory and preclinical drug self-administration data suggest that a history of contingent abused drug exposure impairs performance in operant discrimination procedures, such as delayed nonmatching-to-sample (DNMTS), that are hypothesized to assess components of executive function. However, these preclinical discrimination studies have exclusively used food as the reinforcer and the effects of drugs as reinforcers in these operant procedures are unknown. The present study determined effects of contingent intravenous remifentanil injections on DNMTS performance hypothesized to assess 1 aspect of executive function, working memory. Daily behavioral sessions consisted of 2 components with sequential intravenous remifentanil (0, 0.01-1.0 μg/kg/injection) or food (0, 1-10 pellets) availability in nonopioid dependent male rhesus monkeys (n = 3). Remifentanil functioned as a reinforcer in the DNMTS procedure. Similar delay-dependent DNMTS accuracy was observed under both remifentanil- and food-maintained components, such that higher accuracies were maintained at shorter (0.1-1.0 s) delays and lower accuracies approaching chance performance were maintained at longer (10-32 s) delays. Remifentanil maintained significantly lower initial DNMTS accuracy compared to food. Reinforcer magnitude was not an important determinant of DNMTS accuracy for either remifentanil or food. These results extend the range of experimental procedures under which drugs function as reinforcers. Furthermore, the selective remifentanil-induced decrease in initial DNMTS accuracy is consistent with a selective impairment of attentional, but not memorial, processes. (PsycINFO Database Record

Choice between variable and fixed cocaine injections in male rhesus monkeys

RATIONALE

The schedule of drug availability may enhance choice of a drug. In non-human subjects, reinforcers are chosen more often when available under variable schedules of reinforcement relative to fixed schedules.

OBJECTIVE

To determine whether variable-drug access is an important determinant of cocaine choice by manipulating the schedule, drug dose, and combination of schedule + dose.

METHOD

Four male rhesus monkeys chose between cocaine doses (0.025-0.4 mg/kg/injection). In control conditions, the schedule and dose of each drug delivery were fixed. In other conditions, the reinforcement schedule (i.e., variable-ratio schedule), dose of each cocaine delivery, or both were variable on one lever while all aspects on the other lever remained fixed.

RESULTS

When cocaine dose was equal on average (0.1 mg/kg/injection), 2 of 4 subjects chose cocaine associated with the variable schedule more than the fixed schedule. All subjects chose the variable dose that was equal on average to the fixed dose, and this difference was statistically significant. Three of 4 subjects chose cocaine associated with the variable combination over the fixed option (when the dose was equal on average). During dose-response determinations (when dose on the variable and fixed options were not equal), making the schedule, dose, or both variable generally did not alter cocaine's potency as a reinforcer.

CONCLUSION

While many factors contribute to drug choice, unpredictable drug access is a feature that may be common in the natural environment and could play a key role in the allocation of behavior to drug alternatives by patients with substance-use disorders.

Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty

We make choices based on the values of expected outcomes, informed by previous experience in similar settings. When the outcomes of our decisions consistently violate expectations, new learning is needed to maximize rewards. Yet not every surprising event indicates a meaningful change in the environment. Even when conditions are stable overall, outcomes of a single experience can still be unpredictable due to small fluctuations (i.e., expected uncertainty) in reward or costs. In the present work, we investigate causal contributions of the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) in rats to learning under expected outcome uncertainty in a novel delay-based task that incorporates both predictable fluctuations and directional shifts in outcome values. We demonstrate that OFC is required to accurately represent the distribution of wait times to stabilize choice preferences despite trial-by-trial fluctuations in outcomes, whereas BLA is necessary for the facilitation of learning in response to surprising events.

DOI:http://dx.doi.org/10.7554/eLife.27483.001

Nobody likes waiting – we opt for online shopping to avoid standing in lines, grow impatient in traffic, and often prefer restaurants that serve food quickly. When making decisions, humans and other animals try to maximize the benefits by weighing up the costs and rewards associated with a situation. Many regions in the brain help us choose the best options based on quality and size of rewards, and required waiting times. Even before we make decisions, the activity in these brain regions predicts what we will choose.

Sometimes, however, unexpected changes can lead to longer waiting times and our preferences suddenly become less desirable. The brain can detect such changes by comparing the outcomes we anticipate to those we experience. When the outcomes are surprising, specific areas in the brain such as the amygdala and the orbitofrontal cortex help us learn to make better choices. However, as surprising events can occur purely by chance, we need to be able to ignore irrelevant surprises and only learn from meaningful ones. Until now, it was not clear whether the amygdala and orbitofrontal cortex play specific roles in successfully learning under such conditions.

Stolyarova and Izquierdo trained rats to select between two images and rewarded them with sugar pellets after different delays. If rats chose one of these images they received the rewards after a predictable delay that was about 10 seconds, while choosing the other one produced variable delays – sometimes the time intervals were either very short or very long. Then, the waiting times for one of the alternatives changed unexpectedly. Rats with healthy brains quickly learned to choose the option with the shorter waiting time.

Stolyarova and Izquierdo repeated the experiments with rats that had damage in a part of the amygdala. These rats learned more slowly, particularly when the variable option changed for the better. Rats with damage to the orbitofrontal cortex failed to learn at all. Stolyarova and Izquierdo then examined the rats’ behavior during delays. Rats with damage to the orbitofrontal cortex could not distinguish between meaningful and irrelevant surprises and always looked for the food pellet (i.e. anticipated a reward) at the average delay interval.

These findings highlight two brain regions that help us distinguish meaningful surprises from irrelevant ones. A next step will be to examine how the amygdala and orbitofrontal cortex interact during learning and see if changes to the activity of these brain regions may affect responses. Advanced methods to non-invasively manipulate brain activity in humans may help people who find it hard to cope with changes; or individuals suffering from substance use disorders, who often struggle to give up drugs that provide them immediate and predictable rewards.

DOI:http://dx.doi.org/10.7554/eLife.27483.002

Delay discounting of the μ-opioid receptor agonist remifentanil in rhesus monkeys

Although increased impulsivity (delay discounting) is an important risk factor for drug abuse, the impact of delay on drug taking has received relatively little attention. This study examined delay discounting of the μ-opioid receptor agonist remifentanil in rhesus monkeys (n=4) responding for intravenous infusions under a concurrent choice procedure. Dose-effect curves for remifentanil were determined by varying the dose available on one lever (0.001-0.32 μg/kg/infusion) while keeping the dose available on the other lever (0.1 μg/kg/infusion) the same. Dose-effect curves were determined when both infusions were delivered immediately and when delivery of the fixed dose was delayed (15-180 s). When both doses of remifentanil were delivered immediately, monkeys chose the large dose. Delaying delivery of the fixed dose reduced choice of that dose and increased choice of small immediately available doses. Extending previous studies, these results show that the effects of delay on choice between two doses of a μ-opioid receptor agonist are consistent with hyperbolic discounting. Delaying delivery of a preferred reinforcer (e.g. large dose of drug) reduces its effectiveness and increases the effectiveness of small immediately available doses. This effect of delay, particularly on drug self-administration, might contribute to drug abuse.

Shallow discounting of delayed cocaine by male rhesus monkeys when immediate food is the choice alternative

Huskinson et al. (2015) recently examined delay discounting in monkeys choosing between an immediate drug (cocaine) reinforcer and a delayed nondrug (food) reinforcer. The present experiment examined the reverse situation: choice between immediate nondrug (food) and delayed drug (cocaine) reinforcers. Whereas the former choice situation exemplifies drug abuse from a delay-discounting perspective, our interest in the latter choice situation is derived from the observation that drug abusers, who characteristically are associated with impulsive choice, typically must devote considerable time to procuring drugs, often at the expense of immediate nondrug alternatives. Accordingly, we analyzed 3 male rhesus monkeys' choices between immediate food and delayed cocaine (0.1 and 0.2 mg/kg/injection) using a hyperbolic model that allowed us to compare discounting rates between qualitatively different reinforcers. Choice of immediate food increased with food amount, and choice functions generally shifted leftward as delay to cocaine increased, indicating a decrease in the subjective value of cocaine. Compared with our previous delay-discounting experiment with immediate cocaine versus delayed food, both doses of delayed cocaine were discounted at a shallow rate. The present results demonstrate that rhesus monkeys will tolerate relatively long delays in an immediate-food versus delayed-drug situation, suggesting that in intertemporal choices between cocaine and food, the subjective value of cocaine is less affected by the delay until reinforcement than is the subjective value of delayed food. More generally, the present findings suggest that although drug abusers may choose impulsively when immediate drug reinforcement is available, they exercise self-control in the acquisition of a highly preferred, delayed drug reinforcer. (PsycINFO Database Record

The absence of numbers to express the amount may affect delay discounting with humans

Human delay discounting is usually studied with experimental protocols that use symbols to express delay and amount. In order to further understand discounting, we evaluated whether the absence of numbers to represent reward amounts affects discount rate in general, and whether the magnitude effect is generalized to nonsymbolic situations in particular. In Experiment 1, human participants were exposed to a delay-discounting task in which rewards were presented using dots to represent monetary rewards (nonsymbolic); under this condition the magnitude effect did not occur. Nevertheless, the magnitude effect was observed when equivalent reward amounts were presented using numbers (symbolic). Moreover, in estimation tasks, magnitude increments produced underestimation of large amounts. In Experiment 2, participants were exposed only to the nonsymbolic discounting task and were required to estimate reward amounts in each trial. Consistent with Experiment 1, the absence of numbers representing reward amounts produced similar discount rates of small and large rewards. These results suggest that value of nonsymbolic rewards is a nonlinear function of amount and that value attribution depends on perceived difference between the immediate and the delayed nonsymbolic rewards.

The effect of reinforcer magnitude on probability and delay discounting of experienced outcomes in a computer game task in humans

Delay and uncertainty of receipt both reduce the subjective value of reinforcers. Delay has a greater impact on the subjective value of smaller reinforcers than of larger ones while the reverse is true for uncertainty. We investigated the effect of reinforcer magnitude on discounting of delayed and uncertain reinforcers using a novel approach: embedding relevant choices within a computer game. Participants made repeated choices between smaller, certain, immediate outcomes and larger, but delayed or uncertain outcomes while experiencing the result of each choice. Participants' choices were generally well described by the hyperbolic discounting function. Smaller numbers of points were discounted more steeply than larger numbers as a function of delay but not probability. The novel experiential choice task described is a promising approach to investigating both delay and probability discounting in humans.

Delay discounting: Pigeon, rat, human--does it matter?

Delay discounting refers to the decrease in subjective value of an outcome as the time to its receipt increases. Across species and situations, animals discount delayed rewards, and their discounting is well-described by a hyperboloid function. The current review begins with a comparison of discounting models and the procedures used to assess delay discounting in nonhuman animals. We next discuss the generality of discounting, reviewing the effects of different variables on the degree of discounting delayed reinforcers by nonhuman animals. Despite the many similarities in discounting observed between human and nonhuman animals, several differences have been proposed (e.g., the magnitude effect; nonhuman animals discount over a matter of seconds whereas humans report willing to wait months, if not years before receiving a reward), raising the possibility of fundamental species differences in intertemporal choice. After evaluating these differences, we discuss delay discounting from an adaptationist perspective. The pervasiveness of discounting across species and situations suggests it is a fundamental process underlying decision making.

Delay discounting of food by rhesus monkeys: Cocaine and food choice in isomorphic and allomorphic situations

Research on delay discounting has focused largely on nondrug reinforcers in an isomorphic context in which choice is between alternatives that involve the same type of reinforcer. Less often, delay discounting has been studied with drug reinforcers in a more ecologically valid allomorphic context where choice is between alternatives involving different types of reinforcers. The present experiment is the first to examine discounting of drug and nondrug reinforcers in both isomorphic and allomorphic situations using a theoretical model (i.e., the hyperbolic discounting function) that allows for comparisons of discounting rates between reinforcer types and amounts. The goal of the current experiment was to examine discounting of a delayed, nondrug reinforcer (food) by male rhesus monkeys when the immediate alternative was either food (isomorphic situation) or cocaine (allomorphic situation). In addition, we sought to determine whether there was a magnitude effect with delayed food in the allomorphic situation. Choice of immediate food and immediate cocaine increased with amount and dose, respectively. Choice functions for immediate food and cocaine generally shifted leftward as delay increased. Compared to isomorphic situations in which food was the immediate alternative, delayed food was discounted more steeply in allomorphic situations where cocaine was the immediate alternative. Notably, discounting was not affected by the magnitude of the delayed reinforcer. These data indicate that how steeply a delayed nondrug reinforcer is discounted may depend more on the qualitative characteristics of the immediate reinforcer and less on the magnitude of the delayed one.

Relatively high motivation for context-evoked reward produces the magnitude effect in rats

Using a concurrent-chain schedule, we demonstrated the effect of absolute reinforcement (i.e., the magnitude effect) on choice behavior in rats. In general, animals' simultaneous choices conform to a relative reinforcement ratio between alternatives. However, studies in pigeons and rats have found that on a concurrent-chain schedule, the overall reinforcement ratio, or absolute amount, also influences choice. The effect of reinforcement amount has also been studied in inter-temporal choice situations, and this effect has been referred to as the magnitude effect. The magnitude effect has been observed in humans under various conditions, but little research has assessed it in animals (e.g., pigeons and rats). The present study confirmed the effect of reinforcement amount in rats during simultaneous and inter-temporal choice situations. We used a concurrent-chain procedure to examine the cause of the magnitude effect during inter-temporal choice. Our results suggest that rats can use differences in reinforcement amount as a contextual cue during choice, and the direction of the magnitude effect in rats might be similar to humans when using the present procedure. Furthermore, our results indicate that the magnitude effect was caused by the initial-link effect when the reinforcement amount was relatively small, while a loss aversion tendency was observed when the reinforcement amount changed within a session. The emergence of the initial-link effect and loss aversion suggests that rats make choices through cognitive processes predicted by prospect theory.

Working and waiting for better rewards: self-control in two monkey species (Cebus apella and Macaca mulatta)

Self-control is typically defined as choosing a greater, delayed reward over a lesser, more immediate reward. However, in nature, there are other costs besides delay associated with obtaining the greatest outcome including increased effort, potential punishment, and low probability of reward. Effort is an interesting case because it sometimes impairs self-control, by acting as an additional cost, and at other times facilitates self-control, by distracting one from impulsive options. Additionally, different species may perform differently in effortful self-control tasks, based on their natural ecology. To gain insight into these aspects of self-control behavior, we examined capuchin monkeys' and rhesus monkeys' self-control in separate working and waiting choice tasks. We hypothesized that capuchins would show greater self-control in the working task, given their naturally higher activity level, whereas rhesus would perform similarly in both tasks. Rhesus performed as predicted, whereas contrary to our hypothesis, capuchins exhibited lesser performance in the working task. Nonetheless, these results may still stem from inherent species differences interacting with details of the methodology. Capuchins, being highly energetic and social monkeys, may have divided their energy and attention between the working task and other elements of the test environment such as visible group mates or manipulanda.

Decision-making strategies: ignored to the detriment of healthcare training and delivery?

Context: People do not always make health-related decisions which reflect their best interest – best interest being defined as the decision they would make if they carefully considered the options and fully understood the information available. A substantial literature has developed in behavioral economics and social psychology that seeks to elucidate the patterns in individual decision-making. While this is particularly relevant to healthcare, the insights from these fields have only been applied in a limited way. To address the health challenges of the twenty-first century, healthcare providers and healthcare systems designers need to more fully understand how individuals are making decisions. Methods: We provide an overview of the theories of behavioral economics and social psychology that relate to how individuals make health-related decisions. The concentration on health-related decisions leads to a focus on three topics: (1) mental shortcuts and motivated reasoning; (2) implications of time; and (3) implications of affect. The first topic is relevant because health-related decisions are often made in a hurry without a full appreciation of the implications and the deliberation they warrant. The second topic is included because the link between a decision and its health-related outcomes can involve a significant time lag. The final topic is included because health and affect are so often linked. Findings: The literature reviewed has implications for healthcare training and delivery. Selection for medical training must consider the skills necessary to understand and adapt to how patients make decisions. Training on the insights garnered from behavioral economics and social psychology would better prepare healthcare providers to effectively support their clients to lead healthy lives. Healthcare delivery should be structured to respond to the way in which decisions are made. Conclusions: These patterns in decision-making call into question basic assumptions our healthcare system makes about the best way to treat patients and deliver care. This literature has implications for the way we train physicians and deliver care.

A systematic assessment of delay discounting in relation to cocaine and nicotine dependence

Delay discounting is a measure of impulsivity describing how a reinforcer loses value as the delay to its receipt increases. Greater delay discounting is reliably observed among those with different substance use disorders (SUDs) compared to the general population. Nevertheless, the relation between delay discounting and the type and number of substances used remains unclear. The aim of this study was to compare delay discounting across four groups of participants: cocaine- and nicotine-dependent participants, cocaine-dependent only participants, nicotine-dependent only participants, and non-dependent controls. One hundred and seven participants completed a computerized delay discounting task for hypothetical monetary values. Data were fit to Mazur's hyperbolic equation to derive the discounting rate k. Results showed that delay discounting was significantly greater in the cocaine- and nicotine-dependent group, compared to the nicotine-dependent only group, compared to control group. Delay discounting was also greater in the cocaine-dependent only group relative to the nicotine-dependent only and control groups, but no differences were observed between the cocaine- and nicotine-dependent group and the cocaine-dependent only group. This study provides evidence that delay discounting differs depending on the type of SUD but not on the number of SUDs.

Response factors in delay discounting: evidence for Pavlovian influences on delay discounting in pigeons

Pigeons completed a delay-discounting task where in different conditions the required response was either key pecking or treadle pressing. Because of stimulus-reinforcer relations that are known to form between localized visual cues and the delivery of food (e.g., autoshaping), we predicted that there would be steeper rates of discounting with key pecking than treadle pressing. To account for possible effort differences between key pecking and treadle pressing, pigeons also completed a discounting task where multiple key pecks were required to gain access to the food. The rates of discounting for the key peck and effort-equivalence discounting procedures were similar, and both were steeper than the rate of discounting for the treadle-pressing procedure. While it is tacitly assumed that behavior in choice situations is largely under the control of operant contingencies, the present results suggest that when developing animal analogs to study discounting in a discrete-trial choice procedure, the stimulus-reinforcer relations (Pavlovian conditioning) may need to be taken into consideration.

A concurrent-choice analysis of amount-dependent temporal discounting

Amount-dependent temporal discounting refers to the differential rates at which the values of large and small rewards are discounted over time. A lower rate of discounting of larger rewards is known as the magnitude effect. The present study aimed to establish a magnitude effect in humans using a concurrent-choice procedure. Participants indicated their strength of preference between hypothetical outcomes that differed in monetary value, and in the delay at which they were available. Most studies of temporal discounting measure preference using indifference points estimated from a titration procedure. The present study measured preference using a concurrent choice procedure. The main analysis demonstrated temporal discounting and a magnitude effect. Further analysis showed that the result was consistent with the matching law.

Impulsivity and drugs of abuse: a juice-reinforced operant procedure for determining within-session delay discounting functions in rhesus monkeys

INTRODUCTION

Impulsivity is a behavioral trait that is thought to contribute to a variety of disorders, including drug abuse. Efficient, sensitive procedures are needed for studying drug effects on impulsivity (e.g., delay discounting) in nonhumans.

METHODS

Three monkeys responded under an operant choice procedure whereby responses on one lever resulted in immediate delivery of 0.15 ml of juice [Hawaiian Punch®] and responses on another lever resulted in delivery of 0.75 ml of juice, either immediately or after a delay (3.75-40 s). The delay to the larger reinforcer increased within-session across discrete blocks allowing for generation of delay-discounting functions within sessions.

RESULTS

Without delay, monkeys chose the larger reinforcer nearly exclusively. With increasing delay, monkeys progressively switched their choice from the larger to the smaller reinforcer in a delay-dependent manner. In 2 monkeys, acute administration of morphine (0.1 and 0.32 mg/kg) or ketamine (0.1 and 0.32 mg/kg) but not diazepam (0.1-1.0 mg/kg) dose-dependently shifted the delay-discounting functions to the left, indicating increased discounting. In one monkey, daily morphine treatment (0.32 mg/kg/day, 3-hour pretreatment) produced a rapid, sustained leftward shift in the delay-discounting function; the curve returned to the pre-drug position 6 days after discontinuation of treatment.

DISCUSSION

This delay discounting procedure is sensitive to both behavioral and pharmacological manipulations and appears to be particularly sensitive to chronic drug treatment and drug withdrawal. Given the importance of drug dependence and withdrawal in the initiation, maintenance of, and relapse to drug taking, this procedure should be useful to study one aspect of this process.

Delay discounting in rhesus monkeys: equivalent discounting of more and less preferred sucrose concentrations

Humans discount larger amounts of a delayed reinforcer less steeply than smaller amounts, but studies with pigeons and rats have yet to reveal such a magnitude effect, suggesting that the effect may be unique to humans. The present study examined whether the magnitude effect is observed in a species phylogenetically closer to humans, by comparing the rates at which rhesus monkeys discounted 10% and 20% concentrations of sucrose. There were no systematic differences in the rates at which the monkeys discounted the two sucrose concentrations, despite the fact that they strongly preferred the 20% concentration. Interestingly, the monkeys discounted delayed sucrose at a rate higher than was observed with delayed cocaine, and lower than was observed with delayed saccharin in previous studies (Freeman et al. Behavioural Processes, 82, 214-218, 2009; Woolverton et al. Experimental and Clinical Psychopharmacology, 15, 238-244, 2007). Taken together, these findings suggest that although both quantitative and qualitative differences can affect monkeys' preferences between immediate reinforcers, qualitative differences between types of reinforcers (e.g., sucrose vs. cocaine) can affect monkeys' discounting rates in a way that quantitative differences within a reinforcer (e.g., 10% vs. 20% sucrose) do not.

Suppression of cocaine self-administration in monkeys: effects of delayed punishment

RATIONALE

Delaying presentation of a drug can decrease its effectiveness as a reinforcer, but the effect of delaying punishment of drug self-administration is unknown.

OBJECTIVE

This study examined whether a histamine injection could punish cocaine self-administration in a drug-drug choice, whether delaying histamine would decrease its effectiveness, and whether the effects of delay could be described within a delay discounting framework.

METHODS

Monkeys were implanted with double-lumen catheters to allow separate injection of cocaine and histamine. In discrete trials, subjects first chose between cocaine (50 or 100 μg/kg/inj) alone and an injection of the same dose of cocaine followed immediately by an injection of histamine (0.37-50 μg/kg). Next, they chose between cocaine followed immediately by histamine and cocaine followed by an equal but delayed dose of histamine.

RESULTS

When choosing between cocaine alone and cocaine followed immediately by histamine, preference increased with histamine dose from indifference to >80% choice of cocaine alone. When choosing between cocaine followed by immediate histamine and cocaine followed by delayed histamine, monkeys showed strong position preferences. When delayed histamine was associated with the nonpreferred position, preference for that option increased with delay from ≤30% to >85%. The corresponding decrease in choice of the preferred position was well described by a hyperboloid discounting function.

CONCLUSIONS

Histamine can function as a punisher in the choice between injections of cocaine and delay can decrease its effectiveness as a punisher. The effects of delaying punishment of drug self-administration can be conceptualized within the delay discounting framework.

The orbitofrontal cortex and response selection

Orbitofrontal cortex (OFC) function is often characterized in terms of stimulus-reward mapping; however, more recent evidence suggests that the OFC may play a role in selecting and representing extended actions. First, previously encoded reward associations in the OFC could be used to inform responding in novel but similar situations. Second, when evaluated in tasks requiring the animal to perform extended actions, response selective activity can be recorded in the OFC. Finally, the interaction between the OFC and hippocampus illustrates OFC's role in response selection. The OFC may facilitate reward-guided memory retrieval by selecting the memories most relevant to achieve a goal. This model for OFC function places it within the hierarchy of increasingly complex action representations that support decision making.

Behavioral toxicology of cognition: extrapolation from experimental animal models to humans: behavioral toxicology symposium overview

A variety of behavioral instruments are available for assessing important aspects of cognition in both animals and humans and, in many cases, the same instruments can be used in both. While nonhuman primates are phylogenetically closest to humans, rodents, pigeons and other animals also offer behaviors worthy of note. Delay Discounting procedures are as useful as any in studies of impulsivity and may have utility in shedding light on processes associated with drug abuse. Specific memory tests such as Visual Paired Comparisons tasks (similar to the Fagan test of infant intelligence) can be modified to allow for assessment of different aspects of memory such as spatial memory. Use of these and other specific memory tasks can be used to directly monitor aspects of cognitive development in infant animals, particularly in nonhuman primates such as monkeys, and children and to draw inferences with respect to possible neuroanatomical substrates sub-serving their functions. Tasks for assessing working memory such as Variable Delayed Response (VDR), modified VDR and Spatial Working Memory tasks are now known to be affected in Parkinson's disease (PD). These and other cognitive function tasks are being used in a monkey model of PD to assess the ability of anti-Parkinson's disease therapies to ameliorate these cognitive deficits without diminishing their therapeutic effects on motor dysfunction. Similarly, in a rat model of the cognitive deficits associated with perinatal exposure to polychlorinated biphenyls (PCBs), clear parallels with children can be seen in at least two areas of executive function: cognitive flexibility and response inhibition. In the rat model, discrimination reversal tasks were utilized to assess cognitive flexibility, a function often assessed in humans using the Wisconsin Card Sorting Task. Response inhibition was assessed using performance in a Differential Reinforcement of Low Response Rates (DRL) task. As the data continue to accumulate, it becomes more clear that our attempts to adapt animal-appropriate tasks for the study of important aspects of human cognition have proven to be very fruitful.

Future decision-making without episodic mental time travel

Deficits in episodic memory are associated with deficits in the ability to imagine future experiences (i.e., mental time travel). We show that K.C., a person with episodic amnesia and an inability to imagine future experiences, nonetheless systematically discounts the value of future rewards, and his discounting is within the range of controls in terms of both rate and consistency. Because K.C. is neither able to imagine personal uses for the rewards nor provide a rationale for selecting larger future rewards over smaller current rewards, this study demonstrates a dissociation between imagining and making decisions involving the future. Thus, although those capable of mental time travel may use it in making decisions about future rewards, these results demonstrate that it is not required for such decisions.

Discounting in Pigeons When the Choice is between Two Delayed Rewards: Implications for Species Comparisons

Studies of delay discounting typically have involved choices between smaller, immediate outcomes and larger, delayed outcomes. In a study of delay discounting in humans, Green et al. (2005) added a period of time prior to both outcomes, creating a delay common to both. They found that the subjective value of the more delayed reward was well described by a hyperboloid discounting function and that the degree to which that outcome was discounted decreased as the common delay increased. In two experiments, we examined the effect of adding a common delay on the discounting of food rewards in pigeons. In Experiment 1, an adjusting-amount procedure was used to establish discounting functions when the common delay was 0, 3, 5, and 10 s, and different stimuli signaled time to the smaller, sooner and larger, later rewards. In contrast to humans, the pigeons showed increases in the degree of discounting when a common delay was added. In Experiment 2, the delay common to both rewards and the delay unique to the larger, later reward were each specifically signaled. With this procedure, the degree of discounting decreased as the common delay increased, a result consistent with that obtained with humans (Green et al., 2005). These findings reveal fundamental similarities between pigeons' and humans' choice behavior, and provide strong interspecies support for the hypothesis that choice between delayed outcomes is based on comparison of their hyperbolically discounted present subjective values.

Decision salience signals in posterior cingulate cortex

Despite its phylogenetic antiquity and clinical importance, the posterior cingulate cortex (CGp) remains an enigmatic nexus of attention, memory, motivation, and decision making. Here we show that CGp neurons track decision salience – the degree to which an option differs from a standard – but not the subjective value of a decision. To do this, we recorded the spiking activity of CGp neurons in monkeys choosing between options varying in reward-related risk, delay to reward, and social outcomes, each of which varied in level of decision salience. Firing rates were higher when monkeys chose the risky option, consistent with their risk-seeking preferences, but were also higher when monkeys chose the delayed and social options, contradicting their preferences. Thus, across decision contexts, neuronal activity was uncorrelated with how much monkeys valued a given option, as inferred from choice. Instead, neuronal activity signaled the deviation of the chosen option from the standard, independently of how it differed. The observed decision salience signals suggest a role for CGp in the flexible allocation of neural resources to motivationally significant information, akin to the role of attention in selective processing of sensory inputs.

Delay discounting of qualitatively different reinforcers in rats

Humans discount larger delayed rewards less steeply than smaller rewards, whereas no such magnitude effect has been observed in rats (and pigeons). It remains possible that rats' discounting is sensitive to differences in the quality of the delayed reinforcer even though it is not sensitive to amount. To evaluate this possibility, Experiment 1 examined discounting of qualitatively different food reinforcers: highly preferred versus nonpreferred food pellets. Similarly, Experiment 2 examined discounting of highly preferred versus nonpreferred liquid reinforcers. In both experiments, an adjusting-amount procedure was used to determine the amount of immediate reinforcer that was judged to be of equal subjective value to the delayed reinforcer. The amount and quality of the delayed reinforcer were varied across conditions. Discounting was well described by a hyperbolic function, but no systematic effects of the quantity or the quality of the delayed reinforcer were observed.

Behavioral characterization of adult male and female rhesus monkeys exposed to cocaine throughout gestation

RATIONALE

In utero cocaine exposure has been associated with alterations in the dopamine (DA) system in monkeys. However, the behavioral outcomes of prenatal cocaine exposure in adulthood are poorly understood.

OBJECTIVES

To assess several behavioral measures in 14-year-old rhesus monkeys exposed to cocaine in utero and controls (n = 10 per group).

MATERIALS AND METHODS

For these studies, two unconditioned behavioral tasks, novel object reactivity and locomotor activity, and two conditioned behavioral tasks, response extinction and delay discounting, were examined. In addition, cerebrospinal fluid (CSF) samples were analyzed for concentrations of the monoamine metabolites homovanillic acid (HVA) and 5-hydroxyindole acetic acid (5-HIAA).

RESULTS

No differences in CSF concentrations of 5-HIAA and HVA, latencies to touch a novel object or locomotor activity measures were observed between groups or sexes. However, prenatally cocaine-exposed monkeys required a significantly greater number of sessions to reach criteria for extinction of food-reinforced behavior than control monkeys. On the delay-discounting task, male prenatally cocaine-exposed monkeys switched preference from the larger reinforcer to the smaller one at shorter delay values than male control monkeys; no differences were observed in females.

CONCLUSIONS

These findings suggest that prenatal cocaine exposure results in long-term neurobehavioral deficits that are influenced by sex of the individual.

Effects of time between trials on rats' and pigeons' choices with probabilistic delayed reinforcers

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.

Within-subject comparison of degree of delay discounting using titrating and fixed sequence procedures

Different procedures are often used across experiments to estimate the degree of delay discounting, a common measure of impulsivity. In all procedures, participants indicate their choice between a reward available immediately and one available after a delay. The present experiment determined whether there are differences in the degree of discounting for a hypothetical $100 produced by a procedure that titrates the immediate amount (titrating sequence procedure) versus a procedure that presents a fixed sequence of immediate amounts (fixed sequence procedure) using a within-subject design. The adult human participants showed no significant differences in degree of discounting between procedures as assessed by a hyperboloid model and the Area Under the Curve. Furthermore, the Area Under the Curve values from the two procedures showed a strong positive correlation. These findings suggest there may be no systematic difference between the degree of delay discounting as estimated by the titrating sequence and fixed sequence procedures. Given the apparent similarities in the results, it appears researchers may be justified in basing their choice of which procedure to use on convenience.

Are people really more patient than other animals? Evidence from human discounting of real liquid rewards

In previous studies, researchers have found that humans discount delayed rewards orders of magnitude less steeply than do other animals. Humans also discount smaller delayed reward amounts more steeply than larger amounts, whereas animals apparently do not. These differences between humans and animals might reflect differences in the types of rewards studied and/or the fact that animals actually had to wait for their rewards. In the present article, we report the results of three experiments in which people made choices involving liquid rewards delivered and consumed after actual delays, thereby bridging the gap between animal and human studies. Under these circumstances, humans, like animals, discounted the value of rewards delayed by seconds; however, unlike animals, they still showed an effect of reward amount. Human discounting was well described by the same hyperboloid function that has previously been shown to describe animal discounting of delayed food and water rewards, as well as human discounting of real and hypothetical monetary rewards.