Intertrial interval duration and impulsive choice

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Timing and delay discounting in attention-deficit/hyperactivity disorder: A translational approach

Attention-deficit/hyperactivity disorder (ADHD) is a common neurodevelopmental disorder that often presents with abnormal time perception and increased impulsive choice behavior. The spontaneously hypertensive rat (SHR) is the most widely used preclinical model of the ADHD-Combined and ADHD-Hyperactive/Impulsive subtypes of the disorder. However, when testing the spontaneously hypertensive rat from Charles River (SHR/NCrl) on timing and impulsive choice tasks, the appropriate control strain is not clear, and it is possible that one of the possible control strains, the Wistar Kyoto from Charles River (WKY/NCrl), is an appropriate model for ADHD-Predominately Inattentive. Our goals were to test the SHR/NCrl, WKY/NCrl, and Wistar (WI; the progenitor strain for the SHR/NCrl and WKY/NCrl) strains on time perception and impulsive choice tasks to assess the validity of SHR/NCrl and WKY/NCrl as models of ADHD, and the validity of the WI strain as a control. We also sought to assess impulsive choice behavior in humans diagnosed with the three subtypes of ADHD and compare them with our findings from the preclinical models. We found SHR/NCrl rats timed faster and were more impulsive than WKY/NCrl and WI rats, and human participants diagnosed with ADHD were more impulsive compared to controls, but there were no differences between the three ADHD subtypes.

"Distractor" effects in delay discounting of probability by pigeons

Impulsive behavior can be measured by performance on a successive delay-discounting task, in which a response to a stimulus provides a small reinforcer sooner (SS), but in the absence of a response, a larger reinforcer later (LL). Previous research suggests that the presence of a concurrent "distractor" stimulus, to which responding has no programed consequence, can result in increased LL reinforcers. In the present experiments, we used differences in the probability of reinforcement between SS and LL (rather than magnitude of reinforcement) and tested the hypothesis that the concurrent stimulus may become a Pavlovian conditioned stimulus. For the Red-Only group, a response to the SS stimulus resulted in a reinforcer with a low probability (SS), whereas the absence of a response resulted in a reinforcer with a high probability (LL). For the Red-Green group, (analogous to the more typical simultaneous choice between an SS and LL stimulus) the absence of a response to the SS stimulus replaced the SS stimulus with the LL stimulus and a response to the LL stimulus resulted in the reinforcer. Thus, for the Red-Green group, the concurrent stimulus should have been less effective because responding to the concurrent stimulus was not immediately followed by the reinforcer. In Experiment 1, the concurrent stimulus was a yellow key-light; in Experiment 2, it was a houselight. In both experiments, the concurrent stimulus was effective in increasing the number of LL reinforcers and the effect was larger for the Red-Only group than for the Red-Green group.

Variations on a theme by Rachlin: Probability discounting

Rachlin and colleagues laid the groundwork for treating the discounting of probabilistic goods as a variant of the discounting of delayed goods. This approach was seminal for a large body of subsequent research. The present paper finds the original development problematic: In converting probability to delay, the authors incorrectly dropped trial duration. The subsumption of probability by delay is also empirically questionable, as those are different functions of variables such as magnitude of outcome and commodity versus money. A variant of Rachlin's theme treats human discounting studies as psychophysical matching experiments, in which one compound stimulus is adjusted to equal another. It is assumed that a function of amount (its utility) is multiplied by a function of probability (its weight). Conjoint measurement establishes the nature of these functions, yielding a logarithmic transform on amount, and a Prelec function on probability. This model provides a good and parsimonious account of probability discounting in diverse data sets. Variant representations of the data are explored. By inserting the probabilistically discounted utility into the additive utility theory of delay discounting, a general theory of probabilistic intertemporal choice is achieved.

The irrelevancy of the inter-trial interval in delay-discounting experiments on an animal model of ADHD

Delay discounting involves choosing between a small, immediate reward, and a larger but delayed one. As the delay between choice and large reward gets longer, people with ADHD tend to become impulsive faster than controls, indicated by a switch in preference from the large to the smaller reward. Choosing the smaller reward when the larger is considered reward maximizing is labeled impulsive behaviour. It is well documented that increased delays between choice and reward affects choice preference in both humans and other animals. Other variables such as the inter-trial interval or trial length are observed to have an effect on human discounting, but their effect on discounting in other animals is largely assumed rather than tested. In the current experiment, we tested this assumption. One group of rats was exposed to increasing delays between choosing the large reward and receiving it, while another group experienced longer inter-trial intervals that were equal in length to the delays in the other group. This ensured that trial length was controlled for in delay discounting, but that the delay function and inter-trial intervals could be manipulated and measured separately. Results showed that while the delay between choice and reward caused impulsive behaviour in rats, the length of the inter-trial interval (and by extension trial length) had no impact on choice behaviour. A follow-up experiment found this to be the case even if the length of the inter-trial interval was signaled with audio cues. These results suggest that rats, and possibly animals in general, are insensitive to time between trials, and therefore cannot easily represent human counterparts on the task.

Some theoretical notes on spatial discounting

Spatial discounting is a largely underexplored area of decision-making research, both theoretically and empirically, especially when compared to intertemporal choice, which has received significant attention in psychology and animal behaviour. Spatial decision problems seem to share some of the same features of a temporal decision problem (namely, the risk of reward objects disappearing and the opportunity cost of waiting), but there are several additional factors that affect the appropriate discount function for distant rewards. These include more significant opportunity costs, changes in the distances to all the other available opportunities, the post-reward costs of getting back home, the complex energetics associated with locomotion and all the additional risks faced by travelling itself. This paper organises and explores these factors and suggests some normative models that should predict the adaptive behaviour of animals and humans.

Sex differences in nicotine-induced impulsivity and its reversal with bupropion in rats

BACKGROUND

Enhancement in cognitive impulsivity and the resulting alterations in decision making serve as a contributing factor for the development and maintenance of substance-use disorders. Nicotine-induced increases in impulsivity has been previously reported in male humans and rodents. Although the potential for sex differences in nicotine-induced impulsivity has not been examined.

AIMS AND METHODS

In the present study, male and female Sprague Dawley rats were submitted to a delay discounting task, in which several consecutive measures of self-control were taken. Firstly, rats were tested with vehicle, and next with nicotine doses of 0.4 and 0.8 mg/kg. Thereafter, chronic treatment with bupropion started, and the animals were tested again. Half the animals continued to receive 0.8 mg/kg of nicotine, while the rest received nicotine and also a daily dose of 30 mg/kg of bupropion.

RESULTS

When the animals were first tested with nicotine, female rats showed a significant nicotine dose dependent increase of impulsive behaviour, whereas male rats only showed a decrease on their elections of the larger but delayed reward under the highest dose of 0.8 mg/kg of nicotine. Treatment with bupropion blocked the effect of nicotine on decision making in female rats, as they showed results close to their baseline levels. On the other hand, bupropion did not affect the nicotine-induced delay discounting in male rats.

CONCLUSION

These findings demonstrate sexually dimorphic effects of nicotine on cognitive impulsivity which may help to shed light on nicotine use vulnerabilities observed in women.

An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals

Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.

Serotonin (5-HT) plays many important roles in reward, punishment, patience and beyond, and optogenetic stimulation of 5-HT neurons has not crisply parsed them. The authors report a novel analysis of a reward-based decision-making experiment, and show that 5-HT stimulation increases the learning rate, but only on a select subset of choices.

The role of temporal intervals on reinforcer accumulation

Animals accumulate reinforcers when they forgo the opportunity to consume available food in favor of acquiring additional food for later consumption. Laboratory research has shown that reinforcer accumulation is facilitated when an interval (either spatial or temporal) separates earning from consuming reinforcers. However, there has been no systematic investigation on the interval separating consuming reinforcers from earning additional reinforcers. This oversight is problematic because this second interval is an integral part of much of the previous research on reinforcer accumulation. The purpose of the current study was to determine the independent contributions of these two temporal intervals on reinforcer accumulation in rats. Each left lever press earned a single food pellet; delivery of the accumulated pellet(s) occurred upon a right lever press. Conditions varied based on the presence of either an intertrial interval (ITI) that separated pellet delivery from the further opportunity to accumulate more pellets, or a delay-to-reinforcement that separated the right lever press from the delivery of the accumulated pellet(s). Delay and ITI values of 0, 5, 10 and 20 s were investigated. The delay-to-reinforcement conditions produced greater accumulation relative to the ITI conditions, despite accumulation increasing the density of reinforcement more substantially in the ITI conditions. This finding suggests that the temporal separation between reinforcer accumulation and subsequent delivery and consumption was a more critical variable in controlling reinforcer accumulation.