Females in the forefront: time-based intervention effects on impulsive choice and interval timing in female rats

Active and passive waiting in impulsive choice: Effects of fixed-interval and fixed-time delays

Behavioral interventions to improve self-control, preference for a larger-later (LL) reward over a smaller-sooner (SS) reward, involve experience with delayed rewards. Whether they involve timing processes remains controversial. In rats, there have been inconsistent results on whether timing processes may be involved in intervention-induced improvements in self-control. Interventions that improved self-control with corresponding timing improvements used fixed-interval (FI) delays, whereas interventions that failed to find corresponding timing improvements used fixed-time (FT) delays. The FI schedule includes a response contingency (active waiting), whereas the FT schedule delivers reward automatically (passive waiting). The present study compared the effects of FI and FT schedules in interventions and impulsive choice tasks to evaluate effects on self-control and timing behavior. The impulsive choice task evaluated preference for an SS option (one pellet after 10-, 15-, 20-, 25-, and 30-s delays) versus an LL option (two pellets after a 30-s delay). The intervention task included forced-choice SS (one pellet after 10 s) and LL (two pellets after 30 s) sessions under FI or FT schedules. FI schedules produced greater sensitivity to SS delay in the impulsive choice task. Both FI and FT interventions increased LL choices. Following choice testing, temporal bisection and peak interval tasks revealed better timing precision for rats with an FI delay experience. Overall, the FI choice contingency was associated with improved temporal attention and timing precision.

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Effects of the estrous cycle on impulsive choice and interval timing in female rats

Research in humans and animals shows differences in impulsive choice, which is a failure to wait for larger, delayed rewards, when comparing males and females. It is possible that fluctuations in sex hormones (estradiol and progesterone) across the reproductive cycle contribute to sex differences in impulsive choice. The current study delivered an impulsive choice task with peak interval trials to female rats while estrous cycles, the rodent reproductive cycle, were tracked over the course of the task. Female rats were more sensitive to changes in delay in the proestrus phase of the estrous cycle and made more larger-later choices when in estrus, particularly when the delay to the smaller reward was short. Estradiol increases dramatically during proestrus while progesterone peaks during estrus, suggesting that estradiol and progesterone may affect impulsive choice through mechanisms such as delay discounting, delay aversion, and/or timing processes. Analyses of timing of the choice task delays showed inconsistent effects of the estrous cycle across delays, suggesting that reward-timing interactions may have complicated how hormone fluctuations affected interval timing. Further research is needed to determine the mechanism underlying increased larger-later choices during the estrus phase, increased delay sensitivity during the proestrus phase, and variability in interval timing across delays and estrous cycle stages.

A behavioral timing intervention upregulates striatal serotonergic markers and reduces impulsive action in adult male mice

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a "free" pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

Reducing impulsive choice: VIII. Effects of delay-exposure training in female rats

Impulsive choice may play an important role in serious health-related decisions, like addiction tendencies. Thus, there is merit in exploring interventions that reduce impulsive choice. Delay-exposure training involves extended experience with delayed reinforcement. Following training, delay-exposed rats make fewer impulsive choices than control rats. The reducing effects of delay exposure training on impulsive choice have been replicated in male rats seven times. For the first time, this study evaluated the effects of delay exposure training in female rats. Thirty-six rats were randomly assigned to either delay-exposure or immediacy-exposure training. Then, rats underwent two impulsive choice assessments in which they chose between one immediate pellet or three delayed pellets. In the first assessment, delays increased within-sessions, across trial blocks from 0, 8, 16, to 32 s. In the second assessment, delays to the larger reward increased between-sessions, from 8, 16, 32, to 4 s. Unlike findings with male rats, delay-exposure training produced a reduction in impulsive choice only in the initial five sessions in female rats. Possible reasons for the lack of lasting effect in female rats are discussed and future research directions are identified.

Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay

Interventions exposing rats to delayed-reward contingencies attenuate suboptimal impulsive choices, a preference for a smaller-sooner (SS) over a larger-later (LL) reward. Interventions may potentially improve delay-tolerance, timing of delays, and/or discrimination of reward magnitudes. Generalization from the intervention to impulsive choice under different procedures can provide insights into the processes that underlie the intervention effects. Experiment 1 tested intervention effects on systematic-delay (SYS) and adjusting-delay (ADJ) procedures, predicting that intervention effects would be more effective on the SYS procedure with predictable delays. The ADJ procedure did not benefit significantly from intervention, but the SYS procedure, unexpectedly, showed greater impulsive choices following intervention. Experiment 2 tested whether short (5 s) SS intervention delays may have promoted greater impulsivity in the SYS impulsive choice procedure in Experiment 1. Short SS delays in choice and intervention procedures increased impulsive choices in comparison to longer (10 s) delays. Incongruent SS delays in the intervention/choice procedures resulted in negative intervention effects. The results suggest that short SS delays are detrimental to self-control and that specific temporal information generalizes from the intervention to the SYS choice task, but not the ADJ choice task.

Effects of immediate-reinforcement training on delay discounting behavior in rats

Chronic exposure to delayed reinforcement has been shown to increase choice for larger, later reinforcement in a subsequent delay discounting task. In the 3 experiments presented in this paper, the opposite was tested: effects of chronic exposure to immediate reinforcement on choice in a subsequent delay discounting task. In Experiment 1, larger, later reinforcement choice was significantly reduced as a result of exposure to immediate reinforcement, compared to a maturation/handle control group, in experienced, male Lewis rats. In Experiment 2, with naive male and female Wistar rats, and Experiment 3, with naïve male Long Evans rats, the impact of exposure to immediate reinforcement was less robust, but directionally consistent with results from Experiment 1. These results align with some previous work reporting that exposure to immediate reinforcement may decrease choice for larger, later reinforcement in a delay discounting task, and/or blunt maturational increases in choice for larger, later reinforcement. These findings have implications for future research investigating experience-based interventions to manipulate delay discounting behavior. They also have clinical implications for understanding and treating disorders involving maladaptive choice.

Hazard function effects on promoting self-control in variable interval time-based interventions in rats

The present experiments investigated properties of time-based interventions used to increase self-control. Rats received impulsive-choice assessments before and after interventions that consisted of different distributions of delays to reinforcement. In Experiment 1, rats received an intervention with an increasing hazard function where delays were more evenly distributed, a decreasing hazard function where delays were mostly short, or a constant hazard function where delays were exponentially distributed. Surprisingly, rats that received the decreasing hazard function made the most self-controlled choices. Response rates during intervention trials showed that rats anticipated reinforcement based on the shape of the distributions they received. In Experiment 2, rats received an intervention with a decreasing hazard function with a steep slope or a shallow slope. Both time-based interventions increased self-control and produced similar response-rate patterns, indicating that the slope of the decreasing hazard function may not play a strong role in intervention efficacy. While this research aligns with previous literature showing that time-based interventions improved self-control, exposure to short delays produced the biggest improvements. Ultimately, exposure to short delays may increase the subjective value of the larger-later choice while occasional long delays may promote the ability to wait, which may have important implications for translational applications.

A time-based intervention to treat impulsivity in male and female rats

Time-based interventions have emerged as promising treatments for disorders associated with impulsivity. These interventions can be implemented to test their efficacy in preventing or treating impulsive choice in animal models of diseases related to impulsivity such as drug abuse. Impulsive choice is typically defined as choosing a smaller-sooner (SS) reward over a larger-later (LL) reward when the LL is relatively more optimal. Previous research has shown that these interventions promote LL choices in males and females, but sex differences have not been assessed. Because sex differences can complicate the application of therapies, it is critical to compare the effects of the intervention in males and females. The intervention group received exposure to 10-s and 30-s interval schedules, and the control rats received no delay to reward. Different impulsive choice tasks were used to assess the intervention efficacy across the two experiments. Following the intervention, reductions in impulsive choice were found in male and female rats, but the degree of improvement was inconsistent across sex and task. Bayesian analyses that combined the results revealed robust evidence of an overall intervention effect with the intervention group showing greater self-control, but there was no evidence for the intervention affecting males and females differently. Taken together, these results suggest that time-based interventions are effective tools to treat impulsivity in both males and females and offer promising translational capability to humans.

Reducing impulsive choice: VI. Delay-exposure training reduces aversion to delay-signaling stimuli

Delay-exposure (DE) training consistently and robustly reduces impulsive choice in rats, but the behavioral mechanisms behind this effect are not yet understood. The present study evaluated if DE training works by mitigating aversion to delay-signaling stimuli-those encountered when rats chose the larger-later reward in impulsive choice assessments. Fifty-seven rats were randomly assigned to 120 days of training with delayed reinforcement, training with immediate reinforcement (IE), or to a no-training Control group. Consistent with prior experiments, DE rats made significantly fewer impulsive choices than IE or Control rats. Subsequently, in a separate assessment of delay aversion, rats were given the opportunity to press a lever to temporarily escape from stimuli correlated with long or short time-intervals to food. When these escape opportunities terminated delay-signaling stimuli in the impulsive-choice task, DE rats escaped significantly less than IE and Control rats. When escapes terminated FI-signaling stimuli (a procedure in which there is no response-reinforcer delay), the difference only approached significance. These results support the hypothesis that DE training reduces impulsive choice, in part, by reducing aversion to delay-signaling stimuli. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Changing Delay Discounting and Impulsive Choice: Implications for Addictions, Prevention, and Human Health

Delay discounting describes the tendency to devalue delayed consequences or future prospects. The degree to which an individual discounts delayed events appears trait-like in that it is stable over time and across functionally similar situations. Steeply discounting delayed rewards is correlated with most substance-use disorders, the severity of these disorders, rates of relapse to drug use, and a host of other maladaptive decisions impacting human health. Longitudinal data suggest steep delay discounting and high levels of impulsive choice are predictive of subsequent drug taking, which suggests (though does not establish) that reducing delay discounting could have a preventive health-promoting effect. Experimental manipulations that produce momentary or long-lasting reductions in delay discounting or impulsive choice are reviewed, and behavioral mechanisms that may underlie these effects are discussed. Shortcomings of each manipulation technique are discussed and areas for future research are identified. While much work remains, it is clear that impulsive decision-making can be reduced, despite its otherwise trait-like qualities. Such findings invite technique refinement, translational research, and hope.

Interventions aimed at changing impulsive choice in rats: Effects of immediate and relatively long delay to reward training

A relatively strong preference for smaller-sooner rewards (SSR) over larger-later rewards (LLR) is associated with a host of maladaptive behavioral patterns. As such, the clinical implications for increasing preference for LLR are profound. There is a growing body of literature that suggests extended exposure to delayed reward may increase preference for LLR in rats. However, questions remain about the underlying mechanism driving this effect and the extent to which extended exposure to immediate rewards may decrease LLR choice. In Experiment 1, we tested effects of a differential-reinforcement-of-low-rates schedule (DRL) to increase LLR choice using a pretest/posttest design with Wistar rats as subjects. We compared this group to a group of rats exposed to a differential-reinforcement-of-high-rates schedule (DRH). The DRH intervention has never been employed in this research context, but explicitly programs an immediate response-reinforcement requirement. In Experiment 2, we tested effects of an intervention with a delay longer than those used in the delay discounting pretest and posttest. No previous research has tested effects of an intervention delay this long, relative to the delay discounting task. We compared this group to a group exposed to a delay that was part of the delay discounting pretest and posttest and to a group exposed to a traditional no-delay, fixed-ratio (FR) 2 control intervention. In both experiments, we found that exposure to delayed rewards in the intervention phase significantly increased LLR choice relative to pretest performance. These findings replicate and extend a growing body of literature showing that delay exposure increases preference for LLR. We also found significant decreases in LLR choice from pretest to posttest in the DRH and no-delay intervention groups in Experiments 1 and 2, respectively. This is the first report of such an effect and has implications for understanding and interpreting effects of delay exposure training in past and future research. Our results also suggested no relationship between improved temporal tracking of reward and increases in LLR choice as a result of delay exposure training.

Durability and generalizability of time-based intervention effects on impulsive choice in rats

Impulsive choice involves choosing a smaller-sooner (SS) reward over a larger-later (LL) reward. Due to the importance of timing processes in impulsive choice, time-based interventions have been developed to decrease impulsive choice. The present set of experiments assessed the durability and generalizability of time-based interventions. Experiment 1 assessed fixed interval (FI) or variable interval (VI) intervention efficacy over 9 months. The FI intervention decreased impulsive choice, and this effect persisted over time, but the VI intervention effects were only apparent when tested immediately after the intervention. Experiment 2 examined the generalizability of the FI and VI interventions on choice tasks manipulating the SS delay, LL delay, or LL magnitude. The FI intervention decreased sensitivity to delay, promoting LL choices in both delay tasks, but the VI intervention only altered choices when manipulating the SS delay. Experiment 3 further examined the FI intervention effects on tasks that manipulated the LL delay or magnitude immediately following the intervention. The intervention decreased sensitivity to both delay and magnitude. The experiments indicate that the FI intervention is effective at decreasing impulsive choice behavior for an extended period across changing delays and magnitudes, suggesting a relatively broad effect on choice behavior.