The effects of a time-based intervention on experienced middle-aged rats

Active and passive waiting in impulsive choice: Effects of fixed-interval and fixed-time delays

Behavioral interventions to improve self-control, preference for a larger-later (LL) reward over a smaller-sooner (SS) reward, involve experience with delayed rewards. Whether they involve timing processes remains controversial. In rats, there have been inconsistent results on whether timing processes may be involved in intervention-induced improvements in self-control. Interventions that improved self-control with corresponding timing improvements used fixed-interval (FI) delays, whereas interventions that failed to find corresponding timing improvements used fixed-time (FT) delays. The FI schedule includes a response contingency (active waiting), whereas the FT schedule delivers reward automatically (passive waiting). The present study compared the effects of FI and FT schedules in interventions and impulsive choice tasks to evaluate effects on self-control and timing behavior. The impulsive choice task evaluated preference for an SS option (one pellet after 10-, 15-, 20-, 25-, and 30-s delays) versus an LL option (two pellets after a 30-s delay). The intervention task included forced-choice SS (one pellet after 10 s) and LL (two pellets after 30 s) sessions under FI or FT schedules. FI schedules produced greater sensitivity to SS delay in the impulsive choice task. Both FI and FT interventions increased LL choices. Following choice testing, temporal bisection and peak interval tasks revealed better timing precision for rats with an FI delay experience. Overall, the FI choice contingency was associated with improved temporal attention and timing precision.

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

A behavioral timing intervention upregulates striatal serotonergic markers and reduces impulsive action in adult male mice

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a "free" pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

Effects of immediate-reinforcement training on delay discounting behavior in rats

Chronic exposure to delayed reinforcement has been shown to increase choice for larger, later reinforcement in a subsequent delay discounting task. In the 3 experiments presented in this paper, the opposite was tested: effects of chronic exposure to immediate reinforcement on choice in a subsequent delay discounting task. In Experiment 1, larger, later reinforcement choice was significantly reduced as a result of exposure to immediate reinforcement, compared to a maturation/handle control group, in experienced, male Lewis rats. In Experiment 2, with naive male and female Wistar rats, and Experiment 3, with naïve male Long Evans rats, the impact of exposure to immediate reinforcement was less robust, but directionally consistent with results from Experiment 1. These results align with some previous work reporting that exposure to immediate reinforcement may decrease choice for larger, later reinforcement in a delay discounting task, and/or blunt maturational increases in choice for larger, later reinforcement. These findings have implications for future research investigating experience-based interventions to manipulate delay discounting behavior. They also have clinical implications for understanding and treating disorders involving maladaptive choice.

Hazard function effects on promoting self-control in variable interval time-based interventions in rats

The present experiments investigated properties of time-based interventions used to increase self-control. Rats received impulsive-choice assessments before and after interventions that consisted of different distributions of delays to reinforcement. In Experiment 1, rats received an intervention with an increasing hazard function where delays were more evenly distributed, a decreasing hazard function where delays were mostly short, or a constant hazard function where delays were exponentially distributed. Surprisingly, rats that received the decreasing hazard function made the most self-controlled choices. Response rates during intervention trials showed that rats anticipated reinforcement based on the shape of the distributions they received. In Experiment 2, rats received an intervention with a decreasing hazard function with a steep slope or a shallow slope. Both time-based interventions increased self-control and produced similar response-rate patterns, indicating that the slope of the decreasing hazard function may not play a strong role in intervention efficacy. While this research aligns with previous literature showing that time-based interventions improved self-control, exposure to short delays produced the biggest improvements. Ultimately, exposure to short delays may increase the subjective value of the larger-later choice while occasional long delays may promote the ability to wait, which may have important implications for translational applications.

A time-based intervention to treat impulsivity in male and female rats

Time-based interventions have emerged as promising treatments for disorders associated with impulsivity. These interventions can be implemented to test their efficacy in preventing or treating impulsive choice in animal models of diseases related to impulsivity such as drug abuse. Impulsive choice is typically defined as choosing a smaller-sooner (SS) reward over a larger-later (LL) reward when the LL is relatively more optimal. Previous research has shown that these interventions promote LL choices in males and females, but sex differences have not been assessed. Because sex differences can complicate the application of therapies, it is critical to compare the effects of the intervention in males and females. The intervention group received exposure to 10-s and 30-s interval schedules, and the control rats received no delay to reward. Different impulsive choice tasks were used to assess the intervention efficacy across the two experiments. Following the intervention, reductions in impulsive choice were found in male and female rats, but the degree of improvement was inconsistent across sex and task. Bayesian analyses that combined the results revealed robust evidence of an overall intervention effect with the intervention group showing greater self-control, but there was no evidence for the intervention affecting males and females differently. Taken together, these results suggest that time-based interventions are effective tools to treat impulsivity in both males and females and offer promising translational capability to humans.

Reducing impulsive choice: VI. Delay-exposure training reduces aversion to delay-signaling stimuli

Delay-exposure (DE) training consistently and robustly reduces impulsive choice in rats, but the behavioral mechanisms behind this effect are not yet understood. The present study evaluated if DE training works by mitigating aversion to delay-signaling stimuli-those encountered when rats chose the larger-later reward in impulsive choice assessments. Fifty-seven rats were randomly assigned to 120 days of training with delayed reinforcement, training with immediate reinforcement (IE), or to a no-training Control group. Consistent with prior experiments, DE rats made significantly fewer impulsive choices than IE or Control rats. Subsequently, in a separate assessment of delay aversion, rats were given the opportunity to press a lever to temporarily escape from stimuli correlated with long or short time-intervals to food. When these escape opportunities terminated delay-signaling stimuli in the impulsive-choice task, DE rats escaped significantly less than IE and Control rats. When escapes terminated FI-signaling stimuli (a procedure in which there is no response-reinforcer delay), the difference only approached significance. These results support the hypothesis that DE training reduces impulsive choice, in part, by reducing aversion to delay-signaling stimuli. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Changing Delay Discounting and Impulsive Choice: Implications for Addictions, Prevention, and Human Health

Delay discounting describes the tendency to devalue delayed consequences or future prospects. The degree to which an individual discounts delayed events appears trait-like in that it is stable over time and across functionally similar situations. Steeply discounting delayed rewards is correlated with most substance-use disorders, the severity of these disorders, rates of relapse to drug use, and a host of other maladaptive decisions impacting human health. Longitudinal data suggest steep delay discounting and high levels of impulsive choice are predictive of subsequent drug taking, which suggests (though does not establish) that reducing delay discounting could have a preventive health-promoting effect. Experimental manipulations that produce momentary or long-lasting reductions in delay discounting or impulsive choice are reviewed, and behavioral mechanisms that may underlie these effects are discussed. Shortcomings of each manipulation technique are discussed and areas for future research are identified. While much work remains, it is clear that impulsive decision-making can be reduced, despite its otherwise trait-like qualities. Such findings invite technique refinement, translational research, and hope.

Interventions aimed at changing impulsive choice in rats: Effects of immediate and relatively long delay to reward training

A relatively strong preference for smaller-sooner rewards (SSR) over larger-later rewards (LLR) is associated with a host of maladaptive behavioral patterns. As such, the clinical implications for increasing preference for LLR are profound. There is a growing body of literature that suggests extended exposure to delayed reward may increase preference for LLR in rats. However, questions remain about the underlying mechanism driving this effect and the extent to which extended exposure to immediate rewards may decrease LLR choice. In Experiment 1, we tested effects of a differential-reinforcement-of-low-rates schedule (DRL) to increase LLR choice using a pretest/posttest design with Wistar rats as subjects. We compared this group to a group of rats exposed to a differential-reinforcement-of-high-rates schedule (DRH). The DRH intervention has never been employed in this research context, but explicitly programs an immediate response-reinforcement requirement. In Experiment 2, we tested effects of an intervention with a delay longer than those used in the delay discounting pretest and posttest. No previous research has tested effects of an intervention delay this long, relative to the delay discounting task. We compared this group to a group exposed to a delay that was part of the delay discounting pretest and posttest and to a group exposed to a traditional no-delay, fixed-ratio (FR) 2 control intervention. In both experiments, we found that exposure to delayed rewards in the intervention phase significantly increased LLR choice relative to pretest performance. These findings replicate and extend a growing body of literature showing that delay exposure increases preference for LLR. We also found significant decreases in LLR choice from pretest to posttest in the DRH and no-delay intervention groups in Experiments 1 and 2, respectively. This is the first report of such an effect and has implications for understanding and interpreting effects of delay exposure training in past and future research. Our results also suggested no relationship between improved temporal tracking of reward and increases in LLR choice as a result of delay exposure training.

Frontal brain injury chronically impairs timing behavior in rats

Traumatic brain injury (TBI) affects over 2.8 million people annually, and has been shown to increase motor impulsivity in both humans and animals. However, the root cause of this behavioral disinhibition is not fully understood. The goal of the current study was to evaluate whether timing behavior is disrupted after TBI, which could potentially explain increases in impulsive responding. Twenty-one male three-month old Long-Evans rats were trained on a fixed interval-18 s schedule. Following training, rats were placed on the Peak Interval Procedure, with intermittent peak trials. On peak trials, no behaviors were reinforced and response rates were recorded to determine timing ability. After reaching a stable baseline, rats received bilateral frontal TBI (n = 12) using controlled cortical impact or sham procedures (n = 9). After one week recovery, rats were re-assessed on the Peak Procedure for six weeks. An amphetamine challenge was carried out after behavior reached stable post-injury performance. TBI caused a chronic decrease/acceleration in peak time, increase in response variability, and reduction in response rate. The shifted peak time suggests that altered perception of time may contribute to impairments in response inhibition after TBI. Amphetamine significantly increased response variability, with TBI animals demonstrating greater sensitivity, but did not affect peak time in either group. These data suggest that timing may not be the sole factor explaining impulsive action after TBI given that amphetamine reduced motor impulsivity in prior studies. Further investigations will be needed to dissociate the effects of amphetamine on TBI with regard to timing behavior.

Females in the forefront: time-based intervention effects on impulsive choice and interval timing in female rats

Impulsive choice has been implicated in substance abuse, gambling, obesity, and other maladaptive behaviors. Deficits in interval timing may increase impulsive choices, and therefore, could serve as an avenue through which suboptimal impulsive choices can be moderated. Temporal interventions have successfully attenuated impulsive choices in male rats, but the efficacy of a temporal intervention has yet to be assessed in female rats. As such, this experiment examined timing and choice behavior in female rats, and evaluated the ability of a temporal intervention to mitigate impulsive choice behavior. The temporal intervention administered in this study was successful in reducing impulsive choices compared to a control group. Results of a temporal bisection task indicated that the temporal intervention increased long responses at the shorter durations. Further, results from the peak trials within the choice task combined with the progressive interval task suggest that the intervention increased sensitivity to delay and enhanced timing confidence. Overall, these results indicate that a temporal intervention can be a successful avenue for reducing impulsive choice behavior in female rats, and could contribute to the development of behavioral interventions to prevent impulsive choice and maladaptive behaviors that can be applied to both sexes.

Impulsive choice and pre-exposure to delays: iv. effects of delay- and immediacy-exposure training relative to maturational changes in impulsivity

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay-exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay-exposure training shifting preference toward self-control or immediacy-exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay- and immediacy-exposure training to a no-training control group and evaluated within-subject changes in impulsive choice across 51 male Wistar rats. Delay-exposed rats made significantly fewer impulsive choices than immediacy-exposed and control rats. Between-group differences in impulsive choice were not observed in the latter two groups. While delay-exposed rats showed large, significant pre- to posttraining reductions in impulsive choice, immediacy-exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.

Durability and generalizability of time-based intervention effects on impulsive choice in rats

Impulsive choice involves choosing a smaller-sooner (SS) reward over a larger-later (LL) reward. Due to the importance of timing processes in impulsive choice, time-based interventions have been developed to decrease impulsive choice. The present set of experiments assessed the durability and generalizability of time-based interventions. Experiment 1 assessed fixed interval (FI) or variable interval (VI) intervention efficacy over 9 months. The FI intervention decreased impulsive choice, and this effect persisted over time, but the VI intervention effects were only apparent when tested immediately after the intervention. Experiment 2 examined the generalizability of the FI and VI interventions on choice tasks manipulating the SS delay, LL delay, or LL magnitude. The FI intervention decreased sensitivity to delay, promoting LL choices in both delay tasks, but the VI intervention only altered choices when manipulating the SS delay. Experiment 3 further examined the FI intervention effects on tasks that manipulated the LL delay or magnitude immediately following the intervention. The intervention decreased sensitivity to both delay and magnitude. The experiments indicate that the FI intervention is effective at decreasing impulsive choice behavior for an extended period across changing delays and magnitudes, suggesting a relatively broad effect on choice behavior.

Nucleus accumbens core lesions induce sub-optimal choice and reduce sensitivity to magnitude and delay in impulsive choice tasks

The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making.

Increased temporal discounting after chronic stress in CHL1-deficient mice is reversed by 5-HT2C agonist Ro 60-0175

Schizophrenia is a neurodevelopmental disorder in which impaired decision-making and goal-directed behaviors are core features. One of the genes associated with schizophrenia is the Close Homolog of L1 (CHL1); CHL1-deficient mice are considered a model of schizophrenia-like deficits, including sensorimotor gating, interval timing and spatial memory impairments. Here we investigated temporal discounting in CHL1-deficient (KO) mice and their wild-type littermates. Although no discounting differences were found under baseline conditions, CHL1-KO mice showed increased impulsive choice following chronic unpredictable stress (fewer % larger-later choices, and reduced area under the discounting curve). Stressed CHL1-KO mice also showed decreased neuronal activation (number of cFos positive neurons) in the discounting task in the prelimbic cortex and dorsal striatum, areas thought to be part of executive and temporal processing circuits. Impulsive choice alterations were reversed by the 5-HT2C agonist Ro 60-0175. Our results provide evidence for a gene x environment, double-hit model of stress-related decision-making impairments, and identify CHL1-deficient mice as a mouse model for these deficits in regard to schizophrenia-like phenotypes.