Mechanisms of impulsive choice: I. Individual differences in interval timing and reward processing

Hyperbolae

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable-interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.

Choice impulsivity after repeated social stress is associated with increased perineuronal nets in the medial prefrontal cortex

Repeated stress can predispose to substance abuse. However, behavioral and neurobiological adaptations that link stress to substance abuse remain unclear. This study investigates whether intermittent social defeat (ISD), a stress protocol that promotes drug-seeking behavior, alters intertemporal decision-making and cortical inhibitory function in the medial prefrontal cortex (mPFC). Male long evans rats were trained in a delay discounting task (DDT) where rats make a choice between a fast (1 s) small reward (1 sugar pellet) and a large reward (3 sugar pellets) that comes with a time delay (10 s or 20 s). A decreased preference for delayed rewards was used as an index of choice impulsivity. Rats were exposed to ISD and tested in the DDT 24 h after each stress episode, and one- and two-weeks after the last stress episode. Immunohistochemistry was performed in rat's brains to evaluate perineuronal nets (PNNs) and parvalbumin GABA interneurons (PV) labeling as markers of inhibitory function in mPFC. ISD significantly decreased the preference for delayed large rewards in low impulsive, but not high impulsive, animals. ISD also increased the density of PNNs in the mPFC. These results suggest that increased choice impulsivity and cortical inhibition predispose animals to seek out rewards after stress.

Active and passive waiting in impulsive choice: Effects of fixed-interval and fixed-time delays

Behavioral interventions to improve self-control, preference for a larger-later (LL) reward over a smaller-sooner (SS) reward, involve experience with delayed rewards. Whether they involve timing processes remains controversial. In rats, there have been inconsistent results on whether timing processes may be involved in intervention-induced improvements in self-control. Interventions that improved self-control with corresponding timing improvements used fixed-interval (FI) delays, whereas interventions that failed to find corresponding timing improvements used fixed-time (FT) delays. The FI schedule includes a response contingency (active waiting), whereas the FT schedule delivers reward automatically (passive waiting). The present study compared the effects of FI and FT schedules in interventions and impulsive choice tasks to evaluate effects on self-control and timing behavior. The impulsive choice task evaluated preference for an SS option (one pellet after 10-, 15-, 20-, 25-, and 30-s delays) versus an LL option (two pellets after a 30-s delay). The intervention task included forced-choice SS (one pellet after 10 s) and LL (two pellets after 30 s) sessions under FI or FT schedules. FI schedules produced greater sensitivity to SS delay in the impulsive choice task. Both FI and FT interventions increased LL choices. Following choice testing, temporal bisection and peak interval tasks revealed better timing precision for rats with an FI delay experience. Overall, the FI choice contingency was associated with improved temporal attention and timing precision.

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity

Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

A behavioral timing intervention upregulates striatal serotonergic markers and reduces impulsive action in adult male mice

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a "free" pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Interoceptive accuracy correlates with precision of time perception in the millisecond range

It has been proposed that accuracy in time perception is related to interoceptive accuracy and vagal activity. However, studies investigating time perception in the supra-second range have provided mixed results, and few studies have investigated the sub-second range. Moreover, there is a lack of studies investigating the relationship between precision in time perception and interoceptive accuracy. A recent meta-analytic review of neuroimaging studies proposed a dynamic interaction between two types of timing processing-an endogenous time keeping mechanism and the use of exogenous temporal cues. Interoceptive accuracy may affect both accuracy and precision of primary temporal representations, as they are generated based on the endogenous time keeping mechanism. Temporal accuracy may vary when adapted to the environmental context. In contrast, temporal precision contains some constant noise, which may maintain the relationship with interoceptive accuracy. Based on these assumptions, we hypothesized that interoceptive accuracy would be associated with temporal precision in the sub-second range, while vagal activity would be associated with temporal accuracy. We used the temporal generalization task, which allowed us to calculate the indices of temporal accuracy and temporal precision in line with the existing research, and also compute the index of participants' sensitivity according to the signal detection theory. Specifically, we investigated whether (1) interoceptive accuracy would correlate with temporal accuracy, temporal precision, or sensitivity and (2) resting-state vagal activity would correlate with temporal accuracy, temporal precision, or sensitivity. The results indicated that interoceptive accuracy was positively correlated with temporal precision as well as sensitivity, but not with temporal accuracy, in the sub-second range time perception. Vagal activity was negatively correlated only with sensitivity. Furthermore, we found a moderation effect of sensitivity on the relationship between vagal activity and perceived duration, which affected the association between vagal activity and temporal accuracy. These findings suggest the importance of precision as an aspect of time perception, which future studies should further explore in relation to interoception and vagal activity, and of the moderation effects of factors such as participants' sensitivity in this context.

Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay

Interventions exposing rats to delayed-reward contingencies attenuate suboptimal impulsive choices, a preference for a smaller-sooner (SS) over a larger-later (LL) reward. Interventions may potentially improve delay-tolerance, timing of delays, and/or discrimination of reward magnitudes. Generalization from the intervention to impulsive choice under different procedures can provide insights into the processes that underlie the intervention effects. Experiment 1 tested intervention effects on systematic-delay (SYS) and adjusting-delay (ADJ) procedures, predicting that intervention effects would be more effective on the SYS procedure with predictable delays. The ADJ procedure did not benefit significantly from intervention, but the SYS procedure, unexpectedly, showed greater impulsive choices following intervention. Experiment 2 tested whether short (5 s) SS intervention delays may have promoted greater impulsivity in the SYS impulsive choice procedure in Experiment 1. Short SS delays in choice and intervention procedures increased impulsive choices in comparison to longer (10 s) delays. Incongruent SS delays in the intervention/choice procedures resulted in negative intervention effects. The results suggest that short SS delays are detrimental to self-control and that specific temporal information generalizes from the intervention to the SYS choice task, but not the ADJ choice task.

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

Effects of immediate-reinforcement training on delay discounting behavior in rats

Chronic exposure to delayed reinforcement has been shown to increase choice for larger, later reinforcement in a subsequent delay discounting task. In the 3 experiments presented in this paper, the opposite was tested: effects of chronic exposure to immediate reinforcement on choice in a subsequent delay discounting task. In Experiment 1, larger, later reinforcement choice was significantly reduced as a result of exposure to immediate reinforcement, compared to a maturation/handle control group, in experienced, male Lewis rats. In Experiment 2, with naive male and female Wistar rats, and Experiment 3, with naïve male Long Evans rats, the impact of exposure to immediate reinforcement was less robust, but directionally consistent with results from Experiment 1. These results align with some previous work reporting that exposure to immediate reinforcement may decrease choice for larger, later reinforcement in a delay discounting task, and/or blunt maturational increases in choice for larger, later reinforcement. These findings have implications for future research investigating experience-based interventions to manipulate delay discounting behavior. They also have clinical implications for understanding and treating disorders involving maladaptive choice.

Hazard function effects on promoting self-control in variable interval time-based interventions in rats

The present experiments investigated properties of time-based interventions used to increase self-control. Rats received impulsive-choice assessments before and after interventions that consisted of different distributions of delays to reinforcement. In Experiment 1, rats received an intervention with an increasing hazard function where delays were more evenly distributed, a decreasing hazard function where delays were mostly short, or a constant hazard function where delays were exponentially distributed. Surprisingly, rats that received the decreasing hazard function made the most self-controlled choices. Response rates during intervention trials showed that rats anticipated reinforcement based on the shape of the distributions they received. In Experiment 2, rats received an intervention with a decreasing hazard function with a steep slope or a shallow slope. Both time-based interventions increased self-control and produced similar response-rate patterns, indicating that the slope of the decreasing hazard function may not play a strong role in intervention efficacy. While this research aligns with previous literature showing that time-based interventions improved self-control, exposure to short delays produced the biggest improvements. Ultimately, exposure to short delays may increase the subjective value of the larger-later choice while occasional long delays may promote the ability to wait, which may have important implications for translational applications.

Quantifying the contribution of individual variation in timing to delay-discounting

Delay-discounting studies in neuroscience, psychology, and economics have been mostly focused on concepts of self-control, reward evaluation, and discounting. Another important relationship to consider is the link between intertemporal choice and time perception. We presented 50 college students with timing tasks on the range of seconds to minutes and intertemporal-choice tasks on both the time-scale of seconds and of days. We hypothesized that individual differences in time perception would influence decisions about short experienced delays but not long delays. While we found some evidence that individual differences in internal clock speed account for some unexplained variance between choices across time-horizons, overall our findings suggest a nominal contribution of the altered sense of time in intertemporal choice.

High impulsive choice is accompanied by an increase in dopamine release in rat dorsolateral striatum

Dopamine neurotransmission has been consistently associated with individual differences in impulsive choice. Clinical and preclinical evidence suggests that low striatal dopamine D₂ signaling predisposes to engage in impulsive behaviors. Although dopamine D₂ signaling controls dopamine (DA) extracellular levels, the relationship between striatal dopamine extracellular levels and impulsive choice remains poorly understood. Using quantitative microdialysis, we investigated whether extracellular DA levels in rat dorsolateral striatum (DLS) correlates with preference for an immediate small reward or for a delayed larger reward. Rats were tested in a delay-discounting task and classified as high impulsive (HI) or low impulsive (LI) according to the area under the discounting curve (AUC). No-net flux microdialysis experiments, assessing basal DA release, DA-uptake, and DA extracellular concentration (DA C_ext), were carried out in dorsolateral striatum (DLS) of urethane-anesthetized rats. Rats classified as HI showed a higher DA release compared with LI rats. Differences in DLS DA-uptake and DA C_ext were non-significant. Importantly, a significant negative correlation was observed between AUC and DA release, indicating that the lower the AUC, the higher the DLS DA release. This finding shows that DA release is augmented in the DLS of rats classified as HI, suggesting that a hyper-activated nigro-striatal pathway contributes to impulsive choice.

Time perception and patience: individual differences in interval timing precision predict choice impulsivity in European starlings, Sturnus vulgaris

Impulsivity, in the sense of the extent rewards are devalued as the time until their realization increases, is linked to various negative outcomes in humans, yet understanding of the cognitive mechanisms underlying it is limited. Variation in the imprecision of interval timing is a possible contributor to variation in impulsivity. We use a numerical model to generate predictions concerning the effect of timing imprecision on impulsivity. We distinguish between fixed imprecision (the imprecision that applies even when timing the very shortest time intervals) and proportional imprecision (the rate at which imprecision increases as the interval becomes longer). The model predicts that impulsivity should increase with increasing fixed imprecision, but decrease with increasing proportional imprecision. We present data from a cohort of European starlings (Sturnus vulgaris, n = 28) in which impulsivity had previously been measured through an intertemporal choice paradigm. We tested interval timing imprecision in the same individuals using a tri-peak temporal reproduction procedure. We found repeatable individual differences in both fixed and proportional imprecision. As predicted, birds with greater proportional imprecision in interval timing made fewer impulsive choices, whilst those with greater fixed imprecision tended to make more. Contradictory observations in the literature regarding the direction of association between timing imprecision and impulsivity might be clarified by distinguishing between fixed and proportional components of imprecision.

Delay discounting of different outcomes: Review and theory

Steep delay discounting is characterized by a preference for small immediate outcomes relative to larger delayed outcomes and is predictive of drug abuse, risky sexual behaviors, and other maladaptive behaviors. Nancy M. Petry was a pioneer in delay discounting research who demonstrated that people discount delayed monetary gains less steeply than they discount substances with abuse liability. Subsequent research found steep discounting for not only drugs, but other nonmonetary outcomes such as food, sex, and health. In this systematic review, we evaluate the hypotheses proposed to explain differences in discounting as a function of the type of outcome and explore the trait- and state-like nature of delay discounting. We found overwhelming evidence for the state-like quality of delay discounting: Consistent with Petry and others' work, nonmonetary outcomes are discounted more steeply than monetary outcomes. We propose two hypotheses that together may account for this effect: Decreasing Future Preference and Decreasing Future Worth. We also found clear evidence that delay discounting has trait-like qualities: People who steeply discount monetary outcomes steeply discount nonmonetary outcomes as well. The implication is that changing delay discounting for one outcome could change discounting for other outcomes.

A time-based intervention to treat impulsivity in male and female rats

Time-based interventions have emerged as promising treatments for disorders associated with impulsivity. These interventions can be implemented to test their efficacy in preventing or treating impulsive choice in animal models of diseases related to impulsivity such as drug abuse. Impulsive choice is typically defined as choosing a smaller-sooner (SS) reward over a larger-later (LL) reward when the LL is relatively more optimal. Previous research has shown that these interventions promote LL choices in males and females, but sex differences have not been assessed. Because sex differences can complicate the application of therapies, it is critical to compare the effects of the intervention in males and females. The intervention group received exposure to 10-s and 30-s interval schedules, and the control rats received no delay to reward. Different impulsive choice tasks were used to assess the intervention efficacy across the two experiments. Following the intervention, reductions in impulsive choice were found in male and female rats, but the degree of improvement was inconsistent across sex and task. Bayesian analyses that combined the results revealed robust evidence of an overall intervention effect with the intervention group showing greater self-control, but there was no evidence for the intervention affecting males and females differently. Taken together, these results suggest that time-based interventions are effective tools to treat impulsivity in both males and females and offer promising translational capability to humans.

Linking Delay Discounting and Substance Use Disorders: Genotypes and Phenotypes

Research supports the idea that "delay discounting," also known as temporal discounting, intertemporal choice, or impulsive choice, is a transdisease process with a strong connection to substance use disorders (SUDs) and other psychopathologies, like attention deficit hyperactivity disorder and depression. This article briefly reviews the evidence used to conclude that delay discounting is heritable and should be considered to be an endophenotype, as well as evidence of its behavioral and genetic associations with SUDs. It also discusses the limitations that should be considered when evaluating the strength of these associations. Finally, this article briefly describes research examining relationships among delay discounting and SUD-associated intermediate phenotypes to better understand the conceptual relationships underlying the links between SUDs and delay discounting, and identifies research gaps that should be addressed.

Reducing impulsive choice: VI. Delay-exposure training reduces aversion to delay-signaling stimuli

Delay-exposure (DE) training consistently and robustly reduces impulsive choice in rats, but the behavioral mechanisms behind this effect are not yet understood. The present study evaluated if DE training works by mitigating aversion to delay-signaling stimuli-those encountered when rats chose the larger-later reward in impulsive choice assessments. Fifty-seven rats were randomly assigned to 120 days of training with delayed reinforcement, training with immediate reinforcement (IE), or to a no-training Control group. Consistent with prior experiments, DE rats made significantly fewer impulsive choices than IE or Control rats. Subsequently, in a separate assessment of delay aversion, rats were given the opportunity to press a lever to temporarily escape from stimuli correlated with long or short time-intervals to food. When these escape opportunities terminated delay-signaling stimuli in the impulsive-choice task, DE rats escaped significantly less than IE and Control rats. When escapes terminated FI-signaling stimuli (a procedure in which there is no response-reinforcer delay), the difference only approached significance. These results support the hypothesis that DE training reduces impulsive choice, in part, by reducing aversion to delay-signaling stimuli. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Changing Delay Discounting and Impulsive Choice: Implications for Addictions, Prevention, and Human Health

Delay discounting describes the tendency to devalue delayed consequences or future prospects. The degree to which an individual discounts delayed events appears trait-like in that it is stable over time and across functionally similar situations. Steeply discounting delayed rewards is correlated with most substance-use disorders, the severity of these disorders, rates of relapse to drug use, and a host of other maladaptive decisions impacting human health. Longitudinal data suggest steep delay discounting and high levels of impulsive choice are predictive of subsequent drug taking, which suggests (though does not establish) that reducing delay discounting could have a preventive health-promoting effect. Experimental manipulations that produce momentary or long-lasting reductions in delay discounting or impulsive choice are reviewed, and behavioral mechanisms that may underlie these effects are discussed. Shortcomings of each manipulation technique are discussed and areas for future research are identified. While much work remains, it is clear that impulsive decision-making can be reduced, despite its otherwise trait-like qualities. Such findings invite technique refinement, translational research, and hope.

Dietary effects on the determinants of food choice: Impulsive choice, discrimination, incentive motivation, preference, and liking in male rats

The current study sought to understand how long-term exposure to diets high in saturated fat and refined sugar affected impulsive choice behavior, discrimination abilities, incentive motivation, food preferences, and liking of fat and sugar in male rats. The results showed that 8 weeks of dietary exposure impaired impulsive choice behavior; rats exposed to diets high in processed fat or sugar were more sensitive to changes in delay, a marker of impulsivity. For the high-fat group, these deficits in impulsive choice may stem from poor time discrimination, as their performance was impaired on a temporal discrimination task. The high-fat group also showed reduced magnitude sensitivity in the impulsive choice task, and they earned fewer rewards during lever press training indicating potentially reduced incentive motivation. The high-fat group also developed a preference for high-fat foods compared to the chow and high-sugar group who both preferred sugar. In contrast, dietary exposure did not alter the liking of fat or sugar as measured by a taste reactivity task. Together, the results suggest that the alterations in impulsive choice, time discrimination, incentive motivation, and food preferences induced by consumption of a high-fat diet could make individuals vulnerable to overeating, and thus obesity.

Interventions aimed at changing impulsive choice in rats: Effects of immediate and relatively long delay to reward training

A relatively strong preference for smaller-sooner rewards (SSR) over larger-later rewards (LLR) is associated with a host of maladaptive behavioral patterns. As such, the clinical implications for increasing preference for LLR are profound. There is a growing body of literature that suggests extended exposure to delayed reward may increase preference for LLR in rats. However, questions remain about the underlying mechanism driving this effect and the extent to which extended exposure to immediate rewards may decrease LLR choice. In Experiment 1, we tested effects of a differential-reinforcement-of-low-rates schedule (DRL) to increase LLR choice using a pretest/posttest design with Wistar rats as subjects. We compared this group to a group of rats exposed to a differential-reinforcement-of-high-rates schedule (DRH). The DRH intervention has never been employed in this research context, but explicitly programs an immediate response-reinforcement requirement. In Experiment 2, we tested effects of an intervention with a delay longer than those used in the delay discounting pretest and posttest. No previous research has tested effects of an intervention delay this long, relative to the delay discounting task. We compared this group to a group exposed to a delay that was part of the delay discounting pretest and posttest and to a group exposed to a traditional no-delay, fixed-ratio (FR) 2 control intervention. In both experiments, we found that exposure to delayed rewards in the intervention phase significantly increased LLR choice relative to pretest performance. These findings replicate and extend a growing body of literature showing that delay exposure increases preference for LLR. We also found significant decreases in LLR choice from pretest to posttest in the DRH and no-delay intervention groups in Experiments 1 and 2, respectively. This is the first report of such an effect and has implications for understanding and interpreting effects of delay exposure training in past and future research. Our results also suggested no relationship between improved temporal tracking of reward and increases in LLR choice as a result of delay exposure training.

Experimental reductions of delay discounting and impulsive choice: A systematic review and meta-analysis

Many behaviors posing significant risks to public health are characterized by repeated decisions to forego better long-term outcomes in the face of immediate temptations. Steeply discounting the value of delayed outcomes often underlies a pattern of impulsive choice. Steep delay discounting is correlated with addictions (e.g., substance abuse, obesity) and behaviors such as seatbelt use and risky sexual activity. As evidence accumulates suggesting steep delay discounting plays a causal role in these maladaptive behaviors, researchers have begun testing methods for reducing discounting. In this first systematic and comprehensive review of this literature, the findings of 92 articles employing different methodologies to reduce discounting are evaluated narratively and meta-analytically. Although most of the methods reviewed produced significant reductions in discounting, they varied in effect sizes. Most methods were ideal for influencing one-off choices (e.g., framing and priming manipulations), although other successful manipulations, such as episodic future thinking, could be incorporated into existing therapies designed to produce longer-lasting changes in decision-making. The largest and longest-lasting effects were produced by learning-based manipulations, although translational research is needed to determine the generality and clinical utility of these methods. Methodological shortcomings in the existing literature and suggestions for ameliorating these issues are discussed. This review reveals a variety of methods with translational potential, which, through continued refinement, may prove effective in reducing impulsive choice and its associated maladaptive decisions that negatively impact quality of life. (PsycINFO Database Record

Females in the forefront: time-based intervention effects on impulsive choice and interval timing in female rats

Impulsive choice has been implicated in substance abuse, gambling, obesity, and other maladaptive behaviors. Deficits in interval timing may increase impulsive choices, and therefore, could serve as an avenue through which suboptimal impulsive choices can be moderated. Temporal interventions have successfully attenuated impulsive choices in male rats, but the efficacy of a temporal intervention has yet to be assessed in female rats. As such, this experiment examined timing and choice behavior in female rats, and evaluated the ability of a temporal intervention to mitigate impulsive choice behavior. The temporal intervention administered in this study was successful in reducing impulsive choices compared to a control group. Results of a temporal bisection task indicated that the temporal intervention increased long responses at the shorter durations. Further, results from the peak trials within the choice task combined with the progressive interval task suggest that the intervention increased sensitivity to delay and enhanced timing confidence. Overall, these results indicate that a temporal intervention can be a successful avenue for reducing impulsive choice behavior in female rats, and could contribute to the development of behavioral interventions to prevent impulsive choice and maladaptive behaviors that can be applied to both sexes.

5-HT1a Receptor Involvement in Temporal Memory and the Response to Temporal Ambiguity

It has previously been demonstrated that rats trained on the peak-interval procedure to associate two different cues with two different fixed interval schedules will generate a scalar peak function at an intermediate time when presented with the compound cue. This response pattern has been interpreted as resulting from the simultaneous retrieval of different temporal memories, and a consequential averaging process to resolve the ambiguity. In the present set of studies, we investigated the role that serotonin 1a receptors play in this process. In Experiment 1, rats were trained on a peak-interval procedure to associate the interoceptive states induced by saline and the 5-HT1a agonist, 8-OH-DPAT, with a 5 s or 20 s fixed-interval schedule signaled by the same tone cue (counter-balanced). While peak functions following administration of saline were centered at the appropriate time (5 s or 20 s), peak functions following administration of the agonist were centered around 7 s, irrespective of the reinforced time during training, suggesting agonist-induced disruption in selective temporal memory retrieval, resulting in increased ambiguity regarding the appropriate time at which to respond. In Experiment 2, rats were trained in a peak-interval procedure to associate a tone cue with a 10 s fixed interval and a light cue with a 20 s fixed interval. Administration of the 5-HT1a antagonist, WAY-100635, had no impact on timing when single cues were presented, but altered the intermediate, scalar, response to the stimulus compound, suggesting antagonist-induced disruption in the processes used to deal with temporal memory ambiguity. Together, these data suggest that manipulations of 5HT transmission at the 5-HT1a receptor cause changes in the temporal pattern of responding that are consistent with alterations in temporal memory processes and responses to temporal ambiguity.

Reducing impulsive choice: V. The role of timing in delay-exposure training

Impulsive decision-making is common in addiction-related disorders, with some research suggesting it plays a causal role in their development. Therefore, reducing impulsive decision-making may prevent or reduce addiction-related behaviors. Recent research shows that prolonged experience with response-contingent delayed reward (delay exposure [DE] training) reduces impulsive choice in rats, but it is unclear what behavioral mechanisms underlie this effect. The present study evaluated whether improvements in interval timing mediate the effects of DE training on impulsive choice. Thirty-nine Long-Evans rats were randomly assigned to groups completing DE, immediacy-, or no-exposure training, followed by impulsive-choice and timing tasks (temporal bisection). Despite replicating the DE effect on impulsive choice, timing accuracy and precision were unaffected by DE training and unrelated to impulsive choice. The present findings did not replicate previous reports that timing precision and impulsive choice are related, which may be due to between-laboratory differences in impulsive choice tasks. Continued research to identify candidate behavioral mechanisms of DE training may assist in improving training efficacy and facilitating translational efforts.

Differential reinforcement of low rates differentially decreased timing precision

Timing processes have been implicated as potential mechanisms that underlie self-controlled choice. To investigate the impact of an intervention that has been shown to increase self-controlled choice on timing processes, accuracy and precision of temporal discrimination were assessed in an 18-s peak procedure (18-s fixed interval trials; 54-s peak trials). During an intervention phase, mice in three treatment groups experienced differential reinforcement of low rate (DRL) schedules of reinforcement of 27 s, 18 s, or 9 s. A fourth group received continued exposure to the peak procedure. After the DRL intervention, timing was reassessed using the peak procedure. In contrast to previous reports, the DRL intervention resulted in less precise timing as indicated by increased peak spread and disrupted single-trial measures of temporal control. These effects were only detected just after the DRL intervention suggesting a transient effect of DRL exposure on timing. The increase in peak spread in the present experiment suggests delay exposure via DRL schedules may produce a "dose-dependent" effect on temporal discrimination, which may also increase self-controlled choice.

Durability and generalizability of time-based intervention effects on impulsive choice in rats

Impulsive choice involves choosing a smaller-sooner (SS) reward over a larger-later (LL) reward. Due to the importance of timing processes in impulsive choice, time-based interventions have been developed to decrease impulsive choice. The present set of experiments assessed the durability and generalizability of time-based interventions. Experiment 1 assessed fixed interval (FI) or variable interval (VI) intervention efficacy over 9 months. The FI intervention decreased impulsive choice, and this effect persisted over time, but the VI intervention effects were only apparent when tested immediately after the intervention. Experiment 2 examined the generalizability of the FI and VI interventions on choice tasks manipulating the SS delay, LL delay, or LL magnitude. The FI intervention decreased sensitivity to delay, promoting LL choices in both delay tasks, but the VI intervention only altered choices when manipulating the SS delay. Experiment 3 further examined the FI intervention effects on tasks that manipulated the LL delay or magnitude immediately following the intervention. The intervention decreased sensitivity to both delay and magnitude. The experiments indicate that the FI intervention is effective at decreasing impulsive choice behavior for an extended period across changing delays and magnitudes, suggesting a relatively broad effect on choice behavior.

Discounting: A practical guide to multilevel analysis of choice data

Multilevel modeling provides the ability to simultaneously evaluate the discounting of individuals and groups by examining choices between smaller sooner and larger later rewards. A multilevel logistic regression approach is advocated in which sensitivity to relative reward magnitude and relative delay are considered as separate contributors to choice. Examples of how to fit choice data using multilevel logistic models are provided to help researchers in the adoption of these methods.

Nucleus accumbens core lesions induce sub-optimal choice and reduce sensitivity to magnitude and delay in impulsive choice tasks

The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making.

Decline of prefrontal cortical-mediated executive functions but attenuated delay discounting in aged Fischer 344 × brown Norway hybrid rats

Despite the fact that prefrontal cortex (PFC) function declines with age, aged individuals generally show an enhanced ability to delay gratification, as evident by less discounting of delayed rewards in intertemporal choice tasks. The present study was designed to evaluate relationships between 2 aspects of PFC-dependent cognition (working memory and cognitive flexibility) and intertemporal choice in young (6 months) and aged (24 months) Fischer 344 × brown Norway F1 hybrid rats. Rats were also evaluated for motivation to earn rewards using a progressive ratio task. As previously reported, aged rats showed attenuated discounting of delayed rewards, impaired working memory, and impaired cognitive flexibility compared with young. Among aged rats, greater choice of delayed reward was associated with preserved working memory, impaired cognitive flexibility, and less motivation to work for food. These relationships suggest that age-related changes in PFC and incentive motivation contribute to variance in intertemporal choice within the aged population. Cognitive impairments mediated by PFC are unlikely, however, to fully account for the enhanced ability to delay gratification that accompanies aging.

Diet-induced impulsivity: Effects of a high-fat and a high-sugar diet on impulsive choice in rats

Impulsive choice is a common charactertistic among individuals with gambling problems, obesity, and substance abuse issues. Impulsive choice has been classified as a trans-disease process, and understanding the etiology of trait impulsivity could help to understand how diseases and disorders related to impulsive choice are manifested. The Western diet is a possible catalyst of impulsive choice as individuals who are obese and who eat diets high in fat and sugar are typically more impulsive. However, such correlational evidence is unable to discern the direction and causal nature of the relationship. The present study sought to determine how diet may directly contribute to impulsive choice. After 8 weeks of dietary exposure (high-fat, high-sugar, chow), the rats were tested on an impulsive choice task, which presented choices between a smaller-sooner reward (SS) and a larger-later reward (LL). Then, the rats were transferred to a chow diet and retested on the impulsive choice task. The high-sugar and high-fat groups made significantly more impulsive choices than the chow group. Both groups became more self-controlled when they were off the diet, but there were some residual effects of the diet on choice behavior. These results suggest that diet, specifically one high in processed fat or sugar, induces impulsive choice. This diet-induced impulsivity could be a precursor to other disorders that are characterized by impulsivity, such as diet-induced obesity, and could offer potential understanding of the trans-disease nature of impulsive choice.

Differential effects of social and novelty enrichment on individual differences in impulsivity and behavioral flexibility

Early life experience profoundly impacts behavior and cognitive functions in rats. The present study investigated how the presence of conspecifics and/or novel objects, could independently influence individual differences in impulsivity and behavioral flexibility. Twenty-four rats were reared in an isolated condition, an isolated condition with a novel object, a pair-housed social condition, or a pair-housed social condition with a novel object. The rats were then tested on an impulsive choice task, a behavioral flexibility task, and an impulsive action task. Novelty enrichment produced an overall increase in impulsive choice, while social enrichment decreased impulsive choice in the absence of novelty enrichment and also produced an overall increase in impulsive action. In the behavioral flexibility task, social enrichment increased regressive errors, whereas both social and novelty enrichment reduced never-reinforced errors. Individual differences analyses indicated a significant relationship between performance in the behavioral flexibility and impulsive action tasks, which may reflect a common psychological correlate of action inhibition. Moreover, there was a relationship between delay sensitivity in the impulsive choice task and performance on the DRL and behavioral flexibility tasks, suggesting a dual role for timing and inhibitory processes in driving the interrelationship between these tasks. Overall, these results indicate that social and novelty enrichment produce distinct effects on impulsivity and adaptability, suggesting the need to parse out the different elements of enrichment in future studies. Further research is warranted to better understand how individual differences in sensitivity to enrichment affect individuals' interactions with and the resulting consequences of the rearing environment.

Impulsive Rats Exhibit Blunted Dopamine Release Dynamics during a Delay Discounting Task Independent of Cocaine History

The inability to wait for a large, delayed reward when faced with a small, immediate one, known as delay discounting, has been implicated in a number of disorders including substance abuse. Individual differences in impulsivity on the delay discounting task are reflected in differences in neural function, including in the nucleus accumbens (NAc) core. We examined the role of a history of cocaine self-administration, as well as individual differences in impulsivity, on rapid dopamine (DA) release dynamics in the NAc core. Rats with a history of cocaine or water/saline self-administration were tested on delay discounting while being simultaneously assayed for rapid DA release using electrochemical methods. In controls, we found that cue DA release was modulated by reward delay and magnitude, consistent with prior reports. A history of cocaine had no effect on either delay discounting or DA release dynamics. Nonetheless, independent of drug history, individual differences in impulsivity were related to DA release in the NAc core. First, high impulsive animals exhibited dampened cue DA release during the delay discounting task. Second, reward delay and magnitude in high impulsive animals failed to robustly modulate changes in cue DA release. Importantly, these two DAergic mechanisms were uncorrelated with each other and, together, accounted for a high degree of variance in impulsive behavior. Collectively, these findings demonstrate two distinct mechanisms by which rapid DA signaling may influence impulsivity, and illustrate the importance of NAc core DA release dynamics in impulsive behavior.

A framework for understanding and advancing intertemporal choice research using rodent models

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit. Even though great advances have been made, our understanding of the neurobiology of intertemporal choice is still in its infancy because of the complex and dynamic nature of this form of decision making. We close by briefly discussing recommendations for the future study of intertemporal choice research.

The effects of a time-based intervention on experienced middle-aged rats

Impulsive behavior is a common symptom in Attention Deficit Hyperactivity Disorder, schizophrenia, drug abuse, smoking, obesity and compulsive gambling. Stable levels of impulsive choice have been found in humans and rats and a recent study reported significant test-retest reliability of impulsive choice behavior after 1 and 5 months in rats. Time-based behavioral interventions have been successful in decreasing impulsive choices. These interventions led to improvements in the ability to time and respond more appropriately to adventitious choices. The current study examined the use of a time-based intervention in experienced, middle-aged rats. This intervention utilized a variable interval schedule previously found to be successful in improving timing and decreasing impulsive choice. This study found that the intervention led to a decrease in impulsive choices and there was a significant correlation between the improvement in self-control and post-intervention temporal precision in middle-aged rats. Although there were no overall group differences in bisection performance, individual differences were observed, suggesting an improvement in timing. This is an important contribution to the field because previous studies have utilized only young rats and because previous research indicates a decrease in general timing abilities with age.

Reward Contrast Effects on Impulsive Choice and Timing in Rats

Despite considerable interest in impulsive choice as a predictor of a variety of maladaptive behaviors, the mechanisms that drive choice behavior are still poorly understood. The present study sought to examine the influence of one understudied variable, reward magnitude contrast, on choice and timing behavior as changes in magnitude commonly occur within choice procedures. In addition, assessments of indirect effects on choice behavior through magnitude-timing interactions were assessed by measuring timing within the choice task. Rats were exposed to choice procedures composed of different pairs of magnitudes of rewards for either the smaller-sooner (SS) or larger-later (LL) option. In Phase 2, the magnitude of reward either increased or decreased by 1 pellet in different groups (LL increase = 1v1→1v2; SS decrease = 2v2 → 1v2; SS increase = 1v2 → 2v2), followed by a return to baseline in Phase 3. Choice behavior was affected by the initial magnitudes experienced in the task, demonstrating a strong anchor effect. The nature of the change in magnitude affected choice behavior as well. Timing behavior was also affected by the reward contrast manipulation albeit to a lesser degree and the timing and choice effects were correlated. The results suggest that models of choice behavior should incorporate reinforcement history, reward contrast elements, and magnitude-timing interactions, but that direct effects of reward contrast on choice should be given more weight than the indirect reward-timing interactions. A better understanding of the factors that contribute to choice behavior could supply key insights into this important individual differences variable.

I can't wait: Methods for measuring and moderating individual differences in impulsive choice

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

The effect of acute morphine on delay discounting in dependent and non-dependent rats

RATIONALE

Chronic opiate use is associated with increased impulsivity in both humans and animals, and previous studies suggest that acute morphine can increase impulsivity in non-dependent rats. However, the extent to which chronic opiate usage modulates the effect of acute morphine is unknown.

METHODS

Rats were trained to delay discount 20 % sucrose solution and then randomly assigned to either a dependent group that received a nightly 30 mg/kg subcutaneous dose of morphine or a non-dependent group that received a nightly saline injection. Once dependence was established, rats were then assigned to one of four acute morphine doses (0, 1.25, 2.5, 5 mg/kg). For 5 days, delay discounting curves were determined 22.5 h after maintenance doses and 1 h after their prescribed acute injections.

RESULTS

In non-dependent rats, 2.5 and 5 mg/kg doses of morphine caused decreased preference for the large reward at all delays. Acute morphine had no effect on discounting curves in dependent rats.

CONCLUSIONS

Morphine dependence can cause tolerance to the effects of acute morphine on delay discounting.

Mechanisms of impulsive choice: II. Time-based interventions to improve self-control

Impulsive choice behavior has been proposed as a primary risk factor for other maladaptive behaviors (e.g., gambling, substance abuse). Recent research has suggested that timing processes may play a key role in impulsive choice behavior, and could provide an avenue for altering impulsive choice. Accordingly, the current experiments assessed a set of time-based behavioral interventions to increase self-control while simultaneously assessing effects on timing processes within the impulsive choice task. Three experiments assessed temporal interventions using a differential reinforcement of low rates task (Experiment 1) and exposure to either a variable or fixed interval schedule (Experiments 2-3). The efficacy of the interventions was assessed in Sprague-Dawley (Experiments 1-2) and Lewis (Experiment 3) rat strains. Impulsive choice behavior was assessed by measuring preferences of a smaller-sooner (SS) versus a larger-later (LL) reward, while timing of the SS and LL durations was measured during peak trials within the impulsive choice procedure. The rats showed an increased preference for the LL following all three time-based interventions and also displayed increased temporal precision. These results add to the increasing evidence that supports a possible role for temporal processing in impulsive choice behavior and supply novel behavioral interventions to decrease impulsive behavior.

Relative gains, losses, and reference points in probabilistic choice in rats

Theoretical reference points have been proposed to differentiate probabilistic gains from probabilistic losses in humans, but such a phenomenon in non-human animals has yet to be thoroughly elucidated. Three experiments evaluated the effect of reward magnitude on probabilistic choice in rats, seeking to determine reference point use by examining the effect of previous outcome magnitude(s) on subsequent choice behavior. Rats were trained to choose between an outcome that always delivered reward (low-uncertainty choice) and one that probabilistically delivered reward (high-uncertainty). The probability of high-uncertainty outcome receipt and the magnitudes of low-uncertainty and high-uncertainty outcomes were manipulated within and between experiments. Both the low- and high-uncertainty outcomes involved variable reward magnitudes, so that either a smaller or larger magnitude was probabilistically delivered, as well as reward omission following high-uncertainty choices. In Experiments 1 and 2, the between groups factor was the magnitude of the high-uncertainty-smaller (H-S) and high-uncertainty-larger (H-L) outcome, respectively. The H-S magnitude manipulation differentiated the groups, while the H-L magnitude manipulation did not. Experiment 3 showed that manipulating the probability of differential losses as well as the expected value of the low-uncertainty choice produced systematic effects on choice behavior. The results suggest that the reference point for probabilistic gains and losses was the expected value of the low-uncertainty choice. Current theories of probabilistic choice behavior have difficulty accounting for the present results, so an integrated theoretical framework is proposed. Overall, the present results have implications for understanding individual differences and corresponding underlying mechanisms of probabilistic choice behavior.

Mechanisms of Individual Differences in Impulsive and Risky Choice in Rats

Individual differences in impulsive and risky choice are key risk factors for a variety of maladaptive behaviors such as drug abuse, gambling, and obesity. In our rat model, ordered individual differences are stable across choice parameters, months of testing, and span a broad spectrum, suggesting that rats, like humans, exhibit trait-level impulsive and risky choice behaviors. In addition, impulsive and risky choices are highly correlated, suggesting a degree of correlation between these two traits. An examination of the underlying cognitive mechanisms has suggested an important role for timing processes in impulsive choice. In addition, in an examination of genetic factors in impulsive choice, the Lewis rat strain emerged as a possible animal model for studying disordered impulsive choice, with this strain demonstrating deficient delay processing. Early rearing environment also affected impulsive behaviors, with rearing in an enriched environment promoting adaptable and more self-controlled choices. The combined results with impulsive choice suggest an important role for timing and reward sensitivity in moderating impulsive behaviors. Relative reward valuation also affects risky choice, with manipulation of objective reward value (relative to an alternative reference point) resulting in loss chasing behaviors that predicted overall risky choice behaviors. The combined results are discussed in relation to domain-specific versus domain-general subjective reward valuation processes and the potential neural substrates of impulsive and risky choice.

Everywhere and everything: The power and ubiquity of time

Anticipatory timing plays a critical role in many aspects of human and non-human animal behavior. Timing has been consistently observed in the range of milliseconds to hours, and demonstrates a powerful influence on the organization of behavior. Anticipatory timing is acquired early in associative learning and appears to guide association formation in important ways. Importantly, timing participates in regulating goal-directed behaviors in many schedules of reinforcements, and plays a critical role in value-based decision making under concurrent schedules. In addition to playing a key role in fundamental learning processes, timing often dominates when temporal cues are available concurrently with other stimulus dimensions. Such control by the passage of time has even been observed when other cues provide more accurate information and can lead to sub-optimal behaviors. The dominance of temporal cues in governing anticipatory behavior suggests that time may be inherently more salient than many other stimulus dimensions. Discussions of the interface of the timing system with other cognitive processes are provided to demonstrate the powerful and primitive nature of time as a stimulus dimension.