Effects of reward bundling on male rats' preference for larger-later food rewards

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

A potential role of hippocampus on impulsivity and alcohol consumption through CB1R

There are a few studies suggesting that the hippocampus is involved in the regulation of impulsivity, and which attempt to explain drug seeking behavior in addiction. In addition, cannabinoid receptor 1 (CB1R) is highly expressed in the hippocampus (HPP). To further understand the potential role of the hippocampal CB1R in impulsive and drug seeking behaviors, we characterized impulsivity in adolescent and adult male rats, by means of a delay discounting task (DDT) by evaluating preference and seeking motivation for alcohol (10 % v/v) consumption, and analyzing CB1R expression in CA1, CA3 and the dentate gyrus (DG) of the HPP as well as in the medial prefrontal cortex (mPFC). Our results show that adolescent rats display more impulsive choices than adult rats in the DDT. The k value is statistically higher in adolescents, further supporting that they are more impulsive. Besides, adolescent rats have higher forced and voluntary alcohol consumption and display a higher alcohol conditioned place preference (CPP) vs. adult rats. In addition, CB1R expression in CA3 and the DG is higher in adolescent vs. adult rats. Our data further support the role of the hippocampus in impulsivity with the potential involvement of the endocannabinoid system, considering that CB1R in CA3 and DG is higher in adolescents, who display impulsivity and alcohol seeking and consumption.

Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay

Interventions exposing rats to delayed-reward contingencies attenuate suboptimal impulsive choices, a preference for a smaller-sooner (SS) over a larger-later (LL) reward. Interventions may potentially improve delay-tolerance, timing of delays, and/or discrimination of reward magnitudes. Generalization from the intervention to impulsive choice under different procedures can provide insights into the processes that underlie the intervention effects. Experiment 1 tested intervention effects on systematic-delay (SYS) and adjusting-delay (ADJ) procedures, predicting that intervention effects would be more effective on the SYS procedure with predictable delays. The ADJ procedure did not benefit significantly from intervention, but the SYS procedure, unexpectedly, showed greater impulsive choices following intervention. Experiment 2 tested whether short (5 s) SS intervention delays may have promoted greater impulsivity in the SYS impulsive choice procedure in Experiment 1. Short SS delays in choice and intervention procedures increased impulsive choices in comparison to longer (10 s) delays. Incongruent SS delays in the intervention/choice procedures resulted in negative intervention effects. The results suggest that short SS delays are detrimental to self-control and that specific temporal information generalizes from the intervention to the SYS choice task, but not the ADJ choice task.

Delaying Gratification's role in the relationship between facets of mindfulness and substance use outcomes

The present study examined whether five facets of mindfulness (acting with awareness, describing, observing, non-judging of inner experience, and non-reactivity to inner experience) relate to alcohol and cannabis outcomes (quantity of use and negative consequences) via delaying gratification. Participants were college students across seven universities who used alcohol (n = 1467) and/or cannabis (n = 647) in the last 30 days. For the alcohol analytic sample, the majority of students identified as being female (71.1%) and reported a mean age of 20.04 (Median = 19.00; SD = 3.47) years. For the cannabis analytic sample, the majority of students identified as being female (69.7%) and reported a mean age of 19.63 (Median = 19.00; SD = 2.85). Within our model, significant indirect effects were only found for two mindfulness facets. Specifically, the indirect effects from acting with awareness (alcohol indirect β = -0.026, SE = 0.006; cannabis indirect β = -0.019, SE = 0.008) and observing (alcohol indirect β = -0.017, SE = 0.005; cannabis indirect β = -0.015, SE = 0.007) to consequences via delaying gratification and quantity were significant for both the alcohol and cannabis models, such that acting with awareness and observing were associated with higher delaying gratification; which in turn was associated with lower quantity of use, which in turn was associated with fewer consequences for both the cannabis and alcohol models. Our findings suggest that delaying gratification is important to understanding the relationship between specific mindfulness facets (i.e., acting with awareness and observing) and alcohol and cannabis outcomes. Mindfulness interventions aimed at reducing substance use may benefit from targeting delaying gratification through increasing acting with awareness and observing.

Choice Bundling Increases Valuation of Delayed Losses More Than Gains in Cigarette Smokers

Choice bundling, in which a single choice produces a series of repeating consequences over time, increases valuation of delayed monetary and non-monetary gains. Interventions derived from this manipulation may be an effective method for mitigating the elevated delay discounting rates observed in cigarette smokers. No prior work, however, has investigated whether the effects of choice bundling generalize to reward losses. In the present study, an online panel of cigarette smokers (N = 302), recruited using survey firms Ipsos and InnovateMR, completed assessments for either monetary gains or losses (randomly assigned). In Step 1, participants completed a delay-discounting task to establish Effective Delay 50 (ED50), or the delay required for an outcome to lose half of its value. In Step 2, participants completed three conditions of an adjusting-amount task, choosing between a smaller, sooner (SS) adjusting amount and a larger, later (LL) fixed amount. The bundle size (i.e., number of consequences) was manipulated across conditions, where a single choice produced either 1 (control), 3, or 9 consequences over time (ascending/descending order counterbalanced). The delay to the first LL amount in each condition, as well as the intervals between all additional SS and LL amounts (where applicable), were set to individual participants’ ED50 values from Step 1 to control for differences in discounting of gains and losses. Results from Step 1 showed significantly higher ED50 values (i.e., less discounting) for losses compared to gains (p < 0.001). Results from Step 2 showed that choice bundling significantly increased valuation of both LL gains and losses (p < 0.001), although effects were significantly greater for losses (p < 0.01). Sensitivity analyses replicated these conclusions. Future research should examine the potential clinical utility of choice bundling, such as development of motivational interventions that emphasize both the bundled health gains associated with smoking cessation and the health losses associated with continued smoking.

Choice bundling, unpacked: Observed and predicted effects on intertemporal choice in an additive model of hyperbolic delay discounting

One method known to increase preference for larger, later rewards (LLRs) over smaller, sooner rewards (SSRs) is choice bundling, in which a single choice produces a series of repeating consequences over time. The present study examined whether effects of choice bundling on preference for LLRs: (1) increase with the number of rewards in the bundle (i.e., bundle size); (2) are independent of differences in reward magnitude between conditions; and (3) accord with predictions of an additive model of hyperbolic delay discounting, in which the value of a bundle of rewards can be expressed as the summed discounted value of all rewards in that bundle. Participants (N = 252) completed a choice task to assess valuation of monetary LLRs at bundle sizes of 1 (control), 3, and 9 rewards per choice (ascending/descending order counterbalanced). To control for the magnitude effect, the total reward amounts were held constant across conditions. Choice bundling significantly increased LLR preference (p < .001), with the largest effect observed at the largest bundle size. The descending bundle-size order produced significantly greater LLR preference than the ascending order (p < .05), although order did not significantly interact with bundle size. Difference scores between observed measures and those predicted by an additive model of hyperbolic discounting were small and not significantly different than zero, but were not equivalent to zero. Future research should investigate the clinical utility of choice bundling for reducing the maladaptive health behavior (e.g., substance use) with which delay discounting is associated.

Steep Discounting of Future Rewards as an Impulsivity Phenotype: A Concise Review

This chapter provides an overview over the behavioral economic index of impulsivity known as delay discounting. Specifically, delay discounting refers to an individual's preference for smaller immediate rewards over a larger delayed rewards. The more precipitously an individual discounts future rewards, the more impulsive they are considered to be. First, the chapter reviews the nature of delay discounting as a psychological process and juxtaposes it with nominally similar processes, including other facets of impulsivity. Second, the chapter reviews the links between delay discounting and numerous health behaviors, including addiction, attention deficit/hyperactivity disorder, and obesity. Third, the determinants of individual variation in delay discounting are discussed, including both genetic and environmental contributions. Finally, the chapter evaluates delay discounting as a potentially modifiable risk factor and the status of clinical interventions designed to reduce delay discounting to address deficits in self-control in a variety of maladaptive behaviors.

A brief review of choice bundling: A strategy to reduce delay discounting and bolster self-control

Choice bundling is a behavioral economic strategy designed to reduce excessive delay discounting and bolster self-control. Choice bundling entails aggregating a series of individual, identical intertemporal decisions (e.g., should I smoke today?) into a single choice (e.g., should I smoke this month?). In this brief review, we succinctly summarize delay discounting and how it has been linked to lapses in self-control, using substance use as an exemplar. Next, we describe how choice bundling may theoretically work to counter excess discounting rates. Finally, we review the extant empirical research on choice bundling and offer recommendations for future research.

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Changing Delay Discounting and Impulsive Choice: Implications for Addictions, Prevention, and Human Health

Delay discounting describes the tendency to devalue delayed consequences or future prospects. The degree to which an individual discounts delayed events appears trait-like in that it is stable over time and across functionally similar situations. Steeply discounting delayed rewards is correlated with most substance-use disorders, the severity of these disorders, rates of relapse to drug use, and a host of other maladaptive decisions impacting human health. Longitudinal data suggest steep delay discounting and high levels of impulsive choice are predictive of subsequent drug taking, which suggests (though does not establish) that reducing delay discounting could have a preventive health-promoting effect. Experimental manipulations that produce momentary or long-lasting reductions in delay discounting or impulsive choice are reviewed, and behavioral mechanisms that may underlie these effects are discussed. Shortcomings of each manipulation technique are discussed and areas for future research are identified. While much work remains, it is clear that impulsive decision-making can be reduced, despite its otherwise trait-like qualities. Such findings invite technique refinement, translational research, and hope.

Experimental reductions of delay discounting and impulsive choice: A systematic review and meta-analysis

Many behaviors posing significant risks to public health are characterized by repeated decisions to forego better long-term outcomes in the face of immediate temptations. Steeply discounting the value of delayed outcomes often underlies a pattern of impulsive choice. Steep delay discounting is correlated with addictions (e.g., substance abuse, obesity) and behaviors such as seatbelt use and risky sexual activity. As evidence accumulates suggesting steep delay discounting plays a causal role in these maladaptive behaviors, researchers have begun testing methods for reducing discounting. In this first systematic and comprehensive review of this literature, the findings of 92 articles employing different methodologies to reduce discounting are evaluated narratively and meta-analytically. Although most of the methods reviewed produced significant reductions in discounting, they varied in effect sizes. Most methods were ideal for influencing one-off choices (e.g., framing and priming manipulations), although other successful manipulations, such as episodic future thinking, could be incorporated into existing therapies designed to produce longer-lasting changes in decision-making. The largest and longest-lasting effects were produced by learning-based manipulations, although translational research is needed to determine the generality and clinical utility of these methods. Methodological shortcomings in the existing literature and suggestions for ameliorating these issues are discussed. This review reveals a variety of methods with translational potential, which, through continued refinement, may prove effective in reducing impulsive choice and its associated maladaptive decisions that negatively impact quality of life. (PsycINFO Database Record

'Waiting impulsivity' in isolation-reared and socially-reared rats: effects of amphetamine

BACKGROUND

Rats reared in social isolation exhibit various cognitive and behavioural abnormalities in adulthood. However, impulsivity following this treatment still remains unclear, especially in response to medications used in attention deficit hyperactivity disorder, such as amphetamine.

METHODS

Using an isolation-rearing (IR) manipulation, the present study examined the effects of IR on impulsive action and impulsive choice when also treated with doses of D-amphetamine, by employing the five-choice serial reaction time task (5-CSRTT) and a temporal discounting of reward task (TDRT), respectively.

RESULTS

IR rats showed similar acquisition of the 5-CSRTT. Amphetamine increased premature responding in both groups; however, IR rats showed less responding overall. For the TDRT, IR rats revealed a greater preference for the large but delayed reward during task acquisition (i.e. were less impulsive) with a higher rate of nose poking during the delay, and exhibited a compressed dose-response function (i.e. reduced dose sensitivity) for amphetamine.

DISCUSSION

Impulsive action and impulsive choice were reduced in IR rats under certain conditions, and a blunted response to D-amphetamine was found on these measures. These reductions in impulsivity contrast with locomotor hyperactivity normally shown in IR rats and the findings have implications for the utility of IR as a model of psychopathology.

Does impulsivity change rate dependently following stimulant administration? A translational selective review and re-analysis

RATIONALE

Rate dependence refers to an orderly relationship between a baseline measure of behavior and the change in that behavior following an intervention. The most frequently observed rate-dependent effect is an inverse relationship between the baseline rate of behavior and response rates following an intervention. A previous report of rate dependence in delay discounting suggests that the discounting of delayed reinforcers, and perhaps, other impulsivity measures, may change rate dependently following acute and chronic administration of potentially therapeutic medications in both preclinical and clinical studies.

OBJECTIVE

The aim of the current paper was to review the effects of stimulants on delay discounting and other impulsivity tasks.

METHODS

All studies identified from the literature were required to include (1) an objective measure of impulsivity; (2) administration of amphetamine, methylphenidate, or modafinil; (3) presentation of a pre- and postdrug administration impulsivity measure; and (4) the report of individual drug effects or results in groups split by baseline or vehicle impulsivity. Twenty-five research reports were then reanalyzed for evidence consistent with rate dependence.

RESULTS

Of the total possible instances, 67 % produced results consistent with rate dependence. Specifically, 72, 45, and 80 % of the data sets were consistent with rate dependence following amphetamine, methylphenidate, and modafinil administration, respectively.

CONCLUSIONS

These results suggest that rate dependence is a more robust phenomenon than reported in the literature. Impulsivity studies should consider this quantitative signature as a process to determine the effects of variables and as a potential prognostic tool to evaluate the effectiveness of future interventions.

I can't wait: Methods for measuring and moderating individual differences in impulsive choice

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Impulsive delayed reward discounting as a genetically-influenced target for drug abuse prevention: a critical evaluation

This review evaluates the viability of delayed reward discounting (DRD), an index of how much an individual devalues a future reward based on its delay in time, for genetically-informed drug abuse prevention. A review of the literature suggests that impulsive DRD is robustly associated with drug addiction and meets most of the criteria for being an endophenotype, albeit with mixed findings for specific molecular genetic influences. Several modes of experimental manipulation have been demonstrated to reduce DRD acutely. These include behavioral strategies, such as mindfulness, reward bundling, and episodic future thinking; pharmacological interventions, including noradrenergic agonists, adrenergic agonists, and multiple monoamine agonists; and neuromodulatory interventions, such as transcranial magnetic stimulation and transcranial direct current stimulation. However, the generalization of these interventions to positive clinical outcomes remains unclear and no studies to date have examined interventions on DRD in the context of prevention. Collectively, these findings suggest it would be premature to target DRD for genetically-informed prevention. Indeed, given the evidence of environmental contributions to impulsive DRD, whether genetically-informed secondary prevention would ever be warranted is debatable. Progress in identifying polymorphisms associated with DRD profiles could further clarify the underlying biological systems for pharmacological and neuromodulatory interventions, and, as a qualitatively different risk factor from existing prevention programs, impulsive DRD is worthy of investigation at a more general level as a novel and promising drug abuse prevention target.

Delay-specific stimuli and genotype interact to determine temporal discounting in a rapid-acquisition procedure

The importance of delay discounting to many socially important behavior problems has stimulated investigations of biological and environmental mechanisms responsible for variations in the form of the discount function. The extant experimental research, however, has yielded disparate results, raising important questions regarding Gene X Environment interactions. The present study determined the influence of stimuli that uniquely signal delays to reinforcement on delay discounting in two inbred mouse strains using a rapid-acquisition procedure. BALB/c and C57BL/6 mice responded under a six-component, concurrent-chained schedule in which the terminal-link delays preceding the larger-reinforcer were presented randomly across components of an individual session. Across conditions, components were presented either with or without delay-specific auditory stimuli, i.e., as multiple or mixed schedules. A generalized matching-based model was used to incorporate the impact of current and previous component reinforcer-delay ratios on current component response allocation. Sensitivity to reinforcer magnitude and delay were higher for BALB/c mice, but within-component preference reached final levels faster for C57Bl/6 mice. For BALB/c mice, acquisition of preference across blocks of a component was faster under the multiple than the mixed schedule, but final levels of sensitivity to reinforcement were unaffected by schedule. The speed of acquisition of preference was not different across schedules for C57Bl/6 mice, but sensitivity to reinforcement was higher under the multiple than the mixed schedule. Overall, differences in the acquisition and final form of the discount function were determined by a Gene X Environment interaction, but the presence of delay-specific stimuli attenuated genotype-dependent differences in magnitude and delay sensitivity.

Do the adjusting-delay and increasing-delay tasks measure the same construct: delay discounting?

Delay discounting describes the subjective devaluation of a reward when it is delayed. In animals, the adjusting-delay (AD) and increasing-delay (ID) tasks often are used to assess individual differences in, and drug effects on, delay discounting. No study to date, however, has compared systematically the measures of discounting produced in these tasks. The current study examined the correlation between measures of delay discounting derived from AD and ID procedures. Twenty rats completed 30 sessions under each task (order counterbalanced across rats). Quantitative measures of delay discounting produced by the two tasks were positively correlated, suggesting that the AD and ID tasks measure the same underlying facet of impulsive choice (i.e. individual or conjoint sensitivities to reward delay and magnitude). The measures derived from either task, however, depended on the sequences in which the tasks were experienced. That is, pre-exposure to one task decreased discounting of delayed rewards in the second task. Consistent with other published findings, exposure to delayed consequences during the initial discounting assessment might explain this effect. Despite the observed correlation between ID and AD indifference delays, we suggest that the ID procedure might be a more appropriate procedure for pharmacological studies.

Ethanol self-administration in rats responding under concurrent schedules for milk or ethanol plus milk

The relative reinforcing strength of drugs can be characterized by the distribution of operant behavior during the availability of other reinforcing stimuli. 'Choice' procedures are not widely used in rats, with the exception of ethanol self-administration in which there often is a choice between ethanol and water, which typically does not maintain much responding. A procedure was developed to evaluate the relative reinforcing strength of ethanol in rats when a similar appetitive reinforcer is concurrently available. Rats were trained to respond on two levers under concurrent fixed-ratio schedules of reinforcement with milk (1-50%) or ethanol+milk (4-32% ethanol+5-10% milk). Daily 60-min sessions began with a forced sample of each reinforcer, followed by the concurrent schedules. Under this schedule, rats preferentially allocated their responding to the ethanol-associated lever under conditions of ethanol+5% milk versus 5% milk, but neither preferred nor avoided ethanol when ethanol+10% milk versus 10% milk was available. When 8% ethanol+5% milk was available, 85±6% of responses were directed toward the ethanol-associated lever and the mean ethanol intake was 1.55±0.10 g/kg. The response rate decreased monotonically with the concentration of ethanol. Naltrexone injections did not affect the distribution of responding, but slightly decreased ethanol intake. It is concluded that stable behavior can be maintained under concurrent fixed-ratio schedules of ethanol and milk presentation in rats, resulting in intake of behaviorally active amounts of ethanol.