The effect of changes in criterion value on differential reinforcement of low rate schedule performance

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Differential reinforcement of low rates differentially decreased timing precision

Timing processes have been implicated as potential mechanisms that underlie self-controlled choice. To investigate the impact of an intervention that has been shown to increase self-controlled choice on timing processes, accuracy and precision of temporal discrimination were assessed in an 18-s peak procedure (18-s fixed interval trials; 54-s peak trials). During an intervention phase, mice in three treatment groups experienced differential reinforcement of low rate (DRL) schedules of reinforcement of 27 s, 18 s, or 9 s. A fourth group received continued exposure to the peak procedure. After the DRL intervention, timing was reassessed using the peak procedure. In contrast to previous reports, the DRL intervention resulted in less precise timing as indicated by increased peak spread and disrupted single-trial measures of temporal control. These effects were only detected just after the DRL intervention suggesting a transient effect of DRL exposure on timing. The increase in peak spread in the present experiment suggests delay exposure via DRL schedules may produce a "dose-dependent" effect on temporal discrimination, which may also increase self-controlled choice.

Early Alterations in Operant Performance and Prominent Huntingtin Aggregation in a Congenic F344 Rat Line of the Classical CAGn51trunc Model of Huntington Disease

The transgenic rat model of Huntington disease expressing a fragment of mutant HTT (tgHD rat) has been thoroughly characterized and reproduces hallmark symptoms of human adult-onset HD. Pursuing the optimization of this model for evaluation of translational therapeutic approaches, the F344 inbred rat strain was considered as advantageous genetic background for the expression of the HD transgenic construct. In the present study, a novel congenic line of the SPRDtgHD transgenic model of HD, carrying 51 CAG repeats, was generated on the F344 rat genetic background. To assess the behavioral phenotype, classical assays investigating motor function, emotion, and sensorimotor gating were applied, along with automated screening of metabolic and activity parameters as well as operant conditioning tasks. The neuropathological phenotype was analyzed by immunohistochemistry and ex vivo magnetic resonance imaging. F344tgHD rats displayed markedly reduced anxiety-like behavior in the social interaction test and elevated impulsivity traits already at 3 months of age. Neuropathologically, reduced striatal volume and pronounced aggregation of mutant huntingtin in several brain regions were detected at later disease stage. In conclusion, the congenic F344tgHD model reproduces key aspects of the human HD phenotype, substantiating its value for translational therapeutic approaches.

KU32 prevents 5-fluorouracil induced cognitive impairment

Chemotherapy induced cognitive impairment (i.e. chemobrain) involves acute and long-term deficits in memory, executive function, and processing speed. Animal studies investigating these cognitive deficits have had mixed results, potentially due to variability in the complexity of behavioral tasks across experiments. Further, common chemotherapy treatments such as 5-fluorouracil (5-FU) break down myelin integrity corresponding to hippocampal neurodegenerative deficits and mitochondrial dysfunction. There is little evidence, however, of pharmacological treatments that may target mitochondrial dysfunction. Using a differential reinforcement of low rates (DRL) task combining spatial and temporal components, the current study evaluated the preventative effects of the pharmacological agent KU32 on the behavior of rats treated with 5-FU (5-FU+Saline vs. 5FU+KU32). DRL performance was analyzed the day after the first set of injections (D1), the day after the second set of injections (D7) and the last day of the experiment (D14). The 5FU+KU32 group earned significantly more reinforcers on the DRL task at D7 and D14 than the 5FU+Saline group. Further, the 5FU+KU32 group showed significantly better temporal discrimination. The 5FU+KU32 showed within-group improvement in temporal discrimination from D7 to D14. No significant differences were observed in spatial discrimination, however, those in the 5FU+Saline group responded more frequently on T3 compared to the 5FU+KU32 group, highlighting temporal discrimination differences between groups. The current data suggest that KU32 shows promise in the prevention of chemotherapy induced impairments in temporal discrimination.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Mechanisms of impulsive choice: II. Time-based interventions to improve self-control

Impulsive choice behavior has been proposed as a primary risk factor for other maladaptive behaviors (e.g., gambling, substance abuse). Recent research has suggested that timing processes may play a key role in impulsive choice behavior, and could provide an avenue for altering impulsive choice. Accordingly, the current experiments assessed a set of time-based behavioral interventions to increase self-control while simultaneously assessing effects on timing processes within the impulsive choice task. Three experiments assessed temporal interventions using a differential reinforcement of low rates task (Experiment 1) and exposure to either a variable or fixed interval schedule (Experiments 2-3). The efficacy of the interventions was assessed in Sprague-Dawley (Experiments 1-2) and Lewis (Experiment 3) rat strains. Impulsive choice behavior was assessed by measuring preferences of a smaller-sooner (SS) versus a larger-later (LL) reward, while timing of the SS and LL durations was measured during peak trials within the impulsive choice procedure. The rats showed an increased preference for the LL following all three time-based interventions and also displayed increased temporal precision. These results add to the increasing evidence that supports a possible role for temporal processing in impulsive choice behavior and supply novel behavioral interventions to decrease impulsive behavior.

Mechanisms of Individual Differences in Impulsive and Risky Choice in Rats

Individual differences in impulsive and risky choice are key risk factors for a variety of maladaptive behaviors such as drug abuse, gambling, and obesity. In our rat model, ordered individual differences are stable across choice parameters, months of testing, and span a broad spectrum, suggesting that rats, like humans, exhibit trait-level impulsive and risky choice behaviors. In addition, impulsive and risky choices are highly correlated, suggesting a degree of correlation between these two traits. An examination of the underlying cognitive mechanisms has suggested an important role for timing processes in impulsive choice. In addition, in an examination of genetic factors in impulsive choice, the Lewis rat strain emerged as a possible animal model for studying disordered impulsive choice, with this strain demonstrating deficient delay processing. Early rearing environment also affected impulsive behaviors, with rearing in an enriched environment promoting adaptable and more self-controlled choices. The combined results with impulsive choice suggest an important role for timing and reward sensitivity in moderating impulsive behaviors. Relative reward valuation also affects risky choice, with manipulation of objective reward value (relative to an alternative reference point) resulting in loss chasing behaviors that predicted overall risky choice behaviors. The combined results are discussed in relation to domain-specific versus domain-general subjective reward valuation processes and the potential neural substrates of impulsive and risky choice.

Everywhere and everything: The power and ubiquity of time

Anticipatory timing plays a critical role in many aspects of human and non-human animal behavior. Timing has been consistently observed in the range of milliseconds to hours, and demonstrates a powerful influence on the organization of behavior. Anticipatory timing is acquired early in associative learning and appears to guide association formation in important ways. Importantly, timing participates in regulating goal-directed behaviors in many schedules of reinforcements, and plays a critical role in value-based decision making under concurrent schedules. In addition to playing a key role in fundamental learning processes, timing often dominates when temporal cues are available concurrently with other stimulus dimensions. Such control by the passage of time has even been observed when other cues provide more accurate information and can lead to sub-optimal behaviors. The dominance of temporal cues in governing anticipatory behavior suggests that time may be inherently more salient than many other stimulus dimensions. Discussions of the interface of the timing system with other cognitive processes are provided to demonstrate the powerful and primitive nature of time as a stimulus dimension.

Caramiphen edisylate: an optimal antidote against organophosphate poisoning

Potent cholinesterase inhibitors such as sarin, induce an array of harmful effects including hypersecretion, convulsions and ultimately death. Surviving subjects demonstrate damage in specific brain regions that lead to cognitive and neurological dysfunctions. An early accumulation of acetylcholine in the synaptic clefts was suggested as the trigger of a sequence of neurochemical events such as an excessive outpour of glutamate and activation of its receptors. Indeed, alterations in NMDA and AMPA central receptors' densities were detected in brains of poisoned animals. Attempts to improve the current cholinergic-based treatment by adding potent anticonvulsants or antiglutamatergic drugs produced unsatisfactory results. In light of recent events in Syria and the probability of various scenarios of military or terrorist attacks involving organophosphate (OP) nerve agent, research should focus on finding markedly improved countermeasures. Caramiphen, an antimuscarinic drug with antiglutamatergic and GABAergic facilitating properties, was evaluated in a wide range of animals and experimental protocols against OP poisoning. Its remarkable efficacy against OP exposure was established both in prophylactic and post-exposure therapies in both small and large animals. The present review will highlight the outstanding neuroprotective effect of caramiphen as the optimal candidate for the treatment of OP-exposed subjects.

Aggression and aspects of impulsivity in wild-type rats

Aggression is closely related to impulsive behavior both in humans and in animals. To avoid potential negative consequences, aggressive behavior is kept in control by strong inhibitory mechanisms. Failure of these inhibitory mechanisms results in violent behavior. In the present experiments, we investigated whether aggressive behavior is related to impulsive behavior. Furthermore, we investigated if violent behavior can be distinguished from "normal" aggressive behavior in terms of impulsivity levels. We used rats of the wild-type Groningen strain, rats of this strain differ widely in their level of offensive aggression expressed toward an unfamiliar intruder male, ranging from no aggression at all to very high levels of intense and sometimes violent behavior. Violent behavior was displayed by some of the animals that were given repeated winning experience. We used behavioral performance in an unpredictable operant conditioning paradigm for food reinforcement (variable interval 15) and performance in a differential-reinforcement of low rate (DRL-60s) responding as determinants for impulsivity. We predicted that offensive aggression is correlated with behavioral flexibility measured by the VI-15 procedure and that aggressive behavior is characterized by low behavioral inhibition on the DRL task. In addition we expected that violent animals would be characterized by extremely low levels of behavioral inhibition on the DRL task. We showed that the level of offensive aggression indeed positively correlated with VI-15 performance. In addition, we showed that behavioral performance on the DRL procedure is similar in low and high aggressive rats. However, violent animals can be dissociated by a lower efficiency of lever pressing on a DRL-60s schedule of reinforcement.

Variable delay-to-signal: a fast paradigm for assessment of aspects of impulsivity in rats

Testing impulsive behavior in rodents is challenging and labor-intensive. We developed a new behavioral paradigm—the Variable Delay-to-Signal (VDS) test—that provides rapid and simultaneous assessment of response and decision impulsivity in rodents. Presentation of a light at variable delays signals the permission for action (nose poke) contingent with a reward. 2 blocks of 25 trials at 3 s delay flank a block of 70 trials in which light is presented with randomly selected 6 or 12 s delays. Exposure to such large delays boosts the rate of premature responses when the delay drops to 3 s in the final block, an effect that is blunted by an acute methamphetamine challenge and that correlates with the delay-discounting (DD) paradigm (choice impulsivity). Finally, as expected, treatment with the NMDA antagonist MK-801 caused a generalized response increase in all VDS blocks. The pharmacological validation, particularly with methamphetamine which has a well established dual effect on response and decision impulsivity, and the correlations between the impulsive behavior in the DD and VDS paradigms, suggests that the later is able to provide, in a single session, a multi-dimensional assessment of impulsive behavior.