Delay aversion but preference for large and rare rewards in two choice tasks: implications for the measurement of self-control parameters

Constitutive depletion of brain serotonin differentially affects rats' social and cognitive abilities

Central serotonin appears a promising transdiagnostic marker of psychiatric disorders and a modulator of some of their key behavioral symptoms. In adult male Tph2 ^-/- rats, constitutively lacking central serotonin, we tested individual's cognitive, social and non-social abilities and characterized group's social organization under classical and ethological testing conditions. Using unsupervised machine learning, we identified the functions most dependent on serotonin. Although serotonin depletion did not affect cognitive performances in classical testing, in the home-cage it induced compulsive aggression and sexual behavior, hyperactive and hypervigilant stereotyped behavior, reduced self-care and exacerbated corticosterone levels. This profile recalled symptoms of impulse control and anxiety disorders. Serotonin appeared essential for behavioral adaptation to dynamic social environments. Our animal model challenges the essential role of serotonin in decision-making, flexibility, impulsivity, and risk-taking. These findings highlight the importance of studying everyday life functions within the dynamic social living environment to model complexity in animal models.

Precision Behavioral Management (PBM) and Cognitive Control as a Potential Therapeutic and Prophylactic Modality for Reward Deficiency Syndrome (RDS): Is There Enough Evidence?

This brief commentary aims to provide an overview of the available and relatively new precision management of reward deficiencies manifested as substance and behavioral disorders. Current and future advances, concepts, and the substantial evidential basis of this potential therapeutic and prophylactic treatment modality are presented. Precision Behavioral Management (PBM), conceptualized initially as Precision Addiction Management (PAM), certainly deserves consideration as an important modality for the treatment of impaired cognitive control in reward processing as manifested in people with neurobiologically expressed Reward Deficiency Syndrome (RDS).

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

The role of opioidergic system in modulating cost/benefit decision-making in alcohol-preferring AA rats and Wistar rats

Research has highlighted the association of a positive family history of alcoholism with a positive treatment response to opioid antagonists in those with a gambling disorder. However, the role of the opioidergic system in gambling behavior is not well understood, and preclinical studies are needed to clarify this. In this study, Alko Alcohol (AA) and Wistar rats went through operant lever pressing training where the task was to choose the more profitable of two options. Different sized sucrose rewards guided the lever choices, and the probability of gaining rewards changed slowly to a level where choosing the smaller reward was the most profitable option. After training, rats were administered subcutaneously with opioid agonist morphine or opioid antagonist naltrexone to study the impact of opioidergic mechanisms on cost/benefit decisions. No difference was found in the decision-making between AA rats or Wistar rats after the morphine administration, but control data revealed a minor decision enhancing effect in AA rats. Naltrexone had no impact on the decisions in AA rats but promoted unprofitable decisions in Wistar rats. Supporting behavioral data showed that in both rat strains morphine increased, and naltrexone decreased, sucrose consumption. Naltrexone also increased the time to accomplish the operant task. The results suggest that opioid agonists could improve decision-making in cost-benefit settings in rats that are naturally prone to high alcohol drinking. The naltrexone results are ambiguous but may partly explain why opioid antagonists lack a positive pharmacotherapeutic effect in some subgroups of gamblers.

Adolescent impulsivity as a sex-dependent and subtype-dependent predictor of impulsivity, alcohol drinking and dopamine D2 receptor expression in adult rats

Impulsivity is a personality trait associated with a heightened risk for drug use and other psychiatric conditions. Because impulsivity-related disorders typically emerge during adolescence, there has been interest in exploring methods for identifying adolescents that will be at risk to develop substance use disorders in adulthood. Here, we used a rodent model to assess inhibitory control (impulsive action) and impulsive decision making (impulsive choice) during adolescence (43-50 days old) or adulthood (93-100 days old) and then examined the impact of development on these impulsivity traits by re-testing rats 50 days later. Impulsive action was not stable from adolescence to adulthood in male rats and was lowest in adult male rats, relative to adolescents and female rats. Impulsive choice was stable across development and unaffected by age or sex. Next, we examined the connection between our model of impulsivity and two measures relevant to substance abuse research: the initiation of voluntary alcohol drinking and dopamine D₂ receptor (D₂ R) expression in the prelimbic prefrontal cortex. Consumption of saccharin-sweetened ethanol during 30-minute sessions in adulthood was associated with adolescent, but not adult, impulsive action, particularly in male rats. Prelimbic D₂ R expression was reduced in individuals with high levels of impulsive choice, and this relationship appeared to be strongest among female rats. The results of this study demonstrate that impulsive choice, along with its connection to D₂ R expression, is relatively unchanged by the process of development. For impulsive action, however, individual levels of impulsivity during adolescence predict drinking in adulthood despite changes in the measure during development.

Amphetamine primes enhanced motivation toward uncertain choices in rats with genetic alcohol preference

RATIONALE

Comorbidity with gambling disorder (GD) and alcohol use disorder (AUD) is well documented.

OBJECTIVE

The purpose of our study was to examine the influence of genetic alcohol drinking tendency on reward-guided decision making behavior of rats and the impact of dopamine releaser D-amphetamine on this behavior.

METHODS

In this study, Alko alcohol (AA) and Wistar rats went through long periods of operant lever pressing training where the task was to choose the profitable of two options. The lever choices were guided by different-sized sucrose rewards (one or three pellets), and the probability of gaining the larger reward was slowly changed to a level where choosing the smaller reward would be the most profitable in the long run. After training, rats were injected (s.c.) with dopamine releaser D-amphetamine (0.3, 1.0 mg/kg) to study the impact of rapid dopamine release on this learned decision making behavior.

RESULTS

Administration of D-amphetamine promoted unprofitable decision making of AA rats more robustly when compared to Wistar rats. At the same time, D-amphetamine reduced lever pressing responses. Interestingly, we found that this reduction in lever pressing was significantly greater in Wistar rats than in AA rats and it was not linked to motivation to consume sucrose.

CONCLUSIONS

Our results indicate that conditioning to the lever pressing in uncertain environments is more pronounced in AA than in Wistar rats and indicate that the reinforcing effects of a gambling-like environment act as a stronger conditioning factor for rats that exhibit a genetic tendency for high alcohol drinking.

Activation of 5-HT7 receptor by administration of its selective agonist, LP-211, modifies explorative-curiosity behavior in rats in two paradigms which differ in visuospatial parameters

AIMS

The serotonin 7 receptor (5-HT7R) subtype, coded by Htr7 gene, is broadly expressed in the central nervous system (CNS) with clear involvement in behavioral functions such as learning/memory, regulation of mood, and circadian rhythms. In this study, we assessed effects of 5-HT7R stimulation by administration of its selective agonist, LP-211 (0.25 mg/kg i.p.), in adult Wistar-Han rats.

METHODS

We used two different explorative-curiosity tests. Drug was administered either before one side-chamber familiarization (CF/V group) or immediately after it, to act on consolidation of familiarization (V/CF group).

RESULTS

Exp. 1 for novelty seeking in black/white boxes (BWB), with door opening after 5 minutes in the familiar chamber, showed that (i) time spent in the novel environment (significantly higher than in familiar chamber for controls) is enhanced in V/CF group (potentiated recognition for a "visual" consolidation) and not different in CF/V group; (ii) activity and chamber transitions, made by CF/V rats, are significantly higher than for other groups (interference on recognition for a "spatial" acquisition). Exp. 2 for novelty preference in D- vs L-shaped chambers (D/L), with start from neutral center, gave different results: (i) time spent in the novel environment by CF/V group is significantly higher than other groups (potentiated "cognitive" acquisition); (ii) chamber transitions made by V/CF group are significantly higher than other groups (potentiated "emotional" consolidation).

CONCLUSION

These apparently conflicting results may reflect LP-211 effects on visual vs spatial memory (D/L apparatus has more pronounced hippocampal components than BWB). However, further experiments are needed to analyze more in depth the mechanisms involved.

Modeling risky decision-making in nonhuman animals: shared core features

Understanding the neural mechanisms of risky decision-making is critical to developing appropriate treatments for psychiatric disorders, problem gambling, and addiction to drugs of abuse. Probing neurobiological mechanisms requires the use of nonhuman animal models (particularly rhesus macaques, rats, and mice). However, there is considerable variation across species in risk preferences. Nevertheless, there are shared core features of risky decision-making present across species. As demonstrated with a wide variety of behavioral paradigms, modulators of risk preference observed in humans are readily replicated in model species. Thus, risky decision-making represents an important implementation of reward-guided decision-making that is feasibly modeled across species.

Methylphenidate and Atomoxetine-Responsive Prefrontal Cortical Genetic Overlaps in "Impulsive" SHR/NCrl and Wistar Rats

Impulsivity, the predisposition to act prematurely without foresight, is associated with a number of neuropsychiatric disorders, including attention-deficit/hyperactivity disorder (ADHD). Identifying genetic underpinnings of impulsive behavior may help decipher the complex etiology and neurobiological factors of disorders marked by impulsivity. To identify potential genetic factors of impulsivity, we examined common differentially expressed genes (DEGs) in the prefrontal cortex (PFC) of adolescent SHR/NCrl and Wistar rats, which showed marked decrease in preference for the large but delayed reward, compared with WKY/NCrl rats, in the delay discounting task. Of these DEGs, we examined drug-responsive transcripts whose mRNA levels were altered following treatment (in SHR/NCrl and Wistar rats) with drugs that alleviate impulsivity, namely, the ADHD medications methylphenidate and atomoxetine. Prefrontal cortical genetic overlaps between SHR/NCrl and Wistar rats in comparison with WKY/NCrl included genes associated with transcription (e.g., Btg2, Fos, Nr4a2), synaptic plasticity (e.g., Arc, Homer2), and neuron apoptosis (Grik2, Nmnat1). Treatment with methylphenidate and/or atomoxetine increased choice of the large, delayed reward in SHR/NCrl and Wistar rats and changed, in varying degrees, mRNA levels of Nr4a2, Btg2, and Homer2, genes with previously described roles in neuropsychiatric disorders characterized by impulsivity. While further studies are required, we dissected potential genetic factors that may influence impulsivity by identifying genetic overlaps in the PFC of "impulsive" SHR/NCrl and Wistar rats. Notably, these are also drug-responsive transcripts which may be studied further as biomarkers to predict response to ADHD drugs, and as potential targets for the development of treatments to improve impulsivity.

Adolescent transitions in reflexive and non-reflexive behavior: Review of fear conditioning and impulse control in rodent models

Adolescence is a time of critical brain changes that pave the way for adult learning processes. However, the extent to which learning in adolescence is best characterized as a transitional linear progression from childhood to adulthood, or represents a period that differs from earlier and later developmental stages, remains unclear. Here we examine behavioral literature on associative fear conditioning and complex choice behavior with rodent models. Many aspects of fear conditioning are intact by adolescence and do not differ from adult patterns. Sufficient evidence, however, suggests that adolescent learning cannot be characterized simply as an immature precursor to adulthood. Across different paradigms assessing choice behavior, literature suggests that adolescent animals typically display more impulsive patterns of responding compared to adults. The extent to which the development of basic conditioning processes serves as a scaffold for later adult decision making is an additional research area that is important for theory, but also has widespread applications for numerous psychological conditions.

Down-regulation of serotonin and dopamine transporter genes in individual rats expressing a gambling-prone profile: A possible role for epigenetic mechanisms

Gambling Disorder (GD) is characterized by excessive gambling despite adverse consequences on individual functioning. In spite of some positive findings, it is difficult to draw any conclusion on the genetics of GD. Indeed, beyond DNA sequence variation, other regulatory mechanisms (like those that engage epigenetics) may explain gene alterations in this addictive disease. Wistar male rats underwent an operant task for the evaluation of individual propensity to gamble. Few rats, after having learnt to prefer nose-poking for a large over a small food reward, were sacrificed to obtain a baseline profile of gene expression at both central and peripheral levels. In the remaining rats, probability of occurrence of large-reward delivery decreased progressively to very low levels. Thus, rats were faced with temptation to "gamble", i.e. to nose-poke for a binge reward, whose delivery was omitted the majority of times. After 3weeks of testing, rats showing a clear-cut profile of either gambling proneness or aversion were selected and sacrificed after the last session. A selective down-regulation of i) serotonin transporter in prefrontal cortex, ii) tyrosine hydroxylase in ventral striatum, iii) dopamine transporter in lymphocytes was evidenced in "gambler" vs "non-gambler" rats. The exposure to such operant task (compared to home-cage alone) modulated ventrostriatal but not prefrontal genes. A consistent increase of DNA methylation, in one specific CpG site at serotonin transporter gene, was evident in prefrontal cortex of "gambler" rats. Elucidation of epigenetic changes occurring during GD progression may pave the way to the development of new therapeutic strategies through specific modulation of epigenetic factors.

Effects of High Frequency Repeated Transcranial Magnetic Stimulation and Continuous Theta Burst Stimulation on Gambling Reinforcement, Delay Discounting, and Stroop Interference in Men with Pathological Gambling

BACKGROUND

Repeated transcranial magnetic stimulation (rTMS) can reduce cravings and improve cognitive function in substance dependent individuals. Whether these benefits extend to individuals with pathological gambling (PG) is unclear. High-frequency rTMS of the medial prefrontal cortex (PFC) and continuous theta burst stimulation (cTBS) of the right dorsolateral PFC can reduce impulsive choice in healthy volunteers.

OBJECTIVE

This study aimed to assess the effects of these two protocols on gambling reinforcement and related responses in otherwise healthy men with PG.

METHODS

Participants (n = 9) underwent active or sham treatments at weekly intervals in a repeated-measures, Latin square design. Subjective and physiological responses were assessed before and after a 15-min slot machine game on each session. Delay discounting and Stroop tasks measured post-game impulsive choice and attentional control.

RESULTS

Multivariate analysis of covariance, controlling for winnings on the slot machine under each treatment, found that rTMS reduced the post-game increase in Desire to Gamble; cTBS reduced amphetamine-like effects, and decreased diastolic blood pressure. Treatment had no significant univariate effects on bet size or speed of play in the game; however, a multivariate effect for the two indices suggested that treatment decreased behavioral activation. Neither treatment reduced impulsive choice, while both treatments increased Stroop interference.

CONCLUSIONS

rTMS and cTBS can reduce gambling reinforcement in non-comorbid men with PG. Separate processes appear to mediate gambling reinforcement and betting behavior as against delay discounting and Stroop interference. Interventions that modify risky as opposed to temporal aspects of decision making may better predict therapeutic response in PG.

The subjective value of probabilistic outcomes: Impact of reward magnitude on choice with uncertain rewards in rats

Interest is rising for animal modelling of Gambling disorder (GD), which is rapidly emerging as a mental health concern. In the present study, we assessed gambling proneness in male Wistar-Han rats using the "Probabilistic Delivery Task" (PDT). This operant protocol is based on choice between either certain, small amounts of food (SS) or larger amounts of food (LLL) delivered (or not) depending on a given (and progressively decreasing) probability. Here, we manipulated the ratio between large and small reward size to assess the impact of different magnitudes on rats' performance. Specifically, we drew a comparison between threefold (2 vs 6 pellets) and fivefold (1 vs 5 pellets) sizes. As a consequence, the "indifferent point" (IP, at which either choice is mathematically equivalent in terms of total foraging) was at 33% and 20% probability of delivery, respectively. Animals tested with the sharper contrast (i.e. fivefold ratio) exhibited sustained preference for LLL far beyond the IP, despite high uncertainty and low payoff, which rendered LLL a sub-optimal option. By contrast, animals facing a slighter contrast (i.e. threefold ratio) were increasingly disturbed by progressive rarefaction of rewards, as expressed by enhanced inadequate nose-poking: this was in accordance with their prompt shift in preference to SS, already shown around the IP. In conclusion, a five-folded LLL-to-SS ratio was not only more attractive, but also less frustrating than a three-folded one. Thus, a profile of gambling proneness in the PDT is more effectively induced by marked contrast between alternative options.

A test of stress, cues, and re-exposure to large wins as potential reinstaters of suboptimal decision making in rats

The present experiment investigated potential reinstaters of suboptimal economic decision making in rats. Rats were first trained on a version of the rat Gambling Task under conditions designed to promote choice of a suboptimal option that occasionally resulted in large “wins” (four sucrose pellets). In a second phase, preference for this economically suboptimal option was reduced by substantially increasing the probability of punishment when this option was chosen. Then, three events were tested for their ability to reinstate choice of the suboptimal option. A brief period of re-exposure to a high frequency of large wins significantly increased choice of the suboptimal option. The pharmacological stressor yohimbine did not reinstate suboptimal choice, but did increase impulsive action as indexed by premature responding. Presentation of cues previously associated with large wins did not alter behavior. Results suggest reinstaters of suboptimal choice may differ from reinstaters of extinguished drug- and food-seeking behavior.

Aging impairs deliberation and behavioral flexibility in inter-temporal choice

Inter-temporal choice depends on multiple, interacting systems, some of which may be compromised with age. Some of these systems may be responsible for ongoing trial-by-trial choice strategies. Some may represent the consequences of action. Some may be necessary for the coupling between anticipated consequences and strategies currently in use, flexibly guiding behavior. When faced with a difficult decision, rats will orient back and forth, a behavior termed "vicarious trial and error" (VTE). Recent experiments have linked the occurrence of VTE to hippocampal search processes and behavioral flexibility. We tested 5 month (n = 6), 9 month (n = 8) and over-27 month-old (n = 10) rats on a Spatial Adjusting Delay Discounting task to examine how aging impacted lap-by-lap strategies and VTE during inter-temporal choice. Rats chose between spatially separated food goals that provided a smaller-sooner or larger-later reward. On each lap, the delay to the larger-later reward was adjusted as a function of the rat's decisions, increasing by 1 s after delayed-side choices and decreasing by 1 s after non-delayed side choices. The strategies that aged rats used differed from those used in young and adult rats. Moreover, aged rats produced reliably more VTE behaviors, for protracted periods of time, uncoupled from behavioral flexibility.

Individual differences in gambling proneness among rats and common marmosets: an automated choice task

Interest is rising for animal modeling of pathological gambling. Using the operant probabilistic-delivery task (PDT), gambling proneness can be evaluated in laboratory animals. Drawing a comparison with rats, this study evaluated the common marmoset (Callithrix jacchus) using a PDT. By nose- or hand-poking, subjects learnt to prefer a large (LLL, 5-6 pellets) over a small (SS, 1-2 pellets) reward and, subsequently, the probability of occurrence of large-reward delivery was decreased progressively to very low levels (from 100% to 17% and 14%). As probability decreased, subjects showed a great versus little shift in preference from LLL to SS reinforcer. Hence, two distinct subpopulations (“non-gambler” versus “gambler”) were differentiated within each species. A proof of the model validity comes from marmosets' reaction to reward-delivery omission. Namely, depending on individual temperament (“gambler” versus “non-gambler”), they showed either persistence (i.e., inadequate pokes towards LLL) or restlessness (i.e., inadequate pokes towards SS), respectively. In conclusion, the marmoset could be a suitable model for preclinical gambling studies. Implementation of the PDT to species other than rats may be relevant for determining its external validity/generalizability and improving its face/construct validity.

Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues

Uncertainty is a component of many gambling games and may play a role in incentive motivation and cue attraction. Uncertainty can increase the attractiveness for predictors of reward in the Pavlovian procedure of autoshaping, visible as enhanced sign-tracking (or approach and nibbles) by rats of a metal lever whose sudden appearance acts as a conditioned stimulus (CS+) to predict sucrose pellets as an unconditioned stimulus (UCS). Here we examined how reward uncertainty might enhance incentive salience as sign-tracking both in intensity and by broadening the range of attractive CS+s. We also examined whether initially induced uncertainty enhancements of CS+ attraction can endure beyond uncertainty itself, and persist even when Pavlovian prediction becomes 100% certain. Our results show that uncertainty can broaden incentive salience attribution to make CS cues attractive that would otherwise not be (either because they are too distal from reward or too risky to normally attract sign-tracking). In addition, uncertainty enhancement of CS+ incentive salience, once induced by initial exposure, persisted even when Pavlovian CS-UCS correlations later rose toward 100% certainty in prediction. Persistence suggests an enduring incentive motivation enhancement potentially relevant to gambling, which in some ways resembles incentive-sensitization. Higher motivation to uncertain CS+s leads to more potent attraction to these cues when they predict the delivery of uncertain rewards. In humans, those cues might possibly include the sights and sounds associated with gambling, which contribute a major component of the play immersion experienced by problematic gamblers.

Nonhuman gamblers: lessons from rodents, primates, and robots

The search for neuronal and psychological underpinnings of pathological gambling in humans would benefit from investigating related phenomena also outside of our species. In this paper, we present a survey of studies in three widely different populations of agents, namely rodents, non-human primates, and robots. Each of these populations offers valuable and complementary insights on the topic, as the literature demonstrates. In addition, we highlight the deep and complex connections between relevant results across these different areas of research (i.e., cognitive and computational neuroscience, neuroethology, cognitive primatology, neuropsychiatry, evolutionary robotics), to make the case for a greater degree of methodological integration in future studies on pathological gambling.

Cross-species approaches to pathological gambling: a review targeting sex differences, adolescent vulnerability and ecological validity of research tools

Decision-making plays a pivotal role in daily life as impairments in processes underlying decision-making often lead to an inability to make profitable long-term decisions. As a case in point, pathological gamblers continue gambling despite the fact that this disrupts their personal, professional or financial life. The prevalence of pathological gambling will likely increase in the coming years due to expanding possibilities of on-line gambling through the Internet and increasing liberal attitudes towards gambling. It therefore represents a growing concern for society. Both human and animal studies rapidly advance our knowledge on brain-behaviour processes relevant for understanding normal and pathological gambling behaviour. Here, we review in humans and animals three features of pathological gambling which hitherto have received relatively little attention: (1) sex differences in (the development of) pathological gambling, (2) adolescence as a (putative) sensitive period for (developing) pathological gambling and (3) avenues for improving ecological validity of research tools. Based on these issues we also discuss how research in humans and animals may be brought in line to maximize translational research opportunities.

Irrational choice under uncertainty correlates with lower striatal D(2/3) receptor binding in rats

Individual differences in dopamine (DA) signaling, including low striatal D(2/3) receptors, may increase vulnerability to substance abuse, although whether this phenotype confers susceptibility to nonchemical addictions is unclear. The degree to which people use "irrational" cognitive heuristics when choosing under uncertainty can determine whether they find gambling addictive. Given that dopaminergic projections to the striatum signal reward expectancy and modulate decision-making, individual differences in DA signaling could influence the extent of such biases. To test this hypothesis, we used a novel task to model biased, risk-averse decision-making in rats. Animals chose between a "safe" lever, which guaranteed delivery of the wager, or an "uncertain" lever, which delivered either double the wager or nothing with 50:50 odds. The bet size varied from one to three sugar pellets. Although the amount at stake did not alter the options' utility, a subgroup of "wager-sensitive" rats increased their preference for the safe lever as the bet size increased, akin to risk aversion. In contrast, wager-insensitive rats slightly preferred the uncertain option consistently. Amphetamine increased choice of the uncertain option in wager-sensitive, but not in wager-insensitive rats, whereas a D(2/3) receptor antagonist decreased uncertain lever choice in wager-insensitive rats alone. Micro-PET and autoradiography using [(11)C]raclopride confirmed a strong correlation between high wager sensitivity and low striatal D(2/3) receptor density. These data suggest that the propensity for biased decision-making under uncertainty is influenced by striatal D(2/3) receptor expression, and provide novel support for the hypothesis that susceptibility to chemical and behavioral addictions may share a common neurobiological basis.

Age-related differences in impulsivity among adolescent and adult Sprague-Dawley rats

Adolescence is an ontogenetic period characterized by numerous hormonal, neural, and behavioral changes. In animal models, adolescents exhibit greater levels of novelty-seeking behavior and risk-taking relative to adults, behaviors associated in humans with increases in impulsivity and elevated propensities to engage in drug and alcohol seeking behaviors. The current series of experiments sought to explore possible age-related differences in impulsivity when indexed using delay discounting in adolescent (postnatal day [P] 25-27) and adult (P68-71) female (Experiment 1) and male (Experiment 2) Sprague-Dawley rats. In both experiments, adolescents exhibited significantly greater levels of impulsive-like behavior in this test relative to adults-even when data were adjusted to account for baseline differences in activity levels (i.e., general nose-poking behavior) across age. Taken together, these results extend to both sexes previous findings of adolescent-associated elevations in impulsivity observed among male mice using delay discounting, as well as among male rats using other procedures to index impulsivity. That these age differences were observed among both male and female rats suggests that impulsivity may be a pervasive feature of adolescence, and contributes to the expression of risky behaviors during this ontogenetic period.

Delay discounting in C57BL/6J and DBA/2J mice: adolescent-limited and life-persistent patterns of impulsivity

Impulsivity is a defining characteristic of adolescence. Compared to adults, for example, adolescents engage in higher rates of drug and alcohol experimentation, risky sexual practices, and criminal activity. Such behavior may reflect reduced sensitivity to long-term consequences of behavior during adolescence. Recently, our lab has attempted to refine mouse procedures to study developmental trends in decision making in the laboratory. In the present experiment, we examined sensitivity to delayed rewards in C57BL/6J (B6) and DBA/2J (D2) mice during adolescence and adulthood using an adaptation of a 2-week delay discounting procedure developed by Adriani and Laviola (2003). During training, mice could choose between a 20- or 100-μl drop of milk delivered after a 1-s delay. During testing, the delay to the large drop of milk was increased from 1 to 100 seconds. As the delay to the larger volume increased, preference shifted to the smaller, more immediate option. In adolescence, both strains showed similar shifts in preference. In contrast, adult B6 mice were less sensitive to increasing delays than were adult D2 mice, who continued to perform much as their adolescent counterparts. A subsequent resistance-to-extinction test ruled out the possibility that the slower change in the adult B6 mice was due to perseverative responding. The present findings suggest that B6 and D2 strains may be differentially suited to uncovering the biological mechanism of short-term and long-term patterns of impulsive behavior.

Loss in risk-taking: absence of optimal gain or reduction in one's own resources?

Determining how living beings react to tasks that reflect realistic situations of risk has given rise to a vast literature. However, I argue that the methodologies traditionally used to test humans and nonhumans relative to risk often fail to achieve their goal. When risk is modelled in laboratory, potential decision cost (or potential loss) typically denotes an absence of optimal gain. In contrast, when risk occurs in real-life situations, potential loss denotes the reduction in an individual's limited resources - whether energetic, social, financial, etc. This conceptual difference about the nature of risk may have important implications for the understanding of the parameters that control risk-taking behaviour.

Brain processes in discounting: consequences of adolescent methylphenidate exposure

Traits of inattention, impulsivity, and motor hyperactivity characterize children diagnosed with attention-deficit/hyperactivity disorder (ADHD), whose inhibitory control is reduced. In animal models, crucial developmental phases or experimental transgenic conditions account for peculiarities, such as sensation-seeking and risk-taking behaviors, and reproduce the beneficial effects of psychostimulants. An "impulsive" behavioral profile appears to emerge more extremely in rats when forebrain dopamine (DA) systems undergo remodeling, as in adolescence, or with experimental manipulation tapping onto the dopamine transporter (DAT). Ritalin(®) (methylphenidate, MPH), a DAT-blocking drug, is prescribed for ADHD therapy but is also widely abused by human adolescents. Administration of MPH during rats' adolescence causes a long-term modulation of their self-control, in terms of reduced intolerance to delay and diminished proneness for risk when reward is uncertain. Exactly the opposite profile emerges when exogenous alteration of DAT levels is achieved via lentiviral transfection. Both adolescent MPH exposure and DAT-targeting transfection lead to enduring hyperfunction of dorsal striatum and hypofunction of ventral striatum. Together with upregulation of prefronto-cortical phospho-creatine, striatal upregulation of selected genes (like serotonin 7 receptor gene) suggests that enhanced inhibitory control is generated by adolescent MPH exposure. Operant tasks, which assess the balance between motivational drives and inhibitory self-control, are thus useful for investigating reward-discounting processes and their modulation by DAT-targeting tools. In summary, due to the complexity of human studies, preclinical investigations of rodent models are necessary to understand better both the neurobiology of ADHD-like symptoms' etiology and the long-term therapeutic safety of adolescent MPH exposure.

Contrafreeloading in grizzly bears: implications for captive foraging enrichment

Although traditional feeding regimens for captive animals were focused on meeting physiological needs to assure good health, more recently emphasis has also been placed on non-nutritive aspects of feeding. The provision of foraging materials to diversify feeding behavior is a common practice in zoos but selective consumption of foraging enrichment items over more balanced "chow" diets could lead to nutrient imbalance. One alternative is to provide balanced diets in a contrafreeloading paradigm. Contrafreeloading occurs when animals choose resources that require effort to exploit when identical resources are freely available. To investigate contrafreeloading and its potential as a theoretical foundation for foraging enrichment, we conducted two experiments with captive grizzly bears (Ursus arctos horribilis). In Experiment 1, bears were presented with five foraging choices simultaneously: apples, apples in ice, salmon, salmon in ice, and plain ice under two levels of food restriction. Two measures of contrafreeloading were considered: weight of earned food consumed and time spent working for earned food. More free than earned food was eaten, with only two bears consuming food extracted from ice, but all bears spent more time manipulating ice containing salmon or apples than plain ice regardless of level of food restriction. In Experiment 2, food-restricted bears were presented with three foraging choices simultaneously: apples, apples inside a box, and an empty box. Although they ate more free than earned food, five bears consumed food from boxes and all spent more time manipulating boxes containing apples than empty boxes. Our findings support the provision of contrafreeloading opportunities as a foraging enrichment strategy for captive wildlife.

Social withdrawal and gambling-like profile after lentiviral manipulation of DAT expression in the rat accumbens

Dysfunction of brain dopamine transporter (DAT) has been associated with sensation seeking and impulse-control disorders. We recently generated a new animal model by stereotaxical inoculation of lentiviral vectors, which allowed localized intra-accumbal delivery of modulators for DAT gene: GFP (green fluorescent protein) control, silencers (Sil), a regulatable enhancer (DAT+), or both (DAT+Sil). Wistar male rats were followed both for socio-emotional profiles and for propensity to seek risky, uncertain rewards. Elevated anxiety and affiliation towards an unfamiliar partner emerged in Sil rats. Interestingly, in DAT+Sil rats (and Sil rats to a lesser extent) levels of playful social interaction were markedly reduced compared to controls. These DAT+Sil rats displayed a marked 'gambling-like' profile (i.e. preference for a large/uncertain over a small/sure reward), which disappeared upon doxycycline-induced switch-off onto DAT enhancer, but consistently reappeared with doxycycline removal. MRI-guided 1H-MRS (at 4.7 T) examinations in vivo (under anaesthesia) revealed changes in the bioenergetic metabolites (phosphocreatine and total creatine) for DAT+Sil rats, indicating a functional up-regulation of dorsal striatum (Str) and conversely a down-regulation of ventral striatum (i.e. nucleus accumbens, NAc). A combined profile of (1) enhanced proneness to gambling and (2) strong social withdrawal is thus associated with altered DAT-induced balance within forebrain dopamine systems. In fact, risk of developing a gambling-prone, social-avoidant psychopathology might be associated with (1) dominant semi-automatic strategies and/or habits, developed within Str circuits, and (2) reduced NAc function, with poorer feedback adjustment on decisions by aversive experiences.

Insight into the relationship between impulsivity and substance abuse from studies using animal models

Drug use disorders are often accompanied by deficits in the capacity to efficiently process reward-related information and to monitor, suppress, or override reward-controlled behavior when goals are in conflict with aversive or immediate outcomes. This emerging deficit in behavioral flexibility and impulse control may be a central component of the progression to addiction, as behavior becomes increasingly driven by drugs and drug-associated cues at the expense of more advantageous activities. Understanding how neural mechanisms implicated in impulse control are affected by addictive drugs may therefore prove a useful strategy in the search for new treatment options. Animal models of impulsivity and addiction could make a significant contribution to this endeavor. Here, some of the more common behavioral paradigms used to measure different aspects of impulsivity across species are outlined, and the importance of the response to reward-paired cues in such paradigms is discussed. Naturally occurring differences in forms of impulsivity have been found to be predictive of future drug self-administration, but drug exposure can also increase impulsive responding. Such data are in keeping with the suggestion that impulsivity may contribute to multiple stages within the spiral of addiction. From a neurobiological perspective, converging evidence from rat, monkey, and human studies suggest that compromised functioning within the orbitofrontal cortex may critically contribute to the cognitive sequelae of drug abuse. Changes in gene transcription and protein expression within this region may provide insight into the mechanism underlying drug-induced cortical hypofunction, reflecting new molecular targets for the treatment of uncontrolled drug-seeking and drug-taking behavior.

Home cage testing of delay discounting in rats

Testing rodents in their home cages has become increasingly popular. Since human intervention, handling, and transport are minimized, behavior can be recorded undisturbed and continuously. Currently existing home cage systems are too complex if only relatively simple operant-learning tests are to be carried out in rats. For that purpose, a new low-cost computer-controlled operant panel was designed, which can be placed inside the home cage. A pilot study was carried out, using an intolerance-to-delay protocol, classically developed for testing behavioral impulsivity. Male adult rats were tested in their home cages, containing the operant panel provided with nose-poking holes. Nose poking in one hole resulted in the immediate delivery of one food pellet (small-soon, SS), whereas nose poking in the other hole delivered five food pellets after a delay (large-late), which was increased progressively each day (0-150 sec). The two daily sessions, spaced 8 h apart, lasted 1 h each, and the time-out after food delivery was 90 sec. A clear-cut shift toward preference for SS, which is considered a classical index of cognitive impulsivity, was shown at the longest delay. It is noteworthy that rats shifted when the delay interval was longer than the mean intertrial interval-that is, when they experienced more than one delay-equivalent odds against discounting (see Adriani & Laviola, 2006). The shortened training (2 days) and testing (5 days) phases, as allowed by prolonged and multiple daily sessions, can be advantageous in testing rodents during selected short phases of development. Current research is focusing on further validation of this and similar protocols.

Lentiviral vector-mediated gene transfer and RNA silencing technology in neuronal dysfunctions

Lentiviral-mediated gene transfer in vivo or in cultured mammalian neurons can be used to address a wide variety of biological questions, to design animal models for specific neurodegenerative pathologies, or to test potential therapeutic approaches in a variety of brain disorders. Lentiviruses can infect nondividing cells, thereby allowing stable gene transfer in postmitotic cells such as mature neurons. An important contribution has been the use of inducible vectors: the same animal can thus be used repeatedly in the doxycycline-on or -off state, providing a powerful mean for assessing the function of a gene candidate in a disorder within a specific neuronal circuit. Furthermore, lentivirus vectors provide a unique tool to integrate siRNA expression constructs with the aim to locally knockdown expression of a specific gene, enabling to assess the function of a gene in a very specific neuronal pathway. Lentiviral vector-mediated delivery of short hairpin RNA results in persistent knockdown of gene expression in the brain. Therefore, the use of lentiviruses for stable expression of siRNA in brain is a powerful aid to probe gene functions in vivo and for gene therapy of diseases of the central nervous system. In this chapter, I review the applications of lentivirus-mediated gene transfer in the investigation of specific gene candidates involved in major brain disorders and neurodegenerative processes. Major applications have been in polyglutamine disorders, such as synucleinopathies and Parkinson's disease, or in investigating gene function in Huntington's disease, dystonia, or muscular dystrophy. Recently, lentivirus gene transfer has been an invaluable tool for evaluation of gene function in behavioral disorders such as drug addiction and attention-deficit hyperactivity disorder or in learning and cognition.

Cognitive impulsivity in animal models: role of response time and reinforcing rate in delay intolerance with two-choice operant tasks

Impulsivity, a key symptom of ADHD (attention-deficit hyperactivity disorder), is also common in obsessive-compulsive and addictive disorders. There is rising interest in animal models of inhibitory-control impairment. Adolescent rats were tested daily in the intolerance-to-delay (ID) task (session 25 min, timeout 20 s), involving choice between either immediate small amount of food (SS), or larger amount of food after a delay (LL). The mixed 5-HT(1A/7) agonist (8-OH-DPAT, 0 or 0.060 mg/kg i.p.) was administered acutely just before the last three sessions at highest delays. In addition to the classical choice parameter (percent LL preference), the spontaneous waiting (termed response time, RT) occurring between end of a timeout (TO) and next nose-poke was calculated. The pace between consecutive reinforcer deliveries is given by the mean inter-trial interval (mITI, i.e. TO + RT). Hence, the impact of any given delay may be proportional to this pace and be expressed as delay-equivalent odds, i.e. the extent by which delays are multiples of the mITI. Data revealed that RT/mITI increased sharply from around 15 s/35 s to around 30 s/50 s when imposed delay changed from 30 s to 45 s (i.e. odds from 0.91 to 1.06). This suggests that rats adopted a strategy allowing them to keep in pace with perceived reinforcing rate. The increasing delay constraint directly influenced the length of rats' spontaneous waiting (RT) before next decision. For higher delays, with odds >1, rats shifted to a clear-cut SS preference, which is devoid of any exogenous temporal constraint. A challenge with 8-OH-DPAT (0 or 0.060 mg/kg i.p.) decreased impulsive choice but also increased RT. Thus, tapping onto 5-HT(1A/7) receptors slightly enhanced RT/mITI values, possibly reflecting ability of rats to cope with slower reinforcing rates and/or with delay-cancelled reward paces. In summary, delay-induced states of aversion may arise from the innate tendency to rely on a regular rate of reinforcement. Conversely, a drug-enhanced capacity to cope with delay may involve an internal ability to adjust expectancy about such a reinforcing rate.

Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task

Pathological gambling (PG) is characterized by persistent, maladaptive gambling behavior, which disrupts personal and professional life. Animal models of gambling behavior could make a significant contribution to improving our understanding of the neural and neurochemical basis of gambling, and the treatment of PG. When gambling, failing to win critically results in the loss of resources wagered as well as the absence of additional gain. Here, we have incorporated these concepts into a novel rat gambling task (rGT), based, in part, on the 'Iowa' gambling task (IGT) commonly used clinically to measure gambling-like behavior. Rats choose among four different options to earn as many sugar pellets as possible within 30 min. Each option is associated with the delivery of a different amount of reward, but also with a different probability and duration of punishing time-out periods during which reward cannot be earned. The schedules are designed such that persistent choice of options linked with larger rewards result in fewer pellets earned per unit time. Rats learn to avoid these risky options to maximize their earnings, comparable with the optimal strategy in the IGT. Both d-amphetamine and the 5-HT(1A) receptor agonist, 8-OH-DPAT, impaired task performance. In contrast, the dopamine D(2) receptor antagonist, eticlopride, improved performance, whereas the D(1) receptor antagonist, SCH23390, had no effect. These data suggest that both serotonergic and dopaminergic agents can impair and improve gambling performance, and indicate that the rGT will be a useful tool to study the biological basis of gambling.

Gender differences in delay-discounting under mild food restriction

Impulsivity, a core symptom of attention-deficit hyperactivity disorder (ADHD), is tested in animal models by delay-discounting tasks. So far, mainly male subjects have been used in this paradigm at severe levels of food restriction. Here we studied the impulsive behaviour of CD-1 adult male and female mice at mild levels of food restriction. Mice maintained at 90 +/- 5% of ad libitum bodyweight, were tested in operant chambers provided with nose-poking holes. Nose poking in one hole resulted in the immediate delivery of one food pellet (small-soon, SS), whereas nose poking in the other hole delivered five food pellets after a delay (large-late, LL), which was increased progressively each day (0-150 s). Two subgroups emerged: individuals that shifted at short delays ("steep") and individuals that did not shift, even at the highest delays ("flat"). Analysis showed that "steep" females shifted at shorter delays than "steep" males, while no difference existed between males and females within the "flat" sub-population. In home-cage circadian activity as well as in a novelty-seeking test, females were more active than males. It can be concluded from these results that female mice are more impulsive than male mice under mild food restriction. This is in contrast with findings in earlier studies with more severe food restriction. Therefore, an alternative explanation is that females are more explorative, and that different features might be tested in delay-discounting paradigms, depending on restriction levels.

Dopaminergic modulation of risk-based decision making

Psychopharmacological studies have implicated the mesolimbic dopamine (DA) system in the mediation of cost/benefit evaluations about delay or effort-related costs associated with larger rewards. However, the role of DA in risk-based decision making remains relatively unexplored. The present study investigated the effects of systemic manipulations of DA transmission on risky choice using a probabilistic discounting task. Over discrete trials, rats chose between two levers; a press on the 'small/certain' lever always delivered one reward pellet, whereas a press on the other, 'large/risky' lever delivered four pellets, but the probability of receiving reward decreased across the four trial blocks (100, 50, 25, 12.5%). In separate groups of well-trained rats we assessed the effects of the DA releaser amphetamine, as well as receptor selective agonists and antagonists. Amphetamine consistently increased preference for the large/risky lever; an effect that was blocked or attenuated by co-administration of either D(1) (SCH23390) or D(2) (eticlopride) receptor antagonists. Blockade of either of these receptors alone induced risk aversion. Conversely, stimulation of D(1) (SKF81297) or D(2) (bromocriptine) receptors also increased risky choice. In contrast, activation of D(3) receptors with PD128,907 reduced choice of the large/risky lever. Likewise, D(3) antagonism with nafadotride potentiated the amphetamine-induced increase in risky choice. Blockade or stimulation of D(4) receptors did not reliably alter behavior. These findings indicate that DA has a critical role in mediating risk-based decision making, with increased activation of D(1) and D(2) receptors biasing choice toward larger, probabilistic rewards, whereas D(3) receptors appear to exert opposing effects on this form of decision making.

Increased impulsive behavior and risk proneness following lentivirus-mediated dopamine transporter over-expression in rats' nucleus accumbens

Multiple theories have been proposed for sensation seeking and vulnerability to impulse-control disorders [Zuckerman M, Kuhlman DM (2000) Personality and risk-taking: Common biosocial factors. J Pers 68:999-1029], and many of these rely on a dopamine system deficit. Available animal models reproduce only some behavioral symptoms and seem devoid of construct validity. We used lentivirus tools for over-expressing or silencing the dopamine transporter (DAT) and we evaluated the resulting behavioral profiles in terms of motivation and self-control. Wistar adult rats received stereotaxic inoculation of a lentivirus that allowed localized intra-accumbens delivery of a DAT gene enhancer/silencer, or the green fluorescent protein, GFP. These animals were studied for intolerance to delay, risk proneness and novelty seeking. As expected, controls shifted their demanding from a large reward toward a small one when the delivery of the former was increasingly delayed (or uncertain). Interestingly, in the absence of general locomotor effects, DAT over-expressing rats showed increased impulsivity (i.e. a more marked shift of demanding from the large/delayed toward the small/soon reward), and increased risk proneness (i.e. a less marked shift from the large/uncertain toward the small/sure reward), compared with controls. Rats with enhanced or silenced DAT expression did not show any significant preference for a novel environment. In summary, consistent with literature on comorbidity between attention-deficit/hyperactivity disorder and pathological gambling, we demonstrate that DAT over-expression in rats' nucleus accumbens leads to impulsive and risk prone phenotype. Thus, a reduced dopaminergic tone following altered accumbal DAT function may subserve a sensation-seeker phenotype and the vulnerability to impulse-control disorders.

Rats bred for high alcohol drinking are more sensitive to delayed and probabilistic outcomes

Alcoholics and heavy drinkers score higher on measures of impulsivity than nonalcoholics and light drinkers. This may be because of factors that predate drug exposure (e.g. genetics). This study examined the role of genetics by comparing impulsivity measures in ethanol-naive rats selectively bred based on their high [high alcohol drinking (HAD)] or low [low alcohol drinking (LAD)] consumption of ethanol. Replicates 1 and 2 of the HAD and LAD rats, developed by the University of Indiana Alcohol Research Center, completed two different discounting tasks. Delay discounting examines sensitivity to rewards that are delayed in time and is commonly used to assess 'choice' impulsivity. Probability discounting examines sensitivity to the uncertain delivery of rewards and has been used to assess risk taking and risk assessment. High alcohol drinking rats discounted delayed and probabilistic rewards more steeply than LAD rats. Discount rates associated with probabilistic and delayed rewards were weakly correlated, while bias was strongly correlated with discount rate in both delay and probability discounting. The results suggest that selective breeding for high alcohol consumption selects for animals that are more sensitive to delayed and probabilistic outcomes. Sensitivity to delayed or probabilistic outcomes may be predictive of future drinking in genetically predisposed individuals.

Comparison of probabilistic choice models in humans

Background

Probabilistic choice has been attracting attention in psychopharmacology and neuroeconomics. Several parametric models have been proposed for probabilistic choice; entropy model, Prelec's probability weight function, and hyperbola-like probability discounting functions.

Methods

In order to examine (i) fitness of the probabilistic models to behavioral data, (ii) relationships between the parameters and psychological processes, e.g., aversion to possible non-gain in each probabilistic choice and aversion to unpredictability, we estimated the parameters and AICc (Akaike Information Criterion with small sample correction) of the probabilistic choice models by assessing the points of subjective equality at seven probability values (95%–5%). We examined both fitness of the models parametrized by utilizing AICc, and the relationships between the model parameters and equation-free parameter of aversion to possible non-gain.

Results

Our results have shown that (i) the goodness of fitness for group data was [Entropy model>Prelec's function>General hyperbola>Simple hyperbola]; while Prelec's function best fitted individual data, (ii) aversion to possible non-gain and aversion to unpredictability are distinct psychological processes.

Conclusion

Entropy and Prelec models can be utilized in psychopharmacological and neuroeconomic studies of risky decision-making.

1H MRS-detectable metabolic brain changes and reduced impulsive behavior in adult rats exposed to methylphenidate during adolescence

Administration of methylphenidate (MPH, Ritalin) to children affected by attention deficit hyperactivity disorder (ADHD) is an elective therapy, which however raises concerns for public health, due to possible persistent neuro-behavioral alterations. We investigated potential long-term consequences at adulthood of MPH exposure during adolescence, by means of behavioral and brain MRS assessment in drug-free state. Wistar adolescent rats (30- to 44-day-old) were treated with MPH (0 or 2 mg/kg once/day for 14 days) and then left undisturbed until adulthood. Levels of impulsive behavior were assessed in the intolerance-to-delay task: Food-restricted rats were tested in operant chambers with two nose-poking holes, delivering one food pellet immediately, or five pellets after a delay whose length was increased over days. MPH-exposed animals showed a less marked shifting profile from the large/late to the small/soon reward, suggesting reduced basal levels of impulsivity, compared to controls. In vivo MRI-guided 1H MRS examinations at 4.7 T in anaesthetised animals revealed long-term biochemical changes in the dorsal striatum (STR), nucleus accumbens (NAcc), and prefrontal cortex (PFC) of MPH-exposed rats. Notably, total creatine and taurine, metabolites respectively involved in bioenergetics and synaptic efficiency, were up-regulated in the STR and conversely down-regulated in the NAcc of MPH-exposed rats. A strong correlation was evident between non-phosphorylated creatine in the STR and behavioral impulsivity. Moreover, unaltered total creatine and increased phospho-creatine/creatine ratio were detected in the PFC, suggesting improved cortical energetic performance. Because of this enduring rearrangement in the forebrain function, MPH-exposed animals may be more efficient when faced with delay of reinforcement. In summary, MPH exposure during adolescence produced enduring MRS-detectable biochemical modifications in brain reward-related circuits, which may account for increased self-control capacity of adult rats.