Reversal learning as a measure of impulsive and compulsive behavior in addictions

A Systematic Review on the Influences of Neurotoxicological Xenobiotic Compounds on Inhibitory Control

Background: Impulsive and compulsive traits represent a variety of maladaptive behaviors defined by the difficulties to stop an improper response and the control of a repeated behavioral pattern without sensitivity to changing contingencies, respectively. Otherwise, human beings are continuously exposed to plenty neurotoxicological agents which have been systematically linked to attentional, learning, and memory dysfunctions, both preclinical and clinical studies. Interestingly, the link between both impulsive and compulsive behaviors and the exposure to the most important xenobiotic compounds have been extensively developed; although the information has been rarely summarized. For this, the present systematic review schedule and analyze in depth the most important works relating different subtypes of the above-mentioned behaviors with 4 of the most important xenobiotic compounds: Lead (Pb), Methylmercury (MeHg), Polychlorinated biphenyls (PCB), and Organophosphates (OP) in both preclinical and clinical models.

Methods: Systematic search strategy on PubMed databases was developed, and the most important information was structured both in text and in separate tables based on rigorous methodological quality assessment.

Results: For Lead, Methylmercury, Polychlorinated biphenyls and organophosphates, a total of 44 (31 preclinical), 34 (21), 38 (23), and 30 (17) studies were accepted for systematic synthesis, respectively. All the compounds showed an important empirical support on their role in the modulation of impulsive and, in lesser degree, compulsive traits, stronger and more solid in animal models with inconclusive results in humans in some cases (i.e., MeHg). However, preclinical and clinical studies have systematically focused on different subtypes of the above-mentioned behaviors, as well as impulsive choice or habit conformations have been rarely studied.

Discussion: The strong empirical support in preclinical studies contrasts with the lack of connection between preclinical and clinical models, as well as the different methodologies used. Further research should be focused on dissipate these differences as well as deeply study impulsive choice, decision making, risk taking, and cognitive flexibility, both in experimental animals and humans.

Effects of chronic cocaine self-administration and N-acetylcysteine on learning, cognitive flexibility, and reinstatement in nonhuman primates

RATIONALE

Cocaine use disorder (CUD) is associated with cognitive deficits that have been linked to poor treatment outcomes. An improved understanding of cocaine's deleterious effects on cognition may help optimize pharmacotherapies. Emerging evidence implicates abnormalities in glutamate neurotransmission in CUD and drugs that normalize glutamatergic homeostasis (e.g., N-acetylcysteine [NAC]) may attenuate CUD-related relapse behavior.

OBJECTIVES

The present studies examined the impact of chronic cocaine exposure on touchscreen-based models of learning (repeated acquisition) and cognitive flexibility (discrimination reversal) and, also, the ability of NAC to modulate cocaine self-administration and its capacity to reinstate drug-seeking behavior.

METHODS

First, stable repeated acquisition and discrimination reversal performance was established. Next, high levels of cocaine-taking behavior (2.13-3.03 mg/kg/session) were maintained for 150 sessions during which repeated acquisition and discrimination reversal performance was probed periodically. Finally, the effects of NAC treatment were examined on cocaine self-administration and, subsequently, extinction and reinstatement.

RESULTS

Cocaine self-administration significantly impaired performance under both cognitive tasks; however, discrimination reversal was disrupted considerably more than acquisition. Performance eventually approximated baseline levels during chronic exposure. NAC treatment did not perturb ongoing self-administration behavior but was associated with significantly quicker extinction of drug-lever responding. Cocaine-primed reinstatement did not significantly differ between groups.

CONCLUSIONS

The disruptive effects of cocaine on learning and cognitive flexibility are profound but performance recovered during chronic exposure. Although the effects of NAC on models of drug-taking and drug-seeking behavior in monkeys are less robust than reported in rodents, they nevertheless suggest a role for glutamatergic modulators in CUD treatment programs.

A single factor explanation for associative learning performance on colour discrimination problems in common pheasants (Phasianus colchicus)

It remains unclear whether performance of non-human animals on cognitive test batteries can be explained by domain general cognitive processes, as is found in humans. The persistence of this dispute is likely to stem from a lack of clarity of the psychological or neural processes involved. One broadly accepted cognitive process, that may predict performance in a range of psychometric tasks, is associative learning. We therefore investigated intra-individual performances on tasks that incorporate processes of associative learning, by assessing the speed of acquisition and reversal learning in up to 187 pheasants (Phasianus colchicus) on four related binary colour discrimination tasks. We found a strong, positive significant bivariate relationship between an individual's acquisition and reversal learning performances on one cue set. Weak, positive significant bivariate relationships were also found between an individual's performance on pairs of reversal tasks and between the acquisition and reversal performances on different cue sets. A single factor, robust to parallel analysis, explained 36% of variation in performance across tasks. Inter-individual variation could not be explained by differential prior experience, age, sex or body condition. We propose that a single factor explanation, which we call 'a', summarises the covariance among scores obtained from these visual discrimination tasks, as they all assess capacities for associative learning. We argue that 'a' may represent an underlying cognitive ability exhibited by an individual, which manifests across a variety of tasks requiring associative processes.

Inhibitory control in BALB/c mice sub-strains during extinction learning

Dysregulation of executive function (EF) involves alterations in cognitive flexibility / control and is underscored by learning impairments in neurodevelopmental disorders. Here, we examine cognitive inflexibility in BALB/cJ mice (a mouse model showing diminished sociability, increased anxiety and inattentive behaviour) and closely related "reference" BALB/cByJ mice. We used an appetitive extinction paradigm to investigate if cognitive flexibility measures are different between learning acquisition and extinction. The two BALB/c sub-strains learned to respond to a stimulus in a touchscreen operant chamber, after which the reward was removed and responses should be inhibited. Both mice sub-strains showed a different rate of learning while acquiring the task, in which the BALB/cJ mice were faster learners compared to the BALB/cByJ mice. This was not observed during the extinction phase, in which the BALB/cJ mice were able to extinguish responding to unrewarded stimuli equally. Within the BALB/cJ sub-strain, variation in the ability to inhibit a learnt response was observed when comparing them to similar grouped BALB/cByJ mice: BALB/cJ animals that reached the criterion were more reward driven, while BALB/cJ mice failing to reach the set criterion during extinction processing make more mistakes. Additionally, the changes observed during acquisition, were driven by animals not reaching the extinction criterion. Our results suggest that the BALB/c mice sub-strains may use different strategies to learn during appetitive extinction. This may be useful in the phenotypic dissection of cognitive flexibility in BALB/c sub-strains and their mapping on genetic variance revealed by next-generation sequencing in future studies.

Effects of long-term high-fat food or methamphetamine intake and serotonin 2C receptors on reversal learning in female rhesus macaques

Perseverative behavior has been highly implicated in addiction. Activation of serotonin 2C receptors (5-HT_2CRs) attenuates cocaine and high caloric food intake, but whether a 5-HT_2CR agonist can reduce high caloric diet (HCD) or methamphetamine (METH) intake and response perseveration remains unknown. Clarifying the role of 5-HT_2CRs in these behaviors will improve knowledge of neurochemical processes that regulate flexible decision-making and whether improvements in decision-making are accompanied by decreases in HCD or METH intake. This study evaluated the effects of long-term HCD and METH intake on reversal learning in female rhesus monkeys. The effects of the 5-HT_2CR agonist WAY163909 on reversal learning before and after extended HCD or METH intake, and on food intake, was also tested. Moreover, we examined whether the 5-HT_2CR is necessary for the effects of WAY163909. WAY163909 was given prior to reversal learning at baseline and after extended HCD or METH intake, and prior to measures of food intake. Extended intake of METH or the HCD increased perseverative errors during reversal. WAY163909 increased correct responses and decreased perseverative errors, both before and after extended HCD or METH intake. Similarly, WAY163909 decreased consumption of a HCD, but not a low caloric diet. The effects of WAY163909 on all these measures were blocked by co-administration with a 5-HT_2CR antagonist. These data indicate that long-term HCD or METH intake disrupts flexible decision-making. Further, the results suggest that reductions in food intake produced by WAY163909 are associated with parallel improvements in decision-making strategies, underscoring the role of the 5-HT_2CR for these behavioral effects.

Translational tests involving non-reward: methodological considerations

This review is concerned with methods for assessing the processing of unrewarded responses in experimental animals and the mechanisms underlying performance of these tasks. A number of clinical populations, including Parkinson's disease, depression, compulsive disorders, and schizophrenia demonstrate either abnormal processing or learning from non-rewarded responses in laboratory-based reinforcement learning tasks. These effects are hypothesized to result from disturbances in modulatory neurotransmitter systems, including dopamine and serotonin. Parallel work in experimental animals has revealed consistent behavioral patterns associated with non-reward and, consistent with the human literature, modulatory roles for specific neurotransmitters. Classical tests involving an important reward omission component include appetitive extinction, ratio schedules of responding, reversal learning, and delay and probability discounting procedures. In addition, innovative behavioral tests have recently been developed leverage probabilistic feedback to specifically assay accommodation of, and learning from, non-rewarded responses. These procedures will be described and reviewed with discussion of the behavioral and neural determinants of performance. A final section focusses specifically on the benefits of trial-by-trial analysis of responding during such tasks, and the implications of such analyses for the translation of findings to clinical studies.

Predicting change: Approximate inference under explicit representation of temporal structure in changing environments

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants' beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants' behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior.

Selective Role of the Putamen in Serial Reversal Learning in the Marmoset

Fronto-striatal circuitry involving the orbitofrontal cortex has been identified as mediating successful reversal of stimulus-outcome contingencies. The region of the striatum that most contributes to reversal learning remains unclear, with studies in primates implicating both caudate nucleus and putamen. We trained four marmosets on a touchscreen-based serial reversal task and implanted each with cannulae targeting both putamen and caudate bilaterally. This allowed reversible inactivation of the two areas within the same monkeys, but across separate sessions, to directly investigate their respective contributions to reversal performance. Behavioral sensitivity to the GABAA agonist muscimol varied across subjects and between brain regions, so each marmoset received a range of doses. Intermediate doses of intra-putamen muscimol selectively impaired reversal performance, leaving the baseline discrimination phase unchanged. There was no effect of low doses and high doses were generally disruptive. By contrast, low doses of intra-caudate muscimol improved reversal performance, while high doses impaired both reversal and baseline discrimination performance. These data provide evidence for a specific role of the putamen in serial reversal learning, which may reflect the more habitual nature of repeated reversals using the same stimulus pair.

Training improves inhibitory control in water rescue dogs

Inhibitory control is a collection of several processes that are aimed to refrain from any impulsive response in the subject during inappropriate situations. Evidence suggests that in dogs, the inhibitory control is affected by domestication process, but also experiences during ontogeny could be an important driver in acquiring inhibitory control. The aim of the study was to compare the performance of highly trained dogs (i.e., water rescue dogs) and pet dogs in the A-not-B task. In this procedure, the animals have to inhibit their urge of going to a previous reinforced place. The results showed that the trained dogs committed fewer errors in the task than the pet dogs suggesting a better inhibitory control. This result could indicate that inhibitory control is a flexible ability affected by ontogenetic processes such as the training experience.

Association of Brain Reward Learning Response With Harm Avoidance, Weight Gain, and Hypothalamic Effective Connectivity in Adolescent Anorexia Nervosa

IMPORTANCE

Anorexia nervosa (AN) is associated with adolescent onset, severe low body weight, and high mortality as well as high harm avoidance. The brain reward system could have an important role in the perplexing drive for thinness and food avoidance in AN.

OBJECTIVE

To test whether brain reward learning response to taste in adolescent AN is altered and associated with treatment response, striatal-hypothalamic connectivity, and elevated harm avoidance.

DESIGN, SETTING, AND PARTICIPANTS

In this cross-sectional multimodal brain imaging study, adolescents and young adults with AN were matched with healthy controls at a university brain imaging facility and eating disorder treatment program. During a sucrose taste classical conditioning paradigm, violations of learned associations between conditioned visual and unconditioned taste stimuli evoked the dopamine-related prediction error (PE). Dynamic effective connectivity during sweet taste receipt was studied to investigate hierarchical brain activation across the brain network that regulates eating. The study was conducted from July 2012 to May 2017, and data were analyzed from June 2017 to December 2017.

MAIN OUTCOMES AND MEASURES

Prediction error brain reward response across the insula, caudate, and orbitofrontal cortex; dynamic effective connectivity between hypothalamus and ventral striatum; and treatment weight gain, harm avoidance scores, and salivary cortisol levels and their correlations with PE brain response.

RESULTS

Of 56 female participants with AN included in the study, the mean (SD) age was 16.6 (2.5) years, and the mean (SD) body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) was 15.9 (0.9); of 52 matched female controls, the mean (SD) age was 16.0 (2.8) years, and the mean (SD) BMI was 20.9 (2.1). Prediction error response was elevated in participants with AN in the caudate head, nucleus accumbens, and insula (multivariate analysis of covariance: Wilks λ, 0.707; P = .02; partial η2 = 0.296), which correlated negatively with sucrose taste pleasantness. Bilateral AN orbitofrontal gyrus rectus PE response was positively correlated with harm avoidance (right ρ, 0.317; 95% CI, 0.091 to 0.539; P < .02; left ρ, 0.336; 95% CI, 0.112 to 0.550; P < .01) but negatively correlated with treatment BMI change (right ρ, -0.282; 95% CI, -0.534 to -0.014; P < .04; left ρ, -0.268; 95% CI, -0.509 to -0.018; P < .045). Participants with AN showed effective connectivity from ventral striatum to hypothalamus, and connectivity strength was positively correlated with insula and orbitofrontal PE response. Right frontal cortex PE response was associated with cortisol, which correlated with body dissatisfaction.

CONCLUSIONS AND RELEVANCE

These results further support elevated PE signal in AN and suggest a link between PE and elevated harm avoidance, brain connectivity, and weight gain in AN. Prediction error may have a central role in adolescent AN in driving anxiety and ventral striatal-hypothalamus circuit-controlled food avoidance.

Are some individuals generally more behaviorally plastic than others? An experiment with sailfin mollies

Individuals within the same population generally differ among each other not only in their behavioral traits but also in their level of behavioral plasticity (i.e., in their propensity to modify their behavior in response to changing conditions). If the proximate factors underlying individual differences in behavioral plasticity were the same for any measure of plasticity, as commonly assumed, one would expect plasticity to be repeatable across behaviors and contexts. However, this assumption remains largely untested. Here, we conducted an experiment with sailfin mollies (Poecilia latipinna) whose behavioral plasticity was estimated both as the change in their personality traits or mating behavior across a social gradient and using their performance on a reversal-learning task. We found that the correlations between pairwise measures of plasticity were weak and non-significant, thus indicating that the most plastic individuals were not the same in all the tests. This finding might arise because either individuals adjust the magnitude of their behavioral responses depending on the benefits of plasticity, and/or individuals expressing high behavioral plasticity in one context are limited by neural and/or physiological constraints in the amount of plasticity they can express in other contexts. Because the repeatability of behavioral plasticity may have important evolutionary consequences, additional studies are needed to assess the importance of trade-offs between conflicting selection pressures on the maintenance of intra-individual variation in behavioral plasticity.

Sex: A Significant Risk Factor for Neurodevelopmental and Neurodegenerative Disorders

Males and females sometimes significantly differ in their propensity to develop neurological disorders. Females suffer more from mood disorders such as depression and anxiety, whereas males are more susceptible to deficits in the dopamine system including Parkinson's disease (PD), attention-deficit hyperactivity disorder (ADHD) and autism. Despite this, biological sex is rarely considered when making treatment decisions in neurological disorders. A better understanding of the molecular mechanism(s) underlying sex differences in the healthy and diseased brain will help to devise diagnostic and therapeutic strategies optimal for each sex. Thus, the aim of this review is to discuss the available evidence on sex differences in neuropsychiatric and neurodegenerative disorders regarding prevalence, progression, symptoms and response to therapy. We also discuss the sex-related factors such as gonadal sex hormones and sex chromosome genes and how these might help to explain some of the clinically observed sex differences in these disorders. In particular, we highlight the emerging role of the Y-chromosome gene, SRY, in the male brain and its potential role as a male-specific risk factor for disorders such as PD, autism, and ADHD in many individuals.

Touchscreen technology in the study of cognition-related behavior

There is a growing need for new translational animal models designed to capture complex behavioral phenotypes implicated in addiction and other neuropsychiatric conditions. For example, a complete understanding of the effects of commonly abused drugs, as well as candidate medications, requires assessments of their effects on learning, memory, attention, and other cognition-related behavior. Modern touch-sensitive technology provides an extremely flexible means to expose an experimental subject to a variety of complex behavioral tasks designed to assay dimensions of cognitive function before, during, and after drug administration. In addition to tailored variants of gold-standard cognitive assessments, touchscreen chambers offer the ability to develop novel tasks based upon the researcher's needs. This methods perspective presents (i) a brief review of previous touchscreen-based animal studies, (ii) a primer on the construction of a touch-sensitive experimental chamber, and (iii) data from a proof-of-concept study examining cross-species continuity in performance across a diverse assortment of animal subjects (rats, marmosets, squirrel monkeys, and rhesus macaques) using the repeated acquisition task - a modern variant of a traditional animal model of learning. Taken together, the procedures and data discussed in this review illustrate the point that contemporary touchscreen methodology can be tailored to desired experimental goals and adapted to provide formal similarity in cognition-related tasks across experimental species. Moreover, touchscreen methodology allows for the development of new translational models that emerge through laboratory and clinical discovery to capture important dimensions of complex behavior and cognitive function.

Intra-individual variation in performance on novel variants of similar tasks influences single factor explanations of general cognitive processes

Intra-individual variation in performance within and across cognitive domains may confound interpretations of both domain-general and domain-specific abilities. Such variation is rarely considered in animal test batteries. We investigate individual consistency in performance by presenting pheasant chicks (n = 31), raised under standardized conditions, with nine different cognitive tasks. Among these tasks were two replicated novel variants of colour learning and colour reversal problems, tests of positional learning and memory, as well as two different tasks that captured multiple putative measures of inhibitory control and motor-related performance. These task variants were also used to compare subjects' performance on alternative test batteries comprised of different task combinations. Subjects' performance improved with experience, yet we found relatively little consistency in their performance, both within similar tasks using different paradigms and across different tasks. Parallel analysis revealed non-significant factors when all nine tasks were included in a principal axis factor analysis. However, when different combinations of six of the nine tasks were included in principal axis factoring, 14 of 84 combinations revealed significant main factors, explaining between 28 and 35% of the variance in task performance. While comparable findings have been suggested to reflect domain-general intelligence in other species, we found no evidence to suggest that a single factor encompassed a diverse range of cognitive abilities in pheasants. Instead, we reveal how single factor explanations of cognitive processes can be influenced by test battery composition and intra-individual variation in performance across tasks. Our findings highlight the importance of conducting multiple tests within specific domains to ensure robust cognitive measures are obtained.

Sex-Dependent Effects of the Histone Deacetylase Inhibitor, Sodium Valproate, on Reversal Learning After Developmental Arsenic Exposure

Several studies have demonstrated that exposure to arsenic in drinking water adversely affects brain development and cognitive function in adulthood. While the mechanism by which arsenic induces adverse neurological outcomes remains elusive, studies suggest a link between reduced levels of histone acetylation and impaired performance on a variety of behavioral tasks following arsenic exposure. Using our developmental arsenic exposure (DAE) paradigm, we have previously reported reduced histone acetylation and associated histone acetyltransferase enzyme expression in the frontal cortex of C57BL/6J adult male mice, with no changes observed in the female frontal cortex. In the present study, we sought to determine if DAE produced sex-dependent deficits in frontal cortical executive function using the Y-maze acquisition and reversal learning tasks, which are specific for assessing cognitive flexibility. Further, we tested whether the administration of valproic acid, a class I-IIa histone deacetylase inhibitor, was able to mitigate behavioral and biochemical changes resulting from DAE. As anticipated, DAE inhibited acquisition and reversal learning performance in adult male, but not female, mice. Valproate treatment for 2 weeks restored reversal performance in the male arsenic-exposed offspring, while not affecting female performance. Protein levels of HDACs 1, 2, and 5 were elevated following behavioral assessment but only in DAE male mice; restoration of appropriate HDAC levels occurred after valproate treatment and was concurrent with improved behavioral performance, particularly during reversal learning. Female frontal cortical levels of HDAC enzymes were not impacted by DAE or valproate treatment. Finally, mRNA expression levels of brain-derived neurotrophic factor, Bdnf, which has been implicated in the control of frontal cortical flexibility and is regulated by HDAC5, were elevated in DAE male mice and restored to normal levels following HDACi treatment. Levels of mRNA encoding glutamate receptor ionotropic NMDA type subunits, which have been linked to cognitive flexibility, were not related to the reversal learning deficit in the DAE mice and were not altered by HDACi treatments. These findings demonstrate that DAE alters frontal cortical HDAC levels and Bdnf expression in males, but not females, and that these molecular changes are associated with sex-dependent differences in cognitive flexibility in a reversal-learning task.

Effects of HIV-1 TAT protein and methamphetamine exposure on visual discrimination and executive function in mice

Mild neurocognitive impairments are common in people with human immunodeficiency virus (HIV) infection. HIV-encoded proteins, such as trans-activator of transcription (TAT), contribute to neuropathology and cognitive function in medicated subjects. The combination of TAT and comorbid methamphetamine use may further impair neurocognitive function in HIV-positive individuals by affecting dopaminergic systems in the brain. The current study examined the effects of TAT protein expression and methamphetamine exposure on cognitive function and dopamine systems in mice. Transgenic mice with inducible brain expression of the TAT protein were exposed to a binge methamphetamine regimen. TAT expression was induced via a doxycycline-containing diet during the final stage of the regimen and maintained throughout cognitive testing. Learning and executive function were assessed using an operant visual discrimination protocol, with a strategy switch and reversal. TAT expression and methamphetamine exposure improved visual discrimination learning. Combined TAT expression and methamphetamine exposure increased perseverative errors during reversal learning. TAT expression altered reversal learning by improving early stage, but impairing late stage, learning. TAT expression was also associated with an increase in dopamine transporter expression in the caudate putamen. These results highlight that TAT expression and methamphetamine exposure likely affect a range of selective cognitive processes, with some potentially improving function under certain conditions.

Greater mindful eating practice is associated with better reversal learning

Mindfulness-based interventions are thought to reduce compulsive behavior such as overeating by promoting behavioral flexibility. Here the main aim was to provide support for mindfulness-mediated improvements in reversal learning, a direct measure of behavioral flexibility. We investigated whether an 8-week mindful eating intervention improved outcome-based reversal learning relative to an educational cooking (i.e., active control) intervention in a non-clinical population. Sixty-five healthy participants with a wide BMI range (19–35 kg/m²), who were motivated to change their eating habits, performed a deterministic reversal learning task that enabled the investigation of reward- and punishment-based reversal learning at baseline and following the intervention. No group differences in reversal learning were observed. However, time invested in the mindful eating, but not the educational cooking intervention correlated positively with changes in reversal learning, in a manner independent of outcome valence. These findings suggest that greater amount of mindfulness practice can lead to increased behavioral flexibility, which, in turn, might help overcome compulsive eating in clinical populations.

Dopamine D2 Receptors in Dopaminergic Neurons Modulate Performance in a Reversal Learning Task in Mice

Neuroimaging studies in animal models and human subjects have each revealed that relatively low striatal dopamine D2-like receptor binding potential is associated with poor impulse control and with vulnerability for addiction-related behaviors. These studies cannot, however, disambiguate the roles for various pools of D2 receptors found in the striatum (e.g., those expressed on medium spiny striato-pallidal neurons vs on dopamine-releasing nerve terminals) in these behavioral outcomes. To clarify the role of the latter pool, namely, D2 autoreceptors, we studied mice carrying a conditional DRD2 gene, with or without Cre-recombinase expressed under the transcriptional control of the dopamine transporter gene locus (autoDrd2-KO, n = 19 and controls, n = 21). These mice were tested for locomotor response to cocaine, and spatial reversal learning was assessed in operant conditioning chambers. As predicted, compared to control mice, autoDrd2-KO animals demonstrated heightened sensitivity to the locomotor stimulating effect of cocaine (10 mg/kg, i.p.), confirming previous research using a similar genetic model. In the spatial reversal learning task, autoDrd2-KO mice were slower to reach a learning criterion and had difficulty sustaining a prolonged nose poke response, measurements conceptually related to impaired response inhibition. Rate of learning of the initial discrimination and latencies to collect rewards, to initiate trials and to produce a response were unaffected by genetic deletion of D2 autoreceptors, discarding possible motor and motivational factors. Together, these findings confirm the role of D2 autoreceptors in reversal learning and suggest a broader involvement in behavioral inhibition mechanisms.

Shaping vulnerability to addiction - the contribution of behavior, neural circuits and molecular mechanisms

Substance use disorders continue to impose increasing medical, financial and emotional burdens on society in the form of morbidity and overdose, family disintegration, loss of employment and crime, while advances in prevention and treatment options remain limited. Importantly, not all individuals exposed to abused substances effectively develop the disease. Genetic factors play a significant role in determining addiction vulnerability and interactions between innate predisposition, environmental factors and personal experiences are also critical. Thus, understanding individual differences that contribute to the initiation of substance use as well as on long-term maladaptations driving compulsive drug use and relapse propensity is of critical importance to reduce this devastating disorder. In this paper, we discuss current topics in the field of addiction regarding individual vulnerability related to behavioral endophenotypes, neural circuits, as well as genetics and epigenetic mechanisms. Expanded knowledge of these factors is of importance to improve and personalize prevention and treatment interventions in the future.

Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience

The stagnation in drug development for schizophrenia highlights the need for better translation between basic and clinical research. Understanding the neurobiology of schizophrenia presents substantial challenges but a key feature continues to be the involvement of subcortical dopaminergic dysfunction in those with psychotic symptoms. Our contemporary knowledge regarding dopamine dysfunction has clarified where and when dopaminergic alterations may present in schizophrenia. For example, clinical studies have shown patients with schizophrenia show increased presynaptic dopamine function in the associative striatum, rather than the limbic striatum as previously presumed. Furthermore, subjects deemed at high risk of developing schizophrenia show similar presynaptic dopamine abnormalities in the associative striatum. Thus, our view of subcortical dopamine function in schizophrenia continues to evolve as we accommodate this newly acquired information. However, basic research in animal models has been slow to incorporate these clinical findings. For example, psychostimulant-induced locomotion, the commonly utilised phenotype for positive symptoms in rodents, is heavily associated with dopaminergic activation in the limbic striatum. This anatomical misalignment has brought into question how we assess positive symptoms in animal models and represents an opportunity for improved translation between basic and clinical research. The current review focuses on the role of subcortical dopamine dysfunction in psychosis and schizophrenia. We present and discuss alternative phenotypes that may provide a more translational approach to assess the neurobiology of positive symptoms in schizophrenia. Incorporation of recent clinical findings is essential if we are to develop meaningful translational animal models.

TGF-β Signaling in Dopaminergic Neurons Regulates Dendritic Growth, Excitatory-Inhibitory Synaptic Balance, and Reversal Learning

Neural circuits involving midbrain dopaminergic (DA) neurons regulate reward and goal-directed behaviors. Although local GABAergic input is known to modulate DA circuits, the mechanism that controls excitatory/inhibitory synaptic balance in DA neurons remains unclear. Here, we show that DA neurons use autocrine transforming growth factor β (TGF-β) signaling to promote the growth of axons and dendrites. Surprisingly, removing TGF-β type II receptor in DA neurons also disrupts the balance in TGF-β1 expression in DA neurons and neighboring GABAergic neurons, which increases inhibitory input, reduces excitatory synaptic input, and alters phasic firing patterns in DA neurons. Mice lacking TGF-β signaling in DA neurons are hyperactive and exhibit inflexibility in relinquishing learned behaviors and re-establishing new stimulus-reward associations. These results support a role for TGF-β in regulating the delicate balance of excitatory/inhibitory synaptic input in local microcircuits involving DA and GABAergic neurons and its potential contributions to neuropsychiatric disorders.

Cognitive test batteries in animal cognition research: evaluating the past, present and future of comparative psychometrics

For the past two decades, behavioural ecologists have documented consistent individual differences in behavioural traits within species and found evidence for animal "personality". It is only relatively recently, however, that increasing numbers of researchers have begun to investigate individual differences in cognitive ability within species. It has been suggested that cognitive test batteries may provide an ideal tool for this growing research endeavour. In fact, cognitive test batteries have now been used to examine the causes, consequences and underlying structure of cognitive performance within and between many species. In this review, we document the existing attempts to develop cognitive test batteries for non-human animals and review the claims that these studies have made in terms of the structure and evolution of cognition. We argue that our current test battery methods could be improved on multiple fronts, from the design of tasks, to the domains targeted and the species tested. Refining and optimising test battery design will provide many benefits. In future, we envisage that well-designed cognitive test batteries may provide answers to a range of exciting questions, including giving us greater insight into the evolution and structure of cognition.

Contrafreeloading in Rats Is Adaptive and Flexible: Support for an Animal Model of Compulsive Checking

Contrafreeloading involves working unnecessarily to obtain a reward that is otherwise freely available. It has been observed in numerous species and can be adaptive when it provides an organism with updated information about available resources. Humans frequently update their knowledge of the environment through checking behaviors. Compulsive checking occurs when such actions are performed with excessive frequency. In a putative animal model of compulsive checking, rats treated chronically with the dopamine agonist quinpirole display exaggerated contrafreeloading for water. Although this effect has been attributed to behavioral rigidity, some evidence suggests the behavior remains somewhat flexible and may be adaptive under certain conditions. We assessed the ability of quinpirole-treated rats with contrafreeloading experience to adapt to changing contingencies by requiring them to alternate between response levers. Rats treated with quinpirole or saline were first trained to obtain water by pressing either of two levers. Next, free water was made available for 8 days, and contrafreeloading was measured. Rates of contrafreeloading were significantly higher in the drug-treated rats than in controls. On the following 5 days, each reward caused the associated lever to become inactive until a reward was earned from the alternate lever. Quinpirole-treated rats learned this new response requirement more quickly than controls. Thus, exaggerated checking behavior induced by chronic quinpirole treatment can be advantageous when environmental contingencies change. These results provide support for this animal model of compulsive checking and hint at the presence of a specialized neural checking module involving the dopamine system.

Alterations in brain activation in response to prolonged morphine withdrawal-induced behavioral inflexibility in rats

RATIONALE

The inability to stop a repetitive maladaptive behavior is a main problem in addictive disorders. Neuroadaptations that are associated with behavioral inflexibility may be involved in compulsive drug use.

OBJECTIVES

The aim of the present study was to investigate the pattern of behavioral inflexibility during morphine withdrawal and map brain activation that is linked to alterations in flexibility.

METHODS

We first analyzed the effects of chronic morphine exposure on reversal learning after 2-week (short-term) and 6-week (prolonged) morphine withdrawal. We then compared the level of neuronal activation using cFos immunohistochemistry in 15 brain areas between rats that underwent morphine withdrawal and saline-control rats after a test of reversal learning.

RESULTS

Only prolonged morphine withdrawal impaired reversal learning. Rats that exhibited impairments in reversal learning presented a significant decrease in cFos expression in the orbitofrontal cortex (OFC), including the medial, lateral, and ventral OFC. cFos expression significantly increased in the dorsomedial striatum and major subregions of the medial prefrontal cortex (mPFC) in the morphine group. Rats that underwent prolonged morphine withdrawal exhibited no significant changes in cFos expression in the dorsolateral striatum, nucleus accumbens, amygdala, paraventricular thalamic nucleus, or motor cortex. The rats that underwent short-term withdrawal did not present any changes in cFos expression in any of these brain regions.

CONCLUSION

Altogether, these data suggest that alterations in the function of the frontal cortex and its striatal connections during the late morphine withdrawal phase may underlie the disruption of inhibitory control in opioid dependence.

Acute Δ-9-tetrahydrocannabinol administration in female rats attenuates immediate responses following losses but not multi-trial reinforcement learning from wins

Δ-9-Tetrahydrocannabinol (THC) is the main psychoactive component of marijuana and has potent effects on decision-making, including a proposed reduction in cognitive flexibility. We demonstrate here that acute THC administration differentially affects some of the processes that contribute to cognitive flexibility. Specifically, THC reduces lose-shift responding in which female rats tend to immediately shift choice responses away from options that result in reward omission on the previous trial. THC, however, did not impair the ability of rats to flexibly bias responses toward feeders with higher probability of reward in a reversal task. This response adaptation developed over several trials, suggesting that THC did not impair slower forms of reinforcement learning needed to choose among options with unequal utility. This dissociation of THC's effects on innate/rapid and learned/gradual decision-making processes was unexpected, but is supported by emerging evidence that lose-shift responding is mediated by neural mechanisms distinct from those involved in other forms of reinforcement learning. The present data suggest that, at least in some tasks, the apparent reductions in cognitive flexibility by THC may be explained by the immediate effects on loss sensitivity, rather than impairments of all processes used for choice adaptation.

Stuttering Thoughts: Negative Self-Referent Thinking Is Less Sensitive to Aversive Outcomes in People with Higher Levels of Depressive Symptoms

Learning theories of depression have proposed that depressive cognitions, such as negative thoughts with reference to oneself, can develop through a reinforcement learning mechanism. This negative self-reference is considered to be positively reinforced by rewarding experiences such as genuine support from others after negative self-disclosure, and negatively reinforced by avoidance of potential aversive situations. The learning account additionally predicts that negative self-reference would be maintained by an inability to adjust one’s behavior when negative self-reference no longer leads to such reward. To test this prediction, we designed an adapted version of the reversal-learning task. In this task, participants were reinforced to choose and engage in either negative or positive self-reference by probabilistic economic reward and punishment. Although participants were initially trained to choose negative self-reference, the stimulus-reward contingencies were reversed to prompt a shift toward positive self-reference (Study 1) and a further shift toward negative self-reference (Study 2). Model-based computational analyses showed that depressive symptoms were associated with a low learning rate of negative self-reference, indicating a high level of reward expectancy for negative self-reference even after the contingency reversal. Furthermore, the difficulty in updating outcome predictions of negative self-reference was significantly associated with the extent to which one possesses negative self-images. These results suggest that difficulty in adjusting action-outcome estimates for negative self-reference increases the chance to be faced with negative aspects of self, which may result in depressive symptoms.

Basic reversal-learning capacity in flies suggests rudiments of complex cognition

The most basic models of learning are reinforcement learning models (for instance, classical and operant conditioning) that posit a constant learning rate; however many animals change their learning rates with experience. This process is sometimes studied by reversing an existing association between cues and rewards, and measuring the rate of relearning. Augmented reversal-learning, where learning rates increase with practice, can be an important component of behavioral flexibility; and may provide insight into higher cognition. Previous studies of reversal-learning in Drosophila have not measured learning rates, but have tended to focus on measuring gross deficits in reversal-learning, as the ratio of two timepoints. These studies have uncovered a diversity of mechanisms underlying reversal-learning, but natural genetic variation in this trait has yet to be assessed. We conducted a reversal-learning regime on a diverse panel of Drosophila melanogaster genotypes. We found highly significant genetic variation in their baseline ability to learn. We also found that they have a consistent, and strong (1.3×), increase in their learning speed with reversal. We found no evidence, however, that there was genetic variation in their ability to increase their learning rates with experience. This may suggest that Drosophila have a hitherto unrecognized ability to integrate acquired information, and improve their decision making; but that their mechanisms for doing so are under strong constraints.

Behavioral and Neural Signatures of Reduced Updating of Alternative Options in Alcohol-Dependent Patients during Flexible Decision-Making

Addicted individuals continue substance use despite the knowledge of harmful consequences and often report having no choice but to consume. Computational psychiatry accounts have linked this clinical observation to difficulties in making flexible and goal-directed decisions in dynamic environments via consideration of potential alternative choices. To probe this in alcohol-dependent patients (n = 43) versus healthy volunteers (n = 35), human participants performed an anticorrelated decision-making task during functional neuroimaging. Via computational modeling, we investigated behavioral and neural signatures of inference regarding the alternative option. While healthy control subjects exploited the anticorrelated structure of the task to guide decision-making, alcohol-dependent patients were relatively better explained by a model-free strategy due to reduced inference on the alternative option after punishment. Whereas model-free prediction error signals were preserved, alcohol-dependent patients exhibited blunted medial prefrontal signatures of inference on the alternative option. This reduction was associated with patients' behavioral deficit in updating the alternative choice option and their obsessive-compulsive drinking habits. All results remained significant when adjusting for potential confounders (e.g., neuropsychological measures and gray matter density). A disturbed integration of alternative choice options implemented by the medial prefrontal cortex appears to be one important explanation for the puzzling question of why addicted individuals continue drug consumption despite negative consequences.

SIGNIFICANCE STATEMENT

In addiction, patients maintain substance use despite devastating consequences and often report having no choice but to consume. These clinical observations have been theoretically linked to disturbed mechanisms of inference, for example, to difficulties when learning statistical regularities of the environmental structure to guide decisions. Using computational modeling, we demonstrate disturbed inference on alternative choice options in alcohol addiction. Patients neglecting "what might have happened" was accompanied by blunted coding of inference regarding alternative choice options in the medial prefrontal cortex. An impaired integration of alternative choice options implemented by the medial prefrontal cortex might contribute to ongoing drug consumption in the face of evident negative consequences.

Dopamine D3 Receptor Availability Is Associated with Inflexible Decision Making

Dopamine D2/3 receptor signaling is critical for flexible adaptive behavior; however, it is unclear whether D2, D3, or both receptor subtypes modulate precise signals of feedback and reward history that underlie optimal decision making. Here, PET with the radioligand [(11)C]-(+)-PHNO was used to quantify individual differences in putative D3 receptor availability in rodents trained on a novel three-choice spatial acquisition and reversal-learning task with probabilistic reinforcement. Binding of [(11)C]-(+)-PHNO in the midbrain was negatively related to the ability of rats to adapt to changes in rewarded locations, but not to the initial learning. Computational modeling of choice behavior in the reversal phase indicated that [(11)C]-(+)-PHNO binding in the midbrain was related to the learning rate and sensitivity to positive, but not negative, feedback. Administration of a D3-preferring agonist likewise impaired reversal performance by reducing the learning rate and sensitivity to positive feedback. These results demonstrate a previously unrecognized role for D3 receptors in select aspects of reinforcement learning and suggest that individual variation in midbrain D3 receptors influences flexible behavior. Our combined neuroimaging, behavioral, pharmacological, and computational approach implicates the dopamine D3 receptor in decision-making processes that are altered in psychiatric disorders.

SIGNIFICANCE STATEMENT

Flexible decision-making behavior is dependent upon dopamine D2/3 signaling in corticostriatal brain regions. However, the role of D3 receptors in adaptive, goal-directed behavior has not been thoroughly investigated. By combining PET imaging with the D3-preferring radioligand [(11)C]-(+)-PHNO, pharmacology, a novel three-choice probabilistic discrimination and reversal task and computational modeling of behavior in rats, we report that naturally occurring variation in [(11)C]-(+)-PHNO receptor availability relates to specific aspects of flexible decision making. We confirm these relationships using a D3-preferring agonist, thus identifying a unique role of midbrain D3 receptors in decision-making processes.

Immune activation in lactating dams alters sucklings' brain cytokines and produces non-overlapping behavioral deficits in adult female and male offspring: A novel neurodevelopmental model of sex-specific psychopathology

Early immune activation (IA) in rodents, prenatal through the mother or early postnatal directly to the neonate, is widely used to produce behavioral endophenotypes relevant to schizophrenia and depression. Given that maternal immune response plays a crucial role in the deleterious effects of prenatal IA, and lactation is a critical vehicle of immunological support to the neonate, we predicted that immune activation of the lactating dam will produce long-term abnormalities in the sucklings. Nursing dams were injected on postnatal day 4 with the viral mimic poly-I:C (4mg/kg) or saline. Cytokine assessment was performed in dams' plasma and milk 2h, and in the sucklings' hippocampus, 6h and 24h following poly-I:C injection. Male and female sucklings were assessed in adulthood for: a) performance on behavioral tasks measuring constructs considered relevant to schizophrenia (selective attention and executive control) and depression (despair and anhedonia); b) response to relevant pharmacological treatments; c) brain structural changes. Maternal poly-I:C injection caused cytokine alterations in the dams' plasma and milk, as well as in the sucklings' hippocampus. Lactational poly-I:C exposure led to sex-dimorphic (non-overlapping) behavioral abnormalities in the adult offspring, with male but not female offspring exhibiting attentional and executive function abnormalities (manifested in persistent latent inhibition and slow reversal) and hypodopaminergia, and female but not male offspring exhibiting despair and anhedonia (manifested in increased immobility in the forced swim test and reduced saccharine preference) and hyperdopaminergia, mimicking the known sex-bias in schizophrenia and depression. The behavioral double-dissociation predicted distinct pharmacological profiles, recapitulating the pharmacology of negative/cognitive symptoms and depression. In-vivo imaging revealed hippocampal and striatal volume reductions in both sexes, as found in both disorders. This is the first evidence for the emergence of long-term behavioral and brain abnormalities after lactational exposure to an inflammatory agent, supporting a causal link between early immune activation and disrupted neuropsychodevelopment. That such exposure produces schizophrenia- or depression-like phenotype depending on sex, resonates with notions that risk factors are transdiagnostic, and that sex is a susceptibility factor for neurodevelopmental psychopathologies.

Neural Signatures of Cognitive Flexibility and Reward Sensitivity Following Nicotinic Receptor Stimulation in Dependent Smokers: A Randomized Trial

IMPORTANCE

Withdrawal from nicotine is an important contributor to smoking relapse. Understanding how reward-based decision making is affected by abstinence and by pharmacotherapies such as nicotine replacement therapy and varenicline tartrate may aid cessation treatment.

OBJECTIVE

To independently assess the effects of nicotine dependence and stimulation of the nicotinic acetylcholine receptor on the ability to interpret valence information (reward sensitivity) and subsequently alter behavior as reward contingencies change (cognitive flexibility) in a probabilistic reversal learning task.

DESIGN, SETTING, AND PARTICIPANTS

Nicotine-dependent smokers and nonsmokers completed a probabilistic reversal learning task during acquisition of functional magnetic resonance imaging (fMRI) in a 2-drug, double-blind placebo-controlled crossover design conducted from January 21, 2009, to September 29, 2011. Smokers were abstinent from cigarette smoking for 12 hours for all sessions. In a fully Latin square fashion, participants in both groups underwent MRI twice while receiving varenicline and twice while receiving a placebo pill, wearing either a nicotine or a placebo patch. Imaging analysis was performed from June 15, 2015, to August 10, 2016.

MAIN OUTCOME AND MEASURES

A well-established computational model captured effects of smoking status and administration of nicotine and varenicline on probabilistic reversal learning choice behavior. Neural effects of smoking status, nicotine, and varenicline were tested for on MRI contrasts that captured reward sensitivity and cognitive flexibility.

RESULTS

The study included 24 nicotine-dependent smokers (12 women and 12 men; mean [SD] age, 35.8 [9.9] years) and 20 nonsmokers (10 women and 10 men; mean [SD] age, 30.4 [7.2] years). Computational modeling indicated that abstinent smokers were biased toward response shifting and that their decisions were less sensitive to the available evidence, suggesting increased impulsivity during withdrawal. These behavioral impairments were mitigated with nicotine and varenicline. Similarly, decreased mesocorticolimbic activity associated with cognitive flexibility in abstinent smokers was restored to the level of nonsmokers following stimulation of nicotinic acetylcholine receptors (familywise error-corrected P < .05). Conversely, neural signatures of decreased reward sensitivity in smokers (vs nonsmokers; familywise error-corrected P < .05) in the dorsal striatum and anterior cingulate cortex were not mitigated by nicotine or varenicline.

CONCLUSIONS AND RELEVANCE

There was a double dissociation between the effects of chronic nicotine dependence on neural representations of reward sensitivity and acute effects of stimulation of nicotinic acetylcholine receptors on behavioral and neural signatures of cognitive flexibility in smokers. These chronic and acute pharmacologic effects were observed in overlapping mesocorticolimbic regions, suggesting that available pharmacotherapies may alleviate deficits in the same circuitry for certain mental computations but not for others.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT00830739.

Measures of Dogs' Inhibitory Control Abilities Do Not Correlate across Tasks

Inhibitory control, the ability to overcome prepotent but ineffective behaviors, has been studied extensively across species, revealing the involvement of this ability in many different aspects of life. While various different paradigms have been created in order to measure inhibitory control, only a limited number of studies have investigated whether such measurements indeed evaluate the same underlying mechanism, especially in non-human animals. In humans, inhibitory control is a complex construct composed of distinct behavioral processes rather than of a single unified measure. In the current study, we aimed to investigate the validity of inhibitory control paradigms in dogs. Sixty-seven dogs were tested in a battery consisting of frequently used inhibitory control tests. Additionally, dog owners were asked to complete an impulsivity questionnaire about their dog. No correlation of dogs' performance across tasks was found. In order to understand whether there are some underlying behavioral aspects explaining dogs' performance across tests, we performed principle component analyses. Results revealed that three components (persistency, compulsivity and decision speed) explained the variation across tasks. The questionnaire and dogs' individual characteristics (i.e., age and sex) provided only limited information for the derived components. Overall, results suggest that no unique measurement for inhibitory control exists in dogs, but tests rather measure different aspects of this ability. Considering the context-specificity of inhibitory control in dogs and most probably also in other non-human animals, extreme caution is needed when making conclusions about inhibitory control abilities based on a single test.

Dorsolateral prefrontal cortex contributes to the impaired behavioral adaptation in alcohol dependence

Substance-dependent individuals often lack the ability to adjust decisions flexibly in response to the changes in reward contingencies. Prediction errors (PEs) are thought to mediate flexible decision-making by updating the reward values associated with available actions. In this study, we explored whether the neurobiological correlates of PEs are altered in alcohol dependence. Behavioral, and functional magnetic resonance imaging (fMRI) data were simultaneously acquired from 34 abstinent alcohol-dependent patients (ADP) and 26 healthy controls (HC) during a probabilistic reward-guided decision-making task with dynamically changing reinforcement contingencies. A hierarchical Bayesian inference method was used to fit and compare learning models with different assumptions about the amount of task-related information subjects may have inferred during the experiment. Here, we observed that the best-fitting model was a modified Rescorla-Wagner type model, the “double-update” model, which assumes that subjects infer the knowledge that reward contingencies are anti-correlated, and integrate both actual and hypothetical outcomes into their decisions. Moreover, comparison of the best-fitting model's parameters showed that ADP were less sensitive to punishments compared to HC. Hence, decisions of ADP after punishments were loosely coupled with the expected reward values assigned to them. A correlation analysis between the model-generated PEs and the fMRI data revealed a reduced association between these PEs and the BOLD activity in the dorsolateral prefrontal cortex (DLPFC) of ADP. A hemispheric asymmetry was observed in the DLPFC when positive and negative PE signals were analyzed separately. The right DLPFC activity in ADP showed a reduced correlation with positive PEs. On the other hand, ADP, particularly the patients with high dependence severity, recruited the left DLPFC to a lesser extent than HC for processing negative PE signals. These results suggest that the DLPFC, which has been linked to adaptive control of action selection, may play an important role in cognitive inflexibility observed in alcohol dependence when reinforcement contingencies change. Particularly, the left DLPFC may contribute to this impaired behavioral adaptation, possibly by impeding the extinction of the actions that no longer lead to a reward.

•

Alcohol-dependent patients (ADP) had difficulty adapting to the reversals.

•

The impaired adaptation was associated with a decrease in punishment sensitivity.

•

The dorsolateral prefrontal cortex (DLPFC) of ADP failed to track prediction errors.

•

A reduced tracking of the negative prediction error was present in the left DLPFC.

•

The clinical severity of dependence was correlated with abnormal DLPFC activity.

BALB/c Mice Can Learn Touchscreen Visual Discrimination and Reversal Tasks Faster than C57BL/6 Mice

Touchscreen technology is increasingly being used to characterize cognitive performance in rodent models of neuropsychiatric disorders. Researchers are attracted to the automated system and translational potential for touchscreen-based tasks. However, training time is extensive and some mouse strains have struggled to learn touchscreen tasks. Here we compared the performance of commonly used C57BL/6 mice against the BALB/c mice, which are considered a poor performing strain, using a touchscreen task. BALB/c and C57BL/6 mice were trained to operate the touchscreens before learning a visual discrimination (VD) and reversal task. Following touchscreen testing, these strains were assessed for differences in locomotion and learned helplessness. BALB/c mice finished training in nearly half the number of sessions taken by C57BL/6 mice. Following training, mice learned a VD task where BALB/c mice again reached criteria in fewer than half the sessions required for C57BL/6 mice. Once acquired, there were no strain differences in % correct responses, correction trials or response latency. BALB/c mice also learnt the reversal task in significantly fewer sessions than C57BL/6 mice. On the open field test C57BL/6 mice traveled further and spent more time in the center, and spent less time immobile than BALB/c mice on the forced swim test (FST). After touchscreen testing, strains exhibited well-established behavioral traits demonstrating the extensive training and handling from touchscreen testing did not alter their behavioral phenotype. These results suggest that BALB/c mice can be examined using touchscreen tasks and that task adaptations may improve feasibility for researchers using different strains.

Touch screen assays of behavioural flexibility and error characteristics in Eastern grey squirrels (Sciurus carolinensis)

Behavioural flexibility allows animals to adjust their behaviours according to changing environmental demands. Such flexibility is frequently assessed by the discrimination–reversal learning task. We examined grey squirrels’ behavioural flexibility, using a simultaneous colour discrimination–reversal learning task on a touch screen. Squirrels were trained to select their non-preferred colour in the discrimination phase, and their preferred colour was rewarded in a subsequent reversal phase. We used error rates to divide learning in each phase into three stages (perseveration, chance level and ‘learned’) and examined response inhibition and head-switching during each stage. We found consistent behavioural patterns were associated with each learning stage: in the perseveration stage, at the beginning of each training phase, squirrels showed comparable response latencies to correct and incorrect stimuli, along with a low level of head-switching. They quickly overcame perseveration, typically in one to three training blocks. In the chance-level stage, response latencies to both stimuli were low, but during initial discrimination squirrels showed more head-switches than in the previous stage. This suggests that squirrels were learning the current reward contingency by responding rapidly to a stimulus, but with increased attention to both stimuli. In the learned stage, response latencies to the correct stimulus and the number of head-switches were at their highest, whereas incorrect response latencies were at their lowest, and differed significantly from correct response latencies. These results suggest increased response inhibition and attention allowed the squirrels to minimise errors. They also suggest that errors in the ‘learned’ stage were related to impulsive emission of the pre-potent or previously learned responses.

Impulse Control Disorders - The Continuum Hypothesis

The group Parkinson Inside Out is composed of health professionals and academic researchers who have been diagnosed with Parkinson's Disease. In our discussions we try to make use of both our inside perspective as patients, and our outside perspective as professionals. In this paper, we apply the two perspectives to the Impulse Control Disorders. These impulsive behaviour patterns are thought to be relatively uncommon side effects of some of the medication used in dopamine replacement therapy. The phenomenon is usually described as relatively rare (<15%), and mainly confined to patients with special vulnerabilities. In contrast, we propose that having some problems with controlling impulses is a very common experience for patients undergoing dopamine replacement therapy. They result from difficulties in decision making engendered by variations in dopamine accessibility in the reward centre of the brain. Only in a minority do the consequences grow to the damaging proportions of a disorder, but most patients are probably affected to some degree. Seeing, and measuring, decision difficulties as a continuous dimension, rather than as a discrete category, brings increased possibilities for early detection and continuous monitoring. With reliable measures of the propensity for impulsive decision making, it may become possible to both reap the benefits and avoid the dangers of the dopamine agonists. We point to ways of empirically testing our continuity hypothesis.

Rats' midsession reversal performance: the nature of the response

The midsession reversal task involves a simple simultaneous discrimination that predictably reverses midway through a session. Under various conditions, pigeons generally both anticipate the reversal and perseverate once it has occurred, whereas rats tend to make very few of either kind of error. In the present research, we investigated the hypothesis that the difference in performance between rats and pigeons is related to the nature of the responses made. We hypothesized that rats could have been better at bridging the intertrial interval by keeping the relevant paw close to the lever while eating, whereas the pigeons had to remove their beak from the response key and insert it into the feeder, thus making it difficult to mediate the response last made. In the present experiment, in successive phases, rats were trained to leverpress or nose-poke on a 40-trial midsession reversal, an 80-trial midsession reversal, and a variable-location reversal. The results showed that the leverpress group acquired the task faster than the nose-poke group, but that both groups reached comparable levels of performance. Thus, the difference in the natures of the responses cannot fully account for the differences in accuracy between rats and pigeons. Additionally, differences in the types of errors made by the two groups suggest that the nature of the response plays different roles in the performance of this task.

Testing the predictions of coping styles theory in threespined sticklebacks

Coping styles theory provides a framework for understanding individual variation in how animals respond to environmental change, and predicts how individual differences in stress responsiveness and behavior might relate to cognitive differences. According to coping styles theory, proactive individuals are bolder, less reactive to stressors, and more routinized than their reactive counterparts. A key tenet of coping styles theory is that variation in coping styles is maintained by tradeoffs with behavioral flexibility: proactive individuals excel in stable environments while more flexible, reactive individuals perform better in variable environments. Here, we assess evidence for coping styles within a natural population of threespined sticklebacks (Gasterosteus aculeatus). We developed a criterion-based learning paradigm to evaluate individual variation in initial and reversal learning. We observed strong individual differences in boldness, cortisol production, and learning performance. Consistent with coping styles, fish that released more cortisol were more timid in response to a predator attack and slower to learn a color discrimination task. However, there was no evidence that reactive individuals performed better when the environment changed (when the rewarded color was reversed). The failure to detect trade-offs between behavioral routinization and flexibility prompts other explanations for the maintenance of differing coping styles.

Measuring discrimination- and reversal learning in mouse models within 4 days and without prior food deprivation

Many neurological and psychiatric disorders are characterized by deficits in cognitive flexibility. Modeling cognitive flexibility in mice enables the investigation of mechanisms underlying these deficits. The majority of currently available behavioral tests targeting this cognitive domain are reversal learning tasks that require scheduled food restriction, extended training periods and labor-intensive, and stress-inducing animal handling. Here, we describe a novel 4-day (4-d) continuously running task measuring discrimination- and reversal learning in an automated home cage (CognitionWall DL/RL task) that largely eliminates these limitations. In this task, mice can earn unlimited number of food rewards by passing through the correct hole of the three-holed CognitionWall. To assess the validity and sensitivity of this novel task, the performance of C57BL/6J mice, amyloid precursor protein/presenilin1 transgenic (APP/PS1) mice, α-calmodulin kinase-II (αCaMKII) T305D knock-in mice, and mice with an orbitofrontal cortex lesion were examined. We found that C57BL/6J mice reach stable performance levels within the 4 d of the task, while experiencing only slight reductions in weight and no major effects on circadian rhythm. The task detected learning deficits in APP/PS1 transgenic and αCaMKII T305D mutant mice. Additionally, we established that the orbitofrontal cortex underlies reversal learning performance in our task. Because of its short duration and the absence of food deprivation and concurrent weight loss, this novel automated home-cage task substantially improves comprehensive preclinical assessment of cognitive functions in mouse models of psychiatric and neurological disorders and also enables analysis during specific developmental stages.

Blunted Dopamine Transmission in Addiction: Potential Mechanisms and Implications for Behavior

Positron emission tomography (PET) imaging consistently shows blunted striatal dopamine release and decreased dopamine D2 receptor availability in addiction. Here, we review the preclinical and clinical studies indicating that this neurobiological phenotype is likely to be both a consequence of chronic drug consumption and a vulnerability factor in the development of addiction. We propose that, behaviorally, blunted striatal dopamine transmission could reflect the increased impulsivity and altered cost/benefit computations that are associated with addiction. The factors that influence blunted striatal dopamine transmission in addiction are unknown. Herein, we give an overview of various factors, genetic, environmental, and social, that are known to affect dopamine transmission and that have been associated with the vulnerability to develop addiction. Altogether, these data suggest that blunted dopamine transmission and decreased D2 receptor availability are biomarkers both for the development of addiction and resistance to treatment. These findings support the view that blunted dopamine reflects impulsive behavior and deficits in motivation, which lead to the escalation of drug use.

Pallidal spiking activity reflects learning dynamics and predicts performance

The basal ganglia (BG) network has been divided into interacting actor and critic components, modulating the probabilities of different state-action combinations through learning. Most models of learning and decision making in the BG focus on the roles of the striatum and its dopaminergic inputs, commonly overlooking the complexities and interactions of BG downstream nuclei. In this study, we aimed to reveal the learning-related activity of the external segment of the globus pallidus (GPe), a downstream structure whose computational role has remained relatively unexplored. Recording from monkeys engaged in a deterministic three-choice reversal learning task, we found that changes in GPe discharge rates predicted subsequent behavioral shifts on a trial-by-trial basis. Furthermore, the activity following the shift encoded whether it resulted in reward or not. The frequent changes in stimulus-outcome contingencies (i.e., reversals) allowed us to examine the learning-related neural activity and show that GPe discharge rates closely matched across-trial learning dynamics. Additionally, firing rates exhibited a linear decrease in sequences of correct responses, possibly reflecting a gradual shift from goal-directed execution to automaticity. Thus, modulations in GPe spiking activity are highest for attention-demanding aspects of behavior (i.e., switching choices) and decrease as attentional demands decline (i.e., as performance becomes automatic). These findings are contrasted with results from striatal tonically active neurons, which show none of these task-related modulations. Our results demonstrate that GPe, commonly studied in motor contexts, takes part in cognitive functions, in which movement plays a marginal role.

Compulsivity Across the Pathological Misuse of Drug and Non-Drug Rewards

Behavioral adaptation is required for the successful navigation of a constantly changing environment. Impairments in behavioral flexibility are commonly observed in psychiatric disorders including those of addiction. This study investigates two distinct facets of compulsivity, namely reversal learning and attentional set shifting, implicating orbitofrontal and lateral prefrontal regions respectively, across disorders of primary and secondary rewards. Obese subjects with and without binge eating disorder (BED), individuals with compulsive sexual behaviors (CSB), alcohol dependence (AD) and pathological video-gaming (VG) were tested with two computerized tasks: the probabilistic reversal task (trials to criterion and win-stay/lose-shift errors) and the intra/extra-dimensional set shift task (IED). Individuals with AD and pathological VG were slower at reversal learning irrespective of valence, with AD subjects more likely to perseverate after losses. Compared to obese subjects without BED, BED subjects were worse at reversal learning to wins but better at losses highlighting valence effects as a function of binge eating. CSB subjects demonstrated enhanced sensitivity to reward outcomes with faster acquisition and greater perseveration with higher magnitude rewards. We further show an impairment in attentional set shifting in individuals with BED and AD relative to healthy volunteers (HV). This study provides evidence for commonalities and differences in two distinct dimensions of behavioral inflexibility across disorders of compulsivity. We summarize studies on compulsivity subtypes within this same patient population. We emphasize commonalities in AD and BED with impairments across a range of compulsivity indices, perhaps supporting pathological binge eating as a form of behavioral addiction. We further emphasize commonalities in reversal learning across disorders and the crucial role of valence effects. These findings highlight the role of behavioral inflexibility and compulsivity as a relevant domain in defining dimensional psychiatry and the identification of relevant cognitive endophenotypes as targets for therapeutic modulation.

Effects of Dexamphetamine and Music on Reversal Learning

Background:

Reversal learning has proven to be a valuable task in assessing the inhibitory process that is central to executive control. Psycho-stimulants and music are prevalent factors that influence cognition.

Objectives:

The present study aimed at investigating the influences of dexamphetamine and music on inhibitory control.

Materials and Methods:

This experimental study was conducted between May and June 2014 in the laboratory animal center of Shiraz University of Medical Sciences, Shiraz, Iran. Thirty mice were divided to five groups including a control group, a witness group, and three experimental groups. Food availability was restricted in order to maintain the subjects at 85% of their free-feeding body weight for behavioral testing. After discrimination learning, animals received four injections of 2 mg/kg dexamphetamine at two-hour intervals. The music group was exposed to music half an hour before reversal learning.

Results:

According to the results of the repeated measure analysis of variance (ANOVA), music increased errors (mean difference: -2.40, 95% CI: -3.59 to -1.22), yet dexamphetamine had no significant effect on reversal learning. Due to various advantages, we transited to the mixed model that showed increasing (Beta: 2.2 95% CI: 0.26 to 4.13) and borderline (Beta: 1.8 95% CI: -0.13 to 3.73) effects on the number of errors for dexamphetamine and music group, respectively.

Conclusions:

Drug-treated subjects were impaired in their ability to modulate behavior, based upon changing information about stimulus-reward associations, possibly due to the inability to inhibit their response. These effects may have relevance to some mental disorders such as drug-abuse, schizophrenia, and obsessive-compulsive disorder.

Chronic methamphetamine self-administration alters cognitive flexibility in male rats

RATIONALE

Methamphetamine (meth) addiction is a chronically relapsing disorder that often produces persistent cognitive deficits. These include decreased cognitive flexibility, which may prevent meth addicts from altering their habitual drug abuse and leave them more susceptible to relapse. Multiple factors including low rates of compliance with research study participation and varied drug use patterns make the relationship between cognitive flexibility and relapse difficult to establish in clinical populations.

OBJECTIVES

Here, we combined an extended-access meth self-administration paradigm with an automated set-shifting task in rats to directly compare cognitive flexibility performance with meth-seeking behavior.

METHODS

Rats were pre-trained on an automated visual discrimination task, followed by 14 days of extended access (6 h/day) of meth or sucrose self-administration. They were then tested in the set-shifting task on strategy shift and reversal and subsequently assessed for cue-induced reinstatement of meth seeking.

RESULTS

Rats with a history of meth, but not sucrose, self-administration had selective deficits in reversal learning. Specifically, meth rats had an increase in the total number of errors and perseverative errors (corresponding to the old stimulus-reward association) following the reversal shift, which correlated with prior stable meth self-administration. However, no relationship was seen between errors during the reversal and cue-induced reinstatement.

CONCLUSION

The lack of association between meth-induced reversal deficits and cue-induced reinstatement to meth seeking indicates that these two domains may constitute independent pathologies of meth addiction.

Hypothesizing Music Intervention Enhances Brain Functional Connectivity Involving Dopaminergic Recruitment: Common Neuro-correlates to Abusable Drugs

The goal of this review is to explore the clinical significance of music listening on neuroplasticity and dopaminergic activation by understanding the role of music therapy in addictive behavior treatment. fMRI data has shown that music listening intensely modifies mesolimbic structural changes responsible for reward processing (e.g., nucleus accumbens [NAc]) and may control the emotional stimuli's effect on autonomic and physiological responses (e.g., hypothalamus). Music listening has been proven to induce the endorphinergic response blocked by naloxone, a common opioid antagonist. NAc opioid transmission is linked to the ventral tegmental area (VTA) dopamine release. There are remarkable commonalities between listening to music and the effect of drugs on mesolimbic dopaminergic activation. It has been found that musical training before the age of 7 results in changes in white-matter connectivity, protecting carriers with low dopaminergic function (DRD2A1 allele, etc.) from poor decision-making, reward dependence, and impulsivity. In this article, we briefly review a few studies on the neurochemical effects of music and propose that these findings are relevant to the positive clinical findings observed in the literature. We hypothesize that music intervention enhances brain white matter plasticity through dopaminergic recruitment and that more research is needed to explore the efficacy of these therapies.

Compulsivity in obsessive-compulsive disorder and addictions

Compulsive behaviors are driven by repetitive urges and typically involve the experience of limited voluntary control over these urges, a diminished ability to delay or inhibit these behaviors, and a tendency to perform repetitive acts in a habitual or stereotyped manner. Compulsivity is not only a central characteristic of obsessive-compulsive disorder (OCD) but is also crucial to addiction. Based on this analogy, OCD has been proposed to be part of the concept of behavioral addiction along with other non-drug-related disorders that share compulsivity, such as pathological gambling, skin-picking, trichotillomania and compulsive eating. In this review, we investigate the neurobiological overlap between compulsivity in substance-use disorders, OCD and behavioral addictions as a validation for the construct of compulsivity that could be adopted in the Research Domain Criteria (RDoC). The reviewed data suggest that compulsivity in OCD and addictions is related to impaired reward and punishment processing with attenuated dopamine release in the ventral striatum, negative reinforcement in limbic systems, cognitive and behavioral inflexibility with diminished serotonergic prefrontal control, and habitual responding with imbalances between ventral and dorsal frontostriatal recruitment. Frontostriatal abnormalities of compulsivity are promising targets for neuromodulation and other interventions for OCD and addictions. We conclude that compulsivity encompasses many of the RDoC constructs in a trans-diagnostic fashion with a common brain circuit dysfunction that can help identifying appropriate prevention and treatment targets.

Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats

Corticostriatal circuitry supports flexible reward learning and emotional behavior from the critical neurodevelopmental stage of adolescence through adulthood. It is still poorly understood how prescription drug exposure in adolescence may impact these outcomes in the long-term. We studied adolescent methylphenidate (MPH) and fluoxetine (FLX) exposure in rats and their impact on learning and emotion in adulthood. In Experiment 1, male and female rats were administered MPH, FLX, or saline (SAL), and compared with methamphetamine (mAMPH) treatment beginning in postnatal day (PND) 37. The rats were then tested on discrimination and reversal learning in adulthood. In Experiment 2, animals were administered MPH or SAL also beginning in PND 37 and later tested in adulthood for anxiety levels. In Experiment 3, we analyzed striatal dopamine D1 and D2 receptor expression in adulthood following either extensive learning (after Experiment 1) or more brief emotional measures (after Experiment 2). We found sex differences in discrimination learning and attenuated reversal learning after MPH and only sex differences in adulthood anxiety. In learners, there was enhanced striatal D1, but not D2, after either adolescent MPH or mAMPH. Lastly, also in learners, there was a sex x treatment group interaction for D2, but not D1, driven by the MPH-pretreated females, who expressed significantly higher D2 levels compared to SAL. These results show enduring effects of adolescent MPH on reversal learning in rats. Developmental psychostimulant exposure may interact with learning to enhance D1 expression in adulthood, and affect D2 expression in a sex-dependent manner.