Reinforcement learning across the rat estrous cycle

Fecal microbiota transplantation of patients with anorexia nervosa did not alter flexible behavior in rats

OBJECTIVE

Patients with anorexia nervosa (AN) are often anxious, display inflexible behavior and disrupted reward processing. Emerging evidence suggests that gut dysbiosis in patients contributes to the disease phenotype and progression.

METHODS

In a preclinical study, we explored whether AN-derived microbiota impacts cognitive flexibility, anxiety, and dopamine signaling using fecal microbiota transplantation (FMT) in tyrosine hydroxylase-cre rats. We performed probabilistic reversal learning task (PRLT) at the baseline, after antibiotic treatment, and following FMT from patients with AN and controls. We assessed flexible behavior, task engagement, and ventral tegmental area (VTA) dopamine signaling during and in the absence of reward. Furthermore, anxiety-like behavior was evaluated with open field (OF) and elevated plus maze (EPM) tests.

RESULTS

Neither antibiotic-induced dysbiosis nor AN FMT led to significant alterations in the number of reversals or lever press strategies after reinforced or nonreinforced lever presses (win and lose-stay) in the PRLT. However, the number of initiated trials decreased after antibiotic treatment while remaining unchanged after FMT. No significant differences were observed in VTA dopamine activity, anxiety measures in the OF and EPM tests. Microbiome analysis revealed limited overlap between the microbiota of the donors and recipients.

DISCUSSION

No evidence was found that the microbiota of patients compared to controls, nor a depleted microbiome impacts cognitive flexibility. Nonetheless, antibiotic-induced dysbiosis resulted in reduced task engagement during the PRLT. The relatively low efficiency of the FMT is a limitation of our study and highlights the need for improved protocols to draw robust conclusions in future studies.

PUBLIC SIGNIFICANCE

While our study did not reveal direct impacts of AN-associated gut microbiota on cognitive flexibility or anxiety behaviors in our preclinical model, we observed a decrease in task engagement after antibiotic-induced dysbiosis, underscoring that the presence of a gut microbiome matters. Our findings underscore the need for further refinement in FMT protocols to better elucidate the complex interplay between gut microbiota and behaviors characteristic of anorexia nervosa.

Psilocybin restrains activity-based anorexia in female rats by enhancing cognitive flexibility: contributions from 5-HT1A and 5-HT2A receptor mechanisms

Psilocybin has shown promise for alleviating symptoms of depression and is currently in clinical trials for the treatment of anorexia nervosa (AN), a condition that is characterised by persistent cognitive inflexibility. Considering that enhanced cognitive flexibility after psilocybin treatment is reported to occur in individuals with depression, it is plausible that psilocybin could improve symptoms of AN by breaking down cognitive inflexibility. A mechanistic understanding of the actions of psilocybin is required to tailor the clinical application of psilocybin to individuals most likely to respond with positive outcomes. This can only be achieved using incisive neurobiological approaches in animal models. Here, we use the activity-based anorexia (ABA) rat model and comprehensively assess aspects of reinforcement learning to show that psilocybin (post-acutely) improves body weight maintenance in female rats and facilitates cognitive flexibility, specifically via improved adaptation to the initial reversal of reward contingencies. Further, we reveal the involvement of signalling through the serotonin (5-HT) 1 A and 5-HT2A receptor subtypes in specific aspects of learning, demonstrating that 5-HT1A antagonism negates the cognitive enhancing effects of psilocybin. Moreover, we show that psilocybin elicits a transient increase and decrease in cortical transcription of these receptors (Htr2a and Htr1a, respectively), and a further reduction in the abundance of Htr2a transcripts in rats exposed to the ABA model. Together, these findings support the hypothesis that psilocybin could ameliorate cognitive inflexibility in the context of AN and highlight a need to better understand the therapeutic mechanisms independent of 5-HT2A receptor binding.

State and rate-of-change encoding in parallel mesoaccumbal dopamine pathways

The nervous system uses fast- and slow-adapting sensory detectors in parallel to enable neuronal representations of external states and their temporal dynamics. It is unknown whether this dichotomy also applies to internal representations that have no direct correlation in the physical world. Here we find that two distinct dopamine (DA) neuron subtypes encode either a state or its rate-of-change. In mice performing a reward-seeking task, we found that the animal's behavioral state and rate-of-change were encoded by the sustained activity of DA neurons in medial ventral tegmental area (VTA) DA neurons and transient activity in lateral VTA DA neurons, respectively. The neural activity patterns of VTA DA cell bodies matched DA release patterns within anatomically defined mesoaccumbal pathways. Based on these results, we propose a model in which the DA system uses two parallel lines for proportional-differential encoding of a state variable and its temporal dynamics.

Female rats prefer to forage food from males, an effect that is not influenced by stress

As social beings, animals and humans alike make real life decisions that are often influenced by other members. Most current research has focused on the influence of same-sex peers on individual decision-making, with potential opposite sex effect scarcely explored. Here, we developed a behavioral model to observe food foraging decision-making in female rats under various social situations. We found that female rats preferred to forage food from male over female rats or from the no-rat storage side. Female rats were more likely to forage food from familiar males than from unfamiliar. This opposite-sex preference was not altered by the lure of sweet food, or with estrous cycle, nor under stress conditions. These results suggest that the opposite sex influences food foraging decision-making in female rats. The behavioral model established could facilitate future investigation into the underlying neurobiological mechanisms.

Postnatal ethanol exposure impairs social behavior and operant extinction in the adult female mouse offspring

Fetal Alcohol Spectrum Disorder (FASD) comprises a group of neurodevelopmental deficits caused by alcohol exposure during pregnancy. Clinical studies suggest that while the male progeny experiences serious neurodevelopmental defects, female patients have more severe cognitive, social, and affective symptoms. Other than sex, dose, frequency, and timing of exposure determine the neurobehavioral outcomes in young and adult progeny. In this regard, human studies indicate that some individuals relapse during late-term gestational periods. In mice, this interval corresponds to the first 10 days after birth (postnatal, P0-P10). In our model of postnatal ethanol exposure (PEE^P0-P10), we tested whether adult female and male offspring show deficits in sociability, anxiety-like, reward consumption, and action-outcome associations. We report that female PEE^P0-P10 offspring have mild social impairments and altered extinction of operant responding in the absence of anxiety-like traits and reward consumption defects. None of these deficits were detected in the male PEE^P0-P10 offspring. Our data provide novel information on sex-specific neurobehavioral outcomes of postnatal ethanol exposure in female adult offspring.

Inhibition of ventral tegmental area projections to the nucleus accumbens shell increases premature responding in the five-choice serial reaction time task in rats

Exaggerated impulsivity and attentional impairments are hallmarks of certain disorders of behavioural control such as attention-deficit/hyperactivity disorder (ADHD), schizophrenia and addiction. Pharmacological studies have implicated elevated dopamine (DA) levels in the nucleus accumbens shell (NAcbS) in impulsive actions. The NAcbS receives its DA input from the ventral tegmental area (VTA), and we have previously shown that optogenetic activation of VTA-NAcbS projections impaired impulse control and attention in the five-choice serial reaction time task (5-CSRTT) in rats. To better understand the role of VTA-NAcbS projections in impulsivity and attention, the present study sought to inhibit this projection using optogenetics. We demonstrate that inhibiting VTA-NAcbS efferents during the last seconds of the inter-trial interval (i.e. immediately before presentation of the instructive cue) induces exaggerated impulsive action, in the absence of changes in attentional or motivational parameters in the 5-CSRTT. Together with our earlier observations, this suggests that impulse control in the 5-CSRTT is tightly controlled by VTA-NAcbS activity, with deviations in both directions resulting in increased impulsivity.

A neural substrate of sex-dependent modulation of motivation

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Cognitive performance during adulthood in a rat model of neonatal diffuse white matter injury

RATIONALE

Infants born prematurely risk developing diffuse white matter injury (WMI), which is associated with impaired cognitive functioning and an increased risk of autism spectrum disorder. Recently, our rat model of preterm diffuse WMI induced by combined fetal inflammation and postnatal hypoxia showed impaired motor performance, anxiety-like behaviour and autism-like behaviour in juvenile rats, especially males. Immunohistochemistry showed delayed myelination in the sensory cortex and impaired oligodendrocyte differentiation.

OBJECTIVE

To assess long-term cognitive deficits in this double-hit rat model of diffuse WMI, animals were screened on impulsivity, attention and cognitive flexibility in adulthood using the 5-choice serial reaction time task (5CSRTT) and a probabilistic reversal learning task, tests that require a proper functioning prefrontal cortex. Thereafter, myelination deficits were evaluated by immunofluorescent staining in adulthood.

RESULTS

Overall, little effect of WMI or sex was found in the cognitive tasks. WMI animals showed subtle differences in performance in the 5CSRTT. Manipulating 5CSRTT parameters resulted in performance patterns previously seen in the literature. Sex differences were found in perseverative responses and omitted trials: female WMI rats seem to be less flexible in the 5CSRTT but not in the reversal learning task. Males collected rewards faster in the probabilistic reversal learning task. These findings are explained by temporally rather than permanently affected myelination and by the absence of extensive injury to prefrontal cortical subregions, confirmed by immunofluorescent staining in both adolescence and adulthood.

CONCLUSION

This rat model of preterm WMI does not lead to long-term cognitive deficits as observed in prematurely born human infants.

Sex Differences in Substance Use Disorders: A Neurobiological Perspective

Clinical studies provide fundamental knowledge of substance use behaviors (substance of abuse, patterns of use, relapse rates). The combination of neuroimaging approaches reveal correlation between substance use disorder (SUD) and changes in neural structure, function, and neurotransmission. Here, we review these advances, placing special emphasis on sex specific findings from structural neuroimaging studies of those dependent on alcohol, nicotine, cannabis, psychostimulants, or opioids. Recent clinical studies in SUD analyzing sex differences reveal neurobiological changes that are differentially impacted in common reward processing regions such as the striatum, hippocampus, amygdala, insula, and corpus collosum. We reflect on the contribution of sex hormones, period of drug use and abstinence, and the potential impact of these factors on the interpretation of the reported findings. With the overall recognition that SUD impacts the brains of females and males differentially, it is of fundamental importance that future research is designed with sex as a variable of study in this field. Improved understanding of neurobiological changes in males and females in SUD will advance knowledge underlying sex-specific susceptibility and the neurobiological impact in these disorders. Together these findings will inform future treatments that are tailor designed for improved efficacy in females and males with SUD.

Prolonged Social Isolation, Started Early in Life, Impairs Cognitive Abilities in Rats Depending on Sex

Background: The chronic stress of social isolation is a valid predictor of cognitive pathology. This study aimed to compare the effects of long-term social isolation on female versus male Wistar rats’ learning and memory. We hypothesized that prolonged social isolation stress, which starts early in life, would affect learning in a sex-dependent manner. Methods: Social isolation started at the edge of early to mid-adolescence and lasted 9 months. The rat’s cognitive abilities were assessed by habituation and reactivity to novelty in the open field (OF) test, spatial memory in the Morris water maze (MWM), and the conditioned passive avoidance (PA) reflex. Basal serum corticosterone levels were assessed using an enzyme-linked immunosorbent assay. Results: Regardless of the housing conditions, females habituated to the OF under low illumination slower than males. Under bright light, the single-housed rats showed hyporeactivity to novelty. In the MWM, all the rats learned to locate the platform; however, on the first training day, the single-housed females’ speed was lower relative to other groups. Four months later, in the post-reminder probe trial, the single-housed rats reached the area around the platform site later, and only males, regardless of housing conditions, preferred the target quadrant. Single-housed rats, irrespective of sex, showed a PA deficit. There was a more pronounced conditioned fear in the single-housed males than in females. In both male and female rats, basal corticosterone levels in rat blood serum after 9 months of social isolation did not differ from that in the group-housed rats of the corresponding sex. Meanwhile, females’ basal corticosterone level was higher than in males, regardless of the housing conditions. The relative weight of the adrenal glands was increased only in single-housed females. Conclusions: Under long-term social isolation, started early in life, single-housed females compared with males showed more pronounced cognitive impairments in the MWM and PA paradigm, findings that specify their greater vulnerability to the stress of prolonged social isolation.

Pro-social preference in an automated operant two-choice reward task under different housing conditions: Exploratory studies on pro-social decision making

In this study, we aimed to develop a behavioral task that measures pro-social decision making in rats. A fully automated, operant pro-social two-choice task is introduced that quantifies pro-social preferences for a mutual food reward in a set-up with tightly controlled task contingencies. Pairs of same-sex adult Wistar rats were placed in an operant chamber divided into two compartments (one rat per compartment), separated by a transparent barrier with holes that allowed the rats to see, hear, smell, but not touch each other. Test rats could earn a sucrose pellet either for themselves (own reward) or for themselves and the partner (both reward) by means of lever pressing. On average, male rats showed a 60 % preference for the lever that yielded a food reward for both themselves and their partner. In contrast, females did not show lever preference, regardless of the estrous cycle phase. Next, the impact of juvenile environmental factors on male rat social decision making was studied. Males were group-housed from postnatal day 26 onwards in complex housing Marlau™ cages that provided social and physical enrichment and stimulation in the form of novelty. Complex housed males did not show a preference for the pro-social lever.

Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex

RATIONALE

During value-based decision-making, organisms make choices on the basis of reward expectations, which have been formed during prior action-outcome learning. Although it is known that neuronal manipulations of different subregions of the rat prefrontal cortex (PFC) have qualitatively different effects on behavioral tasks involving value-based decision-making, it is unclear how these regions contribute to the underlying component processes.

OBJECTIVES

Assessing how different regions of the rodent PFC contribute to component processes of value-based decision-making behavior, including reward (or positive feedback) learning, punishment (or negative feedback) learning, response persistence, and exploration versus exploitation.

METHODS

We performed behavioral modeling of data of rats in a probabilistic reversal learning task after pharmacological inactivation of five PFC subregions, to assess how inactivation of these different regions affected the structure of responding of animals in the task.

RESULTS

Our results show reductions in reward and punishment learning after PFC subregion inactivation. The prelimbic, infralimbic, lateral orbital, and medial orbital PFC particularly contributed to punishment learning, and the prelimbic and lateral orbital PFC to reward learning. In addition, response persistence depended on the infralimbic and medial orbital PFC. As a result, pharmacological inactivation of the infralimbic and lateral orbitofrontal cortex reduced the number of reversals achieved, whereas inactivation of the prelimbic and medial orbitofrontal cortex decreased the number of rewards obtained. Finally, using simulated data, we explain discrepancies with a previous study and demonstrate complex, interacting relationships between conventional measures of probabilistic reversal learning performance, such as win-stay/lose-switch behavior, and component processes of value-based decision-making.

CONCLUSIONS

Together, our data suggest that distinct components of value-based learning and decision-making are generated in medial and orbital PFC regions, displaying functional specialization and overlap, with a prominent role of large parts of the PFC in negative feedback processing.

Differential contributions of striatal dopamine D1 and D2 receptors to component processes of value-based decision making

Dopamine has been implicated in value-based learning and decision making by signaling reward prediction errors and facilitating cognitive flexibility, incentive motivation, and voluntary movement. Dopamine receptors can roughly be divided into the D1 and D2 subtypes, and it has been hypothesized that these two types of receptors have an opposite function in facilitating reward-related and aversion-related behaviors, respectively. Here, we tested the contribution of striatal dopamine D1 and D2 receptors to processes underlying value-based learning and decision making in rats, employing a probabilistic reversal learning paradigm. Using computational trial-by-trial analysis of task behavior after systemic or intracranial treatment with dopamine D1 and D2 receptor agonists and antagonists, we show that negative feedback learning can be modulated through D2 receptor signaling and positive feedback learning through D1 receptor signaling in the ventral striatum. Furthermore, stimulation of D2 receptors in the ventral or dorsolateral (but not dorsomedial) striatum promoted explorative choice behavior, suggesting an additional function of dopamine in these areas in value-based decision making. Finally, treatment with most dopaminergic drugs affected response latencies and number of trials completed, which was also seen after infusion of D2, but not D1 receptor-acting drugs into the striatum. Together, our data support the idea that dopamine D1 and D2 receptors have complementary functions in learning on the basis of emotionally valenced feedback, and provide evidence that dopamine facilitates value-based and motivated behaviors through distinct striatal regions.