Characterisation of behaviours relevant to apathy syndrome in the aged male rat

Apathy is a complex psychiatric syndrome characterised by motivational deficit, emotional blunting and cognitive changes. It occurs alongside a broad range of neurological disorders, but also occurs in otherwise healthy ageing. Despite its clinical prevalence, apathy does not yet have a designated treatment strategy. Generation of a translational animal model of apathy syndrome would facilitate the development of novel treatments. Given the multidimensional nature of apathy, a model cannot be achieved with a single behavioural test. Using a battery of behavioural tests we investigated whether aged rats exhibit behavioural deficits across different domains relevant to apathy. Using the effort for reward and progressive ratio tasks we found that aged male rats (21-27 months) show intact reward motivation. Using the novelty supressed feeding test and position-based object exploration we found aged rats showed increased anxiety-like behaviour inconsistent with emotional blunting. The sucrose preference test and reward learning assay showed intact reward sensitivity and reward-related cognition in aged rats. However, using a bowl-digging version of the probabilistic reversal learning task, we found a deficit in cognitive flexibility in aged rats that did not translate across to a touchscreen version of the task. While these data reveal important changes in cognitive flexibility and anxiety associated with ageing, aged rats do not show deficits across other behavioural domains relevant to apathy. This suggests that aged rats are not a suitable model for age-related apathy syndrome. These findings contrast with previous work in mice, revealing important species differences in behaviours relevant to apathy syndrome in ageing.

Reach-to-Grasp and tactile discrimination task: A new task for the study of sensory-motor learning

Although active touch in rodents arises from the forepaws as well as whiskers, most research on active touch only focuses on whiskers. This results in a paucity of tasks designed to assess the process of active touch with a forepaw. We develop a new experimental task, the Reach-to-Grasp and Tactile Discrimination task (RGTD task), to examine active touch with a forepaw in rodents, particularly changes in processes of active touch during motor skill learning. In the RGTD task, animals are required to (1) extend their forelimb to an object, (2) grasp the object, and (3) manipulate the grasped object with the forelimb. The animals must determine the direction of the manipulation based on active touch sensations arising during the period of the grasping. In experiment 1 of the present study, we showed that rats can learn the RGTD task. In experiment 2, we confirmed that the rats are capable of reversal learning of the RGTD task. The RGTD task shared most of the reaching movements involved with conventional forelimb reaching tasks. From the standpoint of a discrimination task, the RGTD task enables rigorous experimental control, for example by removing bias in the stimulus-response correspondence, and makes it possible to utilize diverse experimental procedures that have been difficult in prior tasks.

Conditional deletion of neurexin-2 impaired behavioral flexibility to alterations in action-outcome contingency

Neurexins (Nrxns) are critical for synapse organization and their mutations have been documented in autism spectrum disorder, schizophrenia, and epilepsy. We recently reported that conditional deletion of Nrxn2, under the control of Emx1Cre promoter, predominately expressed in the neocortex and hippocampus (Emx1-Nrxn2 cKO mice) induced stereotyped patterns of behavior in mice, suggesting behavioral inflexibility. In this study, we investigated the effects of Nrxn2 deletion through two different conditional approaches targeting presynaptic cortical neurons projecting to dorsomedial striatum on the flexibility between goal-directed and habitual actions in response to devaluation of action-outcome (A-O) contingencies in an instrumental learning paradigm or upon reversal of A-O contingencies in a water T-maze paradigm. Nrxn2 deletion through both the conditional approaches induced an inability of mice to discriminate between goal-directed and habitual action strategies in their response to devaluation of A-O contingency. Emx1-Nrxn2 cKO mice exhibited reversal learning deficits, indicating their inability to adopt new action strategies. Overall, our studies showed that Nrxn2 deletion through two distinct conditional deletion approaches impaired flexibility in response to alterations in A-O contingencies. These investigations can lay the foundation for identification of novel genetic factors underlying behavioral inflexibility.

Executive functions and brain morphology of male and female dominant and subordinate cichlid fish

BACKGROUND

Living in a social dominance hierarchy presents different benefits and challenges for dominant and subordinate males and females, which might in turn affect their cognitive needs. Despite the extensive research on social dominance in group-living species, there is still a knowledge gap regarding how social status impacts brain morphology and cognitive abilities.

METHODS

Here, we tested male and female dominants and subordinates of Neolamprologus pulcher, a social cichlid fish species with size-based hierarchy. We ran three executive cognitive function tests for cognitive flexibility (reversal learning test), self-control (detour test), and working memory (object permanence test), followed by brain and brain region size measurements.

RESULTS

Performance was not influenced by social status or sex. However, dominants exhibited a brain-body slope that was relatively steeper than that of subordinates. Furthermore, individual performance in reversal learning and detour tests correlated with brain morphology, with some trade-offs among major brain regions like telencephalon, cerebellum, and optic tectum.

CONCLUSION

As individuals' brain growth strategies varied depending on social status without affecting executive functions, the different associated challenges might yield a potential effect on social cognition instead. Overall, the findings highlight the importance of studying the individual and not just species to understand better how the individual's ecology might shape its brain and cognition.

Ghrelin decreases sensitivity to negative feedback and increases prediction-error related caudate activity in humans, a randomized controlled trial

The stomach-derived hormone ghrelin plays not only a role in feeding, starvation, and survival, but it has been suggested to also be involved in the stress response, in neuropsychiatric conditions, and in alcohol and drug use disorders. Mechanisms related to reward processing might mediate ghrelin's broader effects on complex behaviors, as indicated by animal studies and mostly correlative human studies. Here, using a within-subject double-blind placebo-controlled design with intravenous ghrelin infusion in healthy volunteers (n = 30), we tested whether ghrelin alters sensitivity to reward and punishment in a reward learning task. Parameters were derived from a computational model of participants' task behavior. The reversal learning task with monetary rewards was performed during functional brain imaging to investigate ghrelin effects on brain signals related to reward prediction errors. Compared to placebo, ghrelin decreased punishment sensitivity (t = -2.448, p = 0.021), while reward sensitivity was unaltered (t = 0.8, p = 0.43). We furthermore found increased prediction-error related activity in the dorsal striatum during ghrelin administration (region of interest analysis: t-values ≥ 4.21, p-values ≤ 0.044). Our results support a role for ghrelin in reward processing that extends beyond food-related rewards. Reduced sensitivity to negative outcomes and increased processing of prediction errors may be beneficial for food foraging when hungry but could also relate to increased risk taking and impulsivity in the broader context of addictive behaviors.

Optogenetic stimulation of corticostriatal circuits improves behavioral flexibility in mice with prenatal alcohol exposure

Fetal alcohol spectrum disorder (FASD) is the most common preventable form of developmental and neurobehavioral disability. Animal models have demonstrated that even low to moderate prenatal alcohol exposure (PAE) is sufficient to impair behavioral flexibility in multiple domains. Previously, utilizing a moderate limited access drinking in the dark paradigm, we have shown that PAE 1) impairs touchscreen pairwise visual reversal in male adult offspring 2) leads to small but significant decreases in orbitofrontal (OFC) firing rates 3) significantly increases dorsal striatum (dS) activity and 4) aberrantly sustains OFC-dS synchrony across early reversal. In the current study, we examined whether optogenetic stimulation of OFC-dS projection neurons would be sufficient to rescue the behavioral inflexibility induced by PAE in male C57BL/6J mice. Following discrimination learning, we targeted OFC-dS projections using a retrograde adeno-associated virus (AAV) delivered to the dS which expressed channel rhodopsin (ChR2). During the first four sessions of reversal learning, we delivered high frequency optogenetic stimulation to the OFC via optic fibers immediately following correct choice responses. Our results show that optogenetic stimulation significantly reduced the number of sessions, incorrect responses, and correction errors required to move past the early perseverative phase for both PAE and control mice. In addition, OFC-dS stimulation during early reversal learning reduced the increased sessions, correct and incorrect responding seen in PAE mice during the later learning phase of reversal but did not significantly alter later performance in control ChR2 mice. Taken together these results suggest that stimulation of OFC-dS projections can improve early reversal learning in PAE and control mice, and these improvements can persist even into later stages of the task days later. These studies provide an important foundation for future clinical approaches to improve executive control in those with FASD. This article is part of the Special Issue on "PFC circuit function in psychiatric disease and relevant models".

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Dynamic regulation of corticostriatal glutamatergic synaptic expression during reversal learning in male mice

Behavioral flexibility, one of the core executive functions of the brain, has been shown to be an essential skill for survival across species. Corticostriatal circuits play a critical role in mediating behavioral flexibility. The molecular mechanisms underlying these processes are still unclear. Here, we measured how synaptic glutamatergic α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR) and N-methyl-D-aspartic acid receptor (NMDAR) expression dynamically changed during specific stages of learning and reversal. Following training to well-established stages of discrimination and reversal learning on a touchscreen visual task, lateral orbitofrontal cortex (OFC), dorsal striatum (dS) as well as medial prefrontal cortex (mPFC), basolateral amygdala (BLA) and piriform cortex (Pir) were micro dissected from male mouse brain and the expression of glutamatergic receptor subunits in the synaptic fraction were measured via immunoblotting. We found that the GluN2B subunit of NMDAR in the OFC remained stable during initial discrimination learning but significantly increased in the synaptic fraction during mid-reversal stages, the period during which the OFC has been shown to play a critical role in updating outcome expectancies. In contrast, both GluA1 and GluA2 subunits of the AMPAR significantly increased in the dS synaptic fraction as new associations were learned late in reversal. Expression of NMDAR and AMPAR subunits did not significantly differ across learning stages in any other brain region. Together, these findings further support the involvement of OFC-dS circuits in moderating well-learned associations and flexible behavior and suggest that dynamic synaptic expression of NMDAR and AMPAR in these circuits may play a role in mediating efficient learning during discrimination and the ability to update previously learned associations as environmental contingencies change.

Comparable roles for serotonin in rats and humans for computations underlying flexible decision-making

Serotonin is critical for adapting behavior flexibly to meet changing environmental demands. Cognitive flexibility is important for successful attainment of goals, as well as for social interactions, and is frequently impaired in neuropsychiatric disorders, including obsessive-compulsive disorder. However, a unifying mechanistic framework accounting for the role of serotonin in behavioral flexibility has remained elusive. Here, we demonstrate common effects of manipulating serotonin function across two species (rats and humans) on latent processes supporting choice behavior during probabilistic reversal learning, using computational modelling. The findings support a role of serotonin in behavioral flexibility and plasticity, indicated, respectively, by increases or decreases in choice repetition ('stickiness') or reinforcement learning rates following manipulations intended to increase or decrease serotonin function. More specifically, the rate at which expected value increased following reward and decreased following punishment (reward and punishment 'learning rates') was greatest after sub-chronic administration of the selective serotonin reuptake inhibitor (SSRI) citalopram (5 mg/kg for 7 days followed by 10 mg/kg twice a day for 5 days) in rats. Conversely, humans given a single dose of an SSRI (20 mg escitalopram), which can decrease post-synaptic serotonin signalling, and rats that received the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT), which destroys forebrain serotonergic neurons, exhibited decreased reward learning rates. A basic perseverative tendency ('stickiness'), or choice repetition irrespective of the outcome produced, was likewise increased in rats after the 12-day SSRI regimen and decreased after single dose SSRI in humans and 5,7-DHT in rats. These common effects of serotonergic manipulations on rats and humans-identified via computational modelling-suggest an evolutionarily conserved role for serotonin in plasticity and behavioral flexibility and have clinical relevance transdiagnostically for neuropsychiatric disorders.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Dopamine release in human associative striatum during reversal learning

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Meta-reinforcement learning via orbitofrontal cortex

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.

Reversal learning in those with early psychosis features contingency-dependent changes in loss response and learning

INTRODUCTION

People with psychotic disorders commonly feature broad decision-making impairments that impact their functional outcomes. Specific associative/reinforcement learning problems have been demonstrated in persistent psychosis. But these phenotypes may differ in early psychosis, suggesting that aspects of cognition decline over time.

METHODS

The present proof-of-concept study examined goal-directed action and reversal learning in controls and those with early psychosis.

RESULTS

Equivalent performance was observed between groups during outcome-specific devaluation, and reversal learning at an 80:20 contingency (reward probability for high:low targets). But when the low target reward probability was increased (80:40) those with early psychosis altered their response to loss, whereas controls did not. Computational modelling confirmed that in early psychosis there was a change in punishment learning that increased the chance of staying with the same stimulus after a loss, multiple trials into the future. In early psychosis, the magnitude of this response was greatest in those with higher IQ and lower clinical severity scores.

CONCLUSIONS

We show preliminary evidence that those with early psychosis present with a phenotype that includes altered responding to loss and hyper-adaptability in response to outcome changes. This may reflect a compensatory response to overcome the milieu of corticostriatal changes associated with psychotic disorders.

How uncertain are you? Disentangling expected and unexpected uncertainty in pupil-linked brain arousal during reversal learning

During decision making, we are continuously faced with two sources of uncertainty regarding the links between stimuli, our actions, and outcomes. On the one hand, our expectations are often probabilistic, that is, stimuli or actions yield the expected outcome only with a certain probability (expected uncertainty). On the other hand, expectations might become invalid due to sudden, unexpected changes in the environment (unexpected uncertainty). Several lines of research show that pupil-linked brain arousal is a sensitive indirect measure of brain mechanisms underlying uncertainty computations. Thus, we investigated whether it is involved in disentangling these two forms of uncertainty. To this aim, we measured pupil size during a probabilistic reversal learning task. In this task, participants had to figure out which of two response options led to reward with higher probability, whereby sometimes the identity of the more advantageous response option was switched. Expected uncertainty was manipulated by varying the reward probability of the advantageous choice option, whereas the level of unexpected uncertainty was assessed by using a Bayesian computational model estimating change probability and resulting uncertainty. We found that both aspects of unexpected uncertainty influenced pupil responses, confirming that pupil-linked brain arousal is involved in model updating after unexpected changes in the environment. Furthermore, high level of expected uncertainty impeded the detection of sudden changes in the environment, both on physiological and behavioral level. These results emphasize the role of pupil-linked brain arousal and underlying neural structures in handling situations in which the previously established contingencies are no longer valid.

Modulation of ventromedial orbitofrontal cortical glutamatergic activity affects the explore-exploit balance and influences value-based decision-making

The balance between exploration and exploitation is essential for decision-making. The present study investigated the role of ventromedial orbitofrontal cortex (vmOFC) glutamate neurons in mediating value-based decision-making by first using optogenetics to manipulate vmOFC glutamate activity in rats during a probabilistic reversal learning (PRL) task. Rats that received vmOFC activation during informative feedback completed fewer reversals and exhibited reduced reward sensitivity relative to rats. Analysis with a Q-learning computational model revealed that increased vmOFC activity did not affect the learning rate but instead promoted maladaptive exploration. By contrast, vmOFC inhibition increased the number of completed reversals and increased exploitative behavior. In a separate group of animals, calcium activity of vmOFC glutamate neurons was recorded using fiber photometry. Complementing our results above, we found that suppression of vmOFC activity during the latter part of rewarded trials was associated with improved PRL performance, greater win-stay responding and selecting the correct choice on the next trial. These data demonstrate that excessive vmOFC activity during reward feedback disrupted value-based decision-making by increasing the maladaptive exploration of lower-valued options. Our findings support the premise that pharmacological interventions that normalize aberrant vmOFC glutamate activity during reward feedback processing may attenuate deficits in value-based decision-making.

A bidirectional competitive interaction between circHomer1 and Homer1b within the orbitofrontal cortex regulates reversal learning

Although circular RNAs (circRNAs) are enriched in the brain, their relevance for brain function and psychiatric disorders is poorly understood. Here, we show that circHomer1 is inversely associated with relative HOMER1B mRNA isoform levels in both the orbitofrontal cortex (OFC) and stem-cell-derived neuronal cultures of subjects with psychiatric disorders. We further demonstrate that in vivo circHomer1 knockdown (KD) within the OFC can inhibit the synaptic expression of Homer1b mRNA. Furthermore, we show that circHomer1 directly binds to Homer1b mRNA and that Homer1b-specific KD increases synaptic circHomer1 levels and improves OFC-mediated behavioral flexibility. Importantly, double circHomer1 and Homer1b in vivo co-KD results in a complete rescue in circHomer1-associated alterations in both chance reversal learning and synaptic gene expression. Lastly, we uncover an RNA-binding protein that can directly bind to circHomer1 and promote its biogenesis. Taken together, our data provide mechanistic insights into the importance of circRNAs in brain function and disease.

The selective 5-HT2A receptor agonist 25CN-NBOH does not affect reversal learning in mice

Psychedelic 5-hydroxytryptamine 2A receptor (5-HT2AR) agonists are showing promise in the treatment of psychiatric disorders, such as treatment-resistant depression and anxiety. Human studies suggest that enhanced cognitive flexibility may contribute to their clinical efficacy. Both improvement and impairment of cognitive flexibility has been reported with 5-HT2AR ligands, making the link between 5-HT2AR pharmacology and cognitive flexibility equivocal. We tested the selective 5-HT2AR agonist 25CN-NBOH in healthy male C57BL/6JOlaHsd mice in a touchscreen-based mouse reversal learning test. No effects were observed on acquisition of the new stimulus-reward contingency, learning errors, or perseverative responses during reversal. Our results suggest that 25CN-NBOH does not affect reversal learning in the schedule used in this study.

Relocation of an Extrasynaptic GABAA Receptor to Inhibitory Synapses Freezes Excitatory Synaptic Strength and Preserves Memory

The excitatory synapse between hippocampal CA3 and CA1 pyramidal neurons exhibits long-term potentiation (LTP), a positive feedback process implicated in learning and memory in which postsynaptic depolarization strengthens synapses, promoting further depolarization. Without mechanisms for interrupting positive feedback, excitatory synapses could strengthen inexorably, corrupting memory storage. Here, we reveal a hidden form of inhibitory synaptic plasticity that prevents accumulation of excitatory LTP. We developed a knockin mouse that allows optical control of endogenous α5-subunit-containing γ-aminobutyric acid (GABA)A receptors (α5-GABARs). Induction of excitatory LTP relocates α5-GABARs, which are ordinarily extrasynaptic, to inhibitory synapses, quashing further NMDA receptor activation necessary for inducing more excitatory LTP. Blockade of α5-GABARs accelerates reversal learning, a behavioral test for cognitive flexibility dependent on repeated LTP. Hence, inhibitory synaptic plasticity occurs in parallel with excitatory synaptic plasticity, with the ensuing interruption of the positive feedback cycle of LTP serving as a possible critical early step in preserving memory.

The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection

Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions.

Reinforcement Learning during Adolescence in Rats

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Sex-dependent effects of chronic intermittent voluntary alcohol consumption on attentional, not motivational, measures during probabilistic learning and reversal

Background

Forced alcohol (ethanol, EtOH) exposure has been shown to cause significant impairments on reversal learning, a widely-used assay of cognitive flexibility, specifically on fully-predictive, deterministic versions of this task. However, previous studies have not adequately considered voluntary EtOH consumption and sex effects on probabilistic reversal learning. The present study aimed to fill this gap in the literature.

Methods

Male and female Long-Evans rats underwent either 10 weeks of voluntary intermittent 20% EtOH access or water only (H2O) access. Rats were then pretrained to initiate trials and learn stimulus-reward associations via touchscreen response, and subsequently required to select between two visual stimuli, rewarded with probability 0.70 or 0.30. In the final phase, reinforcement contingencies were reversed.

Results

We found significant sex differences on several EtOH-drinking variables, with females reaching a higher maximum EtOH consumption, exhibiting more high-drinking days, and escalating their EtOH at a quicker rate compared to males. During early abstinence, EtOH drinkers (and particularly EtOH-drinking females) made more initiation omissions and were slower to initiate trials than H2O drinking controls, especially during pretraining. A similar pattern in trial initiations was also observed in discrimination, but not in reversal learning. EtOH drinking rats were unaffected in their reward collection and stimulus response times, indicating intact motivation and motor responding. Although there were sex differences in discrimination and reversal phases, performance improved over time. We also observed sex-independent drinking group differences in win-stay and lose-shift strategies specific to the reversal phase.

Conclusions

Females exhibit increased vulnerability to EtOH effects in early learning: there were sex-dependent EtOH effects on attentional measures during pretraining and discrimination phases. We also found sex-independent EtOH effects on exploration strategies during reversal. Future studies should aim to uncover the neural mechanisms for changes in attention and exploration in both acute and prolonged EtOH withdrawal.

Possible association between spindle frequency and reversal-learning in aged family dogs

In both humans and dogs sleep spindle occurrence between acquisition and recall of a specific memory correlate with learning performance. However, it is not known whether sleep spindle characteristics are also linked to performance beyond the span of a day, except in regard to general mental ability in humans. Such a relationship is likely, as both memory and spindle expression decline with age in both species (in dogs specifically the density and amplitude of slow spindles). We investigated if spindle amplitude, density (spindles/minute) and/or frequency (waves/second) correlate with performance on a short-term memory and a reversal-learning task in old dogs (> 7 years), when measurements of behavior and EEG were on average a month apart. Higher frequencies of fast (≥ 13 Hz) spindles on the frontal and central midline electrodes, and of slow spindles (≤ 13 Hz) on the central midline electrode were linked to worse performance on a reversal-learning task. The present findings suggest a role for spindle frequency as a biomarker of cognitive aging across species: Changes in spindle frequency are associated with dementia risk and onset in humans and declining learning performance in the dog.

Midsession reversal learning: Pigeons learn what stimulus to avoid

The midsession reversal task involves a simple simultaneous discrimination in which, each session, choice of 1 stimulus (S1) is correct for the first 40 trials of each session, and choice of the other stimulus (S2) is correct for the remaining 40 trials. After considerable training with this task, pigeons typically continue to choose S2 too early (making anticipatory errors) and continue choosing S1 for following the reversal (making perseverative errors). Errors can be reduced, however, by decreasing the probability of reinforcement for correct S2 choices or by increasing the response requirement for S2 choices. Increasing the number of S2 stimuli (over trials, 1 S2 stimulus on each trial), however, does not reduce errors. Instead, it results in an increase in anticipatory errors but no change in perseverative errors. In the present experiment, we increased the number of S1 stimuli (over trials, 1 S1 stimulus on each trial) and found an increase in the number of perseverative errors but no change in anticipatory errors. The results suggest that the pigeons acquire this task by learning which stimuli to avoid, rather than which stimuli to choose, although it is also possible that these effects result from attention drawn to the variable stimuli when they are incorrect. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning

Impaired cognitive flexibility in visual reversal-learning tasks has been observed in a wide range of neurological and neuropsychiatric disorders. Although both human and animal studies have implicated striatal D2-like and D1-like receptors (D2R; D1R) in this form of flexibility, less is known about the contribution they make within distinct sub-regions of the striatum and the different phases of visual reversal learning. The present study investigated the involvement of D2R and D1R during the early (perseverative) phase of reversal learning as well as in the intermediate and late stages (new learning) after microinfusions of D2R and D1R antagonists into the nucleus accumbens core and shell (NAcC; NAcS), the anterior and posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS) on a touchscreen visual serial reversal-learning task. Reversal learning was improved after dopamine receptor blockade in the nucleus accumbens; the D1R antagonist, SCH23390, in the NAcS and the D2R antagonist, raclopride, in the NAcC selectively reduced early, perseverative errors. In contrast, reversal learning was impaired by D2R antagonism, but not D1R antagonism, in the dorsal striatum: raclopride increased errors in the intermediate phase after DMS infusions, and increased errors across phases after DLS infusions. These findings indicate that D1R and D2R modulate different stages of reversal learning through effects localised to different sub-regions of the striatum. Thus, deficits in behavioral flexibility observed in disorders linked to dopamine perturbations may be attributable to specific D1R and D2R dysfunction in distinct striatal sub-regions.

Cortical Foxp2 Supports Behavioral Flexibility and Developmental Dopamine D1 Receptor Expression

Genetic studies have associated FOXP2 variation with speech and language disorders and other neurodevelopmental disorders (NDDs) involving pathology of the cortex. In this brain region, FoxP2 is expressed from development into adulthood, but little is known about its downstream molecular and behavioral functions. Here, we characterized cortex-specific Foxp2 conditional knockout mice and found a major deficit in reversal learning, a form of behavioral flexibility. In contrast, they showed normal activity levels, anxiety, and vocalizations, save for a slight decrease in neonatal call loudness. These behavioral phenotypes were accompanied by decreased cortical dopamine D1 receptor (D1R) expression at neonatal and adult stages, while general cortical development remained unaffected. Finally, using single-cell transcriptomics, we identified at least five excitatory and three inhibitory D1R-expressing cell types in neonatal frontal cortex, and we found changes in D1R cell type composition and gene expression upon cortical Foxp2 deletion. Strikingly, these alterations included non-cell-autonomous changes in upper layer neurons and interneurons. Together, these data support a role for Foxp2 in the development of dopamine-modulated cortical circuits and behaviors relevant to NDDs.

Multi-domain cognitive assessment of male mice shows space radiation is not harmful to high-level cognition and actually improves pattern separation

Astronauts on interplanetary missions - such as to Mars - will be exposed to space radiation, a spectrum of highly-charged, fast-moving particles that includes 56Fe and 28Si. Earth-based preclinical studies show space radiation decreases rodent performance in low- and some high-level cognitive tasks. Given astronaut use of touchscreen platforms during training and space flight and given the ability of rodent touchscreen tasks to assess functional integrity of brain circuits and multiple cognitive domains in a non-aversive way, here we exposed 6-month-old C57BL/6J male mice to whole-body space radiation and subsequently assessed them on a touchscreen battery. Relative to Sham treatment, 56Fe irradiation did not overtly change performance on tasks of visual discrimination, reversal learning, rule-based, or object-spatial paired associates learning, suggesting preserved functional integrity of supporting brain circuits. Surprisingly, 56Fe irradiation improved performance on a dentate gyrus-reliant pattern separation task; irradiated mice learned faster and were more accurate than controls. Improved pattern separation performance did not appear to be touchscreen-, radiation particle-, or neurogenesis-dependent, as 56Fe and 28Si irradiation led to faster context discrimination in a non-touchscreen task and 56Fe decreased new dentate gyrus neurons relative to Sham. These data urge revisitation of the broadly-held view that space radiation is detrimental to cognition.

The basolateral amygdala-medial prefrontal cortex circuitry regulates behavioral flexibility during appetitive reversal learning

Environmental cues can become predictors of food availability through Pavlovian conditioning. Two forebrain regions important in this associative learning are the basolateral amygdala (BLA) and medial prefrontal cortex (mPFC). Recent work showed the BLA-mPFC pathway is activated when a cue reliably signals food, suggesting the BLA informs the mPFC of the cue's value. The current study tested this hypothesis by altering the value of 2 food cues using reversal learning and illness-induced devaluation paradigms. Rats that received unilateral excitotoxic lesions of the BLA and mPFC contralaterally placed, along with ipsilateral and sham controls, underwent discriminative conditioning, followed by reversal learning and then devaluation. All groups successfully discriminated between 2 auditory stimuli that were followed by food delivery (conditional stimulus [CS] +) or not rewarded (CS-), demonstrating this learning does not require BLA-mPFC communication. When the outcomes of the stimuli were reversed, the rats with disconnected BLA-mPFC (contralateral condition) showed increased responding to the CSs, especially to the rCS + (original CS-) during the first session, suggesting impaired cue memory recall and behavioral inhibition compared to the other groups. For devaluation, all groups successfully learned conditioned taste aversion; however, there was no evidence of cue devaluation or differences between groups. Interestingly, at the end of testing, the nondevalued contralateral group was still responding more to the original CS + (rCS-) compared to the devalued contralateral group. These results suggest a potential role for BLA-mPFC communication in guiding appropriate responding during periods of behavioral flexibility when the outcomes, and thus the values, of learned cues are altered. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Withdrawal from escalated cocaine self-administration impairs reversal learning by disrupting the effects of negative feedback on reward exploitation: a behavioral and computational analysis

Addiction is regarded as a disorder of inflexible choice with behavior dominated by immediate positive rewards over longer-term negative outcomes. However, the psychological mechanisms underlying the effects of self-administered drugs on behavioral flexibility are not well understood. To investigate whether drug exposure causes asymmetric effects on positive and negative outcomes we used a reversal learning procedure to assess how reward contingencies are utilized to guide behavior in rats previously exposed to intravenous cocaine self-administration (SA). Twenty-four rats were screened for anxiety in an open field prior to acquisition of cocaine SA over six daily sessions with subsequent long-access cocaine SA for 7 days. Control rats (n = 24) were trained to lever-press for food under a yoked schedule of reinforcement. Higher rates of cocaine SA were predicted by increased anxiety and preceded impaired reversal learning, expressed by a decrease in lose-shift as opposed to win-stay probability. A model-free reinforcement learning algorithm revealed that rats with high, but not low cocaine escalation failed to exploit previous reward learning and were more likely to repeat the same response as the previous trial. Eight-day withdrawal from high cocaine escalation was associated, respectively, with increased and decreased dopamine receptor D2 (DRD2) and serotonin receptor 2C (HTR2C) expression in the ventral striatum compared with controls. Dopamine receptor D1 (DRD1) expression was also significantly reduced in the orbitofrontal cortex of high cocaine-escalating rats. These findings indicate that withdrawal from escalated cocaine SA disrupts how negative feedback is used to guide goal-directed behavior for natural reinforcers and that trait anxiety may be a latent variable underlying this interaction.

Reversal learning in Drosophila larvae

Adjusting behavior to changed environmental contingencies is critical for survival, and reversal learning provides an experimental handle on such cognitive flexibility. Here, we investigate reversal learning in larval Drosophila Using odor-taste associations, we establish olfactory reversal learning in the appetitive and the aversive domain, using either fructose as a reward or high-concentration sodium chloride as a punishment, respectively. Reversal learning is demonstrated both in differential and in absolute conditioning, in either valence domain. In differential conditioning, the animals are first trained such that an odor A is paired, for example, with the reward whereas odor B is not (A+/B); this is followed by a second training phase with reversed contingencies (A/B+). In absolute conditioning, odor B is omitted, such that the animals are first trained with paired presentations of A and reward, followed by unpaired training in the second training phase. Our results reveal "true" reversal learning in that the opposite associative effects of both the first and the second training phase are detectable after reversed-contingency training. In what is a surprisingly quick, one-trial contingency adjustment in the Drosophila larva, the present study establishes a simple and genetically easy accessible study case of cognitive flexibility.

The effect of learning on heart rate and behavior of European starlings (Sturnus vulgaris)

Wild-caught European starlings (Sturnus vulgaris) were exposed to a learning task to determine whether heart rate (HR) and behavior responses to the learning activated the sympathetic nervous system. Birds learned to discriminate between images of opposite convexity (concave and convex) based on shading cues in a closed economy (food only available through task completion). Once learned, the task was changed in three ways: (a) manipulating the angle and shape of the image; (b) altering the availability of the task; and (c) reversing the positive stimulus. HR, used as an index of catecholamine release, was measured during each change to determine whether having to alter previously established behaviors to learn new behaviors elicited a sympathetic response. Starlings decreased their HR during the initial discrimination training and did not alter their HR when presented with modified images or when the positive stimulus was reversed. However, HR increased when the task became unavailable and decreased upon its return, suggesting that preventing task performance was perceived as stressful. Birds also modified their behavior when tasks were changed. The number of trials per minute decreased during the reversal treatment, as did the success rate, suggesting that starlings may try to conserve energy when access to food diminishes. This is also supported by the decrease in perch hops per minute when the task was unavailable and the subsequent increase upon its return. Overall, these results suggest that learning per se does not activate the sympathetic nervous system and, therefore, is not a stressor for wild birds.

Contributions of anterior cingulate cortex and basolateral amygdala to decision confidence and learning under uncertainty

The subjective sense of certainty, or confidence, in ambiguous sensory cues can alter the interpretation of reward feedback and facilitate learning. We trained rats to report the orientation of ambiguous visual stimuli according to a spatial stimulus-response rule that must be learned. Following choice, rats could wait a self-timed delay for reward or initiate a new trial. Waiting times increase with discrimination accuracy, demonstrating that this measure can be used as a proxy for confidence. Chemogenetic silencing of BLA shortens waiting times overall whereas ACC inhibition renders waiting times insensitive to confidence-modulating attributes of visual stimuli, suggesting contribution of ACC but not BLA to confidence computations. Subsequent reversal learning is enhanced by confidence. Both ACC and BLA inhibition block this enhancement but via differential adjustments in learning strategies and consistent use of learned rules. Altogether, we demonstrate dissociable roles for ACC and BLA in transmitting confidence and learning under uncertainty.

A computational account of threat-related attentional bias

Visual selective attention acts as a filter on perceptual information, facilitating learning and inference about important events in an agent's environment. A role for visual attention in reward-based decisions has previously been demonstrated, but it remains unclear how visual attention is recruited during aversive learning, particularly when learning about multiple stimuli concurrently. This question is of particular importance in psychopathology, where enhanced attention to threat is a putative feature of pathological anxiety. Using an aversive reversal learning task that required subjects to learn, and exploit, predictions about multiple stimuli, we show that the allocation of visual attention is influenced significantly by aversive value but not by uncertainty. Moreover, this relationship is bidirectional in that attention biases value updates for attended stimuli, resulting in heightened value estimates. Our findings have implications for understanding biased attention in psychopathology and support a role for learning in the expression of threat-related attentional biases in anxiety.

Maladaptive striatal plasticity and abnormal reward-learning in cervical dystonia

In monogenetic generalized forms of dystonia, in vitro neurophysiological recordings have demonstrated direct evidence for abnormal plasticity at the level of the cortico-striatal synapse. It is unclear whether similar abnormalities contribute to the pathophysiology of cervical dystonia, the most common type of focal dystonia. We investigated whether abnormal cortico-striatal synaptic plasticity contributes to abnormal reward-learning behavior in patients with focal dystonia. Forty patients and 40 controls performed a reward gain and loss avoidance reversal learning task. Participant's behavior was fitted to a computational model of the basal ganglia incorporating detailed cortico-striatal synaptic learning rules. Model comparisons were performed to assess the ability of four hypothesized receptor specific abnormalities of cortico-striatal long-term potentiation (LTP) and long-term depression (LTD): increased or decreased D1:LTP/LTD and increased or decreased D2: LTP/LTD to explain abnormal behavior in patients. Patients were selectively impaired in the post-reversal phase of the reward task. Individual learning rates in the reward reversal task correlated with the severity of the patient's motor symptoms. A model of the striatum with decreased D2:LTP/ LTD best explained the patient's behavior, suggesting excessive D2 cortico-striatal synaptic depotentiation could underpin biased reward-learning in patients with cervical dystonia. Reversal learning impairment in cervical dystonia may be a behavioral correlate of D2-specific abnormalities in cortico-striatal synaptic plasticity. Reinforcement learning tasks with computational modeling could allow the identification of molecular targets for novel treatments based on their ability to restore normal reward-learning behavior in these patients.

The MK2 cascade regulates mGluR-dependent synaptic plasticity and reversal learning

The ability to either erase or update the memories of a previously learned spatial task is an essential process that is required to modify behaviour in a changing environment. Current evidence suggests that the neural representation of such cognitive flexibility involves the balancing of synaptic potentiation (acquisition of memories) with synaptic depression (modulation and updating previously acquired memories). Here we demonstrate that the p38 MAPK/MAPK-activated protein kinase 2 (MK2) cascade is required to maintain the precise tuning of long-term potentiation and long-term depression at CA1 synapses of the hippocampus which is correlated with efficient reversal learning. Using the MK2 knockout (KO) mouse, we show that mGluR-LTD, but not NMDAR-LTD, is markedly impaired in mice aged between 4 and 5 weeks (juvenile) to 7 months (mature adult). Although the amplitude of LTP was the same as in wildtype mice, priming of LTP by the activation of group I metabotropic receptors was impaired in MK2 KO mice. Consistent with unaltered LTP amplitude and compromised mGluR-LTD, MK2 KO mice had intact spatial learning when performing the Barnes maze task, but showed specific deficits in selecting the most efficient combination of search strategies to perform the task reversal. Findings from this study suggest that the mGluR-p38-MK2 cascade is important for cognitive flexibility by regulating LTD amplitude and the priming of LTP.

Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2/3 receptor agents

RATIONALE

Disorders of compulsivity such as stimulant use disorder (SUD) and obsessive-compulsive disorder (OCD) are characterised by deficits in behavioural flexibility, some of which have been captured using probabilistic reversal learning (PRL) paradigms.

OBJECTIVES

This study used computational modelling to characterise the reinforcement learning processes underlying patterns of PRL behaviour observed in SUD and OCD and to show how the dopamine D2/3 receptor agonist pramipexole and the D2/3 antagonist amisulpride affected these responses.

METHODS

We applied a hierarchical Bayesian method to PRL data across three groups: individuals with SUD, OCD, and healthy controls. Participants completed three sessions where they received placebo, pramipexole, and amisulpride, in a double-blind placebo-controlled, randomised design. We compared seven models using a bridge sampling estimate of the marginal likelihood.

RESULTS

Stimulus-bound perseveration, a measure of the degree to which participants responded to the same stimulus as before irrespective of outcome, was significantly increased in SUD, but decreased in OCD, compared to controls (on placebo). Individuals with SUD also exhibited reduced reward-driven learning, whilst both the SUD and OCD groups showed increased learning from punishment (nonreward). Pramipexole and amisulpride had similar effects on the control and OCD groups; both increased punishment-driven learning. These D2/3-modulating drugs affected the SUD group differently, remediating reward-driven learning and reducing aspects of perseverative behaviour, amongst other effects.

CONCLUSIONS

We provide a parsimonious computational account of how perseverative tendencies and reward- and punishment-driven learning differentially contribute to PRL in SUD and OCD. D2/3 agents modulated these processes and remediated deficits in SUD in particular, which may inform therapeutic effects.

Regional Striatal Cholinergic Involvement in Human Behavioral Flexibility

Animal studies have shown that the striatal cholinergic system plays a role in behavioral flexibility but, until recently, this system could not be studied in humans due to a lack of appropriate noninvasive techniques. Using proton magnetic resonance spectroscopy, we recently showed that the concentration of dorsal striatal choline (an acetylcholine precursor) changes during reversal learning (a measure of behavioral flexibility) in humans. The aim of the present study was to examine whether regional average striatal choline was associated with reversal learning. A total of 22 participants (mean age = 25.2 years, range = 18-32 years, 13 female) reached learning criterion in a probabilistic learning task with a reversal component. We measured choline at rest in both the dorsal and ventral striatum using magnetic resonance spectroscopy. Task performance was described using a simple reinforcement learning model that dissociates the contributions of positive and negative prediction errors to learning. Average levels of choline in the dorsal striatum were associated with performance during reversal, but not during initial learning. Specifically, lower levels of choline in the dorsal striatum were associated with a lower number of perseverative trials. Moreover, choline levels explained interindividual variance in perseveration over and above that explained by learning from negative prediction errors. These findings suggest that the dorsal striatal cholinergic system plays an important role in behavioral flexibility, in line with evidence from the animal literature and our previous work in humans. Additionally, this work provides further support for the idea of measuring choline with magnetic resonance spectroscopy as a noninvasive way of studying human cholinergic neurochemistry.SIGNIFICANCE STATEMENT Behavioral flexibility is a crucial component of adaptation and survival. Evidence from the animal literature shows that the striatal cholinergic system is fundamental to reversal learning, a key paradigm for studying behavioral flexibility, but this system remains understudied in humans. Using proton magnetic resonance spectroscopy, we showed that choline levels at rest in the dorsal striatum are associated with performance specifically during reversal learning. These novel findings help to bridge the gap between animal and human studies by demonstrating the importance of cholinergic function in the dorsal striatum in human behavioral flexibility. Importantly, the methods described here cannot only be applied to furthering our understanding of healthy human neurochemistry, but also to extending our understanding of cholinergic disorders.

Impaired instrumental reversal learning is associated with increased medial prefrontal cortex activity in Sapap3 knockout mouse model of compulsive behavior

Convergent functional neuroimaging findings implicate hyperactivity across the prefrontal cortex (PFC) and striatum in the neuropathology of obsessive compulsive disorder (OCD). The impact of cortico-striatal circuit hyperactivity on executive functions subserved by these circuits is unclear, because impaired recruitment of PFC has also been observed in OCD patients during paradigms assessing cognitive flexibility. To investigate the relationship between cortico-striatal circuit disturbances and cognitive functioning relevant to OCD, Sapap3 knockout mice (KOs) and littermate controls were tested in an instrumental reversal-learning paradigm to assess cognitive flexibility. Cortical and striatal activation associated with reversal learning was assessed via quantitative analysis of expression of the immediate early gene cFos and generalized linear mixed-effects models. Sapap3-KOs displayed heterogeneous reversal-learning performance, with almost half (n = 13/28) failing to acquire the reversed contingency, while the other 15/28 had similar acquisition as controls. Notably, reversal impairments were not correlated with compulsive grooming severity. cFos analysis revealed that reversal performance declined as medial PFC (mPFC) activity increased in Sapap3-KOs. No such relationship was observed in controls. Our studies are among the first to describe cognitive impairments in a transgenic OCD-relevant model, and demonstrate pronounced heterogeneity among Sapap3-KOs. These findings suggest that increased neural activity in mPFC is associated with impaired reversal learning in Sapap3-KOs, providing a likely neural basis for this observed heterogeneity. The Sapap3-KO model is thus a useful tool for future mechanistic studies to determine how mPFC hyperactivity contributes to OCD-relevant cognitive dysfunction.

Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes

Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.

Does opportunistic testing bias cognitive performance in primates? Learning from drop-outs

Dropouts are a common issue in cognitive tests with non-human primates. One main reason for dropouts is that researchers often face a trade-off between obtaining a sufficiently large sample size and logistic restrictions, such as limited access to testing facilities. The commonly-used opportunistic testing approach deals with this trade-off by only testing those individuals who readily participate and complete the cognitive tasks within a given time frame. All other individuals are excluded from further testing and data analysis. However, it is unknown if this approach merely excludes subjects who are not consistently motivated to participate, or if these dropouts systematically differ in cognitive ability. If the latter holds, the selection bias resulting from opportunistic testing would systematically affect performance scores and thus comparisons between individuals and species. We assessed the potential effects of opportunistic testing on cognitive performance in common marmosets (Callithrix jacchus) and squirrel monkeys (Saimiri sciureus) with a test battery consisting of six cognitive tests: two inhibition tasks (Detour Reaching and A-not-B), one cognitive flexibility task (Reversal Learning), one quantity discrimination task, and two memory tasks. Importantly, we used a full testing approach in which subjects were given as much time as they required to complete each task. For each task, we then compared the performance of subjects who completed the task within the expected number of testing days with those subjects who needed more testing time. We found that the two groups did not differ in task performance, and therefore opportunistic testing would have been justified without risking biased results. If our findings generalise to other species, maximising sample sizes by only testing consistently motivated subjects will be a valid alternative whenever full testing is not feasible.

Structural and Functional Neuroimaging of Polygenic Risk for Schizophrenia: A Recall-by-Genotype-Based Approach

Risk profile scores (RPS) derived from genome-wide association studies (GWAS) explain a considerable amount of susceptibility for schizophrenia (SCZ). However, little is known about how common genetic risk factors for SCZ influence the structure and function of the human brain, largely due to the constraints of imaging sample sizes. In the current study, we use a novel recall-by-genotype (RbG) methodological approach, where we sample young adults from a population cohort (Avon Longitudinal Study of Parents and Children: N genotyped = 8365) based on their SCZ-RPS. We compared 197 healthy individuals at extremes of low (N = 99) or high (N = 98) SCZ-RPS with behavioral tests, and structural and functional magnetic resonance imaging (fMRI). We first provide methodological details that will inform the design of future RbG studies for common SCZ genetic risk. We further provide an between group analysis of the RbG individuals (low vs high SCZ-RPS) who underwent structural neuroimaging data (T1-weighted scans) and fMRI data during a reversal learning task. While we found little evidence for morphometric differences between the low and high SCZ-RPS groups, we observed an impact of SCZ-RPS on blood oxygen level-dependent (BOLD) signal during reward processing in the ventral striatum (PFWE-VS-CORRECTED = .037), a previously investigated broader reward-related network (PFWE-ROIS-CORRECTED = .008), and across the whole brain (PFWE-WHOLE-BRAIN-CORRECTED = .013). We also describe the study strategy and discuss specific challenges of RbG for SCZ risk (such as SCZ-RPS related homoscedasticity). This study will help to elucidate the behavioral and imaging phenotypes that are associated with SCZ genetic risk.

Predicting change: Approximate inference under explicit representation of temporal structure in changing environments

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants' beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants' behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior.

Selective Role of the Putamen in Serial Reversal Learning in the Marmoset

Fronto-striatal circuitry involving the orbitofrontal cortex has been identified as mediating successful reversal of stimulus-outcome contingencies. The region of the striatum that most contributes to reversal learning remains unclear, with studies in primates implicating both caudate nucleus and putamen. We trained four marmosets on a touchscreen-based serial reversal task and implanted each with cannulae targeting both putamen and caudate bilaterally. This allowed reversible inactivation of the two areas within the same monkeys, but across separate sessions, to directly investigate their respective contributions to reversal performance. Behavioral sensitivity to the GABAA agonist muscimol varied across subjects and between brain regions, so each marmoset received a range of doses. Intermediate doses of intra-putamen muscimol selectively impaired reversal performance, leaving the baseline discrimination phase unchanged. There was no effect of low doses and high doses were generally disruptive. By contrast, low doses of intra-caudate muscimol improved reversal performance, while high doses impaired both reversal and baseline discrimination performance. These data provide evidence for a specific role of the putamen in serial reversal learning, which may reflect the more habitual nature of repeated reversals using the same stimulus pair.

Dopamine D2-like receptor stimulation blocks negative feedback in visual and spatial reversal learning in the rat: behavioural and computational evidence

RATIONALE

Dopamine D2-like receptors (D2R) are important drug targets in schizophrenia and Parkinson's disease, but D2R ligands also cause cognitive inflexibility such as poor reversal learning. The specific role of D2R in reversal learning remains unclear.

OBJECTIVES

We tested the hypotheses that D2R agonism impairs reversal learning by blocking negative feedback and that antagonism of D1-like receptors (D1R) impairs learning from positive feedback.

METHODS

Male Lister Hooded rats were trained on a novel visual reversal learning task. Performance on "probe trials", during which the correct or incorrect stimulus was presented with a third, probabilistically rewarded (50% of trials) and therefore intermediate stimulus, revealed individual learning curves for the processes of positive and negative feedback. The effects of D2R and D1R agonists and antagonists were evaluated. A separate cohort was tested on a spatial probabilistic reversal learning (PRL) task after D2R agonism. Computational reinforcement learning modelling was applied to choice data from the PRL task to evaluate the contribution of latent factors.

RESULTS

D2R agonism with quinpirole dose-dependently impaired both visual reversal and PRL. Analysis of the probe trials on the visual task revealed a complete blockade of learning from negative feedback at the 0.25 mg/kg dose, while learning from positive feedback was intact. Estimated parameters from the model that best described the PRL choice data revealed a steep and selective decrease in learning rate from losses. D1R antagonism had a transient effect on the positive probe trials.

CONCLUSIONS

D2R stimulation impairs reversal learning by blocking the impact of negative feedback.

Indices of comparative cognition: assessing animal models of human brain function

Understanding the cognitive capacities of animals is important, because (a) several animal models of human neurodegenerative disease are considered poor representatives of the human equivalent and (b) cognitive capacities may provide insight into alternative animal models. We used a three-stage process of cognitive and neuroanatomical comparison (using sheep as an example) to assess the appropriateness of a species to model human brain function. First, a cognitive task was defined via a reinforcement-learning algorithm where values/constants in the algorithm were taken as indirect measures of neurophysiological attributes. Second, cognitive data (values/constants) were generated for the example species (sheep) and compared to other species. Third, cognitive data were compared with neuroanatomical metrics for each species (endocranial volume, gyrification index, encephalisation quotient, and number of cortical neurons). Four breeds of sheep (n = 15/sheep) were tested using the two-choice discrimination-reversal task. The 'reversal index' was used as a measure of constants within the learning algorithm. Reversal index data ranked sheep as third in a table of species that included primates, dogs, and pigs. Across all species, number of cortical neurons correlated strongest against the reversal index (r2 = 0.66, p = 0.0075) followed by encephalization quotient (r2 = 0.42, p = 0.03), endocranial volume (r2 = 0.30, p = 0.08), and gyrification index (r2 = 0.16, p = 0.23). Sheep have a high predicted level of cognitive capacity and are thus a valid alternative model for neurodegenerative research. Using learning algorithms within cognitive tasks increases the resolution of methods of comparative cognition and can help to identify the most relevant species to model human brain function and dysfunction.

Granulocyte Colony Stimulating Factor Enhances Reward Learning through Potentiation of Mesolimbic Dopamine System Function

Deficits in motivation and cognition are hallmark symptoms of multiple psychiatric diseases. These symptoms are disruptive to quality of life and often do not improve with available medications. In recent years there has been increased interest in the role of the immune system in neuropsychiatric illness, but to date no immune-related treatment strategies have come to fruition. The cytokine granulocyte-colony stimulating factor (G-CSF) is known to have trophic and neuroprotective properties in the brain, and we recently identified it as a modulator of neuronal and behavioral plasticity. By combining operant tasks that assess discrete aspects of motivated behavior and decision-making in male mice and rats with subsecond dopamine monitoring via fast-scan cyclic voltammetry, we defined the role of G-CSF in these processes as well as the neural mechanism by which it modulates dopamine function to exert these effects. G-CSF enhanced motivation for sucrose as well as cognitive flexibility as measured by reversal learning. These behavioral outcomes were driven by mesolimbic dopamine system plasticity, as systemically administered G-CSF increased evoked dopamine release in the nucleus accumbens independent of clearance mechanisms. Importantly, sustained increases in G-CSF were required for these effects as acute exposure did not enhance behavioral outcomes and decreased dopamine release. These effects seem to be a result of the ability of G-CSF to alter local inflammatory signaling cascades, particularly tumor necrosis factor α. Together, these data show G-CSF as a potent modulator of the mesolimbic dopamine circuit and its ability to appropriately attend to salient stimuli.SIGNIFICANCE STATEMENT Emerging evidence has highlighted the importance of the immune system in psychiatric diseases states. However, the effects of peripheral cytokines on motivation and cognitive function are largely unknown. Here, we report that granulocyte-colony stimulating factor (G-CSF), a pleiotropic cytokine with known trophic and neuroprotective properties in the brain, acts directly on dopaminergic circuits to enhance their function. These changes in dopaminergic dynamics enhance reward learning and motivation for natural stimuli. Together, these results suggest that targeting immune factors may provide a new avenue for therapeutic intervention in the multiple psychiatric disorders that are characterized by motivational and cognitive deficits.

Impaired cognitive plasticity and goal-directed control in adolescent obsessive-compulsive disorder

BACKGROUND

Youths with obsessive-compulsive disorder (OCD) experience severe distress and impaired functioning at school and at home. Critical cognitive domains for daily functioning and academic success are learning, memory, cognitive flexibility and goal-directed behavioural control. Performance in these important domains among teenagers with OCD was therefore investigated in this study.

METHODS

A total of 36 youths with OCD and 36 healthy comparison subjects completed two memory tasks: Pattern Recognition Memory (PRM) and Paired Associates Learning (PAL); as well as the Intra-Extra Dimensional Set Shift (IED) task to quantitatively gauge learning as well as cognitive flexibility. A subset of 30 participants of each group also completed a Differential-Outcome Effect (DOE) task followed by a Slips-of-Action Task, designed to assess the balance of goal-directed and habitual behavioural control.

RESULTS

Adolescent OCD patients showed a significant learning and memory impairment. Compared with healthy comparison subjects, they made more errors on PRM and PAL and in the first stages of IED involving discrimination and reversal learning. Patients were also slower to learn about contingencies in the DOE task and were less sensitive to outcome devaluation, suggesting an impairment in goal-directed control.

CONCLUSIONS

This study advances the characterization of juvenile OCD. Patients demonstrated impairments in all learning and memory tasks. We also provide the first experimental evidence of impaired goal-directed control and lack of cognitive plasticity early in the development of OCD. The extent to which the impairments in these cognitive domains impact academic performance and symptom development warrants further investigation.

The Temporal Dynamics of Arc Expression Regulate Cognitive Flexibility

Neuronal activity regulates the transcription and translation of the immediate-early gene Arc/Arg3.1, a key mediator of synaptic plasticity. Proteasome-dependent degradation of Arc tightly limits its temporal expression, yet the significance of this regulation remains unknown. We disrupted the temporal control of Arc degradation by creating an Arc knockin mouse (ArcKR) where the predominant Arc ubiquitination sites were mutated. ArcKR mice had intact spatial learning but showed specific deficits in selecting an optimal strategy during reversal learning. This cognitive inflexibility was coupled to changes in Arc mRNA and protein expression resulting in a reduced threshold to induce mGluR-LTD and enhanced mGluR-LTD amplitude. These findings show that the abnormal persistence of Arc protein limits the dynamic range of Arc signaling pathways specifically during reversal learning. Our work illuminates how the precise temporal control of activity-dependent molecules, such as Arc, regulates synaptic plasticity and is crucial for cognition.

Rats with ventral hippocampal damage are impaired at various forms of learning including conditioned inhibition, spatial navigation, and discriminative fear conditioning to similar contexts

The ventral hippocampus (vHPC) has been implicated in learning and memory functions that seem to differ from its dorsal counterpart. The goal of this series of experiments was to provide further insight into the functional contributions of the vHPC. Our previous work implicated the vHPC in spatial learning, inhibitory learning, and fear conditioning to context. However, the specific role of vHPC on these different forms of learning are not clear. Accordingly, we assessed the effects of neurotoxic lesions of the ventral hippocampus on retention of a conditioned inhibitory association, early versus late spatial navigation in the water task, and discriminative fear conditioning to context under high ambiguity conditions. The results showed that the vHPC was necessary for the expression of conditioned inhibition, early spatial learning, and discriminative fear conditioning to context when the paired and unpaired contexts have high cue overlap. We argue that this pattern of effects, combined with previous work, suggests a key role for vHPC in the utilization of broad contextual representations for inhibition and discriminative memory in high ambiguity conditions.

Serial reversal learning and cognitive flexibility in two species of Neotropical parrots (Diopsittaca nobilis and Pionites melanocephala)

Serial reversal learning of colour discriminations was assessed as an index of cognitive flexibility in two captive species of Neotropical parrots. Both species showed similar performances across serial reversals and no between species differences were observed. In a second task subjects' performances were assessed after they experienced either a low or high pre-reversal learning criterion. If reversal performances improve through processes of associative learning, a high pre-reversal criterion is expected to strengthen previously learned associations and hence impede post-reversal performances. Conversely, highly reinforced associations may facilitate the use of conditional rules that can be generalised across reversals and improve post-reversal performances. We found that high criterion subjects made fewer post-reversal errors and required fewer trials to reach criterion, than low criterion subjects. Red-shouldered macaws and black-headed caiques may therefore demonstrate capacities for solving serial reversal problems by applying conditional rules, rather than learning solely by associative processes. Such performances coincide with findings in great apes, but contrast with findings in monkeys and prosimians, which generally show impaired reversal performances when trained to a highly rigorous pre-reversal criterion. Overall, these findings suggest an evolutionary convergence of cognitive flexibility between parrots and non-human great apes.