5-HT 2A and 5-HT 2C receptor antagonism differentially modulate reinforcement learning and cognitive flexibility: behavioural and computational evidence

RATIONALE

Cognitive flexibility, the ability to adapt behaviour in response to a changing environment, is disrupted in several neuropsychiatric disorders, including obsessive-compulsive disorder and major depressive disorder. Evidence suggests that flexibility, which can be operationalised using reversal learning tasks, is modulated by serotonergic transmission. However, how exactly flexible behaviour and associated reinforcement learning (RL) processes are modulated by 5-HT action on specific receptors is unknown.

OBJECTIVES

We investigated the effects of 5-HT2A receptor (5-HT2AR) and 5-HT2C receptor (5-HT2CR) antagonism on flexibility and underlying RL mechanisms.

METHODS

Thirty-six male Lister hooded rats were trained on a touchscreen visual discrimination and reversal task. We evaluated the effects of systemic treatments with the 5-HT2AR and 5-HT2CR antagonists M100907 and SB-242084, respectively, on reversal learning and performance on probe trials where correct and incorrect stimuli were presented with a third, probabilistically rewarded, stimulus. Computational models were fitted to task choice data to extract RL parameters, including a novel model designed specifically for this task.

RESULTS

5-HT2AR antagonism impaired reversal learning only after an initial perseverative phase, during a period of random choice and then new learning. 5-HT2CR antagonism, on the other hand, impaired learning from positive feedback. RL models further differentiated these effects. 5-HT2AR antagonism decreased punishment learning rate (i.e. negative feedback) at high and low doses. The low dose also decreased reinforcement sensitivity (beta) and increased stimulus and side stickiness (i.e., the tendency to repeat a choice regardless of outcome). 5-HT2CR antagonism also decreased beta, but reduced side stickiness.

CONCLUSIONS

These data indicate that 5-HT2A and 5-HT2CRs both modulate different aspects of flexibility, with 5-HT2ARs modulating learning from negative feedback as measured using RL parameters and 5-HT2CRs for learning from positive feedback assessed through conventional measures.

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders

BACKGROUND

Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior.

AIMS

It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity.

METHOD

We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data.

RESULTS

We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls.

CONCLUSIONS

Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification.

Cognitive flexibility: neurobehavioral correlates of changing one's mind

Behavioral and cognitive flexibility allow adaptation to a changing environment. Most tasks used to investigate flexibility require switching reactively in response to deterministic task-response rules. In daily life, flexibility often involves a volitional decision to change behavior. This can be instigated by environmental signals, but these are frequently unreliable. We report results from a novel "change your mind" task, which assesses volitional switching under uncertainty without the need for rule-based learning. Participants completed a two-alternative choice task, and following spurious feedback, were presented with the same stimulus again. Subjects had the opportunity to repeat or change their response. Forty healthy participants completed the task while undergoing a functional magnetic resonance imaging scan. Participants predominantly repeated their choice but changed more when their first response was incorrect or when the feedback was negative. Greater activations for changing were found in the inferior frontal junction, anterior insula (AI), anterior cingulate, and dorsolateral prefrontal cortex. Changing responses were also accompanied by reduced connectivity from the AI and orbitofrontal cortices to the occipital cortex. Using multivariate pattern analysis of brain activity, we predicted with 77% reliability whether participants would change their mind. These findings extend our understanding of cognitive flexibility in daily life by assessing volitional decision-making.

Sex-dependent effects of early life stress on reinforcement learning and limbic cortico-striatal functional connectivity

Major depressive disorder (MDD) is a stress-related condition hypothesized to involve aberrant reinforcement learning (RL) with positive and negative stimuli. The present study investigated whether repeated early maternal separation (REMS) stress, a procedure widely recognized to cause depression-like behaviour, affects how subjects learn from positive and negative feedback. The REMS procedure was implemented by separating male and female rats from their dam for 6 h each day from post-natal day 5-19. Control rat offspring were left undisturbed during this period. Rats were tested as adults for behavioral flexibility and feedback sensitivity on a probabilistic reversal learning task. A computational approach based on RL theory was used to derive latent behavioral variables related to reward learning and flexibility. To assess underlying brain substrates, a seed-based functional MRI connectivity analysis was applied both before and after an additional adulthood stressor in control and REMS rats. Female but not male rats exposed to REMS stress showed increased response 'stickiness' (repeated responses regardless of reward outcome). Following repeated adulthood stress, reduced functional connectivity from the basolateral amygdala (BLA) to the dorsolateral striatum (DLS), cingulate cortex (Cg), and anterior insula (AI) cortex was observed in females. By contrast, control male rats exposed to the second stressor showed impaired learning from negative feedback (i.e., non-reward) and reduced functional connectivity from the BLA to the DLS and AI compared to maternally separated males. RL in male rats exposed to REMS was unaffected. The fMRI data further revealed that connectivity between the mOFC and other prefrontal cortical and subcortical structures was positively correlated with response 'stickiness'. These findings reveal differences in how females and males respond to early life adversity and subsequent stress. These effects may be mediated by functional divergence in resting-state connectivity between the basolateral amygdala and fronto-striatal brain regions.