5-HT 2A and 5-HT 2C receptor antagonism differentially modulate reinforcement learning and cognitive flexibility: behavioural and computational evidence

RATIONALE

Cognitive flexibility, the ability to adapt behaviour in response to a changing environment, is disrupted in several neuropsychiatric disorders, including obsessive-compulsive disorder and major depressive disorder. Evidence suggests that flexibility, which can be operationalised using reversal learning tasks, is modulated by serotonergic transmission. However, how exactly flexible behaviour and associated reinforcement learning (RL) processes are modulated by 5-HT action on specific receptors is unknown.

OBJECTIVES

We investigated the effects of 5-HT2A receptor (5-HT2AR) and 5-HT2C receptor (5-HT2CR) antagonism on flexibility and underlying RL mechanisms.

METHODS

Thirty-six male Lister hooded rats were trained on a touchscreen visual discrimination and reversal task. We evaluated the effects of systemic treatments with the 5-HT2AR and 5-HT2CR antagonists M100907 and SB-242084, respectively, on reversal learning and performance on probe trials where correct and incorrect stimuli were presented with a third, probabilistically rewarded, stimulus. Computational models were fitted to task choice data to extract RL parameters, including a novel model designed specifically for this task.

RESULTS

5-HT2AR antagonism impaired reversal learning only after an initial perseverative phase, during a period of random choice and then new learning. 5-HT2CR antagonism, on the other hand, impaired learning from positive feedback. RL models further differentiated these effects. 5-HT2AR antagonism decreased punishment learning rate (i.e. negative feedback) at high and low doses. The low dose also decreased reinforcement sensitivity (beta) and increased stimulus and side stickiness (i.e., the tendency to repeat a choice regardless of outcome). 5-HT2CR antagonism also decreased beta, but reduced side stickiness.

CONCLUSIONS

These data indicate that 5-HT2A and 5-HT2CRs both modulate different aspects of flexibility, with 5-HT2ARs modulating learning from negative feedback as measured using RL parameters and 5-HT2CRs for learning from positive feedback assessed through conventional measures.

Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Much evidence suggests that reversal learning is mediated by cortico-striatal circuitries with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, but potential differential roles of lateral (lOFC) and medial (mOFC) portions in visual reversal learning have yet to be determined. We investigated the effects of pharmacological inactivation of mOFC and lOFC on a deterministic serial visual reversal learning task for rats. For reference, we also targeted other areas previously implicated in reversal learning: prelimbic (PrL) and infralimbic (IL) prefrontal cortex, and basolateral amygdala (BLA). Inactivating mOFC and lOFC produced opposite effects; lOFC impairing, and mOFC improving, performance in the early, perseverative phase specifically. Additionally, mOFC inactivation enhanced negative feedback sensitivity, while lOFC inactivation diminished feedback sensitivity in general. mOFC and lOFC inactivation also affected novel visual discrimination learning differently; lOFC inactivation paradoxically improved learning, and mOFC inactivation had no effect. We also observed dissociable roles of the OFC and the IL/PrL. Whereas the OFC inactivation affected only perseveration, IL/PrL inactivation improved learning overall. BLA inactivation did not affect perseveration, but improved the late phase of reversal learning. These results support opponent roles of the rodent mOFC and lOFC in deterministic visual reversal learning.

Dissociable contributions of mediodorsal and anterior thalamic nuclei in visual attentional performance: A comparison using nicotinic and muscarinic cholinergic receptor antagonists

BACKGROUND

Thalamic subregions mediate various cognitive functions, including attention, inhibitory response control and decision making. Such neuronal activity is modulated by cholinergic thalamic afferents and deterioration of such modulatory signaling has been theorised to contribute to cognitive decline in neurodegenerative disorders. However, the thalamic subnuclei and cholinergic receptors involved in cognitive functioning remain largely unknown.

AIMS

We investigated whether muscarinic or nicotinic receptors in the mediodorsal thalamus and anterior thalamus contribute to rats' performance in the five-choice serial reaction time task, which measures sustained visual attention and impulsive action.

METHODS

Male Long-Evans rats were trained in the five-choice serial reaction time task then surgically implanted with guide cannulae targeting either the mediodorsal thalamus or anterior thalamus. Reversible inactivation of either the mediodorsal thalamus or anterior thalamus were achieved with infusions of the γ-aminobutyric acid-ergic agonists muscimol and baclofen prior to behavioural assessment. To investigate cholinergic mechanisms, we also assessed the behavioural effects of locally administered nicotinic (mecamylamine) and muscarinic (scopolamine) receptor antagonists.

RESULTS

Reversible inactivation of the mediodorsal thalamus severely impaired discriminative accuracy and response speed and increased omissions. Inactivation of the anterior thalamus produced less profound effects, with impaired accuracy at the highest dose. In contrast, blocking cholinergic transmission in these regions did not significantly affect five-choice serial reaction time task performance.

CONCLUSIONS/INTERPRETATIONS

These findings show the mediodorsal thalamus plays a key role in visuospatial attentional performance that is independent of local cholinergic neurotransmission.

Glutamatergic and Serotonergic Modulation of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Adapting behavior to a dynamic environment requires both steadiness when the environment is stable and behavioral flexibility in response to changes. Much evidence suggests that cognitive flexibility, which can be operationalized in reversal learning tasks, is mediated by cortico-striatal circuitries, with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, and we have previously reported differential roles of lateral (lOFC) and medial (mOFC) regions in a touchscreen serial visual reversal learning task for rats using pharmacological inactivation. Here, we investigated the effects of pharmacological overactivation of these regions using a glutamate transporter 1 (GLT-1) inhibitor, dihydrokainate (DHK), which increases extracellular glutamate by blocking its reuptake. We also tested the impact of antagonism of the serotonin 2A receptor (5-HT_2AR), which modulates glutamate action, in the mOFC and lOFC on the same task. Overactivation induced by DHK produced dissociable effects in the mOFC and lOFC, with more prominent effects in the mOFC, specifically improving performance in the early, perseveration phase. Intra-lOFC DHK increased the number of omitted responses without affecting errors. In contrast, blocking the 5-HT_2AR in the lOFC impaired reversal learning overall, while mOFC 5-HT_2AR blockade had no effect. These results further support dissociable roles of the rodent mOFC and lOFC in deterministic visual reversal learning and indicate that modulating glutamate transmission through blocking the GLT-1 and the 5-HT_2AR have different roles in these two structures.

This study further supports dissociable roles of specific orbitofrontal subregions, as well as glutamatergic and serotonergic transmission in these subregions, in cognitive flexibility. This knowledge will add to the understanding of specific neural mechanisms underlying inflexible behaviour across psychiatric disorders.