Comparable roles for serotonin in rats and humans for computations underlying flexible decision-making

Serotonin is critical for adapting behavior flexibly to meet changing environmental demands. Cognitive flexibility is important for successful attainment of goals, as well as for social interactions, and is frequently impaired in neuropsychiatric disorders, including obsessive-compulsive disorder. However, a unifying mechanistic framework accounting for the role of serotonin in behavioral flexibility has remained elusive. Here, we demonstrate common effects of manipulating serotonin function across two species (rats and humans) on latent processes supporting choice behavior during probabilistic reversal learning, using computational modelling. The findings support a role of serotonin in behavioral flexibility and plasticity, indicated, respectively, by increases or decreases in choice repetition ('stickiness') or reinforcement learning rates following manipulations intended to increase or decrease serotonin function. More specifically, the rate at which expected value increased following reward and decreased following punishment (reward and punishment 'learning rates') was greatest after sub-chronic administration of the selective serotonin reuptake inhibitor (SSRI) citalopram (5 mg/kg for 7 days followed by 10 mg/kg twice a day for 5 days) in rats. Conversely, humans given a single dose of an SSRI (20 mg escitalopram), which can decrease post-synaptic serotonin signalling, and rats that received the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT), which destroys forebrain serotonergic neurons, exhibited decreased reward learning rates. A basic perseverative tendency ('stickiness'), or choice repetition irrespective of the outcome produced, was likewise increased in rats after the 12-day SSRI regimen and decreased after single dose SSRI in humans and 5,7-DHT in rats. These common effects of serotonergic manipulations on rats and humans-identified via computational modelling-suggest an evolutionarily conserved role for serotonin in plasticity and behavioral flexibility and have clinical relevance transdiagnostically for neuropsychiatric disorders.

Perseveration and Shifting in Obsessive-Compulsive Disorder as a Function of Uncertainty, Punishment, and Serotonergic Medication

Background

The nature of cognitive flexibility deficits in obsessive-compulsive disorder (OCD), which historically have been tested with probabilistic reversal learning tasks, remains elusive. Here, a novel deterministic reversal task and inclusion of unmedicated patients in the study sample illuminated the role of fixed versus uncertain rules/contingencies and of serotonergic medication. Additionally, our understanding of probabilistic reversal was enhanced through theoretical computational modeling of cognitive flexibility in OCD.

Methods

We recruited 49 patients with OCD, 21 of whom were unmedicated, and 43 healthy control participants matched for age, IQ, and gender. Participants were tested on 2 tasks: a novel visuomotor deterministic reversal learning task with 3 reversals (feedback rewarding/punishing/neutral) measuring accuracy/perseveration and a 2-choice visual probabilistic reversal learning task with uncertain feedback and a single reversal measuring win-stay and lose-shift. Bayesian computational modeling provided measures of learning rate, reinforcement sensitivity, and stimulus stickiness.

Results

Unmedicated patients with OCD were impaired on the deterministic reversal task under punishment only at the first and third reversals compared with both control participants and medicated patients with OCD, who had no deficit. Perseverative errors were correlated with OCD severity. On the probabilistic reversal task, unmedicated patients were only impaired at reversal, whereas medicated patients were impaired at both the learning and reversal stages. Computational modeling showed that the overall change was reduced feedback sensitivity in both OCD groups.

Conclusions

Both perseveration and increased shifting can be observed in OCD, depending on test conditions including the predictability of reinforcement. Perseveration was related to clinical severity and remediated by serotonergic medication.

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders

BACKGROUND

Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior.

AIMS

It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity.

METHOD

We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data.

RESULTS

We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls.

CONCLUSIONS

Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification.

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Adaptive human learning utilizes reward prediction errors (RPEs) that scale the differences between expected and actual outcomes to optimize future choices. Depression has been linked with biased RPE signaling and an exaggerated impact of negative outcomes on learning which may promote amotivation and anhedonia. The present proof-of-concept study combined computational modeling and multivariate decoding with neuroimaging to determine the influence of the selective competitive angiotensin II type 1 receptor antagonist losartan on learning from positive or negative outcomes and the underlying neural mechanisms in healthy humans. In a double-blind, between-subjects, placebo-controlled pharmaco-fMRI experiment, 61 healthy male participants (losartan, n = 30; placebo, n = 31) underwent a probabilistic selection reinforcement learning task incorporating a learning and transfer phase. Losartan improved choice accuracy for the hardest stimulus pair via increasing expected value sensitivity towards the rewarding stimulus relative to the placebo group during learning. Computational modeling revealed that losartan reduced the learning rate for negative outcomes and increased exploitatory choice behaviors while preserving learning for positive outcomes. These behavioral patterns were paralleled on the neural level by increased RPE signaling in orbitofrontal-striatal regions and enhanced positive outcome representations in the ventral striatum (VS) following losartan. In the transfer phase, losartan accelerated response times and enhanced VS functional connectivity with left dorsolateral prefrontal cortex when approaching maximum rewards. These findings elucidate the potential of losartan to reduce the impact of negative outcomes during learning and subsequently facilitate motivational approach towards maximum rewards in the transfer of learning. This may indicate a promising therapeutic mechanism to normalize distorted reward learning and fronto-striatal functioning in depression.

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans

BACKGROUND

The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans.

METHODS

Healthy volunteers received intravenous LSD (75 μg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history).

RESULTS

Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase.

CONCLUSIONS

Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.

Association of Environmental Uncertainty With Altered Decision-making and Learning Mechanisms in Youths With Obsessive-Compulsive Disorder

Importance

Adults with obsessive-compulsive disorder (OCD) display perseverative behavior in stable environments but exhibit vacillating choice when payoffs are uncertain. These findings may be associated with intolerance of uncertainty and compulsive behaviors; however, little is known about the mechanisms underlying learning and decision-making in youths with OCD because research into this population has been limited.

Objective

To investigate cognitive mechanisms associated with decision-making in youths with OCD by using executive functioning tasks and computational modeling.

Design, Setting, and Participants

In this cross-sectional study, 50 youths with OCD (patients) and 53 healthy participants (controls) completed a probabilistic reversal learning (PRL) task between January 2014 and March 2020. A separate sample of 27 patients and 46 controls completed the Wisconsin Card Sorting Task (WCST) between January 2018 and November 2020. The study took place at the University of Cambridge in the UK.

Main Outcomes and Measures

Decision-making mechanisms were studied by fitting hierarchical bayesian reinforcement learning models to the 2 data sets and comparing model parameters between participant groups. Model parameters included reward and punishment learning rates (feedback sensitivity), reinforcement sensitivity and decision consistency (exploitation), and stickiness (perseveration). Associations of receipt of serotonergic medication with performance were assessed.

Results

In total, 50 patients (29 female patients [58%]; median age, 16.6 years [IQR, 15.3-18.0 years]) and 53 controls (30 female participants [57%]; median age, 16.4 years [IQR, 14.8-18.0 years]) completed the PRL task. A total of 27 patients (18 female patients [67%]; median age, 16.1 years [IQR, 15.2-17.2 years]) and 46 controls (28 female participants [61%]; median age, 17.2 [IQR, 16.3-17.6 years]) completed the WCST. During the reversal phase of the PRL task, patients made fewer correct responses (mean [SD] proportion: 0.83 [0.16] for controls and 0.61 [0.31] for patients; 95% CI, -1.31 to -0.64) and switched choices more often following false-negative feedback (mean [SD] proportion: 0.09 [0.16] for controls vs 0.27 [0.34] for patients; 95% CI, 0.60-1.26) and true-positive feedback (mean [SD] proportion: 0.93 [0.17] for controls vs 0.73 [0.34] for patients; 95% CI, -2.17 to -1.31). Computational modeling revealed that patients displayed enhanced reward learning rates (mean difference [MD], 0.21; 95% highest density interval [HDI], 0.04-0.38) but decreased punishment learning rates (MD, -0.29; 95% HDI, -0.39 to -0.18), reinforcement sensitivity (MD, -4.91; 95% HDI, -9.38 to -1.12), and stickiness (MD, -0.35; 95% HDI, -0.57 to -0.11) compared with controls. There were no group differences on standard WCST measures and computational model parameters. However, patients who received serotonergic medication showed slower response times (mean [SD], 1420.49 [279.71] milliseconds for controls, 1471.42 [212.81] milliseconds for patients who were unmedicated, and 1738.25 [349.23] milliseconds for patients who were medicated) (control vs medicated MD, -320.26 [95% CI, -547.00 to -88.68]) and increased unique errors (mean [SD] proportion: 0.001 [0.004] for controls, 0.002 [0.004] for patients who were unmedicated, and 0.008 [0.01] for patients who were medicated) (control vs medicated MD, -0.007 [95% CI, -3.14 to -0.36]) on the WCST.

Conclusions and Relevance

The results of this cross-sectional study indicated that youths with OCD showed atypical probabilistic reversal learning but were generally unimpaired on the deterministic WCST, although unexpected results were observed for patients receiving serotonergic medication. These findings have implications for reframing the understanding of early-onset OCD as a disorder in which decision-making is associated with uncertainty in the environment, a potential target for therapeutic treatment. These results provide continuity with findings in adults with OCD.

Serotonin depletion amplifies distinct human social emotions as a function of individual differences in personality

Serotonin is involved in a wide range of mental capacities essential for navigating the social world, including emotion and impulse control. Much recent work on serotonin and social functioning has focused on decision-making. Here we investigated the influence of serotonin on human emotional reactions to social conflict. We used a novel computerised task that required mentally simulating social situations involving unjust harm and found that depleting the serotonin precursor tryptophan-in a double-blind randomised placebo-controlled design-enhanced emotional responses to the scenarios in a large sample of healthy volunteers (n = 73), and interacted with individual differences in trait personality to produce distinctive human emotions. Whereas guilt was preferentially elevated in highly empathic participants, annoyance was potentiated in those high in trait psychopathy, with medium to large effect sizes. Our findings show how individual differences in personality, when combined with fluctuations of serotonin, may produce diverse emotional phenotypes. This has implications for understanding vulnerability to psychopathology, determining who may be more sensitive to serotonin-modulating treatments, and casts new light on the functions of serotonin in emotional processing.

Effect of Tryptophan Depletion on Conditioned Threat Memory Expression: Role of Intolerance of Uncertainty

Background

Responding emotionally to danger is critical for survival. Normal functioning also requires flexible alteration of emotional responses when a threat becomes safe. Aberrant threat and safety learning occur in many psychiatric disorders, including posttraumatic stress disorder, obsessive-compulsive disorder, and schizophrenia, in which emotional responses can persist pathologically. While there is evidence that threat and safety learning can be modulated by the serotonin systems, there have been few studies in humans. We addressed a critical clinically relevant question: How does lowering serotonin affect memory retention of conditioned threat and safety memory?

Methods

Forty-seven healthy participants underwent conditioning to two stimuli predictive of threat on day 1. One stimulus but not the other was subsequently presented in an extinction session. Emotional responding was assessed by the skin conductance response. On day 2, we employed acute dietary tryptophan depletion to lower serotonin temporarily, in a double-blind, placebo-controlled, randomized between-groups design. We then tested for the retention of conditioned threat and extinction memory. We also measured self-reported intolerance of uncertainty, known to modulate threat memory expression.

Results

The expression of emotional memory was attenuated in participants who had undergone tryptophan depletion. Individuals who were more intolerant of uncertainty showed even greater attenuation of emotion following depletion.

Conclusions

These results support the view that serotonin is involved in predicting aversive outcomes and refine our understanding of the role of serotonin in the persistence of emotional responsivity, with implications for individual differences in vulnerability to psychopathology.

Association between childhood trauma and risk for obesity: a putative neurocognitive developmental pathway

BACKGROUND

Childhood trauma increases the risk for adult obesity through multiple complex pathways, and the neural substrates are yet to be determined.

METHODS

Participants from three population-based neuroimaging cohorts, including the IMAGEN cohort, the UK Biobank (UKB), and the Human Connectome Project (HCP), were recruited. Voxel-based morphometry analysis of both childhood trauma and body mass index (BMI) was performed in the longitudinal IMAGEN cohort; validation of the findings was performed in the UKB. White-matter connectivity analysis was conducted to study the structural connectivity between the identified brain region and subdivisions of the hypothalamus in the HCP.

RESULTS

In IMAGEN, a smaller frontopolar cortex (FPC) was associated with both childhood abuse (CA) (β = - .568, 95%CI - .942 to - .194; p = .003) and higher BMI (β = - .086, 95%CI - .128 to - .043; p < .001) in male participants, and these findings were validated in UKB. Across seven data collection sites, a stronger negative CA-FPC association was correlated with a higher positive CA-BMI association (β = - 1.033, 95%CI - 1.762 to - .305; p = .015). Using 7-T diffusion tensor imaging data (n = 156), we found that FPC was the third most connected cortical area with the hypothalamus, especially the lateral hypothalamus. A smaller FPC at age 14 contributed to higher BMI at age 19 in those male participants with a history of CA, and the CA-FPC interaction enabled a model at age 14 to account for some future weight gain during a 5-year follow-up (variance explained 5.8%).

CONCLUSIONS

The findings highlight that a malfunctioning, top-down cognitive or behavioral control system, independent of genetic predisposition, putatively contributes to excessive weight gain in a particularly vulnerable population, and may inform treatment approaches.

Probabilistic reversal learning under acute tryptophan depletion in healthy humans: a conventional analysis

The involvement of serotonin in responses to negative feedback is well established. Acute serotonin reuptake inhibition has enhanced sensitivity to negative feedback (SNF), modelled by behaviour in probabilistic reversal learning (PRL) paradigms. Whilst experiments employing acute tryptophan depletion (ATD) in humans, to reduce serotonin synthesis, have shown no clear effect on SNF, sample sizes have been small. We studied a large sample of healthy volunteers, male and female, and found ATD had no effect on core behavioural measures in PRL. These results indicate that ATD effects can differ from other manipulations of serotonin expected to have a parallel or opposing action.

Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2/3 receptor agents

RATIONALE

Disorders of compulsivity such as stimulant use disorder (SUD) and obsessive-compulsive disorder (OCD) are characterised by deficits in behavioural flexibility, some of which have been captured using probabilistic reversal learning (PRL) paradigms.

OBJECTIVES

This study used computational modelling to characterise the reinforcement learning processes underlying patterns of PRL behaviour observed in SUD and OCD and to show how the dopamine D2/3 receptor agonist pramipexole and the D2/3 antagonist amisulpride affected these responses.

METHODS

We applied a hierarchical Bayesian method to PRL data across three groups: individuals with SUD, OCD, and healthy controls. Participants completed three sessions where they received placebo, pramipexole, and amisulpride, in a double-blind placebo-controlled, randomised design. We compared seven models using a bridge sampling estimate of the marginal likelihood.

RESULTS

Stimulus-bound perseveration, a measure of the degree to which participants responded to the same stimulus as before irrespective of outcome, was significantly increased in SUD, but decreased in OCD, compared to controls (on placebo). Individuals with SUD also exhibited reduced reward-driven learning, whilst both the SUD and OCD groups showed increased learning from punishment (nonreward). Pramipexole and amisulpride had similar effects on the control and OCD groups; both increased punishment-driven learning. These D2/3-modulating drugs affected the SUD group differently, remediating reward-driven learning and reducing aspects of perseverative behaviour, amongst other effects.

CONCLUSIONS

We provide a parsimonious computational account of how perseverative tendencies and reward- and punishment-driven learning differentially contribute to PRL in SUD and OCD. D2/3 agents modulated these processes and remediated deficits in SUD in particular, which may inform therapeutic effects.

Persistence of Amygdala-Hippocampal Connectivity and Multi-Voxel Correlation Structures During Awake Rest After Fear Learning Predicts Long-Term Expression of Fear

After encoding, memories undergo a process of consolidation that determines long-term retention. For conditioned fear, animal models postulate that consolidation involves reactivations of neuronal assemblies supporting fear learning during postlearning "offline" periods. However, no human studies to date have investigated such processes, particularly in relation to long-term expression of fear. We tested 24 participants using functional MRI on 2 consecutive days in a fear conditioning paradigm involving 1 habituation block, 2 acquisition blocks, and 2 extinction blocks on day 1, and 2 re-extinction blocks on day 2. Conditioning blocks were preceded and followed by 4.5-min rest blocks. Strength of spontaneous recovery of fear on day 2 served as a measure of long-term expression of fear. Amygdala connectivity primarily with hippocampus increased progressively during postacquisition and postextinction rest on day 1. Intraregional multi-voxel correlation structures within amygdala and hippocampus sampled during a block of differential fear conditioning furthermore persisted after fear learning. Critically, both these main findings were stronger in participants who exhibited spontaneous recovery 24 h later. Our findings indicate that neural circuits activated during fear conditioning exhibit persistent postlearning activity that may be functionally relevant in promoting consolidation of the fear memory.

Young and old Pavlovian fear memories can be modified with extinction training during reconsolidation in humans

Extinction training during reconsolidation has been shown to persistently diminish conditioned fear responses across species. We investigated in humans if older fear memories can benefit similarly. Using a Pavlovian fear conditioning paradigm we compared standard extinction and extinction after memory reactivation 1 d or 7 d following acquisition. Participants who underwent extinction during reconsolidation showed no evidence of fear recovery, whereas fear responses returned in participants who underwent standard extinction. We observed this effect in young and old fear memories. Extending the beneficial use of reconsolidation to older fear memories in humans is promising for therapeutic applications.