The role of serotonin in nonnormative risky choice: the effects of tryptophan supplements on the "reflection effect" in healthy adult volunteers

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Decision-making under risk and ambiguity in adults with Tourette syndrome

BACKGROUND

Tourette syndrome (TS) as well as its most common comorbidities are associated with a higher propensity for risky behaviour in everyday life. However, it is unclear whether this increased risk propensity in real-life contexts translates into a generally increased attitude towards risk. We aimed to assess decision-making under risk and ambiguity based on prospect theory by considering the effects of comorbidities and medication.

METHODS

Fifty-four individuals with TS and 32 healthy controls performed risk and ambiguity decision-making tasks under both gains and losses conditions. Behavioural and computational parameters were evaluated using (i) univariate analysis to determine parameters difference taking independently; (ii) supervised multivariate analysis to evaluate whether our parameters could jointly account for between-group differences (iii) unsupervised multivariate analysis to explore the potential presence of sub-groups.

RESULTS

Except for general 'noisier' (less consistent) decisions in TS, we showed no specific risk-taking behaviour in TS or any relation with tics severity or antipsychotic medication. However, the presence of comorbidities was associated with distortion of decision-making. Specifically, TS with obsessive-compulsive disorder comorbidity was associated with a higher risk-taking profile to increase gain and a higher risk-averse profile to decrease loss. TS with attention-deficit hyperactivity disorder comorbidity was associated with risk-seeking in the ambiguity context to reduce a potential loss.

CONCLUSIONS

Impaired valuation of risk and ambiguity was not related to TS per se. Our findings are important for clinical practice: the involvement of individuals with TS in real-life risky situations may actually rather result from other factors such as psychiatric comorbidities.

Risk taking for potential losses but not gains increases with time of day

Humans exhibit distinct risk preferences when facing choices involving potential gains and losses. These preferences are believed to be subject to neuromodulatory influence, particularly from dopamine and serotonin. As neuromodulators manifest circadian rhythms, this suggests decision making under risk might be affected by time of day. Here, in a large subject sample collected using a smartphone application, we found that risky options with potential losses were increasingly chosen over the course of the day. We observed this result in both a within-subjects design (N = 2599) comparing risky options chosen earlier and later in the day in the same individuals, and in a between-subjects design (N = 26,720) showing our effect generalizes across ages and genders. Using computational modelling, we show this diurnal change in risk preference reflects a decrease in sensitivity to increasing losses, but no change was observed in the relative impacts of gains and losses on choice (i.e., loss aversion). Thus, our findings reveal a striking diurnal modulation in human decision making, a pattern with potential importance for real-life decisions that include voting, medical decisions, and financial investments.

Influences of social uncertainty and serotonin on gambling decisions

In many instances in life, our decisions’ outcomes hinge on someone else’s choices (i.e., under social uncertainty). Behavioral and pharmacological work has previously focused on different types of uncertainty, such as risk and ambiguity, but not so much on risk behaviors under social uncertainty. Here, in two different studies using a double-blind, placebo-controlled, within-subject design, we administrated citalopram (a selective-serotonin-reuptake inhibitor) to male participants and investigated decisions in a gambling task under social and nonsocial uncertainty. In the social condition, gamble outcomes were determined by another participant. In the nonsocial condition, gamble outcomes were determined by a coin toss. We observed increased gamble acceptance under social uncertainty, especially for gambles with lower gains and higher losses, which might be indicative of a positivity bias in social expectations in conditions of high uncertainty about others’ behaviors. A similar effect was found for citalopram, which increased overall acceptance behavior for gambles irrespective of the source of uncertainty (social/nonsocial). These results provide insights into the cognitive and neurochemical processes underlying decisions under social uncertainty, with implications for research in risk-taking behaviors in healthy and clinical populations.

Dopamine increases risky choice while D2 blockade shortens decision time

Dopamine is crucially involved in decision-making and overstimulation within dopaminergic pathways can lead to impulsive behaviour, including a desire to take risks and reduced deliberation before acting. These behavioural changes are side effects of treatment with dopaminergic drugs in Parkinson disease, but their likelihood of occurrence is difficult to predict and may be influenced by the individual's baseline endogenous dopamine state, and indeed correlate with sensation-seeking personality traits. We here collected data on a standard gambling task in healthy volunteers given either placebo, 2.5 mg of the dopamine antagonist haloperidol or 100/25 mg of the dopamine precursor levodopa in a within-subject design. We found an increase in risky choices on levodopa. Choices were, however, made faster on haloperidol with no effect of levodopa on deliberation time. Shortened deliberation times on haloperidol occurred in low sensation-seekers only, suggesting a correlation between sensation-seeking personality trait and baseline dopamine levels. We hypothesise that levodopa increases risk-taking behaviour via overstimulation at both D1 and D2 receptor level, while a single low dose of haloperidol, as previously reported (Frank and O'Reilly 2006), may block D2 receptors pre- and post-synaptically and may paradoxically lead to higher striatal dopamine acting on remaining striatal D1 receptors, causing speedier decision without influencing risk tolerance. These effects could also fit with a recently proposed computational model of the basal ganglia (Moeller and Bogacz 2019; Moeller et al. 2021). Furthermore, our data suggest that the actual dopaminergic drug effect may be dependent on the individual's baseline dopamine state, which may influence our therapeutic decision as clinicians in the future.

The Effect of the 5-HT4 Agonist, Prucalopride, on a Functional Magnetic Resonance Imaging Faces Task in the Healthy Human Brain

Depression is a common and often recurrent illness with significant negative impact on a global scale. Current antidepressants are ineffective for up to one third of people with depression, many of whom experience persistent symptomatology. 5-HT₄ receptor agonists show promise in both animal models of depression and cognitive deficit. We therefore studied the effect of the 5-HT₄ partial agonist prucalopride (1 mg daily for 6 days) on the neural processing of emotional faces in 43 healthy participants using a randomised placebo-controlled design. Participants receiving prucalopride were more accurate at identifying the gender of emotional faces. In whole brain analyses, prucalopride was also associated with reduced activation in a network of regions corresponding to the default mode network. However, there was no evidence that prucalopride treatment produced a positive bias in the neural processing of emotional faces. Our study provides further support for a pro-cognitive effect of 5-HT₄ receptor agonism in humans. While our current behavioural and neural investigations do not suggest an antidepressant-like profile of prucalopride in humans, it will be important to study a wider dose range in future studies.

To learn or to gain: neural signatures of exploration in human decision-making

Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.

Selective effects of serotonin on choices to gather more information

BACKGROUND

Gathering and evaluating information leads to better decisions, but often at cost. The balance between information seeking and exploitation features in neurodevelopmental, mood, psychotic and substance-related disorders. Serotonin's role has been highlighted by experimental reduction of its precursor, tryptophan.

AIMS

We tested the boundaries and applicability of this role by asking whether changes to information sampling would be observed following acute doses of serotonergic and catecholaminergic clinical treatments. We used a variant of the Information Sampling Task (IST) to measure how much information a person requires before they make a decision. This task allows participants to sample information until satisfied to make a choice.

METHODS

In separate double-blind placebo-controlled experiments, we tested 27 healthy participants on/off 20 mg of the serotonin reuptake inhibitor (SRI) citalopram, and 22 participants on/off 40 mg of the noradrenergic reuptake inhibitor atomoxetine. The IST variant minimised effects of temporal impulsivity and loss aversion. Analyses used a variety of participant prior expectations of sampling spaces in the IST, including a new prior that accounts for learning of likely states across trials. We analysed behaviour by a new method that also accounts for baseline individual differences of risk preference.

RESULTS

Baseline preferences demonstrated risk aversion. Citalopram decreased the expected utility of choices and probability of being correct based on informational content of samples collected, suggesting participants collected less useful information before making a choice. Atomoxetine did not influence information seeking.

CONCLUSION

Acute changes of serotonin activity by way of a single SRI dose alter information-seeking behaviour.

Eating to dare - Nutrition impacts human risky decision and related brain function

Macronutrient composition modulates plasma amino acids that are precursors of neurotransmitters and can impact brain function and decisions. Neurotransmitter serotonin has been shown to regulate not only food intake, but also economic decisions. We investigated whether an acute nutrition-manipulation inducing plasma tryptophan fluctuation affects brain function, thereby affecting risky decisions. Breakfasts differing in carbohydrate/protein ratios were offered to test changes in risky decision-making while metabolic and neural dynamics were tracked. We identified that a high-carbohydrate/protein breakfast increased plasma tryptophan/LNAA (large neutral amino acids) ratio which mapped to individual risk propensity changes. The nutrition-manipulation and tryptophan/LNAA fluctuation effects on risk propensity changes were further modulated by individual differences in body fat mass. Using fMRI, we further identified activation in the parietal lobule during risk-processing, of which activities 1) were sensitive to the tryptophan/LNAA fluctuation, 2) were modulated by individual's body fat mass, and 3) predicted the risk propensity changes in decision-making. Our results provide evidence for a personalized nutrition-driven modulation on human risky decision and its metabolic and neural mechanisms.

A systematic review of the effect of L-tryptophan supplementation on mood and emotional functioning

L-tryptophan (TRP), one of the essential amino acids in humans, is a precursor of serotonin, and hence its intake is closely related to the suppression of depressed and anxious moods. We did a systematic review of RCTs to examine the effects of tryptophan intake on the mood of healthy adults by searching PubMed, the Cochrane Library, and Ichu-shi according to PRISMA guidelines. As a result, 11 RCTs met the criteria and were accepted. Four RCTs showed the effects of tryptophan intake on negative feelings and happy feelings in healthy individuals, with significant differences between the treatment and the control groups. This suggests that TRP intake may be an effective approach to decrease anxiety and increase positive mood in healthy individuals. On the other hand, the effectiveness of TRP for aggressive feelings was not recognized. Reviewing these 11 RCTs, we concluded that taking 0.14-3 g of TRP per day in addition to the usual meal can be expected to improve the mood of healthy individuals. In order to estimate the optimum amount of TRP intake more accurately, further studies need to be conducted with more appropriate settings of intake period, intake frequency, and intake method.

Risk-seeking for losses is associated with 5-HTTLPR, but not with transient changes in 5-HT levels

RATIONALE

Serotonin (5-HT) plays a key role in different aspects of value-based decision-making. A recent framework proposed that tonic 5-HT (together with dopamine, DA) codes future average reward expectations, providing a baseline against which possible choice outcomes are compared to guide decision-making.

OBJECTIVES

To test whether high 5-HT levels decrease loss aversion, risk-seeking for gains, and risk-seeking for losses.

METHODS

In a first session, 611 participants were genotyped for 5-HTTLPR and performed a mixed gambles (MGA) task and two probability discounting tasks for gains and losses, respectively (PDG/PDL). Afterwards, a subsample of 105 participants (44 with S/S, 6 with S/L, 55 with L/L genotype) completed the pharmacological study using a crossover design with tryptophan depletion (ATD), loading (ATL), and balanced (BAL) conditions. The same decision constructs were assessed.

RESULTS

We found increased risk-seeking for losses in S/S compared to L/L individuals at the first visit (p = 0.002). Neither tryptophan depletion nor loading affected decision-making, nor did we observe an interaction between intervention and 5-HTTLPR genotype.

CONCLUSION

Our data do not support the idea that transient changes of tonic 5-HT affect value-based decision-making. We provide evidence for an association of 5-HTTLPR with risk-seeking for losses, independent of acute 5-HT levels. This indicates that the association of 5-HTTLPR and risk-seeking for losses is mediated via other mechanisms, possibly by differences in the structural development of neural circuits of the 5-HT system during early life phases.

Psilocybin and MDMA reduce costly punishment in the Ultimatum Game

Disruptions in social decision-making are becoming evident in many psychiatric conditions. These are studied using paradigms investigating the psychological mechanisms underlying interpersonal interactions, such as the Ultimatum Game (UG). Rejection behaviour in the UG represents altruistic punishment – the costly punishment of norm violators – but the mechanisms underlying it require clarification. To investigate the psychopharmacology of UG behaviour, we carried out two studies with healthy participants, employing serotonergic agonists: psilocybin (open-label, within-participant design, N = 19) and 3,4-methylenedioxymethamphetamine (MDMA; placebo-controlled, double-blind, crossover design, N = 20). We found that both MDMA and psilocybin reduced rejection of unfair offers (odds ratio: 0.57 and 0.42, respectively). The reduction in rejection rate following MDMA was associated with increased prosociality (R² = 0.26, p = 0.025). In the MDMA study, we investigated third-party decision-making and proposer behaviour. MDMA did not reduce rejection in the third-party condition, but produced an increase in the amount offered to others (Cohen’s d = 0.82). We argue that these compounds altered participants’ conceptualisation of ‘social reward’, placing more emphasis on the direct relationship with interacting partners. With these compounds showing efficacy in drug-assisted psychotherapy, these studies are an important step in the further characterisation of their psychological effects.

Sex differences in risk-based decision making in adolescents with conduct disorder

Altered decision making processes and excessive risk-seeking behaviours are key features of conduct disorder (CD). Previous studies have provided compelling evidence of abnormally increased preference for risky options, higher sensitivity to rewards, as well as blunted responsiveness to aversive outcomes in adolescents with CD. However, most studies published to date have focused on males only; thus, it is not known whether females with CD show similar alterations in decision making. The current study investigated potential sex differences in decision making and risk-seeking behaviours in adolescents with CD. Forty-nine adolescents with CD (23 females) and 51 control subjects (27 females), aged 11-18 years, performed a computerised task assessing decision making under risk-the Risky Choice Task. Participants made a series of decisions between two gamble options that varied in terms of their expected values and probability of gains and losses. This enabled the participants' risk preferences to be determined. Taking the sample as a whole, adolescents with CD exhibited increased risk-seeking behaviours compared to healthy controls. However, we found a trend towards a sex-by-group interaction, suggesting that these effects may vary by sex. Follow-up analyses showed that males with CD made significantly more risky choices than their typically developing counterparts, while females with CD did not differ from typically developing females in their risk-seeking behaviours. Our results provide preliminary evidence that sex may moderate the relationship between CD and alterations in risk attitudes and reward processing, indicating that there may be sex differences in the developmental pathways and neuropsychological deficits that lead to CD.

Shared and Disorder-Specific Neurocomputational Mechanisms of Decision-Making in Autism Spectrum Disorder and Obsessive-Compulsive Disorder

Autism spectrum disorder (ASD) and obsessive-compulsive disorder (OCD) often share phenotypes of repetitive behaviors, possibly underpinned by abnormal decision-making. To compare neural correlates underlying decision-making between these disorders, brain activation of boys with ASD (N = 24), OCD (N = 20) and typically developing controls (N = 20) during gambling was compared, and computational modeling compared performance. Patients were unimpaired on number of risky decisions, but modeling showed that both patient groups had lower choice consistency and relied less on reinforcement learning compared to controls. ASD individuals had disorder-specific choice perseverance abnormalities compared to OCD individuals. Neurofunctionally, ASD and OCD boys shared dorsolateral/inferior frontal underactivation compared to controls during decision-making. During outcome anticipation, patients shared underactivation compared to controls in lateral inferior/orbitofrontal cortex and ventral striatum. During reward receipt, ASD boys had disorder-specific enhanced activation in inferior frontal/insular regions relative to OCD boys and controls. Results showed that ASD and OCD individuals shared decision-making strategies that differed from controls to achieve comparable performance to controls. Patients showed shared abnormalities in lateral-(orbito)fronto-striatal reward circuitry, but ASD boys had disorder-specific lateral inferior frontal/insular overactivation, suggesting that shared and disorder-specific mechanisms underpin decision-making in these disorders. Findings provide evidence for shared neurobiological substrates that could serve as possible future biomarkers.

Comparison of neural substrates of temporal discounting between youth with autism spectrum disorder and with obsessive-compulsive disorder

BACKGROUND

Autism spectrum disorder (ASD) and obsessive-compulsive disorder (OCD) share abnormalities in hot executive functions such as reward-based decision-making, as measured in the temporal discounting task (TD). No studies, however, have directly compared these disorders to investigate common/distinct neural profiles underlying such abnormalities. We wanted to test whether reward-based decision-making is a shared transdiagnostic feature of both disorders with similar neurofunctional substrates or whether it is a shared phenotype with disorder-differential neurofunctional underpinnings.

METHODS

Age and IQ-matched boys with ASD (N = 20), with OCD (N = 20) and 20 healthy controls, performed an individually-adjusted functional magnetic resonance imaging (fMRI) TD task. Brain activation and performance were compared between groups.

RESULTS

Boys with ASD showed greater choice-impulsivity than OCD and control boys. Whole-brain between-group comparison revealed shared reductions in ASD and OCD relative to control boys for delayed-immediate choices in right ventromedial/lateral orbitofrontal cortex extending into medial/inferior prefrontal cortex, and in cerebellum, posterior cingulate and precuneus. For immediate-delayed choices, patients relative to controls showed reduced activation in anterior cingulate/ventromedial prefrontal cortex reaching into left caudate, which, at a trend level, was more decreased in ASD than OCD patients, and in bilateral temporal and inferior parietal regions.

CONCLUSIONS

This first fMRI comparison between youth with ASD and with OCD, using a reward-based decision-making task, shows predominantly shared neurofunctional abnormalities during TD in key ventromedial, orbital- and inferior fronto-striatal, temporo-parietal and cerebellar regions of temporal foresight and reward processing, suggesting trans-diagnostic neurofunctional deficits.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

The role of serotonin in personality inference: tryptophan depletion impairs the identification of neuroticism in the face

Serotonergic mechanisms mediate the expression of personality traits (such as impulsivity, aggression and anxiety) that are linked to vulnerability to psychological illnesses, and modulate the identification of emotional expressions in the face as well as learning about broader classes of appetitive and aversive signals. Faces with neutral expressions signal a variety of socially relevant information, such that inferences about the big five personality traits, including Neuroticism, Extraversion and Agreeableness, can be accurately made on the basis of emotionally neutral facial photographs. Given the close link between Neuroticism and psychological distress, we investigated the effects of diminished central serotonin activity (achieved by tryptophan depletion) upon the accuracy of 52 healthy (non-clinical) adults' discriminations of personality from facial characteristics. All participants were able to discriminate reliably four of the big five traits. However, the tryptophan-depleted participants were specifically less accurate in discriminating Neuroticism than the matched non-depleted participants. These data suggest that central serotonin activity modulates the identification of not only negative facial emotional expression but also a broader class of signals about personality characteristics linked to psychological distress.

Dissociable effects of basolateral amygdala lesions on decision making biases in rats when loss or gain is emphasized

Individuals switch from risk seeking to risk aversion when mathematically identical options are described in terms of loss versus gains, as exemplified in the reflection and framing effects. Determining the neurobiology underlying such cognitive biases could inform our understanding of decision making in health and disease. Although reports vary, data using human subjects have implicated the amygdala in such biases. Animal models enable more detailed investigation of neurobiological mechanisms. We therefore tested whether basolateral amygdala (BLA) lesions would affect risk preference for gains or losses in rats. Choices in both paradigms were always between options of equal expected value-a guaranteed outcome, or the 50:50 chance of double or nothing. In the loss-chasing task, most rats exhibited strong risk seeking preferences, gambling at the risk of incurring double the penalty, regardless of the size of the guaranteed loss. In the betting task, the majority of animals were equivocal in their choice, irrespective of bet size; however, a wager-sensitive subgroup progressively shifted away from the uncertain option as the bet size increased, which is reminiscent of risk aversion. BLA lesions increased preference for the smaller guaranteed loss in the loss-chasing task, without affecting choice on the betting task, which is indicative of reduced risk seeking for losses, but intact risk aversion for gains. These data support the hypothesis that the amygdala plays a more prominent role in choice biases related to losses. Given the importance of the amygdala in representing negative affect, the aversive emotional reaction to loss, rather than aberrant estimations of probability or loss magnitude, may underlie risk seeking for losses.

A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making

There is significant evidence that in addition to reward-punishment based decision making, the Basal Ganglia (BG) contributes to risk-based decision making (Balasubramani et al., 2014). Despite this evidence, little is known about the computational principles and neural correlates of risk computation in this subcortical system. We have previously proposed a reinforcement learning (RL)-based model of the BG that simulates the interactions between dopamine (DA) and serotonin (5HT) in a diverse set of experimental studies including reward, punishment and risk based decision making (Balasubramani et al., 2014). Starting with the classical idea that the activity of mesencephalic DA represents reward prediction error, the model posits that serotoninergic activity in the striatum controls risk-prediction error. Our prior model of the BG was an abstract model that did not incorporate anatomical and cellular-level data. In this work, we expand the earlier model into a detailed network model of the BG and demonstrate the joint contributions of DA-5HT in risk and reward-punishment sensitivity. At the core of the proposed network model is the following insight regarding cellular correlates of value and risk computation. Just as DA D1 receptor (D1R) expressing medium spiny neurons (MSNs) of the striatum were thought to be the neural substrates for value computation, we propose that DA D1R and D2R co-expressing MSNs are capable of computing risk. Though the existence of MSNs that co-express D1R and D2R are reported by various experimental studies, prior existing computational models did not include them. Ours is the first model that accounts for the computational possibilities of these co-expressing D1R-D2R MSNs, and describes how DA and 5HT mediate activity in these classes of neurons (D1R-, D2R-, D1R-D2R- MSNs). Starting from the assumption that 5HT modulates all MSNs, our study predicts significant modulatory effects of 5HT on D2R and co-expressing D1R-D2R MSNs which in turn explains the multifarious functions of 5HT in the BG. The experiments simulated in the present study relates 5HT to risk sensitivity and reward-punishment learning. Furthermore, our model is shown to capture reward-punishment and risk based decision making impairment in Parkinson's Disease (PD). The model predicts that optimizing 5HT levels along with DA medications might be essential for improving the patients' reward-punishment learning deficits.

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

Serotonin 2A receptors contribute to the regulation of risk-averse decisions

Pharmacological studies point to a role of the neurotransmitter serotonin (5-HT) in regulating the preference for risky decisions, yet the functional contribution of specific 5-HT receptors remains to be clarified. We used pharmacological fMRI to investigate the role of the 5-HT2A receptors in processing negative outcomes and regulating risk-averse behavior. During fMRI, twenty healthy volunteers performed a gambling task under two conditions: with or without blocking the 5-HT2A receptors. The volunteers repeatedly chose between small, likely rewards and large, unlikely rewards. Choices were balanced in terms of expected utility and potential loss. Acute blockade of the 5-HT2A receptors with ketanserin made participants more risk-averse. Ketanserin selectively reduced the neural response of the frontopolar cortex to negative outcomes that were caused by low-risk choices and were associated with large missed rewards. In the context of normal 5-HT2A receptor function, ventral striatum displayed a stronger response to low-risk negative outcomes in risk-taking as opposed to risk-averse individuals. This (negative) correlation between the striatal response to low-risk negative outcomes and risk-averse choice behavior was abolished by 5-HT2A receptor blockade. The results provide the first evidence for a critical role of 5-HT2A receptor function in regulating risk-averse behavior. We suggest that the 5-HT2A receptor system facilitates risk-taking behavior by modulating the outcome evaluation of "missed" reward. These results have implications for understanding the neural basis of abnormal risk-taking behavior, for instance in pathological gamblers.

Short-term quetiapine treatment alters the use of reinforcement signals during risky decision-making and promotes the choice of negative expected values in healthy adult males

Effective decision-making can involve using environmental signals about the possible good and bad outcomes, and their probabilities, to select optimal actions. Problematic decision-making in psychiatric disorders, and particularly bipolar illness, may result from disrupted use of these reinforcement cues, leading to actions that reflect or precipitate pathological changes in mood. Previous experiments indicate that the processing of reinforcement cues while selecting between risky actions can be influenced by dopamine and serotonin activity. Quetiapine is an atypical antipsychotic agent with a complex pharmacology, including antagonist actions at 5-HT2A and, to a lesser extent, D2 receptors. Here, we investigated the effects of (short-term) treatment with quetiapine on the risky decision-making of healthy human adults. Twenty participants received 150 mg of quetiapine XL for 7 d, whereas 20 age- and IQ-matched participants received a placebo. On the eighth day, all participants completed a risky decision-making task that involved making a series of choices between two simultaneously presented gambles that differed in the magnitudes of their possible gains and losses, and the probabilities with which these outcomes were delivered. Quetiapine treatment was associated with a marked tendency to choose options with negative expected values compared with placebo treatment in male but not female participants. Our results demonstrate that antagonism of serotonin and dopamine receptor activity can alter the way individuals use information about gains and losses when selecting between risky actions, possibly reflecting gender-specific differences in risk attitudes. These effects may be beneficial by correcting decision-making biases that feature in mood disorders.

A dynamic, embodied paradigm to investigate the role of serotonin in decision-making

Serotonin (5-HT) is a neuromodulator that has been attributed to cost assessment and harm aversion. In this review, we look at the role 5-HT plays in making decisions when subjects are faced with potential harmful or costly outcomes. We review approaches for examining the serotonergic system in decision-making. We introduce our group’s paradigm used to investigate how 5-HT affects decision-making. In particular, our paradigm combines techniques from computational neuroscience, socioeconomic game theory, human–robot interaction, and Bayesian statistics. We will highlight key findings from our previous studies utilizing this paradigm, which helped expand our understanding of 5-HT’s effect on decision-making in relation to cost assessment. Lastly, we propose a cyclic multidisciplinary approach that may aid in addressing the complexity of exploring 5-HT and decision-making by iteratively updating our assumptions and models of the serotonergic system through exhaustive experimentation.

Dopamine modulates risk-taking as a function of baseline sensation-seeking trait

Trait sensation-seeking, defined as a need for varied, complex, and intense sensations, represents a relatively underexplored hedonic drive in human behavioral neuroscience research. It is related to increased risk for a range of behaviors including substance use, gambling, and risky sexual practice. Individual differences in self-reported sensation-seeking have been linked to brain dopamine function, particularly at D2-like receptors, but so far no causal evidence exists for a role of dopamine in sensation-seeking behavior in humans. Here, we investigated the effects of the selective D2/D3 agonist cabergoline on performance of a probabilistic risky choice task in healthy humans using a sensitive within-subject, placebo-controlled design. Cabergoline significantly influenced the way participants combined different explicit signals regarding probability and loss when choosing between response options associated with uncertain outcomes. Importantly, these effects were strongly dependent on baseline sensation-seeking score. Overall, cabergoline increased sensitivity of choice to information about probability of winning; while decreasing discrimination according to magnitude of potential losses associated with different options. The largest effects of the drug were observed in participants with lower sensation-seeking scores. These findings provide evidence that risk-taking behavior in humans can be directly manipulated by a dopaminergic drug, but that the effectiveness of such a manipulation depends on baseline differences in sensation-seeking trait. This emphasizes the importance of considering individual differences when investigating manipulation of risky decision-making, and may have relevance for the development of pharmacotherapies for disorders involving excessive risk-taking in humans, such as pathological gambling.

Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience

RATIONALE

We have previously shown that tryptophan depletion enhances punishment but not reward prediction (Cools et al. in Neuropsychopharmacology 33:2291-2299, 2008b). This provided evidence for a valence-specific role of serotonin (which declines under depleted tryptophan) in aversive processing. Recent theoretical (Dayan and Huys in PLoS Comput Biol 4:e4, 2008) and experimental (Crockett et al. in J Neurosci 29:11993-11999, 2009) approaches have, however, further specified this role by showing that serotonin is critical for punishment-induced inhibition.

OBJECTIVES

We sought to examine the role of serotonin in punishment-induced inhibition. We also examined the impact of induced mood on this effect to assess whether effects of tryptophan depletion on affective inhibition are moderated by mood.

METHODS

Healthy females consumed a balanced amino acid mixture with (N = 20) or without (N = 21) the serotonin precursor tryptophan. Each subject completed either negative or neutral mood induction. All subjects completed the reward and punishment reversal learning task adopted in the previous study.

RESULTS

We demonstrate a punishment prediction impairment in individuals who consumed tryptophan which was absent in individuals who were depleted of tryptophan. This effect was impervious to mood state.

CONCLUSIONS

Our results suggest that serotonin promotes the inhibition of responses to punishing outcomes. This may lead to reduced punishment prediction accuracy in the presence of tryptophan and may contribute to resilience to affective disorders. Reduction of serotonin via tryptophan depletion then removes this inhibition. As such, we highlight a mechanism by which reduced serotonin can contribute to disorders of impulsivity and compulsivity as well as disorders of emotion.

Neurogenetics and Epigenetics in Impulsive Behaviour: Impact on Reward Circuitry

Adverse, unfavourable life conditions, particularly during early life stages and infancy, can lead to epigenetic regulation of genes involved in stress-response, behavioral disinhibition, and cognitive-emotional systems. Over time, the ultimate final outcome can be expressed through behaviors bedeviled by problems with impulse control, such as eating disorders, alcoholism, and indiscriminate social behavior. While many reward gene polymorphisms are involved in impulsive behaviors, a polymorphism by itself may not translate to the development of a particular behavioral disorder unless it is impacted by epigenetic effects. Brain-derived neurotrophic factor (BDNF) affects the development and integrity of the noradrenergic, dopaminergic, serotonergic, glutamatergic, and cholinergic neurotransmitter systems, and plasma levels of the neurotrophin are associated with both cognitive and aggressive impulsiveness. Epigenetic mechanisms associated with a multitude of environmental factors, including premature birth, low birth weight, prenatal tobacco exposure, non-intact family, young maternal age at birth of the target child, paternal history of antisocial behavior, and maternal depression, alter the developmental trajectories for several neuropsychiatric disorders. These mechanisms affect brain development and integrity at several levels that determine structure and function in resolving the final behavioral expressions.

Rapid decision-making under risk

Impulsivity is often characterized by rapid decisions under risk, but most current tests of decision-making do not impose time pressures on participants' choices. Here we introduce a new traffic lights test which requires people to choose whether to program a risky, early eye movement before a traffic light turns green (earning them high rewards or a penalty) or wait for the green light before responding to obtain a small reward instead. Young participants demonstrated bimodal responses: an early, high-risk and a later, low-risk set of choices. By contrast, elderly people invariably waited for the green light and showed little risk-taking. Performance could be modeled as a race between two rise-to-threshold decision processes, one triggered by the green light and the other initiated before it. The test provides a useful measure of rapid decision-making under risk, with the potential to reveal how this process alters with aging or in patient groups.

Epigenetics in Developmental Disorder: ADHD and Endophenotypes

Heterogeneity in attention-deficit/hyperactivity disorder (ADHD), with complex interactive operations of genetic and environmental factors, is expressed in a variety of disorder manifestations: severity, co-morbidities of symptoms, and the effects of genes on phenotypes. Neurodevelopmental influences of genomic imprinting have set the stage for the structural-physiological variations that modulate the cognitive, affective, and pathophysiological domains of ADHD. The relative contributions of genetic and environmental factors provide rapidly proliferating insights into the developmental trajectory of the condition, both structurally and functionally. Parent-of-origin effects seem to support the notion that genetic risks for disease process debut often interact with the social environment, i.e., the parental environment in infants and young children. The notion of endophenotypes, markers of an underlying liability to the disorder, may facilitate detection of genetic risks relative to a complex clinical disorder. Simple genetic association has proven insufficient to explain the spectrum of ADHD. At a primary level of analysis, the consideration of epigenetic regulation of brain signalling mechanisms, dopamine, serotonin, and noradrenaline is examined. Neurotrophic factors that participate in the neurogenesis, survival, and functional maintenance of brain systems, are involved in neuroplasticity alterations underlying brain disorders, and are implicated in the genetic predisposition to ADHD, but not obviously, nor in a simple or straightforward fashion. In the context of intervention, genetic linkage studies of ADHD pharmacological intervention have demonstrated that associations have fitted the "drug response phenotype," rather than the disorder diagnosis. Despite conflicting evidence for the existence, or not, of genetic associations between disorder diagnosis and genes regulating the structure and function of neurotransmitters and brain-derived neurotrophic factor (BDNF), associations between symptoms-profiles endophenotypes and single nucleotide polymorphisms appear reassuring.

Effect of hydrolysed egg protein on brain tryptophan availability

Serotonin synthesis critically depends on plasma levels of tryptophan (TRP). Earlier studies have shown that for mood and cognitive benefits to occur, the ratio between TRP and other large neutral amino acids (LNAA) has to be increased by approximately 40 %. The present study investigated the dose-dependent effects of a TRP-rich hydrolysed protein (egg-protein hydrolysate, EPH) on the plasma TRP:LNAA. Moreover, it was investigated whether EPH could increase TRP:LNAA in the presence of 2 g of milk protein (MP). In a randomised double-blind crossover design, plasma amino acids were measured every 30 min for 3·5 h after ingestion of a drink containing either three different doses of 4, 8 and 12 g EPH containing 270, 560 or 800 mg of TRP, respectively, the combination of 4 g EPH and 2 g MP (74 mg TRP), or 4 g MP (148 mg TRP) in twenty healthy subjects with a mean age of 52 years. All three EPH doses caused significant increases of TRP:LNAA above 40 % at 30, 60 and 90 min after consumption in a dose-dependent manner. Compared with the 4 g EPH, the increase in TRP:LNAA in the 4 g EPH with 2 g MP condition was significantly lower at 60 min (63 v. 44 %, P < 0·001) and did not differ significantly at 90 min (58 v. 53 %, P>0·05). The present study showed that a low dose of 4 g EPH with even the addition of 2 g MP was sufficient to increase the ratio of TRP:LNAA above 40 %. Thus, EPH offers a viable ingredient to increase TRP availability.

Serotonin and dopamine: unifying affective, activational, and decision functions

Serotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcement learning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance, compared with DA, we have a much less specific understanding about the mechanisms by which serotonin affects behavior. Here, we draw on recent work on computational models of dopaminergic function to suggest a framework by which many of the seemingly diverse functions associated with both DA and serotonin-comprising both affective and activational ones, as well as a number of other functions not overtly related to either-can be seen as consequences of a single root mechanism.

The roles of dopamine and serotonin in decision making: evidence from pharmacological experiments in humans

Neurophysiological experiments in primates, alongside neuropsychological and functional magnetic resonance investigations in humans, have significantly enhanced our understanding of the neural architecture of decision making. In this review, I consider the more limited database of experiments that have investigated how dopamine and serotonin activity influences the choices of human adults. These include those experiments that have involved the administration of drugs to healthy controls, experiments that have tested genotypic influences upon dopamine and serotonin function, and, finally, some of those experiments that have examined the effects of drugs on the decision making of clinical samples. Pharmacological experiments in humans are few in number and face considerable methodological challenges in terms of drug specificity, uncertainties about pre- vs post-synaptic modes of action, and interactions with baseline cognitive performance. However, the available data are broadly consistent with current computational models of dopamine function in decision making and highlight the dissociable roles of dopamine receptor systems in the learning about outcomes that underpins value-based decision making. Moreover, genotypic influences on (interacting) prefrontal and striatal dopamine activity are associated with changes in choice behavior that might be relevant to understanding exploratory behaviors and vulnerability to addictive disorders. Manipulations of serotonin in laboratory tests of decision making in human participants have provided less consistent results, but the information gathered to date indicates a role for serotonin in learning about bad decision outcomes, non-normative aspects of risk-seeking behavior, and social choices involving affiliation and notions of fairness. Finally, I suggest that the role played by serotonin in the regulation of cognitive biases, and representation of context in learning, point toward a role in the cortically mediated cognitive appraisal of reinforcers when selecting between actions, potentially accounting for its influence upon the processing salient aversive outcomes and social choice.

Serotonin and dopamine play complementary roles in gambling to recover losses

Continued gambling to recover losses--'loss chasing'--is a prominent feature of social and pathological gambling. However, little is known about the neuromodulators that influence this behavior. In three separate experiments, we investigated the role of serotonin activity, D(2)/D(3) receptor activity, and beta-adrenoceptor activity on the loss chasing of age and IQ-matched healthy adults randomized to treatment or an appropriate control/placebo. In Experiment 1, participants consumed amino-acid drinks that did or did not contain the serotonin precursor, tryptophan. In Experiment 2, participants received a single 176 μg dose of the D(2)/D(3) receptor agonist, pramipexole, or placebo. In Experiment 3, participants received a single 80 mg dose of the beta-adrenoceptor blocker, propranolol, or placebo. Following treatment, participants completed a computerized loss-chasing game. Mood and heart rate were measured at baseline and following treatment. Tryptophan depletion significantly reduced the number of decisions made to chase losses, and the number of consecutive decisions to chase, in the absence of marked changes in mood. By contrast, pramipexole significantly increased the value of losses chased and diminished the value of losses surrendered. Propranolol markedly reduced heart rate, but produced no significant changes in loss-chasing behavior. Loss chasing can be thought of as an aversively motivated escape behavior controlled, in part, by the marginal value of continued gambling relative to the value of already accumulated losses. Serotonin and dopamine appear to play dissociable roles in the tendency of individuals to gamble to recover, or to seek to 'escape' from, previous losses. Serotonergic activity seems to promote the availability of loss chasing as a behavioral option, whereas D(2)/D(3) receptor activity produces complex changes in the value of losses judged worth chasing. Sympathetic arousal, at least as mediated by beta-adrenoceptors, does not play a major role in laboratory-based loss-chasing choices.

Risk-taking behavior in a gambling task associated with variations in the tryptophan hydroxylase 2 gene: relevance to psychiatric disorders

Decision making, choosing the best option from the possible outcomes, is impaired in many psychiatric conditions including affective disorders. We tested the hypothesis that variations in serotonergic genes (TPH2, TPH1, SLC6A4, HTR1A), which influence serotonin availability, affect choice behavior in a probabilistic gambling task. A population cohort (N=1035) completed a paper-and-pencil gambling task, filled out personality and symptom questionnaires and gave consent for the use of their DNA in a genetic association study. A subgroup of subjects (N=69) also completed a computer version of the task. The gambling task was designed to estimate an individual's tendency to take a risk when choosing between a smaller but more certain 'win' and a larger, less probable one. We genotyped seven haplotype tagging SNPs in the TPH2 gene, and previously reported functional polymorphisms from the other genes (rs1800532, 5HTTLPR, and rs6295). Carriers of the more prevalent TPH2 haplotype, which was previously associated with less active enzyme variant, showed reduced risk taking on both tasks compared with subjects not carrying the common haplotype. The effect of TPH2 haplotypes on risk-taking was independent of current depression and anxiety symptoms, neuroticism and impulsiveness scores. We did not find an association between functional polymorphisms in the TPH1, SLC6A4, HTR1A genes and risk-taking behavior. In conclusion, our study demonstrates the role of the TPH2 gene and the serotonin system in risk taking and suggests that TPH2 gene may contribute to the expression of psychiatric phenotypes through altered decision making.