Serotonin shapes risky decision making in monkeys

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Role of serotonin in modulation of decision-making in Parkinson's disease

BACKGROUND

Dysfunction of dopaminergic pathways has been considered to play a pivotal role in Parkinson's disease (PD), affecting the processing of emotional and rewarding information, and potentially leading to symptoms of depression or apathy. However, some aspects of motivation in PD might be affected by non-dopaminergic mechanisms.

AIM AND METHOD

The objective of this experimental medicine study was to investigate the contribution of serotonergic modulation via administration of citalopram (20 mg) for 7 days on motivated decision-making in twenty PD patients, measured using several different computerised tasks and clinical questionnaires that probe different aspects of decision-making. Twenty healthy controls were additionally tested without medication to assess any baseline differences between the two groups.

RESULTS

Results indicated that PD patients were overall less motivated than controls on an effort- and reward-based decision-making task. Citalopram increased or decreased willingness to exert effort for reward, depending on whether baseline motivation was high or low, respectively. A task assessing decision-making under risk revealed higher levels of risk aversion for potential losses in PD patients, which neither serotonin nor the patient's regular dopaminergic medication seemed to restore. However, citalopram in PD was associated with more risk-seeking choices for gains, although patients and controls did not differ on this at baseline.

CONCLUSION

The results provide evidence for a role of the serotonergic system in influencing some aspects of motivated decision-making in PD processes.

The calcium-sensing receptor (CaSR) regulates zebrafish sensorimotor decision making via a genetically defined cluster of hindbrain neurons

Decision making is a fundamental nervous system function that ranges widely in complexity and speed of execution. We previously established larval zebrafish as a model for sensorimotor decision making and identified the G-protein-coupled calcium-sensing receptor (CaSR) to be critical for this process. Here, we report that CaSR functions in neurons to dynamically regulate the bias between two behavioral outcomes: escapes and reorientations. By employing a computational guided transgenic strategy, we identify a genetically defined neuronal cluster in the hindbrain as a key candidate site for CaSR function. Finally, we demonstrate that transgenic CaSR expression targeting this cluster consisting of a few hundred neurons shifts behavioral bias in wild-type animals and restores decision making deficits in CaSR mutants. Combined, our data provide a rare example of a G-protein-coupled receptor that biases vertebrate sensorimotor decision making via a defined neuronal cluster.

The neural substrates of risky rewards and losses in healthy volunteers and patient groups: a PET imaging study

BACKGROUND

Risk is an essential trait of most daily decisions. Our behaviour when faced with risks involves evaluation of many factors including the outcome probabilities, the valence (gains or losses) and past experiences. Several psychiatric disorders belonging to distinct diagnostic categories, including pathological gambling and addiction, show pathological risk-taking and implicate abnormal dopaminergic, opioidergic and serotonergic neurotransmission. In this study, we adopted a transdiagnostic approach to delineate the neurochemical substrates of decision making under risk.

METHODS

We recruited 39 participants, including 17 healthy controls, 15 patients with pathological gambling and seven binge eating disorder patients, who completed an anticipatory risk-taking task. Separately, participants underwent positron emission tomography (PET) imaging with three ligands, [¹⁸F]fluorodopa (FDOPA), [¹¹C]MADAM and [¹¹C]carfentanil to assess presynaptic dopamine synthesis capacity and serotonin transporter and mu-opioid receptor binding respectively.

RESULTS

Risk-taking behaviour when faced with gains positively correlated with dorsal cingulate [¹¹C]carfentanil binding and risk-taking to losses positively correlated with [¹¹C]MADAM binding in the caudate and putamen across all subjects.

CONCLUSIONS

We show distinct neurochemical substrates underlying risk-taking with the dorsal cingulate cortex mu-opioid receptor binding associated with rewards and dorsal striatal serotonin transporter binding associated with losses. Risk-taking and goal-directed control appear to dissociate between dorsal and ventral fronto-striatal systems. Our findings thus highlight the potential role of pharmacological agents or neuromodulation on modifying valence-specific risk-taking biases.

To learn or to gain: neural signatures of exploration in human decision-making

Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.

Selective effects of serotonin on choices to gather more information

BACKGROUND

Gathering and evaluating information leads to better decisions, but often at cost. The balance between information seeking and exploitation features in neurodevelopmental, mood, psychotic and substance-related disorders. Serotonin's role has been highlighted by experimental reduction of its precursor, tryptophan.

AIMS

We tested the boundaries and applicability of this role by asking whether changes to information sampling would be observed following acute doses of serotonergic and catecholaminergic clinical treatments. We used a variant of the Information Sampling Task (IST) to measure how much information a person requires before they make a decision. This task allows participants to sample information until satisfied to make a choice.

METHODS

In separate double-blind placebo-controlled experiments, we tested 27 healthy participants on/off 20 mg of the serotonin reuptake inhibitor (SRI) citalopram, and 22 participants on/off 40 mg of the noradrenergic reuptake inhibitor atomoxetine. The IST variant minimised effects of temporal impulsivity and loss aversion. Analyses used a variety of participant prior expectations of sampling spaces in the IST, including a new prior that accounts for learning of likely states across trials. We analysed behaviour by a new method that also accounts for baseline individual differences of risk preference.

RESULTS

Baseline preferences demonstrated risk aversion. Citalopram decreased the expected utility of choices and probability of being correct based on informational content of samples collected, suggesting participants collected less useful information before making a choice. Atomoxetine did not influence information seeking.

CONCLUSION

Acute changes of serotonin activity by way of a single SRI dose alter information-seeking behaviour.

Platelet MAO activity and COMT Val158Met genotype interaction predicts visual working memory updating efficiency

The exact mechanism how serotonergic and dopaminergic systems relate to one another in working memory (WM) updating is unknown. Platelet monoamine oxidase (MAO) has been used as a marker for central serotonergic capacity, and catechol-O-methyltransferase (COMT) as a marker for central dopaminergic capacity. This study aimed to describe the interaction of platelet MAO activity and COMT Val158Met genotype in visual working memory updating: the ability to replace old information with new within hundreds of milliseconds. Previous studies suggest that platelet MAO activity and COMT Val158Met genotype could have an interaction effect on working memory. However, there are no studies that have directly examined the interaction of these biomarkers in WM updating. We used a 2-back updating task with facial expressions and defined updating efficiency as response times for correct responses. 455 subjects from a population representative sample were included. Mixed models were used for data analysis with an aim to study the interaction of COMT Val158Met genotype (Val/Val, Val/Met and Met/Met) and the level of MAO activity (high vs low). Education, IQ, sex, simple reaction times, and overall updating accuracy were included as covariates. We found that the effect of COMT Val158Met on updating efficiency depends on the level of platelet MAO activity. Low MAO in contrast to high MAO was associated with an increase in updating efficiency in Val/Met but a decrease in Met/Met. The results are discussed in the context of serotonin and dopamine functions in brain regions related to WM. The findings support the view that serotonin modulates dopaminergic activation in updating and contribute to understanding the role of serotonin in PFC, top-down inhibitory signals, and its interactions with dopamine in WM processes.

The description-experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

The experimental investigation of decision-making in humans relies on two distinct types of paradigms, involving either description- or experience-based choices. In description-based paradigms, decision variables (i.e. payoffs and probabilities) are explicitly communicated by means of symbols. In experience-based paradigms decision variables are learnt from trial-by-trial feedback. In the decision-making literature, 'description-experience gap' refers to the fact that different biases are observed in the two experimental paradigms. Remarkably, well-documented biases of description-based choices, such as under-weighting of rare events and loss aversion, do not apply to experience-based decisions. Here, we argue that the description-experience gap represents a major challenge, not only to current decision theories, but also to the neuroeconomics research framework, which relies heavily on the translation of neurophysiological findings between human and non-human primate research. In fact, most non-human primate neurophysiological research relies on behavioural designs that share features of both description- and experience-based choices. As a consequence, it is unclear whether the neural mechanisms built from non-human primate electrophysiology should be linked to description-based or experience-based decision-making processes. The picture is further complicated by additional methodological gaps between human and non-human primate neuroscience research. After analysing these methodological challenges, we conclude proposing new lines of research to address them. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Variation in primate decision-making under uncertainty and the roots of human economic behaviour

Uncertainty is a ubiquitous component of human economic behaviour, yet people can vary in their preferences for risk across populations, individuals and different points in time. As uncertainty also characterizes many aspects of animal decision-making, comparative research can help evaluate different potential mechanisms that generate this variation, including the role of biological differences or maturational change versus cultural learning, as well as identify human-unique components of economic decision-making. Here, we examine decision-making under risk across non-human primates, our closest relatives. We first review theoretical approaches and current methods for understanding decision-making in animals. We then assess the current evidence for variation in animal preferences between species and populations, between individuals based on personality, sex and age, and finally, between different contexts and individual states. We then use these primate data to evaluate the processes that can shape human decision-making strategies and identify the primate foundations of human economic behaviour. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Impulsive decisions are those that favor immediate over delayed rewards, involve the acceptance of undue risk or uncertainty, or fail to adapt to environmental changes. Pathological levels of impulsive decision-making have been observed in individuals with mental illness, but there may be substantial heterogeneity in the processes that drive impulsive choices. Understanding this behavioral heterogeneity may be critical for understanding associated diverseness in the neural mechanisms that give rise to impulsivity. The application of reinforcement learning algorithms in the deconstruction of impulsive decision-making phenotypes can help bridge the gap between biology and behavior and provide insights into the biobehavioral heterogeneity of impulsive choice. This chapter will review the literature on the neurobiological mechanisms of impulsive decision-making in nonhuman animals; specifically, the role of the amine neuromodulatory systems (dopamine, serotonin, norepinephrine, and acetylcholine) in impulsive decision-making and reinforcement learning processes is discussed. Ultimately, the integration of reinforcement learning algorithms with sophisticated behavioral and neuroscience techniques may be critical for advancing the understanding of the neurochemical basis of impulsive decision-making.

Bipolar oscillations between positive and negative mood states in a computational model of Basal Ganglia

Bipolar disorder is characterized by mood swings-oscillations between manic and depressive states. The swings (oscillations) mark the length of an episode in a patient's mood cycle (period), and can vary from hours to years. The proposed modeling study uses decision making framework to investigate the role of basal ganglia network in generating bipolar oscillations. In this model, the basal ganglia system performs a two-arm bandit task in which one of the arms (action responses) leads to a positive outcome, while the other leads to a negative outcome. We explore the dynamics of key reward and risk related parameters in the system while the model agent receives various outcomes. Particularly, we study the system using a model that represents the fast dynamics of decision making, and a module to capture the slow dynamics that describe the variation of some meta-parameters of fast dynamics over long time scales. The model is cast at three levels of abstraction: (1) a two-dimensional dynamical system model, that is a simple two variable model capable of showing bistability for rewarding and punitive outcomes; (2) a phenomenological basal ganglia model, to extend the implications from the reduced model to a cortico-basal ganglia setup; (3) a detailed network model of basal ganglia, that incorporates detailed cellular level models for a more realistic understanding. In healthy conditions, the model chooses positive action and avoids negative one, whereas under bipolar conditions, the model exhibits slow oscillations in its choice of positive or negative outcomes, reminiscent of bipolar oscillations. Phase-plane analyses on the simple reduced dynamical system with two variables reveal the essential parameters that generate pathological 'bipolar-like' oscillations. Phenomenological and network models of the basal ganglia extend that logic, and interpret bipolar oscillations in terms of the activity of dopaminergic and serotonergic projections on the cortico-basal ganglia network dynamics. The network's dysfunction, specifically in terms of reward and risk sensitivity, is shown to be responsible for the pathological bipolar oscillations. The study proposes a computational model that explores the effects of impaired serotonergic neuromodulation on the dynamics of the cortico basal ganglia network, and relates this impairment to abstract mood states (manic and depressive episodes) and oscillations of bipolar disorder.

The effect of 5-HT1A receptor antagonist on reward-based decision-making

When choosing the best action from several alternatives, we compare each value that depends on the balance between benefit and cost. Previous studies have shown that animals and humans with low brain serotonin (5-HT) level tend to choose smaller immediate reward. We used a decision-making schedule task to investigate whether 5-HT1A receptor is responsible for the decisions related to reward. In this task, the monkeys chose either of two different alternatives that were comprised of 1-4 drops of liquid reward (benefit) and 1-4 repeats of a color discrimination trial (workload cost), then executed the chosen schedule. By the administration of 5-HT1A antagonist, WAY100635, the choice tendency did not change, however, the sensitivity to the amount of reward in the schedule part was diminished. The 5-HT1A could have a role in maintaining reward value to keep track with the promised reward rather than modulating workload discounting of reward value.

Monoaminergic modulation of decision-making under risk of punishment in a rat model

The ability to decide advantageously among options that vary in both their risks and rewards is critical for survival and well-being. Previous work shows that some forms of risky decision-making are robustly modulated by monoamine signaling, but it is less clear how monoamine signaling modulates decision-making under risk of explicit punishment. The goal of these experiments was to determine how this form of decision-making is modulated by dopamine, serotonin, and norepinephrine signaling, using a task in which rats choose between a small, 'safe' food reward and a large food reward associated with variable risks of punishment. Preference for the large, risky reward (risk-taking) was reduced by administration of a D2/3 dopamine receptor agonist (bromocriptine) and a selective D2 agonist (sumanirole). The selective D3 agonist PD128907 appeared to attenuate reward discrimination abilities but did not affect risk-taking per se. In contrast, drugs targeting serotonergic and noradrenergic signaling had few if any effects on choice behavior. These data suggest that in contrast to other forms of risky decision-making, decision-making under risk of punishment is selectively modulated by dopamine signaling, predominantly through D2 receptors.

Anabolic-androgenic steroids and cognitive effort discounting in male rats

Anabolic-androgenic steroids (AAS) are drugs of abuse that impair behavior and cognition. In a rodent model of AAS abuse, testosterone-treated male rats expend more physical effort, by repeatedly pressing a lever for a large reward in an operant discounting task. However, since modern society prioritizes cognitive over physical effort, it is important to determine if AAS limit cognitive effort. Here we tested the effects of AAS on a novel cognitive-effort discounting task. Each operant chamber had 3 nose-pokes, opposite 2 levers and a pellet dispenser. Rats pressed a lever to illuminate 1 nose-poke; they responded in the illuminated nose-poke to receive sugar pellets. For the 'easy' lever, the light remained on for 1 s, and a correct response earned 1 pellet. For the 'hard' lever, the light duration decreased from 1 s to 0.1 s across 5 blocks of trials, and a correct response earned 4 pellets. As the duration of the nose-poke light decreased, all rats decreased their choice of the hard lever in a modest discounting curve. Task accuracy also decreased significantly across the 5 blocks of trials. However, there was no effect of testosterone on choice of the hard lever or task accuracy. Antagonism of dopamine D1 or D2 receptors had no effect on lever choice or task accuracy. However, serotonin depletion significantly decreased preference for the hard lever, and impaired task accuracy. Thus, physical effort discounting depends on dopamine activity, while cognitive effort discounting task is sensitive to serotonin. AAS impair physical effort discounting, but not cognitive effort discounting.

Injury-Related Deaths according to Environmental, Demographic, and Lifestyle Factors

Background

Environmental, demographic, and lifestyle variables have been associated with injury-related deaths. The current study identifies the simultaneous association of selected environmental, demographic, and lifestyle variables with deaths from homicide, unintentional injuries, and suicide.

Materials and Methods

Analyses are based on county-level mortality data in the contiguous United States, 2011–15. Basic summary statistics and Poisson regression were used to evaluate the data.

Results

The selected causes of death were impacted differently by age, sex, and race: for homicide, mortality rates were greater in ages 20–39, males, and blacks; for unintentional injuries, the rates increased with age, most noticeably in the oldest age group, and were highest among males and whites; and for suicide, the rates tended to increase with age and were greater in males and whites. Mortality rates from homicide were positively associated with poverty, cigarette smoking, air temperature, and leisure-time physical inactivity. They were negatively associated with precipitation and sunlight. Mortality rates from unintentional injuries were positively associated with altitude, cigarette smoking, air temperature, poverty, obesity, and precipitation. They were negatively associated with population density. Mortality rates from suicides were positively associated with altitude, cigarette smoking, obesity, air temperature, and precipitation and negatively associated with population density.

Conclusion

The results confirm and extend previous research in which death from homicide, unintentional injuries, and suicide are distinctly associated with a combination of environmental, demographic, and lifestyle variables. The findings may be useful in developing strategies for reducing injury-related deaths.

Risk-seeking for losses is associated with 5-HTTLPR, but not with transient changes in 5-HT levels

RATIONALE

Serotonin (5-HT) plays a key role in different aspects of value-based decision-making. A recent framework proposed that tonic 5-HT (together with dopamine, DA) codes future average reward expectations, providing a baseline against which possible choice outcomes are compared to guide decision-making.

OBJECTIVES

To test whether high 5-HT levels decrease loss aversion, risk-seeking for gains, and risk-seeking for losses.

METHODS

In a first session, 611 participants were genotyped for 5-HTTLPR and performed a mixed gambles (MGA) task and two probability discounting tasks for gains and losses, respectively (PDG/PDL). Afterwards, a subsample of 105 participants (44 with S/S, 6 with S/L, 55 with L/L genotype) completed the pharmacological study using a crossover design with tryptophan depletion (ATD), loading (ATL), and balanced (BAL) conditions. The same decision constructs were assessed.

RESULTS

We found increased risk-seeking for losses in S/S compared to L/L individuals at the first visit (p = 0.002). Neither tryptophan depletion nor loading affected decision-making, nor did we observe an interaction between intervention and 5-HTTLPR genotype.

CONCLUSION

Our data do not support the idea that transient changes of tonic 5-HT affect value-based decision-making. We provide evidence for an association of 5-HTTLPR with risk-seeking for losses, independent of acute 5-HT levels. This indicates that the association of 5-HTTLPR and risk-seeking for losses is mediated via other mechanisms, possibly by differences in the structural development of neural circuits of the 5-HT system during early life phases.

Sex Differences in Risk Preference and c-Fos Expression in Paraventricular Thalamic Nucleus of Rats During Gambling Task

Different biological requirements between males and females may cause sex differences in decision preference when choosing between taking a risk to get a higher gain or taking a lower but sure gain. Several studies have tested this assumption in rats, however the conclusion remains controversial because the previous real-world like gambling tasks contained a learning component to track a global payoff of probabilistic outcome in addition to risk preference. Therefore, we modified a simple gambling task allowing us to exclude such learning effect, and investigated the sex difference in risk preference of rats and its neural basis. The task required water deprived rats to choose between a risky option which provided four drops of water or no reward at a 50% random chance vs. a sure option which provided predictable amount x (x = 1, 2, 3, 4). The amount and the risk were explicitly instructed so that different choice conditions could be tested trial by trial without re-learning of reward contingency. Although both sexes correctly chose the sure option with the same level of accuracy when the sure option provided the best offer (x = 4), they exhibited different choice performances when two options had the same expected value (x = 2). Males and females both preferred to take risky choices than sure choices (risk seeking), but males were more risk seeking than females. Outcome-history analysis of their choice pattern revealed that females reduced their risk preference after losing risky choices, whereas males did not. Rather, as losses continued, reaction time for subsequent risky choices got shorter in males. Given that significant sex difference features mainly emerged after negative experiences, male and female rats may evaluate an unsuccessful outcome of their decision in different manners. Furthermore, c-Fos expression in the paraventricular nucleus of the thalamus (PV) was higher in the gambling task than for the control task in males while c-fos levels did not differ in females. The present study provides a clear evidence of sex differences in risk preference in rats and suggests that the PV is a candidate region contributing to sex differences in risky decision making.

Information about peer choices shapes human risky decision-making

Humans frequently make choices that involve risk for health and well-being. At the same time, information about others’ choices is omnipresent due to new forms of social media and information technology. However, while past research has shown that peers can exert a strong influence on such risky choices, understanding how information about risky decisions of others affects one’s own risky decisions is still lacking. We therefore developed a behavioral task to measure how information about peer choices affects risky decision-making and call it the social Balloon Analogue Risk Task (sBART). We tested this novel paradigm in a sample of 52 college young adults. Here we show that risky decisions were influenced in the direction of the perceived choices of others – riskier choices of others led to riskier behavior whereas safer choices of others led to less risky behavior. These findings indicate that information about peer choices is sufficient to shape one’s own risky behavior.

Decision-making under risk of gain in young children and mangabey monkeys

In contrast to human adults, risk proneness in the gain domain is usually observed in both young children and non-human primates. It is currently unclear what mechanism might be underlying such economic preferences. We investigated decision-making under risk of gain in toddlers and monkeys. The choices of 2.5-year-old children and red-capped mangabeys (Cercocebus torquatus torquatus) were examined in a gambling task for food reward in which participants have to choose between two options, a secure option and a risky option. In contrast to monkeys, toddlers showed a strong preference for the risky option over the safe option. In order to test the hypothesis that risky choices in participants reflect inhibitory control difficulties, toddlers and mangabeys were presented in Experiment 2 with a situation analogous to that used in Experiment 1 except for the fact that the opaque cover under which was placed the secure option was replaced by a transparent cover. In this second experiment, toddlers continued to show a preference for the risky option over the safe option. In contrast, mangabeys showed a preference for the safe option over the risky option in Trial 1 but they shifted their economic preferences in Trial 2. We argue that decision-making strategies under risk of gain in both toddlers and mangabeys (a) do not reflect poor behavioral control and (b) are not reducible to perception-action couplings.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

Learning Reward Uncertainty in the Basal Ganglia

Learning the reliability of different sources of rewards is critical for making optimal choices. However, despite the existence of detailed theory describing how the expected reward is learned in the basal ganglia, it is not known how reward uncertainty is estimated in these circuits. This paper presents a class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum. In the models, the tendency to seek (or avoid) options with variable reward can be controlled by increasing (or decreasing) the tonic level of dopamine. The models are consistent with the physiology of and synaptic plasticity in the basal ganglia, they explain the effects of dopaminergic manipulations on choices involving risks, and they make multiple experimental predictions.

Doubt in the Insula: Risk Processing in Obsessive-Compulsive Disorder

Extensive cleaning or checking of patients with obsessive-compulsive disorder (OCD) are often interpreted as strategies to avoid harm and as an expression of the widespread belief that OCD patients are more risk-averse. However, despite its clinical significance, the neural basis of risk attitude in OCD is unknown. Here, we investigated neural activity during risk processing using functional magnetic resonance imaging and simultaneously assessed risk attitude using a separate behavioral paradigm in OCD patients with different symptoms versus healthy controls (HCs). We found opposite insula responses to high versus low risk in OCD patients compared to HCs: a positive correlation between insula activity and risk-aversion in patients versus a negative correlation in controls. Although OCD patients overall were not more risk-averse than controls, there were differences between subgroups of OCD patients: patients with doubt/checking symptoms were more risk-averse than other patients. Taken together, OCD patients show a reversed pattern of risk processing by the insula compared to HCs. Moreover, the data suggest that increased activation of the insula signals an abnormal urge to avoid risks in the subpopulation of OCD patients with doubt and checking symptoms. These results indicate a role for the insula in excessive risk-avoidance relevant to OCD.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

Valence-dependent influence of serotonin depletion on model-based choice strategy

Human decision-making arises from both reflective and reflexive mechanisms, which underpin goal-directed and habitual behavioural control. Computationally, these two systems of behavioural control have been described by different learning algorithms, model-based and model-free learning, respectively. Here, we investigated the effect of diminished serotonin (5-hydroxytryptamine) neurotransmission using dietary tryptophan depletion (TD) in healthy volunteers on the performance of a two-stage decision-making task, which allows discrimination between model-free and model-based behavioural strategies. A novel version of the task was used, which not only examined choice balance for monetary reward but also for punishment (monetary loss). TD impaired goal-directed (model-based) behaviour in the reward condition, but promoted it under punishment. This effect on appetitive and aversive goal-directed behaviour is likely mediated by alteration of the average reward representation produced by TD, which is consistent with previous studies. Overall, the major implication of this study is that serotonin differentially affects goal-directed learning as a function of affective valence. These findings are relevant for a further understanding of psychiatric disorders associated with breakdown of goal-directed behavioural control such as obsessive-compulsive disorders or addictions.

The effects of acute fluoxetine administration on temporal discounting in youth with ADHD

BACKGROUND

Serotonin is under-researched in attention deficit hyperactivity disorder (ADHD), despite accumulating evidence for its involvement in impulsiveness and the disorder. Serotonin further modulates temporal discounting (TD), which is typically abnormal in ADHD relative to healthy subjects, underpinned by reduced fronto-striato-limbic activation. This study tested whether a single acute dose of the selective serotonin reuptake inhibitor (SSRI) fluoxetine up-regulates and normalizes reduced fronto-striato-limbic neurofunctional activation in ADHD during TD.

METHOD

Twelve boys with ADHD were scanned twice in a placebo-controlled randomized design under either fluoxetine (between 8 and 15 mg, titrated to weight) or placebo while performing an individually adjusted functional magnetic resonance imaging TD task. Twenty healthy controls were scanned once. Brain activation was compared in patients under either drug condition and compared to controls to test for normalization effects.

RESULTS

Repeated-measures whole-brain analysis in patients revealed significant up-regulation with fluoxetine in a large cluster comprising right inferior frontal cortex, insula, premotor cortex and basal ganglia, which further correlated trend-wise with TD performance, which was impaired relative to controls under placebo, but normalized under fluoxetine. Fluoxetine further down-regulated default mode areas of posterior cingulate and precuneus. Comparisons between controls and patients under either drug condition revealed normalization with fluoxetine in right premotor-insular-parietal activation, which was reduced in patients under placebo.

CONCLUSIONS

The findings show that a serotonin agonist up-regulates activation in typical ADHD dysfunctional areas in right inferior frontal cortex, insula and striatum as well as down-regulating default mode network regions in the context of impulsivity and TD.

Behavioral and emotional adverse events of drugs frequently used in the treatment of bipolar disorders: clinical and theoretical implications

BACKGROUND

Behavioral and emotional adverse events induced by drugs commonly prescribed to patients with bipolar disorders are of paramount importance to clinical practice and research. However, no reviews on the topic have been published so far.

METHODS

An extensive search was performed. Reports were reviewed if they described behavioral side effects related to pharmacological treatments for bipolar disorders in healthy subjects or patients with different neuropsychiatric disorders. For this review, lithium, antipsychotics, anticonvulsants and selective serotonin reuptake inhibitors were included.

RESULTS

Apathy or emotional blunting, diminished sexual desire, and inability to cry were reported to be associated with exposure to selective serotonin reuptake inhibitors. Neuroleptic-induced deficit syndrome/emotional detachment and obsessive-compulsive symptomatology and decision-making modifications. A lithium-related amotivational syndrome was also reported in the literature. Furthermore, hypersexuality and obsessive-compulsive symptoms have been noted in subjects treated with lamotrigine.

LIMITATIONS

Primary studies on drug-related adverse events are scant so far and most of the data currently available derive from case reports. Moreover, most of the evidence reviewed is based on studies performed on healthy subjects and patients with neuropsychiatric conditions other than bipolar disorders.

DISCUSSION

There is a remarkable dearth of data on behavioral adverse events of pharmacological treatment for bipolar disorders. However, the pieces of evidence available at present, though scant and scattered, suggest that different behavioral adverse events may be related to pharmacological treatment for these disorders. The implications of these findings for research and management of patients with mood disorders are discussed.

HEALTH SELECTION THEORY: AN EXPLANATION FOR THE PARADOX BETWEEN PERCEIVED MALE WELL-BEING AND MORTALITY

Paradoxically, men report better health and quality of life than women, but men experience higher mortality rates than women at most ages. One conclusion from these findings is that men have been selected to disregard signs of ill health, or even to deceive themselves about their health, to their detriment because presenting themselves as healthy has fitness benefits. We hypothesize that men have been sexually selected to present themselves to women as healthy but that the cost of not attending to their minor health problems results in earlier mortality than women. We present a review of the human and primate literature that supports health selection theory, the hypothesis that females have preferentially selected males who present themselves as healthy.

A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making

There is significant evidence that in addition to reward-punishment based decision making, the Basal Ganglia (BG) contributes to risk-based decision making (Balasubramani et al., 2014). Despite this evidence, little is known about the computational principles and neural correlates of risk computation in this subcortical system. We have previously proposed a reinforcement learning (RL)-based model of the BG that simulates the interactions between dopamine (DA) and serotonin (5HT) in a diverse set of experimental studies including reward, punishment and risk based decision making (Balasubramani et al., 2014). Starting with the classical idea that the activity of mesencephalic DA represents reward prediction error, the model posits that serotoninergic activity in the striatum controls risk-prediction error. Our prior model of the BG was an abstract model that did not incorporate anatomical and cellular-level data. In this work, we expand the earlier model into a detailed network model of the BG and demonstrate the joint contributions of DA-5HT in risk and reward-punishment sensitivity. At the core of the proposed network model is the following insight regarding cellular correlates of value and risk computation. Just as DA D1 receptor (D1R) expressing medium spiny neurons (MSNs) of the striatum were thought to be the neural substrates for value computation, we propose that DA D1R and D2R co-expressing MSNs are capable of computing risk. Though the existence of MSNs that co-express D1R and D2R are reported by various experimental studies, prior existing computational models did not include them. Ours is the first model that accounts for the computational possibilities of these co-expressing D1R-D2R MSNs, and describes how DA and 5HT mediate activity in these classes of neurons (D1R-, D2R-, D1R-D2R- MSNs). Starting from the assumption that 5HT modulates all MSNs, our study predicts significant modulatory effects of 5HT on D2R and co-expressing D1R-D2R MSNs which in turn explains the multifarious functions of 5HT in the BG. The experiments simulated in the present study relates 5HT to risk sensitivity and reward-punishment learning. Furthermore, our model is shown to capture reward-punishment and risk based decision making impairment in Parkinson's Disease (PD). The model predicts that optimizing 5HT levels along with DA medications might be essential for improving the patients' reward-punishment learning deficits.

Identifying the Basal Ganglia network model markers for medication-induced impulsivity in Parkinson's disease patients

Impulsivity, i.e. irresistibility in the execution of actions, may be prominent in Parkinson's disease (PD) patients who are treated with dopamine precursors or dopamine receptor agonists. In this study, we combine clinical investigations with computational modeling to explore whether impulsivity in PD patients on medication may arise as a result of abnormalities in risk, reward and punishment learning. In order to empirically assess learning outcomes involving risk, reward and punishment, four subject groups were examined: healthy controls, ON medication PD patients with impulse control disorder (PD-ON ICD) or without ICD (PD-ON non-ICD), and OFF medication PD patients (PD-OFF). A neural network model of the Basal Ganglia (BG) that has the capacity to predict the dysfunction of both the dopaminergic (DA) and the serotonergic (5HT) neuromodulator systems was developed and used to facilitate the interpretation of experimental results. In the model, the BG action selection dynamics were mimicked using a utility function based decision making framework, with DA controlling reward prediction and 5HT controlling punishment and risk predictions. The striatal model included three pools of Medium Spiny Neurons (MSNs), with D1 receptor (R) alone, D2R alone and co-expressing D1R-D2R. Empirical studies showed that reward optimality was increased in PD-ON ICD patients while punishment optimality was increased in PD-OFF patients. Empirical studies also revealed that PD-ON ICD subjects had lower reaction times (RT) compared to that of the PD-ON non-ICD patients. Computational modeling suggested that PD-OFF patients have higher punishment sensitivity, while healthy controls showed comparatively higher risk sensitivity. A significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients. Our results highlight the power of computational modelling for identifying neuronal circuitry implicated in learning, and its impairment in PD. The results presented here not only show that computational modelling can be used as a valuable tool for understanding and interpreting clinical data, but they also show that computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression.

A comparison of genetic variants between proficient low- and high-risk sport participants

Athletes participating in high-risk sports consistently report higher scores on sensation-seeking measures than do low-risk athletes or non-athletic controls. To determine whether genetic variants commonly associated with sensation seeking were over-represented in such athletes, proficient practitioners of high-risk (n = 141) and low-risk sports (n = 132) were compared for scores on sensation seeking and then genotyped at 33 polymorphic loci in 14 candidate genes. As expected, athletes participating in high-risk sports score higher on sensation seeking than did low-risk sport athletes (P < .01). Genotypes were associated with high-risk sport participation for two genes (stathmin, (P = .004) and brain-derived neurotrophic factor (P = .03)) as well as when demographically matched subsets of the sport cohorts were compared (P < .05); however, in all cases, associations did not survive correction for multiple testing.

Improving Research in the Emerging Field of Cross-Cultural Sociogenetics

We offer a critical overview of studies associating genetic differences in the 5-HTTLPR VNTR in the serotonin-transporter gene with societal differences. We also highlight recent findings from individual-level research on 5-HTTLPR generating new hypotheses concerning the effect of genes on culture. We provide an expanded national index reflecting 5-HTTLPR S-allele prevalence as an improved tool for future research. Our preliminary tests of this tool suggest that national S-allele prevalence is not associated with individualism as has been claimed, but with national neuroticism, IQ and school achievement, Hofstede’s fifth dimension of long-term orientation, and Minkov’s societal hypometropia—a measure of risk acceptance and short-term vision in life history strategy. We encourage detailed research of these associations in future studies.

Do tufted capuchin monkeys play the odds? Flexible risk preferences in Sapajus spp

As humans, several non-human animal species avoid risk, defined as "variability in rate of gain". However, non-human primate studies revealed a more complicated picture, with different species ranging from risk aversion to risk proneness. Within an ecological rationality framework, a species' feeding ecology should influence its risk preferences, as it has been shown in bonobos and chimpanzees. Although the feeding ecology hypothesis is promising, it has not been yet verified in species other than apes. Here, we aimed to assess whether this hypothesis holds true in tufted capuchin monkeys (Sapajus spp.). Ten capuchins were presented with choices between a "safe" option and a "risky" option in three conditions differing for the probability of receiving the larger reward when selecting the risky option. Similarly to chimpanzees, capuchins were risk prone. However, capuchins' behaviour was not the result of a bias towards the choice of the risky option, since-when facing options with different probabilities of obtaining the larger outcome-they were able to flexibly modify their preferences. Capuchins' decision-making under risk mirrors their risk-prone behaviour in the wild, where they often rely on unpredictable and/or hazardous food sources, thus satisfying the feeding ecology hypothesis.

Individual differences in gambling proneness among rats and common marmosets: an automated choice task

Interest is rising for animal modeling of pathological gambling. Using the operant probabilistic-delivery task (PDT), gambling proneness can be evaluated in laboratory animals. Drawing a comparison with rats, this study evaluated the common marmoset (Callithrix jacchus) using a PDT. By nose- or hand-poking, subjects learnt to prefer a large (LLL, 5-6 pellets) over a small (SS, 1-2 pellets) reward and, subsequently, the probability of occurrence of large-reward delivery was decreased progressively to very low levels (from 100% to 17% and 14%). As probability decreased, subjects showed a great versus little shift in preference from LLL to SS reinforcer. Hence, two distinct subpopulations (“non-gambler” versus “gambler”) were differentiated within each species. A proof of the model validity comes from marmosets' reaction to reward-delivery omission. Namely, depending on individual temperament (“gambler” versus “non-gambler”), they showed either persistence (i.e., inadequate pokes towards LLL) or restlessness (i.e., inadequate pokes towards SS), respectively. In conclusion, the marmoset could be a suitable model for preclinical gambling studies. Implementation of the PDT to species other than rats may be relevant for determining its external validity/generalizability and improving its face/construct validity.

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

Nonhuman gamblers: lessons from rodents, primates, and robots

The search for neuronal and psychological underpinnings of pathological gambling in humans would benefit from investigating related phenomena also outside of our species. In this paper, we present a survey of studies in three widely different populations of agents, namely rodents, non-human primates, and robots. Each of these populations offers valuable and complementary insights on the topic, as the literature demonstrates. In addition, we highlight the deep and complex connections between relevant results across these different areas of research (i.e., cognitive and computational neuroscience, neuroethology, cognitive primatology, neuropsychiatry, evolutionary robotics), to make the case for a greater degree of methodological integration in future studies on pathological gambling.

Computational model of precision grip in Parkinson's disease: a utility based approach

We propose a computational model of Precision Grip (PG) performance in normal subjects and Parkinson's Disease (PD) patients. Prior studies on grip force generation in PD patients show an increase in grip force during ON medication and an increase in the variability of the grip force during OFF medication (Ingvarsson et al., 1997; Fellows et al., 1998). Changes in grip force generation in dopamine-deficient PD conditions strongly suggest contribution of the Basal Ganglia, a deep brain system having a crucial role in translating dopamine signals to decision making. The present approach is to treat the problem of modeling grip force generation as a problem of action selection, which is one of the key functions of the Basal Ganglia. The model consists of two components: (1) the sensory-motor loop component, and (2) the Basal Ganglia component. The sensory-motor loop component converts a reference position and a reference grip force, into lift force and grip force profiles, respectively. These two forces cooperate in grip-lifting a load. The sensory-motor loop component also includes a plant model that represents the interaction between two fingers involved in PG, and the object to be lifted. The Basal Ganglia component is modeled using Reinforcement Learning with the significant difference that the action selection is performed using utility distribution instead of using purely Value-based distribution, thereby incorporating risk-based decision making. The proposed model is able to account for the PG results from normal and PD patients accurately (Ingvarsson et al., 1997; Fellows et al., 1998). To our knowledge the model is the first model of PG in PD conditions.

Serotonin 2A receptors contribute to the regulation of risk-averse decisions

Pharmacological studies point to a role of the neurotransmitter serotonin (5-HT) in regulating the preference for risky decisions, yet the functional contribution of specific 5-HT receptors remains to be clarified. We used pharmacological fMRI to investigate the role of the 5-HT2A receptors in processing negative outcomes and regulating risk-averse behavior. During fMRI, twenty healthy volunteers performed a gambling task under two conditions: with or without blocking the 5-HT2A receptors. The volunteers repeatedly chose between small, likely rewards and large, unlikely rewards. Choices were balanced in terms of expected utility and potential loss. Acute blockade of the 5-HT2A receptors with ketanserin made participants more risk-averse. Ketanserin selectively reduced the neural response of the frontopolar cortex to negative outcomes that were caused by low-risk choices and were associated with large missed rewards. In the context of normal 5-HT2A receptor function, ventral striatum displayed a stronger response to low-risk negative outcomes in risk-taking as opposed to risk-averse individuals. This (negative) correlation between the striatal response to low-risk negative outcomes and risk-averse choice behavior was abolished by 5-HT2A receptor blockade. The results provide the first evidence for a critical role of 5-HT2A receptor function in regulating risk-averse behavior. We suggest that the 5-HT2A receptor system facilitates risk-taking behavior by modulating the outcome evaluation of "missed" reward. These results have implications for understanding the neural basis of abnormal risk-taking behavior, for instance in pathological gamblers.

Food-derived serotonergic modulators: effects on mood and cognition

The most frequently described drugs in the treatment of mood disorders are selective serotonin reuptake and monoamine oxidase (MAO) inhibitors, enhancing serotonin levels in the brain. However, side-effects have been reported for these drugs. Because serotonin levels in the brain are dependent on the availability of the food-derived precursor tryptophan, foods such as chicken, soyabeans, cereals, tuna, nuts and bananas may serve as an alternative to improve mood and cognition. Here we discuss the effects of high- or low-tryptophan-containing food, as well as plant extracts with a modest monoamine reuptake and MAO-A inhibition functional profile, on mood and cognition in healthy and vulnerable human subjects and rodents. Together the studies suggest that there is an inverted U-shaped curve for plasma tryptophan levels, with low and too high tryptophan levels impairing cognition, and moderate to high tryptophan levels improving cognition. This relationship is found for both healthy and vulnerable subjects. Whereas this relationship may also exist for mood, the inverted U-shaped curve for plasma tryptophan levels and mood may be based on different tryptophan concentrations in healthyv.vulnerable individuals. Animal studies are emerging and allow further understanding of effects and the mode of action of food-derived serotonergic components on mood, cognition and mechanisms. Ultimately, insight into the concentrations of tryptophan and other serotonergic components in food having beneficial effects on mood and cognition in healthy, but particularly vulnerable, subjects may support well-being in our highly demanding society.

Playing it safe but losing anyway--serotonergic signaling of negative outcomes in dorsomedial prefrontal cortex in the context of risk-aversion

Risk avoidance is an important determinant of human behavior. The neurotransmitter serotonin has been implicated in processing negative outcomes caused by risky decisions. However, it is unclear whether serotonin provides a neurobiological link between making a risk aversive decision and the response to a negative outcome. Using pharmacological fMRI, we manipulated the availability of serotonin in healthy volunteers while performing a gambling task. The same group of participants was studied in three fMRI sessions: (i) during intravenous administration of the SSRI citalopram to increase the serotonergic tone, (ii) after acute tryptophan depletion (ATD) to reduce central serotonin levels, or (iii) without interventions. ATD and citalopram had opposite effects on outcome related activity in dorsomedial prefrontal cortex (dmPFC) and amygdala. Relative to the control condition, ATD increased and citalopram decreased the neural response to negative outcomes in dmPFC. Conversely, ATD decreased and citalopram increased the neural response to negative outcomes in left amygdala. Critically, these pharmacological effects were restricted to negative outcomes that were caused by low-risk decisions and led to a high missed reward. ATD and citalopram did not alter the neural response to positive outcomes in dmPFC, but relative to ATD, citalopram produced a bilateral increase in the amygdala response to large wins caused by high-risk choices. The results show a selective involvement of the serotonergic system in neocortical processing of negative outcomes resulting from risk-averse decisions, thereby linking risk aversion and processing of negative outcomes in goal-directed behaviors.

Assaying the effect of levodopa on the evaluation of risk in healthy humans

In humans, dopamine is implicated in reward and risk-based decision-making. However, the specific effects of dopamine augmentation on risk evaluation are unclear. Here we sought to measure the effect of 100 mg oral levodopa, which enhances synaptic release of dopamine, on choice behaviour in healthy humans. We use a paradigm without feedback or learning, which solely isolates effects on risk evaluation. We present two studies (n = 20; n = 20) employing a randomised, placebo-controlled, within-subjects design. We manipulated different dimensions of risk in a controlled economic paradigm. We test effects on risk-reward tradeoffs, assaying both aversion to variance (the spread of possible outcomes) and preference for relative losses and gains (asymmetry of outcomes - skewness), dissociating this from potential non-specific effects on choice randomness using behavioural modelling. There were no systematic effects of levodopa on risk attitudes, either for variance or skewness. However, there was a drift towards more risk-averse behaviour over time, indicating that this paradigm was sensitive to detect changes in risk-preferences. These findings suggest that levodopa administration does not change the evaluation of risk. One possible reason is that dopaminergic influences on decision making may be due to changing the response to reward feedback.

Contextual factors explain risk-seeking preferences in rhesus monkeys

In contrast to humans and most other animals, rhesus macaques strongly prefer risky rewards to safe ones with similar expected value. Why macaques prefer risk while other animals typically avoid it remains puzzling and challenges the idea that monkeys provide a model for human economic behavior. Here we argue that monkeys’ risk-seeking preferences are neither mysterious nor unique. Risk-seeking in macaques is possibly induced by specific elements of the tasks that have been used to measure their risk preferences. The most important of these elements are (1) very small stakes, (2) serially repeated gambles with short delays between trials, and (3) task parameters that are learned through experience, not described verbally. Together, we hypothesize that these features will readily induce risk-seeking in monkeys, humans, and rats. Thus, elements of task design that are often ignored when comparing studies of risk attitudes can easily overwhelm basal risk preferences. More broadly, these results highlight the fundamental importance of understanding the psychological basis of economic decisions in interpreting preference data and corresponding neural measures.

Coding of reward probability and risk by single neurons in animals

Probability and risk are important factors for value-based decision making and optimal foraging. In order to survive in an unpredictable world, organisms must be able to assess the probability and risk attached to future events and use this information to generate adaptive behavior. Recent studies in non-human primates and rats have shown that both probability and risk are processed in a distributed fashion throughout the brain at the level of single neurons. Reward probability has mainly been shown to be coded by phasic increases and decreases in firing rates in neurons in the basal ganglia, midbrain, parietal, and frontal cortex. Reward variance is represented in orbitofrontal and posterior cingulate cortex and through a sustained response of dopaminergic midbrain neurons.

Variance-sensitive choice in lemurs: constancy trumps quantity

Numerous studies have demonstrated that animals' tolerance for risk when foraging can be affected by changes in metabolic state. Specifically, animals on a negative energy budget increase their preferences for risk, while animals on a positive energy budget are typically risk-averse. The malleability of these preferences may be evolutionarily advantageous, and important for maximizing chances of survival during brief periods of energetic stress. However, animals adapted to living in unpredictable conditions are unlikely to benefit from risk-seeking strategies, and instead are expected to reduce energetic demands while maintaining risk-aversion. We measured risk preferences in lemurs, a group of primates restricted to the island of Madagascar. Lemurs have evolved diverse anatomical and behavioral traits for survival in a harsh and unpredictable ecology, and these traits have been explained as forms of anatomical and behavioral risk reduction. We therefore predicted that lemurs would also be risk-averse in a behavioral task that offered subjects a choice between a small certain reward, and an uncertain but potentially large reward. In Experiment 1, the average rewards associated with the constant and variable options were equal and lemurs exhibited high levels of risk-aversion, replicating a phenomenon that has been demonstrated in dozens of taxa. In Experiment 2, we gradually increased the average value of the variable option relative to the constant option. Lemurs' preferences tracked these changes and subjects became more risk-seeking as the risk premium increased. However, many subjects maintained high levels of risk-aversion even when the average payout of the variable option yielded double that of the constant option. These results are consistent with the notion that lemur cognition has evolved to minimize risk in an unpredictable island environment.

Risk-sensitivity in sensorimotor control

Recent advances in theoretical neuroscience suggest that motor control can be considered as a continuous decision-making process in which uncertainty plays a key role. Decision-makers can be risk-sensitive with respect to this uncertainty in that they may not only consider the average payoff of an outcome, but also consider the variability of the payoffs. Although such risk-sensitivity is a well-established phenomenon in psychology and economics, it has been much less studied in motor control. In fact, leading theories of motor control, such as optimal feedback control, assume that motor behaviors can be explained as the optimization of a given expected payoff or cost. Here we review evidence that humans exhibit risk-sensitivity in their motor behaviors, thereby demonstrating sensitivity to the variability of "motor costs." Furthermore, we discuss how risk-sensitivity can be incorporated into optimal feedback control models of motor control. We conclude that risk-sensitivity is an important concept in understanding individual motor behavior under uncertainty.

Serotonin and dopamine: unifying affective, activational, and decision functions

Serotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcement learning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance, compared with DA, we have a much less specific understanding about the mechanisms by which serotonin affects behavior. Here, we draw on recent work on computational models of dopaminergic function to suggest a framework by which many of the seemingly diverse functions associated with both DA and serotonin-comprising both affective and activational ones, as well as a number of other functions not overtly related to either-can be seen as consequences of a single root mechanism.