Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Optogenetic activation of dorsal raphe serotonin neurons induces brain-wide activation

Serotonin is a neuromodulator that affects multiple behavioral and cognitive functions. Nonetheless, how serotonin causes such a variety of effects via brain-wide projections and various receptors remains unclear. Here we measured brain-wide responses to optogenetic stimulation of serotonin neurons in the dorsal raphe nucleus (DRN) of the male mouse brain using functional MRI with an 11.7 T scanner and a cryoprobe. Transient activation of DRN serotonin neurons caused brain-wide activation, including the medial prefrontal cortex, the striatum, and the ventral tegmental area. The same stimulation under anesthesia with isoflurane decreased brain-wide activation, including the hippocampal complex. These brain-wide response patterns can be explained by DRN serotonergic projection topography and serotonin receptor expression profiles, with enhanced weights on 5-HT1 receptors. Together, these results provide insight into the DR serotonergic system, which is consistent with recent discoveries of its functions in adaptive behaviors.

Methylphenidate, but not citalopram, decreases impulsive choice in rats performing a temporal discounting task

Introduction

Drugs targeting monoamine systems remain the most common treatment for disorders with impulse control impairments. There is a body of literature suggesting that drugs affecting serotonin reuptake and dopamine reuptake can modulate distinct aspects of impulsivity - though such tests are often performed using distinct behavioral tasks prohibiting easy comparisons.

Methods

Here, we directly compare pharmacologic agents that affect dopamine (methylphenidate) vs serotonin (citalopram) manipulations on choice impulsivity in a temporal discounting task where rats could choose between a small, immediate reward or a large reward delayed at either 2 or 10s. In control conditions, rats preferred the large reward at a small (2s) delay and discounted the large reward at a long (10s) delay.

Results

Methylphenidate, a dopamine transport inhibitor that blocks reuptake of dopamine, dose-dependently increased large reward preference in the long delay (10s) block. Citalopram, a selective serotonin reuptake inhibitor, had no effect on temporal discounting behavior. Impulsive behavior on the temporal discounting task was at least partially mediated by the nucleus accumbens shell. Bilateral lesions to the nucleus accumbens shell reduced choice impulsivity during the long delay (10s) block. Following lesions, methylphenidate did not impact impulsivity.

Discussion

Our results suggest that striatal dopaminergic systems modulate choice impulsivity via actions within the nucleus accumbens shell, whereas serotonin systems may regulate different aspects of behavioral inhibition/impulsivity.

Synthetic surprise as the foundation of the psychedelic experience

Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Local 5-HT signaling bi-directionally regulates the coincidence time window for associative learning

The coincidence between conditioned stimulus (CS) and unconditioned stimulus (US) is essential for associative learning; however, the mechanism regulating the duration of this temporal window remains unclear. Here, we found that serotonin (5-HT) bi-directionally regulates the coincidence time window of olfactory learning in Drosophila and affects synaptic plasticity of Kenyon cells (KCs) in the mushroom body (MB). Utilizing GPCR-activation-based (GRAB) neurotransmitter sensors, we found that KC-released acetylcholine (ACh) activates a serotonergic dorsal paired medial (DPM) neuron, which in turn provides inhibitory feedback to KCs. Physiological stimuli induce spatially heterogeneous 5-HT signals, which proportionally gate the intrinsic coincidence time windows of different MB compartments. Artificially reducing or increasing the DPM neuron-released 5-HT shortens or prolongs the coincidence window, respectively. In a sequential trace conditioning paradigm, this serotonergic neuromodulation helps to bridge the CS-US temporal gap. Altogether, we report a model circuitry for perceiving the temporal coincidence and determining the causal relationship between environmental events.

Fifty years on: Serotonin and depression

It has been over 50 years since the original serotonin hypothesis was proposed by the British Psychiatrist Alec Coppen. Recently, some authors have questioned the validity of the hypothesis. In this narrative review, we summarise the evidence for the serotonin hypothesis of depression, focusing on psychopharmacology and molecular imaging, as well as systems-level neuroscience.

A behavioral timing intervention upregulates striatal serotonergic markers and reduces impulsive action in adult male mice

Many studies support the hypothesis that time-based interventions reduce impulsive behavior in rodents. However, few studies have directly assessed 1) how such interventions affect impulsive action rather than impulsive choice, 2) if intervention effects differ by sex, and 3) how time-based interventions affect neurochemistry in regions mediating decision-making and reward. Thus, we assessed how a fixed-interval (FI) intervention initiated during late adolescence and extending into adulthood affected dopaminergic and serotonergic analytes in the frontal cortex and striatum and subsequent impulsive action in adult male and female mice. Beginning on postnatal day (PND) 45, mice were either trained on a progressive series of FI schedules (FI 20, 40, & 60 s) or remained in the home cage. Following the intervention, increases in striatal serotonergic analytes were found in FI-exposed males and females (n = 8/sex/group) with few changes found in the frontal cortex. Impulsive action was assessed in the remaining mice (n = 10/sex/group) using a fixed-ratio waiting-for-reward (FR-wait) task in which completion of an FR-25 component initiated a "free" pellet component in which pellets were delivered at increasing intervals according to a fixed delay increment that varied across sessions. Responses reset the additive delay and initiated a new FR-25 component. FI-exposed males, but not females, showed fewer delay resets and no-wait resets relative to control mice. Importantly, FI-exposure did not affect discrimination reversal performance in either sex. These data suggest that time-based interventions may reduce impulsive action in addition to impulsive choice perhaps with increased male sensitivity. Additionally, time-based interventions appear to operate through striatal serotonergic augmentation.

What Can Reinforcement Learning Models of Dopamine and Serotonin Tell Us about the Action of Antidepressants?

Although evidence suggests that antidepressants are effective at treating depression, the mechanisms behind antidepressant action remain unclear, especially at the cognitive/computational level. In recent years, reinforcement learning (RL) models have increasingly been used to characterise the roles of neurotransmitters and to probe the computations that might be altered in psychiatric disorders like depression. Hence, RL models might present an opportunity for us to better understand the computational mechanisms underlying antidepressant effects. Moreover, RL models may also help us shed light on how these computations may be implemented in the brain (e.g., in midbrain, striatal, and prefrontal regions) and how these neural mechanisms may be altered in depression and remediated by antidepressant treatments. In this paper, we evaluate the ability of RL models to help us understand the processes underlying antidepressant action. To do this, we review the preclinical literature on the roles of dopamine and serotonin in RL, draw links between these findings and clinical work investigating computations altered in depression, and appraise the evidence linking modification of RL processes to antidepressant function. Overall, while there is no shortage of promising ideas about the computational mechanisms underlying antidepressant effects, there is insufficient evidence directly implicating these mechanisms in the response of depressed patients to antidepressant treatment. Consequently, future studies should investigate these mechanisms in samples of depressed patients and assess whether modifications in RL processes mediate the clinical effect of antidepressant treatments.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the “brake” on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory “drive.” For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the “drive” and “brake” circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Complementary roles of serotonergic and cholinergic systems in decisions about when to act

Decision-making not only involves deciding about which action to choose but when and whether to initiate an action in the first place. Macaque monkeys tracked number of dots on a screen and could choose when to make a response. The longer the animals waited before responding, the more dots appeared on the screen and the higher the probability of reward. Monkeys waited longer before making a response when a trial’s value was less than the environment’s average value. Recordings of brain activity with fMRI revealed that activity in dorsal raphe nucleus (DRN)—a key source of serotonin (5-HT)—tracked average value of the environment. By contrast, activity in the basal forebrain (BF)—an important source of acetylcholine (ACh)—was related to decision time to act as a function of immediate and recent past context. Interactions between DRN and BF and the anterior cingulate cortex (ACC), another region with action initiation-related activity, occurred as a function of the decision time to act. Next, we performed two psychopharmacological studies. Manipulating systemic 5-HT by citalopram prolonged the time macaques waited to respond for a given opportunity. This effect was more evident during blocks with long inter-trial intervals (ITIs) where good opportunities were sparse. Manipulating systemic acetylcholine (ACh) by rivastigmine reduced the time macaques waited to respond given the immediate and recent past context, a pattern opposite to the effect observed with 5-HT. These findings suggest complementary roles for serotonin/DRN and acetylcholine/BF in decisions about when to initiate an action.

•

Both immediate context and wider environment influence decisions about when to act

•

DRN and 5-HT mediate the influence of wider environment

•

BF and ACh mediate the influence of immediate context

Serotonin neurons modulate learning rate through uncertainty

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

5-HT2C receptor perturbation has bidirectional influence over instrumental vigour and restraint

The serotonin (5-HT) system, particularly the 5-HT2C receptor, has consistently been implicated in behavioural control. However, while some studies have focused on the role 5-HT2C receptors play in regulating motivation to work for reward, others have highlighted its importance in response restraint. To date, it is unclear how 5-HT transmission at this receptor regulates the balance of response invigoration and restraint in anticipation of future reward. In addition, it remains to be established how 5-HT2C receptors gate the influence of internal versus cue-driven processes over reward-guided actions. To elucidate these issues, we investigated the effects of administering the 5-HT2C receptor antagonist SB242084, both systemically and directly into the nucleus accumbens core (NAcC), in rats performing a Go/No-Go task for small or large rewards. The results were compared to the administration of d-amphetamine into the NAcC, which has previously been shown to promote behavioural activation. Systemic perturbation of 5-HT2C receptors-but crucially not intra-NAcC infusions-consistently boosted rats' performance and instrumental vigour on Go trials when they were required to act. Concomitantly, systemic administration also reduced their ability to withhold responding for rewards on No-Go trials, particularly late in the holding period. Notably, these effects were often apparent only when the reward on offer was small. By contrast, inducing a hyperdopaminergic state in the NAcC with d-amphetamine strongly impaired response restraint on No-Go trials both early and late in the holding period, as well as speeding action initiation. Together, these findings suggest that 5-HT2C receptor transmission, outside the NAcC, shapes the vigour of ongoing goal-directed action as well as the likelihood of responding as a function of expected reward.

The dorsal raphe nucleus in the control of energy balance

Energy balance is orchestrated by an extended network of highly interconnected nuclei across the central nervous system. While much is known about the hypothalamic circuits regulating energy homeostasis, the 'extra-hypothalamic' circuits involved are relatively poorly understood. In this review, we focus on the brainstem's dorsal raphe nucleus (DRN), integrating decades of research linking this structure to the physiologic and behavioral responses that maintain proper energy stores. DRN neurons sense and respond to interoceptive and exteroceptive cues related to energy imbalance and in turn induce appropriate alterations in energy intake and expenditure. The DRN is also molecularly differentiable, with different populations playing distinct and often opposing roles in controlling energy balance. These populations are integrated into the extended circuit known to regulate energy balance. Overall, this review summarizes the key evidence demonstrating an important role for the DRN in regulating energy balance.

Serotonergic regulation of the dopaminergic system: Implications for reward-related functions

Serotonin is a critical neuromodulator involved in development and behavior. Its role in reward is however still debated. Here, we first review classical studies involving electrical stimulation protocols and pharmacological approaches. Contradictory results on the serotonergic' involvement in reward emerge from these studies. These differences might be ascribable to either the diversity of cellular types within the raphe nuclei or/and the specific projection pathways of serotonergic neurons. We continue to review more recent work, using optogenetic approaches to activate serotonergic cells in the Raphe to VTA pathway. From these studies, it appears that activation of this pathway can lead to reinforcement learning mediated through the excitation of dopaminergic neurons by serotonergic neurons co-transmitting glutamate. Finally, given the importance of serotonin during development on adult emotion, the effect of abnormal early-life levels of serotonin on the dopaminergic system will also be discussed. Understanding the interaction between the serotonergic and dopaminergic systems during development and adulthood is critical to gain insight into the specific facets of neuropsychiatric disorders.

Developmental exposure to the synthetic progestin, 17α-hydroxyprogesterone caproate, disrupts the mesocortical serotonin pathway and alters impulsive decision-making in rats

The synthetic progestin, 17α-hydroxyprogesterone caproate (17-OHPC), is administered to women at risk for preterm birth during a critical period of fetal development for mesocortical pathways. Yet, little information is available regarding the potential effects of 17-OHPC on the developing fetal brain. In rat models, the mesocortical serotonin pathway is sensitive to progestins. Progesterone receptor (PR) is expressed in layer 3 pyramidal neurons of medial prefrontal cortex (mPFC) and in serotonergic neurons of the dorsal raphe. The present study tested the hypothesis that exposure to 17-OHPC during development disrupts serotonergic innervation of the mPFC in adolescence and impairs behavior mediated by this pathway in adulthood. Administration of 17-OHPC from postnatal days 1-14 decreased the density of SERT-ir fibers within superficial and deep layers and decreased the density of synaptophysin-ir boutons in all layers of prelimbic mPFC at postnatal day 28. In addition, rats exposed to 17-OHPC during development were less likely to make impulsive choices in the Delay Discounting task, choosing the larger, delayed reward more often than controls at moderate delay times. Interestingly, 17-OHPC exposed rats were more likely to fail to make any choice (i.e., increased omissions) compared to controls at longer delays, suggesting disruptions in decision-making. These results suggest that further investigation is warranted in the clinical use of 17-OHPC to better inform a risk/benefit analysis of progestin use in pregnancy.

Reciprocal Lateral Hypothalamic and Raphe GABAergic Projections Promote Wakefulness

The lateral hypothalamus (LH), together with multiple neuromodulatory systems of the brain, such as the dorsal raphe nucleus (DR), is implicated in arousal, yet interactions between these systems are just beginning to be explored. Using a combination of viral tracing, circuit mapping, electrophysiological recordings from identified neurons, and combinatorial optogenetics in mice, we show that GABAergic neurons in the LH selectively inhibit GABAergic neurons in the DR, resulting in increased firing of a substantial fraction of its neurons that ultimately promotes arousal. These DRGABA neurons are wake active and project to multiple brain areas involved in the control of arousal, including the LH, where their specific activation potently influences local network activity leading to arousal from sleep. Our results show how mutual inhibitory projections between the LH and the DR promote wakefulness and suggest a complex arousal control by intimate interactions between long-range connections and local circuit dynamics.SIGNIFICANCE STATEMENT: Multiple brain systems including the lateral hypothalamus and raphe serotonergic system are involved in the regulation of the sleep/wake cycle, yet the interaction between these systems have remained elusive. Here we show that mutual disinhibition mediated by long range inhibitory projections between these brain areas can promote wakefulness. The main importance of this work relies in revealing the interaction between a brain area involved in autonomic regulation and another in controlling higher brain functions including reward, patience, mood and sensory coding.

Modulating the Neuromodulators: Dopamine, Serotonin, and the Endocannabinoid System

Dopamine (DA), serotonin (5-hydroxytryptamine, 5-HT), and endocannabinoids (ECs) are key neuromodulators involved in many aspects of motivated behavior, including reward processing, reinforcement learning, and behavioral flexibility. Among the longstanding views about possible relationships between these neuromodulators is the idea of DA and 5-HT acting as opponents. This view has been challenged by emerging evidence that 5-HT supports reward seeking via activation of DA neurons in the ventral tegmental area. Adding an extra layer of complexity to these interactions, the endocannabinoid system is uniquely placed to influence dopaminergic and serotonergic neurotransmission. In this review we discuss how these three neuromodulatory systems interact at the cellular and circuit levels. Technological advances that facilitate precise identification and control of genetically targeted neuronal populations will help to achieve a better understanding of the complex relationship between these essential systems, and the potential relevance for motivated behavior.

Soft Drink Consumption in Young Mexican Adults Is Associated with Higher Total Body Fat Percentage in Men but Not in Women

A high consumption of soft drinks (SDs) has been linked with the development of anthropometric and metabolic alterations. We evaluate the association between SD consumption and some anthropometric and metabolic variables. This study is an observational study, using a sample of 394 university students, of which 158 were men (40.1%) and 238 women (59.9%), between 18 and 30 years. An SD intake questionnaire provided the consumption of different SDs. The participants' weight, height, and waist and hip circumferences were collected. Metabolic biomarkers were analyzed. The average intake of caloric SDs (CSDs) was 1193.6 ± 1534.8 mL/week and 84.5 ± 115.02 mL/week for non-caloric SDs (NCSDs). Sex differences were found in the amount of SD consumption and these statistical differences were driven by those men subjects with a high total body fat percentage (TBF%). In men, correlations were found between the intake of CSDs and the body mass index, waist and hip circumferences, TBF%, and visceral fat percentage. In woman, a correlation was found with glucose and triglycerides. The prediction model revealed that the intake of CSDs predicts TBF% and low-density lipoprotein only in men. A high amount of CSD consumption in men was associated with a high TBF%, and this may be predictive of future development of metabolic abnormalities.

Serotonergic projections to the orbitofrontal and medial prefrontal cortices differentially modulate waiting for future rewards

Optogenetic activation of serotonergic neurons in the dorsal raphe nucleus (DRN) enhances patience when waiting for future rewards, and this effect is maximized by both high probability and high timing uncertainty of reward. Here, we explored which serotonin projection areas contribute to these effects using optogenetic axon terminal stimulation. We found that serotonin stimulation in the orbitofrontal cortex (OFC) is nearly as effective as that in the DRN for promoting waiting, while in the nucleus accumbens, it does not promote waiting. We also found that serotonin stimulation in the medial prefrontal cortex (mPFC) promotes waiting only when the timing of future rewards is uncertain. Our Bayesian decision model of waiting assumed that the OFC and mPFC calculate the posterior probability of reward delivery separately. These results suggest that serotonin in the mPFC affects evaluation of time committed, while serotonin in the OFC is responsible for overall valuation of delayed rewards.

The Role of Tryptophan and Tyrosine in Executive Function and Reward Processing

The serotonergic precursor tryptophan and the dopaminergic precursor tyrosine have been shown to be important modulators of mood, behaviour and cognition. Specifically, research on the function of tryptophan has characterised this molecule as particularly relevant in the context of pathological disorders such as depression. Moreover, a large body of evidence has now been accumulated to suggest that tryptophan may also be involved in executive function and reward processing. Despite some clear differentiation with tryptophan, the data reviewed in this paper illustrates that tyrosine shares similar functions with tryptophan in the regulation of executive function and reward, and that these processes in turn, rather than acting in isolation, causally influence each other.

Psychological mechanisms and functions of 5-HT and SSRIs in potential therapeutic change: Lessons from the serotonergic modulation of action selection, learning, affect, and social cognition

Uncertainty regarding which psychological mechanisms are fundamental in mediating SSRI treatment outcomes and wide-ranging variability in their efficacy has raised more questions than it has solved. Since subjective mood states are an abstract scientific construct, only available through self-report in humans, and likely involving input from multiple top-down and bottom-up signals, it has been difficult to model at what level SSRIs interact with this process. Converging translational evidence indicates a role for serotonin in modulating context-dependent parameters of action selection, affect, and social cognition; and concurrently supporting learning mechanisms, which promote adaptability and behavioural flexibility. We examine the theoretical basis, ecological validity, and interaction of these constructs and how they may or may not exert a clinical benefit. Specifically, we bridge crucial gaps between disparate lines of research, particularly findings from animal models and human clinical trials, which often seem to present irreconcilable differences. In determining how SSRIs exert their effects, our approach examines the endogenous functions of 5-HT neurons, how 5-HT manipulations affect behaviour in different contexts, and how their therapeutic effects may be exerted in humans - which may illuminate issues of translational models, hierarchical mechanisms, idiographic variables, and social cognition.

Stress drives deliberative tendencies by influencing vicarious trial and error in decision making

Previous studies have reported the effects of stress on decision making. However, the wide range of findings make it difficult to identify the fundamental effects of stress on decision making and, therefore, how stress affects decision making remains unknown. To investigate the influence of stress on decision making, we employed "vicarious trial and error" (VTE), which refers to a rat's behavior of orienting the head toward options at a decision point. VTE is thought to reflect mental simulation for possible options preceding a decision. We examined effects of acute restraint stress on VTE in a T-maze choice task. VTE depended on learning and past reward outcomes. Acute restraint stress before rats ran the T-maze choice task induced VTE, especially in trials with low demand of VTE, and increased the number of head orientations and time spent during each VTE. On the other hand, stress did not affect task performance (probability of advantageous choice) and patterns of behavioral choice (win-stay lose-shift, exploration-exploitation). In addition, stress activated serotonergic and noradrenergic neurons in the dorsal raphe nucleus and locus coeruleus, which are modulators of impulsivity and attentional control in decision making. These results suggest that stress in decision making drives the VTE process, which may lead to deep consideration, over-thinking, and indecisiveness.

Reward Contributions to Serotonergic Functions

The brain serotonin systems participate in numerous aspects of reward processing, although it remains elusive how exactly serotonin signals regulate neural computation and reward-related behavior. The application of optogenetics and imaging techniques during the last decade has provided many insights. Here, we review recent progress on the organization and physiology of the dorsal raphe serotonin neurons and the relationships between their activity and behavioral functions in the context of reward processing. We also discuss several interesting theories on serotonin's function and how these theories may be reconciled by the possibility that serotonin, acting in synergy with coreleased glutamate, tracks and calculates the so-called beneficialness of the current state to guide an animal's behavior in dynamic environments.

Serotonergic modulation across sensory modalities

The serotonergic system has been widely studied across animal taxa and different functional networks. This modulatory system is therefore well positioned to compare the consequences of neuromodulation for sensory processing across species and modalities at multiple levels of sensory organization. Serotonergic neurons that innervate sensory networks often bidirectionally exchange information with these networks but also receive input representative of motor events or motivational state. This convergence of information supports serotonin's capacity for contextualizing sensory information according to the animal's physiological state and external events. At the level of sensory circuitry, serotonin can have variable effects due to differential projections across specific sensory subregions, as well as differential serotonin receptor type expression within those subregions. Functionally, this infrastructure may gate or filter sensory inputs to emphasize specific stimulus features or select among different streams of information. The near-ubiquitous presence of serotonin and other neuromodulators within sensory regions, coupled with their strong effects on stimulus representation, suggests that these signaling pathways should be considered integral components of sensory systems.

Separable gain control of ongoing and evoked activity in the visual cortex by serotonergic input

Controlling gain of cortical activity is essential to modulate weights between internal ongoing communication and external sensory drive. Here, we show that serotonergic input has separable suppressive effects on the gain of ongoing and evoked visual activity. We combined optogenetic stimulation of the dorsal raphe nucleus (DRN) with wide-field calcium imaging, extracellular recordings, and iontophoresis of serotonin (5-HT) receptor antagonists in the mouse visual cortex. 5-HT1A receptors promote divisive suppression of spontaneous activity, while 5-HT2A receptors act divisively on visual response gain and largely account for normalization of population responses over a range of visual contrasts in awake and anesthetized states. Thus, 5-HT input provides balanced but distinct suppressive effects on ongoing and evoked activity components across neuronal populations. Imbalanced 5-HT1A/2A activation, either through receptor-specific drug intake, genetically predisposed irregular 5-HT receptor density, or change in sensory bombardment may enhance internal broadcasts and reduce sensory drive and vice versa.

Neural circuitry and mechanisms of waiting impulsivity: relevance to addiction

Impatience-the failure to wait or tolerate delayed rewards (e.g. food, drug and monetary incentives)-is a common behavioural tendency in humans. However, when rigidly and rapidly expressed with limited regard for future, often negative consequences, impatient or impulsive actions underlie and confer susceptibility for such diverse brain disorders as drug addiction, attention-deficit hyperactivity disorder (ADHD) and major depressive disorder. Consequently, 'waiting' impulsivity has emerged as a candidate endophenotype to inform translational research on underlying neurobiological mechanisms and biomarker discovery for many of the so-called impulse-control disorders. Indeed, as reviewed in this article, this research enterprise has revealed a number of unexpected targets and mechanisms for intervention. However, in the context of drug addiction, impulsive decisions that maximize short-term gains (e.g. acute drug consumption) over longer-term punishment (e.g. unemployment, homelessness, personal harm) defines one aspect of impulsivity, which may or may not be related to rapid, unrestrained actions over shorter timescales. We discuss the relevance of this distinction in impulsivity subtypes for drug addiction with reference to translational research in humans and other animals. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

A neural network for information seeking

Humans and other animals often show a strong desire to know the uncertain rewards their future has in store, even when they cannot use this information to influence the outcome. However, it is unknown how the brain predicts opportunities to gain information and motivates this information-seeking behavior. Here we show that neurons in a network of interconnected subregions of primate anterior cingulate cortex and basal ganglia predict the moment of gaining information about uncertain rewards. Spontaneous increases in their information prediction signals are followed by gaze shifts toward objects associated with resolving uncertainty, and pharmacologically disrupting this network reduces the motivation to seek information. These findings demonstrate a cortico-basal ganglia mechanism responsible for motivating actions to resolve uncertainty by seeking knowledge about the future.

Animals resolve uncertainty by seeking knowledge about the future. How the brain controls this is unclear. The authors show that a network including primate anterior cingulate cortex and basal ganglia encodes opportunities to gain information about uncertain rewards and mediates information seeking.

No evidence for the involvement of serotonin or the 5-HTTLPR genotype in intertemporal choice in a larger community sample

BACKGROUND

Serotonin has been implicated in impulsive behaviours such as temporal discounting. While animal studies and theoretical approaches suggest that reduced tonic serotonin levels increase temporal discounting rates and vice versa, evidence from human studies is scarce and inconclusive. Furthermore, an important modulator of serotonin signalling, a genetic variation in the promoter region of the serotonin transporter gene (5-HTTLPR), has not been investigated for temporal discounting so far.

OBJECTIVE

First, the purpose of this study was to test for a significant association between 5-HTTLPR and temporal discounting. Second, we wished to investigate the effect of high/low tonic serotonin levels on intertemporal choice and blood oxygen-level-dependent response, controlling for 5-HTTLPR.

METHODS

We tested the association of 5-HTTLPR with temporal discounting rates using an intertemporal choice task in 611 individuals. We then manipulated tonic serotonin levels with acute tryptophan interventions (depletion, loading, balanced) in a subsample of 45 short (S)-allele and 45 long (L)/L-allele carriers in a randomised double-blind crossover design using functional magnetic resonance imaging and an intertemporal choice task.

RESULTS

Overall, we did not find any effect of serotonin and 5-HTTLPR on temporal discounting rates or the brain networks associated with valuation and cognitive control.

CONCLUSION

Our findings indicate that serotonin may not be directly involved in choices including delays on longer timescales such as days, weeks or months. We speculate that serotonin plays a stronger role in dynamic intertemporal choice tasks where the delays are on a timescale of seconds and hence are therefore directly experienced during the experiment.

The effect of 5-HT1A receptor antagonist on reward-based decision-making

When choosing the best action from several alternatives, we compare each value that depends on the balance between benefit and cost. Previous studies have shown that animals and humans with low brain serotonin (5-HT) level tend to choose smaller immediate reward. We used a decision-making schedule task to investigate whether 5-HT1A receptor is responsible for the decisions related to reward. In this task, the monkeys chose either of two different alternatives that were comprised of 1-4 drops of liquid reward (benefit) and 1-4 repeats of a color discrimination trial (workload cost), then executed the chosen schedule. By the administration of 5-HT1A antagonist, WAY100635, the choice tendency did not change, however, the sensitivity to the amount of reward in the schedule part was diminished. The 5-HT1A could have a role in maintaining reward value to keep track with the promised reward rather than modulating workload discounting of reward value.

Prior Activation of 5-HT7 Receptors Modulates the Conditioned Place Preference With Methylphenidate

The serotonin receptor subtype 7 (5-HT7R) is clearly involved in behavioral functions such as learning/memory, mood regulation and circadian rhythm. Recent discoveries proposed modulatory physiological roles for serotonergic systems in reward-guided behavior. However, the interplay between serotonin (5-HT) and dopamine (DA) in reward-related behavioral adaptations needs to be further assessed. TP-22 is a recently developed arylpiperazine-based 5-HT7R agonist, which is also showing high affinity and selectivity towards D1 receptors. Here, we report that TP-22 displays D1 receptor antagonist activity. Moreover, we describe the first in vivo tests with TP-22: first, a pilot experiment (assessing dosage and timing of action) identified the 0.25 mg/kg i.v. dosage for locomotor stimulation of rats. Then, a conditioned place preference (CPP) test with the DA-releasing psychostimulant drug, methylphenidate (MPH), involved three rat groups: prior i.v. administration of TP-22 (0.25 mg/kg), or vehicle (VEH), 90 min before MPH (5 mg/kg), was intended for modulation of conditioning to the white chamber (saline associated to the black chamber); control group (SAL) was conditioned with saline in both chambers. Prior TP-22 further increased the stimulant effect of MPH on locomotor activity. During the place-conditioning test, drug-free activity of TP-22+MPH subjects remained steadily elevated, while VEH+MPH subjects showed a decline. Finally, after a priming injection of TP-22 in MPH-free conditions, rats showed a high preference for the MPH-associated white chamber, which conversely had vanished in VEH-primed MPH-conditioned subjects. Overall, the interaction between MPH and pre-treatment with TP-22 seems to improve both locomotor stimulation and the conditioning of motivational drives to environmental cues. Together with recent studies, a main modulatory role of 5-HT7R for the processing of rewards can be suggested. In the present study, TP-22 proved to be a useful psychoactive tool to better elucidate the role of 5-HT7R and its interplay with DA in reward-related behavior.

Encoding prediction signals during appetitive and aversive Pavlovian conditioning in the primate lateral hypothalamus

The lateral hypothalamus (LH), which plays a role in homeostatic functions such as appetite regulation, is also linked to arousal and motivational behavior. However, little is known about how these components are encoded in the LH. Thus cynomolgus monkeys were conditioned with two distinct contexts, i.e., an appetitive context with available rewards and an aversive context with predicted air puffs. Different LH neuron groups encoded different degrees of expectation, predictability, and risks of rewards in a specific manner. A nearly equal number of one-third of the recorded LH neurons showed a positive or negative correlation between their response to visual conditioned stimuli (CS) that predicted the probabilistic delivery of rewards (0%, 50%, and 100%) and the associative values. For another one-third of recorded neurons, a nearly equal number showed a positive or negative correlation between their responses to rewards [appetitive unconditioned stimulus (US)] and reward predictability. Some neurons exhibited their highest or lowest trace-period responses in the 50% reward trials. These response modulations were represented independently and overlaid on a consistent excitatory or inhibitory response across the conditioning events. LH neurons also showed consistent responses in the aversive context. However, the responses to aversive conditioning events depending on the air puff value and predictability were less common. The multifaceted modulation of consistent activity related to outcome predictions may reflect motivational and arousal signals. Furthermore, it may underlie the role the LH plays in the integration and relay of signals to cortices for adaptive and goal-directed physiological and behavioral responses to environmental changes.

NEW & NOTEWORTHY The lateral hypothalamus (LH) is implicated in motivational and arousal behavior; however, the detailed information carried by single LH neurons remains unclear. We demonstrate that primate LH neurons encode multiple combinations of signals concerning different degrees of expectation, appreciation, and uncertainty of rewards in consistent responses across conditioning events and between different contexts. This multifaceted modulation of activity may underlie the role of the LH as a critical node integrating motivational signals with arousal signals.

Can delay discounting deliver on the promise of RDoC?

The National Institute of Mental Health launched the Research Domain Criteria (RDoC) initiative to better understand dimensions of behavior and identify targets for treatment. Examining dimensions across psychiatric illnesses has proven challenging, as reliable behavioral paradigms that are known to engage specific neural circuits and translate across diagnostic populations are scarce. Delay discounting paradigms seem to be an exception: they are useful for understanding links between neural systems and behavior in healthy individuals, with potential for assessing how these mechanisms go awry in psychiatric illnesses. This article reviews relevant literature on delay discounting (or the rate at which the value of a reward decreases as the delay to receipt increases) in humans, including methods for examining it, its putative neural mechanisms, and its application in psychiatric research. There exist rigorous and reproducible paradigms to evaluate delay discounting, standard methods for calculating discount rate, and known neural systems probed by these paradigms. Abnormalities in discounting have been associated with psychopathology ranging from addiction (with steep discount rates indicating relative preference for immediate rewards) to anorexia nervosa (with shallow discount rates indicating preference for future rewards). The latest research suggests that delay discounting can be manipulated in the laboratory. Extensively studied in cognitive neuroscience, delay discounting assesses a dimension of behavior that is important for decision-making and is linked to neural substrates and to psychopathology. The question now is whether manipulating delay discounting can yield clinically significant changes in behavior that promote health. If so, then delay discounting could deliver on the RDoC promise.

5-HT1a Receptor Involvement in Temporal Memory and the Response to Temporal Ambiguity

It has previously been demonstrated that rats trained on the peak-interval procedure to associate two different cues with two different fixed interval schedules will generate a scalar peak function at an intermediate time when presented with the compound cue. This response pattern has been interpreted as resulting from the simultaneous retrieval of different temporal memories, and a consequential averaging process to resolve the ambiguity. In the present set of studies, we investigated the role that serotonin 1a receptors play in this process. In Experiment 1, rats were trained on a peak-interval procedure to associate the interoceptive states induced by saline and the 5-HT1a agonist, 8-OH-DPAT, with a 5 s or 20 s fixed-interval schedule signaled by the same tone cue (counter-balanced). While peak functions following administration of saline were centered at the appropriate time (5 s or 20 s), peak functions following administration of the agonist were centered around 7 s, irrespective of the reinforced time during training, suggesting agonist-induced disruption in selective temporal memory retrieval, resulting in increased ambiguity regarding the appropriate time at which to respond. In Experiment 2, rats were trained in a peak-interval procedure to associate a tone cue with a 10 s fixed interval and a light cue with a 20 s fixed interval. Administration of the 5-HT1a antagonist, WAY-100635, had no impact on timing when single cues were presented, but altered the intermediate, scalar, response to the stimulus compound, suggesting antagonist-induced disruption in the processes used to deal with temporal memory ambiguity. Together, these data suggest that manipulations of 5HT transmission at the 5-HT1a receptor cause changes in the temporal pattern of responding that are consistent with alterations in temporal memory processes and responses to temporal ambiguity.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

Neuro-computational account of how mood fluctuations arise and affect decision making

The influence of mood on choices is a well-established but poorly understood phenomenon. Here, we suggest a three-fold neuro-computational account: (1) the integration of positive and negative events over time induce mood fluctuations, (2) which are underpinned by variations in the baseline activities of critical brain valuation regions, (3) which in turn modulate the relative weights assigned to key dimensions of choice options. We validate this model in healthy participants, using feedback in a quiz task to induce mood fluctuations, and a choice task (accepting vs. declining a motor challenge) to reveal their effects. Using fMRI, we demonstrate the pivotal role of the ventromedial prefrontal cortex and anterior insula, in which baseline activities respectively increase and decrease with theoretical mood level and respectively enhance the weighting of potential gains and losses during decision making. The same mechanisms might explain how decisions are biased in mood disorders at longer timescales.

Opinion on monoaminergic contributions to traits and temperament

This article critically reviews evidence relating temperamental traits and personality factors to the monoamine neurotransmitters, especially dopamine and serotonin. The genetic evidence is not yet considered to be conclusive and it is argued that basic neuroscience research on the neural basis of behaviour in experimental animals should be taken more into account. While questionnaire and lexical methodology including the 'Five Factor' theory has been informative (mostly for the traits relevant to social functioning, i.e. personality), biologically oriented approaches should be employed with more objective, theoretically grounded measures of cognition and behaviour, combined with neuroimaging and psychopharmacology, where appropriate. This strategy will enable specific functions of monoamines and other neuromodulators such as acetylcholine and neuropeptides (such as orexin) to be defined with respect to their roles in modulating activity in specific neural networks-leading to a more realistic definition of their interactive roles in complex, biologically based traits (i.e. temperament).This article is part of the theme issue 'Diverse perspectives on diversity: multi-disciplinary approaches to taxonomies of individual differences'.

Activation of serotonin neurons promotes active persistence in a probabilistic foraging task

The neuromodulator serotonin (5-HT) has been implicated in a variety of functions that involve patience or impulse control. Many of these effects are consistent with a long-standing theory that 5-HT promotes behavioral inhibition, a motivational bias favoring passive over active behaviors. To further test this idea, we studied the impact of 5-HT in a probabilistic foraging task, in which mice must learn the statistics of the environment and infer when to leave a depleted foraging site for the next. Critically, mice were required to actively nose-poke in order to exploit a given site. We show that optogenetic activation of 5-HT neurons in the dorsal raphe nucleus increases the willingness of mice to actively attempt to exploit a reward site before giving up. These results indicate that behavioral inhibition is not an adequate description of 5-HT function and suggest that a unified account must be based on a higher-order function.

Intrinsic and Extrinsic Neuromodulation of Olfactory Processing

Neuromodulation is a ubiquitous feature of neural systems, allowing flexible, context specific control over network dynamics. Neuromodulation was first described in invertebrate motor systems and early work established a basic dichotomy for neuromodulation as having either an intrinsic origin (i.e., neurons that participate in network coding) or an extrinsic origin (i.e., neurons from independent networks). In this conceptual dichotomy, intrinsic sources of neuromodulation provide a “memory” by adjusting network dynamics based upon previous and ongoing activation of the network itself, while extrinsic neuromodulators provide the context of ongoing activity of other neural networks. Although this dichotomy has been thoroughly considered in motor systems, it has received far less attention in sensory systems. In this review, we discuss intrinsic and extrinsic modulation in the context of olfactory processing in invertebrate and vertebrate model systems. We begin by discussing presynaptic modulation of olfactory sensory neurons by local interneurons (LNs) as a mechanism for gain control based on ongoing network activation. We then discuss the cell-class specific effects of serotonergic centrifugal neurons on olfactory processing. Finally, we briefly discuss the integration of intrinsic and extrinsic neuromodulation (metamodulation) as an effective mechanism for exerting global control over olfactory network dynamics. The heterogeneous nature of neuromodulation is a recurring theme throughout this review as the effects of both intrinsic and extrinsic modulation are generally non-uniform.

Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments

Neuromodulatory systems, including the noradrenergic, serotonergic, dopaminergic, and cholinergic systems, track environmental signals, such as risks, rewards, novelty, effort, and social cooperation. These systems provide a foundation for cognitive function in higher organisms; attention, emotion, goal-directed behavior, and decision-making derive from the interaction between the neuromodulatory systems and brain areas, such as the amygdala, frontal cortex, hippocampus, and sensory cortices. Given their strong influence on behavior and cognition, these systems also play a key role in disease states and are the primary target of many current treatment strategies. The fact that these systems interact with each other either directly or indirectly, however, makes it difficult to understand how a failure in one or more systems can lead to a particular symptom or pathology. In this review, we explore experimental evidence, as well as focus on computational and theoretical models of neuromodulation. Better understanding of neuromodulatory systems may lead to the development of novel treatment strategies for a number of brain disorders.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

Learning and Stress Shape the Reward Response Patterns of Serotonin Neurons

The ability to predict reward promotes animal survival. Both dopamine neurons in the ventral tegmental area and serotonin neurons in the dorsal raphe nucleus (DRN) participate in reward processing. Although the learning effects on dopamine neurons have been extensively characterized, it remains largely unknown how the response of serotonin neurons evolves during learning. Moreover, although stress is known to strongly influence reward-related behavior, we know very little about how stress modulates neuronal reward responses. By monitoring Ca²⁺ signals during the entire process of Pavlovian conditioning, we here show that learning differentially shapes the response patterns of serotonin neurons and dopamine neurons in mice of either sex. Serotonin neurons gradually develop a slow ramp-up response to the reward-predicting cue, and ultimately remain responsive to the reward, whereas dopamine neurons increase their response to the cue but reduce their response to the reward. For both neuron types, the responses to the cue and the reward depend on reward value, are reversible when the reward is omitted, and are rapidly reinstated by restoring the reward. We also found that stressors including head restraint and fearful context substantially reduce the response strength of both neuron types, to both the cue and the reward. These results reveal the dynamic nature of the reward responses, support the hypothesis that DRN serotonin neurons signal the current likelihood of receiving a net benefit, and suggest that the inhibitory effect of stress on the reward responses of serotonin neurons and dopamine neurons may contribute to stress-induced anhedonia.SIGNIFICANCE STATEMENT Both serotonin neurons in the dorsal raphe and dopamine neurons in the ventral tegmental area are intimately involved in reward processing. Using long-term fiber photometry of Ca²⁺ signals from freely behaving mice, we here show that learning produces a ramp-up activation pattern in serotonin neurons that differs from that in dopamine neurons, indicating complementary roles for these two neuron types in reward processing. Moreover, stress treatment substantially reduces the reward responses of both serotonin neurons and dopamine neurons, suggesting a possible physiological basis for stress-induced anhedonia.

Fractionating impulsivity: neuropsychiatric implications

The ability to make decisions and act quickly without hesitation can be advantageous in many settings. However, when persistently expressed, impulsive decisions and actions are considered risky, maladaptive and symptomatic of such diverse brain disorders as attention-deficit hyperactivity disorder, drug addiction and affective disorders. Over the past decade, rapid progress has been made in the identification of discrete neural networks that underlie different forms of impulsivity - from impaired response inhibition and risky decision making to a profound intolerance of delayed rewards. Herein, we review what is currently known about the neural and psychological mechanisms of impulsivity, and discuss the relevance and application of these new insights to various neuropsychiatric disorders.

An emerging role for the lateral habenula in aggressive behavior

Inter-male aggression is an essential component of social behavior in organisms from insects to humans. However, when expressed inappropriately, aggression poses significant threats to the mental and physical health of both the aggressor and the target. Inappropriate aggression is a common feature of numerous neuropsychiatric disorders in humans and has been hypothesized to result from the atypical activation of reward circuitry in response to social targets. The lateral habenula (LHb) has recently been identified as a major node of the classical reward circuitry and inhibits the release of dopamine from the midbrain to signal negative valence. Here, we discuss the evidence linking LHb function to aggression and its valence, arguing that strong LHb outputs to the ventral tegmental area (VTA) and the dorsal raphe nucleus (DRN) are likely to play roles in aggression and its rewarding components. Future studies should aim to elucidate how various inputs and outputs of the LHb shape motivation and reward in the context of aggression.

Fatigue modulates dopamine availability and promotes flexible choice reversals during decision making

During decisions, animals balance goal achievement and effort management. Despite physical exercise and fatigue significantly affecting the levels of effort that an animal exerts to obtain a reward, their role in effort-based choice and the underlying neurochemistry are incompletely known. In particular, it is unclear whether fatigue influences decision (cost-benefit) strategies flexibly or only post-decision action execution and learning. To answer this question, we trained mice on a T-maze task in which they chose between a high-cost, high-reward arm (HR), which included a barrier, and a low-cost, low-reward arm (LR), with no barrier. The animals were parametrically fatigued immediately before the behavioural tasks by running on a treadmill. We report a sharp choice reversal, from the HR to LR arm, at 80% of their peak workload (PW), which was temporary and specific, as the mice returned to choose the HC when the animals were successively tested at 60% PW or in a two-barrier task. These rapid reversals are signatures of flexible choice. We also observed increased subcortical dopamine levels in fatigued mice: a marker of individual bias to use model-based control in humans. Our results indicate that fatigue levels can be incorporated in flexible cost-benefits computations that improve foraging efficiency.