Appetitive and aversive information coding in the primate dorsal raphé nucleus

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

Degeneracy and stability in neural circuits of dopamine and serotonin neuromodulators: A theoretical consideration

Degenerate neural circuits perform the same function despite being structurally different. However, it is unclear whether neural circuits with interacting neuromodulator sources can themselves degenerate while maintaining the same neuromodulatory function. Here, we address this by computationally modeling the neural circuits of neuromodulators serotonin and dopamine, local glutamatergic and GABAergic interneurons, and their possible interactions, under reward/punishment-based conditioning tasks. The neural modeling is constrained by relevant experimental studies of the VTA or DRN system using, e.g., electrophysiology, optogenetics, and voltammetry. We first show that a single parsimonious, sparsely connected neural circuit model can recapitulate several separate experimental findings that indicated diverse, heterogeneous, distributed, and mixed DRNVTA neuronal signaling in reward and punishment tasks. The inability of this model to recapitulate all observed neuronal signaling suggests potentially multiple circuits acting in parallel. Then using computational simulations and dynamical systems analysis, we demonstrate that several different stable circuit architectures can produce the same observed network activity profile, hence demonstrating degeneracy. Due to the extensive D2-mediated connections in the investigated circuits, we simulate the D2 receptor agonist by increasing the connection strengths emanating from the VTA DA neurons. We found that the simulated D2 agonist can distinguish among sub-groups of the degenerate neural circuits based on substantial deviations in specific neural populations' activities in reward and punishment conditions. This forms a testable model prediction using pharmacological means. Overall, this theoretical work suggests the plausibility of degeneracy within neuromodulator circuitry and has important implications for the stable and robust maintenance of neuromodulatory functions.

Inhibition of anandamide breakdown reduces pain and restores LTP and monoamine levels in the rat hippocampus via the CB1 receptor following osteoarthritis

Chronic pain is a persistent, complex condition that contributes to impaired mood, anxiety and emotional problems. Osteoarthritis (OA) is one of the major causes of chronic pain in adults and elderly people. A substantial body of evidence demonstrate that hippocampal neural circuits, especially monoamine dopamine and serotonin levels, contributes to negative affect and avoidance motivation experienced during pain. Current pharmacological strategies for OA patients are unsatisfying and the endocannabinoid system modulation might represent an alternative for the treatment of OA-related pain. In the present study, we used a rat model of osteoarthritis induced by intra-articular injection of sodium monoiodoacetate to assess, 28 days post-induction, the contribution of endocannabinoid system on the possible alteration in pain perception and affective behavior, in LTP and monoamine levels in the lateral entorhinal cortex-dentate gyrus pathway. The results show that OA-related chronic pain induces working memory impairment and depressive-like behavior appearance, diminishes LTP, decreases dopamine levels and increases serotonin levels in the rat dentate gyrus. URB597 administration (i.p., 1 mg/kg) reduces hyperalgesia and mechanical allodynia, improves recognition memory and depressive-live behavior, restores LTP and normalizes monoamine levels in the hippocampus. The effect was observed 60-120 min post-treatment and was blocked by AM251, which proves the action of URB597 via the CB₁ receptor. Therefore, our study confirms the role of anandamide in OA-related chronic pain management at the behavioral and hippocampal levels. This article is part of the Special Issue on 'Advances in mechanisms and therapeutic targets relevant to pain'.

Median raphe serotonergic neurons projecting to the interpeduncular nucleus control preference and aversion

Appropriate processing of reward and aversive information is essential for survival. Although a critical role of serotonergic neurons in the dorsal raphe nucleus (DRN) in reward processing has been shown, the lack of rewarding effects with selective serotonin reuptake inhibitors (SSRIs) implies the presence of a discrete serotonergic system playing an opposite role to the DRN in the processing of reward and aversive stimuli. Here, we demonstrated that serotonergic neurons in the median raphe nucleus (MRN) of mice process reward and aversive information in opposite directions to DRN serotonergic neurons. We further identified MRN serotonergic neurons, including those projecting to the interpeduncular nucleus (5-HT^MRN→IPN), as a key mediator of reward and aversive stimuli. Moreover, 5-HT receptors, including 5-HT_2A receptors in the interpeduncular nucleus, are involved in the aversive properties of MRN serotonergic neural activity. Our findings revealed an essential function of MRN serotonergic neurons, including 5-HT^MRN→IPN, in the processing of reward and aversive stimuli.

Single-cell activity and network properties of dorsal raphe nucleus serotonin neurons during emotionally salient behaviors

The serotonin system modulates a wide variety of emotional behaviors and states, including reward processing, anxiety, and social interaction. To reveal the underlying patterns of neural activity, we visualized serotonergic neurons in the dorsal raphe nucleus (DRN5-HT) of mice using miniaturized microscopy during diverse emotional behaviors. We discovered ensembles of cells with highly correlated activity and found that DRN5-HT neurons are preferentially recruited by emotionally salient stimuli as opposed to neutral stimuli. Individual DRN5-HT neurons responded to diverse combinations of salient stimuli, with some preference for valence and sensory modality. Anatomically defined subpopulations projecting to either a reward-related structure (the ventral tegmental area) or an anxiety-related structure (the bed nucleus of the stria terminalis) contained all response types but were enriched in reward- and anxiety-responsive cells, respectively. Our results suggest that the DRN serotonin system responds to emotional salience using ensembles with mixed selectivity and biases in downstream connectivity.

Medial and orbital frontal cortex in decision-making and flexible behavior

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Potassium Channel Conductance Is Involved in Phenylephrine-Induced Spontaneous Firing of Serotonergic Neurons in the Dorsal Raphe Nucleus

The serotonergic (5-HT) network from the dorsal raphe nucleus (DRN) of the brain has been demonstrated to regulate cognition, emotion, and behaviors, including learning and the sleep-wake cycle. Dysregulation of the activity of 5-HT neurons in the DRN is thought to play an important role in emotional disorders. The activity of 5-HT neurons is regulated by norepinephrine (NE) released from the projection terminals of noradrenergic input from the locus coeruleus (LC) via activation of the α1-adrenoceptor. However, insight into the molecular mechanism underlying this NE-induced regulation of 5-HT neuron activity is not clear. In this study, using the agonist of α1-adrenoceptor phenylephrine (PE), brain slices, and patch clamp, we found that A-type, Kv7/KCNQ, and calcium-activated low-conductance K+ channels (SK) underlie PE-induced spontaneous firing in DRN 5-HT neurons. Using single-cell PCR and immunofluorescence, we also identified the isoforms of these K+ channel families that might contribute to the NE/PE-induced spontaneous firing of DRN 5-HT neurons.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

MED1/BDNF/TrkB pathway is involved in thalamic hemorrhage-induced pain and depression by regulating microglia

Central post-stroke pain (CPSP) and associated depression remain poorly understood and pharmacological treatments are unsatisfactory. Recently, microglia activation was suggested to be involved in CPSP pathophysiology. The goal of this study was to investigate the effectiveness of a co-ultramicronized combination of N-palmitoylethanolamide and luteolin (PEALut) in a mouse model of thalamic hemorrhage (TH)-induced CPSP. TH was established through the collagenase-IV injection in thalamic ventral-posterolateral-nucleus. PEALut effects in CPSP-associated behaviors were evaluated during a 28-days observation period. We found that repeated administrations of co-ultra PEALut significantly reduced mechanical hypersensitivity after TH, as compared to vehicle, by reducing the early microglial activation in the perilesional site. Moreover, PEALut prevented the development of depressive-like behavior (21 days post-TH). These effects were associated with the restoration of synaptic plasticity in LEC-DG pathway and monoamines levels found impaired in TH mice. Hippocampal MED1 and TrkB expressions were significantly increased in TH compared to sham mice 21 days post-TH, whereas BDNF levels were decreased. PEALut restored MED1/TrkB/BDNF expression in mice. Remarkably, we found significant overexpression of MED1 in the human autoptic brain specimens after stroke, indicating a translational potential of our findings. These results pave the way for better-investigating depression in TH- induced CPSP, together with the involvement of MED1/TrkB/BDNF pathway, proposing PEALut as an adjuvant treatment.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the “brake” on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory “drive.” For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the “drive” and “brake” circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Complementary roles of serotonergic and cholinergic systems in decisions about when to act

Decision-making not only involves deciding about which action to choose but when and whether to initiate an action in the first place. Macaque monkeys tracked number of dots on a screen and could choose when to make a response. The longer the animals waited before responding, the more dots appeared on the screen and the higher the probability of reward. Monkeys waited longer before making a response when a trial’s value was less than the environment’s average value. Recordings of brain activity with fMRI revealed that activity in dorsal raphe nucleus (DRN)—a key source of serotonin (5-HT)—tracked average value of the environment. By contrast, activity in the basal forebrain (BF)—an important source of acetylcholine (ACh)—was related to decision time to act as a function of immediate and recent past context. Interactions between DRN and BF and the anterior cingulate cortex (ACC), another region with action initiation-related activity, occurred as a function of the decision time to act. Next, we performed two psychopharmacological studies. Manipulating systemic 5-HT by citalopram prolonged the time macaques waited to respond for a given opportunity. This effect was more evident during blocks with long inter-trial intervals (ITIs) where good opportunities were sparse. Manipulating systemic acetylcholine (ACh) by rivastigmine reduced the time macaques waited to respond given the immediate and recent past context, a pattern opposite to the effect observed with 5-HT. These findings suggest complementary roles for serotonin/DRN and acetylcholine/BF in decisions about when to initiate an action.

•

Both immediate context and wider environment influence decisions about when to act

•

DRN and 5-HT mediate the influence of wider environment

•

BF and ACh mediate the influence of immediate context

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Neuromodulation of prefrontal cortex cognitive function in primates: the powerful roles of monoamines and acetylcholine

The primate prefrontal cortex (PFC) subserves our highest order cognitive operations, and yet is tremendously dependent on a precise neurochemical environment for proper functioning. Depletion of noradrenaline and dopamine, or of acetylcholine from the dorsolateral PFC (dlPFC), is as devastating as removing the cortex itself, and serotonergic influences are also critical to proper functioning of the orbital and medial PFC. Most neuromodulators have a narrow inverted U dose response, which coordinates arousal state with cognitive state, and contributes to cognitive deficits with fatigue or uncontrollable stress. Studies in monkeys have revealed the molecular signaling mechanisms that govern the generation and modulation of mental representations by the dlPFC, allowing dynamic regulation of network strength, a process that requires tight regulation to prevent toxic actions, e.g., as occurs with advanced age. Brain imaging studies in humans have observed drug and genotype influences on a range of cognitive tasks and on PFC circuit functional connectivity, e.g., showing that catecholamines stabilize representations in a baseline-dependent manner. Research in monkeys has already led to new treatments for cognitive disorders in humans, encouraging future research in this important field.

Serotonergic regulation of the dopaminergic system: Implications for reward-related functions

Serotonin is a critical neuromodulator involved in development and behavior. Its role in reward is however still debated. Here, we first review classical studies involving electrical stimulation protocols and pharmacological approaches. Contradictory results on the serotonergic' involvement in reward emerge from these studies. These differences might be ascribable to either the diversity of cellular types within the raphe nuclei or/and the specific projection pathways of serotonergic neurons. We continue to review more recent work, using optogenetic approaches to activate serotonergic cells in the Raphe to VTA pathway. From these studies, it appears that activation of this pathway can lead to reinforcement learning mediated through the excitation of dopaminergic neurons by serotonergic neurons co-transmitting glutamate. Finally, given the importance of serotonin during development on adult emotion, the effect of abnormal early-life levels of serotonin on the dopaminergic system will also be discussed. Understanding the interaction between the serotonergic and dopaminergic systems during development and adulthood is critical to gain insight into the specific facets of neuropsychiatric disorders.

Rhythms, Reward, and Blues: Consequences of Circadian Photoperiod on Affective and Reward Circuit Function

Circadian disruptions, along with altered affective and reward states, are commonly associated with psychiatric disorders. In addition to genetics, the enduring influence of environmental factors in programming neural networks is of increased interest in assessing the underpinnings of mental health. The duration of daylight or photoperiod is known to impact both the serotonin and dopamine systems, which are implicated in mood and reward-based disorders. This review first examines the effects of circadian disruption and photoperiod in the serotonin system in both human and preclinical studies. We next highlight how brain regions crucial for the serotoninergic system (i.e., dorsal raphe nucleus; DRN), and dopaminergic (i.e., nucleus accumbens; NAc and ventral tegmental area; VTA) system are intertwined in overlapping circuitry, and play influential roles in the pathology of mood and reward-based disorders. We then focus on human and animal studies that demonstrate the impact of circadian factors on the dopaminergic system. Lastly, we discuss how environmental factors such as circadian photoperiod can impact the neural circuits that are responsible for regulating affective and reward states, offering novel insights into the biological mechanisms underlying the pathophysiology, systems, and therapeutic treatments necessary for mood and reward-based disorders.

The Role of Dorsal Raphe Serotonin Neurons in the Balance between Reward and Aversion

BACKGROUND

Reward processing is fundamental for animals to survive and reproduce. Many studies have shown the importance of dorsal raphe nucleus (DRN) serotonin (5-HT) neurons in this process, but the strongly correlative link between the activity of DRN 5-HT neurons and rewarding/aversive potency is under debate. Our primary objective was to reveal this link using two different strategies to transduce DRN 5-HT neurons.

METHODS

For transduction of 5-HT neurons in wildtype mice, adeno-associated virus (AAV) bearing the mouse tryptophan hydroxylase 2 (TPH2) gene promoter was used. For transduction in Tph2-tTA transgenic mice, AAVs bearing the tTA-dependent TetO enhancer were used. To manipulate the activity of 5-HT neurons, optogenetic actuators (CheRiff, eArchT) were expressed by AAVs. For measurement of rewarding/aversive potency, we performed a nose-poke self-stimulation test and conditioned place preference (CPP) test.

RESULTS

We found that stimulation of DRN 5-HT neurons and their projections to the ventral tegmental area (VTA) increased the number of nose-pokes in self-stimulation test and CPP scores in both targeting methods. Concomitantly, CPP scores were decreased by inhibition of DRN 5-HT neurons and their projections to VTA.

CONCLUSION

Our findings indicate that the activity of DRN 5-HT neurons projecting to the VTA is a key modulator of balance between reward and aversion.

Molecular and anatomical organization of the dorsal raphe nucleus

The dorsal raphe nucleus (DRN) is an important source of neuromodulators and has been implicated in a wide variety of behavioral and neurological disorders. The DRN is subdivided into distinct anatomical subregions comprised of multiple cell types, and its complex cellular organization has impeded efforts to investigate the distinct circuit and behavioral functions of its subdomains. Here we used single-cell RNA sequencing, in situ hybridization, anatomical tracing, and spatial correlation analysis to map the transcriptional and spatial profiles of cells from the mouse DRN. Our analysis of 39,411 single-cell transcriptomes revealed at least 18 distinct neuron subtypes and 5 serotonergic neuron subtypes with distinct molecular and anatomical properties, including a serotonergic neuron subtype that preferentially innervates the basal ganglia. Our study lays out the molecular organization of distinct serotonergic and non-serotonergic subsystems, and will facilitate the design of strategies for further dissection of the DRN and its diverse functions.

Modulation of anterior cingulate cortex reward and penalty signalling in medication-naive young-adult subjects with depressive symptoms following acute dose lurasidone

BACKGROUND

Aberrations in reward and penalty processing are implicated in depression and putatively reflect altered dopamine signalling. This study exploits the advantages of a placebo-controlled design to examine how a novel D2 antagonist with adjunctive antidepressant properties modifies activity in the brain's reward network in depression.

METHODS

We recruited 43 medication-naïve subjects across the range of depression severity (Beck's Depression Inventory-II score range: 0-43), including healthy volunteers, as well as people meeting full-criteria for major depressive disorder. In a double-blind placebo-controlled cross-over design, all subjects received either placebo or lurasidone (20 mg) across two visits separated by 1 week. Functional magnetic resonance imaging with the Monetary Incentive Delay (MID) task assessed reward functions via neural responses during anticipation and receipt of gains and losses. Arterial spin labelling measured cerebral blood flow (CBF) at rest.

RESULTS

Lurasidone altered fronto-striatal activity during anticipation and outcome phases of the MID task. A significant three-way Medication-by-Depression severity-by-Outcome interaction emerged in the anterior cingulate cortex (ACC) after correction for multiple comparisons. Follow-up analyses revealed significantly higher ACC activation to losses in high- v. low depression participants in the placebo condition, with a normalisation by lurasidone. This effect could not be accounted for by shifts in resting CBF.

CONCLUSIONS

Lurasidone acutely normalises reward processing signals in individuals with depressive symptoms. Lurasidone's antidepressant effects may arise from reducing responses to penalty outcomes in individuals with depressive symptoms.

Manipulation of dorsal raphe serotonergic neurons modulates active coping to inescapable stress and anxiety-related behaviors in mice and rats

Major depression and anxiety disorders are a social and economic burden worldwide. Serotonergic signaling has been implicated in the pathophysiology of these disorders and thus has been a crucial target for pharmacotherapy. However, the precise mechanisms underlying these disorders are still unclear. Here, we used species-optimized lentiviral vectors that were capable of efficient and specific transduction of serotonergic neurons in mice and rats for elucidation of serotonergic roles in anxiety-like behaviors and active coping behavior in both species. Immunohistochemical analyses revealed that lentiviral vectors with an upstream sequence of tryptophan hydroxylase 2 gene efficiently transduced serotonergic neurons with a specificity of approximately 95% in both mice and rats. Electrophysiological recordings showed that these lentiviral vectors induced sufficient expression of optogenetic tools for precise control of serotonergic neurons. Using these vectors, we demonstrate that acute activation of serotonergic neurons in the dorsal raphe nucleus increases active coping with inescapable stress in rats and mice in a time-locked manner, and that acute inhibition of these neurons increases anxiety-like behaviors specifically in rats. These findings further our understanding of the pathophysiological role of dorsal raphe serotonergic neurons in different species and the role of these neurons as therapeutic targets in major depression and anxiety disorders.

Encoding prediction signals during appetitive and aversive Pavlovian conditioning in the primate lateral hypothalamus

The lateral hypothalamus (LH), which plays a role in homeostatic functions such as appetite regulation, is also linked to arousal and motivational behavior. However, little is known about how these components are encoded in the LH. Thus cynomolgus monkeys were conditioned with two distinct contexts, i.e., an appetitive context with available rewards and an aversive context with predicted air puffs. Different LH neuron groups encoded different degrees of expectation, predictability, and risks of rewards in a specific manner. A nearly equal number of one-third of the recorded LH neurons showed a positive or negative correlation between their response to visual conditioned stimuli (CS) that predicted the probabilistic delivery of rewards (0%, 50%, and 100%) and the associative values. For another one-third of recorded neurons, a nearly equal number showed a positive or negative correlation between their responses to rewards [appetitive unconditioned stimulus (US)] and reward predictability. Some neurons exhibited their highest or lowest trace-period responses in the 50% reward trials. These response modulations were represented independently and overlaid on a consistent excitatory or inhibitory response across the conditioning events. LH neurons also showed consistent responses in the aversive context. However, the responses to aversive conditioning events depending on the air puff value and predictability were less common. The multifaceted modulation of consistent activity related to outcome predictions may reflect motivational and arousal signals. Furthermore, it may underlie the role the LH plays in the integration and relay of signals to cortices for adaptive and goal-directed physiological and behavioral responses to environmental changes.

NEW & NOTEWORTHY The lateral hypothalamus (LH) is implicated in motivational and arousal behavior; however, the detailed information carried by single LH neurons remains unclear. We demonstrate that primate LH neurons encode multiple combinations of signals concerning different degrees of expectation, appreciation, and uncertainty of rewards in consistent responses across conditioning events and between different contexts. This multifaceted modulation of activity may underlie the role of the LH as a critical node integrating motivational signals with arousal signals.

5-HT release in nucleus accumbens rescues social deficits in mouse autism model

Dysfunction in prosocial interactions is a core symptom of autism spectrum disorder. However, the neural mechanisms that underlie sociability are poorly understood, limiting the rational development of therapies to treat social deficits. Here we show in mice that bidirectional modulation of the release of serotonin (5-HT) from dorsal raphe neurons in the nucleus accumbens bidirectionally modifies sociability. In a mouse model of a common genetic cause of autism spectrum disorder-a copy number variation on chromosome 16p11.2-genetic deletion of the syntenic region from 5-HT neurons induces deficits in social behaviour and decreases dorsal raphe 5-HT neuronal activity. These sociability deficits can be rescued by optogenetic activation of dorsal raphe 5-HT neurons, an effect requiring and mimicked by activation of 5-HT1b receptors in the nucleus accumbens. These results demonstrate an unexpected role for 5-HT action in the nucleus accumbens in social behaviours, and suggest that targeting this mechanism may prove therapeutically beneficial.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

Intrinsic and Extrinsic Neuromodulation of Olfactory Processing

Neuromodulation is a ubiquitous feature of neural systems, allowing flexible, context specific control over network dynamics. Neuromodulation was first described in invertebrate motor systems and early work established a basic dichotomy for neuromodulation as having either an intrinsic origin (i.e., neurons that participate in network coding) or an extrinsic origin (i.e., neurons from independent networks). In this conceptual dichotomy, intrinsic sources of neuromodulation provide a “memory” by adjusting network dynamics based upon previous and ongoing activation of the network itself, while extrinsic neuromodulators provide the context of ongoing activity of other neural networks. Although this dichotomy has been thoroughly considered in motor systems, it has received far less attention in sensory systems. In this review, we discuss intrinsic and extrinsic modulation in the context of olfactory processing in invertebrate and vertebrate model systems. We begin by discussing presynaptic modulation of olfactory sensory neurons by local interneurons (LNs) as a mechanism for gain control based on ongoing network activation. We then discuss the cell-class specific effects of serotonergic centrifugal neurons on olfactory processing. Finally, we briefly discuss the integration of intrinsic and extrinsic neuromodulation (metamodulation) as an effective mechanism for exerting global control over olfactory network dynamics. The heterogeneous nature of neuromodulation is a recurring theme throughout this review as the effects of both intrinsic and extrinsic modulation are generally non-uniform.

Serotonin Decreases the Gain of Visual Responses in Awake Macaque V1

Serotonin, an important neuromodulator in the brain, is implicated in affective and cognitive functions. However, its role even for basic cortical processes is controversial. For example, in the mammalian primary visual cortex (V1), heterogenous serotonergic modulation has been observed in anesthetized animals. Here, we combined extracellular single-unit recordings with iontophoresis in awake animals. We examined the role of serotonin on well-defined tuning properties (orientation, spatial frequency, contrast, and size) in V1 of two male macaque monkeys. We find that in the awake macaque the modulatory effect of serotonin is surprisingly uniform: it causes a mainly multiplicative decrease of the visual responses and a slight increase in the stimulus-selective response latency. Moreover, serotonin neither systematically changes the selectivity or variability of the response, nor the interneuronal correlation unexplained by the stimulus ("noise-correlation"). The modulation by serotonin has qualitative similarities with that for a decrease in stimulus contrast, but differs quantitatively from decreasing contrast. It can be captured by a simple additive change to a threshold-linear spiking nonlinearity. Together, our results show that serotonin is well suited to control the response gain of neurons in V1 depending on the animal's behavioral or motivational context, complementing other known state-dependent gain-control mechanisms.SIGNIFICANCE STATEMENT Serotonin is an important neuromodulator in the brain and a major target for drugs used to treat psychiatric disorders. Nonetheless, surprisingly little is known about how it shapes information processing in sensory areas. Here we examined the serotonergic modulation of visual processing in the primary visual cortex of awake behaving macaque monkeys. We found that serotonin mainly decreased the gain of the visual responses, without systematically changing their selectivity, variability, or covariability. This identifies a simple computational function of serotonin for state-dependent sensory processing, depending on the animal's affective or motivational state.

An Update on the Role of Serotonin and its Interplay with Dopamine for Reward

The specific role of serotonin and its interplay with dopamine (DA) in adaptive, reward guided behavior as well as drug dependance, still remains elusive. Recently, novel methods allowed cell type specific anatomical, functional and interventional analyses of serotonergic and dopaminergic circuits, promising significant advancement in understanding their functional roles. Furthermore, it is increasingly recognized that co-release of neurotransmitters is functionally relevant, understanding of which is required in order to interpret results of pharmacological studies and their relationship to neural recordings. Here, we review recent animal studies employing such techniques with the aim to connect their results to effects observed in human pharmacological studies and subjective effects of drugs. It appears that the additive effect of serotonin and DA conveys significant reward related information and is subjectively highly euphorizing. Neither DA nor serotonin alone have such an effect. This coincides with optogenetically targeted recordings in mice, where the dopaminergic system codes reward prediction errors (PE), and the serotonergic system mainly unsigned PE. Overall, this pattern of results indicates that joint activity between both systems carries essential reward information and invites parallel investigation of both neurotransmitter systems.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

Learning and Stress Shape the Reward Response Patterns of Serotonin Neurons

The ability to predict reward promotes animal survival. Both dopamine neurons in the ventral tegmental area and serotonin neurons in the dorsal raphe nucleus (DRN) participate in reward processing. Although the learning effects on dopamine neurons have been extensively characterized, it remains largely unknown how the response of serotonin neurons evolves during learning. Moreover, although stress is known to strongly influence reward-related behavior, we know very little about how stress modulates neuronal reward responses. By monitoring Ca²⁺ signals during the entire process of Pavlovian conditioning, we here show that learning differentially shapes the response patterns of serotonin neurons and dopamine neurons in mice of either sex. Serotonin neurons gradually develop a slow ramp-up response to the reward-predicting cue, and ultimately remain responsive to the reward, whereas dopamine neurons increase their response to the cue but reduce their response to the reward. For both neuron types, the responses to the cue and the reward depend on reward value, are reversible when the reward is omitted, and are rapidly reinstated by restoring the reward. We also found that stressors including head restraint and fearful context substantially reduce the response strength of both neuron types, to both the cue and the reward. These results reveal the dynamic nature of the reward responses, support the hypothesis that DRN serotonin neurons signal the current likelihood of receiving a net benefit, and suggest that the inhibitory effect of stress on the reward responses of serotonin neurons and dopamine neurons may contribute to stress-induced anhedonia.SIGNIFICANCE STATEMENT Both serotonin neurons in the dorsal raphe and dopamine neurons in the ventral tegmental area are intimately involved in reward processing. Using long-term fiber photometry of Ca²⁺ signals from freely behaving mice, we here show that learning produces a ramp-up activation pattern in serotonin neurons that differs from that in dopamine neurons, indicating complementary roles for these two neuron types in reward processing. Moreover, stress treatment substantially reduces the reward responses of both serotonin neurons and dopamine neurons, suggesting a possible physiological basis for stress-induced anhedonia.

Information processing in the vertebrate habenula

The habenula is a brain region that has gained increasing popularity over the recent years due to its role in processing value-related and experience-dependent information with a strong link to depression, addiction, sleep and social interactions. This small diencephalic nucleus is proposed to act as a multimodal hub or a switchboard, where inputs from different brain regions converge. These diverse inputs to the habenula carry information about the sensory world and the animal's internal state, such as reward expectation or mood. However, it is not clear how these diverse habenular inputs interact with each other and how such interactions contribute to the function of habenular circuits in regulating behavioral responses in various tasks and contexts. In this review, we aim to discuss how information processing in habenular circuits, can contribute to specific behavioral programs that are attributed to the habenula.

Anterior cingulate is a source of valence-specific information about value and uncertainty

Anterior cingulate cortex (ACC) is thought to control a wide range of reward, punishment, and uncertainty-related behaviors. However, how it does so is unclear. Here, in a Pavlovian procedure in which monkeys displayed a diverse repertoire of reward-related, punishment-related, and uncertainty-related behaviors, we show that many ACC-neurons represent expected value and uncertainty in a valence-specific manner, signaling value or uncertainty predictions about either rewards or punishments. Other ACC-neurons signal prediction information about rewards and punishments by displaying excitation to both (rather than excitation to one and inhibition to the other). This diversity in valence representations may support the role of ACC in many behavioral states that are either enhanced by reward and punishment (e.g., vigilance) or specific to either reward or punishment (e.g., approach and avoidance). Also, this first demonstration of punishment-uncertainty signals in the brain suggests that ACC could be a target for the treatment of uncertainty-related disorders of mood.

Rewards or punishments elicit diverse behavioral responses; however, the neural circuits underlying such flexibility are unclear. Here Monosov shows that this diversity could be supported by neurons in the anterior cingulate that represent expected value and uncertainty in a valence-specific manner.

Activity patterns of serotonin neurons underlying cognitive flexibility

Serotonin is implicated in mood and affective disorders. However, growing evidence suggests that a core endogenous role is to promote flexible adaptation to changes in the causal structure of the environment, through behavioral inhibition and enhanced plasticity. We used long-term photometric recordings in mice to study a population of dorsal raphe serotonin neurons, whose activity we could link to normal reversal learning using pharmacogenetics. We found that these neurons are activated by both positive and negative prediction errors, and thus report signals similar to those proposed to promote learning in conditions of uncertainty. Furthermore, by comparing the cue responses of serotonin and dopamine neurons, we found differences in learning rates that could explain the importance of serotonin in inhibiting perseverative responding. Our findings show how the activity patterns of serotonin neurons support a role in cognitive flexibility, and suggest a revised model of dopamine-serotonin opponency with potential clinical implications.

Can the Lateral Habenula Crack the Serotonin Code?

The lateral habenula (LHb) and the serotonergic system both contribute to motivational states by encoding rewarding and aversive signals. Converging evidence suggests that perturbation of these systems is critical for the pathophysiology of mood disorders. Anatomical and functional studies indicate that the serotonergic system and the LHb are interconnected in a forward-feedback loop. However, how serotonin release modifies the synaptic and cellular properties of LHb neurons and whether this has any behavioral repercussions remain poorly investigated. In this review article, we discuss insights gained from rodents and humans regarding the implications of the serotonin system and the LHb in aversion encoding and related disorders. We then describe the type, properties and pharmacology of serotonergic receptors expressed throughout the LHb. Finally, we discuss physiological data reporting how serotonergic signaling modifies synaptic transmission and neuronal activity within the LHb. Altogether, we combine a mechanistic- and circuit-level knowledge to provide an overview on how the LHb integrates serotonergic signals, a process potentially contributing to LHb-dependent encoding of valenced external stimuli.

Neurons in the primate dorsal striatum signal the uncertainty of object-reward associations

To learn, obtain reward and survive, humans and other animals must monitor, approach and act on objects that are associated with variable or unknown rewards. However, the neuronal mechanisms that mediate behaviours aimed at uncertain objects are poorly understood. Here we demonstrate that a set of neurons in an internal-capsule bordering regions of the primate dorsal striatum, within the putamen and caudate nucleus, signal the uncertainty of object–reward associations. Their uncertainty responses depend on the presence of objects associated with reward uncertainty and evolve rapidly as monkeys learn novel object–reward associations. Therefore, beyond its established role in mediating actions aimed at known or certain rewards, the dorsal striatum also participates in behaviours aimed at reward-uncertain objects.

The dorsal striatum (DS) is a brain region that is thought to aim actions at certain or known rewards. Here, the authors show that an internal-capsule bordering region of the primate DS signals the uncertainty of object-reward associations, suggesting a novel role for the DS in behavior under uncertainty.

Activation and inhibition of tph2 serotonergic neurons operate in tandem to influence larval zebrafish preference for light over darkness

Serotonergic neurons have been implicated in a broad range of processes, but the principles underlying their effects remain a puzzle. Here, we ask how these neurons influence the tendency of larval zebrafish to swim in the light and avoid regions of darkness. Pharmacological inhibition of serotonin synthesis reduces dark avoidance, indicating an involvement of this neuromodulator. Calcium imaging of tph2-expressing cells demonstrates that a rostral subset of dorsal raphe serotonergic neurons fire continuously while the animal is in darkness, but are inhibited in the light. Optogenetic manipulation of tph2 neurons by channelrhodopsin or halorhodopsin expression modifies preference, confirming a role for these neurons. In particular, these results suggest that fish prefer swimming in conditions that elicits lower activity in tph2 serotonergic neurons in the rostral raphe.

Serotonin neurons in the dorsal raphe nucleus encode reward signals

The dorsal raphe nucleus (DRN) is involved in organizing reward-related behaviours; however, it remains unclear how genetically defined neurons in the DRN of a freely behaving animal respond to various natural rewards. Here we addressed this question using fibre photometry and single-unit recording from serotonin (5-HT) neurons and GABA neurons in the DRN of behaving mice. Rewards including sucrose, food, sex and social interaction rapidly activate 5-HT neurons, but aversive stimuli including quinine and footshock do not. Both expected and unexpected rewards activate 5-HT neurons. After mice learn to wait for sucrose delivery, most 5-HT neurons fire tonically during waiting and then phasically on reward acquisition. Finally, GABA neurons are activated by aversive stimuli but inhibited when mice seek rewards. Thus, DRN 5-HT neurons positively encode a wide range of reward signals during anticipatory and consummatory phases of reward responses. Moreover, GABA neurons play a complementary role in reward processing.

Ascending serotonin neuron diversity under two umbrellas

Forebrain serotonin relevant for many psychological disorders arises in the hindbrain, primarily within the dorsal and median raphe nuclei (DR and MR). These nuclei are heterogeneous, containing several distinct groups of serotonin neurons. Here, new insight into the afferent and efferent connectivity of these areas is reviewed in correlation with their developmental origin. These data suggest that the caudal third of the DR, the area originally designated B6, may be misidentified as part of the DR as it shares many features of connectivity with the MR. By considering the rostral DR independently and affiliating the B6 to the MR, the diverse subgroups of serotonin neurons can be arranged with more coherence into two umbrella groups, each with distinctive domains of influence. Serotonin neurons within the rostral DR are uniquely interconnected with brain areas associated with emotion and motivation such as the amygdala, accumbens and ventral pallidum. In contrast serotonin neurons in the B6 and MR are characterized by their dominion over the septum and hippocampus. This distinction between the DR and B6/MR parallels their developmental origin and likely impacts their role in both behavior and psychopathology. Implications and further subdivisions within these areas are discussed.

Reward processing by the dorsal raphe nucleus: 5-HT and beyond

The dorsal raphe nucleus (DRN) represents one of the most sensitive reward sites in the brain. However, the exact relationship between DRN neuronal activity and reward signaling has been elusive. In this review, we will summarize anatomical, pharmacological, optogenetics, and electrophysiological studies on the functions and circuit mechanisms of DRN neurons in reward processing. The DRN is commonly associated with serotonin (5-hydroxytryptamine; 5-HT), but this nucleus also contains neurons of the neurotransmitter phenotypes of glutamate, GABA and dopamine. Pharmacological studies indicate that 5-HT might be involved in modulating reward- or punishment-related behaviors. Recent optogenetic stimulations demonstrate that transient activation of DRN neurons produces strong reinforcement signals that are carried out primarily by glutamate. Moreover, activation of DRN 5-HT neurons enhances reward waiting. Electrophysiological recordings reveal that the activity of DRN neurons exhibits diverse behavioral correlates in reward-related tasks. Studies so far thus demonstrate the strong power of DRN neurons in reward signaling and at the same time invite additional efforts to dissect the roles and mechanisms of different DRN neuron types in various processes of reward-related behaviors.