Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dorsal raphe to basolateral amygdala corticotropin-releasing factor circuit regulates cocaine-memory reconsolidation

Environmental stimuli elicit drug craving and relapse in cocaine users by triggering the retrieval of strong cocaine-related contextual memories. Retrieval can also destabilize drug memories, requiring reconsolidation, a protein synthesis-dependent storage process, to maintain memory strength. Corticotropin-releasing factor (CRF) signaling in the basolateral amygdala (BLA) is necessary for cocaine-memory reconsolidation. We have hypothesized that a critical source of CRF in the BLA is the dorsal raphe nucleus (DR) based on its neurochemistry, anatomical connectivity, and requisite involvement in cocaine-memory reconsolidation. To test this hypothesis, male and female Sprague-Dawley rats received adeno-associated viruses to express Gi-coupled designer receptors exclusively activated by designer drugs (DREADDs) selectively in CRF neurons of the DR and injection cannulae directed at the BLA. The rats were trained to self-administer cocaine in a distinct environmental context then received extinction training in a different context. Next, they were briefly re-exposed to the cocaine-predictive context to destabilize (reactivate) cocaine memories. Intra-BLA infusions of the DREADD agonist deschloroclozapine (DCZ; 0.1 mM, 0.5 µL/hemisphere) immediately after memory reactivation attenuated cocaine-memory strength, relative to vehicle infusion. This was indicated by a selective, DCZ-induced and memory reactivation-dependent decrease in drug-seeking behavior in the cocaine-predictive context in DREADD-expressing males and females at test compared to respective controls. Notably, BLA-projecting DR CRF neurons that exhibited increased c-Fos expression during memory reconsolidation co-expressed the glutamatergic neuronal marker, vesicular glutamate transporter 3. Together, these findings suggest that the DRCRF → BLA circuit is engaged to maintain cocaine-memory strength after memory destabilization, and this phenomenon may be mediated by DR CRF and/or glutamate release in the BLA.

The lateral habenula: A hub for value-guided behavior

The habenula is an evolutionarily highly conserved diencephalic brain region divided into two major parts, medial and lateral. Over the past two decades, studies of the lateral habenula (LHb), in particular, have identified key functions in value-guided behavior in health and disease. In this review, we focus on recent insights into LHb connectivity and its functional relevance for different types of aversive and appetitive value-guided behavior. First, we give an overview of the anatomical organization of the LHb and its main cellular composition. Next, we elaborate on how distinct LHb neuronal subpopulations encode aversive and appetitive stimuli and on their involvement in more complex decision-making processes. Finally, we scrutinize the afferent and efferent connections of the LHb and discuss their functional implications for LHb-dependent behavior. A deepened understanding of distinct LHb circuit components will substantially contribute to our knowledge of value-guided behavior.

Requisite role of dorsal raphé in contextual cocaine-memory reconsolidation

Memory reconsolidation is a process by which labile drug memories are restabilized in long-term memory stores, permitting their enduring control over drug-seeking behaviors. In the present study, we investigated the involvement of the dorsal raphé nuclei (DRN) in cocaine-memory reconsolidation. Sprague-Dawley rats (male, female) were trained to self-administer cocaine in a distinct environmental context to establish contextual drug memories. They then received extinction training in a different context. Next, the rats were re-exposed to the cocaine-predictive context for 15 min to reactivate their cocaine memories or remained in their home cages (no-reactivation control). Memory reactivation was sufficient to increase c-Fos expression, an index of neuronal activation, in the DRN, but not in the median raphé nuclei, during reconsolidation, compared to no reactivation. To determine whether DRN neuronal activity was necessary for cocaine-memory reconsolidation, rats received intra-DRN baclofen plus muscimol (BM; GABAB/A agonists) or vehicle microinfusions immediately after or 6 h after a memory reactivation session conducted with or without lever access. The effects of DRN functional inactivation on long-term memory strength, as indicated by the magnitude of context-induced cocaine seeking, were assessed 72 h later. Intra-DRN BM treatment immediately after memory reactivation with or without lever access attenuated subsequent context-induced cocaine-seeking behavior, independent of sex. Conversely, BM treatment in the adjacent periaqueductal gray (PAG) immediately after memory reactivation, or BM treatment in the DRN 6 h after memory reactivation, did not alter responding. Together, these findings indicate that the DRN plays a requisite role in maintaining cocaine-memory strength during reconsolidation.

Dorsal Raphe to Basolateral Amygdala Corticotropin-Releasing Factor Circuit Regulates Cocaine-Memory Reconsolidation

Environmental stimuli elicit drug craving and relapse in cocaine users by triggering the retrieval of strong cocainerelated contextual memories. Retrieval can also destabilize drug memories, requiring reconsolidation, a protein synthesis-dependent storage process, to maintain memory strength. Corticotropin-releasing factor (CRF) signaling in the basolateral amygdala (BLA) is necessary for cocainememory reconsolidation. We have hypothesized that a critical source of CRF in the BLA is the dorsal raphe nucleus (DR) based on its neurochemistry, anatomical connectivity, and requisite involvement in cocaine-memory reconsolidation. To test this hypothesis, male and female Sprague-Dawley rats received adeno-associated viruses to express Gi-coupled designer receptors exclusively activated by designer drugs (DREADDs) selectively in CRF neurons of the DR and injection cannulae directed at the BLA. The rats were trained to self-administer cocaine in a distinct environmental context then received extinction training in a different context. They were then briefly reexposed to the cocaine-predictive context to destabilize (reactivate) cocaine memories. Intra-BLA infusions of the DREADD agonist deschloroclozapine (DCZ; 0.1 mM, 0.5 μL/hemisphere) after memory reactivation attenuated cocaine-memory strength, relative to vehicle infusion. This was indicated by a selective, DCZ-induced and memory reactivation-dependent decrease in drug-seeking behavior in the cocaine-predictive context in DREADD-expressing males and females at test compared to respective controls. Notably, BLA-projecting DR CRF neurons that exhibited increased c-Fos expression during memory reconsolidation co-expressed glutamatergic and serotonergic neuronal markers. Together, these findings suggest that the DRCRF → BLA circuit is engaged to maintain cocaine-memory strength after memory destabilization, and this phenomenon may be mediated by DR CRF, glutamate, and/or serotonin release in the BLA.

Apathy and Motivation: Biological Basis and Drug Treatment

Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.

Serotonin neurons in mating female mice are activated by male ejaculation

Sexual stimulation triggers changes in female physiology and behavior, including sexual satiety and preparing the uterus for pregnancy. Serotonin (5-HT) is an important regulator of reproductive physiology and sexual receptivity, but the relationship between sexual stimulation and 5-HT neural activity in females is poorly understood. Here, we investigated dorsal raphe 5-HT neural activity in female mice during sexual behavior. We found that 5-HT neural activity in mating females peaked specifically upon male ejaculation and remained elevated above baseline until disengagement. Artificial intravaginal mechanical stimulation was sufficient to elicit increased 5-HT neural activity but the delivery of ejaculatory fluids was not. Distal penis expansion ("penile cupping") at ejaculation and forceful expulsion of ejaculatory fluid each provided sufficient mechanical stimulation to elicit 5-HT neuron activation. Our study identifies a female ejaculation-specific signal in a major neuromodulatory system and shows that intravaginal mechanosensory stimulation is necessary and sufficient to drive this signal.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Serotonin neurons in mating female mice are activated by male ejaculation

Sexual stimulation triggers changes in female physiology and behavior, including sexual satiety and preparing the uterus for pregnancy. Serotonin is an important regulator of reproductive physiology and sexual receptivity, but the relationship between sexual stimulation and serotonin neural activity in females is poorly understood. Here, we investigated dorsal raphe serotonin neural activity in females during sexual behavior. We found that serotonin neural activity in mating females peaked specifically upon male ejaculation, and remained elevated above baseline until disengagement. Artificial intravaginal mechanical stimulation was sufficient to elicit increased 5-HT neural activity but the delivery of ejaculatory fluids was not. Distal penis erectile enlargement ("penile cupping") at ejaculation and forceful expulsion of ejaculatory fluid each provided sufficient mechanical stimulation to elicit serotonin neuron activation. Our study identifies a female ejaculation-specific signal in a major neuromodulatory system and shows that intravaginal mechanosensory stimulation is necessary and sufficient to drive this signal.

Medial Prefrontal Cortex Serotonin Input Regulates Cognitive Flexibility in Mice

The medial prefrontal cortex (mPFC) regulates cognitive flexibility and emotional behavior. Neurons that release serotonin project to the mPFC, and serotonergic drugs influence emotion and cognition. Yet, the specific roles of endogenous serotonin release in the mPFC on neurophysiology and behavior are unknown. We show that axonal serotonin release in the mPFC directly inhibits the major mPFC output neurons. In serotonergic neurons projecting from the dorsal raphe to the mPFC, we find endogenous activity signatures pre-reward retrieval and at reward retrieval during a cognitive flexibility task. In vivo optogenetic activation of this pathway during pre-reward retrieval selectively improved extradimensional rule shift performance while inhibition impaired it, demonstrating sufficiency and necessity for mPFC serotonin release in cognitive flexibility. Locomotor activity and anxiety-like behavior were not affected by either optogenetic manipulation. Collectively, our data reveal a powerful and specific modulatory role of endogenous serotonin release from dorsal raphe-to-mPFC projecting neurons in cognitive flexibility.

Amphetamine-induced prolonged disturbances in tissue levels of dopamine and serotonin in the rat brain

BACKGROUND

A hallmark of psychostimulants is the persistence of neurobiological changes they produce. The difficulty in reversing long-time effects of psychostimulants use is why addiction therapy is so ineffective. This study aimed to look for such drug-induced changes that can be detected even after many weeks of abstinence.

METHODS

Rats were given 12 doses of amphetamine (Amph) at 1.5 mg/kg. The rewarding effect of Amph was assessed using ultrasonic vocalization. After 14 and 28 days of abstinence, tissue levels of dopamine (DA), serotonin (5-HT), and their metabolites were measured in the prefrontal cortex (PFC), nucleus accumbens (Acb), dorsomedial (CPuM), and dorsolateral (CPuL) striatum.

RESULTS

After 28 days of abstinence, DA levels were increased in the dorsal striatum while 5-HT levels were decreased in all brain regions studied. The opposite direction of changes in DA and 5-HT tissue levels observed in the dorsal striatum may be related to the changes in the emotional state during abstinence and may contribute to the incubation of craving and relapses. Tissue levels of 5-HT and DA showed intra- and inter-structural correlations, most pronounced after 14 days of abstinence. Most of them were absent in the control group (ctrl), which may indicate that their appearance was related to the changes induced by earlier Amph administration. We did not find any associations between reward sensitivity and the persistence of Amph-induced neurochemical disturbances.

CONCLUSIONS

Administration of 12 moderate doses of Amph causes prolonged changes in DA and 5-HT tissue levels. The direction and severity of the changes are dependent on the brain region and the neurotransmitter studied.

Neural circuits underlying habituation of visually evoked escape behaviors in larval zebrafish

Larval zebrafish that are exposed repeatedly to dark looming stimuli will quickly habituate to these aversive signals and cease to respond with their stereotypical escape swims. A dark looming stimulus can be separated into two independent components: one that is characterized by an overall spatial expansion, where overall luminance is maintained at the same level, and a second, that represents an overall dimming within the whole visual field in the absence of any motion energy. Using specific stimulation patterns that isolate these independent components, we first extracted the behavioral algorithms that dictate how these separate information channels interact with each other and across the two eyes during the habituation process. Concurrent brain wide imaging experiments then permitted the construction of circuit models that suggest the existence of two separate neural pathways. The first is a looming channel which responds specifically to expanding edges presented to the contralateral eye and relays that information to the brain stem escape network to generate directed escapes. The second is a dimming-specific channel that could be either monocular or binocularly responsive, and that appears to specifically inhibit escape response when activated. We propose that this second channel is under strong contextual modulation and that it is primarily responsible for the incremental silencing of successive dark looming-evoked escapes.

Temporal derivative computation in the dorsal raphe network revealed by an experimentally driven augmented integrate-and-fire modeling framework

By means of an expansive innervation, the serotonin (5-HT) neurons of the dorsal raphe nucleus (DRN) are positioned to enact coordinated modulation of circuits distributed across the entire brain in order to adaptively regulate behavior. Yet the network computations that emerge from the excitability and connectivity features of the DRN are still poorly understood. To gain insight into these computations, we began by carrying out a detailed electrophysiological characterization of genetically identified mouse 5-HT and somatostatin (SOM) neurons. We next developed a single-neuron modeling framework that combines the realism of Hodgkin-Huxley models with the simplicity and predictive power of generalized integrate-and-fire models. We found that feedforward inhibition of 5-HT neurons by heterogeneous SOM neurons implemented divisive inhibition, while endocannabinoid-mediated modulation of excitatory drive to the DRN increased the gain of 5-HT output. Our most striking finding was that the output of the DRN encodes a mixture of the intensity and temporal derivative of its input, and that the temporal derivative component dominates this mixture precisely when the input is increasing rapidly. This network computation primarily emerged from prominent adaptation mechanisms found in 5-HT neurons, including a previously undescribed dynamic threshold. By applying a bottom-up neural network modeling approach, our results suggest that the DRN is particularly apt to encode input changes over short timescales, reflecting one of the salient emerging computations that dominate its output to regulate behavior.

Median raphe serotonergic neurons projecting to the interpeduncular nucleus control preference and aversion

Appropriate processing of reward and aversive information is essential for survival. Although a critical role of serotonergic neurons in the dorsal raphe nucleus (DRN) in reward processing has been shown, the lack of rewarding effects with selective serotonin reuptake inhibitors (SSRIs) implies the presence of a discrete serotonergic system playing an opposite role to the DRN in the processing of reward and aversive stimuli. Here, we demonstrated that serotonergic neurons in the median raphe nucleus (MRN) of mice process reward and aversive information in opposite directions to DRN serotonergic neurons. We further identified MRN serotonergic neurons, including those projecting to the interpeduncular nucleus (5-HT^MRN→IPN), as a key mediator of reward and aversive stimuli. Moreover, 5-HT receptors, including 5-HT_2A receptors in the interpeduncular nucleus, are involved in the aversive properties of MRN serotonergic neural activity. Our findings revealed an essential function of MRN serotonergic neurons, including 5-HT^MRN→IPN, in the processing of reward and aversive stimuli.

Neural circuits provide insights into reward and aversion

Maladaptive changes in the neural circuits associated with reward and aversion result in some common symptoms, such as drug addiction, anxiety, and depression. Historically, the study of these circuits has been hampered by technical limitations. In recent years, however, much progress has been made in understanding the neural mechanisms of reward and aversion owing to the development of technologies such as cell type-specific electrophysiology, neuronal tracing, and behavioral manipulation based on optogenetics. The aim of this paper is to summarize the latest findings on the mechanisms of the neural circuits associated with reward and aversion in a review of previous studies with a focus on the ventral tegmental area (VTA), nucleus accumbens (NAc), and basal forebrain (BF). These findings may inform efforts to prevent and treat mental illnesses associated with dysfunctions of the brain's reward and aversion system.

Tonic activity in lateral habenula neurons acts as a neutral valence brake on reward-seeking behavior

Survival requires both the ability to persistently pursue goals and the ability to determine when it is time to stop, an adaptive balance of perseverance and disengagement. Neural activity in the lateral habenula (LHb) has been linked to negative valence, but its role in regulating the balance between engaged reward seeking and disengaged behavioral states remains unclear. Here, we show that LHb neural activity is tonically elevated during minutes-long periods of disengagement from reward-seeking behavior, both when due to repeated reward omission (negative valence) and when sufficient reward has been consumed (positive valence). Furthermore, we show that LHb inhibition extends ongoing reward-seeking behavioral states but does not prompt task re-engagement. We find no evidence for similar tonic activity changes in ventral tegmental area dopamine neurons. Our findings support a framework in which tonic activity in LHb neurons suppresses engagement in reward-seeking behavior in response to both negatively and positively valenced factors.

Single-cell activity and network properties of dorsal raphe nucleus serotonin neurons during emotionally salient behaviors

The serotonin system modulates a wide variety of emotional behaviors and states, including reward processing, anxiety, and social interaction. To reveal the underlying patterns of neural activity, we visualized serotonergic neurons in the dorsal raphe nucleus (DRN5-HT) of mice using miniaturized microscopy during diverse emotional behaviors. We discovered ensembles of cells with highly correlated activity and found that DRN5-HT neurons are preferentially recruited by emotionally salient stimuli as opposed to neutral stimuli. Individual DRN5-HT neurons responded to diverse combinations of salient stimuli, with some preference for valence and sensory modality. Anatomically defined subpopulations projecting to either a reward-related structure (the ventral tegmental area) or an anxiety-related structure (the bed nucleus of the stria terminalis) contained all response types but were enriched in reward- and anxiety-responsive cells, respectively. Our results suggest that the DRN serotonin system responds to emotional salience using ensembles with mixed selectivity and biases in downstream connectivity.

Potassium Channel Conductance Is Involved in Phenylephrine-Induced Spontaneous Firing of Serotonergic Neurons in the Dorsal Raphe Nucleus

The serotonergic (5-HT) network from the dorsal raphe nucleus (DRN) of the brain has been demonstrated to regulate cognition, emotion, and behaviors, including learning and the sleep-wake cycle. Dysregulation of the activity of 5-HT neurons in the DRN is thought to play an important role in emotional disorders. The activity of 5-HT neurons is regulated by norepinephrine (NE) released from the projection terminals of noradrenergic input from the locus coeruleus (LC) via activation of the α1-adrenoceptor. However, insight into the molecular mechanism underlying this NE-induced regulation of 5-HT neuron activity is not clear. In this study, using the agonist of α1-adrenoceptor phenylephrine (PE), brain slices, and patch clamp, we found that A-type, Kv7/KCNQ, and calcium-activated low-conductance K+ channels (SK) underlie PE-induced spontaneous firing in DRN 5-HT neurons. Using single-cell PCR and immunofluorescence, we also identified the isoforms of these K+ channel families that might contribute to the NE/PE-induced spontaneous firing of DRN 5-HT neurons.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A neurobiological perspective on social influence: Serotonin and social adaptation

Humans are inherently social beings. Being suggestible to each other's expectations enables pro-social skills that are crucial for social learning and adaptation. Despite its high relevance for psychiatry, the neurobiological mechanisms underlying social adaptation are still not well understood. This review therefore provides a conceptual framework covering various distinct mechanisms underlying social adaptation and explores the neuropharmacology - in particular the role of the serotonin (5-HT) system - modulating these mechanisms. This article therefore reviews empirical results on social influence processing and reconciles them with recent findings from psychedelic research on social processing to elucidate neurobiological and neuropharmacological underpinnings of social adaptation. Various computational, neurobiological, and neurochemical processes are involved in distinct mechanisms underlying social adaptation such as the multisensory process of social information integration that is crucial for the forming of self-representation and representations of social norms. This is again associated with self- and other-perception during social interactions as well as value-based decision making that guides our behaviour in daily interactions. We highlight the critical role of 5-HT in these processes and suggest that 5-HT can facilitate social learning and may represent an important target for treating psychiatric disorders characterized by impairments in social functioning. This framework also has important implications for psychedelic-assisted therapy as well as for the development of novel treatment approaches and future research directions.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the “brake” on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory “drive.” For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the “drive” and “brake” circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Complementary roles of serotonergic and cholinergic systems in decisions about when to act

Decision-making not only involves deciding about which action to choose but when and whether to initiate an action in the first place. Macaque monkeys tracked number of dots on a screen and could choose when to make a response. The longer the animals waited before responding, the more dots appeared on the screen and the higher the probability of reward. Monkeys waited longer before making a response when a trial’s value was less than the environment’s average value. Recordings of brain activity with fMRI revealed that activity in dorsal raphe nucleus (DRN)—a key source of serotonin (5-HT)—tracked average value of the environment. By contrast, activity in the basal forebrain (BF)—an important source of acetylcholine (ACh)—was related to decision time to act as a function of immediate and recent past context. Interactions between DRN and BF and the anterior cingulate cortex (ACC), another region with action initiation-related activity, occurred as a function of the decision time to act. Next, we performed two psychopharmacological studies. Manipulating systemic 5-HT by citalopram prolonged the time macaques waited to respond for a given opportunity. This effect was more evident during blocks with long inter-trial intervals (ITIs) where good opportunities were sparse. Manipulating systemic acetylcholine (ACh) by rivastigmine reduced the time macaques waited to respond given the immediate and recent past context, a pattern opposite to the effect observed with 5-HT. These findings suggest complementary roles for serotonin/DRN and acetylcholine/BF in decisions about when to initiate an action.

•

Both immediate context and wider environment influence decisions about when to act

•

DRN and 5-HT mediate the influence of wider environment

•

BF and ACh mediate the influence of immediate context

Serotonin neurons modulate learning rate through uncertainty

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Serotonergic regulation of the dopaminergic system: Implications for reward-related functions

Serotonin is a critical neuromodulator involved in development and behavior. Its role in reward is however still debated. Here, we first review classical studies involving electrical stimulation protocols and pharmacological approaches. Contradictory results on the serotonergic' involvement in reward emerge from these studies. These differences might be ascribable to either the diversity of cellular types within the raphe nuclei or/and the specific projection pathways of serotonergic neurons. We continue to review more recent work, using optogenetic approaches to activate serotonergic cells in the Raphe to VTA pathway. From these studies, it appears that activation of this pathway can lead to reinforcement learning mediated through the excitation of dopaminergic neurons by serotonergic neurons co-transmitting glutamate. Finally, given the importance of serotonin during development on adult emotion, the effect of abnormal early-life levels of serotonin on the dopaminergic system will also be discussed. Understanding the interaction between the serotonergic and dopaminergic systems during development and adulthood is critical to gain insight into the specific facets of neuropsychiatric disorders.

The contributions of entorhinal cortex and hippocampus to error driven learning

Computational models proposed that the medial temporal lobe (MTL) contributes importantly to error-driven learning, though little direct in-vivo evidence for this hypothesis exists. To test this, we recorded in the entorhinal cortex (EC) and hippocampus (HPC) as macaques performed an associative learning task using an error-driven learning strategy, defined as better performance after error relative to correct trials. Error-detection signals were more prominent in the EC relative to HPC. Early in learning hippocampal but not EC neurons signaled error-driven learning by increasing their population stimulus-selectivity following error trials. This same pattern was not seen in another task where error-driven learning was not used. After learning, different populations of cells in both the EC and HPC signaled long-term memory of newly learned associations with enhanced stimulus-selective responses. These results suggest prominent but differential contributions of EC and HPC to learning from errors and a particularly important role of the EC in error-detection.

Ku et al. recorded in the entorhinal cortex (EC) and hippocampus (HPC) of macaques during associative learning tasks in order to test the computational model prediction that they contribute to error-driven learning. They demonstrate that the EC and HPC have prominent but differential contributions to learning from errors, with the EC having a particularly prominent role in error-detection.

Influence of ovarian hormones on value-based decision-making systems: Contribution to sexual dimorphisms in mental disorders

Women and men exhibit differences in behavior when making value-based decisions. Various hypotheses have been proposed to explain these findings, stressing differences in functional lateralization of the brain, functional activation, neurotransmitter involvement and more recently, sex hormones. While a significant interaction of neurotransmitter systems and sex hormones has been shown for both sexes, decision-making in women might be particularly affected by variations of ovarian hormones. In this review we have gathered information from animal and human studies on how ovarian hormones affect decision-making processes in females by interacting with neurotransmitter systems at functionally relevant brain locations and thus modify the computation of decision aspects. We also review previous findings on impaired decision-making in animals and clinical populations with substance use disorder and depression, emphasizing how little we know about the role of ovarian hormones in aberrant decision-making.

Habenula kisspeptin retrieves morphine impaired fear memory in zebrafish

The habenula is an evolutionarily conserved brain structure, which has recently been implicated in fear memory. In the zebrafish, kisspeptin (Kiss1) is predominantly expressed in the habenula, which has been implicated as a modulator of fear response. Hence, in the present study, we questioned whether Kiss1 has a role in fear memory and morphine-induced fear memory impairment using an odorant cue (alarm substances, AS)-induced fear avoidance paradigm in adult zebrafish, whereby the fear-conditioned memory can be assessed by a change of basal place preference (= avoidance) of fish due to AS-induced fear experience. Subsequently, to examine the possible role of Kiss1 neurons-serotonergic pathway, kiss1 mRNA and serotonin levels were measured. AS exposure triggered fear episodes and fear-conditioned place avoidance. Morphine treatment followed by AS exposure, significantly impaired fear memory with increased time-spent in AS-paired compartment. However, fish administered with Kiss1 (10^–21 mol/fish) after morphine treatment had significantly lower kiss1 mRNA levels but retained fear memory. In addition, the total brain serotonin levels were significantly increased in AS- and Kiss1-treated groups as compared to control and morphine treated group. These results suggest that habenular Kiss1 might be involved in consolidation or retrieval of fear memory through the serotonin system.

Reward Contributions to Serotonergic Functions

The brain serotonin systems participate in numerous aspects of reward processing, although it remains elusive how exactly serotonin signals regulate neural computation and reward-related behavior. The application of optogenetics and imaging techniques during the last decade has provided many insights. Here, we review recent progress on the organization and physiology of the dorsal raphe serotonin neurons and the relationships between their activity and behavioral functions in the context of reward processing. We also discuss several interesting theories on serotonin's function and how these theories may be reconciled by the possibility that serotonin, acting in synergy with coreleased glutamate, tracks and calculates the so-called beneficialness of the current state to guide an animal's behavior in dynamic environments.

Circuits and functions of the lateral habenula in health and in disease

The past decade has witnessed exponentially growing interest in the lateral habenula (LHb) owing to new discoveries relating to its critical role in regulating negatively motivated behaviour and its implication in major depression. The LHb, sometimes referred to as the brain's 'antireward centre', receives inputs from diverse limbic forebrain and basal ganglia structures, and targets essentially all midbrain neuromodulatory systems, including the noradrenergic, serotonergic and dopaminergic systems. Its unique anatomical position enables the LHb to act as a hub that integrates value-based, sensory and experience-dependent information to regulate various motivational, cognitive and motor processes. Dysfunction of the LHb may contribute to the pathophysiology of several psychiatric disorders, especially major depression. Recently, exciting progress has been made in identifying the molecular and cellular mechanisms in the LHb that underlie negative emotional state in animal models of drug withdrawal and major depression. A future challenge is to translate these advances into effective clinical treatments.

The Role of Dorsal Raphe Serotonin Neurons in the Balance between Reward and Aversion

BACKGROUND

Reward processing is fundamental for animals to survive and reproduce. Many studies have shown the importance of dorsal raphe nucleus (DRN) serotonin (5-HT) neurons in this process, but the strongly correlative link between the activity of DRN 5-HT neurons and rewarding/aversive potency is under debate. Our primary objective was to reveal this link using two different strategies to transduce DRN 5-HT neurons.

METHODS

For transduction of 5-HT neurons in wildtype mice, adeno-associated virus (AAV) bearing the mouse tryptophan hydroxylase 2 (TPH2) gene promoter was used. For transduction in Tph2-tTA transgenic mice, AAVs bearing the tTA-dependent TetO enhancer were used. To manipulate the activity of 5-HT neurons, optogenetic actuators (CheRiff, eArchT) were expressed by AAVs. For measurement of rewarding/aversive potency, we performed a nose-poke self-stimulation test and conditioned place preference (CPP) test.

RESULTS

We found that stimulation of DRN 5-HT neurons and their projections to the ventral tegmental area (VTA) increased the number of nose-pokes in self-stimulation test and CPP scores in both targeting methods. Concomitantly, CPP scores were decreased by inhibition of DRN 5-HT neurons and their projections to VTA.

CONCLUSION

Our findings indicate that the activity of DRN 5-HT neurons projecting to the VTA is a key modulator of balance between reward and aversion.

Neurobehavioral effects of acute and chronic lead exposure in a desert rodent Meriones shawi: Involvement of serotonin and dopamine

Lead (Pb) is a non physiological metal that has been implicated in toxic processes affecting several organs and biological systems, including the central nervous system. Several studies have focused on changes in lead-associated neurobehavioral and neurochemical alterations that occur due to Pb exposure. The present study evaluates the effects of acute and chronic Pb acetate exposure on serotoninergic and dopaminergic systems within the dorsal raphe nucleus, regarding motor activity and anxiety behaviours. Experiments were carried out on adult male Meriones shawi exposed to acute lead acetate intoxication (25 mg/kg b.w., 3 i.p. injections) or to a chronic lead exposure (0,5%) in drinking water from intrauterine age to adult age. Immunohistochemical staining demonstrated that both acute and chronic lead exposure increased anti-serotonin (anti-5HT) and tyrosine hydroxylase (anti-TH) immuno-reactivities in the dorsal raphe nucleus. In parallel, our results demonstrated that a long term Pb-exposure, but not an acute lead intoxication, induced behavioural alterations including, hyperactivity (open field test), and anxiogenic like-effects. Such neurobehavioral impairments induced by Pb-exposure in Meriones shawi may be related to dopaminergic and serotoninergic injuries identified in the dorsal raphe nucleus.

Control of synaptic plasticity in deep cortical networks

Humans and many other animals have an enormous capacity to learn about sensory stimuli and to master new skills. However, many of the mechanisms that enable us to learn remain to be understood. One of the greatest challenges of systems neuroscience is to explain how synaptic connections change to support maximally adaptive behaviour. Here, we provide an overview of factors that determine the change in the strength of synapses, with a focus on synaptic plasticity in sensory cortices. We review the influence of neuromodulators and feedback connections in synaptic plasticity and suggest a specific framework in which these factors can interact to improve the functioning of the entire network.

Identification of neuron-type specific promoters in monkey genome and their functional validation in mice

Viral gene delivery is one of the most versatile techniques for elucidating the mechanisms underlying brain dysfunction, such as neuropsychiatric disorders. Due to the complexity of the brain, expression of genetic tools, such as channelrhodopsin and calcium sensors, often has to be restricted to a specified cell type within a circuit implicated in these disorders. Only a handful of promoters targeting neuronal subtypes are currently used for viral gene delivery. Here, we isolated conserved promoter regions of several subtype-specific genes from the macaque genome and investigated their functionality in the mouse brain when used within lentiviral vectors (LVVs). Immunohistochemical analysis revealed that transgene expression induced by the promoter sequences for somatostatin (SST), cholecystokinin (CCK), parvalbumin (PV), serotonin transporter (SERT), vesicular acetylcholine transporter (vAChT), substance P (SP) and proenkephalin (PENK) was largely colocalized with specific markers for the targeted neuronal populations. Moreover, by combining these results with in silico predictions of transcription factor binding to the isolated sequences, we identified transcription factors possibly underlying cell-type specificity. These findings lay a foundation for the expansion of the current toolbox of promoters suitable for elucidating these neuronal phenotypes.

Ventral midbrain stimulation induces perceptual learning and cortical plasticity in primates

Practice improves perception and enhances neural representations of trained visual stimuli, a phenomenon known as visual perceptual learning (VPL). While attention to task-relevant stimuli plays an important role in such learning, Pavlovian stimulus-reinforcer associations are sufficient to drive VPL, even subconsciously. It has been proposed that reinforcement facilitates perceptual learning through the activation of neuromodulatory centers, but this has not been directly confirmed in primates. Here, we paired task-irrelevant visual stimuli with microstimulation of a dopaminergic center, the ventral tegmental area (VTA), in macaques. Pairing VTA microstimulation with a task-irrelevant visual stimulus increased fMRI activity and improved classification of fMRI activity patterns selectively for the microstimulation-paired stimulus. Moreover, pairing VTA microstimulation with a task-irrelevant visual stimulus improved the subject’s capacity to discriminate that stimulus. This is the first causal demonstration of the role of neuromodulatory centers in VPL in primates.

Practice can improve the perception of stimuli used to achieve a task (perceptual learning). Here, the authors show in monkeys that perceptual learning can be produced even for irrelevant stimuli if the stimuli are paired with stimulation of a dopaminergic centre, the ventral tegmental area (VTA).

Neuropsychopharmacological regulation of performance on creativity-related tasks

A number of factors affect performance on tasks associated with creativity. Two pharmacological systems in particularly been identified as important for their impact on creativity, the noradrenergic system and the dopaminergic systems. Furthermore, stress is also established as an important factor impacting performance, most likely mediated by its effects on these neurotransmitter systems. Herein, we review the current literature on the relationships between stress, the noradrenergic system, the dopaminergic system, and other pharmacological factors and their effects on performance on tasks associated with creativity.

Dorsal raphe serotonin neurons inhibit operant responding for reward via inputs to the ventral tegmental area but not the nucleus accumbens: evidence from studies combining optogenetic stimulation and serotonin reuptake inhibition

The monoamine neurotransmitter serotonin (5-hydroxytryptamine; 5-HT) exerts an inhibitory influence over motivation, but the circuits mediating this are unknown. Here, we used an optogenetic approach to isolate the contribution of dorsal raphe nucleus (DRN) 5-HT neurons and 5-HT innervation of the mesolimbic dopamine (DA) system to motivated behavior in mice. We found that optogenetic stimulation of DRN 5-HT neurons enhanced downstream 5-HT release, but this was not sufficient to inhibit operant responding for saccharin, a measure of motivated behavior. However, combining optogenetic stimulation of DRN 5-HT neurons with a low dose of the selective serotonin reuptake inhibitor (SSRI) citalopram synergistically reduced operant responding. We then examined whether these effects could be recapitulated if optogenetic stimulation specifically targeted 5-HT terminals in the ventral tegmental area (VTA) or nucleus accumbens (NAc) of the mesolimbic DA system. Optogenetic stimulation of 5-HT input to the VTA combined with citalopram treatment produced a synergistic decrease in responding for saccharin, resembling the changes produced by targeting 5-HT neurons in the DRN. However, this effect was not observed when optogenetic stimulation targeted 5-HT terminals in the NAc. Taken together, these results suggest that DRN 5-HT neurons exert an inhibitory influence over operant responding for reward through a direct interaction with the mesolimbic DA system at the level of the VTA. These studies support an oppositional interaction between 5-HT and DA systems in controlling motivation and goal-directed behavior, and have important implications for the development and refinement of treatment strategies for psychiatric disorders such as depression and addiction.

Influence of Serotonin Transporter SLC6A4 Genotype on the Effect of Psychosocial Stress on Cognitive Performance: An Exploratory Pilot Study

BACKGROUND AND OBJECTIVE

Previous research has shown an effect of various psychosocial stressors on unconstrained cognitive flexibility, such as searching through a large set of potential solutions in the lexical-semantic network during verbal problem-solving. Functional magnetic resonance imaging has shown that the presence of the short (S) allele (lacking a 43-base pair repeat) of the promoter region of the gene (SLC6A4) encoding the serotonin transporter (5-HTT) protein is associated with a greater amygdalar response to emotional stimuli and a greater response to stressors. Therefore, we hypothesized that the presence of the S-allele is associated with greater stress-associated impairment in performance on an unconstrained cognitive flexibility task, anagrams.

METHODS

In this exploratory pilot study, 28 healthy young adults were genotyped for long (L)-allele versus S-allele promoter region polymorphism of the 5-HTT gene, SLC6A4. Participants solved anagrams during the Trier Social Stress Test, which included public speaking and mental arithmetic stressors. We compared the participants' cognitive response to stress across genotypes.

RESULTS

A Gene×Stress interaction effect was observed in this small sample. Comparisons revealed that participants with at least one S-allele performed worse during the Stress condition.

CONCLUSIONS

Genetic susceptibility to stress conferred by SLC6A4 appeared to modulate unconstrained cognitive flexibility during psychosocial stress in this exploratory sample. If confirmed, this finding may have implications for conditions associated with increased stress response, including performance anxiety and cocaine withdrawal. Future work is needed both to confirm our findings with a larger sample and to explore the mechanisms of this proposed effect.

Dorsal Raphe Dual Serotonin-Glutamate Neurons Drive Reward by Establishing Excitatory Synapses on VTA Mesoaccumbens Dopamine Neurons

Dorsal raphe (DR) serotonin neurons provide a major input to the ventral tegmental area (VTA). Here, we show that DR serotonin transporter (SERT) neurons establish both asymmetric and symmetric synapses on VTA dopamine neurons, but most of these synapses are asymmetric. Moreover, the DR-SERT terminals making asymmetric synapses on VTA dopamine neurons coexpress vesicular glutamate transporter 3 (VGluT3; transporter for accumulation of glutamate for its synaptic release), suggesting the excitatory nature of these synapses. VTA photoactivation of DR-SERT fibers promotes conditioned place preference, elicits excitatory currents on mesoaccumbens dopamine neurons, increases their firing, and evokes dopamine release in nucleus accumbens. These effects are blocked by VTA inactivation of glutamate and serotonin receptors, supporting the idea of glutamate release in VTA from dual DR SERT-VGluT3 inputs. Our findings suggest a path-specific input from DR serotonergic neurons to VTA that promotes reward by the release of glutamate and activation of mesoaccumbens dopamine neurons.

What Is the Relationship between Dopamine and Effort?

The trade-off between reward and effort is at the heart of most behavioral theories, from ecology to economics. Compared to reward, however, effort remains poorly understood, both at the behavioral and neurophysiological levels. This is important because unwillingness to overcome effort to gain reward is a common feature of many neuropsychiatric and neurological disorders. A recent surge in interest in the neurobiological basis of effort has led to seemingly conflicting results regarding the role of dopamine. We argue here that, upon closer examination, there is actually striking consensus across studies: dopamine primarily codes for future reward but is less sensitive to anticipated effort cost. This strong association between dopamine and the incentive effects of rewards places dopamine in a key position to promote reward-directed action.

A Neuroeconomic Framework for Creative Cognition

Neuroeconomics is the study of the neurobiological bases of subjective preferences and choices. We present a novel framework that synthesizes findings from the literatures on neuroeconomics and creativity to provide a neurobiological description of creative cognition. We propose that value-based decision-making processes and activity in the locus ceruleus-norepinephrine (LC-NE) neuromodulatory system underlie creative cognition, as well as the large-scale brain network dynamics shown to be associated with creativity. This reconceptualization leads to several falsifiable hypotheses that can further understanding of creativity, decision making, and brain network dynamics.

Neuroscience of apathy and anhedonia: a transdiagnostic approach

Apathy and anhedonia are common syndromes of motivation that are associated with a wide range of brain disorders and have no established therapies. Research using animal models suggests that a useful framework for understanding motivated behaviour lies in effort-based decision making for reward. The neurobiological mechanisms underpinning such decisions have now begun to be determined in individuals with apathy or anhedonia, providing an important foundation for developing new treatments. The findings suggest that there might be some shared mechanisms between both syndromes. A transdiagnostic approach that cuts across traditional disease boundaries provides a potentially useful means for understanding these conditions.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

Anatomical Inputs From the Sensory and Value Structures to the Tail of the Rat Striatum

The caudal region of the rodent striatum, called the tail of the striatum (TS), is a relatively small area but might have a distinct function from other striatal subregions. Recent primate studies showed that this part of the striatum has a unique function in encoding long-term value memory of visual objects for habitual behavior. This function might be due to its specific connectivity. We identified inputs to the rat TS and compared those with inputs to the dorsomedial striatum (DMS) in the same animals. The TS directly received anatomical inputs from both sensory structures and value-coding regions, but the DMS did not. First, inputs from the sensory cortex and sensory thalamus to the TS were found; visual, auditory, somatosensory and gustatory cortex and thalamus projected to the TS but not to the DMS. Second, two value systems innervated the TS; dopamine and serotonin neurons in the lateral part of the substantia nigra pars compacta (SNc) and dorsal raphe nucleus projected to the TS, respectively. The DMS received inputs from the separate group of dopamine neurons in the medial part of the SNc. In addition, learning-related regions of the limbic system innervated the TS; the temporal areas and the basolateral amygdala selectively innervated the TS, but not the DMS. Our data showed that both sensory and value-processing structures innervated the TS, suggesting its plausible role in value-guided sensory-motor association for habitual behavior.

Intrinsic and Extrinsic Neuromodulation of Olfactory Processing

Neuromodulation is a ubiquitous feature of neural systems, allowing flexible, context specific control over network dynamics. Neuromodulation was first described in invertebrate motor systems and early work established a basic dichotomy for neuromodulation as having either an intrinsic origin (i.e., neurons that participate in network coding) or an extrinsic origin (i.e., neurons from independent networks). In this conceptual dichotomy, intrinsic sources of neuromodulation provide a “memory” by adjusting network dynamics based upon previous and ongoing activation of the network itself, while extrinsic neuromodulators provide the context of ongoing activity of other neural networks. Although this dichotomy has been thoroughly considered in motor systems, it has received far less attention in sensory systems. In this review, we discuss intrinsic and extrinsic modulation in the context of olfactory processing in invertebrate and vertebrate model systems. We begin by discussing presynaptic modulation of olfactory sensory neurons by local interneurons (LNs) as a mechanism for gain control based on ongoing network activation. We then discuss the cell-class specific effects of serotonergic centrifugal neurons on olfactory processing. Finally, we briefly discuss the integration of intrinsic and extrinsic neuromodulation (metamodulation) as an effective mechanism for exerting global control over olfactory network dynamics. The heterogeneous nature of neuromodulation is a recurring theme throughout this review as the effects of both intrinsic and extrinsic modulation are generally non-uniform.

Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments

Neuromodulatory systems, including the noradrenergic, serotonergic, dopaminergic, and cholinergic systems, track environmental signals, such as risks, rewards, novelty, effort, and social cooperation. These systems provide a foundation for cognitive function in higher organisms; attention, emotion, goal-directed behavior, and decision-making derive from the interaction between the neuromodulatory systems and brain areas, such as the amygdala, frontal cortex, hippocampus, and sensory cortices. Given their strong influence on behavior and cognition, these systems also play a key role in disease states and are the primary target of many current treatment strategies. The fact that these systems interact with each other either directly or indirectly, however, makes it difficult to understand how a failure in one or more systems can lead to a particular symptom or pathology. In this review, we explore experimental evidence, as well as focus on computational and theoretical models of neuromodulation. Better understanding of neuromodulatory systems may lead to the development of novel treatment strategies for a number of brain disorders.