Neurons in monkey dorsal raphe nucleus code beginning and progress of step-by-step schedule, reward expectation, and amount of reward outcome in the reward schedule task

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Reinforcement learning is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g., money, points) that are later exchanged for primary reinforcers (e.g., food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g., the current number of accumulated tokens, choice options, task epoch, trials since the last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n = 5 monkeys, three males and two females). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in the state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

Computational mechanisms underlying motivation to earn symbolic reinforcers

Reinforcement learning (RL) is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g. money, points) that can later be exchanged for primary reinforcers (e.g. food, drink). Although symbolic reinforcers are motivating, little is understood about the neural or computational mechanisms underlying the motivation to earn them. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g. current number of accumulated tokens, choice options, task epoch, trials since last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to capture the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n=5 monkeys). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Medial Prefrontal Cortex Serotonin Input Regulates Cognitive Flexibility in Mice

The medial prefrontal cortex (mPFC) regulates cognitive flexibility and emotional behavior. Neurons that release serotonin project to the mPFC, and serotonergic drugs influence emotion and cognition. Yet, the specific roles of endogenous serotonin release in the mPFC on neurophysiology and behavior are unknown. We show that axonal serotonin release in the mPFC directly inhibits the major mPFC output neurons. In serotonergic neurons projecting from the dorsal raphe to the mPFC, we find endogenous activity signatures pre-reward retrieval and at reward retrieval during a cognitive flexibility task. In vivo optogenetic activation of this pathway during pre-reward retrieval selectively improved extradimensional rule shift performance while inhibition impaired it, demonstrating sufficiency and necessity for mPFC serotonin release in cognitive flexibility. Locomotor activity and anxiety-like behavior were not affected by either optogenetic manipulation. Collectively, our data reveal a powerful and specific modulatory role of endogenous serotonin release from dorsal raphe-to-mPFC projecting neurons in cognitive flexibility.

Median raphe serotonergic neurons projecting to the interpeduncular nucleus control preference and aversion

Appropriate processing of reward and aversive information is essential for survival. Although a critical role of serotonergic neurons in the dorsal raphe nucleus (DRN) in reward processing has been shown, the lack of rewarding effects with selective serotonin reuptake inhibitors (SSRIs) implies the presence of a discrete serotonergic system playing an opposite role to the DRN in the processing of reward and aversive stimuli. Here, we demonstrated that serotonergic neurons in the median raphe nucleus (MRN) of mice process reward and aversive information in opposite directions to DRN serotonergic neurons. We further identified MRN serotonergic neurons, including those projecting to the interpeduncular nucleus (5-HT^MRN→IPN), as a key mediator of reward and aversive stimuli. Moreover, 5-HT receptors, including 5-HT_2A receptors in the interpeduncular nucleus, are involved in the aversive properties of MRN serotonergic neural activity. Our findings revealed an essential function of MRN serotonergic neurons, including 5-HT^MRN→IPN, in the processing of reward and aversive stimuli.

Serotonergic regulation of the dopaminergic system: Implications for reward-related functions

Serotonin is a critical neuromodulator involved in development and behavior. Its role in reward is however still debated. Here, we first review classical studies involving electrical stimulation protocols and pharmacological approaches. Contradictory results on the serotonergic' involvement in reward emerge from these studies. These differences might be ascribable to either the diversity of cellular types within the raphe nuclei or/and the specific projection pathways of serotonergic neurons. We continue to review more recent work, using optogenetic approaches to activate serotonergic cells in the Raphe to VTA pathway. From these studies, it appears that activation of this pathway can lead to reinforcement learning mediated through the excitation of dopaminergic neurons by serotonergic neurons co-transmitting glutamate. Finally, given the importance of serotonin during development on adult emotion, the effect of abnormal early-life levels of serotonin on the dopaminergic system will also be discussed. Understanding the interaction between the serotonergic and dopaminergic systems during development and adulthood is critical to gain insight into the specific facets of neuropsychiatric disorders.

Reward Contributions to Serotonergic Functions

The brain serotonin systems participate in numerous aspects of reward processing, although it remains elusive how exactly serotonin signals regulate neural computation and reward-related behavior. The application of optogenetics and imaging techniques during the last decade has provided many insights. Here, we review recent progress on the organization and physiology of the dorsal raphe serotonin neurons and the relationships between their activity and behavioral functions in the context of reward processing. We also discuss several interesting theories on serotonin's function and how these theories may be reconciled by the possibility that serotonin, acting in synergy with coreleased glutamate, tracks and calculates the so-called beneficialness of the current state to guide an animal's behavior in dynamic environments.

Modulation of anterior cingulate cortex reward and penalty signalling in medication-naive young-adult subjects with depressive symptoms following acute dose lurasidone

BACKGROUND

Aberrations in reward and penalty processing are implicated in depression and putatively reflect altered dopamine signalling. This study exploits the advantages of a placebo-controlled design to examine how a novel D2 antagonist with adjunctive antidepressant properties modifies activity in the brain's reward network in depression.

METHODS

We recruited 43 medication-naïve subjects across the range of depression severity (Beck's Depression Inventory-II score range: 0-43), including healthy volunteers, as well as people meeting full-criteria for major depressive disorder. In a double-blind placebo-controlled cross-over design, all subjects received either placebo or lurasidone (20 mg) across two visits separated by 1 week. Functional magnetic resonance imaging with the Monetary Incentive Delay (MID) task assessed reward functions via neural responses during anticipation and receipt of gains and losses. Arterial spin labelling measured cerebral blood flow (CBF) at rest.

RESULTS

Lurasidone altered fronto-striatal activity during anticipation and outcome phases of the MID task. A significant three-way Medication-by-Depression severity-by-Outcome interaction emerged in the anterior cingulate cortex (ACC) after correction for multiple comparisons. Follow-up analyses revealed significantly higher ACC activation to losses in high- v. low depression participants in the placebo condition, with a normalisation by lurasidone. This effect could not be accounted for by shifts in resting CBF.

CONCLUSIONS

Lurasidone acutely normalises reward processing signals in individuals with depressive symptoms. Lurasidone's antidepressant effects may arise from reducing responses to penalty outcomes in individuals with depressive symptoms.

Neurons in the monkey orbitofrontal cortex mediate reward value computation and decision-making

Choice reflects the values of available alternatives; more valuable options are chosen more often than less valuable ones. Here we studied whether neuronal responses in orbitofrontal cortex (OFC) reflect the value difference between options, and whether there is a causal link between OFC neuronal activity and choice. Using a decision-making task where two visual stimuli were presented sequentially, each signifying a value, we showed that when the second stimulus appears many neurons encode the value difference between alternatives. Later when the choice occurs, that difference signal disappears and a signal indicating the chosen value emerges. Pharmacological inactivation of OFC neurons coding for choice-related values increases the monkey's latency to make a choice and the likelihood that it will choose the less valuable alternative, when the value difference is small. Thus, OFC neurons code for value information that could be used to directly influence choice.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

An Update on the Role of Serotonin and its Interplay with Dopamine for Reward

The specific role of serotonin and its interplay with dopamine (DA) in adaptive, reward guided behavior as well as drug dependance, still remains elusive. Recently, novel methods allowed cell type specific anatomical, functional and interventional analyses of serotonergic and dopaminergic circuits, promising significant advancement in understanding their functional roles. Furthermore, it is increasingly recognized that co-release of neurotransmitters is functionally relevant, understanding of which is required in order to interpret results of pharmacological studies and their relationship to neural recordings. Here, we review recent animal studies employing such techniques with the aim to connect their results to effects observed in human pharmacological studies and subjective effects of drugs. It appears that the additive effect of serotonin and DA conveys significant reward related information and is subjectively highly euphorizing. Neither DA nor serotonin alone have such an effect. This coincides with optogenetically targeted recordings in mice, where the dopaminergic system codes reward prediction errors (PE), and the serotonergic system mainly unsigned PE. Overall, this pattern of results indicates that joint activity between both systems carries essential reward information and invites parallel investigation of both neurotransmitter systems.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

Information Accumulation over Time in Monkey Inferior Temporal Cortex Neurons Explains Pattern Recognition Reaction Time under Visual Noise

We recognize objects even when they are partially degraded by visual noise. We studied the relation between the amount of visual noise (5, 10, 15, 20, or 25%) degrading 8 black-and-white stimuli and stimulus identification in 2 monkeys performing a sequential delayed match-to-sample task. We measured the accuracy and speed with which matching stimuli were identified. The performance decreased slightly (errors increased) as the amount of visual noise increased for both monkeys. The performance remained above 80% correct, even with 25% noise. However, the reaction times markedly increased as the noise increased, indicating that the monkeys took progressively longer to decide what the correct response would be as the amount of visual noise increased, showing that the monkeys trade time to maintain accuracy. Thus, as time unfolds the monkeys act as if they are accumulating the information and/or testing hypotheses about whether the test stimulus is likely to be a match for the sample being held in short-term memory. We recorded responses from 13 single neurons in area TE of the 2 monkeys. We found that stimulus-selective information in the neuronal responses began accumulating when the match stimulus appeared. We found that the greater the amount of noise obscuring the test stimulus, the more slowly stimulus-related information by the 13 neurons accumulated. The noise induced slowing was about the same for both behavior and information. These data are consistent with the hypothesis that area TE neuron population carries information about stimulus identity that accumulates over time in such a manner that it progressively overcomes the signal degradation imposed by adding visual noise.

Serotonergic modulation of the activity of mesencephalic dopaminergic systems: Therapeutic implications

Since their discovery in the mammalian brain, it has been apparent that serotonin (5-HT) and dopamine (DA) interactions play a key role in normal and abnormal behavior. Therefore, disclosure of this interaction could reveal important insights into the pathogenesis of various neuropsychiatric diseases including schizophrenia, depression and drug addiction or neurological conditions such as Parkinson's disease and Tourette's syndrome. Unfortunately, this interaction remains difficult to study for many reasons, including the rich and widespread innervations of 5-HT and DA in the brain, the plethora of 5-HT receptors and the release of co-transmitters by 5-HT and DA neurons. The purpose of this review is to present electrophysiological and biochemical data showing that endogenous 5-HT and pharmacological 5-HT ligands modify the mesencephalic DA systems' activity. 5-HT receptors may control DA neuron activity in a state-dependent and region-dependent manner. 5-HT controls the activity of DA neurons in a phasic and excitatory manner, except for the control exerted by 5-HT_2C receptors which appears to also be tonically and/or constitutively inhibitory. The functional interaction between the two monoamines will also be discussed in view of the mechanism of action of antidepressants, antipsychotics, anti-Parkinsonians and drugs of abuse.

Serotonin neurons in the dorsal raphe nucleus encode reward signals

The dorsal raphe nucleus (DRN) is involved in organizing reward-related behaviours; however, it remains unclear how genetically defined neurons in the DRN of a freely behaving animal respond to various natural rewards. Here we addressed this question using fibre photometry and single-unit recording from serotonin (5-HT) neurons and GABA neurons in the DRN of behaving mice. Rewards including sucrose, food, sex and social interaction rapidly activate 5-HT neurons, but aversive stimuli including quinine and footshock do not. Both expected and unexpected rewards activate 5-HT neurons. After mice learn to wait for sucrose delivery, most 5-HT neurons fire tonically during waiting and then phasically on reward acquisition. Finally, GABA neurons are activated by aversive stimuli but inhibited when mice seek rewards. Thus, DRN 5-HT neurons positively encode a wide range of reward signals during anticipatory and consummatory phases of reward responses. Moreover, GABA neurons play a complementary role in reward processing.

Ongoing behavioral state information signaled in the lateral habenula guides choice flexibility in freely moving rats

The lateral habenula (LHb) plays a role in a wide variety of behaviors ranging from maternal care, to sleep, to various forms of cognition. One prominent theory with ample supporting evidence is that the LHb serves to relay basal ganglia and limbic signals about negative outcomes to midbrain monoaminergic systems. This makes it likely that the LHb is critically involved in behavioral flexibility as all of these systems have been shown to contribute when flexible behavior is required. Behavioral flexibility is commonly examined across species and is impaired in various neuropsychiatric conditions including autism, depression, addiction, and schizophrenia; conditions in which the LHb is thought to play a role. Therefore, a thorough examination of the role of the LHb in behavioral flexibility serves multiple functions including understanding possible connections with neuropsychiatric illnesses and additional insight into its role in cognition in general. Here, we assess the LHb’s role in behavioral flexibility through comparisons of the roles its afferent and efferent pathways are known to play. Additionally, we provide new evidence supporting the LHb contributions to behavioral flexibility through organization of specific goal directed actions under cognitively demanding conditions. Specifically, in the first experiment, a majority of neurons recorded from the LHb were found to correlate with velocity on a spatial navigation task and did not change significantly when reward outcomes were manipulated. Additionally, measurements of local field potential (LFP) in the theta band revealed significant changes in power relative to velocity and reward location. In a second set of experiments, inactivation of the LHb with the gamma-aminobutyric acid (GABA) agonists baclofen and muscimol led to an impairment in a spatial/response based repeated probabilistic reversal learning task. Control experiments revealed that this impairment was likely due to the demands of repeated switching behaviors as rats were unimpaired on initial discrimination acquisition or retention of probabilistic learning. Taken together, these novel findings compliment other work discussed supporting a role for the LHb in action selection when cognitive or emotional demands are increased. Finally, we discuss future mechanisms by which a superior understanding of the LHb can be obtained through additional examination of behavioral flexibility tasks.

Reward processing by the dorsal raphe nucleus: 5-HT and beyond

The dorsal raphe nucleus (DRN) represents one of the most sensitive reward sites in the brain. However, the exact relationship between DRN neuronal activity and reward signaling has been elusive. In this review, we will summarize anatomical, pharmacological, optogenetics, and electrophysiological studies on the functions and circuit mechanisms of DRN neurons in reward processing. The DRN is commonly associated with serotonin (5-hydroxytryptamine; 5-HT), but this nucleus also contains neurons of the neurotransmitter phenotypes of glutamate, GABA and dopamine. Pharmacological studies indicate that 5-HT might be involved in modulating reward- or punishment-related behaviors. Recent optogenetic stimulations demonstrate that transient activation of DRN neurons produces strong reinforcement signals that are carried out primarily by glutamate. Moreover, activation of DRN 5-HT neurons enhances reward waiting. Electrophysiological recordings reveal that the activity of DRN neurons exhibits diverse behavioral correlates in reward-related tasks. Studies so far thus demonstrate the strong power of DRN neurons in reward signaling and at the same time invite additional efforts to dissect the roles and mechanisms of different DRN neuron types in various processes of reward-related behaviors.

Appetitive and aversive information coding in the primate dorsal raphé nucleus

Serotonin is known to play a key role in the regulation of emotional behavior. There have been conflicting hypotheses about whether the central serotonergic system is involved in positive or negative emotional information processing. To reveal whether and how such opposing information processing can be achieved by single neurons in the dorsal raphé nucleus (DRN), the major source of serotonin in the forebrain, we recorded the activity of DRN neurons while monkeys were conditioned in a Pavlovian procedure with two distinct contexts: an appetitive block where a reward was available; and an aversive one where an airpuff was delivered. We found that single DRN neurons were involved in several aspects of both appetitive and aversive information processing. First, more than half of the recorded DRN neurons discriminated between appetitive and aversive contexts by tonic changes in their activity. In the appetitive context, they then kept track of the expected reward value indicated by the conditioned stimuli. Some of them also encoded an error between the obtained and expected values. In the aversive context, the same neurons maintained tonic modulation in their activity throughout the block. However, modulation of their responses to aversive task events depending on airpuff probability was less common. Together, these results indicate that single DRN neurons encode both appetitive and aversive information, but over differing time scales: relatively shorter for appetitive, and longer for aversive. Such temporally distinct processes of value coding in the DRN may provide the neural basis of emotional information processing in different contexts.

Prospective coding of dorsal raphe reward signals by the orbitofrontal cortex

The orbitofrontal cortex (OFC) is important for the cognitive processes of learning and decision making. Previous recordings have revealed that OFC neurons encode predictions of reward outcomes. The OFC is interconnected with the dorsal raphe nucleus (DRN), which is a major serotonin (5-HT) center of the brain. Recent studies have provided increasing evidence that the DRN encodes reward signals. However, it remains unclear how the activity of DRN neurons affects the prospective reward coding of OFC neurons. By combining single-unit recordings from the OFC and optogenetic activation of the DRN in behaving mice, we found that DRN stimulation is sufficient to organize and modulate the anticipatory responses of OFC neurons. During pavlovian conditioning tasks for mice, odorant cues were associated with the delayed delivery of natural rewards of sucrose solution or DRN stimulation. After training, OFC neurons exhibited prospective responses to the sucrose solution. More importantly, the coupling of an odorant with delayed DRN stimulation resulted in tonic excitation or inhibition of OFC neurons during the delay period. The intensity of the prospective responses was affected by the frequency and duration of DRN stimulation. Additionally, DRN stimulation bidirectionally modulated the prospective responses to natural rewards. These experiments indicate that signals from the DRN are incorporated into the brain reward system to shape the cortical prospective coding of rewards.

Serotonergic neurons signal reward and punishment on multiple timescales

Serotonin's function in the brain is unclear. One challenge in testing the numerous hypotheses about serotonin's function has been observing the activity of identified serotonergic neurons in animals engaged in behavioral tasks. We recorded the activity of dorsal raphe neurons while mice experienced a task in which rewards and punishments varied across blocks of trials. We ‘tagged’ serotonergic neurons with the light-sensitive protein channelrhodopsin-2 and identified them based on their responses to light. We found three main features of serotonergic neuron activity: (1) a large fraction of serotonergic neurons modulated their tonic firing rates over the course of minutes during reward vs punishment blocks; (2) most were phasically excited by punishments; and (3) a subset was phasically excited by reward-predicting cues. By contrast, dopaminergic neurons did not show firing rate changes across blocks of trials. These results suggest that serotonergic neurons signal information about reward and punishment on multiple timescales.

DOI:http://dx.doi.org/10.7554/eLife.06346.001

Rewards and punishments can both encourage animals to change their immediate behavior and influence their mood over a longer term, particularly when given repeatedly. A region of the brain that increases its activity in response to rewards and punishments also contains many neurons that communicate with each other by releasing a chemical called serotonin. This chemical is commonly thought to produce feelings of happiness; however, it remains unclear exactly how these particular ‘serotonergic’ neurons help to process rewards and punishments.

The ideal way to work out the role that a type of neuron plays in a behavior is to measure its electrical activity as the behavior is being performed. However, it is difficult to distinguish the activity of serotonergic neurons from the activity of the non-serotonergic neurons around them. To overcome this problem, Cohen et al. used viruses to force serotonergic neurons to make a type of ion channel that produces electrical currents in response to light. Shining light on these neurons via optical fibers and then measuring the neurons' responses helped to develop criteria that can identify which responses are generated by the serotonergic neurons.

Cohen et al. then recorded the activity of serotonergic neurons in thirsty mice as they experienced a series of rewards (for example, a drop of water) or punishments (such as a puff of air to the eye). Each reward or punishment was preceded by a distinct odor, so that the mice learned to anticipate what was coming.

These experiments revealed that serotonergic neurons respond to rewards and punishments by changing two aspects of their electrical activity: by producing short bursts of high activity, and by altering their baseline activity. Some of the serotonergic neurons fired rapidly in response to punishments, but not rewards; others fired rapidly when the mice detected a scent that meant that a reward was about to be given. The average level of reward or punishment the mice received also affected the baseline activity of many of the serotonergic neurons; this effect lasted for several minutes.

Overall, Cohen et al. suggest that serotonergic neurons can affect how mice respond to rewards or punishments in both the short and long term. Future experiments should aim to understand the diversity of the signals that Cohen et al. observed, and to determine how these signals are used to drive behavior. Ultimately, understanding how neural circuits made up of different types of cells work may aid in understanding the neural basis of behavior.

DOI:http://dx.doi.org/10.7554/eLife.06346.002

Serotonergic versus nonserotonergic dorsal raphe projection neurons: differential participation in reward circuitry

The dorsal raphe nucleus (DRN) contains the largest group of serotonin-producing neurons in the brain and projects to regions controlling reward. Although pharmacological studies suggest that serotonin inhibits reward seeking, electrical stimulation of the DRN strongly reinforces instrumental behavior. Here, we provide a targeted assessment of the behavioral, anatomical, and electrophysiological contributions of serotonergic and nonserotonergic DRN neurons to reward processes. To explore DRN heterogeneity, we used a simultaneous two-vector knockout/optogenetic stimulation strategy, as well as cre-induced and cre-silenced vectors in several cre-expressing transgenic mouse lines. We found that the DRN is capable of reinforcing behavior primarily via nonserotonergic neurons, for which the main projection target is the ventral tegmental area (VTA). Furthermore, these nonserotonergic projections provide glutamatergic excitation of VTA dopamine neurons and account for a large majority of the DRN-VTA pathway. These findings help to resolve apparent discrepancies between the roles of serotonin versus the DRN in behavioral reinforcement.

Dorsal raphe neurons signal reward through 5-HT and glutamate

The dorsal raphe nucleus (DRN) in the midbrain is a key center for serotonin (5-hydroxytryptamine; 5-HT)-expressing neurons. Serotonergic neurons in the DRN have been theorized to encode punishment by opposing the reward signaling of dopamine neurons. Here, we show that DRN neurons encode reward, but not punishment, through 5-HT and glutamate. Optogenetic stimulation of DRN Pet-1 neurons reinforces mice to explore the stimulation-coupled spatial region, shifts sucrose preference, drives optical self-stimulation, and directs sensory discrimination learning. DRN Pet-1 neurons increase their firing activity during reward tasks, and this activation can be used to rapidly change neuronal activity patterns in the cortex. Although DRN Pet-1 neurons are often associated with 5-HT, they also release glutamate, and both neurotransmitters contribute to reward signaling. These experiments demonstrate the ability of DRN neurons to organize reward behaviors and might provide insights into the underlying mechanisms of learning facilitation and anhedonia treatment.

Neurophysiology of performance monitoring and adaptive behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of one's actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

Self-choice enhances value in reward-seeking in primates

When an individual chooses one item from two or more alternatives, they compare the values of the expected outcomes. The outcome value can be determined by the associated reward amount, the probability of reward, and the workload required to earn the reward. Rational choice theory states that choices are made to maximize rewards over time, and that the same outcome values lead to an equal likelihood of choices. However, the theory does not distinguish between conditions with the same reward value, even when acquired under different circumstances, and does not always accurately describe real behavior. We have found that allowing a monkey to choose a reward schedule endows the schedule with extra value when compared to performance in an identical schedule that is chosen by another agent (a computer here). This behavior is not consistent with pure rational choice theory. Theoretical analysis using a modified temporal-difference learning model showed an enhanced schedule state value by self-choice. These results suggest that an increased reward value underlies the improved performances by self-choice during reward-seeking behavior.

Serotonin at the nexus of impulsivity and cue reactivity in cocaine addiction

Cocaine abuse and addiction remain great challenges on the public health agendas in the U.S. and the world. Increasingly sophisticated perspectives on addiction to cocaine and other drugs of abuse have evolved with concerted research efforts over the last 30 years. Relapse remains a particularly powerful clinical problem as, even upon termination of drug use and initiation of abstinence, the recidivism rates can be very high. The cycling course of cocaine intake, abstinence and relapse is tied to a multitude of behavioral and cognitive processes including impulsivity (a predisposition toward rapid unplanned reactions to stimuli without regard to the negative consequences), and cocaine cue reactivity (responsivity to cocaine-associated stimuli) cited as two key phenotypes that contribute to relapse vulnerability even years into recovery. Preclinical studies suggest that serotonin (5-hydroxytryptamine; 5-HT) neurotransmission in key neural circuits may contribute to these interlocked phenotypes well as the altered neurobiological states evoked by cocaine that precipitate relapse events. As such, 5-HT is an important target in the quest to understand the neurobiology of relapse-predictive phenotypes, to successfully treat this complex disorder and improve diagnostic and prognostic capabilities. This review emphasizes the role of 5-HT and its receptor proteins in key addiction phenotypes and the implications of current findings to the future of therapeutics in addiction. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

The role of the dorsal raphé nucleus in reward-seeking behavior

Pharmacological experiments have shown that the modulation of brain serotonin levels has a strong impact on value-based decision making. Anatomical and physiological evidence also revealed that the dorsal raphé nucleus (DRN), a major source of serotonin, and the dopamine system receive common inputs from brain regions associated with appetitive and aversive information processing. The serotonin and dopamine systems also have reciprocal functional influences on each other. However, the specific mechanism by which serotonin affects value-based decision making is not clear. To understand the information carried by the DRN for reward-seeking behavior, we measured single neuron activity in the primate DRN during the performance of saccade tasks to obtain different amounts of a reward. We found that DRN neuronal activity was characterized by tonic modulation that was altered by the expected and received reward value. Consistent reward-dependent modulation across different task periods suggested that DRN activity kept track of the reward value throughout a trial. The DRN was also characterized by modulation of its activity in the opposite direction by different neuronal subgroups, one firing strongly for the prediction and receipt of large rewards, with the other firing strongly for small rewards. Conversely, putative dopamine neurons showed positive phasic responses to reward-indicating cues and the receipt of an unexpected reward amount, which supports the reward prediction error signal hypothesis of dopamine. I suggest that the tonic reward monitoring signal of the DRN, possibly together with its interaction with the dopamine system, reports a continuous level of motivation throughout the performance of a task. Such a signal may provide "reward context" information to the targets of DRN projections, where it may be integrated further with incoming motivationally salient information.