Association with reward negatively modulates short latency phasic conditioned responses of dorsal raphe nucleus neurons in freely moving rats

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Potassium Channel Conductance Is Involved in Phenylephrine-Induced Spontaneous Firing of Serotonergic Neurons in the Dorsal Raphe Nucleus

The serotonergic (5-HT) network from the dorsal raphe nucleus (DRN) of the brain has been demonstrated to regulate cognition, emotion, and behaviors, including learning and the sleep-wake cycle. Dysregulation of the activity of 5-HT neurons in the DRN is thought to play an important role in emotional disorders. The activity of 5-HT neurons is regulated by norepinephrine (NE) released from the projection terminals of noradrenergic input from the locus coeruleus (LC) via activation of the α1-adrenoceptor. However, insight into the molecular mechanism underlying this NE-induced regulation of 5-HT neuron activity is not clear. In this study, using the agonist of α1-adrenoceptor phenylephrine (PE), brain slices, and patch clamp, we found that A-type, Kv7/KCNQ, and calcium-activated low-conductance K+ channels (SK) underlie PE-induced spontaneous firing in DRN 5-HT neurons. Using single-cell PCR and immunofluorescence, we also identified the isoforms of these K+ channel families that might contribute to the NE/PE-induced spontaneous firing of DRN 5-HT neurons.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Serotonin neurons modulate learning rate through uncertainty

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

Reward Contributions to Serotonergic Functions

The brain serotonin systems participate in numerous aspects of reward processing, although it remains elusive how exactly serotonin signals regulate neural computation and reward-related behavior. The application of optogenetics and imaging techniques during the last decade has provided many insights. Here, we review recent progress on the organization and physiology of the dorsal raphe serotonin neurons and the relationships between their activity and behavioral functions in the context of reward processing. We also discuss several interesting theories on serotonin's function and how these theories may be reconciled by the possibility that serotonin, acting in synergy with coreleased glutamate, tracks and calculates the so-called beneficialness of the current state to guide an animal's behavior in dynamic environments.

Dorsal raphe serotonin neurons inhibit operant responding for reward via inputs to the ventral tegmental area but not the nucleus accumbens: evidence from studies combining optogenetic stimulation and serotonin reuptake inhibition

The monoamine neurotransmitter serotonin (5-hydroxytryptamine; 5-HT) exerts an inhibitory influence over motivation, but the circuits mediating this are unknown. Here, we used an optogenetic approach to isolate the contribution of dorsal raphe nucleus (DRN) 5-HT neurons and 5-HT innervation of the mesolimbic dopamine (DA) system to motivated behavior in mice. We found that optogenetic stimulation of DRN 5-HT neurons enhanced downstream 5-HT release, but this was not sufficient to inhibit operant responding for saccharin, a measure of motivated behavior. However, combining optogenetic stimulation of DRN 5-HT neurons with a low dose of the selective serotonin reuptake inhibitor (SSRI) citalopram synergistically reduced operant responding. We then examined whether these effects could be recapitulated if optogenetic stimulation specifically targeted 5-HT terminals in the ventral tegmental area (VTA) or nucleus accumbens (NAc) of the mesolimbic DA system. Optogenetic stimulation of 5-HT input to the VTA combined with citalopram treatment produced a synergistic decrease in responding for saccharin, resembling the changes produced by targeting 5-HT neurons in the DRN. However, this effect was not observed when optogenetic stimulation targeted 5-HT terminals in the NAc. Taken together, these results suggest that DRN 5-HT neurons exert an inhibitory influence over operant responding for reward through a direct interaction with the mesolimbic DA system at the level of the VTA. These studies support an oppositional interaction between 5-HT and DA systems in controlling motivation and goal-directed behavior, and have important implications for the development and refinement of treatment strategies for psychiatric disorders such as depression and addiction.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin.

The Role of Multiple Neuromodulators in Reinforcement Learning That Is Based on Competition between Eligibility Traces

The ability to maximize reward and avoid punishment is essential for animal survival. Reinforcement learning (RL) refers to the algorithms used by biological or artificial systems to learn how to maximize reward or avoid negative outcomes based on past experiences. While RL is also important in machine learning, the types of mechanistic constraints encountered by biological machinery might be different than those for artificial systems. Two major problems encountered by RL are how to relate a stimulus with a reinforcing signal that is delayed in time (temporal credit assignment), and how to stop learning once the target behaviors are attained (stopping rule). To address the first problem synaptic eligibility traces were introduced, bridging the temporal gap between a stimulus and its reward. Although, these were mere theoretical constructs, recent experiments have provided evidence of their existence. These experiments also reveal that the presence of specific neuromodulators converts the traces into changes in synaptic efficacy. A mechanistic implementation of the stopping rule usually assumes the inhibition of the reward nucleus; however, recent experimental results have shown that learning terminates at the appropriate network state even in setups where the reward nucleus cannot be inhibited. In an effort to describe a learning rule that solves the temporal credit assignment problem and implements a biologically plausible stopping rule, we proposed a model based on two separate synaptic eligibility traces, one for long-term potentiation (LTP) and one for long-term depression (LTD), each obeying different dynamics and having different effective magnitudes. The model has been shown to successfully generate stable learning in recurrent networks. Although, the model assumes the presence of a single neuromodulator, evidence indicates that there are different neuromodulators for expressing the different traces. What could be the role of different neuromodulators for expressing the LTP and LTD traces? Here we expand on our previous model to include several neuromodulators, and illustrate through various examples how different these contribute to learning reward-timing within a wide set of training paradigms and propose further roles that multiple neuromodulators can play in encoding additional information of the rewarding signal.

Serotonergic modulation of orbitofrontal activity and its relevance for decision making and impulsivity

BACKGROUND

The orbitofrontal cortex seems to play a crucial role in reward-guided learning and decision making, especially for impulsive choice procedures including delayed reward discounting. The central serotonergic system is closely involved in the regulation of impulsivity, but how the serotonergic firing rate and release, best investigated by the loudness dependence of auditory evoked potentials (LDAEP), interact with orbitofrontal activity is still unknown.

METHODS

Twenty healthy volunteers (11 males, 9 females, 31.3 ± 10.6 years old) were studied in a 3T MRI scanner (Philips, Hamburg, Germany) during a delay discounting task, after their LDAEP was recorded using a 32 electrodes EEG machine (Brain Products, Munich, Germany).

RESULTS

Significant positive correlations were only found between the LDAEP and the medial orbitofrontal part of the superior frontal gyrus (SFG/MO) [Δ immediate reward - delayed reward] for the right (r = 0.519; P = 0.019) and left side (r = 0.478; P = 0.033). This relationship was stronger for females compared with males. Orbitofrontal activity was also related to the Barratt Impulsivity Scale.

CONCLUSIONS

This study revealed that low serotonergic activity as measured by a strong LDAEP was related to a high fMRI signal intensity of SFG/MO during immediate reward behavior which is related to impulsivity. Since this relationship was only found for the infralimbic medial and not for the middle or lateral part of the orbitofrontal cortex, an exclusive projection tract of the serotonergic system to this cortical region can be assumed to regulate impulsive reward-orientated decision making. Hum Brain Mapp 38:1507-1517, 2017. © 2016 Wiley Periodicals, Inc.

Complex Multiplexing of Reward-Cue- and Licking-Movement-Related Activity in Single Midline Thalamus Neurons

Midline thalamus is implicated in linking visceral and exteroceptive sensory information with behavior. However, whether neuronal activity is modulated with temporal precision by cues and actions in real time is unknown. Using single-neuron recording and a Pavlovian visual-cue/liquid-reward association task in rats, we discovered phasic responses to sensory cues, appropriately timed to modify information processing in output targets, as well as tonic modulations within and between trials that were differentially reward modulated, which may have distinct arousal functions. Many of the cue-responsive neurons also responded to repetitive licks, consistent with sensorimotor integration. Further, some lick-related neurons were activated only by the first rewarded lick and only if that lick were also part of a conditioned response sequence initiated earlier, consistent with binding action decisions to their ensuing outcome. This rich repertoire of responses provides electrophysiological evidence for midline thalamus as a site of complex information integration for reward-mediated behavior.

SIGNIFICANCE STATEMENT

Disparate brain circuits are involved in sensation, movement, and reward information. These must interact in order for the relationships between cues, actions, and outcomes to be learned. We found that responses of single neurons in midline thalamus to sensory cues are increased when associated with reward. This output may amplify similar signals generated in parallel by the dopamine system. In addition, some neurons coded a three-factor decision in which the neuron fired only if there was a movement, if it was the first one after the reward becoming available, and if it was part of a sequence triggered in response to a preceding cue. These data highlight midline thalamus as an important node integrating multiple types of information for linking sensation, actions, and rewards.

Serotonergic modulation of the activity of mesencephalic dopaminergic systems: Therapeutic implications

Since their discovery in the mammalian brain, it has been apparent that serotonin (5-HT) and dopamine (DA) interactions play a key role in normal and abnormal behavior. Therefore, disclosure of this interaction could reveal important insights into the pathogenesis of various neuropsychiatric diseases including schizophrenia, depression and drug addiction or neurological conditions such as Parkinson's disease and Tourette's syndrome. Unfortunately, this interaction remains difficult to study for many reasons, including the rich and widespread innervations of 5-HT and DA in the brain, the plethora of 5-HT receptors and the release of co-transmitters by 5-HT and DA neurons. The purpose of this review is to present electrophysiological and biochemical data showing that endogenous 5-HT and pharmacological 5-HT ligands modify the mesencephalic DA systems' activity. 5-HT receptors may control DA neuron activity in a state-dependent and region-dependent manner. 5-HT controls the activity of DA neurons in a phasic and excitatory manner, except for the control exerted by 5-HT_2C receptors which appears to also be tonically and/or constitutively inhibitory. The functional interaction between the two monoamines will also be discussed in view of the mechanism of action of antidepressants, antipsychotics, anti-Parkinsonians and drugs of abuse.

Serotonin neurons in the dorsal raphe nucleus encode reward signals

The dorsal raphe nucleus (DRN) is involved in organizing reward-related behaviours; however, it remains unclear how genetically defined neurons in the DRN of a freely behaving animal respond to various natural rewards. Here we addressed this question using fibre photometry and single-unit recording from serotonin (5-HT) neurons and GABA neurons in the DRN of behaving mice. Rewards including sucrose, food, sex and social interaction rapidly activate 5-HT neurons, but aversive stimuli including quinine and footshock do not. Both expected and unexpected rewards activate 5-HT neurons. After mice learn to wait for sucrose delivery, most 5-HT neurons fire tonically during waiting and then phasically on reward acquisition. Finally, GABA neurons are activated by aversive stimuli but inhibited when mice seek rewards. Thus, DRN 5-HT neurons positively encode a wide range of reward signals during anticipatory and consummatory phases of reward responses. Moreover, GABA neurons play a complementary role in reward processing.

Reward processing by the dorsal raphe nucleus: 5-HT and beyond

The dorsal raphe nucleus (DRN) represents one of the most sensitive reward sites in the brain. However, the exact relationship between DRN neuronal activity and reward signaling has been elusive. In this review, we will summarize anatomical, pharmacological, optogenetics, and electrophysiological studies on the functions and circuit mechanisms of DRN neurons in reward processing. The DRN is commonly associated with serotonin (5-hydroxytryptamine; 5-HT), but this nucleus also contains neurons of the neurotransmitter phenotypes of glutamate, GABA and dopamine. Pharmacological studies indicate that 5-HT might be involved in modulating reward- or punishment-related behaviors. Recent optogenetic stimulations demonstrate that transient activation of DRN neurons produces strong reinforcement signals that are carried out primarily by glutamate. Moreover, activation of DRN 5-HT neurons enhances reward waiting. Electrophysiological recordings reveal that the activity of DRN neurons exhibits diverse behavioral correlates in reward-related tasks. Studies so far thus demonstrate the strong power of DRN neurons in reward signaling and at the same time invite additional efforts to dissect the roles and mechanisms of different DRN neuron types in various processes of reward-related behaviors.

5-HT1A Receptor-Mediated Autoinhibition and the Control of Serotonergic Cell Firing

The idea that serotonergic synaptic transmission plays an essential role in the control of mood and the pharmacotherapy of anxiety and depression is one of the cornerstones of modern biological psychiatry. As a result, there is intense interest in understanding the mechanisms controlling the activity of serotonin-synthesizing (serotonergic) neurons. One of the oldest and most durable ideas emerging from this work is that serotonergic neurons are capable of autonomously regulating their own basal firing rate. Serotonergic neurons express on their surface 5-HT1A receptors (autoreceptors) that, when activated, induce the opening of potassium channels that hyperpolarize and thereby inhibit cell firing. Activity-dependent release of serotonin within serotonergic nuclei is thought to activate these autoreceptors, thus completing an autoinhibitory feedback loop. This concept, which was originally proposed in the 1970s, has proven to be enormously fruitful and has guided the interpretation of a broad range of clinical and preclinical work. Yet, remarkably, electrophysiological studies seeking to directly demonstrate this phenomenon, especially in in vitro brain slices, have produced mixed results. Here, we critically review this work with a focus on electrophysiological studies, which directly assess neuronal activity. We also highlight recent work suggesting that 5-HT1A receptor-mediated autoinhibition may play other roles in the control of firing besides acting as a feedback regulator for the pacemaker-like firing rate of serotonergic neurons.

Appetitive and aversive information coding in the primate dorsal raphé nucleus

Serotonin is known to play a key role in the regulation of emotional behavior. There have been conflicting hypotheses about whether the central serotonergic system is involved in positive or negative emotional information processing. To reveal whether and how such opposing information processing can be achieved by single neurons in the dorsal raphé nucleus (DRN), the major source of serotonin in the forebrain, we recorded the activity of DRN neurons while monkeys were conditioned in a Pavlovian procedure with two distinct contexts: an appetitive block where a reward was available; and an aversive one where an airpuff was delivered. We found that single DRN neurons were involved in several aspects of both appetitive and aversive information processing. First, more than half of the recorded DRN neurons discriminated between appetitive and aversive contexts by tonic changes in their activity. In the appetitive context, they then kept track of the expected reward value indicated by the conditioned stimuli. Some of them also encoded an error between the obtained and expected values. In the aversive context, the same neurons maintained tonic modulation in their activity throughout the block. However, modulation of their responses to aversive task events depending on airpuff probability was less common. Together, these results indicate that single DRN neurons encode both appetitive and aversive information, but over differing time scales: relatively shorter for appetitive, and longer for aversive. Such temporally distinct processes of value coding in the DRN may provide the neural basis of emotional information processing in different contexts.

Neuroscience: waiting for serotonin

Serotonin dysfunction is implicated in many neuropsychiatric disorders yet the precise behavioral functions of this neuromodulator are not well understood. A new study employs optogenetic methods to activate serotonin neurons during an effort-demanding waiting behavior and demonstrates that serotonin release increases patience, the capacity for self-control.

Differing properties of putative fast-spiking interneurons in the striatum of two rat strains

Local circuits within the striatum of the basal ganglia include a small number of γ-aminobutyric acid (GABA)-ergic fast-spiking interneurons (FSI). The number of these cells is reduced in disorders of behavioral control, but it is unknown whether this is accompanied by altered electrophysiological properties. The genetically hypertensive (GH) rat strain exhibits impulsiveness and hyperactivity. We investigated if resting-state FSI activity is affected in this strain using extracellular recordings. We also examined the effect of systemic amphetamine (AMPH), a stimulant drug used in the treatment of these particular behavioral deficits. Putative FSI (pFSI) were encountered less often in GH rats compared to the Wistar control strain. pFSI in GH rats also exhibited a higher mean firing rate, higher intraburst firing rate, lower interburst interval, and shorter bursts compared to controls. AMPH increased the mean overall firing rate of Wistar rat pFSI but did not significantly alter the firing properties of this subtype in GH rats. These differences in the resting-state electrophysiological activity of pFSI in GH rats point to them as a cell type of particular interest in understanding striatal functioning across different strains.

Serotonergic neurons signal reward and punishment on multiple timescales

Serotonin's function in the brain is unclear. One challenge in testing the numerous hypotheses about serotonin's function has been observing the activity of identified serotonergic neurons in animals engaged in behavioral tasks. We recorded the activity of dorsal raphe neurons while mice experienced a task in which rewards and punishments varied across blocks of trials. We ‘tagged’ serotonergic neurons with the light-sensitive protein channelrhodopsin-2 and identified them based on their responses to light. We found three main features of serotonergic neuron activity: (1) a large fraction of serotonergic neurons modulated their tonic firing rates over the course of minutes during reward vs punishment blocks; (2) most were phasically excited by punishments; and (3) a subset was phasically excited by reward-predicting cues. By contrast, dopaminergic neurons did not show firing rate changes across blocks of trials. These results suggest that serotonergic neurons signal information about reward and punishment on multiple timescales.

DOI:http://dx.doi.org/10.7554/eLife.06346.001

Rewards and punishments can both encourage animals to change their immediate behavior and influence their mood over a longer term, particularly when given repeatedly. A region of the brain that increases its activity in response to rewards and punishments also contains many neurons that communicate with each other by releasing a chemical called serotonin. This chemical is commonly thought to produce feelings of happiness; however, it remains unclear exactly how these particular ‘serotonergic’ neurons help to process rewards and punishments.

The ideal way to work out the role that a type of neuron plays in a behavior is to measure its electrical activity as the behavior is being performed. However, it is difficult to distinguish the activity of serotonergic neurons from the activity of the non-serotonergic neurons around them. To overcome this problem, Cohen et al. used viruses to force serotonergic neurons to make a type of ion channel that produces electrical currents in response to light. Shining light on these neurons via optical fibers and then measuring the neurons' responses helped to develop criteria that can identify which responses are generated by the serotonergic neurons.

Cohen et al. then recorded the activity of serotonergic neurons in thirsty mice as they experienced a series of rewards (for example, a drop of water) or punishments (such as a puff of air to the eye). Each reward or punishment was preceded by a distinct odor, so that the mice learned to anticipate what was coming.

These experiments revealed that serotonergic neurons respond to rewards and punishments by changing two aspects of their electrical activity: by producing short bursts of high activity, and by altering their baseline activity. Some of the serotonergic neurons fired rapidly in response to punishments, but not rewards; others fired rapidly when the mice detected a scent that meant that a reward was about to be given. The average level of reward or punishment the mice received also affected the baseline activity of many of the serotonergic neurons; this effect lasted for several minutes.

Overall, Cohen et al. suggest that serotonergic neurons can affect how mice respond to rewards or punishments in both the short and long term. Future experiments should aim to understand the diversity of the signals that Cohen et al. observed, and to determine how these signals are used to drive behavior. Ultimately, understanding how neural circuits made up of different types of cells work may aid in understanding the neural basis of behavior.

DOI:http://dx.doi.org/10.7554/eLife.06346.002

Dorsal raphe neurons signal reward through 5-HT and glutamate

The dorsal raphe nucleus (DRN) in the midbrain is a key center for serotonin (5-hydroxytryptamine; 5-HT)-expressing neurons. Serotonergic neurons in the DRN have been theorized to encode punishment by opposing the reward signaling of dopamine neurons. Here, we show that DRN neurons encode reward, but not punishment, through 5-HT and glutamate. Optogenetic stimulation of DRN Pet-1 neurons reinforces mice to explore the stimulation-coupled spatial region, shifts sucrose preference, drives optical self-stimulation, and directs sensory discrimination learning. DRN Pet-1 neurons increase their firing activity during reward tasks, and this activation can be used to rapidly change neuronal activity patterns in the cortex. Although DRN Pet-1 neurons are often associated with 5-HT, they also release glutamate, and both neurotransmitters contribute to reward signaling. These experiments demonstrate the ability of DRN neurons to organize reward behaviors and might provide insights into the underlying mechanisms of learning facilitation and anhedonia treatment.

Neurophysiology of performance monitoring and adaptive behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of one's actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

The role of the dorsal raphé nucleus in reward-seeking behavior

Pharmacological experiments have shown that the modulation of brain serotonin levels has a strong impact on value-based decision making. Anatomical and physiological evidence also revealed that the dorsal raphé nucleus (DRN), a major source of serotonin, and the dopamine system receive common inputs from brain regions associated with appetitive and aversive information processing. The serotonin and dopamine systems also have reciprocal functional influences on each other. However, the specific mechanism by which serotonin affects value-based decision making is not clear. To understand the information carried by the DRN for reward-seeking behavior, we measured single neuron activity in the primate DRN during the performance of saccade tasks to obtain different amounts of a reward. We found that DRN neuronal activity was characterized by tonic modulation that was altered by the expected and received reward value. Consistent reward-dependent modulation across different task periods suggested that DRN activity kept track of the reward value throughout a trial. The DRN was also characterized by modulation of its activity in the opposite direction by different neuronal subgroups, one firing strongly for the prediction and receipt of large rewards, with the other firing strongly for small rewards. Conversely, putative dopamine neurons showed positive phasic responses to reward-indicating cues and the receipt of an unexpected reward amount, which supports the reward prediction error signal hypothesis of dopamine. I suggest that the tonic reward monitoring signal of the DRN, possibly together with its interaction with the dopamine system, reports a continuous level of motivation throughout the performance of a task. Such a signal may provide "reward context" information to the targets of DRN projections, where it may be integrated further with incoming motivationally salient information.