Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards

Reduced serotonergic transmission alters sensitivity to cost and reward via 5-HT1A and 5-HT1B receptors in monkeys

Serotonin (5-HT) deficiency is a core biological pathology underlying depression and other psychiatric disorders whose key symptoms include decreased motivation. However, the exact role of 5-HT in motivation remains controversial and elusive. Here, we pharmacologically manipulated the 5-HT system in macaque monkeys and quantified the effects on motivation for goal-directed actions in terms of incentives and costs. Reversible inhibition of 5-HT synthesis increased errors and reaction times on goal-directed tasks, indicating reduced motivation. Analysis found incentive-dependent and cost-dependent components of this reduction. To identify the receptor subtypes that mediate cost and incentive, we systemically administered antagonists specific to 4 major 5-HT receptor subtypes: 5-HT1A, 5-HT1B, 5-HT2A, and 5-HT4. Positron emission tomography (PET) visualized the unique distribution of each subtype in limbic brain regions and determined the systemic dosage for antagonists that would achieve approximately 30% occupancy. Only blockade of 5-HT1A decreased motivation through changes in both expected cost and incentive; sensitivity to future workload and time delay to reward increased (cost) and reward value decreased (incentive). Blocking the 5-HT1B receptor also reduced motivation through decreased incentive, although it did not affect expected cost. These results suggest that 5-HT deficiency disrupts 2 processes, the subjective valuation of costs and rewards, via 5-HT1A and 5-HT1B receptors, thus leading to reduced motivation.

Attention to Stimuli of Learned versus Innate Biological Value Relies on Separate Neural Systems

The neural bases of attention, a set of neural processes that promote behavioral selection, is a subject of intense investigation. In humans, rewarded cues influence attention, even when those cues are irrelevant to the current task. Because the amygdala plays a role in reward processing, and the activity of amygdala neurons has been linked to spatial attention, we reasoned that the amygdala may be essential for attending to rewarded images. To test this possibility, we used an attentional capture task, which provides a quantitative measure of attentional bias. Specifically, we compared reaction times (RTs) of adult male rhesus monkeys with bilateral amygdala lesions and unoperated controls as they made a saccade away from a high- or low-value rewarded image to a peripheral target. We predicted that: (1) RTs will be longer for high- compared with low-value images, revealing attentional capture by rewarded stimuli; and (2) relative to controls, monkeys with amygdala lesions would exhibit shorter RT for high-value images. For comparison, we assessed the same groups of monkeys for attentional capture by images of predators and conspecifics, categories thought to have innate biological value. In performing the attentional capture task, all monkeys were slowed more by high-value relative to low-value rewarded images. Contrary to our prediction, amygdala lesions failed to disrupt this effect. When presented with images of predators and conspecifics, however, monkeys with amygdala lesions showed significantly diminished attentional capture relative to controls. Thus, separate neural pathways are responsible for allocating attention to stimuli with learned versus innate value.SIGNIFICANCE STATEMENT Valuable objects attract attention. The amygdala is known to contribute to reward processing and the encoding of object reward value. We therefore examined whether the amygdala is necessary for allocating attention to rewarded objects. For comparison, we assessed the amygdala's contribution to attending to objects with innate biological value: predators and conspecifics. We found that the macaque amygdala is necessary for directing attention to images with innate biological value, but not for directing attention to recently learned reward-predictive images. These findings indicate that the amygdala makes selective contributions to attending to valuable objects. The data are relevant to mental health disorders, such as social anxiety disorders and small animal phobias, that arise from biased attention to select categories of objects.

Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions

The brain stem noradrenergic nucleus locus coeruleus (LC) is involved in various costly processes: arousal, stress, and attention. Recent work has pointed toward an implication in physical effort, and indirect evidence suggests that the LC could be also involved in cognitive effort. To assess the dynamic relation between LC activity, effort production, and difficulty, we recorded the activity of 193 LC single units in 5 monkeys performing 2 discounting tasks (a delay discounting task and a force discounting task), as well as a simpler target detection task where conditions were matched for difficulty and only differed in terms of sensory-motor processes. First, LC neurons displayed a transient activation both when monkeys initiated an action and when exerting force. Second, the magnitude of the activation scaled with the associated difficulty, and, potentially, the corresponding amount of effort produced, both for decision and force production. Indeed, at action initiation in both discounting tasks, LC activation increased in conditions associated with lower average engagement rate, i.e., those requiring more cognitive control to trigger the response. Decision-related activation also scaled with response time (RT), over and above task parameters, in line with the idea that it reflects the amount of resources (here time) spent on the decision process. During force production, LC activation only scaled with the amount of force produced in the force discounting task, but not in the control target detection task, where subjective difficulty was equivalent across conditions. Our data show that LC neurons dynamically track the amount of effort produced to face both cognitive and physical challenges with a subsecond precision. This works provides key insight into effort processing and the contribution of the noradrenergic system, which is affected in several pathologies where effort is impaired, including Parkinson disease and depression.

Contributions of the Monkey Inferior Temporal Areas TE and TEO to Visual Categorization

The ability to categorize images is thought to depend on neural processing within the ventral visual stream. Recently, we reported that after removal of architectonic area TE, the terminal region of the ventral stream, monkeys were still able to categorize images as cats or dogs moderately well. Here, we investigate the contribution of TEO, the architectonically defined region located one step earlier than area TE in the ventral stream. Bilateral removal of TEO caused only a mild impairment in categorization. However, combined TE + TEO removal was followed by a severe, long-lasting impairment in categorization. All of the monkeys tested, including those with combined TE + TEO removals, had normal low-level visual functions, such as visual acuity. These results support the conclusion that categorization based on visual similarity is processed in parallel in TE and TEO.

D1- and D2-like receptors differentially mediate the effects of dopaminergic transmission on cost-benefit evaluation and motivation in monkeys

It has been widely accepted that dopamine (DA) plays a major role in motivation, yet the specific contribution of DA signaling at D₁-like receptor (D₁R) and D₂-like receptor (D₂R) to cost–benefit trade-off remains unclear. Here, by combining pharmacological manipulation of DA receptors (DARs) and positron emission tomography (PET) imaging, we assessed the relationship between the degree of D₁R/D₂R blockade and changes in benefit- and cost-based motivation for goal-directed behavior of macaque monkeys. We found that the degree of blockade of either D₁R or D₂R was associated with a reduction of the positive impact of reward amount and increasing delay discounting. Workload discounting was selectively increased by D₂R antagonism. In addition, blocking both D₁R and D₂R had a synergistic effect on delay discounting but an antagonist effect on workload discounting. These results provide fundamental insight into the distinct mechanisms of DA action in the regulation of the benefit- and cost-based motivation, which have important implications for motivational alterations in both neurological and psychiatric disorders.

Using quantitatively controlled pharmacological manipulations, this study teases apart the role of D1- and D2-like dopamine receptors in motivation and goal-directed behavior in monkeys, revealing complementary roles of two dopamine receptor subtypes in the computation of the cost/benefit trade-off to guide action.

Acetylcholine from the nucleus basalis magnocellularis facilitates the retrieval of well-established memory

Retrieval deficit of long-term memory is a cardinal symptom of dementia and has been proposed to associate with abnormalities in the central cholinergic system. Difficulty in the retrieval of memory is experienced by healthy individuals and not limited to patients with neurological disorders that result in forgetfulness. The difficulty of retrieving memories is associated with various factors, such as how often the event was experienced or remembered, but it is unclear how the cholinergic system plays a role in the retrieval of memory formed by a daily routine (accumulated experience). To investigate this point, we trained rats moderately (for a week) or extensively (for a month) to detect a visual cue in a two-alternative forced-choice task. First, we confirmed the well-established memory in the extensively trained group was more resistant to the retrieval problem than recently acquired memory in the moderately trained group. Next, we tested the effect of a cholinesterase inhibitor, donepezil, on the retrieval of memory after a long no-task period in extensively trained rats. Pre-administration of donepezil improved performance and reduced the latency of task initiation compared to the saline-treated group. Finally, we lesioned cholinergic neurons of the nucleus basalis magnocellularis (NBM), which project to the entire neocortex, by injecting the cholinergic toxin 192 IgG-saporin. NBM-lesioned rats showed severely impaired task initiation and performance. These abilities recovered as the trials progressed, though they never reached the level observed in rats with intact NBM. These results suggest that acetylcholine released from the NBM contributes to the retrieval of well-established memory developed by a daily routine.

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Perceptual decisions entail the accumulation of evidence until a decision criterion is reached. The amount of noise in this process is inversely related to the behavioral performance of the decision-maker. Hence, reducing the amount of perceived noise could improve performance in perceptual decisions. In this study, we investigated whether providing monetary reward for correct responses in a perceptual decision-making task would enhance performance based on prior research linking noise reduction to the administration of reward. To this end, thirty-one healthy young adults carried out an incentivized dot tracking task (iDT) during recording of functional magnetic resonance imaging (fMRI). Behavioral responses were fitted to a Bayesian version of the drift-diffusion model that, among other parameters, also includes an estimate of sensory noise. Fifty percent of the trials were incentivized to compare rewarded with unrewarded trials regarding behavior, brain responses and estimates of model parameters. In order to establish a link between the noise parameter and fMRI activity, we correlated percent signal change (PSC) values from nucleus accumbens and caudate nucleus with noise levels in rewarded and unrewarded trials respectively. Although reward did not affect behavioral performance and model parameters, the fMRI analyses showed notable differences in nucleus accumbens, caudate nucleus and rostral anterior cingulate cortex in rewarded relative to unrewarded trials. Furthermore, higher PSC within nucleus accumbens was significantly associated with lower sensory noise levels, which was specific to rewarded trials. This work is consistent with previous findings on reward modulation of brain responses and marks a first step towards elucidating the effects of reward-induced noise suppression during perceptual decision-making.

Noradrenergic But Not Dopaminergic Neurons Signal Task State Changes and Predict Reengagement After a Failure

The two catecholamines, noradrenaline and dopamine, have been shown to play comparable roles in behavior. Both noradrenergic and dopaminergic neurons respond to cues predicting reward availability and novelty. However, even though both are thought to be involved in motivating actions, their roles in motivation have seldom been directly compared. We therefore examined the activity of putative noradrenergic neurons in the locus coeruleus and putative midbrain dopaminergic neurons in monkeys cued to perform effortful actions for rewards. The activity in both regions correlated with engagement with a presented option. By contrast, only noradrenaline neurons were also (i) predictive of engagement in a subsequent trial following a failure to engage and (ii) more strongly activated in nonrepeated trials, when cues indicated a new task condition. This suggests that while both catecholaminergic neurons are involved in promoting action, noradrenergic neurons are sensitive to task state changes, and their influence on behavior extends beyond the immediately rewarded action.

Melanin-concentrating hormone in rat nucleus accumbens or lateral hypothalamus differentially impacts morphine and food seeking behaviors

BACKGROUND

Identifying neural substrates that are differentially affected by drugs of abuse and natural rewards is key to finding a target for an efficacious treatment for substance abuse. Melanin-concentrating hormone is a polypeptide with an inhibitory effect on the mesolimbic dopamine system. Here we test the hypothesis that melanin-concentrating hormone in the lateral hypothalamus and nucleus accumbens shell is differentially involved in the regulation of morphine and food-rewarded behaviors.

METHODS

Male Sprague-Dawley rats were trained with morphine (5.0 mg/kg, subcutaneously) or food pellets (standard chow, 10-14 g) to induce a conditioned place preference, immediately followed by extinction training. Melanin-concentrating hormone (1.0 µg/side) or saline was infused into the nucleus accumbens shell or lateral hypothalamus before the reinstatement primed by morphine or food, and locomotor activity was simultaneously monitored. As the comparison, melanin-concentrating hormone was also microinjected into the nucleus accumbens shell or lateral hypothalamus before the expression of food or morphine-induced conditioned place preference.

RESULTS

Microinfusion of melanin-concentrating hormone into the nucleus accumbens shell (but not into the lateral hypothalamus) prevented the reinstatement of morphine conditioned place preference but had no effect on the reinstatement of food conditioned place preference. In contrast, microinfusion of melanin-concentrating hormone into the lateral hypothalamus (but not in the nucleus accumbens shell) inhibited the reinstatement of food conditioned place preference but had no effect on the reinstatement of morphine conditioned place preference.

CONCLUSIONS

These results suggest a clear double dissociation of melanin-concentrating hormone in morphine/food rewarding behaviors and melanin-concentrating hormone in the nucleus accumbens shell. Melanin-concentrating hormone could be a potential target for therapeutic intervention for morphine abuse without affecting natural rewards.

Anticipatory feelings: Neural correlates and linguistic markers

This review introduces anticipatory feelings (AF) as a new construct related to the process of anticipation and prediction of future events. AF, defined as the state of awareness of physiological and neurocognitive changes that occur within an oganism in the form of a process of adapting to future events, are an important component of anticipation and expectancy. They encompass bodily-related interoceptive and affective components and are influenced by intrapersonal and dispositional factors, such as optimism, hope, pessimism, or worry. In the present review, we consider evidence from animal and human research, including neuroimaging studies, to characterize the brain structures and brain networks involved in AF. The majority of studies reviewed revealed three brain regions involved in future oriented feelings: 1) the insula; 2) the ventromedial prefrontal cortex (vmPFC); and 3) the amygdala. Moreover, these brain regions were confirmed by a meta-analysis, using a platform for large-scale, automated synthesis of fMRI data. Finally, by adopting a neurolinguistic and a big data approach, we illustrate how AF are expressed in language.

Preference for Cocaine is Represented in the Orbitofrontal Cortex by an Increased Proportion of Cocaine Use-Coding Neurons

Cocaine addiction is a harmful preference for drug use over and at the expense of other nondrug-related activities. Here we identify in the rat orbitofrontal cortex (OFC) a mechanism that explains individual preferences between cocaine use and an alternative, nondrug action. OFC neuronal activity was recorded while rats performed each of these 2 actions separately or while they chose between them. First, we found that these actions are encoded by 2 nonoverlapping neuronal populations and that the relative size of the cocaine population represented individual preferences. A larger relative size was only observed in cocaine-preferring individuals. Second, OFC neurons encoding a given individual's preferred action progressively fired more than other action-coding neurons few seconds before the preferred action was actually chosen, suggesting a prechoice neuronal competition for action selection. In cocaine-preferring rats, this manifested by a prechoice ramping-up activity in favor of the cocaine population. Finally, pharmacological manipulation of prechoice activity in favor of the cocaine population caused nondrug-preferring rats to shift their choice to cocaine. Overall, this study suggests that an individual preference for cocaine is represented in the OFC by a population size bias that systematically advantages cocaine use-coding neurons during prechoice competition for action selection.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

Perceptual processing in the ventral visual stream requires area TE but not rhinal cortex

There is an on-going debate over whether area TE, or the anatomically adjacent rhinal cortex, is the final stage of visual object processing. Both regions have been implicated in visual perception, but their involvement in non-perceptual functions, such as short-term memory, hinders clear-cut interpretation. Here, using a two-interval forced choice task without a short-term memory demand, we find that after bilateral removal of area TE, monkeys trained to categorize images based on perceptual similarity (morphs between dogs and cats), are, on the initial viewing, badly impaired when given a new set of images. They improve markedly with a small amount of practice but nonetheless remain moderately impaired indefinitely. The monkeys with bilateral removal of rhinal cortex are, under all conditions, indistinguishable from unoperated controls. We conclude that the final stage of the integration of visual perceptual information into object percepts in the ventral visual stream occurs in area TE.

Using preferred fluids and different reward schedules to motivate rhesus macaques (Macaca mulatta) in cognitive tasks

Rhesus macaques (Macaca mulatta) used in behavioural neuroscience are often required to complete cognitively complex tasks, for which a high level of motivation is essential. To induce motivation, researchers may implement fluid-restriction protocols, whereby freely available water is limited, such that fluid can be used as a reward in the laboratory. A variety of different rewards and schedules are used, but there exists a lack of data assessing their effectiveness. In this study, we aimed to quantify fluid preference in rhesus macaques and to use these preferences to compare the motivational quality of different reward schedules: the monkey's previous reward (i.e. the fluid used to reward them in past studies), their new preferred reward, a variable schedule of previous and preferred reward, and a choice between the previous and preferred rewards. We found that it may be possible to reduce the level of restriction if an adequately motivating preferred reward is identified, but that this is dependent on the animal. Each monkey responded differently to both the fluid-preference assessments and to the different reward schedules. As such, monkeys should not be subject to 'blanket' protocols but should be assessed individually to maintain adequate scientific data collection at the least severe level of fluid restriction.

Non-motor Characterization of the Basal Ganglia: Evidence From Human and Non-human Primate Electrophysiology

Although the basal ganglia have been implicated in a growing list of human behaviors, they include some of the least understood nuclei in the brain. For several decades studies have employed numerous methodologies to uncover evidence pointing to the basal ganglia as a hub of both motor and non-motor function. Recently, new electrophysiological characterization of the basal ganglia in humans has become possible through direct access to these deep structures as part of routine neurosurgery. Electrophysiological approaches for identifying non-motor function have the potential to unlock a deeper understanding of pathways that may inform clinical interventions and particularly neuromodulation. Various electrophysiological modalities can also be combined to reveal functional connections between the basal ganglia and traditional structures throughout the neocortex that have been linked to non-motor behavior. Several reviews have previously summarized evidence for non-motor function in the basal ganglia stemming from behavioral, clinical, computational, imaging, and non-primate animal studies; in this review, instead we turn to electrophysiological studies of non-human primates and humans. We begin by introducing common electrophysiological methodologies for basal ganglia investigation, and then we discuss studies across numerous non-motor domains–emotion, response inhibition, conflict, decision-making, error-detection and surprise, reward processing, language, and time processing. We discuss the limitations of current approaches and highlight the current state of the information.

A common neural code for social and monetary rewards in the human striatum

Although managing social information and decision making on the basis of reward is critical for survival, it remains uncertain whether differing reward type is processed in a uniform manner. Previously, we demonstrated that monetary reward and the social reward of good reputation activated the same striatal regions including the caudate nucleus and putamen. However, it remains unclear whether overlapping activations reflect activities of identical neuronal populations or two overlapping but functionally independent neuronal populations. Here, we re-analyzed the original data and addressed this question using multivariate-pattern-analysis and found evidence that in the left caudate nucleus and bilateral nucleus accumbens, social vs monetary reward were represented similarly. The findings suggest that social and monetary rewards are processed by the same population of neurons within these regions of the striatum. Additional findings demonstrated similar neural patterns when participants experience high social reward compared to viewing others receiving low social reward (potentially inducing schadenfreude). This is possibly an early indication that the same population of neurons may be responsible for processing two different types of social reward (good reputation and schadenfreude). These findings provide a supplementary perspective to previous research, helping to further elucidate the mechanisms behind social vs non-social reward processing.

Contributions of Lateral and Orbital Frontal Regions to Abstract Rule Acquisition and Reversal in Monkeys

The ability to learn and follow abstract rules relies on intact prefrontal regions including the lateral prefrontal cortex (LPFC) and the orbitofrontal cortex (OFC). Here, we investigate the specific roles of these brain regions in learning rules that depend critically on the formation of abstract concepts as opposed to simpler input-output associations. To this aim, we tested monkeys with bilateral removals of either LPFC or OFC on a rapidly learned task requiring the formation of the abstract concept of same vs. different. While monkeys with OFC removals were significantly slower than controls at both acquiring and reversing the concept-based rule, monkeys with LPFC removals were not impaired in acquiring the task, but were significantly slower at rule reversal. Neither group was impaired in the acquisition or reversal of a delayed visual cue-outcome association task without a concept-based rule. These results suggest that OFC is essential for the implementation of a concept-based rule, whereas LPFC seems essential for its modification once established.

Outcome Modulation Across Tasks in the Primate Dorsolateral Prefrontal Cortex

Animals need to learn and to adapt to new and changing environments so that appropriate actions that lead to desirable outcomes are acquired within each context. The prefrontal cortex (PF) is known to underlie such function that directly implies that the outcome of each response must be represented in the brain for behavioral policies update. However, whether such PF signal is context dependent or it is a general representation beyond the specificity of a context is still unclear. Here, we analyzed the activity of neurons in the dorsolateral PF (PFdl) recorded while two monkeys performed two perceptual magnitude discrimination tasks. Both tasks were well known by the monkeys and unexpected changes did not occur but the difficulty of the task varied from trial to trial and thus the monkeys made mistakes in a proportion of trials. We show a context-independent coding of the response outcome with neurons maintaining similar selectivity in both task contexts. Using a classification method of the neural activity, we also show that the trial outcome could be well predicted from the activity of the same neurons in the two contexts. Altogether, our results provide evidence of high degree of outcome generality in PFdl.

Deep Brain Stimulation of the Nucleus Accumbens Core Affects Trait Impulsivity in a Baseline-Dependent Manner

Deep brain stimulation (DBS) of the nucleus accumbens (NA) is explored as a treatment for refractory psychiatric disorders, such as obsessive-compulsive disorder (OCD), depressive disorder (MDD), and substance use disorder (SUD). A common feature of some of these disorders is pathological impulsivity. Here, the effects of NAcore DBS on impulsive choice and impulsive action, two distinct forms of impulsive behavior, were investigated in translational animal tasks, the delayed reward task (DRT) and five-choice serial reaction time task (5-CSRTT), respectively. In both tasks, the effects of NAcore DBS were negatively correlated with baseline impulsive behavior, with more pronounced effects in the 5-CSRTT. To further examine the effects of DBS on trait impulsive action, rats were screened for high (HI) and low (LI) impulsive responding in the 5-CSRTT. NAcore DBS decreased impulsive, premature responding in HI rats under conventional conditions. However, upon challenged conditions to increase impulsive responding, NAcore DBS did not alter impulsivity. These results strongly suggest a baseline-dependent effect of DBS on impulsivity, which is in line with clinical observations.

Cerebellum Transcriptome of Mice Bred for High Voluntary Activity Offers Insights into Locomotor Control and Reward-Dependent Behaviors

The role of the cerebellum in motivation and addictive behaviors is less understood than that in control and coordination of movements. High running can be a self-rewarding behavior exhibiting addictive properties. Changes in the cerebellum transcriptional networks of mice from a line selectively bred for High voluntary running (H) were profiled relative to an unselected Control (C) line. The environmental modulation of these changes was assessed both in activity environments corresponding to 7 days of Free (F) access to running wheel and to Blocked (B) access on day 7. Overall, 457 genes exhibited a significant (FDR-adjusted P-value < 0.05) genotype-by-environment interaction effect, indicating that activity genotype differences in gene expression depend on environmental access to running. Among these genes, network analysis highlighted 6 genes (Nrgn, Drd2, Rxrg, Gda, Adora2a, and Rab40b) connected by their products that displayed opposite expression patterns in the activity genotype contrast within the B and F environments. The comparison of network expression topologies suggests that selection for high voluntary running is linked to a predominant dysregulation of hub genes in the F environment that enables running whereas a dysregulation of ancillary genes is favored in the B environment that blocks running. Genes associated with locomotor regulation, signaling pathways, reward-processing, goal-focused, and reward-dependent behaviors exhibited significant genotype-by-environment interaction (e.g. Pak6, Adora2a, Drd2, and Arhgap8). Neuropeptide genes including Adcyap1, Cck, Sst, Vgf, Npy, Nts, Penk, and Tac2 and related receptor genes also exhibited significant genotype-by-environment interaction. The majority of the 183 differentially expressed genes between activity genotypes (e.g. Drd1) were under-expressed in C relative to H genotypes and were also under-expressed in B relative to F environments. Our findings indicate that the high voluntary running mouse line studied is a helpful model for understanding the molecular mechanisms in the cerebellum that influence locomotor control and reward-dependent behaviors.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Placebo Response is Driven by UCS Revaluation: Evidence, Neurophysiological Consequences and a Quantitative Model

Despite growing scientific interest in the placebo effect and increasing understanding of neurobiological mechanisms, theoretical modeling of the placebo response remains poorly developed. The most extensively accepted theories are expectation and conditioning, involving both conscious and unconscious information processing. However, it is not completely understood how these mechanisms can shape the placebo response. We focus here on neural processes which can account for key properties of the response to substance intake. It is shown that placebo response can be conceptualized as a reaction of a distributed neural system within the central nervous system. Such a reaction represents an integrated component of the response to open substance administration (or to substance intake) and is updated through “unconditioned stimulus (UCS) revaluation learning”. The analysis leads to a theorem, which proves the existence of two distinct quantities coded within the brain, these are the expected or prediction outcome and the reactive response. We show that the reactive response is updated automatically by implicit revaluation learning, while the expected outcome can also be modulated through conscious information processing. Conceptualizing the response to substance intake in terms of UCS revaluation learning leads to the theoretical formulation of a potential neuropharmacological treatment for increasing unlimitedly the effectiveness of a given drug.

Reappraising striatal D1- and D2-neurons in reward and aversion

The striatum has been involved in complex behaviors such as motor control, learning, decision-making, reward and aversion. The striatum is mainly composed of medium spiny neurons (MSNs), typically divided into those expressing dopamine receptor D1, forming the so-called direct pathway, and those expressing D2 receptor (indirect pathway). For decades it has been proposed that these two populations exhibit opposing control over motor output, and recently, the same dichotomy has been proposed for valenced behaviors. Whereas D1-MSNs mediate reinforcement and reward, D2-MSNs have been associated with punishment and aversion. In this review we will discuss pharmacological, genetic and optogenetic studies that indicate that there is still controversy to what concerns the role of striatal D1- and D2-MSNs in this type of behaviors, highlighting the need to reconsider the early view that they mediate solely opposing aspects of valenced behaviour.

Reward functions of the basal ganglia

Besides their fundamental movement function evidenced by Parkinsonian deficits, the basal ganglia are involved in processing closely linked non-motor, cognitive and reward information. This review describes the reward functions of three brain structures that are major components of the basal ganglia or are closely associated with the basal ganglia, namely midbrain dopamine neurons, pedunculopontine nucleus, and striatum (caudate nucleus, putamen, nucleus accumbens). Rewards are involved in learning (positive reinforcement), approach behavior, economic choices and positive emotions. The response of dopamine neurons to rewards consists of an early detection component and a subsequent reward component that reflects a prediction error in economic utility, but is unrelated to movement. Dopamine activations to non-rewarded or aversive stimuli reflect physical impact, but not punishment. Neurons in pedunculopontine nucleus project their axons to dopamine neurons and process sensory stimuli, movements and rewards and reward-predicting stimuli without coding outright reward prediction errors. Neurons in striatum, besides their pronounced movement relationships, process rewards irrespective of sensory and motor aspects, integrate reward information into movement activity, code the reward value of individual actions, change their reward-related activity during learning, and code own reward in social situations depending on whose action produces the reward. These data demonstrate a variety of well-characterized reward processes in specific basal ganglia nuclei consistent with an important function in non-motor aspects of motivated behavior.

Chemogenetic disconnection of monkey orbitofrontal and rhinal cortex reversibly disrupts reward value

To study how the interaction between orbitofrontal (OFC) and rhinal (Rh) cortices influences the judgment of reward size, we reversibly disconnected these regions using the hM4Di-DREADD (Designer Receptor Exclusively Activated by Designer Drug). Repeated inactivation reduced sensitivity to differences in reward size in two monkeys. Results suggest that retrieval of relative stimulus values from memory appears to depend on interaction between Rh and OFC.

Rewards modulate saccade latency but not exogenous spatial attention

The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behavior induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor inhibition of return. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for three blocks of extinction trials. However, this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards

The basal ganglia control body movements, value processing and decision-making. Many studies have shown that the inputs and outputs of each basal ganglia structure are topographically organized, which suggests that the basal ganglia consist of separate circuits that serve distinct functions. A notable example is the circuits that originate from the rostral (head) and caudal (tail) regions of the caudate nucleus, both of which target the superior colliculus. These two caudate regions encode the reward values of visual objects differently: flexible (short-term) values by the caudate head and stable (long-term) values by the caudate tail. These value signals in the caudate guide the orienting of gaze differently: voluntary saccades by the caudate head circuit and automatic saccades by the caudate tail circuit. Moreover, separate groups of dopamine neurons innervate the caudate head and tail and may selectively guide the flexible and stable learning/memory in the caudate regions. Studies focusing on manual handling of objects also suggest that rostrocaudally separated circuits in the basal ganglia control the action differently. These results suggest that the basal ganglia contain parallel circuits for two steps of goal-directed behaviour: finding valuable objects and manipulating the valuable objects. These parallel circuits may underlie voluntary behaviour and automatic skills, enabling animals (including humans) to adapt to both volatile and stable environments. This understanding of the functions and mechanisms of the basal ganglia parallel circuits may inform the differential diagnosis and treatment of basal ganglia disorders.

Cholinergic interneurons in the dorsal and ventral striatum: anatomical and functional considerations in normal and diseased conditions

Striatal cholinergic interneurons (ChIs) are central for the processing and reinforcement of reward-related behaviors that are negatively affected in states of altered dopamine transmission, such as in Parkinson's disease or drug addiction. Nevertheless, the development of therapeutic interventions directed at ChIs has been hampered by our limited knowledge of the diverse anatomical and functional characteristics of these neurons in the dorsal and ventral striatum, combined with the lack of pharmacological tools to modulate specific cholinergic receptor subtypes. This review highlights some of the key morphological, synaptic, and functional differences between ChIs of different striatal regions and across species. It also provides an overview of our current knowledge of the cellular localization and function of cholinergic receptor subtypes. The future use of high-resolution anatomical and functional tools to study the synaptic microcircuitry of brain networks, along with the development of specific cholinergic receptor drugs, should help further elucidate the role of striatal ChIs and permit efficient targeting of cholinergic systems in various brain disorders, including Parkinson's disease and addiction.

Disordered gambling: the evolving concept of behavioral addiction

The reclassification of gambling disorder within the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) addictions category marks an important step for addiction science. The similarities between gambling disorder and the substance use disorders have been well documented. As gambling is unlikely to exert actively damaging effects on the brain, the cognitive sequelae of gambling disorder may provide insights into addictive vulnerabilities; this idea is critically evaluated in light of recent structural imaging data. The second part of the review analyzes a fundamental question of how a behavior can become addictive in the absence of exogenous drug stimulation. The relative potency of drug and nondrug rewards is considered, alongside evidence that cognitive distortions in the processing of chance (for example, the illusion of control and the gambler's fallacy) may constitute an important added ingredient in gambling. Further understanding of these mechanisms at neural and behavioral levels will be critical for the classification of future behavioral addictions, and I consider the current research data for obesity and binge eating, compulsive shopping, and internet gaming disorder.

Behavioral flexibility is increased by optogenetic inhibition of neurons in the nucleus accumbens shell during specific time segments

Behavioral flexibility is vital for survival in an environment of changing contingencies. The nucleus accumbens may play an important role in behavioral flexibility, representing learned stimulus–reward associations in neural activity during response selection and learning from results. To investigate the role of nucleus accumbens neural activity in behavioral flexibility, we used light-activated halorhodopsin to inhibit nucleus accumbens shell neurons during specific time segments of a bar-pressing task requiring a win–stay/lose–shift strategy. We found that optogenetic inhibition during action selection in the time segment preceding a lever press had no effect on performance. However, inhibition occurring in the time segment during feedback of results—whether rewards or nonrewards—reduced the errors that occurred after a change in contingency. Our results demonstrate critical time segments during which nucleus accumbens shell neurons integrate feedback into subsequent responses. Inhibiting nucleus accumbens shell neurons in these time segments, during reinforced performance or after a change in contingencies, increases lose–shift behavior. We propose that the activity of nucleus shell accumbens shell neurons in these time segments plays a key role in integrating knowledge of results into subsequent behavior, as well as in modulating lose–shift behavior when contingencies change.

Brain putamen volume changes in newly-diagnosed patients with obstructive sleep apnea

Obstructive sleep apnea (OSA) is accompanied by cognitive, motor, autonomic, learning, and affective abnormalities. The putamen serves several of these functions, especially motor and autonomic behaviors, but whether global and specific sub-regions of that structure are damaged is unclear. We assessed global and regional putamen volumes in 43 recently-diagnosed, treatment-naïve OSA (age, 46.4 ± 8.8 years; 31 male) and 61 control subjects (47.6 ± 8.8 years; 39 male) using high-resolution T1-weighted images collected with a 3.0-Tesla MRI scanner. Global putamen volumes were calculated, and group differences evaluated with independent samples t-tests, as well as with analysis of covariance (covariates; age, gender, and total intracranial volume). Regional differences between groups were visualized with 3D surface morphometry-based group ratio maps. OSA subjects showed significantly higher global putamen volumes, relative to controls. Regional analyses showed putamen areas with increased and decreased tissue volumes in OSA relative to control subjects, including increases in caudal, mid-dorsal, mid-ventral portions, and ventral regions, while areas with decreased volumes appeared in rostral, mid-dorsal, medial-caudal, and mid-ventral sites. Global putamen volumes were significantly higher in the OSA subjects, but local sites showed both higher and lower volumes. The appearance of localized volume alterations points to differential hypoxic or perfusion action on glia and other tissues within the structure, and may reflect a stage in progression of injury in these newly-diagnosed patients toward the overall volume loss found in patients with chronic OSA. The regional changes may underlie some of the specific deficits in motor, autonomic, and neuropsychologic functions in OSA.

•

Global and regional putamen volumes were examined in newly-diagnosed OSA.

•

Global volumes are higher, but subareas showed increases and decreases.

•

The volume increases suggest transient tissue swelling from hypoxic action.

•

Altered sites likely contribute to motor and other functional deficits in OSA.

Why skill matters

Maximizing rewards per unit time is ideal for success and survival in humans and animals. This goal can be approached by speeding up behavior aiming at rewards and this is done most efficiently by acquiring skills. Importantly, reward-directed skills consist of two components: finding a good object (i.e., object skill) and acting on the object (i.e., action skill), which occur sequentially. Recent studies suggest that object skill is based on high-capacity memory for object-value associations. When a learned object is encountered the corresponding memory is quickly expressed as a value-based gaze bias, leading to the automatic acquisition or avoidance of the object. Object skill thus plays a crucial role in increasing rewards per unit time.

Reward-modulated motor information in identified striatum neurons

It is widely accepted that dorsal striatum neurons participate in either the direct pathway (expressing dopamine D1 receptors) or the indirect pathway (expressing D2 receptors), controlling voluntary movements in an antagonistically balancing manner. The D1- and D2-expressing neurons are activated and inactivated, respectively, by dopamine released from substantia nigra neurons encoding reward expectation. However, little is known about the functional representation of motor information and its reward modulation in individual striatal neurons constituting the two pathways. In this study, we juxtacellularly recorded the spike activity of single neurons in the dorsolateral striatum of rats performing voluntary forelimb movement in a reward-predictable condition. Some of these neurons were identified morphologically by a combination of juxtacellular visualization and in situ hybridization for D1 mRNA. We found that the striatal neurons exhibited distinct functional activations before and during the forelimb movement, regardless of the expression of D1 mRNA. They were often positively, but rarely negatively, modulated by expecting a reward for the correct motor response. The positive reward modulation was independent of behavioral differences in motor performance. In contrast, regular-spiking and fast-spiking neurons in any layers of the motor cortex displayed only minor and unbiased reward modulation of their functional activation in relation to the execution of forelimb movement. Our results suggest that the direct and indirect pathway neurons cooperatively rather than antagonistically contribute to spatiotemporal control of voluntary movements, and that motor information is subcortically integrated with reward information through dopaminergic and other signals in the skeletomotor loop of the basal ganglia.

Different populations of subthalamic neurons encode cocaine vs. sucrose reward and predict future error

The search for treatment of cocaine addiction raises the challenge to find a way to diminish motivation for the drug without decreasing it for natural rewards. Subthalamic nucleus (STN) inactivation decreases motivation for cocaine while increasing motivation for food, suggesting that STN can dissociate different rewards. Here, we investigated how rat STN neurons respond to cues predicting cocaine or sucrose and to reward delivery while rats are performing a discriminative stimuli task. We show that different neuronal populations of STN neurons encode cocaine and sucrose. In addition, we show that STN activity at the cue onset predicts future error. When changing the reward predicted unexpectedly, STN neurons show capacities of adaptation, suggesting a role in reward-prediction error. Furthermore, some STN neurons show a response to executive error (i.e., "oops neurons") that is specific to the missed reward. These results position the STN as a nexus where natural rewards and drugs of abuse are coded differentially and can influence the performance. Therefore, STN can be viewed as a structure where action could be taken for the treatment of cocaine addiction.

Neurons in monkey dorsal raphe nucleus code beginning and progress of step-by-step schedule, reward expectation, and amount of reward outcome in the reward schedule task

The dorsal raphe nucleus is the major source of serotonin in the brain. It is connected to brain regions related to reward processing, and the neurons show activity related to predicted reward outcome. Clinical observations also suggest that it is important in maintaining alertness and its apparent role in addiction seems to be related to reward processing. Here, we examined whether the neurons in dorsal raphe carry signals about reward outcome and task progress during multitrial schedules. We recorded from 98 single neurons in dorsal raphe of two monkeys. The monkeys perform one, two, or three visual discrimination trials (schedule), obtaining one, two, or three drops of liquid. In the valid cue condition, the length and brightness of a visual cue indicated schedule progress and reward amount, respectively. In the random cue condition, the visual cue was randomly presented with respect to schedule length and reward amount. We found information encoded about (1) schedule onset, (2) reward expectation, (3) reward outcome, and (4) reward amount in the mean firing rates. Information theoretic analysis showed that the temporal variation of the neuronal responses contained additional information related to the progress of the schedule toward the reward rather than only discriminating schedule onset or reward/no reward. When considered in light of all that is known about the raphe in anatomy, physiology, and behavior, the rich encoding about both task progress and predicted reward outcome makes the raphe a strong candidate for providing signals throughout the brain to coordinate persistent goal-seeking behavior.

Social information signaling by neurons in primate striatum

Social decisions depend on reliable information about others. Consequently, social primates are motivated to acquire information about the identity, social status, and reproductive quality of others. Neurophysiological and neuroimaging studies implicate the striatum in the motivational control of behavior. Neuroimaging studies specifically implicate the ventromedial striatum in signaling motivational aspects of social interaction. Despite this evidence, precisely how striatal neurons encode social information remains unknown. Therefore, we probed the activity of single striatal neurons in monkeys choosing between visual social information at the potential expense of fluid reward. We show for the first time that a population of neurons located primarily in medial striatum selectively signals social information. Surprisingly, representation of social information was unrelated to simultaneously expressed social preferences. A largely nonoverlapping population of neurons that was not restricted to the medial striatum signaled information about fluid reward. Our findings demonstrate that information about social context and nutritive reward are maintained largely independently in striatum, even when both influence decisions to execute a single action.

Interaction between orbital prefrontal and rhinal cortex is required for normal estimates of expected value

Predicting and valuing potential rewards requires integrating sensory, associative, and contextual information with subjective reward preferences. Previous work has identified regions in the prefrontal cortex and medial temporal lobe believed to be important for each of these functions. For example, activity in the orbital prefrontal cortex (PFo) encodes the specific sensory properties of and preferences for rewards, while activity in the rhinal cortex (Rh) encodes stimulus-stimulus and stimulus-reward associations. Lesions of either structure impair the ability to use visual cues or the history of previous reinforcement to value expected rewards. These areas are linked via reciprocal connections, suggesting it might be their interaction that is critical for estimating expected value. To test this hypothesis, we interrupted direct, intra-hemispheric PFo-Rh interaction in monkeys by performing crossed unilateral ablations of these regions (functional disconnection). We asked whether this circuit is crucial primarily for cue-reward association or for estimating expected value per se, by testing these monkeys, as well as intact controls, on tasks in which expected value was either visually cued or had to be inferred from block-wise changes in reward size in uncued trials. Functional disconnection significantly affected performance in both tasks. Specifically, monkeys with functional disconnection showed less of a difference in error rates and reaction times across reward sizes, in some cases behaving as if they expected rewards to be of equal magnitude. These results support a model whereby information about rewards signaled in PFo is combined with associative and contextual information signaled within Rh to estimate expected value.

Neurobiology of addiction versus drug use driven by lack of choice

Research on the neurobiology of addiction often involves nonhuman animals that are given ready access to drugs for self-administration but without other choices. Here we argue using cocaine as an example that this standard setting may no longer be sufficient and can even lead to the formulation of unrealistic views about the neurobiology of addiction. Addiction as a psychiatric disorder is defined as resulting from brain dysfunctions that affect normal choice-making, not as an expectable response to lack of alternative choices. We encourage neurobiologists involved in addiction research to increase animals' choice during drug access, preferably by supplying alternative rewarding pursuits. Only animals that continue to take and prefer drugs despite and at the expense of other available choices may be considered as having developed an addiction-like behavior in comparison to those that remain able to stop drug use for other pursuits, even after extended drug use. The systematic comparison of these two individual behaviors should reveal new insights about the neurobiology of drug choice and addiction. More generally, this research should also shed a unique light on how the brain 'chooses' among qualitatively different kinds of pursuits.

Differential reward coding in the subdivisions of the primate caudate during an oculomotor task

The basal ganglia play a pivotal role in reward-oriented behavior. The striatum, an input channel of the basal ganglia, is composed of subdivisions that are topographically connected with different cortical and subcortical areas. To test whether reward information is differentially processed in the different parts of the striatum, we compared reward-related neuronal activity along the dorsolateral-ventromedial axis in the caudate nucleus of monkeys performing an asymmetrically rewarded oculomotor task. In a given block, a target in one position was associated with a large reward, whereas the other target was associated with a small reward. The target position-reward value contingency was switched between blocks. We found the following: (1) activity that reflected the block-wise reward contingency emerged before the appearance of a visual target, and it was more prevalent in the dorsal, rather than central and ventral, caudate; (2) activity that was positively related to the reward size of the current trial was evident, especially after reward delivery, and it was more prevalent in the ventral and central, rather than dorsal, caudate; and (3) activity that was modulated by the memory of the outcomes of the previous trials was evident in the dorsal and central caudate. This multiple reward information, together with the target-direction information, was represented primarily by individual caudate neurons, and the different reward information was represented in caudate subpopulations with distinct electrophysiological properties, e.g., baseline firing and spike width. These results suggest parallel processing of different reward information by the basal ganglia subdivisions defined by extrinsic connections and intrinsic properties.

Neuronal reference frames for social decisions in primate frontal cortex

Social decisions are crucial for the success of individuals and the groups that they comprise. Group members respond vicariously to benefits obtained by others, and impairments in this capacity contribute to neuropsychiatric disorders such as autism and sociopathy. We examined the manner in which neurons in three frontal cortical areas encoded the outcomes of social decisions as monkeys performed a reward-allocation task. Neurons in the orbitofrontal cortex (OFC) predominantly encoded rewards that were delivered to oneself. Neurons in the anterior cingulate gyrus (ACCg) encoded reward allocations to the other monkey, to oneself or to both. Neurons in the anterior cingulate sulcus (ACCs) signaled reward allocations to the other monkey or to no one. In this network of received (OFC) and foregone (ACCs) reward signaling, ACCg emerged as an important nexus for the computation of shared experience and social reward. Individual and species-specific variations in social decision-making might result from the relative activation and influence of these areas.

Is working more costly than waiting in monkeys?

We studied how value for instrumental action is discounted by predicted effort and delay. The monkeys were trained to perform instrumental trials that required a bar release when a visual target changed from red-to-green. There were two trial conditions. In delay trials, after the monkeys performed one instrumental trial correctly a reward was delivered 0-7 seconds later. In work trials, the monkeys had to perform 0, 1, or 2 additional instrumental trials to obtain a reward. The lengths of trials in delay matched the time it took to complete work trials. The length of delay or number of trials was indicated by a visual cue presented throughout the trial. Our hypothesis was that the monkeys would all show temporal discounting of reward in the delay trials, and that in the work trials the monkeys' performance might reflect an additional cost due to working. The error rate increased linearly as remaining cost increased for all 8 monkeys. For 4 monkeys the error rate was significantly larger in work trials than in delay trials (effort sensitive monkeys). For the other 4 monkeys there was no significant difference in error rate (effort insensitive monkeys). Since the error rate has an inverse relation with value for action, these results suggest that value is discounted hyperbolically by effort as well as by delay. Error rates generally increased as the testing sessions progressed and the total reward accumulated (i.e., effect of reward devaluation). The relative impact of delay and effort on error rates was reasonably stable within subjects. Thus, within the monkey population there seems to be a significant dichotomy in the sensitivity governing whether working is more costly than waiting, possibly arising from a constitutional or genetic trait.

Intersection of reward and memory in monkey rhinal cortex

In humans and other animals, the vigor with which a reward is pursued depends on its desirability, that is, on the reward's predicted value. Predicted value is generally context-dependent, varying according to the value of rewards obtained in the recent and distant past. Signals related to reward prediction and valuation are believed to be encoded in a circuit centered around midbrain dopamine neurons and their targets in the prefrontal cortex and basal ganglia. Notably absent from this hypothesized reward pathway are dopaminergic targets in the medial temporal lobe. Here we show that a key part of the medial temporal lobe memory system previously reported to be important for sensory mnemonic and perceptual processing, the rhinal cortex (Rh), is required for using memories of previous reward values to predict the value of forthcoming rewards. We tested monkeys with bilateral Rh lesions on a task in which reward size varied across blocks of uncued trials. In this experiment, the only cues for predicting current reward value are the sizes of rewards delivered in previous blocks. Unexpectedly, monkeys with Rh ablations, but not intact controls, were insensitive to differences in predicted reward, responding as if they expected all rewards to be of equal magnitude. Thus, it appears that Rh is critical for using memory of previous rewards to predict the value of forthcoming rewards. These results are in agreement with accumulating evidence that Rh is critical for establishing the relationships between temporally interleaved events, which is a key element of episodic memory.

Complementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys

Rewards have many influences on learning, decision-making, and performance. All seem to rely on complementary actions of two closely related catecholaminergic neuromodulators, dopamine (DA), and noradrenaline (NA). We compared single unit activity of dopaminergic neurons of the substantia nigra pars compacta (SNc) and noradrenergic neurons of the locus coeruleus (LC) in monkeys performing a reward schedule task. Their motivation, indexed using operant performance, increased as they progressed through schedules ending in reward delivery. The responses of dopaminergic and noradrenergic neurons around the time of major task events, visual cues predicting trial outcome and operant action to complete a trial were similar in that they occurred at the same time. They were also similar in that they both responded most strongly to the first cues in schedules, which are the most informative cues. The neuronal responses around the time of the monkeys' actions were different, in that the response intensity profiles changed in opposite directions. Dopaminergic responses were stronger around predictably rewarded correct actions whereas noradrenergic responses were greater around predictably unrewarded correct actions. The complementary response profiles related to the monkeys operant actions suggest that DA neurons might relate to the value of the current action whereas the noradrenergic neurons relate to the psychological cost of that action.

Ventral striatum encodes past and predicted value independent of motor contingencies

The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that VS can encode value independent of motor contingencies and that the role of VS in goal-directed behavior is not just to increase vigor of specific actions when more is at stake.

Encoding of reward expectation by monkey anterior insular neurons

The insula, a cortical brain region that is known to encode information about autonomic, visceral, and olfactory functions, has recently been shown to encode information during reward-seeking tasks in both single neuronal recording and functional magnetic resonance imaging studies. To examine the reward-related activation, we recorded from 170 single neurons in anterior insula of 2 monkeys during a multitrial reward schedule task, where the monkeys had to complete a schedule of 1, 2, 3, or 4 trials to earn a reward. In one block of trials a visual cue indicated whether a reward would or would not be delivered in the current trial after the monkey successfully detected that a red spot turned green, and in other blocks the visual cue was random with respect to reward delivery. Over one-quarter of 131 responsive neurons were activated when the current trial would (certain or uncertain) be rewarded if performed correctly. These same neurons failed to respond in trials that were certain, as indicated by the cue, to be unrewarded. Another group of neurons responded when the reward was delivered, similar to results reported previously. The dynamics of population activity in anterior insula also showed strong signals related to knowing when a reward is coming. The most parsimonious explanation is that this activity codes for a type of expected outcome, where the expectation encompasses both certain and uncertain rewards.

Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex

Background

The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing.

Methods and Findings

We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1–4 sequential color discrimination trials to obtain a reward of 1–3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount.

Conclusions

Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.