Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys

Effects of a 5-HT4 receptor antagonist in the caudate nucleus on the performance of macaques in a delayed reward task

Temporal discounting, in which the recipient of a reward perceives the value of that reward to decrease with delay in its receipt, is associated with impulsivity and psychiatric disorders such as depression. Here, we investigate the role of the serotonin 5-HT4 receptor (5-HT4R) in modulating temporal discounting in the macaque dorsal caudate nucleus (dCDh), the neurons of which have been shown to represent temporally discounted value. We first mapped the 5-HT4R distribution in macaque brains using positron emission tomography (PET) imaging and confirmed dense expression of 5-HT4R in the dCDh. We then examined the effects of a specific 5-HT4R antagonist infused into the dCDh. Blockade of 5-HT4R significantly increased error rates in a goal-directed delayed reward task, indicating an increase in the rate of temporal discounting. This increase was specific to the 5-HT4R blockade because saline controls showed no such effect. The results demonstrate that 5-HT4Rs in the dCDh are involved in reward-evaluation processes, particularly in the context of delay discounting, and suggest that serotonergic transmission via 5-HT4R may be a key component in the neural mechanisms underlying impulsive decisions, potentially contributing to depressive symptoms.

Reward value and internal state differentially drive impulsivity and motivation

Goal-directed behavior is influenced by both reward value as well as internal state. A large body of research has focused on how reward value and internal drives such as hunger influence motivation in rodent models, however less work has focused on how these factors may differentially affect impulsivity. In these studies, we tested the effect of internal drive versus reward value on different facets of reward-related behavior including impulsive action, impulsive choice and, motivation. We varied reward value by changing the concentration of sucrose in the reward outcome, and varied internal drive by manipulating thirst through water restriction. Consistent with the literature we found that both internal state and reward value influenced motivation. However, we found that in high effort paradigms, only internal state influenced motivation with minimal effects of reward value. Interestingly, we found that internal state and reward value differentially influence different subtypes of impulsivity. Internal state, and to a lesser extent, reward value, influenced impulsive action as measured by premature responding. On the other hand, there were minimal effects of either reward value or homeostatic state on impulsive choice as measured by delay discounting. Overall, these studies begin to address how internal state and reward value differentially drive impulsive behavior. Understanding how these factors influence impulsivity is important for developing behavioral interventions and treatment targets for patients with dysregulated motivated or impulsive behavior.

Dynamic inconsistency in great apes

When presented with the option of either an immediate benefit or a larger, later reward, we may behave impatiently by choosing instant gratification. Nonetheless, when we can make the same decision ahead of time and plan for the future, we tend to make more patient choices. Here, we explored whether great apes share this core feature of human decision-making, often referred to as dynamic inconsistency. We found that orangutans, bonobos, and gorillas tended to act impatiently and with considerable variability between individuals when choosing between an immediate reward and a larger-later reward, which is a commonly employed testing method in the field. However, with the inclusion of a front-end delay for both alternatives, their decisions became more patient and homogeneous. These results show that great apes are dynamically inconsistent. They also suggest that, when choosing between future outcomes, they are more patient than previously reported. We advocate for the inclusion of diverse time ranges in comparative research, especially considering the intertwinement of intertemporal choices and future-oriented behavior.

General mechanisms of task engagement in the primate frontal cortex

Staying engaged is necessary to maintain goal-directed behaviors. Despite this, engagement exhibits continuous, intrinsic fluctuations. Even in experimental settings, animals, unlike most humans, repeatedly and spontaneously move between periods of complete task engagement and disengagement. We, therefore, looked at behavior in male macaques (macaca mulatta) in four tasks while recording fMRI signals. We identified consistent autocorrelation in task disengagement. This made it possible to build models capturing task-independent engagement. We identified task general patterns of neural activity linked to impending sudden task disengagement in mid-cingulate gyrus. By contrast, activity centered in perigenual anterior cingulate cortex (pgACC) was associated with maintenance of performance across tasks. Importantly, we carefully controlled for task-specific factors such as the reward history and other motivational effects, such as response vigor, in our analyses. Moreover, we showed pgACC activity had a causal link to task engagement: transcranial ultrasound stimulation of pgACC changed task engagement patterns.

Reduced serotonergic transmission alters sensitivity to cost and reward via 5-HT1A and 5-HT1B receptors in monkeys

Serotonin (5-HT) deficiency is a core biological pathology underlying depression and other psychiatric disorders whose key symptoms include decreased motivation. However, the exact role of 5-HT in motivation remains controversial and elusive. Here, we pharmacologically manipulated the 5-HT system in macaque monkeys and quantified the effects on motivation for goal-directed actions in terms of incentives and costs. Reversible inhibition of 5-HT synthesis increased errors and reaction times on goal-directed tasks, indicating reduced motivation. Analysis found incentive-dependent and cost-dependent components of this reduction. To identify the receptor subtypes that mediate cost and incentive, we systemically administered antagonists specific to 4 major 5-HT receptor subtypes: 5-HT1A, 5-HT1B, 5-HT2A, and 5-HT4. Positron emission tomography (PET) visualized the unique distribution of each subtype in limbic brain regions and determined the systemic dosage for antagonists that would achieve approximately 30% occupancy. Only blockade of 5-HT1A decreased motivation through changes in both expected cost and incentive; sensitivity to future workload and time delay to reward increased (cost) and reward value decreased (incentive). Blocking the 5-HT1B receptor also reduced motivation through decreased incentive, although it did not affect expected cost. These results suggest that 5-HT deficiency disrupts 2 processes, the subjective valuation of costs and rewards, via 5-HT1A and 5-HT1B receptors, thus leading to reduced motivation.

Neural correlates of category learning in monkey inferior temporal cortex

We trained two monkeys implanted with multi-electrode arrays to categorize natural images of cats and dogs, in order to observe changes in neural activity related to category learning. We recorded neural activity from area TE, which is required for normal learning of visual categories based on perceptual similarity. Neural activity during a passive viewing task was compared pre- and post-training. After the category training, the accuracy of abstract category decoding improved. Specifically, the proportion of single units with category selectivity increased, and units sustained their category-specific responses for longer. Visual category learning thus appears to enhance category separability in area TE by driving changes in the stimulus selectivity of individual neurons and by recruiting more units to the active network.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here we provide evidence that the energy landscape around attractor basins in population neural activity in the prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays to reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here, we provide evidence that the energy landscape around attractor basins in population neural activity in prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays-to-reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Neural dynamics underlying self-control in the primate subthalamic nucleus

The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost-benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.

Hypothalamic interaction with reward-related regions during subjective evaluation of foods

The reward system implemented in the midbrain, ventral striatum, orbitofrontal cortex, and ventromedial prefrontal cortex evaluates and compares various types of rewards given to the organisms. It has been suggested that autonomic factors influence reward-related processing via the hypothalamus, but how the hypothalamus modulates the reward system remains elusive. In this functional magnetic resonance imaging study, the hypothalamus was parcellated into individual hypothalamic nuclei performing different autonomic functions using boundary mapping parcellation analyses. The effective interaction during subjective evaluation of foods in a reward task was then investigated between the human hypothalamic nuclei and the reward-related regions. We found significant brain activity decrease in the paraventricular nucleus (PVH) and lateral nucleus in the hypothalamus in food evaluation compared with monetary evaluation. A psychophysiological interaction analysis revealed dual interactions between the PVH and (1) midbrain region and (2) ventromedial prefrontal cortex, with the former correlated with the stronger tendency of participants toward food-seeking. A dynamic causal modeling analysis further revealed unidirectional interactions from the PVH to the midbrain and ventromedial prefrontal cortex. These results suggest that the PVH in the human hypothalamus interacts with the reward-related regions in the cerebral cortex via multiple pathways (i.e., the midbrain pathway and ventromedial prefrontal pathway) to evaluate rewards for subsequent decision-making.

Chemogenetic Disconnection between the Orbitofrontal Cortex and the Rostromedial Caudate Nucleus Disrupts Motivational Control of Goal-Directed Action

The orbitofrontal cortex (OFC) and its major downstream target within the basal ganglia-the rostromedial caudate nucleus (rmCD)-are involved in reward-value processing and goal-directed behavior. However, a causal contribution of the pathway linking these two structures to goal-directed behavior has not been established. Using the chemogenetic technology of designer receptors exclusively activated by designer drugs with a crossed inactivation design, we functionally and reversibly disrupted interactions between the OFC and rmCD in two male macaque monkeys. We injected an adeno-associated virus vector expressing an inhibitory designer receptor, hM4Di, into the OFC and contralateral rmCD, the expression of which was visualized in vivo by positron emission tomography and confirmed by postmortem immunohistochemistry. Functional disconnection of the OFC and rmCD resulted in a significant and reproducible loss of sensitivity to the cued reward value for goal-directed action. This decreased sensitivity was most prominent when monkeys had accumulated a certain amount of reward. These results provide causal evidence that the interaction between the OFC and the rmCD is needed for motivational control of action on the basis of the relative reward value and internal drive. This finding extends the current understanding of the physiological basis of psychiatric disorders in which goal-directed behavior is affected, such as obsessive-compulsive disorder.SIGNIFICANCE STATEMENT In daily life, we routinely adjust the speed and accuracy of our actions on the basis of the value of expected reward. Abnormalities in these kinds of motivational adjustments might be related to behaviors seen in psychiatric disorders such as obsessive-compulsive disorder. In the current study, we show that the connection from the orbitofrontal cortex to the rostromedial caudate nucleus is essential for motivational control of action in monkeys. This finding expands our knowledge about how the primate brain controls motivation and behavior and provides a particular insight into disorders like obsessive-compulsive disorder in which altered connectivity between the orbitofrontal cortex and the striatum has been implicated.

Influence of Rule- and Reward-based Strategies on Inferences of Serial Order by Monkeys

Knowledge of transitive relationships between items can contribute to learning the order of a set of stimuli from pairwise comparisons. However, cognitive mechanisms of transitive inferences based on rank order remain unclear, as are relative contributions of reward associations and rule-based inference. To explore these issues, we created a conflict between rule- and reward-based learning during a serial ordering task. Rhesus macaques learned two lists, each containing five stimuli that were trained exclusively with adjacent pairs. Selection of the higher-ranked item resulted in rewards. "Small reward" lists yielded two drops of fluid reward, whereas "large reward" lists yielded five drops. Following training of adjacent pairs, monkeys were tested on novels pairs. One item was selected from each list, such that a ranking rule could conflict with preferences for large rewards. Differences between the corresponding reward magnitudes had a strong influence on accuracy, but we also observed a symbolic distance effect. That provided evidence of a rule-based influence on decisions. RT comparisons suggested a conflict between rule- and reward-based processes. We conclude that performance reflects the contributions of two strategies and that a model-based strategy is employed in the face of a strong countervailing reward incentive.

How the value of the environment controls persistence in visual search

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.

Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions

The brain stem noradrenergic nucleus locus coeruleus (LC) is involved in various costly processes: arousal, stress, and attention. Recent work has pointed toward an implication in physical effort, and indirect evidence suggests that the LC could be also involved in cognitive effort. To assess the dynamic relation between LC activity, effort production, and difficulty, we recorded the activity of 193 LC single units in 5 monkeys performing 2 discounting tasks (a delay discounting task and a force discounting task), as well as a simpler target detection task where conditions were matched for difficulty and only differed in terms of sensory-motor processes. First, LC neurons displayed a transient activation both when monkeys initiated an action and when exerting force. Second, the magnitude of the activation scaled with the associated difficulty, and, potentially, the corresponding amount of effort produced, both for decision and force production. Indeed, at action initiation in both discounting tasks, LC activation increased in conditions associated with lower average engagement rate, i.e., those requiring more cognitive control to trigger the response. Decision-related activation also scaled with response time (RT), over and above task parameters, in line with the idea that it reflects the amount of resources (here time) spent on the decision process. During force production, LC activation only scaled with the amount of force produced in the force discounting task, but not in the control target detection task, where subjective difficulty was equivalent across conditions. Our data show that LC neurons dynamically track the amount of effort produced to face both cognitive and physical challenges with a subsecond precision. This works provides key insight into effort processing and the contribution of the noradrenergic system, which is affected in several pathologies where effort is impaired, including Parkinson disease and depression.

Single caudate neurons encode temporally discounted value for formulating motivation for action

The term 'temporal discounting' describes both choice preferences and motivation for delayed rewards. Here we show that neuronal activity in the dorsal part of the primate caudate head (dCDh) signals the temporally discounted value needed to compute the motivation for delayed rewards. Macaque monkeys performed an instrumental task, in which visual cues indicated the forthcoming size and delay duration before reward. Single dCDh neurons represented the temporally discounted value without reflecting changes in the animal's physiological state. Bilateral pharmacological or chemogenetic inactivation of dCDh markedly distorted the normal task performance based on the integration of reward size and delay, but did not affect the task performance for different reward sizes without delay. These results suggest that dCDh is involved in encoding the integrated multi-dimensional information critical for motivation.

D1- and D2-like receptors differentially mediate the effects of dopaminergic transmission on cost-benefit evaluation and motivation in monkeys

It has been widely accepted that dopamine (DA) plays a major role in motivation, yet the specific contribution of DA signaling at D₁-like receptor (D₁R) and D₂-like receptor (D₂R) to cost–benefit trade-off remains unclear. Here, by combining pharmacological manipulation of DA receptors (DARs) and positron emission tomography (PET) imaging, we assessed the relationship between the degree of D₁R/D₂R blockade and changes in benefit- and cost-based motivation for goal-directed behavior of macaque monkeys. We found that the degree of blockade of either D₁R or D₂R was associated with a reduction of the positive impact of reward amount and increasing delay discounting. Workload discounting was selectively increased by D₂R antagonism. In addition, blocking both D₁R and D₂R had a synergistic effect on delay discounting but an antagonist effect on workload discounting. These results provide fundamental insight into the distinct mechanisms of DA action in the regulation of the benefit- and cost-based motivation, which have important implications for motivational alterations in both neurological and psychiatric disorders.

Using quantitatively controlled pharmacological manipulations, this study teases apart the role of D1- and D2-like dopamine receptors in motivation and goal-directed behavior in monkeys, revealing complementary roles of two dopamine receptor subtypes in the computation of the cost/benefit trade-off to guide action.

GLP-1 and hunger modulate incentive motivation depending on insulin sensitivity in humans

OBJECTIVE

To regulate food intake, our brain constantly integrates external cues, such as the incentive value of a potential food reward, with internal state signals, such as hunger feelings. Incentive motivation refers to the processes that translate an expected reward into the effort spent to obtain the reward; the magnitude and probability of a reward involved in prompting motivated behaviour are encoded by the dopaminergic (DA) midbrain and its mesoaccumbens DA projections. This type of reward circuity is particularly sensitive to the metabolic state signalled by peripheral mediators, such as insulin or glucagon-like peptide 1 (GLP-1). While in rodents the modulatory effect of metabolic state signals on motivated behaviour is well documented, evidence of state-dependent modulation and the role of incentive motivation underlying overeating in humans is lacking.

METHODS

In a randomised, placebo-controlled, crossover design, 21 lean (body mass index [BMI] < 25 kg/m²) and 16 obese (BMI³ 30 kg/m²) volunteer participants received either liraglutide as a GLP-1 analogue or placebo on two separate testing days. Incentive motivation was measured using a behavioural task in which participants were required to exert physical effort using a handgrip to win different amounts of food and monetary rewards. Hunger levels were measured using visual analogue scales; insulin, glucose, and systemic insulin resistance as assessed by the homeostasis model assessment of insulin resistance (HOMA-IR) were quantified at baseline.

RESULTS

In this report, we demonstrate that incentive motivation increases with hunger in lean humans (F_(1,42) = 5.31, p = 0.026, β = 0.19) independently of incentive type (food and non-food reward). This effect of hunger is not evident in obese humans (F_(1,62) = 1.93, p = 0.17, β = -0.12). Motivational drive related to hunger is affected by peripheral insulin sensitivity (two-way interaction, F_{(1, 35)} = 6.23, p = 0.017, β = -0.281). In humans with higher insulin sensitivity, hunger increases motivation, while poorer insulin sensitivity dampens the motivational effect of hunger. The GLP-1 analogue application blunts the interaction effect of hunger on motivation depending on insulin sensitivity (three-way interaction, F_{(1, 127)} = 5.11, p = 0.026); no difference in motivated behaviour could be found between humans with normal or impaired insulin sensitivity under GLP-1 administration.

CONCLUSION

We report a differential effect of hunger on motivation depending on insulin sensitivity. We further revealed the modulatory role of GLP-1 in adaptive, motivated behaviour in humans and its interaction with peripheral insulin sensitivity and hunger. Our results suggest that GLP-1 might restore dysregulated processes of midbrain DA function and hence motivational behaviour in insulin-resistant humans.

Balancing costs and benefits in primates: ecological and palaeoanthropological views

Maintaining the balance between costs and benefits is challenging for species living in complex and dynamic socio-ecological environments, such as primates, but also crucial for shaping life history, reproductive and feeding strategies. Indeed, individuals must decide to invest time and energy to obtain food, services and partners, with little direct feedback on the success of their investments. Whereas decision-making relies heavily upon cognition in humans, the extent to which it also involves cognition in other species, based on their environmental constraints, has remained a challenging question. Building mental representations relating behaviours and their long-term outcome could be critical for other primates, but there are actually very little data relating cognition to real socio-ecological challenges in extant and extinct primates. Here, we review available data illustrating how specific cognitive processes enable(d) modern primates and extinct hominins to manage multiple resources (e.g. food, partners) and to organize their behaviour in space and time, both at the individual and at the group level. We particularly focus on how they overcome fluctuating and competing demands, and select courses of action corresponding to the best possible packages of potential costs and benefits in reproductive and foraging contexts. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Hypothalamic Interactions with Large-Scale Neural Circuits Underlying Reinforcement Learning and Motivated Behavior

Biological agents adapt behavior to support the survival needs of the individual and the species. In this review we outline the anatomical, physiological, and computational processes that support reinforcement learning (RL). We describe two circuits in the primate brain that are linked to specific aspects of learning and goal-directed behavior. The ventral circuit, that includes the amygdala, ventral medial prefrontal cortex, and ventral striatum, has substantial connectivity with the hypothalamus. The dorsal circuit, that includes inferior parietal cortex, dorsal lateral prefrontal cortex, and the dorsal striatum, has minimal connectivity with the hypothalamus. The hypothalamic connectivity suggests distinct roles for these circuits. We propose that the ventral circuit defines behavioral goals, and the dorsal circuit orchestrates behavior to achieve those goals.

Deschloroclozapine, a potent and selective chemogenetic actuator enables rapid neuronal and behavioral modulations in mice and monkeys

The chemogenetic technology designer receptors exclusively activated by designer drugs (DREADDs) afford remotely reversible control of cellular signaling, neuronal activity and behavior. Although the combination of muscarinic-based DREADDs with clozapine-N-oxide (CNO) has been widely used, sluggish kinetics, metabolic liabilities and potential off-target effects of CNO represent areas for improvement. Here, we provide a new high-affinity and selective agonist deschloroclozapine (DCZ) for muscarinic-based DREADDs. Positron emission tomography revealed that DCZ selectively bound to and occupied DREADDs in both mice and monkeys. Systemic delivery of low doses of DCZ (1 or 3 μg per kg) enhanced neuronal activity via hM3Dq within minutes in mice and monkeys. Intramuscular injections of DCZ (100 μg per kg) reversibly induced spatial working memory deficits in monkeys expressing hM4Di in the prefrontal cortex. DCZ represents a potent, selective, metabolically stable and fast-acting DREADD agonist with utility in both mice and nonhuman primates for a variety of applications.

Noradrenergic But Not Dopaminergic Neurons Signal Task State Changes and Predict Reengagement After a Failure

The two catecholamines, noradrenaline and dopamine, have been shown to play comparable roles in behavior. Both noradrenergic and dopaminergic neurons respond to cues predicting reward availability and novelty. However, even though both are thought to be involved in motivating actions, their roles in motivation have seldom been directly compared. We therefore examined the activity of putative noradrenergic neurons in the locus coeruleus and putative midbrain dopaminergic neurons in monkeys cued to perform effortful actions for rewards. The activity in both regions correlated with engagement with a presented option. By contrast, only noradrenaline neurons were also (i) predictive of engagement in a subsequent trial following a failure to engage and (ii) more strongly activated in nonrepeated trials, when cues indicated a new task condition. This suggests that while both catecholaminergic neurons are involved in promoting action, noradrenergic neurons are sensitive to task state changes, and their influence on behavior extends beyond the immediately rewarded action.

Trait and State-Dependent Risk Attitude of Monkeys Measured in a Single-Option Response Task

Humans and animals show diverse preferences for risks (“trait-like” risk attitude) and shift their preference depending on the state or current needs (“state-dependent” risk attitude). For a better understanding of the neural mechanisms underlying risk-sensitive decisions, useful animal models have been required. Here we examined the risk attitude of three male monkeys in a single-option response task, in which an instrumental lever-release was required to obtain a chance of reward. In each trial, reward condition, either deterministic (100% of 1, 2, 3, and 4 drops of juice) or probabilistic (25, 50, 75, and 100% of 4-drop juice) was randomly selected and assigned by a unique visual cue, allowing the monkeys to evaluate the forthcoming reward. The subjective value of the reward was inferred from their performance. Model-based analysis incorporating known economic models revealed non-linear probability distortion in monkeys; unlike previous studies, they showed a simple convex or concave probability distortion curve. The direction of risk preference was consistent between early and late phases of the testing period, suggesting that our observation reflected the trait-like risk attitude of monkeys, at least under the current experimental setting. Regardless of the baseline risk preference, all monkeys showed an enhancement of risk preference in a session according to the satiation level (i.e., state-dependent risk attitude). Our results suggest that, without choice or cognitive demand, monkeys show naturalistic risk attitude – diverse and flexible like humans. Our novel approach may provide a useful animal model of risk-sensitive decisions, facilitating the investigation of the neural mechanisms of decision-making under risk.

Effects of reward size and context on learning in macaque monkeys

The outcome of an action plays a crucial role in decision-making and reinforcement learning processes. Indeed, both human and animal behavioural studies have shown that different expected reward values, either quantitatively or qualitatively, modulate the motivation of subjects to perform an action and, as a consequence, affect their behavioural performance. Here, we investigated the effect of different amounts of reward on the learning of macaque monkeys using a modified version of the object-in-place task. This task offers the opportunity to shape rapid learning based on a set of external stimuli that enhance an animal's accuracy in terms of solving a problem. We compared the learning of three monkeys among three different reward conditions. Our results demonstrate that the larger the reward, the better the monkey's ability to learn the associations starting with the second presentation of the problem. Moreover, we compared the present results with those of our previous work using the same monkeys in the same task but with a unique reward condition, the intermediate one. Interestingly, the performance of our animals in our previous work matched with their performance in the largest and not intermediate reward condition of the present study These results suggest that learning is mostly influenced by the reward context and not by its absolute value.

Neurons in the monkey orbitofrontal cortex mediate reward value computation and decision-making

Choice reflects the values of available alternatives; more valuable options are chosen more often than less valuable ones. Here we studied whether neuronal responses in orbitofrontal cortex (OFC) reflect the value difference between options, and whether there is a causal link between OFC neuronal activity and choice. Using a decision-making task where two visual stimuli were presented sequentially, each signifying a value, we showed that when the second stimulus appears many neurons encode the value difference between alternatives. Later when the choice occurs, that difference signal disappears and a signal indicating the chosen value emerges. Pharmacological inactivation of OFC neurons coding for choice-related values increases the monkey's latency to make a choice and the likelihood that it will choose the less valuable alternative, when the value difference is small. Thus, OFC neurons code for value information that could be used to directly influence choice.

Reward cues readily direct monkeys' auditory performance resulting in broad auditory cortex modulation and interaction with sites along cholinergic and dopaminergic pathways

In natural settings, the prospect of reward often influences the focus of our attention, but how cognitive and motivational systems influence sensory cortex is not well understood. Also, challenges in training nonhuman animals on cognitive tasks complicate cross-species comparisons and interpreting results on the neurobiological bases of cognition. Incentivized attention tasks could expedite training and evaluate the impact of attention on sensory cortex. Here we develop an Incentivized Attention Paradigm (IAP) and use it to show that macaque monkeys readily learn to use auditory or visual reward cues, drastically influencing their performance within a simple auditory task. Next, this paradigm was used with functional neuroimaging to measure activation modulation in the monkey auditory cortex. The results show modulation of extensive auditory cortical regions throughout primary and non-primary regions, which although a hallmark of attentional modulation in human auditory cortex, has not been studied or observed as broadly in prior data from nonhuman animals. Psycho-physiological interactions were identified between the observed auditory cortex effects and regions including basal forebrain sites along acetylcholinergic and dopaminergic pathways. The findings reveal the impact and regional interactions in the primate brain during an incentivized attention engaging auditory task.

Impact of stimulus format and reward value on quantity discrimination in capuchin and squirrel monkeys

Quantity discrimination abilities are seen in a diverse range of species with similarities in performance patterns, suggesting common underlying cognitive mechanisms. However, methodological factors that impact performance make it difficult to draw broad phylogenetic comparisons of numerical cognition across studies. For example, some Old World monkeys selected a higher quantity stimulus more frequently when choosing between inedible (pebbles) than edible (food) stimuli. In Experiment 1 we presented brown capuchin (Cebus [Sapajus] paella) and squirrel monkeys (Saimiri sciureus) with the same two-choice quantity discrimination task in three different stimulus conditions: edible, inedible, and edible replaced (in which choice stimuli were food items that stood in for the same quantity of food items that were given as a reward). Unlike Old World monkeys, capuchins selected the higher quantity stimulus more in the edible condition and squirrel monkeys showed generally poor performance across all stimulus types. Performance patterns suggested that differences in subjective reward value might motivate differences in choice behavior between and within species. In Experiment 2 we manipulated the subjective reinforcement value of the reward by varying reward type and delay to reinforcement and found that delay to reinforcement had no impact on choice behavior, while increasing the value of the reward significantly improved performance by both species. The results of this study indicate that species presented with identical tasks may respond differently to methodological factors such as stimulus and reward types, resulting in significant differences in choice behavior that may lead to spurious suggestions of species differences in cognitive abilities.

Signaling Incentive and Drive in the Primate Ventral Pallidum for Motivational Control of Goal-Directed Action

Processing incentive and drive is essential for control of goal-directed behavior. The limbic part of the basal ganglia has been emphasized in these processes, yet the exact neuronal mechanism has remained elusive. In this study, we examined the neuronal activity of the ventral pallidum (VP) and its upstream area, the rostromedial caudate (rmCD), while two male macaque monkeys performed an instrumental lever release task in which a visual cue indicated the forthcoming reward size. We found that the activity of some neurons in VP and rmCD reflected the expected reward size transiently following the cue. Reward size coding appeared earlier and stronger in VP than in rmCD. We also found that the activity in these areas was modulated by the satiation level of monkeys, which also occurred more frequently in VP than in rmCD. The information regarding reward size and satiation level was independently signaled in the neuronal populations of these areas. The data thus highlighted the neuronal coding of key variables for goal-directed behavior in VP. Furthermore, pharmacological inactivation of VP induced more severe deficit of goal-directed behavior than inactivation of rmCD, which was indicated by abnormal error repetition and diminished satiation effect on the performance. These results suggest that VP encodes incentive value and internal drive and plays a pivotal role in the control of motivation to promote goal-directed behavior.

SIGNIFICANCE STATEMENT The limbic part of the basal ganglia has been implicated in the motivational control of goal-directed action. Here, we investigated how the ventral pallidum (VP) and the rostromedial caudate (rmCD) encode incentive value and internal drive and control goal-directed behavior. Neuronal recording and subsequent pharmacological inactivation revealed that VP had stronger coding of reward size and satiation level than rmCD. Reward size and satiation level were independently encoded in the neuronal population of these areas. Furthermore, VP inactivation impaired goal-directed behavior more severely than rmCD inactivation. These results highlight the central role of VP in the motivational control of goal-directed action.

Using preferred fluids and different reward schedules to motivate rhesus macaques (Macaca mulatta) in cognitive tasks

Rhesus macaques (Macaca mulatta) used in behavioural neuroscience are often required to complete cognitively complex tasks, for which a high level of motivation is essential. To induce motivation, researchers may implement fluid-restriction protocols, whereby freely available water is limited, such that fluid can be used as a reward in the laboratory. A variety of different rewards and schedules are used, but there exists a lack of data assessing their effectiveness. In this study, we aimed to quantify fluid preference in rhesus macaques and to use these preferences to compare the motivational quality of different reward schedules: the monkey's previous reward (i.e. the fluid used to reward them in past studies), their new preferred reward, a variable schedule of previous and preferred reward, and a choice between the previous and preferred rewards. We found that it may be possible to reduce the level of restriction if an adequately motivating preferred reward is identified, but that this is dependent on the animal. Each monkey responded differently to both the fluid-preference assessments and to the different reward schedules. As such, monkeys should not be subject to 'blanket' protocols but should be assessed individually to maintain adequate scientific data collection at the least severe level of fluid restriction.

Contributions of Lateral and Orbital Frontal Regions to Abstract Rule Acquisition and Reversal in Monkeys

The ability to learn and follow abstract rules relies on intact prefrontal regions including the lateral prefrontal cortex (LPFC) and the orbitofrontal cortex (OFC). Here, we investigate the specific roles of these brain regions in learning rules that depend critically on the formation of abstract concepts as opposed to simpler input-output associations. To this aim, we tested monkeys with bilateral removals of either LPFC or OFC on a rapidly learned task requiring the formation of the abstract concept of same vs. different. While monkeys with OFC removals were significantly slower than controls at both acquiring and reversing the concept-based rule, monkeys with LPFC removals were not impaired in acquiring the task, but were significantly slower at rule reversal. Neither group was impaired in the acquisition or reversal of a delayed visual cue-outcome association task without a concept-based rule. These results suggest that OFC is essential for the implementation of a concept-based rule, whereas LPFC seems essential for its modification once established.

PET imaging-guided chemogenetic silencing reveals a critical role of primate rostromedial caudate in reward evaluation

The rostromedial caudate (rmCD) of primates is thought to contribute to reward value processing, but a causal relationship has not been established. Here we use an inhibitory DREADD (Designer Receptor Exclusively Activated by Designer Drug) to repeatedly and non-invasively inactivate rmCD of macaque monkeys. We inject an adeno-associated viral vector expressing the inhibitory DREADD, hM4Di, into the rmCD bilaterally. To visualize DREADD expression in vivo, we develop a non-invasive imaging method using positron emission tomography (PET). PET imaging provides information critical for successful chemogenetic silencing during experiments, in this case the location and level of hM4Di expression, and the relationship between agonist dose and hM4Di receptor occupancy. Here we demonstrate that inactivating bilateral rmCD through activation of hM4Di produces a significant and reproducible loss of sensitivity to reward value in monkeys. Thus, the rmCD is involved in making normal judgments about the value of reward.

Robust modulation of arousal regulation, performance, and frontostriatal activity through central thalamic deep brain stimulation in healthy nonhuman primates

The central thalamus (CT) is a key component of the brain-wide network underlying arousal regulation and sensory-motor integration during wakefulness in the mammalian brain. Dysfunction of the CT, typically a result of severe brain injury (SBI), leads to long-lasting impairments in arousal regulation and subsequent deficits in cognition. Central thalamic deep brain stimulation (CT-DBS) is proposed as a therapy to reestablish and maintain arousal regulation to improve cognition in select SBI patients. However, a mechanistic understanding of CT-DBS and an optimal method of implementing this promising therapy are unknown. Here we demonstrate in two healthy nonhuman primates (NHPs), Macaca mulatta, that location-specific CT-DBS improves performance in visuomotor tasks and is associated with physiological effects consistent with enhancement of endogenous arousal. Specifically, CT-DBS within the lateral wing of the central lateral nucleus and the surrounding medial dorsal thalamic tegmental tract (DTTm) produces a rapid and robust modulation of performance and arousal, as measured by neuronal activity in the frontal cortex and striatum. Notably, the most robust and reliable behavioral and physiological responses resulted when we implemented a novel method of CT-DBS that orients and shapes the electric field within the DTTm using spatially separated DBS leads. Collectively, our results demonstrate that selective activation within the DTTm of the CT robustly regulates endogenous arousal and enhances cognitive performance in the intact NHP; these findings provide insights into the mechanism of CT-DBS and further support the development of CT-DBS as a therapy for reestablishing arousal regulation to support cognition in SBI patients.

Incubation of food craving is independent of macronutrient composition

Cues previously paired with rewarding stimuli induce a time-dependent increase in the motivational craving state (incubation of craving). Whether there is an increase in craving for high-fat (HF) food over time, which may contribute to overeating and obesity, has not been determined. We hypothesized that cues paired with HF pellets would elicit a greater incubation of craving effect than those paired with standard chow (SC) pellets. Rats exposed to cues associated with either HF or SC pellets demonstrated equivalent levels of craving over an abstinence period of 30 days. Diet preference tests between SC pellets and LabDiet revealed that SC pellets were preferred over LabDiet. Rats reared on SC pellets exclusively, did not display incubation of craving for SC pellets, suggesting that prior history with the food plays an important role in cue-induced seeking behavior. Results identified cues previously associated with food undergo a comparable magnitude of incubation of craving. When ingestive behavior was measured after 30 days of abstinence, rats significantly increased their consumption of HF pellets. Our results indicate that food cues gain importance over time, trigger increased approach behaviors, and increased consumption of HF food following abstinence. This may contribute to overeating and the development of obesity.

Chemogenetic disconnection of monkey orbitofrontal and rhinal cortex reversibly disrupts reward value

To study how the interaction between orbitofrontal (OFC) and rhinal (Rh) cortices influences the judgment of reward size, we reversibly disconnected these regions using the hM4Di-DREADD (Designer Receptor Exclusively Activated by Designer Drug). Repeated inactivation reduced sensitivity to differences in reward size in two monkeys. Results suggest that retrieval of relative stimulus values from memory appears to depend on interaction between Rh and OFC.

Characteristics of fast-spiking neurons in the striatum of behaving monkeys

Inhibitory interneurons are the fundamental constituents of neural circuits that organize network outputs. The striatum as part of the basal ganglia is involved in reward-directed behaviors. However, the role of the inhibitory interneurons in this process remains unclear, especially in behaving monkeys. We recorded the striatal single neuron activity while monkeys performed reward-directed hand or eye movements. Presumed parvalbumin-containing GABAergic interneurons (fast-spiking neurons, FSNs) were identified based on narrow spike shapes in three independent experiments, though they were a small population (4.2%, 42/997). We found that FSNs are characterized by high-frequency and less-bursty discharges, which are distinct from the basic firing properties of the presumed projection neurons (phasically active neurons, PANs). Besides, the encoded information regarding actions and outcomes was similar between FSNs and PANs in terms of proportion of neurons, but the discharge selectivity was higher in PANs than that of FSNs. The coding of actions and outcomes in FSNs and PANs was consistently observed under various behavioral contexts in distinct parts of the striatum (caudate nucleus, putamen, and anterior striatum). Our results suggest that FSNs may enhance the discharge selectivity of postsynaptic output neurons (PANs) in encoding crucial variables for a reward-directed behavior.

Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys

Motivation determines multiple aspects of behavior, including action selection and energization of behavior. Several components of the underlying neural systems have been examined closely, but the specific role of the different neuromodulatory systems in motivation remains unclear. Here, we compare directly the activity of dopaminergic neurons from the substantia nigra pars compacta and noradrenergic neurons from the locus coeruleus in monkeys performing a task manipulating the reward/effort trade-off. Consistent with previous reports, dopaminergic neurons encoded the expected reward, but we found that they also anticipated the upcoming effort cost in connection with its negative influence on action selection. Conversely, the firing of noradrenergic neurons increased with both pupil dilation and effort production in relation to the energization of behavior. Therefore, this work underlines the contribution of dopamine to effort-based decision making and uncovers a specific role of noradrenaline in energizing behavior to face challenges.

Sensitivity of locus ceruleus neurons to reward value for goal-directed actions

The noradrenergic nucleus locus ceruleus (LC) is associated classically with arousal and attention. Recent data suggest that it might also play a role in motivation. To study how LC neuronal responses are related to motivational intensity, we recorded 121 single neurons from two monkeys while reward size (one, two, or four drops) and the manner of obtaining reward (passive vs active) were both manipulated. The monkeys received reward under three conditions: (1) releasing a bar when a visual target changed color; (2) passively holding a bar; or (3) touching and releasing a bar. In the first two conditions, a visual cue indicated the size of the upcoming reward, and, in the third, the reward was constant through each block of 25 trials. Performance levels and lipping intensity (an appetitive behavior) both showed that the monkeys' motivation in the task was related to the predicted reward size. In conditions 1 and 2, LC neurons were activated phasically in relation to cue onset, and this activation strengthened with increasing expected reward size. In conditions 1 and 3, LC neurons were activated before the bar-release action, and the activation weakened with increasing expected reward size but only in task 1. These effects evolved as monkeys progressed through behavioral sessions, because increasing fatigue and satiety presumably progressively decreased the value of the upcoming reward. These data indicate that LC neurons integrate motivationally relevant information: both external cues and internal drives. The LC might provide the impetus to act when the predicted outcome value is low.

Dopaminergic lesions of the dorsolateral striatum in rats increase delay discounting in an impulsive choice task

Dysregulated dopamine transmission in striatal circuitry is associated with impulsivity. The current study evaluated the influence of dopaminergic inputs to the dorsolateral striatum on impulsive choice, one aspect of impulsive behavior. We implemented an operant task that measures impulsive choice in rats via delay discounting wherein intracranial self-stimulation (ICSS) was used as the positive reinforcer. To do so, rats were anesthetized to allow implanting of a stimulating electrode within the lateral hypothalamus of one hemisphere and bilateral dorsal striatal injections of the dopaminergic toxin, 6-OHDA (lesioned) or its vehicle (sham). Following recovery, rats were trained in a delay discounting task wherein they selected between a small ICSS current presented immediately after lever pressing, and a large ICSS current presented following a 0 to 15s delay upon pressing the alternate lever. Task acquisition and reinforcer discrimination were similar for lesioned and sham rats. All rats exhibited an initial preference for the large reinforcer, and as the delay was increased, preference for the large reinforcer was decreased indicating that the subjective value of the large reinforcer was discounted as a function of delay time. However, this discounting effect was significantly enhanced in lesioned rats for the longer delays. These data reveal a contribution of dopaminergic inputs to the dorsolateral striatum on impulsive choice behavior, and provide new insights into neural substrates underlying discounting behaviors.

Supplementary Eye Field Encodes Confidence in Decisions Under Risk

Choices are made with varying degrees of confidence, a cognitive signal representing the subjective belief in the optimality of the choice. Confidence has been mostly studied in the context of perceptual judgments, in which choice accuracy can be measured using objective criteria. Here, we study confidence in subjective value-based decisions. We recorded in the supplementary eye field (SEF) of monkeys performing a gambling task, where they had to use subjective criteria for placing bets. We found neural signals in the SEF that explicitly represent choice confidence independent from reward expectation. This confidence signal appeared after the choice and diminished before the choice outcome. Most of this neuronal activity was negatively correlated with confidence, and was strongest in trials on which the monkey spontaneously withdrew his choice. Such confidence-related activity indicates that the SEF not only guides saccade selection, but also evaluates the likelihood that the choice was optimal. This internal evaluation influences decisions concerning the willingness to bear later costs that follow from the choice or to avoid them. More generally, our findings indicate that choice confidence is an integral component of all forms of decision-making, whether they are based on perceptual evidence or on value estimations.

A new approach to assess gambling-like behavior in laboratory rats: using intracranial self-stimulation as a positive reinforcer

Pathological gambling is one manifestation of impulse control disorders. The biological underpinnings of these disorders remain elusive and treatment is far from ideal. Animal models of impulse control disorders are a critical research tool for understanding this condition and for medication development. Modeling such complex behaviors is daunting, but by its deconstruction, scientists have recapitulated in animals critical aspects of gambling. One aspect of gambling is cost/benefit decision-making wherein one weighs the anticipated costs and expected benefits of a course of action. Risk/reward, delay-based and effort-based decision-making all represent cost/benefit choices. These features are studied in humans and have been translated to animal protocols to measure decision-making processes. Traditionally, the positive reinforcer used in animal studies is food. Here, we describe how intracranial self-stimulation can be used for cost/benefit decision-making tasks and overview our recent studies showing how pharmacological therapies alter these behaviors in laboratory rats. We propose that these models may have value in screening new compounds for the ability to promote and prevent aspects of gambling behavior.

Processing of hedonic and chemosensory features of taste in medial prefrontal and insular networks

Most of the research on cortical processing of taste has focused on either the primary gustatory cortex (GC) or the orbitofrontal cortex (OFC). However, these are not the only areas involved in taste processing. Gustatory information can also reach another frontal region, the medial prefrontal cortex (mPFC), via direct projections from GC. mPFC has been studied extensively in relation to its role in controlling goal-directed action and reward-guided behaviors, yet very little is known about its involvement in taste coding. The experiments presented here address this important point and test whether neurons in mPFC can significantly process the physiochemical and hedonic dimensions of taste. Spiking responses to intraorally delivered tastants were recorded from rats implanted with bundles of electrodes in mPFC and GC. Analysis of single-neuron and ensemble activity revealed similarities and differences between the two areas. Neurons in mPFC can encode the chemosensory identity of gustatory stimuli. However, responses in mPFC are sparser, more narrowly tuned, and have a later onset than in GC. Although taste quality is more robustly represented in GC, taste palatability is coded equally well in the two areas. Additional analysis of responses in neurons processing the hedonic value of taste revealed differences between the two areas in temporal dynamics and sensitivities to palatability. These results add mPFC to the network of areas involved in processing gustatory stimuli and demonstrate significant differences in taste-coding between GC and mPFC.

Self-choice enhances value in reward-seeking in primates

When an individual chooses one item from two or more alternatives, they compare the values of the expected outcomes. The outcome value can be determined by the associated reward amount, the probability of reward, and the workload required to earn the reward. Rational choice theory states that choices are made to maximize rewards over time, and that the same outcome values lead to an equal likelihood of choices. However, the theory does not distinguish between conditions with the same reward value, even when acquired under different circumstances, and does not always accurately describe real behavior. We have found that allowing a monkey to choose a reward schedule endows the schedule with extra value when compared to performance in an identical schedule that is chosen by another agent (a computer here). This behavior is not consistent with pure rational choice theory. Theoretical analysis using a modified temporal-difference learning model showed an enhanced schedule state value by self-choice. These results suggest that an increased reward value underlies the improved performances by self-choice during reward-seeking behavior.

Processing of hedonic and chemosensory features of taste in medial prefrontal and insular networks

Most of the research on cortical processing of taste has focused on either the primary gustatory cortex (GC) or the orbitofrontal cortex (OFC). However, these are not the only areas involved in taste processing. Gustatory information can also reach another frontal region, the medial prefrontal cortex (mPFC), via direct projections from GC. mPFC has been studied extensively in relation to its role in controlling goal-directed action and reward-guided behaviors, yet very little is known about its involvement in taste coding. The experiments presented here address this important point and test whether neurons in mPFC can significantly process the physiochemical and hedonic dimensions of taste. Spiking responses to intraorally delivered tastants were recorded from rats implanted with bundles of electrodes in mPFC and GC. Analysis of single-neuron and ensemble activity revealed similarities and differences between the two areas. Neurons in mPFC can encode the chemosensory identity of gustatory stimuli. However, responses in mPFC are sparser, more narrowly tuned, and have a later onset than in GC. Although taste quality is more robustly represented in GC, taste palatability is coded equally well in the two areas. Additional analysis of responses in neurons processing the hedonic value of taste revealed differences between the two areas in temporal dynamics and sensitivities to palatability. These results add mPFC to the network of areas involved in processing gustatory stimuli and demonstrate significant differences in taste-coding between GC and mPFC.

Interaction between orbital prefrontal and rhinal cortex is required for normal estimates of expected value

Predicting and valuing potential rewards requires integrating sensory, associative, and contextual information with subjective reward preferences. Previous work has identified regions in the prefrontal cortex and medial temporal lobe believed to be important for each of these functions. For example, activity in the orbital prefrontal cortex (PFo) encodes the specific sensory properties of and preferences for rewards, while activity in the rhinal cortex (Rh) encodes stimulus-stimulus and stimulus-reward associations. Lesions of either structure impair the ability to use visual cues or the history of previous reinforcement to value expected rewards. These areas are linked via reciprocal connections, suggesting it might be their interaction that is critical for estimating expected value. To test this hypothesis, we interrupted direct, intra-hemispheric PFo-Rh interaction in monkeys by performing crossed unilateral ablations of these regions (functional disconnection). We asked whether this circuit is crucial primarily for cue-reward association or for estimating expected value per se, by testing these monkeys, as well as intact controls, on tasks in which expected value was either visually cued or had to be inferred from block-wise changes in reward size in uncued trials. Functional disconnection significantly affected performance in both tasks. Specifically, monkeys with functional disconnection showed less of a difference in error rates and reaction times across reward sizes, in some cases behaving as if they expected rewards to be of equal magnitude. These results support a model whereby information about rewards signaled in PFo is combined with associative and contextual information signaled within Rh to estimate expected value.

Is working more costly than waiting in monkeys?

We studied how value for instrumental action is discounted by predicted effort and delay. The monkeys were trained to perform instrumental trials that required a bar release when a visual target changed from red-to-green. There were two trial conditions. In delay trials, after the monkeys performed one instrumental trial correctly a reward was delivered 0-7 seconds later. In work trials, the monkeys had to perform 0, 1, or 2 additional instrumental trials to obtain a reward. The lengths of trials in delay matched the time it took to complete work trials. The length of delay or number of trials was indicated by a visual cue presented throughout the trial. Our hypothesis was that the monkeys would all show temporal discounting of reward in the delay trials, and that in the work trials the monkeys' performance might reflect an additional cost due to working. The error rate increased linearly as remaining cost increased for all 8 monkeys. For 4 monkeys the error rate was significantly larger in work trials than in delay trials (effort sensitive monkeys). For the other 4 monkeys there was no significant difference in error rate (effort insensitive monkeys). Since the error rate has an inverse relation with value for action, these results suggest that value is discounted hyperbolically by effort as well as by delay. Error rates generally increased as the testing sessions progressed and the total reward accumulated (i.e., effect of reward devaluation). The relative impact of delay and effort on error rates was reasonably stable within subjects. Thus, within the monkey population there seems to be a significant dichotomy in the sensitivity governing whether working is more costly than waiting, possibly arising from a constitutional or genetic trait.

Intersection of reward and memory in monkey rhinal cortex

In humans and other animals, the vigor with which a reward is pursued depends on its desirability, that is, on the reward's predicted value. Predicted value is generally context-dependent, varying according to the value of rewards obtained in the recent and distant past. Signals related to reward prediction and valuation are believed to be encoded in a circuit centered around midbrain dopamine neurons and their targets in the prefrontal cortex and basal ganglia. Notably absent from this hypothesized reward pathway are dopaminergic targets in the medial temporal lobe. Here we show that a key part of the medial temporal lobe memory system previously reported to be important for sensory mnemonic and perceptual processing, the rhinal cortex (Rh), is required for using memories of previous reward values to predict the value of forthcoming rewards. We tested monkeys with bilateral Rh lesions on a task in which reward size varied across blocks of uncued trials. In this experiment, the only cues for predicting current reward value are the sizes of rewards delivered in previous blocks. Unexpectedly, monkeys with Rh ablations, but not intact controls, were insensitive to differences in predicted reward, responding as if they expected all rewards to be of equal magnitude. Thus, it appears that Rh is critical for using memory of previous rewards to predict the value of forthcoming rewards. These results are in agreement with accumulating evidence that Rh is critical for establishing the relationships between temporally interleaved events, which is a key element of episodic memory.

Hydration level is an internal variable for computing motivation to obtain water rewards in monkeys

In the process of motivation to engage in a behavior, valuation of the expected outcome is comprised of not only external variables (i.e., incentives) but also internal variables (i.e., drive). However, the exact neural mechanism that integrates these variables for the computation of motivational value remains unclear. Besides, the signal of physiological needs, which serves as the primary internal variable for this computation, remains to be identified. Concerning fluid rewards, the osmolality level, one of the physiological indices for the level of thirst, may be an internal variable for valuation, since an increase in the osmolality level induces drinking behavior. Here, to examine the relationship between osmolality and the motivational value of a water reward, we repeatedly measured the blood osmolality level, while 2 monkeys continuously performed an instrumental task until they spontaneously stopped. We found that, as the total amount of water earned increased, the osmolality level progressively decreased (i.e., the hydration level increased) in an individual-dependent manner. There was a significant negative correlation between the error rate of the task (the proportion of trials with low motivation) and the osmolality level. We also found that the increase in the error rate with reward accumulation can be well explained by a formula describing the changes in the osmolality level. These results provide a biologically supported computational formula for the motivational value of a water reward that depends on the hydration level, enabling us to identify the neural mechanism that integrates internal and external variables.

Ventral striatum encodes past and predicted value independent of motor contingencies

The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that VS can encode value independent of motor contingencies and that the role of VS in goal-directed behavior is not just to increase vigor of specific actions when more is at stake.

Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex

Background

The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing.

Methods and Findings

We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1–4 sequential color discrimination trials to obtain a reward of 1–3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount.

Conclusions

Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.