Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward

Neuroeconomics of decision-making during COVID-19 pandemic

The coronavirus disease 2019 (COVID-19) pandemic reveals the decision-making challenges faced by communities, governments, and international organizations, globally. Policymakers are much concerned about protecting the population from the deadly virus while lacking reliable information on the virus and its spread mechanisms and the effectiveness of possible measures and their (direct and indirect) health and socioeconomic costs. This review aims to highlight the various balanced policy decision that would combine the best obtainable scientific evidence characteristically provided by expert opinions and modeling studies. This article's main goal is to summarize the main significant progress in the understanding of neuroeconomics of decision-making and discuss the anatomy of decision making in the light of COVID-19 pandemic.

Long-range cortical synchronization supports abrupt visual learning

Visual plasticity declines sharply after the critical period, yet we easily learn to recognize new faces and places, even as adults. Such learning is often characterized by a "moment of insight," an abrupt and dramatic improvement in recognition. The mechanisms that support abrupt learning are unknown, but one hypothesis is that they involve changes in synchronization between brain regions. To test this hypothesis, we used a behavioral task in which non-human primates rapidly learned to recognize novel images and to associate them with specific responses. Simultaneous recordings from inferotemporal and prefrontal cortices revealed a transient synchronization of neural activity between these areas that peaked around the moment of insight. Synchronization was strongest between inferotemporal sites that encoded images and reward-sensitive prefrontal sites. Moreover, its magnitude intensified gradually over image exposures, suggesting that abrupt learning is the culmination of a search for informative signals within a circuit linking sensory information to task demands.

Chemogenetic inactivation reveals the inhibitory control function of the prefronto-striatal pathway in the macaque brain

The lateral prefrontal cortex (LPFC) has a strong monosynaptic connection with the caudate nucleus (CdN) of the striatum. Previous human MRI studies have suggested that this LPFC-CdN pathway plays an important role in inhibitory control and working memory. We aimed to validate the function of this pathway at a causal level by pathway-selective manipulation of neural activity in non-human primates. To this end, we trained macaque monkeys on a delayed oculomotor response task with reward asymmetry and expressed an inhibitory type of chemogenetic receptors selectively to LPFC neurons that project to the CdN. Ligand administration reduced the inhibitory control of impulsive behavior, as well as the task-related neuronal responses observed in the local field potentials from the LPFC and CdN. These results show that we successfully suppressed pathway-selective neural activity in the macaque brain, and the resulting behavioral changes suggest that the LPFC-CdN pathway is involved in inhibitory control.

Frontal eye field and caudate neurons make different contributions to reward-biased perceptual decisions

Many decisions require trade-offs between sensory evidence and internal preferences. Potential neural substrates include the frontal eye field (FEF) and caudate nucleus, but their distinct roles are not understood. Previously we showed that monkeys’ decisions on a direction-discrimination task with asymmetric rewards reflected a biased accumulate-to-bound decision process (Fan et al., 2018) that was affected by caudate microstimulation (Doi et al., 2020). Here we compared single-neuron activity in FEF and caudate to each other and to accumulate-to-bound model predictions derived from behavior. Task-dependent neural modulations were similar in both regions. However, choice-selective neurons in FEF, but not caudate, encoded behaviorally derived biases in the accumulation process. Baseline activity in both regions was sensitive to reward context, but this sensitivity was not reliably associated with behavioral biases. These results imply distinct contributions of FEF and caudate neurons to reward-biased decision-making and put experimental constraints on the neural implementation of accumulation-to-bound-like computations.

The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Our decisions often balance what we observe and what we desire. A prime candidate for implementing this complex balancing act is the basal ganglia pathway, but its roles have not yet been examined experimentally in detail. Here, we show that a major input station of the basal ganglia, the caudate nucleus, plays a causal role in integrating uncertain visual evidence and reward context to guide adaptive decision-making. In monkeys making saccadic decisions based on motion cues and asymmetric reward-choice associations, single caudate neurons encoded both sources of information. Electrical microstimulation at caudate sites during motion viewing affected the monkeys’ decisions. These microstimulation effects included coordinated changes in multiple computational components of the decision process that mimicked the monkeys’ similarly coordinated voluntary strategies for balancing visual and reward information. These results imply that the caudate nucleus plays causal roles in coordinating decision processes that balance external evidence and internal preferences.

Reward Processing Deficits During a Spatial Attention Task in Patients With ADHD: An fMRI Study

OBJECTIVE

In this study, we aimed to explore how cues signaling rewards and feedbacks about rewards are processed in ADHD.

METHOD

Inside the scanner, 16 healthy children and 19 children with ADHD completed a spatial attention paradigm where cues informed about the availability of reward and feedbacks were provided about the earned reward.

RESULTS

In ventral anterior thalamus (VA), the controls exhibited greater activation in response to reward-predicting cues, as compared with no-reward cues, whereby in the ADHD group, the reverse pattern was observed (nonreward > reward). For feedbacks; absence of rewards produced greater activation than presence in the left caudate and frontal eye field for the control group, whereas for the ADHD group, the reverse pattern was again observed (reward > nonreward).

DISCUSSION

The present findings indicate that ADHD is associated with difficulty integrating reward contingency information with the orienting and regulatory phases of attention.

Parsing Heterogeneous Striatal Activity

The striatum is an input channel of the basal ganglia and is well known to be involved in reward-based decision making and learning. At the macroscopic level, the striatum has been postulated to contain parallel functional modules, each of which includes neurons that perform similar computations to support selection of appropriate actions for different task contexts. At the single-neuron level, however, recent studies in monkeys and rodents have revealed heterogeneity in neuronal activity even within restricted modules of the striatum. Looking for generality in the complex striatal activity patterns, here we briefly survey several types of striatal activity, focusing on their usefulness for mediating behaviors. In particular, we focus on two types of behavioral tasks: reward-based tasks that use salient sensory cues and manipulate outcomes associated with the cues; and perceptual decision tasks that manipulate the quality of noisy sensory cues and associate all correct decisions with the same outcome. Guided by previous insights on the modular organization and general selection-related functions of the basal ganglia, we relate striatal activity patterns on these tasks to two types of computations: implementation of selection and evaluation. We suggest that a parsing with the selection/evaluation categories encourages a focus on the functional commonalities revealed by studies with different animal models and behavioral tasks, instead of a focus on aspects of striatal activity that may be specific to a particular task setting. We then highlight several questions in the selection-evaluation framework for future explorations.

Cortico-Cortical Interactions during Acquisition and Use of a Neuroprosthetic Skill

A motor cortex-based brain-computer interface (BCI) creates a novel real world output directly from cortical activity. Use of a BCI has been demonstrated to be a learned skill that involves recruitment of neural populations that are directly linked to BCI control as well as those that are not. The nature of interactions between these populations, however, remains largely unknown. Here, we employed a data-driven approach to assess the interaction between both local and remote cortical areas during the use of an electrocorticographic BCI, a method which allows direct sampling of cortical surface potentials. Comparing the area controlling the BCI with remote areas, we evaluated relationships between the amplitude envelopes of band limited powers as well as non-linear phase-phase interactions. We found amplitude-amplitude interactions in the high gamma (HG, 70–150 Hz) range that were primarily located in the posterior portion of the frontal lobe, near the controlling site, and non-linear phase-phase interactions involving multiple frequencies (cross-frequency coupling between 8–11 Hz and 70–90 Hz) taking place over larger cortical distances. Further, strength of the amplitude-amplitude interactions decreased with time, whereas the phase-phase interactions did not. These findings suggest multiple modes of cortical communication taking place during BCI use that are specialized for function and depend on interaction distance.

The neurons in the human brain are densely interlaced, sharing upwards of 100 trillion physical connections. It is widely theorized that this tremendous connectivity is one of the facets of our nervous system that enables human intelligence. In this study, over the course of a week, human subjects learned to use electrical activity recorded directly from the surface of their brain to control a computer cursor. This provided us an opportunity to investigate patterns of interactivity that occur in the brain during the development of a new skill. We demonstrated two fundamentally different forms of interactions, one spanning only neighboring populations of neurons and the other covering much longer distances across the brain. The short-distance interaction type was notably stronger during early phases of learning, lessening with time, whereas the other was not. These findings point to evidence of multiple different forms of task-relevant communication taking place between regions in the human brain, and serve as a building block in our efforts to better understand human intelligence.

Distinct dynamics of ramping activity in the frontal cortex and caudate nucleus in monkeys

The prefronto-striatal network is involved in many cognitive functions, including perceptual decision making and reward-modulated behaviors. For well-trained subjects, neural responses frequently show similar patterns in the prefrontal cortex and striatum, making it difficult to tease apart distinct regional contributions. Here I show that, despite similar mean firing rate patterns, prefrontal and striatal responses differ in other temporal dynamics for both perceptual and reward-based tasks. Compared with simulation results, the temporal dynamics of prefrontal activity are consistent with an accumulation of sensory evidence used to solve a perceptual task but not with an accumulation of reward context-related information used for the development of a reward bias. In contrast, the dynamics of striatal activity is consistent with an accumulation of reward context-related information and with an accumulation of sensory evidence during early stimulus viewing. These results suggest that prefrontal and striatal neurons may have specialized functions for different tasks even with similar average activity.

Double Virus Vector Infection to the Prefrontal Network of the Macaque Brain

To precisely understand how higher cognitive functions are implemented in the prefrontal network of the brain, optogenetic and pharmacogenetic methods to manipulate the signal transmission of a specific neural pathway are required. The application of these methods, however, has been mostly restricted to animals other than the primate, which is the best animal model to investigate higher cognitive functions. In this study, we used a double viral vector infection method in the prefrontal network of the macaque brain. This enabled us to express specific constructs into specific neurons that constitute a target pathway without use of germline genetic manipulation. The double-infection technique utilizes two different virus vectors in two monosynaptically connected areas. One is a vector which can locally infect cell bodies of projection neurons (local vector) and the other can retrogradely infect from axon terminals of the same projection neurons (retrograde vector). The retrograde vector incorporates the sequence which encodes Cre recombinase and the local vector incorporates the "Cre-On" FLEX double-floxed sequence in which a reporter protein (mCherry) was encoded. mCherry thus came to be expressed only in doubly infected projection neurons with these vectors. We applied this method to two macaque monkeys and targeted two different pathways in the prefrontal network: The pathway from the lateral prefrontal cortex to the caudate nucleus and the pathway from the lateral prefrontal cortex to the frontal eye field. As a result, mCherry-positive cells were observed in the lateral prefrontal cortex in all of the four injected hemispheres, indicating that the double virus vector transfection is workable in the prefrontal network of the macaque brain.

Dissociable functions of reward inference in the lateral prefrontal cortex and the striatum

In a complex and uncertain world, how do we select appropriate behavior? One possibility is that we choose actions that are highly reinforced by their probabilistic consequences (model-free processing). However, we may instead plan actions prior to their actual execution by predicting their consequences (model-based processing). It has been suggested that the brain contains multiple yet distinct systems involved in reward prediction. Several studies have tried to allocate model-free and model-based systems to the striatum and the lateral prefrontal cortex (LPFC), respectively. Although there is much support for this hypothesis, recent research has revealed discrepancies. To understand the nature of the reward prediction systems in the LPFC and the striatum, a series of single-unit recording experiments were conducted. LPFC neurons were found to infer the reward associated with the stimuli even when the monkeys had not yet learned the stimulus-reward (SR) associations directly. Striatal neurons seemed to predict the reward for each stimulus only after directly experiencing the SR contingency. However, the one exception was "Exclusive Or" situations in which striatal neurons could predict the reward without direct experience. Previous single-unit studies in monkeys have reported that neurons in the LPFC encode category information, and represent reward information specific to a group of stimuli. Here, as an extension of these, we review recent evidence that a group of LPFC neurons can predict reward specific to a category of visual stimuli defined by relevant behavioral responses. We suggest that the functional difference in reward prediction between the LPFC and the striatum is that while LPFC neurons can utilize abstract code, striatal neurons can code individual associations between stimuli and reward but cannot utilize abstract code.

Decision making: effects of methylphenidate on temporal discounting in nonhuman primates

Decisions are often made based on which option will result in the largest reward. When given a choice between a smaller but immediate reward and a larger delayed reward, however, humans and animals often choose the smaller, an effect known as temporal discounting. Dopamine (DA) neurotransmission is central to reward processing and encodes delayed reward value. Impulsivity, the tendency to act without forethought, is associated with excessive discounting of rewards, which has been documented in patients with attention deficit hyperactivity disorder (ADHD). Both impulsivity and temporal discounting are linked to the dopaminergic system. Methylphenidate (MPH), which blocks the DA transporter and increases extracellular levels of DA in the basal ganglia and prefrontal cortex, is a primary treatment for ADHD and, at low doses, ameliorates impulsivity in both humans and animals. This study tested the hypothesis that low doses of MPH would decrease the discounting rate of rhesus monkeys performing an intertemporal choice task, suggesting a reduction in impulsivity. The results support this hypothesis and provide further evidence for the role of DA in temporal discounting and impulsive behavior.

Cingulate neglect in humans: disruption of contralesional reward learning in right brain damage

Motivational valence plays a key role in orienting spatial attention. Nonetheless, clinical documentation and understanding of motivationally based deficits of spatial orienting in the human is limited. Here in a series of one group-study and two single-case studies, we have examined right brain damaged patients (RBD) with and without left spatial neglect in a spatial reward-learning task, in which the motivational valence of the left contralesional and the right ipsilesional space was contrasted. In each trial two visual boxes were presented, one to the left and one to the right of central fixation. In one session monetary rewards were released more frequently in the box on the left side (75% of trials) whereas in another session they were released more frequently on the right side. In each trial patients were required to: 1) point to each one of the two boxes; 2) choose one of the boxes for obtaining monetary reward; 3) report explicitly the position of reward and whether this position matched or not the original choice. Despite defective spontaneous allocation of attention toward the contralesional space, RBD patients with left spatial neglect showed preserved contralesional reward learning, i.e., comparable to ipsilesional learning and to reward learning displayed by patients without neglect. A notable exception in the group of neglect patients was L.R., who showed no sign of contralesional reward learning in a series of 120 consecutive trials despite being able of reaching learning criterion in only 20 trials in the ipsilesional space. L.R. suffered a cortical-subcortical brain damage affecting the anterior components of the parietal-frontal attentional network and, compared with all other neglect and non-neglect patients, had additional lesion involvement of the medial anterior cingulate cortex (ACC) and of the adjacent sectors of the corpus callosum. In contrast to his lateralized motivational learning deficit, L.R. had no lateral bias in the early phases of attentional processing as he suffered no contralesional visual or auditory extinction on double simultaneous tachistoscopic and dichotic stimulation and detected, with no exception, single contralesional visual and auditory stimuli. In a separate study, we were able to compare L.R. with another RBD patient, G.P., who had a selective lesion in the right ACC, in the adjacent callosal connections and the medial-basal prefrontal cortex. G.P. had no contralesional neglect and displayed normal reward learning both in the left and right side of space. These findings show that contralesional reward learning is generally preserved in RBD patients with left spatial neglect and that this can be exploited in rehabilitation protocols. Contralesional reward learning is severely disrupted in neglect patients when an additional lesion of the ACC is present: however, as demonstrated by the comparison between L.R. and G.P. cases, selective unilateral lesion of the right ACC does not produce motivational neglect for the contralesional space.

Précis on The Cognitive-Emotional Brain

In The Cognitive-Emotional Brain (Pessoa 2013), I describe the many ways that emotion and cognition interact and are integrated in the brain. The book summarizes five areas of research that support this integrative view and makes four arguments to organize each area. (1) Based on rodent and human data, I propose that the amygdala's functions go beyond emotion as traditionally conceived. Furthermore, the processing of emotion-laden information is capacity limited, thus not independent of attention and awareness. (2) Cognitive-emotional interactions in the human prefrontal cortex (PFC) assume diverse forms and are not limited to mutual suppression. Particularly, the lateral PFC is a focal point for cognitive-emotional interactions. (3) Interactions between motivation and cognition can be seen across a range of perceptual and cognitive tasks. Motivation shapes behavior in specific ways – for example, by reducing response conflict or via selective effects on working memory. Traditional accounts, by contrast, typically describe motivation as a global activation independent of particular control demands. (4) Perception and cognition are directly influenced by information with affective or motivational content in powerful ways. A dual competition model outlines a framework for such interactions at the perceptual and executive levels. A specific neural architecture is proposed that embeds emotional and motivational signals into perception and cognition through multiple channels. (5) A network perspective should supplant the strategy of understanding the brain in terms of individual regions. More broadly, in a network view of brain architecture, “emotion” and “cognition” may be used as labels of certain behaviors, but will not map cleanly into compartmentalized pieces of the brain.

The role of the striatum in social behavior

Where and how does the brain code reward during social behavior? Almost all elements of the brain's reward circuit are modulated during social behavior. The striatum in particular is activated by rewards in social situations. However, its role in social behavior is still poorly understood. Here, we attempt to review its participation in social behaviors of different species ranging from voles to humans. Human fMRI experiments show that the striatum is reliably active in relation to others' rewards, to reward inequity and also while learning about social agents. Social contact and rearing conditions have long-lasting effects on behavior, striatal anatomy and physiology in rodents and primates. The striatum also plays a critical role in pair-bond formation and maintenance in monogamous voles. We review recent findings from single neuron recordings showing that the striatum contains cells that link own reward to self or others' actions. These signals might be used to solve the agency-credit assignment problem: the question of whose action was responsible for the reward. Activity in the striatum has been hypothesized to integrate actions with rewards. The picture that emerges from this review is that the striatum is a general-purpose subcortical region capable of integrating social information into coding of social action and reward.

Coding of level of ambiguity within neural systems mediating choice

Data from previous neuroimaging studies exploring neural activity associated with uncertainty suggest varying levels of activation associated with changing degrees of uncertainty in neural regions that mediate choice behavior. The present study used a novel task that parametrically controlled the amount of information hidden from the subject; levels of uncertainty ranged from full ambiguity (no information about probability of winning) through multiple levels of partial ambiguity, to a condition of risk only (zero ambiguity with full knowledge of the probability of winning). A parametric analysis compared a linear model in which weighting increased as a function of level of ambiguity, and an inverted-U quadratic models in which partial ambiguity conditions were weighted most heavily. Overall we found that risk and all levels of ambiguity recruited a common "fronto-parietal-striatal" network including regions within the dorsolateral prefrontal cortex, intraparietal sulcus, and dorsal striatum. Activation was greatest across these regions and additional anterior and superior prefrontal regions for the quadratic function which most heavily weighs trials with partial ambiguity. These results suggest that the neural regions involved in decision processes do not merely track the absolute degree ambiguity or type of uncertainty (risk vs. ambiguity). Instead, recruitment of prefrontal regions may result from greater degree of difficulty in conditions of partial ambiguity: when information regarding reward probabilities important for decision making is hidden or not easily obtained the subject must engage in a search for tractable information. Additionally, this study identified regions of activity related to the valuation of potential gains associated with stimuli or options (including the orbitofrontal and medial prefrontal cortices and dorsal striatum) and related to winning (including orbitofrontal cortex and ventral striatum).

Predeliberation activity in prefrontal cortex and striatum and the prediction of subsequent value judgment

Rational, value-based decision-making mandates selecting the option with highest subjective expected value after appropriate deliberation. We examined activity in the dorsolateral prefrontal cortex (DLPFC) and striatum of monkeys deciding between smaller, immediate rewards and larger, delayed ones. We previously found neurons that modulated their activity in this task according to the animal's choice, while it deliberated (choice neurons). Here we found neurons whose spiking activities were predictive of the spatial location of the selected target (spatial-bias neurons) or the size of the chosen reward (reward-bias neurons) before the onset of the cue presenting the decision-alternatives, and thus before rational deliberation could begin. Their predictive power increased as the values the animals associated with the two decision alternatives became more similar. The ventral striatum (VS) preferentially contained spatial-bias neurons; the caudate nucleus (CD) preferentially contained choice neurons. In contrast, the DLPFC contained significant numbers of all three neuron types, but choice neurons were not preferentially also bias neurons of either kind there, nor were spatial-bias neurons preferentially also choice neurons, and vice versa. We suggest a simple winner-take-all (WTA) circuit model to account for the dissociation of choice and bias neurons. The model reproduced our results and made additional predictions that were borne out empirically. Our data are compatible with the hypothesis that the DLPFC and striatum harbor dissociated neural populations that represent choices and predeliberation biases that are combined after cue onset; the bias neurons have a weaker effect on the ultimate decision than the choice neurons, so their influence is progressively apparent for trials where the values associated with the decision alternatives are increasingly similar.

Preparatory neural networks are impaired in adults with attention-deficit/hyperactivity disorder during the antisaccade task

Adults with attention-deficit/hyperactivity disorder (ADHD) often display executive function impairments, particularly in inhibitory control. The antisaccade task, which measures inhibitory control, requires one to suppress an automatic prosaccade toward a salient visual stimulus and voluntarily make an antisaccade in the opposite direction. ADHD patients not only have longer saccadic reaction times, but also make more direction errors (i.e., a prosaccade was executed toward the stimulus) during antisaccade trials. These deficits may stem from pathology in several brain areas that are important for executive control. Using functional MRI with a rapid event-related design, adults with combined subtype of ADHD (coexistence of attention and hyperactivity problems), who abstained from taking stimulant medication 20 h prior to experiment onset, and age-match controls performed pro- and antisaccade trials that were interleaved with pro- and anti-catch trials (i.e., instruction was presented but no target appeared, requiring no response). This method allowed us to examine brain activation patterns when participants either prepared (during instruction) or executed (after target appearance) correct pro or antisaccades. Behaviorally, ADHD adults displayed several antisaccade deficits, including longer and more variable reaction times and more direction errors, but saccade metrics (i.e., duration, velocity, and amplitude) were normal. When preparing to execute an antisaccade, ADHD adults showed less activation in frontal, supplementary, and parietal eye fields, compared to controls. However, activation in these areas was normal in the ADHD group during the execution of a correct antisaccade. Interestingly, unlike controls, adults with ADHD produced greater activation than controls in dorsolateral prefrontal cortex during antisaccade execution, perhaps as part of compensatory mechanisms to optimize antisaccade production. Overall, these data suggest that the saccade deficits observed in adults with ADHD do not result from an inability to execute a correct antisaccade but rather the failure to properly prepare (i.e., form the appropriate task set) for the antisaccade trial. The data support the view that the executive impairments, including inhibitory control, in ADHD adults are related to poor response preparation.

Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions

In its simplest formulation, reinforcement learning is based on the idea that if an action taken in a particular context is followed by a favorable outcome, then, in the same context, the tendency to produce that action should be strengthened, or reinforced. While reinforcement learning forms the basis of many current theories of basal ganglia (BG) function, these models do not incorporate distinct computational roles for signals that convey context, and those that convey what action an animal takes. Recent experiments in the songbird suggest that vocal-related BG circuitry receives two functionally distinct excitatory inputs. One input is from a cortical region that carries context information about the current “time” in the motor sequence. The other is an efference copy of motor commands from a separate cortical brain region that generates vocal variability during learning. Based on these findings, I propose here a general model of vertebrate BG function that combines context information with a distinct motor efference copy signal. The signals are integrated by a learning rule in which efference copy inputs gate the potentiation of context inputs (but not efference copy inputs) onto medium spiny neurons in response to a rewarded action. The hypothesis is described in terms of a circuit that implements the learning of visually guided saccades. The model makes testable predictions about the anatomical and functional properties of hypothesized context and efference copy inputs to the striatum from both thalamic and cortical sources.

Tracking the temporal evolution of a perceptual judgment using a compelled-response task

Choice behavior and its neural correlates have been intensely studied with tasks in which a subject makes a perceptual judgment and indicates the result with a motor action. Yet a question crucial for relating behavior to neural activity remains unresolved: what fraction of a subject's reaction time (RT) is devoted to the perceptual evaluation step, as opposed to executing the motor report? Making such timing measurements accurately is complicated because RTs reflect both sensory and motor processing, and because speed and accuracy may be traded. To overcome these problems, we designed the compelled-saccade task, a two-alternative forced-choice task in which the instruction to initiate a saccade precedes the appearance of the relevant sensory information. With this paradigm, it is possible to track perceptual performance as a function of the amount of time during which sensory information is available to influence a subject's choice. The result-the tachometric curve-directly reveals a subject's perceptual processing capacity independently of motor demands. Psychophysical data, together with modeling and computer-simulation results, reveal that task performance depends on three separable components: the timing of the motor responses, the speed of the perceptual evaluation, and additional cognitive factors. Each can vary quickly, from one trial to the next, or can show stable, longer-term changes. This novel dissociation between sensory and motor processes yields a precise metric of how perceptual capacity varies under various experimental conditions and serves to interpret choice-related neuronal activity as perceptual, motor, or both.

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.

Reporting the Implementation of the Three Rs in European Primate and Mouse Research Papers: Are We Making Progress?

It is now more than 20 years since both Council of Europe Convention ETS123 and EU Directive 86/609?EEC were introduced, to promote the implementation of the Three Rs in animal experimentation and to provide guidance on animal housing and care. It might therefore be expected that reports of the implementation of the Three Rs in animal research papers would have increased during this period. In order to test this hypothesis, a literature survey of animal-based research was conducted. A randomly-selected sample from 16 high-profile medical journals, of original research papers arising from European institutions that featured experiments which involved either mice or primates, were identified for the years 1986 and 2006 (Total sample = 250 papers). Each paper was scored out of 10 for the incidence of reporting on the implementation of Three Rs-related factors corresponding to Replacement (justification of non-use of non-animal methods), Reduction (statistical analysis of the number of animals needed) and Refinement (housing aspects, i.e. increased cage size, social housing, enrichment of cage environment and food; and procedural aspects, i.e. the use of anaesthesia, analgesia, humane endpoints, and training for procedures with positive reinforcement). There was no significant increase in overall reporting score over time, for either mouse or primate research. By 2006, mouse research papers scored an average of 0 out of a possible 10, and primate research papers scored an average of 1.5. This review provides systematic evidence that animal research is still not properly reported, and supports the call within the scientific community for action to be taken by journals to update their policies.

Motor preparatory activity in posterior parietal cortex is modulated by subjective absolute value

Cortical motor planning is shaped by “subjective absolute value”: planning activity is strongly enhanced for large expected gains in subjects who believe they perform well; conversely, activity is higher for large expected losses in subjects who think they perform poorly.

For optimal response selection, the consequences associated with behavioral success or failure must be appraised. To determine how monetary consequences influence the neural representations of motor preparation, human brain activity was scanned with fMRI while subjects performed a complex spatial visuomotor task. At the beginning of each trial, reward context cues indicated the potential gain and loss imposed for correct or incorrect trial completion. FMRI-activity in canonical reward structures reflected the expected value related to the context. In contrast, motor preparatory activity in posterior parietal and premotor cortex peaked in high “absolute value” (high gain or loss) conditions: being highest for large gains in subjects who believed they performed well while being highest for large losses in those who believed they performed poorly. These results suggest that the neural activity preceding goal-directed actions incorporates the absolute value of that action, predicated upon subjective, rather than objective, estimates of one's performance.

The expected outcome of voluntary actions profoundly shapes human decision making. For instance, expected monetary reward and punishment are powerful modulators of human behavior. Yet how these factors influence brain activity responsible for the preparation of such behavior is not fully understood. This is especially true for demanding tasks, in which the outcome—e.g. reward versus punishment—critically depends on the accuracy of actions. In our human fMRI study we investigated brain activity in specific cortical areas that are related to the planning of voluntary behavior. We show that planning activity in these areas is strongly influenced by the expected monetary gain or loss that subjects associated with their performance in a demanding motor task. Planning activity was highest for large expected gains in subjects who believed that they performed well; conversely, activity was highest for large expected losses in subjects who thought that they performed poorly. This pattern of planning activity was best described by a model which we refer to as the “subjective absolute value model.” We suggest that absolute value signals in motor planning areas can be used to mobilize motor resources in behaviorally relevant situations—both to maximize gains and to avoid losses.

Coding of task reward value in the dorsal raphe nucleus

The dorsal raphe nucleus and its serotonin-releasing neurons are thought to regulate motivation and reward-seeking. These neurons are known to be active during motivated behavior, but the underlying principles that govern their activity are unknown. Here we show that a group of dorsal raphe neurons encode behavioral tasks in a systematic manner, tracking progress toward upcoming rewards. We analyzed dorsal raphe neuron activity recorded while animals performed two reward-oriented saccade tasks. There was a strong correlation between the tonic activity level of a neuron during behavioral tasks and its encoding of reward-related cues and outcomes. Neurons that were tonically excited during the task predominantly carried positive reward signals. Neurons that were tonically inhibited during the task predominantly carried negative reward signals. Neurons that did not change their tonic activity levels during the task had weak reward signals with no tendency for a positive or negative direction. This form of correlated task and reward coding accounted for the majority of systematic variation in dorsal raphe response patterns in our tasks. A smaller component of neural activity reflected detection of reward delivery. Our data suggest that the dorsal raphe nucleus encodes participation in a behavioral task in terms of its future motivational outcomes.

The role of human orbitofrontal cortex in value comparison for incommensurable objects

The human orbitofrontal cortex is strongly implicated in appetitive valuation. Whether its role extends to support comparative valuation necessary to explain probabilistic choice patterns for incommensurable goods is unknown. Using a binary choice paradigm, we derived the subjective values of different bundles of goods, under conditions of both gain and loss. We demonstrate that orbitofrontal activation reflects the difference in subjective value between available options, an effect evident across valuation for both gains and losses. In contrast, activation in dorsal striatum and supplementary motor areas reflects subjects' choice probabilities. These findings indicate that orbitofrontal cortex plays a pivotal role in valuation for incommensurable goods, a critical component process in human decision making.

Influence of reward delays on responses of dopamine neurons

Psychological and microeconomic studies have shown that outcome values are discounted by imposed delays. The effect, called temporal discounting, is demonstrated typically by choice preferences for sooner smaller rewards over later larger rewards. However, it is unclear whether temporal discounting occurs during the decision process when differently delayed reward outcomes are compared or during predictions of reward delays by pavlovian conditioned stimuli without choice. To address this issue, we investigated the temporal discounting behavior in a choice situation and studied the effects of reward delay on the value signals of dopamine neurons. The choice behavior confirmed hyperbolic discounting of reward value by delays on the order of seconds. Reward delay reduced the responses of dopamine neurons to pavlovian conditioned stimuli according to a hyperbolic decay function similar to that observed in choice behavior. Moreover, the stimulus responses increased with larger reward magnitudes, suggesting that both delay and magnitude constituted viable components of dopamine value signals. In contrast, dopamine responses to the reward itself increased with longer delays, possibly reflecting temporal uncertainty and partial learning. These dopamine reward value signals might serve as useful inputs for brain mechanisms involved in economic choices between delayed rewards.

New insights on the subcortical representation of reward

Reward information is represented by many subcortical areas and neuron types, which constitute a complex network. Its output is usually mediated by the basal ganglia where behaviors leading to rewards are disinhibited and behaviors leading to no reward are suppressed. Midbrain dopamine neurons modulate these basal ganglia neurons differentially using signals related to reward-prediction error. Recent studies suggest that other types of subcortical neurons assist, instruct, or work in parallel with dopamine neurons. Such reward-related neurons are found in the areas which have been associated with stress, pain, mood, emotion, memory, and arousal. These results suggest that reward needs to be understood in a larger framework of animal behavior.

Value representations in the primate striatum during matching behavior

Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.

Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype

The dorsal striatum plays a critical role in procedural learning and memory. Current models of basal ganglia assume that striatal neurons and circuitry are critical for the execution of overlearned, habitual sequences of action. However, less is known about how the striatum encodes task information that guides the performance of actions in procedural tasks. To explore the striatal encoding of task information, we compared the behavioral correlates of striatal neurons tested in two tasks: a multiple T-maze task in which reward delivery was entirely predictable based on spatial cues (the Multiple-T task), and a task in which rats ran on a rectangular track, but food delivery depended on the distance traveled on the track and was not dependent solely on spatial location (the Take-5 task). Striatal cells recorded on these tasks were divisible into three cell types: phasic-firing neurons (PFNs), tonically firing neurons (TFNs), and high-firing neurons (HFNs) and similar proportions of each cell type were found in each task. However, the behavioral correlates of each cell type were differentially sensitive to the type of task rats were performing. PFNs were responsive to specific task-parameters on each task. TFNs showed reliable burst-and-pause responses following food delivery and other events that were consistent with tonically active neurons (TANs) on the Take-5 (non-spatial) task but not on the Multiple-T (spatial) task. HFNs showed spatial oscillations on the Multiple-T (spatial) task but not the Take-5 (non-spatial) task. Reconstruction of the rats' position on the maze was highly accurate when using striatal ensembles recorded on the Multiple-T (spatial) task, but not when using ensembles recorded on the Take-5 (non-spatial) task. In contrast, reconstruction of time following food delivery was successful in both tasks. The results indicated a strong task dependency of the quality of the spatial, but not the reward-related, striatal representations on these tasks. These results suggest that striatal spatial representations depend on the degree to which spatial task-parameters can be unambiguously associated with goals.

On the relationship between emotion and cognition

The current view of brain organization supports the notion that there is a considerable degree of functional specialization and that many regions can be conceptualized as either 'affective' or 'cognitive'. Popular examples are the amygdala in the domain of emotion and the lateral prefrontal cortex in the case of cognition. This prevalent view is problematic for a number of reasons. Here, I will argue that complex cognitive-emotional behaviours have their basis in dynamic coalitions of networks of brain areas, none of which should be conceptualized as specifically affective or cognitive. Central to cognitive-emotional interactions are brain areas with a high degree of connectivity, called hubs, which are critical for regulating the flow and integration of information between regions.

Action and outcome encoding in the primate caudate nucleus

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions.

Integration of cognitive and motivational context information in the primate prefrontal cortex

The prefrontal cortex (PFC) appears to be important for processing both cognitive and motivational context information. Primate lateral PFC (LPFC) neurons are involved in cognitive context-dependent stimulus coding by responding differently to an identical stimulus according to the task situation. Such context-dependent LPFC activity appears to be supported by context-representing activity, observed also in LPFC neurons, in which the baseline activity differs as a function of the task. In LPFC, there are also neurons that code stimulus on the basis of motivational context. This motivational context is represented in differential baseline activity as a function of the reward context. In the orbitofrontal cortex (OFC), there are neurons that code stimuli depending on the motivational context as well as neurons that represent motivational context information. Furthermore, we found LPFC neurons that coded the stimulus depending on both the cognitive and motivational context, as well as LPFC neurons that represented both the cognitive and motivational context. For adaptive behavior, it is important to code the meaning of the environmental situation based on the context. While OFC is predominantly concerned with processing motivational context information, LPFC seems to play important roles in integrating the cognitive and motivational context for adaptive goal-directed behavior.

Positive and Negative Modulation of Motor Response in Primate Superior Colliculus by Reward Expectation

Expectation of reward is crucial for goal-directed behavior of animals. However, little is known about how reward information is used in the brain at the time of action. We investigated this question by recording from single neurons in the macaque superior colliculus (SC) while the animal was performing a memory-guided saccade task with an asymmetrical reward schedule. The SC is an ideal structure to ask this question because it receives inputs from many brain areas including the prefrontal cortex and the basal ganglia where reward information is thought to be encoded and sends motor commands to the brain stem saccade generators. We found two groups of SC neurons that encoded reward information in the presaccadic period: positive reward-coding neurons that showed higher activity when reward was expected and negative reward-coding neurons that showed higher activity when reward was not expected. The positive reward-coding usually started even before a cue for target position was presented, whereas the negative reward-coding was largely restricted to the presaccadic period. The two kinds of reward-coding may be useful for the animal to select an appropriate behavior in a complex environment.

Encoding of action history in the rat ventral striatum

In a dynamic environment, animals need to update information about the rewards expected from their alternative actions continually to make optimal choices for its survival. Because the reward resulting from a given action can be substantially delayed, the process of linking a reward to its causative action would be facilitated by memory signals related to the animal's previous actions. Although the ventral striatum has been proposed to play a key role in updating the information about the rewards expected from specific actions, it is not known whether the signals related to previous actions exist in the ventral striatum. In the present study, we recorded neuronal ensemble activity in the rat ventral striatum during a visual discrimination task and investigated whether neuronal activity in the ventral striatum encoded signals related to animal's previous actions. The results show that many neurons modulated their activity according to the animal's goal choice in the previous trial, indicating that memory signals for previous actions are available in the ventral striatum. In contrast, few neurons conveyed signals on impending goal choice of the animal, suggesting the absence of decision signals in the ventral striatum. Memory signals for previous actions might contribute to the process of updating the estimates of rewards expected from alternative actions in the ventral striatum.

Integration of cognitive and motivational information in the primate lateral prefrontal cortex

The prefrontal cortex (PFC), particularly the lateral prefrontal cortex (LPFC), has an important role in cognitive information processing. The area receives projections from sensory association cortices and sends outputs to motor-related areas. Neurons in LPFC code the behavioral significance of stimuli, which can be abstract precursors for complex motor commands and are structured hierarchically. Loss of these neurons leads to a lack of flexibility in decision making, such as seen in stereotyped behaviors. However, to make more appropriate decisions the code for behavioral significance has to reflect the subject's own desires and demands. Indeed, LPFC has connections with reward-related areas, such as the orbitofrontal cortex (OFC), basal ganglia, and medial prefrontal cortex. Recently, many studies have reported reward modulation of neural codes of behavioral significance. Using an asymmetric reward paradigm, we can investigate the functional specificity of LPFC neurons that code both cognitive information and motivational information. In this review, we will discuss details of neuronal properties of LPFC neurons from the viewpoints of cognitive information processing and motivational information processing, and the question of how these two pieces of information are integrated. Abstract coding and contextual representations in the cognitive information processing are functional characteristics of LPFC. Such functional specificity in LPFC cognitive processes is supported by a long-term scale of reward history in the motivational information processing. The integration enables us to make an elaborate decision with respect to goal-directed behavior in complex circumstances.

Functional role of the ventrolateral prefrontal cortex in decision making

To make deliberate decisions, we have to utilize detailed information about the environment and our internal states. The ventral visual pathway provides detailed information on object identity, including color and shape, to the ventrolateral prefrontal cortex (VLPFC). The VLPFC also receives motivational and emotional information from the orbitofrontal cortex and subcortical areas, and computes the behavioral significance of external events; this information can be used for elaborate decision making or design of goal-directed behavior. In this review, we discuss recent advances that are revealing the neural mechanisms that underlie the coding of behavioral significance in the VLPFC, and the functional roles of these mechanisms in decision making and action programming in the brain.

Role of anticipated reward in cognitive behavioral control

The lateral prefrontal cortex (LPFC), which is important for higher cognitive activity, is also concerned with motivational operations; this is exemplified by its activity in relation to expectancy of rewards. In the LPFC, motivational information is integrated with cognitive information, as demonstrated by the enhancement of working-memory-related activity by reward expectancy. Such activity would be expected to induce changes in attention and, subsequently, to modify behavioral performance. Recently, the effects of motivation and emotion on neural activities have been examined in several areas of the brain in relation to cognitive-task performance. Of these areas, the LPFC seems to have the most important role in adaptive goal-directed behavior, by sending top-down attention-control signals to other areas of the brain.