Neuronal coding of prediction errors

The influence of feedback valence in associative learning

The neural systems engaged by intrinsic positive or negative feedback were defined in an associative learning task. Through trial and error, participants learned the arbitrary assignments of a set of stimuli to one of two response categories. Informative feedback was provided on less than 25% of the trials. During positive feedback blocks, half of the trials were eligible for informative feedback; of these, informative feedback was only provided when the response was correct. A similar procedure was used on negative feedback blocks, but here informative feedback was only provided when the response was incorrect. In this manner, we sought to identify regions that were differentially responsive to positive and negative feedback as well as areas that were responsive to both types of informative feedback. Several regions of interest, including the bilateral nucleus accumbens, caudate nucleus, anterior insula, right cerebellar lobule VI, and left putamen, were sensitive to informative feedback regardless of valence. In contrast, several regions were more selective to positive feedback compared to negative feedback. These included the insula, amygdala, putamen, and supplementary motor area. No regions were more strongly activated by negative feedback compared to positive feedback. These results indicate that the neural areas supporting associative learning vary as a function of how that information is learned. In addition, areas linked to intrinsic reinforcement showed considerable overlap with those identified in studies using extrinsic reinforcers.

Rational and mechanistic perspectives on reinforcement learning

This special issue describes important recent developments in applying reinforcement learning models to capture neural and cognitive function. But reinforcement learning, as a theoretical framework, can apply at two very different levels of description: mechanistic and rational. Reinforcement learning is often viewed in mechanistic terms--as describing the operation of aspects of an agent's cognitive and neural machinery. Yet it can also be viewed as a rational level of description, specifically, as describing a class of methods for learning from experience, using minimal background knowledge. This paper considers how rational and mechanistic perspectives differ, and what types of evidence distinguish between them. Reinforcement learning research in the cognitive and brain sciences is often implicitly committed to the mechanistic interpretation. Here the opposite view is put forward: that accounts of reinforcement learning should apply at the rational level, unless there is strong evidence for a mechanistic interpretation. Implications of this viewpoint for reinforcement-based theories in the cognitive and brain sciences are discussed.

Temporally limited role of substantia nigra-central amygdala connections in surprise-induced enhancement of learning

Prediction error plays an important role in modern associative learning theories. For example, the omission of an expected event (surprise) can enhance attention to cues that accompany those omissions, such that subsequent new learning about those cues is more rapid. Many studies from our laboratories have demonstrated that circuitry that includes the amygdala central nucleus (CeA), the cholinergic neurons in the substantia innominata/nucleus basalis region and their innervation of the posterior parietal cortex is critical for this surprise-induced enhancement of attention in learning. We recently showed that midbrain dopamine neurons, known to code prediction error, are also important for surprise-induced enhancement of learning through their interaction with CeA. The present study examined whether in rats the communication between the substantia nigra pars compacta (SNc) and CeA is critical only at the time of surprise, for example to detect prediction error information, or is also needed to maintain and later express that information as enhanced learning. All animals received unilateral CeA lesions and unilateral cannula implants targeting the SNc located contralateral to the lesioned CeA. As the SNc-CeA connections are mainly ipsilateral, inactivating SNc contralateral to the lesioned CeA provided transient blockage of SNc and CeA communication. The results show that SNc-CeA communication is critical for processing prediction error information at the time of surprise, but neither SNc nor SNc-CeA communication is necessary to express that information as enhanced learning later.

Functional grouping and cortical-subcortical interactions in emotion: a meta-analysis of neuroimaging studies

We performed an updated quantitative meta-analysis of 162 neuroimaging studies of emotion using a novel multi-level kernel-based approach, focusing on locating brain regions consistently activated in emotional tasks and their functional organization into distributed functional groups, independent of semantically defined emotion category labels (e.g., "anger," "fear"). Such brain-based analyses are critical if our ways of labeling emotions are to be evaluated and revised based on consistency with brain data. Consistent activations were limited to specific cortical sub-regions, including multiple functional areas within medial, orbital, and inferior lateral frontal cortices. Consistent with a wealth of animal literature, multiple subcortical activations were identified, including amygdala, ventral striatum, thalamus, hypothalamus, and periaqueductal gray. We used multivariate parcellation and clustering techniques to identify groups of co-activated brain regions across studies. These analyses identified six distributed functional groups, including medial and lateral frontal groups, two posterior cortical groups, and paralimbic and core limbic/brainstem groups. These functional groups provide information on potential organization of brain regions into large-scale networks. Specific follow-up analyses focused on amygdala, periaqueductal gray (PAG), and hypothalamic (Hy) activations, and identified frontal cortical areas co-activated with these core limbic structures. While multiple areas of frontal cortex co-activated with amygdala sub-regions, a specific region of dorsomedial prefrontal cortex (dmPFC, Brodmann's Area 9/32) was the only area co-activated with both PAG and Hy. Subsequent mediation analyses were consistent with a pathway from dmPFC through PAG to Hy. These results suggest that medial frontal areas are more closely associated with core limbic activation than their lateral counterparts, and that dmPFC may play a particularly important role in the cognitive generation of emotional states.

A unified framework for addiction: vulnerabilities in the decision process

The understanding of decision-making systems has come together in recent years to form a unified theory of decision-making in the mammalian brain as arising from multiple, interacting systems (a planning system, a habit system, and a situation-recognition system). This unified decision-making system has multiple potential access points through which it can be driven to make maladaptive choices, particularly choices that entail seeking of certain drugs or behaviors. We identify 10 key vulnerabilities in the system: (1) moving away from homeostasis, (2) changing allostatic set points, (3) euphorigenic "reward-like" signals, (4) overvaluation in the planning system, (5) incorrect search of situation-action-outcome relationships, (6) misclassification of situations, (7) overvaluation in the habit system, (8) a mismatch in the balance of the two decision systems, (9) over-fast discounting processes, and (10) changed learning rates. These vulnerabilities provide a taxonomy of potential problems with decision-making systems. Although each vulnerability can drive an agent to return to the addictive choice, each vulnerability also implies a characteristic symptomology. Different drugs, different behaviors, and different individuals are likely to access different vulnerabilities. This has implications for an individual's susceptibility to addiction and the transition to addiction, for the potential for relapse, and for the potential for treatment.

Striatal activity underlies novelty-based choice in humans

The desire to seek new and unfamiliar experiences is a fundamental behavioral tendency in humans and other species. In economic decision making, novelty seeking is often rational, insofar as uncertain options may prove valuable and advantageous in the long run. Here, we show that, even when the degree of perceptual familiarity of an option is unrelated to choice outcome, novelty nevertheless drives choice behavior. Using functional magnetic resonance imaging (fMRI), we show that this behavior is specifically associated with striatal activity, in a manner consistent with computational accounts of decision making under uncertainty. Furthermore, this activity predicts interindividual differences in susceptibility to novelty. These data indicate that the brain uses perceptual novelty to approximate choice uncertainty in decision making, which in certain contexts gives rise to a newly identified and quantifiable source of human irrationality.

Developmental and target-dependent regulation of vesicular glutamate transporter expression by dopamine neurons

Mesencephalic dopamine (DA) neurons have been suggested to use glutamate as a cotransmitter. Here, we suggest a mechanism for this form of cotransmission by showing that a subset of DA neurons both in vitro and in vivo expresses vesicular glutamate transporter 2 (VGluT2). Expression of VGluT2 decreases with age. Moreover, when DA neurons are grown in isolation using a microculture system, there is a marked upregulation of VGluT2 expression. We provide evidence that expression of this transporter is normally repressed through a contact-dependent interaction with GABA and other DA neurons, thus providing a partial explanation for the highly restricted expression of VGluT2 in DA neurons in vivo. Our results demonstrate that the neurotransmitter phenotype of DA neurons is both developmentally and dynamically regulated. These findings may have implications for a better understanding of the fast synaptic action of DA neurons as well as basal ganglia circuitry.

Parkinson's disease and dopaminergic therapy--differential effects on movement, reward and cognition

Cognitive deficits are very common in Parkinson's disease particularly for ‘executive functions’ associated with frontal cortico-striatal networks. Previous work has identified deficits in tasks that require attentional control like task-switching, and reward-based tasks like gambling or reversal learning. However, there is a complex relationship between the specific cognitive problems faced by an individual patient, their stage of disease and dopaminergic treatment. We used a bimodality continuous performance task during fMRI to examine how patients with Parkinson's disease represent the prospect of reward and switch between competing task rules accordingly. The task-switch was not separately cued but was based on the implicit reward relevance of spatial and verbal dimensions of successive compound stimuli. Nineteen patients were studied in relative ‘on’ and ‘off’ states, induced by dopaminergic medication withdrawal (Hoehn and Yahr stages 1–4). Patients were able to successfully complete the task and establish a bias to one or other dimension in order to gain reward. However the lateral prefrontal cortex and caudate nucleus showed a non-linear U-shape relationship between motor disease severity and regional brain activation. Dopaminergic treatment led to a shift in this U-shape function, supporting the hypothesis of differential neurodegeneration in separate motor and cognitive cortico–striato–thalamo–cortical circuits. In addition, anterior cingulate activation associated with reward expectation declined with more severe disease, whereas activation following actual rewards increased with more severe disease. This may facilitate a change in goal-directed behaviours from deferred predicted rewards to immediate actual rewards, particularly when on dopaminergic treatment. We discuss the implications for investigation and optimal treatment of this common condition at different stages of disease.

A phosphatase cascade by which rewarding stimuli control nucleosomal response

Dopamine orchestrates motor behaviour and reward-driven learning. Perturbations of dopamine signalling have been implicated in several neurological and psychiatric disorders, and in drug addiction. The actions of dopamine are mediated in part by the regulation of gene expression in the striatum, through mechanisms that are not fully understood. Here we show that drugs of abuse, as well as food reinforcement learning, promote the nuclear accumulation of 32-kDa dopamine-regulated and cyclic-AMP-regulated phosphoprotein (DARPP-32). This accumulation is mediated through a signalling cascade involving dopamine D1 receptors, cAMP-dependent activation of protein phosphatase-2A, dephosphorylation of DARPP-32 at Ser 97 and inhibition of its nuclear export. The nuclear accumulation of DARPP-32, a potent inhibitor of protein phosphatase-1, increases the phosphorylation of histone H3, an important component of nucleosomal response. Mutation of Ser 97 profoundly alters behavioural effects of drugs of abuse and decreases motivation for food, underlining the functional importance of this signalling cascade.

The influence of personality on neural mechanisms of observational fear and reward learning

Fear and reward learning can occur through direct experience or observation. Both channels can enhance survival or create maladaptive behavior. We used fMRI to isolate neural mechanisms of observational fear and reward learning and investigate whether neural response varied according to individual differences in neuroticism and extraversion. Participants learned object-emotion associations by observing a woman respond with fearful (or neutral) and happy (or neutral) facial expressions to novel objects. The amygdala-hippocampal complex was active when learning the object-fear association, and the hippocampus was active when learning the object-happy association. After learning, objects were presented alone; amygdala activity was greater for the fear (vs. neutral) and happy (vs. neutral) associated object. Importantly, greater amygdala-hippocampal activity during fear (vs. neutral) learning predicted better recognition of learned objects on a subsequent memory test. Furthermore, personality modulated neural mechanisms of learning. Neuroticism positively correlated with neural activity in the amygdala and hippocampus during fear (vs. neutral) learning. Low extraversion/high introversion was related to faster behavioral predictions of the fearful and neutral expressions during fear learning. In addition, low extraversion/high introversion was related to greater amygdala activity during happy (vs. neutral) learning, happy (vs. neutral) object recognition, and faster reaction times for predicting happy and neutral expressions during reward learning. These findings suggest that neuroticism is associated with an increased sensitivity in the neural mechanism for fear learning which leads to enhanced encoding of fear associations, and that low extraversion/high introversion is related to enhanced conditionability for both fear and reward learning.

Spike timing-dependent long-term potentiation in ventral tegmental area dopamine cells requires PKC

Long-term potentiation (LTP) of excitatory synapses on ventral tegmental area (VTA) dopamine (DA) cells is thought to play an important role in mediating some of the behavioral effects of drugs of abuse yet little is known about its underlying mechanisms. We find that spike timing-dependent LTP (STD LTP) in VTA DA cells is absent in slices prepared from mice previously administered cocaine, suggesting that cocaine-induced LTP and STD LTP share underlying mechanisms. This form of STD LTP is dependent on NMDA receptor (NMDAR) activation and a rise in postsynaptic calcium but surprisingly was not affected by an inhibitor of calcium/calmodulin-dependent protein kinase II (CaMKII). It was blocked by antagonists of conventional isoforms of PKC, whereas activation of protein kinase C (PKC) using a phorbol ester enhanced synaptic strength. These results suggest that NMDAR-mediated activation of PKC, but not CaMKII, is a critical trigger for LTP in VTA DA cells.

Segmentation in the perception and memory of events

People make sense of continuous streams of observed behavior in part by segmenting them into events. Event segmentation seems to be an ongoing component of everyday perception. Events are segmented simultaneously at multiple timescales, and are grouped hierarchically. Activity in brain regions including the posterior temporal and parietal cortex and lateral frontal cortex increases transiently at event boundaries. The parsing of ongoing activity into events is related to the updating of working memory, to the contents of long-term memory, and to the learning of new procedures. Event segmentation might arise as a side effect of an adaptive mechanism that integrates information over the recent past to improve predictions about the near future.

Neurobiological mechanisms of cannabinoid addiction

The endocannabinoid system is implicated in the regulation of a variety of physiological processes, among which conditioning, motivation, habit forming, memory, learning, and cognition play pivotal roles in drug reinforcement and reward. In this article we will give a synopsis of last developments in research on cannabinoid actions on brain reward circuits coming from behavioral, neurochemical and electrophysiological studies. Central cannabinoid-induced effects as measured by animal models of addiction, in vivo cerebral microdialysis, in vitro and in vivo electrophysiological recording techniques, will be reviewed. Brain sites that have been implicated in the mediation of addictive cannabinoid properties include primarily the ventral tegmental area, the nucleus accumbens, and the medial prefrontal cortex, although the amygdala, the substantia nigra, the globus pallidus, and the hippocampus have also been shown to be critical structures mediating motivational and reinforcing effects of cannabinoids. Putative neurobiological mechanisms underlying these effects will be delineated.

Risky business: the neuroeconomics of decision making under uncertainty

Many decisions involve uncertainty, or imperfect knowledge about how choices lead to outcomes. Colloquial notions of uncertainty, particularly when describing a decision as 'risky', often carry connotations of potential danger as well. Gambling on a long shot, whether a horse at the racetrack or a foreign oil company in a hedge fund, can have negative consequences, but the impact of uncertainty on decision making extends beyond gambling. Indeed, uncertainty in some form pervades nearly all our choices in daily life. Stepping into traffic to hail a cab, braving an ice storm to be the first at work, or dating the boss's son or daughter also offer potentially great windfalls, at the expense of surety. We continually face trade-offs between options that promise safety and others that offer an uncertain potential for jackpot or bust. When mechanisms for dealing with uncertain outcomes fail, as in mental disorders such as problem gambling or addiction, the results can be disastrous. Thus, understanding decision making-indeed, understanding behavior itself-requires knowing how the brain responds to and uses information about uncertainty.

Reward-related activity in the human motor cortex

The human primary motor cortex (M1) participates in motor learning and response selection, functions that rely on feedback on the success of behavior (i.e. reward). To investigate the possibility that behavioral contingencies alter M1 activity in humans, we tested intracortical inhibition with single and paired (subthreshold/suprathreshold) transcranial magnetic stimulation during a slot machine simulation that delivered variable money rewards for three-way matches and required no movement. A two-way match before the third barrel had stopped (increased reward expectation) was associated with more paired-pulse inhibition than no match. Receiving a large reward on the preceding trial augmented this effect. A control task that manipulated attention to the same stimuli produced no changes in excitability. The origin of this reward-related activity is not clear, although dopaminergic ventral tegmental area neurons project to M1, where they are thought to inhibit output neurons and could be the source of the finding. Transcranial magnetic stimulation of M1 may be useful as a quantitative measure of reward-related activity.

Visuotactile learning and body representation: an ERP study with rubber hands and rubber objects

We studied how the integration of seen and felt tactile stimulation modulates somatosensory processing, and investigated whether visuotactile integration depends on temporal contiguity of stimulation, and its coherence with a preexisting body representation. During training, participants viewed a rubber hand or a rubber object that was tapped either synchronously with stimulation of their own hand, or in an uncorrelated fashion. In a subsequent test phase, somatosensory event-related potentials (ERPs) were recorded to tactile stimulation of the left or right hand, to assess how tactile processing was affected by previous visuotactile experience during training. An enhanced somatosensory N140 component was elicited after synchronous, compared with uncorrelated, visuotactile training, irrespective of whether participants viewed a rubber hand or rubber object. This early effect of visuotactile integration on somatosensory processing is interpreted as a candidate electro-physiological correlate of the rubber hand illusion that is determined by temporal contiguity, but not by preexisting body representations. ERP modulations were observed beyond 200 msec poststimulus, suggesting an attentional bias induced by visuotactile training. These late modulations were absent when the stimulation of a rubber hand and the participant's own hand was uncorrelated during training, suggesting that preexisting body representations may affect later stages of tactile processing.

Substantia nigra/ventral tegmental reward prediction error disruption in psychosis

While dopamine systems have been implicated in the pathophysiology of schizophrenia and psychosis for many years, how dopamine dysfunction generates psychotic symptoms remains unknown. Recent theoretical interest has been directed at relating the known role of midbrain dopamine neurons in reinforcement learning, motivational salience and prediction error to explain the abnormal mental experience of psychosis. However, this theoretical model has yet to be explored empirically. To examine a link between psychotic experience, reward learning and dysfunction of the dopaminergic midbrain and associated target regions, we asked a group of first episode psychosis patients suffering from active positive symptoms and a group of healthy control participants to perform an instrumental reward conditioning experiment. We characterized neural responses using functional magnetic resonance imaging. We observed that patients with psychosis exhibit abnormal physiological responses associated with reward prediction error in the dopaminergic midbrain, striatum and limbic system, and we demonstrated subtle abnormalities in the ability of psychosis patients to discriminate between motivationally salient and neutral stimuli. This study provides the first evidence linking abnormal mesolimbic activity, reward learning and psychosis.

Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback

A simple working memory model based on recurrent network activation is proposed and its application to selection and reinforcement of an action is demonstrated as a solution to the temporal credit assignment problem. Reactivation of recent salient cue states is generated and maintained as a type of salience gated recurrently active working memory, while lower salience distractors are ignored. Cue reactivation during the action selection period allows the cue to select an action while its reactivation at the reward period allows the reinforcement of the action selected by the reactivated state, which is necessarily the action which led to the reward being found. A down-gating of the external input during the reactivation and maintenance prevents interference. A double winner-take-all system which selects only one cue and only one action allows the targeting of the cue-action allocation to be modified. This targeting works both to reinforce a correct cue-action allocation and to punish the allocation when cue-action allocations change. Here we suggest a firing rate neural network implementation of this system based on the basal ganglia anatomy with input from a cortical association layer where reactivations are generated by signals from the thalamus. Striatum medium spiny neurons represent actions. Auto-catalytic feedback from a dopamine reward signal modulates three-way Hebbian long term potentiation and depression at the cortical-striatal synapses which represent the cue-action associations. The model is illustrated by the numerical simulations of a simple example--that of associating a cue signal to a correct action to obtain reward after a delay period, typical of primate cue reward tasks. Through learning, the model shows a transition from an exploratory phase where actions are generated randomly, to a stable directed phase where the animal always chooses the correct action for each experienced state. When cue-action allocations change, we show that this is noticed by the model, the incorrect cue-action allocations are punished and the correct ones discovered.

VTA dopamine neuron bursting is altered in an animal model of depression and corrected by desipramine

Ventral tegmental area (VTA) neuronal activity plays an important role in reward-related learning and motivation. Tracing the bursting signal is important for understanding neural state and understanding communication between individual neurons. The dopaminergic system, which projects from the VTA to other regions in the mesolimbic system, is involved in hedonia and motivation. However, the role of this system in the pathophysiology of depression and its manipulation for treatment of depression has received little attention. Inter-spike interval time series were recorded from the VTA of control Sprague-Dawley and Flinders sensitive line (FSL) rats with or without 14 days of desipramine (5 mg/kg) treatment. Comparison of the firing modes of control and desipramine-treated FSL rats reveals dissimilar patterns. Desipramine treatment normalized depressive-like behavior and elevated the dopaminergic mesolimbic activity, although not to control levels. Mesolimbic neuronal activity is known to occur either in burst or in single-spike firing mode. Herein, we suggest a third mode that is characterized as a "cluster" formed from burst and post-burst activity. A significant reduction in the activity of both bursts and cluster was detected in FSL rats, which was restored by desipramine treatment.

Hippocampal and neocortical interactions during context discrimination: electrophysiological evidence from the rat

There is substantial evidence that hippocampus plays an important role in the processing of contextual information. Its specific role, however, remains unclear. One possibility is that single hippocampal neurons represent context information so that local circuits can construct representations of the current context, and the context that is expected based on past experience. Population codes derived from input by multiple local circuits may then engage match-mismatch algorithms that compare current and expected context information to determine the extent to which an expected context has changed. The results of such match-mismatch comparisons can be used to discriminate contexts. When context changes are detected, efferent messages may be passed on to connected neocortical areas so that informed "decisions" regarding future behavioral and cognitive strategies can be made. Here, a brief review describes evidence that a primary consequence of hippocampal processing is the discrimination of meaningful contexts. Then, the functional significance of neocortical circuits that likely receive hippocampal output messages are described in terms of their contribution to the control of ongoing behavioral and cognitive strategy, especially during active navigation. It is clear from this systems view that studies of spatial navigation continue to provide researchers with an excellent model of hippocampal-neocortical interactions during learning.

Behavioral functions of the mesolimbic dopaminergic system: an affective neuroethological perspective

The mesolimbic dopaminergic (ML-DA) system has been recognized for its central role in motivated behaviors, various types of reward, and, more recently, in cognitive processes. Functional theories have emphasized DA's involvement in the orchestration of goal-directed behaviors and in the promotion and reinforcement of learning. The affective neuroethological perspective presented here views the ML-DA system in terms of its ability to activate an instinctual emotional appetitive state (SEEKING) evolved to induce organisms to search for all varieties of life-supporting stimuli and to avoid harms. A description of the anatomical framework in which the ML system is embedded is followed by the argument that the SEEKING disposition emerges through functional integration of ventral basal ganglia (BG) into thalamocortical activities. Filtering cortical and limbic input that spreads into BG, DA transmission promotes the "release" of neural activity patterns that induce active SEEKING behaviors when expressed at the motor level. Reverberation of these patterns constitutes a neurodynamic process for the inclusion of cognitive and perceptual representations within the extended networks of the SEEKING urge. In this way, the SEEKING disposition influences attention, incentive salience, associative learning, and anticipatory predictions. In our view, the rewarding properties of drugs of abuse are, in part, caused by the activation of the SEEKING disposition, ranging from appetitive drive to persistent craving depending on the intensity of the affect. The implications of such a view for understanding addiction are considered, with particular emphasis on factors predisposing individuals to develop compulsive drug seeking behaviors.

Frequency-dependent gating of synaptic transmission and plasticity by dopamine

The neurotransmitter dopamine (DA) plays an important role in learning by enhancing the saliency of behaviorally relevant stimuli. How this stimulus selection is achieved on the cellular level, however, is not known. Here, in recordings from hippocampal slices, we show that DA acts specifically at the direct cortical input to hippocampal area CA1 (the temporoammonic (TA) pathway) to filter the excitatory drive onto pyramidal neurons based on the input frequency. During low-frequency patterns of stimulation, DA depressed excitatory TA inputs to both CA1 pyramidal neurons and local inhibitory GABAergic interneurons via presynaptic inhibition. In contrast, during high-frequency patterns of stimulation, DA potently facilitated the TA excitatory drive onto CA1 pyramidal neurons, owing to diminished feedforward inhibition. Analysis of DA's effects over a broad range of stimulus frequencies indicates that it acts as a high-pass filter, augmenting the response to high-frequency inputs while diminishing the impact of low-frequency inputs. These modulatory effects of DA exert a profound influence on activity-dependent forms of synaptic plasticity at both TA-CA1 and Schaffer-collateral (SC)-CA1 synapses. Taken together, our data demonstrate that DA acts as a gate on the direct cortical input to the hippocampus, modulating information flow and synaptic plasticity in a frequency-dependent manner.

Changes in neural activity associated with a surprising change in the predictive validity of a conditioned stimulus

Changes in how well a conditioned stimulus (CS) predicts future events can alter the amount of attention paid to that cue. For example, the unexpected violation of a previously established relationship between a CS and another stimulus can increase attentional processing and subsequent conditioning to that cue [J.M. Pearce & G. Hall (1980)Psych. Rev., 106, 532-552]. Previous lesion studies have implicated the central nucleus of the amygdala (CN) and basal forebrain corticopetal cholinergic system in mediating surprise-induced changes in attention. Here, expression of the immediate-early gene c-fos was used to determine which cortical targets of the basal forebrain cholinergic system are activated during an increase in attentional processing. Consistent with previous studies, increased Fos expression was observed in the posterior parietal cortex (PPC) when a visual stimulus no longer reliably predicted occurrence of a tone. Similar results were observed in the secondary auditory cortex; however, there were no significant changes in Fos expression in other auditory or visual cortices or in other cortical association areas that have been implicated in attentional function (frontal, cingulate or retrosplenial cortex). These findings support the notion that the PPC is the primary cortical component of a neural system mediating incremental changes in attention. In addition, an increase in Fos-positive cells was detected in the substantia innominata/nucleus basalis and the CN at the time of surprise. An opposite pattern of results was observed in the basal lateral nucleus of the amygdala, providing evidence for different stimulus-processing mechanisms in regions of the amygdala.

Estimating the hidden learning representations

Successful adaptation relies on the ability to learn the consequence of our actions in different environments. However, understanding the neural bases of this ability still represents one of the great challenges of system neuroscience. In fact, the neuronal plasticity changes occurring during learning cannot be fully controlled experimentally and their evolution is hidden. Our approach is to provide hypotheses about the structure and dynamics of the hidden plasticity changes using behavioral learning theory. In fact, behavioral models of animal learning provide testable predictions about the hidden learning representations by formalizing their relation with the observables of the experiment (stimuli, actions and outcomes). Thus, we can understand whether and how the predicted learning processes are represented at the neural level by estimating their evolution and correlating them with neural data. Here, we present a bayesian model approach to estimate the evolution of the internal learning representations from the observations of the experiment (state estimation), and to identify the set of models' parameters (parameter estimation) and the class of behavioral model (model selection) that are most likely to have generated a given sequence of actions and outcomes. More precisely, we use Sequential Monte Carlo methods for state estimation and the maximum likelihood principle (MLP) for model selection and parameter estimation. We show that the method recovers simulated trajectories of learning sessions on a single-trial basis and provides predictions about the activity of different categories of neurons that should participate in the learning process. By correlating the estimated evolutions of the learning variables, we will be able to test the validity of different models of instrumental learning and possibly identify the neural bases of learning.

Paradoxical modulation of short-term facilitation of dopamine release by dopamine autoreceptors

Electrophysiological studies have demonstrated that dopaminergic neurons burst fire during certain aspects of reward-related behavior; however, the correlation between dopamine release and cell firing is unclear. When complex stimulation patterns that mimic intracranial self-stimulation were employed, dopamine release was shown to exhibit facilitated as well as depressive components (Montague et al. 2004). Understanding the biological mechanisms underlying these variations in dopamine release is necessary to unravel the correlation between unit activity and neurotransmitter release. The dopamine autoreceptor provides negative feedback to dopamine release, inhibiting release on the time scale of a few seconds. Therefore, we investigated this D(2) receptor to see whether it is one of the biological mechanisms responsible for the history-dependent modulation of dopamine release. Striatal dopamine release in anesthetized rats was evoked with stimulus trains that were designed to promote the variability of dopamine release. Consistent with the well established D(2)-mediated autoinhibition, the short-term depressive component of dopamine release was blocked by raclopride, a D(2) antagonist, and enhanced by quinpirole, a D(2)-receptor agonist. Surprisingly, these same drugs exerted a similar effect on the short-term facilitated component: a decrease with raclopride and an increase with quinpirole. These data demonstrate that the commanding control exerted by dopamine autoreceptors over short-term neuroadaptation of dopamine release involves both inhibitory and paradoxically, facilitatory components.

Dynamic causal models of neural system dynamics:current state and future extensions

Complex processes resulting from interaction of multiple elements can rarely be understood by analytical scientific approaches alone; additional, mathematical models of system dynamics are required. This insight, which disciplines like physics have embraced for a long time already, is gradually gaining importance in the study of cognitive processes by functional neuroimaging. In this field, causal mechanisms in neural systems are described in terms of effective connectivity. Recently, dynamic causal modelling (DCM) was introduced as a generic method to estimate effective connectivity from neuroimaging data in a Bayesian fashion. One of the key advantages of DCM over previous methods is that it distinguishes between neural state equations and modality-specific forward models that translate neural activity into a measured signal. Another strength is its natural relation to Bayesian model selection (BMS) procedures. In this article, we review the conceptual and mathematical basis of DCM and its implementation for functional magnetic resonance imaging data and event-related potentials. After introducing the application of BMS in the context of DCM, we conclude with an outlook to future extensions of DCM. These extensions are guided by the long-term goal of using dynamic system models for pharmacological and clinical applications, particularly with regard to synaptic plasticity.

Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala

Animals and humans learn to approach and acquire pleasant stimuli and to avoid or defend against aversive ones. However, both pleasant and aversive stimuli can elicit arousal and attention, and their salience or intensity increases when they occur by surprise. Thus, adaptive behavior may require that neural circuits compute both stimulus valence--or value--and intensity. To explore how these computations may be implemented, we examined neural responses in the primate amygdala to unexpected reinforcement during learning. Many amygdala neurons responded differently to reinforcement depending upon whether or not it was expected. In some neurons, this modulation occurred only for rewards or aversive stimuli, but not both. In other neurons, expectation similarly modulated responses to both rewards and punishments. These different neuronal populations may subserve two sorts of processes mediated by the amygdala: those activated by surprising reinforcements of both valences-such as enhanced arousal and attention-and those that are valence-specific, such as fear or reward-seeking behavior.

Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions

Delusions are maladaptive beliefs about the world. Based upon experimental evidence that prediction error-a mismatch between expectancy and outcome--drives belief formation, this study examined the possibility that delusions form because of disrupted prediction--error processing. We used fMRI to determine prediction-error-related brain responses in 12 healthy subjects and 12 individuals (7 males) with delusional beliefs. Frontal cortex responses in the patient group were suggestive of disrupted prediction-error processing. Furthermore, across subjects, the extent of disruption was significantly related to an individual's propensity to delusion formation. Our results support a neurobiological theory of delusion formation that implicates aberrant prediction-error signalling, disrupted attentional allocation and associative learning in the formation of delusional beliefs.

How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback

This article examines how independent corticostriatal loops linking basal ganglia with cerebral cortex contribute to visual categorization. The first aspect of categorization discussed is the role of the visual corticostriatal loop, which connects the visual cortex and the body/tail of the caudate, in mapping visual stimuli to categories, including evaluating the degree to which this loop may generalize across individual category members. The second aspect of categorization discussed is the selection of appropriate actions or behaviors on the basis of category membership, and the role of the visual corticostriatal loop output and the motor corticostriatal loop, which connects motor planning areas with the putamen, in action selection. The third aspect of categorization discussed is how categories are learned with the aid of feedback linked dopaminergic projections to the basal ganglia. These projections underlie corticostriatal synaptic plasticity across the basal ganglia, and also serve as input to the executive and motivational corticostriatal loops that play a role in strategic use of feedback.

Potentiation phase of spike timing-dependent neuromodulation by a serotonergic interneuron involves an increase in the fraction of transmitter release

In the mollusk, Tritonia diomedea, the serotonergic dorsal swim interneuron (DSI) produces spike timing-dependent neuromodulation (STDN) of the synaptic output of ventral swim interneuron B (VSI) resulting in a biphasic, bidirectional change of synaptic strength characterized by a rapid heterosynaptic potentiation followed by a more prolonged heterosynaptic depression. This study examined the mechanism underlying the potentiation phase of STDN. In the presence of 4-aminopyridine, which blocks the depression phase and enhances transmitter release from VSI, rapidly stimulating VSI led to a steady-state level of transmitter depletion during which potentiation by DSI or serotonin (5-HT) was eliminated. Cumulative plots of excitatory postsynaptic currents were used to estimate changes in the size and replenishment rate of the readily releasable pool (RRP) and the fraction of release. 5-HT application increased transmitter release without altering replenishment rate. The magnitude of 5-HT-evoked potentiation correlated with the increase in the fraction of release. A phenomenological model of the synapse further supported the hypothesis that 5-HT-induced potentiation was caused by an increase in the fraction of release and correctly predicted no change in frequency facilitation. A dynamic version of the model correctly predicted the effect of DSI stimulation under a variety of conditions. Finally, depletion of internal Ca(2+) stores with cyclopiazonic acid showed that Ca(2+) from internal stores is necessary for the 5-HT-induced potentiation. The data indicate that 5-HT released from DSI increases the fraction of the RRP discharged during VSI action potentials using a mechanism that involves Ca(2+) extrusion from internal stores, resulting in time- and state-dependent neuromodulation.

Impact of commitment on performance evaluation in the rostral cingulate motor area

Performance evaluation is a prerequisite for behavioral adaptation. Although the anterior cingulate cortex (ACC) is thought to play a central role in error detection, little is known about the electrophysiological activity of this structure during the performance-monitoring process. We directly addressed this issue by training monkeys to perform a Stroop-like task and then recorded neuronal activity in the rostral cingulate motor area (CMAr), a relatively unexplored region of the ACC known to be involved in motor processing. We found that most CMAr neurons responded during the evaluation period to both positive and negative feedback, but neuronal changes were more important after an error than after a successful trial. Interestingly, this performance-monitoring activity was not directly modulated by the degree of difficulty of the cognitive situation because changes in discharge frequency were similar whatever the level of attentional control imposed on the monkey. Firing activity during the evaluation period increased more, however, in erroneously completed than in incompleted trials and when the reward was delivered in an active rather than passive context, indicating that performance evaluation was conditioned by the degree of commitment of the animal to the task. It would thus seem that CMAr neurons could constitute a system for the evaluation of behavioral performance contingent on the subject's commitment to the task.

Cortical modulations increase in early sessions with brain-machine interface

BACKGROUND

During planning and execution of reaching movements, the activity of cortical motor neurons is modulated by a diversity of motor, sensory, and cognitive signals. Brain-machine interfaces (BMIs) extract part of these modulations to directly control artificial actuators. However, cortical modulations that emerge in the novel context of operating the BMI are poorly understood.

METHODOLOGY/PRINCIPAL FINDINGS

Here we analyzed the changes in neuronal modulations that occurred in different cortical motor areas as monkeys learned to use a BMI to control reaching movements. Using spike-train analysis methods we demonstrate that the modulations of the firing-rates of cortical neurons increased abruptly after the monkeys started operating the BMI. Regression analysis revealed that these enhanced modulations were not correlated with the kinematics of the movement. The initial enhancement in firing rate modulations declined gradually with subsequent training in parallel with the improvement in behavioral performance.

CONCLUSIONS/SIGNIFICANCE

We conclude that the enhanced modulations are related to computational tasks that are significant especially in novel motor contexts. Although the function and neuronal mechanism of the enhanced cortical modulations are open for further inquiries, we discuss their potential role in processing execution errors and representing corrective or explorative activity. These representations are expected to contribute to the formation of internal models of the external actuator and their decoding may facilitate BMI improvement.

Behavioral and neural analysis of associative learning in the honeybee: a taste from the magic well

Equipped with a mini brain smaller than one cubic millimeter and containing only 950,000 neurons, honeybees could be indeed considered as having rather limited cognitive abilities. However, bees display a rich and interesting behavioral repertoire, in which learning and memory play a fundamental role in the framework of foraging activities. We focus on the question of whether adaptive behavior in honeybees exceeds simple forms of learning and whether the neural mechanisms of complex learning can be unraveled by studying the honeybee brain. Besides elemental forms of learning, in which bees learn specific and univocal links between events in their environment, bees also master different forms of non-elemental learning, including categorization, contextual learning and rule abstraction, both in the visual and in the olfactory domain. Different protocols allow accessing the neural substrates of some of these learning forms and understanding how complex problem solving can be achieved by a relatively simple neural architecture. These results underline the enormous richness of experience-dependent behavior in honeybees, its high flexibility, and the fact that it is possible to formalize and characterize in controlled laboratory protocols basic and higher-order cognitive processing using an insect as a model.

Dissociable effects of disconnecting amygdala central nucleus from the ventral tegmental area or substantia nigra on learned orienting and incentive motivation

Recent evidence suggests that the amygdala central nucleus (CeA) and midbrain-striatal dopamine systems are critically involved in the alteration of attentional and emotional processing of initially neutral stimuli by associative learning. In rats, the acquisition of learned orienting responses (ORs) to visual cues paired with food is impaired by lesions of the CeA, and by lesions that disconnect CeA from the dorsolateral striatum (DLS), a region traditionally implicated in elevated responsiveness to sensory stimuli. Similarly, the acquisition of emotional significance to cues paired with food also depends on the function of CeA and of the ventral striatal nucleus accumbens (ACB), a region often considered crucial to acquired reward and motivation. For example, the ability of a cue previously paired with food to increase the rate of food-reinforced instrumental responding (Pavlovian-instrumental transfer, PIT) is eliminated by lesions of the CeA or the accumbens core. In this experiment, we found that lesions that functionally disconnected CeA from the substantia nigra pars compacta (SNc) impaired the acquisition of conditioned orienting to auditory cues paired with food, but had no effect on their ability to enhance instrumental responding, relative to the effects of unilateral lesions of that region. By contrast, lesions that disconnected CeA from the ventral tegmental area (VTA) had no effect on the acquisition of conditioned orienting, but facilitated Pavlovian-instrumental transfer relative to unilateral midbrain lesions, rescuing that function to sham-lesion control levels. Otherwise, unilateral lesions of either midbrain region impaired transfer. Implications of these results for circuit models of amygdalo-striatal interactions in associative learning are discussed.

Blunted response to feedback information in depressive illness

Depressive illness is associated with sustained widespread cognitive deficits, in addition to repeated experience of distressing emotions. An accepted theory, which broadly accounts for features of the syndrome, and its delayed response to antidepressant medication, is lacking. One possibility, which has received considerable attention, is that depressive illness is associated with a specific underlying deficit: a blunted or impaired ability to respond to feedback information. Unlike healthy controls, if patients with a depressive illness commit an error, they can be at increased risk of committing a subsequent error, possibly due to a failure to adjust performance in order to reduce the risk of error. In some speeded tasks, performance adjustment in humans is reliably associated with trial-to-trial change in reaction times (RTs), such as 'post-error slowing'. Previous studies of abnormal response to feedback have not investigated RT change in any detail. We used a combination of quantitative modelling of RTs and fMRI in 15 patients and 14 matched controls to test the hypothesis that depressive illness was associated with a blunted behavioural and neural response to feedback information during a gambling task. The results supported the hypothesis. Controls responded to negative ('lose') feedback by an increase in RT and activation of the anterior cingulate, the extent of which correlated with RT change. Patients did not significantly increase their RTs, nor activate the anterior cingulate. Controls responded to positive ('win') feedback by a reduction in RT and activation of the ventral striatum, the extent of which correlated with RT change. Patients neither reduced their RT nor activated the ventral striatum. RT adjustment correlated with self-reported anhedonia for both patients and controls. This behavioural deficit, together with its associated pattern of abnormal neural activity, implies that the anterior midline cortical substrate for error correction, which includes projections from the monoamine systems, is dysfunctional in depressive illness. Many studies have reported abnormalities of the medial frontal cortex in depressive illness; however, the mechanism by which antidepressant medication acts via the monoamine systems remains elusive. Our results suggest a direct link between the core subjective symptom of anhedonia, replicated neuropsychological deficits, electrophysiological and imaging abnormalities, and hypothesized dysfunction of the error correction system.

Shifting and stopping: fronto-striatal substrates, neurochemical modulation and clinical implications

The neuropsychological basis of attentional set-shifting, task-set switching and stop-signal inhibition is reviewed through comparative studies of humans and experimental animals. Using human functional neuroimaging, plus neuropsychological investigation of patients with frontal damage quantified by structural magnetic resonance imaging, and through parallels with effects of specific lesions of the prefrontal cortex (PFC) and striatum in rats and marmosets, it is possible to define both distinct and overlapping loci for tasks such as extra-dimensional shifting and reversal learning, stop-signal reaction time and task-set switching. Notably, most of the paradigms implicate a locus in the right PFC, specifically the right inferior frontal gyrus, possibly associated with processes of response inhibition. The neurochemical modulation of fronto-striatal circuitry in parallel with effects on task performance has been investigated using specific neuropharmacological agents in animals and by human psychopharmacological investigations, sometimes in conjunction with functional imaging. Evidence is provided for double dissociations of effects of manipulations of prefrontal cortical catecholamine and indoleamine (5-HT) systems that have considerable implications in the treatment of disorders such as Parkinson's disease, attention deficit/hyperactivity disorder and depression, as well as in theoretical notions of how 'fronto-executive' functions are subject to state-dependent influences, probably related to stress, arousal and motivation.

Processing of temporal unpredictability in human and animal amygdala

The amygdala has been studied extensively for its critical role in associative fear conditioning in animals and humans. Noxious stimuli, such as those used for fear conditioning, are most effective in eliciting behavioral responses and amygdala activation when experienced in an unpredictable manner. Here, we show, using a translational approach in mice and humans, that unpredictability per se without interaction with motivational information is sufficient to induce sustained neural activity in the amygdala and to elicit anxiety-like behavior. Exposing mice to mere temporal unpredictability within a time series of neutral sound pulses in an otherwise neutral sensory environment increased expression of the immediate-early gene c-fos and prevented rapid habituation of single neuron activity in the basolateral amygdala. At the behavioral level, unpredictable, but not predictable, auditory stimulation induced avoidance and anxiety-like behavior. In humans, functional magnetic resonance imaging revealed that temporal unpredictably causes sustained neural activity in amygdala and anxiety-like behavior as quantified by enhanced attention toward emotional faces. Our findings show that unpredictability per se is an important feature of the sensory environment influencing habituation of neuronal activity in amygdala and emotional behavior and indicate that regulation of amygdala habituation represents an evolutionary-conserved mechanism for adapting behavior in anticipation of temporally unpredictable events.

The proactive brain: using analogies and associations to generate predictions

Rather than passively 'waiting' to be activated by sensations, it is proposed that the human brain is continuously busy generating predictions that approximate the relevant future. Building on previous work, this proposal posits that rudimentary information is extracted rapidly from the input to derive analogies linking that input with representations in memory. The linked stored representations then activate the associations that are relevant in the specific context, which provides focused predictions. These predictions facilitate perception and cognition by pre-sensitizing relevant representations. Predictions regarding complex information, such as those required in social interactions, integrate multiple analogies. This cognitive neuroscience framework can help explain a variety of phenomena, ranging from recognition to first impressions, and from the brain's 'default mode' to a host of mental disorders.

Hierarchical error processing: Different errors, different systems

Error processing during motor control involves the evaluation of "high-level" errors (i.e., failures to meet a system goal) by a frontal system involving anterior cingulate cortex and the evaluation of "low-level" errors (i.e., discrepancies between actual and desired motor commands) by a posterior system involving posterior parietal cortex. We have recently demonstrated that high-level errors committed within the context of a continuous tracking task elicited an error-related negativity (ERN) -- a component of the event-related brain potential (ERP) generated within medial-frontal cortex that is sensitive to error commission. The purpose of the present study was to demonstrate that low-level motor errors do not elicit an ERN, but may instead evoke other ERP components associated with visual processing and online motor control. Participants performed a computer aiming task in which they manipulated a joystick to move a cursor from a start to a target position. On a random subset of trials the target jumped to a new position at movement onset, requiring the participants to modify their current motor command. Further, on one half of these "target perturbation" trials the cursor did not respond to corrective movements of the joystick. Consistent with our previous findings, we found that the uncorrectable errors elicited an ERN. We also found that the target perturbations on both correctable and uncorrectable trials did not elicit an ERN, but rather evoked two other ERP components, the N100 and P300. These results suggest that medial-frontal cortex is insensitive to low-level motor errors, and are in line with a recent theory that holds that the P300 reflects stimulus-response optimization by the impact of locus coeruleus activity on posterior cortex.

Coordinated accumbal dopamine release and neural activity drive goal-directed behavior

Intracranial self-stimulation (ICSS) activates the neural pathways that mediate reward, including dopaminergic terminal areas such as the nucleus accumbens (NAc). However, a direct role of dopamine in ICSS-mediated reward has been questioned. Here, simultaneous voltammetric and electrophysiological recordings from the same electrode reveal that, at certain sites, the onset of anticipatory dopamine surges and changes in neuronal firing patterns during ICSS are coincident, whereas sites lacking dopamine changes also lack patterned firing. Intrashell microinfusion of a D1, but not a D2 receptor antagonist, blocks ICSS. An iontophoresis approach was implemented to explore the effect of dopamine antagonists on firing patterns without altering behavior. Similar to the microinfusion experiments, ICSS-related firing is selectively attenuated following D1 receptor blockade. This work establishes a temporal link between anticipatory rises of dopamine and firing patterns in the NAc shell during ICSS and suggests that they may play a similar role with natural rewards and during drug self-administration.

Neural signature of fictive learning signals in a sequential investment task

Reinforcement learning models now provide principled guides for a wide range of reward learning experiments in animals and humans. One key learning (error) signal in these models is experiential and reports ongoing temporal differences between expected and experienced reward. However, these same abstract learning models also accommodate the existence of another class of learning signal that takes the form of a fictive error encoding ongoing differences between experienced returns and returns that "could-have-been-experienced" if decisions had been different. These observations suggest the hypothesis that, for all real-world learning tasks, one should expect the presence of both experiential and fictive learning signals. Motivated by this possibility, we used a sequential investment game and fMRI to probe ongoing brain responses to both experiential and fictive learning signals generated throughout the game. Using a large cohort of subjects (n = 54), we report that fictive learning signals strongly predict changes in subjects' investment behavior and correlate with fMRI signals measured in dopaminoceptive structures known to be involved in valuation and choice.

From prediction error to psychosis: ketamine as a pharmacological model of delusions

Recent cognitive neuropsychiatric models of psychosis emphasize the role of attentional disturbances and inappropriate incentive learning in the development of delusions. These models highlight a pre-psychotic period in which the patient experiences perceptual and attentional disruptions. Irrelevant details and numerous associations between stimuli, thoughts and percepts are imbued with inappropriate significance and the attempt to rationalize and account for these bizarre experiences results in the formation of delusions. The present paper discusses delusion formation in terms of basic associative learning processes. Such processes are driven by prediction error signals. Prediction error refers to mismatches between an organism's expectation in a given environment and what actually happens and it is signalled by both dopaminergic and glutamatergic mechanisms. Disruption of these neurobiological systems may underlie delusion formation. We review similarities between acute psychosis and the psychotic state induced by the NMDA receptor antagonist drug ketamine, which impacts upon both dopaminergic and glutamatergic function. We conclude by suggesting that ketamine may provide an appropriate model to investigate the formative stages of symptom evolution in schizophrenia, and thereby provide a window into the earliest and otherwise inaccessible aspects of the disease process.

Decoding of dopaminergic mesolimbic activity and depressive behavior

Dopaminergic mesolimbic and mesocortical systems are involved in hedonia and motivation, two core symptoms of depression. However, their role in the pathophysiology of depression and their manipulation to treat depression has received little attention. Previously, we showed decreased limbic dopamine (DA) neurotransmission in an animal model of depression, Flinder sensitive line (FSL) rats. Here we describe a high correlation between phase-space algorithm of bursting-like activity of DA cells in the ventral tegmental area (VTA) and efficiency of DA release in the accumbens. This bursting-like activity of VTA DA cells of FSL rats is characterized by a low dimension complexity. Treatment with the antidepressant desipramine affected both the dimension complexity of cell firing in the VTA and rate of DA release in the accumbens, as well as alleviating depressive-like behavior. Our findings indicate the potential usefulness of monitoring limbic dopaminergic dynamics in combination with non-linear analysis. Decoding the functionality of the dopaminergic system may help in development of future antidepressant drugs.

Basal Ganglia Mechanisms of Reward-Oriented Eye Movement

Expectation of reward facilitates motor behaviors that enable the animal to approach a location in space where the reward is expected. It is now known that the same expectation of reward profoundly modifies sensory, motor, and cognitive information processing in the brain. However, it is still unclear which brain regions are responsible for causing the reward-approaching behavior. One candidate is the dorsal striatum where cortical and dopaminergic inputs converge. We tested this hypothesis by injecting dopamine antagonists into the caudate nucleus (CD) while the monkey was performing a saccade task with a position-dependent asymmetric reward schedule. We previously had shown that: (1) serial GABAergic connections from the CD to the superior colliculus (SC) via the substantia nigra pars reticulata (SNr) exert powerful control over the initiation of saccadic eye movement and (2) these GABAergic neurons encode target position and are strongly influenced by expected reward, while dopaminergic neurons in the substantia nigra pars compacta (SNc) encode only reward-related information. Before injections of dopamine antagonists the latencies of saccades to a given target were shorter when the saccades were followed by a large reward than when they were followed by a small reward. After injections of dopamine D1 receptor antagonist the reward-dependent latency bias became smaller. This was due to an increase in saccade latency on large-reward trials. After injections of D2 antagonist the latency bias became larger, largely due to an increase in saccade latency on small-reward trials. These results indicate that: (1) dopamine-dependent information processing in the CD is necessary for the reward-dependent modulation of saccadic eye movement and (2) D1 and D2 receptors play differential roles depending on the positive and negative reward outcomes.

Receptor theory and biological constraints on value

Modern economic theories of value derive from expected utility theory. Behavioral evidence points strongly toward departures from linear value weighting, which has given rise to alternative formulations that include prospect theory and rank-dependent utility theory. Many of the nonlinear forms for value assumed by these theories can be derived from the assumption that value is signaled by neurotransmitters in the brain, which obey simple laws of molecular movement. From the laws of mass action and receptor occupancy, we show how behaviorally observed forms of nonlinear value functions can arise.