Dopaminergic reward signals selectively decrease fMRI activity in primate visual cortex

Models of Acetylcholine and Dopamine Signals Differentially Improve Neural Representations

Biological and artificial neural networks (ANNs) represent input signals as patterns of neural activity. In biology, neuromodulators can trigger important reorganizations of these neural representations. For instance, pairing a stimulus with the release of either acetylcholine (ACh) or dopamine (DA) evokes long lasting increases in the responses of neurons to the paired stimulus. The functional roles of ACh and DA in rearranging representations remain largely unknown. Here, we address this question using a Hebbian-learning neural network model. Our aim is both to gain a functional understanding of ACh and DA transmission in shaping biological representations and to explore neuromodulator-inspired learning rules for ANNs. We model the effects of ACh and DA on synaptic plasticity and confirm that stimuli coinciding with greater neuromodulator activation are over represented in the network. We then simulate the physiological release schedules of ACh and DA. We measure the impact of neuromodulator release on the network's representation and on its performance on a classification task. We find that ACh and DA trigger distinct changes in neural representations that both improve performance. The putative ACh signal redistributes neural preferences so that more neurons encode stimulus classes that are challenging for the network. The putative DA signal adapts synaptic weights so that they better match the classes of the task at hand. Our model thus offers a functional explanation for the effects of ACh and DA on cortical representations. Additionally, our learning algorithm yields performances comparable to those of state-of-the-art optimisation methods in multi-layer perceptrons while requiring weaker supervision signals and interacting with synaptically-local weight updates.

Reward Selectively Modulates the Lingering Neural Representation of Recently Attended Objects in Natural Scenes

Theories of reinforcement learning and approach behavior suggest that reward can increase the perceptual salience of environmental stimuli, ensuring that potential predictors of outcome are noticed in the future. However, outcome commonly follows visual processing of the environment, occurring even when potential reward cues have long disappeared. How can reward feedback retroactively cause now-absent stimuli to become attention-drawing in the future? One possibility is that reward and attention interact to prime lingering visual representations of attended stimuli that sustain through the interval separating stimulus and outcome. Here, we test this idea using multivariate pattern analysis of fMRI data collected from male and female humans. While in the scanner, participants searched for examples of target categories in briefly presented pictures of cityscapes and landscapes. Correct task performance was followed by reward feedback that could randomly have either high or low magnitude. Analysis showed that high-magnitude reward feedback boosted the lingering representation of target categories while reducing the representation of nontarget categories. The magnitude of this effect in each participant predicted the behavioral impact of reward on search performance in subsequent trials. Other analyses show that sensitivity to reward-as expressed in a personality questionnaire and in reactivity to reward feedback in the dopaminergic midbrain-predicted reward-elicited variance in lingering target and nontarget representations. Credit for rewarding outcome thus appears to be assigned to the target representation, causing the visual system to become sensitized for similar objects in the future.SIGNIFICANCE STATEMENT How do reward-predictive visual stimuli become salient and attention-drawing? In the real world, reward cues precede outcome and reward is commonly received long after potential predictors have disappeared. How can the representation of environmental stimuli be affected by outcome that occurs later in time? Here, we show that reward acts on lingering representations of environmental stimuli that sustain through the interval between stimulus and outcome. Using naturalistic scene stimuli and multivariate pattern analysis of fMRI data, we show that reward boosts the representation of attended objects and reduces the representation of unattended objects. This interaction of attention and reward processing acts to prime vision for stimuli that may serve to predict outcome.

On the feature specificity of value-driven attention

When an otherwise inconspicuous stimulus is learned to predict a reward, this stimulus will automatically capture visual attention. This learned attentional bias is not specific to the precise object previously associated with reward, but can be observed for different stimuli that share a defining feature with the reward cue. Under certain circumstances, value-driven attentional biases can even transfer to new contexts in which the reward cues were not previously experienced, and can also be evident for different exemplars of a stimulus category, suggesting some degree of tolerance in the scope of the underlying bias. Whether a match to a reward-predictive feature is necessary to support value-driven attention, or whether similar-looking features also receive some degree of elevated priority following associative reward learning, remains an open question. Here, I examine the impact of learned associations between reward and red- and green-colored stimuli on the processing of other colors. The findings show that even though other colors experienced during training were non-predictive with respect to reward, the speed with which targets possessing these colors were identified in a subsequent test phase was affected by their similarity to the high-value color. Thus, value-driven attentional biases for stimulus features are imprecise, as would be predicted by a sensory gain model of value-driven attention.

Reward processing in the value-driven attention network: reward signals tracking cue identity and location

Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value.

How Does Awareness Modulate Goal-Directed and Stimulus-Driven Shifts of Attention Triggered by Value Learning?

In order to behave adaptively, attention can be directed in space either voluntarily (i.e., endogenously) according to strategic goals, or involuntarily (i.e., exogenously) through reflexive capture by salient or novel events. The emotional or motivational value of stimuli can also strongly influence attentional orienting. However, little is known about how reward-related effects compete or interact with endogenous and exogenous attention mechanisms, particularly outside of awareness. Here we developed a visual search paradigm to study subliminal value-based attentional orienting. We systematically manipulated goal-directed or stimulus-driven attentional orienting and examined whether an irrelevant, but previously rewarded stimulus could compete with both types of spatial attention during search. Critically, reward was learned without conscious awareness in a preceding phase where one among several visual symbols was consistently paired with a subliminal monetary reinforcement cue. Our results demonstrated that symbols previously associated with a monetary reward received higher attentional priority in the subsequent visual search task, even though these stimuli and reward were no longer task-relevant, and despite reward being unconsciously acquired. Thus, motivational processes operating independent of conscious awareness may provide powerful influences on mechanisms of attentional selection, which could mitigate both stimulus-driven and goal-directed shifts of attention.

Baseline Perfusion Alterations Due to Acute Application of Quetiapine and Pramipexole in Healthy Adults

BACKGROUND

The dopaminergic system is implicated in many mental processes and neuropsychiatric disorders. Pharmacologically, drugs with dopamine receptor antagonistic and agonistic effects are used, but their effects on functional brain metabolism are not well known.

METHODS

In this randomized crossover, placebo-controlled, and rater-blinded study, 25 healthy adults received an acute dose placebo substance (starch), quetiapine (dopamine receptor antagonist), or pramipexole (dopamine agonist of the nonergoline class) 1 hour before the experiment. Background-suppressed 2D pseudo-continuous arterial spin labeling was used to examine whole-brain baseline cerebral blood flow differences induced by the 3 substances.

RESULTS

We found that quetiapine reduced perfusion in the occipital (early visual areas) and bilateral cerebellar cortex relative to placebo. In contrast, quetiapine enhanced cerebral blood flow (relative to placebo) in the striatal system (putamen and caudate nucleus) but also in the supplementary motor area, insular-, prefrontal- as well as in the pre- and postcentral cortex. Pramipexole increased cerebral blood flow compared with placebo in the caudate nucleus, putamen, middle frontal, supplementary motor area, and brainstem (substantia nigra), but reduced cerebral blood flow in the posterior thalamus, cerebellum, and visual areas. Pramipexole administration resulted in stronger cerebral blood flow relative to quetiapine in the hypothalamus, cerebellum, and substantia nigra.

CONCLUSIONS

Our results indicate that quetiapine and pramipexole differentially modulate regional baseline cerebral blood flow. Both substances act on the dopaminergic system, although they affect distinct regions. Quetiapine altered dopaminergic function in frontal, striatal, and motor regions. In contrast, pramipexole affected cerebral blood flow of the nigrostriatal (striatum and substantia nigra) dopaminergic, but less the fronto-insular system.

How motivation and reward learning modulate selective attention

Motivational stimuli such as rewards elicit adaptive responses and influence various cognitive functions. Notably, increasing evidence suggests that stimuli with particular motivational values can strongly shape perception and attention. These effects resemble both selective top-down and stimulus-driven attentional orienting, as they depend on internal states but arise without conscious will, yet they seem to reflect attentional systems that are functionally and anatomically distinct from those classically associated with frontoparietal cortical networks in the brain. Recent research in human and nonhuman primates has begun to reveal how reward can bias attentional selection, and where within the cognitive system the signals providing attentional priority are generated. This review aims at describing the different mechanisms sustaining motivational attention, their impact on different behavioral tasks, and current knowledge concerning the neural networks governing the integration of motivational influences on attentional behavior.

Gray matter changes related to medication overuse in patients with chronic migraine

Objective

The objective of this article is to investigate the neurological substrates associated with medication overuse (MO) in patients with chronic migraine (CM).

Methods

We recruited age- and sex-matched CM patients with MO (CMwMO), CM patients without MO (CMwoMO), and healthy controls (HCs). Magnetic resonance T1-weighted images were processed by voxel-based morphometry, and the findings were correlated with clinical variables and treatment responses.

Results

A total of 66 patients with CM (half with MO) and 33 HCs completed the study. Patients with CMwMO compared to the patients with CMwoMO showed gray matter volume (GMV) decrease in the orbitofrontal cortex and left middle occipital gyrus as well as GMV increase in the left temporal pole/parahippocampus. The GMV changes explained 31.1% variance of the analgesics use frequency. The patients who responded to treatment had greater GMV in the orbitofrontal cortex ( p = 0.028). Patients with CM (with and without MO), compared with HCs, had decreased GMV at multiple brain areas including the frontal, temporal and occipital lobes, precuneus and cerebellum.

Conclusions

Our study showed GMV changes in CMwMO patients compared to the CMwoMO patients. These three cerebral regions accounted for significant variance in analgesics use frequency. Moreover, the GMV of the orbitofrontal cortex was predictive of the response to MO treatments.

Preclinical Magnetic Resonance Imaging and Spectroscopy Studies of Memory, Aging, and Cognitive Decline

Neuroimaging provides for non-invasive evaluation of brain structure and activity and has been employed to suggest possible mechanisms for cognitive aging in humans. However, these imaging procedures have limits in terms of defining cellular and molecular mechanisms. In contrast, investigations of cognitive aging in animal models have mostly utilized techniques that have offered insight on synaptic, cellular, genetic, and epigenetic mechanisms affecting memory. Studies employing magnetic resonance imaging and spectroscopy (MRI and MRS, respectively) in animal models have emerged as an integrative set of techniques bridging localized cellular/molecular phenomenon and broader in vivo neural network alterations. MRI methods are remarkably suited to longitudinal tracking of cognitive function over extended periods permitting examination of the trajectory of structural or activity related changes. Combined with molecular and electrophysiological tools to selectively drive activity within specific brain regions, recent studies have begun to unlock the meaning of fMRI signals in terms of the role of neural plasticity and types of neural activity that generate the signals. The techniques provide a unique opportunity to causally determine how memory-relevant synaptic activity is processed and how memories may be distributed or reconsolidated over time. The present review summarizes research employing animal MRI and MRS in the study of brain function, structure, and biochemistry, with a particular focus on age-related cognitive decline.

Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.

Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior

Motivation for reward drives adaptive behaviors, whereas impairment of reward perception and experience (anhedonia) can contribute to psychiatric diseases, including depression and schizophrenia. We sought to test the hypothesis that the medial prefrontal cortex (mPFC) controls interactions among specific subcortical regions that govern hedonic responses. By using optogenetic functional magnetic resonance imaging to locally manipulate but globally visualize neural activity in rats, we found that dopamine neuron stimulation drives striatal activity, whereas locally increased mPFC excitability reduces this striatal response and inhibits the behavioral drive for dopaminergic stimulation. This chronic mPFC overactivity also stably suppresses natural reward-motivated behaviors and induces specific new brainwide functional interactions, which predict the degree of anhedonia in individuals. These findings describe a mechanism by which mPFC modulates expression of reward-seeking behavior, by regulating the dynamical interactions between specific distant subcortical regions.

Dopaminergic impact on local and global cortical circuit processing during learning

We have learned to detect, predict and behaviorally respond to important changes in our environment on short and longer time scales. Therefore, brains of humans and higher animals build upon a perceptual and semantic salience stored in their memories mainly generated by associative reinforcement learning. Functionally, the brain needs to extract and amplify a small number of features of sensory input with behavioral relevance to a particular situation in order to guide behavior. In this review, I argue that dopamine action, particularly in sensory cortex, orchestrates layer-dependent local and long-range cortical circuits integrating sensory associated bottom-up and semantically relevant top-down information, respectively. Available evidence reveals that dopamine thereby controls both the selection of perceptually or semantically salient signals as well as feedback processing from higher-order areas in the brain. Sensory cortical dopamine thereby governs the integration of selected sensory information within a behavioral context. This review proposes that dopamine enfolds this function by temporally distinct actions on particular layer-dependent local and global cortical circuits underlying the integration of sensory, and non-sensory cognitive and behavioral variables.

The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications.

Reward breaks through center-surround inhibition via anterior insula

Focusing attention on a target creates a center-surround inhibition such that distractors located close to the target do not capture attention. Recent research showed that a distractor can break through this surround inhibition when associated with reward. However, the brain basis for this reward-based attention is unclear. In this fMRI study, we presented a distractor associated with high or low reward at different distances from the target. Behaviorally the low-reward distractor did not capture attention and thus did not cause interference, whereas the high-reward distractor captured attention only when located near the target. Neural activity in extrastriate cortex mirrored the behavioral pattern. A comparison between the high-reward and the low-reward distractors presented near the target (i.e., reward-based attention) and a comparison between the high-reward distractors located near and far from the target (i.e., spatial attention) revealed a common frontoparietal network, including inferior frontal gyrus and inferior parietal sulcus as well as the visual cortex. Reward-based attention specifically activated the anterior insula (AI). Dynamic causal modelling showed that reward modulated the connectivity from AI to the frontoparietal network but not the connectivity from the frontoparietal network to the visual cortex. Across participants, the reward-based attentional effect could be predicted both by the activity in AI and by the changes of spontaneous functional connectivity between AI and ventral striatum before and after reward association. These results suggest that AI encodes reward-based salience and projects it to the stimulus-driven attentional network, which enables the reward-associated distractor to break through the surround inhibition in the visual cortex.

Reward modulates the effect of visual cortical microstimulation on perceptual decisions

Effective perceptual decisions rely upon combining sensory information with knowledge of the rewards available for different choices. However, it is not known where reward signals interact with the multiple stages of the perceptual decision-making pathway and by what mechanisms this may occur. We combined electrical microstimulation of functionally specific groups of neurons in visual area V5/MT with performance-contingent reward manipulation, while monkeys performed a visual discrimination task. Microstimulation was less effective in shifting perceptual choices towards the stimulus preferences of the stimulated neurons when available reward was larger. Psychophysical control experiments showed this result was not explained by a selective change in response strategy on microstimulated trials. A bounded accumulation decision model, applied to analyse behavioural performance, revealed that the interaction of expected reward with microstimulation can be explained if expected reward modulates a sensory representation stage of perceptual decision-making, in addition to the better-known effects at the integration stage.

DOI:http://dx.doi.org/10.7554/eLife.07832.001

Identifying how an object is moving in three-dimensional (3D) space depends upon a brain region known as V5/MT. The neurons that make up area V5/MT form groups that each have a ‘preference’ for a particular direction of movement and a particular 3D depth. If a group of neurons detects its preferred direction of movement and 3D depth, it will become highly active. The brain can assess which groups of neurons are active, in a process known as integration. This information can then be used to work out the object's movement in space.

The process of integration can be influenced by whether a rewarding outcome is expected to result from identifying the 3D movement correctly. This allows the brain to increase its likelihood of success in situations where a large reward is on offer. Until now, it was thought that the activity in area V5/MT, which takes place before integration, was not affected by the likelihood of receiving a reward.

As well as being ‘naturally’ stimulated by moving objects, the V5/MT neurons can also be ‘artificially’ activated by a technique called microstimulation, which uses a tiny electrode to electrically stimulate groups of neurons. Microstimulation can bias visual perception towards the movement and 3D depth ‘preference’ of the artificially activated neurons. If the V5/MT neurons do receive information about potential rewards from other areas of the brain, we would expect rewards to affect naturally and artificially stimulated neural activity in different ways. On the other hand, if the V5/MT neurons do not receive any information about reward, then it will not matter whether their activity is natural or artificial; the signal that they produce will be the same.

Cicmil et al. gave two monkeys a task in which they could receive rewards for correctly identifying a three-dimensional cylinder's direction of rotation, and applied microstimulation to specific groups of V5/MT neurons on some of the trials. When a larger reward was available, microstimulation was less able to bias the monkeys' choices about the rotation direction of the 3D cylinders.

Overall, Cicmil et al.'s results suggest that the V5/MT neurons are able to incorporate information about reward, before integration occurs. The next step will be to record the activity of area V5/MT to investigate exactly how this happens.

DOI:http://dx.doi.org/10.7554/eLife.07832.002

How does reward compete with goal-directed and stimulus-driven shifts of attention?

In order to behave adaptively, attention can be directed in space either voluntarily (i.e. endogenously) according to strategic goals, or involuntarily (i.e. exogenously) through reflexive capture by salient or novel events. The emotional or motivational values of stimuli can also influence attentional orienting. However, little is known about how reward-related effects compete or interact with endogenous and exogenous attention mechanisms. Here we designed a visual search paradigm in which goal-driven and stimulus-driven shifts of attention were manipulated by classic spatial cueing procedures, while an irrelevant, but previously rewarded stimulus also appeared as a distractor and hence competed with both types of spatial attention during search. Our results demonstrated that stimuli previously associated with a high monetary reward received higher attentional priority in the subsequent visual search task, even though these stimuli and reward were no longer task-relevant, mitigating the attentional orienting induced by both endogenous and exogenous cues.

Dual mechanisms governing reward-driven perceptual learning

In this review, we explore how reward signals shape perceptual learning in animals and humans. Perceptual learning is the well-established phenomenon by which extensive practice elicits selective improvement in one’s perceptual discrimination of basic visual features, such as oriented lines or moving stimuli. While perceptual learning has long been thought to rely on ‘top-down’ processes, such as attention and decision-making, a wave of recent findings suggests that these higher-level processes are, in fact, not necessary.  Rather, these recent findings indicate that reward signals alone, in the absence of the contribution of higher-level cognitive processes, are sufficient to drive the benefits of perceptual learning. Here, we will review the literature tying reward signals to perceptual learning. Based on these findings, we propose dual underlying mechanisms that give rise to perceptual learning: one mechanism that operates ‘automatically’ and is tied directly to reward signals, and another mechanism that involves more ‘top-down’, goal-directed computations.

Covert shifts of spatial attention in the macaque monkey

In the awake state, shifts of spatial attention alternate with periods of sustained attention at a fixed location or object. Human fMRI experiments revealed the critical role of the superior parietal lobule (SPL) in shifting spatial attention, a finding not predicted by human lesion studies and monkey electrophysiology. To investigate whether a potential homolog of the human SPL shifting region exists in monkeys (Macaca mulatta), we adopted an event-related fMRI paradigm that closely resembled a human experiment (Molenberghs et al., 2007). In this paradigm, a pair of relevant and irrelevant shapes was continuously present on the horizontal meridian. Subjects had to covertly detect a dimming of the relevant shape while ignoring the irrelevant dimmings. The events of interest consisted of the replacement of one stimulus pair by the next. During shift but not stay events, the relevant shape of the new pair appeared at the contralateral position relative to the previous one. Spatial shifting events activated parietal areas V6/V6A and medial intraparietal area, caudo-dorsal visual areas, the most posterior portion of the superior temporal sulcus, and several smaller frontal areas. These areas were not activated during passive stimulation with the same sensory stimuli. During stay events, strong direction-sensitive attention signals were observed in a distributed set of contralateral visual, temporal, parietal, and lateral prefrontal areas, the vast majority overlapping with the sensory stimulus representation. We suggest medial intraparietal area and V6/V6A as functional counterparts of human SPL because they contained the most widespread shift signals in the absence of contralateral stay activity, resembling the functional characteristics of the human SPL shifting area.

Differential effects of dopamine-directed treatments on cognition

Dopamine, a prominent neuromodulator, is implicated in many neuropsychiatric disorders. It has wide-ranging effects on both cortical and subcortical brain regions and on many types of cognitive tasks that rely on a variety of different learning and memory systems. As neuroscience and behavioral evidence for the existence of multiple memory systems and their corresponding neural networks accumulated, so did the notion that dopamine's role is markedly different depending on which memory system is engaged. As a result, dopamine-directed treatments will have different effects on different types of cognitive behaviors. To predict what these effects will be, it is critical to understand: which memory system is mediating the behavior; the neural basis of the mediating memory system; the nature of the dopamine projections into that system; and the time course of dopamine after its release into the relevant brain regions. Consideration of these questions leads to different predictions for how changes in brain dopamine levels will affect automatic behaviors and behaviors mediated by declarative, procedural, and perceptual representation memory systems.

Reward activates stimulus-specific and task-dependent representations in visual association cortices

Humans reliably learn which actions lead to rewards. One prominent question is how credit is assigned to environmental stimuli that are acted upon. Recent functional magnetic resonance imaging (fMRI) studies have provided evidence that representations of rewarded stimuli are activated upon reward delivery, providing possible eligibility traces for credit assignment. Our study sought evidence of postreward activation in sensory cortices satisfying two conditions of instrumental learning: postreward activity should reflect the stimulus category that preceded reward (stimulus specificity), and should occur only if the stimulus was acted on to obtain reward (task dependency). Our experiment implemented two tasks in the fMRI scanner. The first was a perceptual decision-making task on degraded face and house stimuli. Stimulus specificity was evident as rewards activated the sensory cortices associated with face versus house perception more strongly after face versus house decisions, respectively, particularly in the fusiform face area. Stimulus specificity was further evident in a psychophysiological interaction analysis wherein face-sensitive areas correlated with nucleus accumbens activity after face-decision rewards, whereas house-sensitive areas correlated with nucleus accumbens activity after house-decision rewards. The second task required participants to make an instructed response. The criterion of task dependency was fulfilled as rewards after face versus house responses activated the respective association cortices to a larger degree when faces and houses were relevant to the performed task. Our study is the first to show that postreward sensory cortex activity meets these two key criteria of credit assignment, and does so independently from bottom-up perceptual processing.

Monkey cortex through fMRI glasses

In 1998 several groups reported the feasibility of fMRI experiments in monkeys, with the goal to bridge the gap between invasive nonhuman primate studies and human functional imaging. These studies yielded critical insights in the neuronal underpinnings of the BOLD signal. Furthermore, the technology has been successful in guiding electrophysiological recordings and identifying focal perturbation targets. Finally, invaluable information was obtained concerning human brain evolution. We here provide a comprehensive overview of awake monkey fMRI studies mainly confined to the visual system. We review the latest insights about the topographic organization of monkey visual cortex and discuss the spatial relationships between retinotopy and category- and feature-selective clusters. We briefly discuss the functional layout of parietal and frontal cortex and continue with a summary of some fascinating functional and effective connectivity studies. Finally, we review recent comparative fMRI experiments and speculate about the future of nonhuman primate imaging.

Tonic effects of the dopaminergic ventral midbrain on the auditory cortex of awake macaque monkeys

This study shows that ongoing electrical stimulation of the dopaminergic ventral midbrain can modify neuronal activity in the auditory cortex of awake primates for several seconds. This was reflected in a decrease of the spontaneous firing and in a bidirectional modification of the power of auditory evoked potentials. We consider that both effects are due to an increase in the dopamine tone in auditory cortex induced by the electrical stimulation. Thus, the dopaminergic ventral midbrain may contribute to the tonic activity in auditory cortex that has been proposed to be involved in associating events of auditory tasks (Brosch et al. Hear Res 271:66-73, 2011) and may modulate the signal-to-noise ratio of the responses to auditory stimuli.

Value is in the eye of the beholder: early visual cortex codes monetary value of objects during a diverted attention task

A central concern in the study of learning and decision-making is the identification of neural signals associated with the values of choice alternatives. An important factor in understanding the neural correlates of value is the representation of the object itself, separate from the act of choosing. Is it the case that the representation of an object within visual areas will change if it is associated with a particular value? We used fMRI adaptation to measure the neural similarity of a set of novel objects before and after participants learned to associate monetary values with the objects. We used a range of both positive and negative values to allow us to distinguish effects of behavioral salience (i.e., large vs. small values) from effects of valence (i.e., positive vs. negative values). During the scanning session, participants made a perceptual judgment unrelated to value. Crucially, the similarity of the visual features of any pair of objects did not predict the similarity of their value, so we could distinguish adaptation effects due to each dimension of similarity. Within early visual areas, we found that value similarity modulated the neural response to the objects after training. These results show that an abstract dimension, in this case, monetary value, modulates neural response to an object in visual areas of the brain even when attention is diverted.

Contributions of cortical feedback to sensory processing in primary visual cortex

Closing the structure-function divide is more challenging in the brain than in any other organ (Lichtman and Denk, 2011). For example, in early visual cortex, feedback projections to V1 can be quantified (e.g., Budd, 1998) but the understanding of feedback function is comparatively rudimentary (Muckli and Petro, 2013). Focusing on the function of feedback, we discuss how textbook descriptions mask the complexity of V1 responses, and how feedback and local activity reflects not only sensory processing but internal brain states.

Lower [3H]Citalopram binding in brain areas related to social cognition in alcoholics

AIMS

In the present study, putative alterations in the serotonin transporter density were evaluated in anterior and posterior insula, posterior cingulate cortex, dorsolateral and dorsomedial prefrontal cortex, hippocampus, parahippocampal gyrus and dorsal raphe nucleus in Cloninger type 1 (n = 9) and type 2 (n = 8) alcoholics and non-alcoholic controls (n = 10).

METHODS

Human whole-hemisphere autoradiography was used to measure [3H]citalopram binding to serotonin transporters in eight brain areas in all post-mortem brains.

RESULTS

Significant differences were observed in the mean [3H]citalopram binding between the study groups, with antisocial type 2 alcoholics showing the lowest binding. Differences between the study groups were prominent in the posterior insula and posterior cingulate cortex, where both alcoholic groups had low [3H]citalopram binding, and in the parahippocampal gyrus where only antisocial type 2 alcoholics had low [3H]citalopram binding when compared with non-alcoholic controls.

CONCLUSION

Although these data are preliminary, and from relatively small diagnostic groups, these results show that alcoholics may have lower serotonergic tone in the brain, thus decreasing social cognition and increasing alcohol-cue reactivity.

Cross-modal effects of value on perceptual acuity and stimulus encoding

Cross-modal interactions are very common in perception. An important feature of many perceptual stimuli is their reward-predicting properties, the utilization of which is essential for adaptive behavior. What is unknown is whether reward associations in one sensory modality influence perception of stimuli in another modality. Here we show that auditory stimuli with high-reward associations increase the sensitivity of visual perception, even when sounds and reward associations are both irrelevant for the visual task. This increased sensitivity correlates with a change in stimulus representation in the visual cortex, indexed by increased multivariate decoding accuracy in simultaneously acquired functional MRI data. Univariate analysis showed that reward associations modulated responses in regions associated with multisensory processing in which the strength of modulation was a better predictor of the magnitude of the behavioral effect than the modulation in classical reward regions. Our findings demonstrate a value-driven cross-modal interaction that affects perception and stimulus encoding, with a resemblance to well-described modulatory effects of attention. We suggest that multisensory processing areas may mediate the transfer of value signals across senses.

Visual cortex activation to drug cues: a meta-analysis of functional neuroimaging papers in addiction and substance abuse literature

BACKGROUND

Although the visual cortex does not typically receive much attention in addiction literature, neuroimaging studies often report significant activity in visual areas when drug users are exposed to drug cues. The purpose of this meta-analysis was to investigate the frequency with which occipital cortex activity is observed during drug cue exposure and to determine its spatial distribution.

METHODS

A comprehensive literature search was performed of human functional neuroimaging studies of drug cue-reactivity. Fifty-five studies were used to determine the frequency with which clusters of significant visual cortex activity during visual drug cues versus non-drug cues were reported. The spatial distribution of visual cortex activations was determined via activation likelihood estimation (ALE; FDR corrected, p<0.01) in a subset of these studies (n=24).

RESULTS

Eighty-six percent of studies that reported fMRI results for drug versus neutral visual cues within a substance-dependent group showed significant drug-elicited activity in the visual cortex. ALE revealed clusters in the left secondary visual cortex (BA 19) and clusters in the primary visual cortex (BA 17) that were consistently activated by drug cues.

CONCLUSIONS

These data demonstrate that the visual cortex, often overlooked in our discussions of the neural circuitry of addiction, consistently discriminates drug cues from neutral cues in substance dependent populations. While it remains unclear whether drug cue-elicited activation in occipital cortex is related to the rewarding properties of the drug and/or attentional mechanisms, these data support further exploration.

Fast transmission from the dopaminergic ventral midbrain to the sensory cortex of awake primates

Motivated by the increasing evidence that auditory cortex is under control of dopaminergic cell structures of the ventral midbrain, we studied how the ventral tegmental area and substantia nigra affect neuronal activity in auditory cortex. We electrically stimulated 567 deep brain sites in total within and in the vicinity of the two dopaminergic ventral midbrain structures and at the same time, recorded local field potentials and neuronal discharges in cortex. In experiments conducted on three awake macaque monkeys, we found that electrical stimulation of the dopaminergic ventral midbrain resulted in short-latency (~35 ms) phasic activations in all cortical layers of auditory cortex. We were also able to demonstrate similar activations in secondary somatosensory cortex and superior temporal polysensory cortex. The electrically evoked responses in these parts of sensory cortex were similar to those previously described for prefrontal cortex. Moreover, these phasic responses could be reversibly altered by the dopamine D1-receptor antagonist SCH23390 for several tens of minutes. Thus, we speculate that the dopaminergic ventral midbrain exerts a temporally precise, phasic influence on sensory cortex using fast-acting non-dopaminergic transmitters and that their effects are modulated by dopamine on a longer timescale. Our findings suggest that some of the information carried by the neuronal discharges in the dopaminergic ventral midbrain, such as the motivational value or the motivational salience, is transmitted to auditory cortex and other parts of sensory cortex. The mesocortical pathway may thus contribute to the representation of non-auditory events in the auditory cortex and to its associative functions.

Reward-priming of location in visual search

Existing visual search research has demonstrated that the receipt of reward will be beneficial for subsequent perceptual and attentional processing of features that have characterized targets, but detrimental for processing of features that have characterized irrelevant distractors. Here we report a similar effect of reward on location. Observers completed a visual search task in which they selected a target, ignored a salient distractor, and received random-magnitude reward for correct performance. Results show that when target selection garnered rewarding outcome attention is subsequently a.) primed to return to the target location, and b.) biased away from the location that was occupied by the salient, task-irrelevant distractor. These results suggest that in addition to priming features, reward acts to guide visual search by priming contextual locations of visual stimuli.

Role of the primate ventral tegmental area in reinforcement and motivation

Monkey electrophysiology suggests that the activity of the ventral tegmental area (VTA) helps regulate reinforcement learning and motivated behavior, in part by broadcasting prediction error signals throughout the reward system. However, electrophysiological studies do not allow causal inferences regarding the activity of VTA neurons with respect to these processes because they require artificial manipulation of neuronal firing. Rodent studies fulfilled this requirement by demonstrating that electrical and optogenetic VTA stimulation can induce learning and modulate downstream structures. Still, the primate dopamine system has diverged significantly from that of rodents, exhibiting greatly expanded and uniquely distributed cortical and subcortical innervation patterns. Here, we bridge the gap between rodent perturbation studies and monkey electrophysiology using chronic electrical microstimulation of macaque VTA (VTA-EM). VTA-EM was found to reinforce cue selection in an operant task and to motivate future cue selection using a Pavlovian paradigm. Moreover, by combining VTA-EM with concurrent fMRI, we demonstrated that VTA-EM increased fMRI activity throughout most of the dopaminergic reward system. These results establish a causative role for primate VTA in regulating stimulus-specific reinforcement and motivation as well as in modulating activity throughout the reward system.

The role of prefrontal catecholamines in attention and working memory

While much progress has been made in identifying the brain regions and neurochemical systems involved in the cognitive processes disrupted in mental illnesses, to date, the level of detail at which neurobiologists can describe the chain of events giving rise to cognitive functions is very rudimentary. Much of the intense interest in understanding cognitive functions is motivated by the hope that it might be possible to understand these complex functions at the level of neurons and neural circuits. Here, we review the current state of the literature regarding how modulations in catecholamine levels within the prefrontal cortex (PFC) alter the neuronal and behavioral correlates of cognitive functions, particularly attention and working memory.

Ventral tegmental area activation promotes firing precision and strength through circuit inhibition in the primary auditory cortex

The activation of the ventral tegmental area (VTA) can rebuild the tonotopic representation in the primary auditory cortex (A1), but the cellular mechanisms remain largely unknown. Here, we investigated the firing patterns and membrane potential dynamics of neurons in A1 under the influence of VTA activation using in vivo intracellular recording. We found that VTA activation can significantly reduce the variability of sound evoked responses and promote the firing precision and strength of A1 neurons. Furthermore, the compressed response window was caused by an early hyperpolarization as a result of enhanced circuit inhibition. Our study suggested a possible mechanism of how the reward system affects information processing in sensory cortex: VTA activation strengthens cortical inhibition, which shortens the response window of post-synaptic cortical neurons and further promotes the precision and strength of neuronal activity.

Reward- and attention-related biasing of sensory selection in visual cortex

Attention to task-relevant features leads to a biasing of sensory selection in extrastriate cortex. Features signaling reward seem to produce a similar bias, but how modulatory effects due to reward and attention relate to each other is largely unexplored. To address this issue, it is critical to separate top-down settings defining reward relevance from those defining attention. To this end, we used a visual search paradigm in which the target's definition (attention to color) was dissociated from reward relevance by delivering monetary reward on search frames where a certain task-irrelevant color was combined with the target-defining color to form the target object. We assessed the state of neural biasing for the attended and reward-relevant color by analyzing the neuromagnetic brain response to asynchronously presented irrelevant distractor probes drawn in the target-defining color, the reward-relevant color, and a completely irrelevant color as a reference. We observed that for the prospect of moderate rewards, the target-defining color but not the reward-relevant color produced a selective enhancement of the neuromagnetic response between 180 and 280 msec in ventral extrastriate visual cortex. Increasing reward prospect caused a delayed attenuation (220-250 msec) of the response to reward probes, which followed a prior (160-180 msec) response enhancement in dorsal ACC. Notably, shorter latency responses in dorsal ACC were associated with stronger attenuation in extrastriate visual cortex. Finally, an analysis of the brain response to the search frames revealed that the presence of the reward-relevant color in search distractors elicited an enhanced response that was abolished after increasing reward size. The present data together indicate that when top-down definitions of reward relevance and attention are separated, the behavioral significance of reward-associated features is still rapidly coded in higher-level cortex areas, thereby commanding effective top-down inhibitory control to counter a selection bias for those features in extrastriate visual cortex.

Neural dynamics of reward probability coding: a Magnetoencephalographic study in humans

Prediction of future rewards and discrepancy between actual and expected outcomes (prediction error) are crucial signals for adaptive behavior. In humans, a number of fMRI studies demonstrated that reward probability modulates these two signals in a large brain network. Yet, the spatio-temporal dynamics underlying the neural coding of reward probability remains unknown. Here, using magnetoencephalography, we investigated the neural dynamics of prediction and reward prediction error computations while subjects learned to associate cues of slot machines with monetary rewards with different probabilities. We showed that event-related magnetic fields (ERFs) arising from the visual cortex coded the expected reward value 155 ms after the cue, demonstrating that reward value signals emerge early in the visual stream. Moreover, a prediction error was reflected in ERF peaking 300 ms after the rewarded outcome and showing decreasing amplitude with higher reward probability. This prediction error signal was generated in a network including the anterior and posterior cingulate cortex. These findings pinpoint the spatio-temporal characteristics underlying reward probability coding. Together, our results provide insights into the neural dynamics underlying the ability to learn probabilistic stimuli-reward contingencies.

What about the "Self" is Processed in the Posterior Cingulate Cortex?

In the past decade, neuroimaging research has begun to identify key brain regions involved in self-referential processing, most consistently midline structures such as the posterior cingulate cortex (PCC). The majority of studies have employed cognitive tasks such as judgment about trait adjectives or mind wandering, that have been associated with increased PCC activity. Conversely, tasks that share an element of present-centered attention (being “on task”), ranging from working memory to meditation, have been associated with decreased PCC activity. Given the complexity of cognitive processes that likely contribute to these tasks, the specific contribution of the PCC to self-related processes still remains unknown. Building on this prior literature, recent studies have employed sampling methods that more precisely link subjective experience to brain activity, such as real-time fMRI neurofeedback. This recent work suggests that PCC activity may represent a sub-component cognitive process of self-reference – “getting caught up in” one’s experience. For example, getting caught up in a drug craving or a particular viewpoint. In this paper, we will review evidence across a number of different domains of cognitive neuroscience that converges in activation and deactivation of the PCC including recent neurophenomenological studies of PCC activity using real-time fMRI neurofeedback.

Reward predictions bias attentional selection

Attention selects stimuli for perceptual and cognitive processing according to an adaptive selection schedule. It has long been known that attention selects stimuli that are task relevant or perceptually salient. Recent evidence has shown that stimuli previously associated with reward persistently capture attention involuntarily, even when they are no longer associated with reward. Here we examine whether the capture of attention by previously reward-associated stimuli is modulated by the processing of current but unrelated rewards. Participants learned to associate two color stimuli with different amounts of reward during a training phase. In a subsequent test phase, these previously rewarded color stimuli were occasionally presented as to-be-ignored distractors while participants performed visual search for each of two differentially rewarded shape-defined targets. The results reveal that attentional capture by formerly rewarded distractors was the largest when both recently received and currently expected reward were the highest in the test phase, even though such rewards were unrelated to the color distractors. Our findings support a model in which value-driven attentional biases acquired through reward learning are maintained via the cognitive mechanisms involved in predicting future rewards.