The human amygdala encodes value and space during decision making

Representation of Anticipated Rewards and Punishments in the Human Brain

Subjective value is a core concept in neuroeconomics, serving as the basis for decision making. Despite the extensive literature on the neural encoding of subjective reward value in humans, the neural representation of punishment value remains relatively understudied. This review synthesizes current knowledge on the neural representation of reward value, including methodologies, involved brain regions, and the concept of a common currency representation of diverse reward types in decision-making and learning processes. We then critically examine existing research on the neural representation of punishment value, highlighting conceptual and methodological challenges in human studies and insights gained from animal research. Finally, we explore how individual differences in reward and punishment processing may be linked to various mental illnesses, with a focus on stress-related psychopathologies. This review advocates for the integration of both rewards and punishments within value-based decision-making and learning frameworks, leveraging insights from cross-species studies and utilizing ecological gamified paradigms to reflect real-life scenarios.

On the Relationship between Value- and Threat-Driven Attentional Capture and Approach-Avoidance Biases

Reward learning and aversive conditioning have consequences for attentional selection, such that stimuli that come to signal reward and threat bias attention regardless of their valence. Appetitive and aversive stimuli have distinctive influences on response selection, such that they activate an approach and an avoidance response, respectively. However, whether the involuntary influence of reward- and threat-history-laden stimuli extends to the manner in which a response is directed remains unclear. Using a feedback-joystick task and a manikin task, which are common paradigms for examining valence-action bias, we demonstrate that reward- and threat-signalling stimuli do not modulate response selection. Stimuli that came to signal reward and threat via training biased attention and invigorated action in general, but they did not facilitate an approach and avoidance response, respectively. We conclude that attention can be biased towards a stimulus as a function of its prior association with reward or aversive outcomes without necessarily influencing approach vs. avoidance tendencies, such that the mechanisms underlying the involuntary control of attention and behaviour evoked by valent stimuli can be decoupled.

Contributions of human amygdala nuclei to resting-state networks

The amygdala is a brain region with a complex internal structure that is associated with psychiatric disease. Methodological limitations have complicated the study of the internal structure of the amygdala in humans. In the current study we examined the functional connectivity between nine amygdaloid nuclei and existing resting-state networks using a high spatial-resolution fMRI dataset. Using data-driven analysis techniques we found that there were three main clusters inside the amygdala that correlated with the somatomotor, ventral attention and default mode networks. In addition, we found that each resting-state networks depended on a specific configuration of amygdaloid nuclei. Finally, we found that co-activity in the cortical-nucleus increased with the severity of self-rated fear in participants. These results highlight the complex nature of amygdaloid connectivity that is not confined to traditional large-scale divisions, implicates specific configurations of nuclei with certain resting-state networks and highlights the potential clinical relevance of the cortical-nucleus in future studies of the human amygdala.

Pupil dilation predicts modulation of direct gaze on action value calculations

Perceiving direct gaze facilitates social cognition and behaviour. We hypothesized that direct gaze modulates decision-making, particularly calculations of action values. To test our hypothesis, we used the reinforcement learning paradigm in situations with or without direct gaze. Forty adults were recruited and participated in pupil size measurements and a two-armed bandit task. The task was conducted with 70% and 30% reward probabilities for each option. During the task, a female showing the Direct Gaze (DG) or Closed Eyes (CE) condition was presented from the start of each trial. The results showed that behavioural bias to choices with 70% reward probability increased more in the DG condition than in the CE condition and the expected reward value. This bias to choices with 70% reward in the DG condition was predicted by pupil dilation to DG. These results suggest that participants over-evaluated the expected reward value in the DG condition, and this DG effect may be related to subjective expectations of rewarding events indexed by pupil dilations. It is considered that perceiving direct gaze is a driver of reward expectations that modulate action value calculations and then cognitive processing and behaviours are facilitated.

Motivational Salience Guides Attention to Valuable and Threatening Stimuli: Evidence from Behavior and Functional Magnetic Resonance Imaging

Rewarding and aversive outcomes have opposing effects on behavior, facilitating approach and avoidance, although we need to accurately anticipate each type of outcome to behave effectively. Attention is biased toward stimuli that have been learned to predict either type of outcome, and it remains an open question whether such orienting is driven by separate systems for value- and threat-based orienting or whether there exists a common underlying mechanism of attentional control driven by motivational salience. Here, we provide a direct comparison of the neural correlates of value- and threat-based attentional capture after associative learning. Across multiple measures of behavior and brain activation, our findings overwhelmingly support a motivational salience account of the control of attention. We conclude that there exists a core mechanism of experience-dependent attentional control driven by motivational salience and that prior characterizations of attention as being value driven or supporting threat monitoring need to be revisited.

Arousal-Biased Competition Explains Reduced Distraction by Reward Cues under Threat

Anxiety is an adaptive neural state that promotes rapid responses under heightened vigilance when survival is threatened. Anxiety has consistently been found to potentiate the attentional processing of physically salient stimuli. However, a recent study demonstrated that a threat manipulation reduces attentional capture by reward-associated stimuli, suggesting a more complex relationship between anxiety and the control of attention. The mechanisms by which threat can reduce the distracting quality of stimuli are unknown. In this study, using functional magnetic resonance imaging (fMRI) on human subjects, we examined the neural correlates of attention to previously reward-associated stimuli with and without the threat of unpredictable electric shock. We replicate enhanced distractor-evoked activity throughout the value-driven attention network (VDAN) in addition to enhanced stimulus-evoked activity generally under threat. Importantly, these two factors interacted such that the representation of previously reward-associated distractors was particularly pronounced under threat. Our results from neuroimaging fit well with the principle of arousal-biased competition (ABC), although such effects are typically associated with behavioral measures of increased attention to stimuli that already possess elevated attentional priority. The findings of our study suggest that ABC can be leveraged to support more efficient ignoring of reward cues, revealing new insights into the functional significance of ABC as a mechanism of attentional control, and provide a mechanistic explanation of how threat reduces attention to irrelevant reward information.

Threat reduces value-driven but not salience-driven attentional capture

What we direct our attention to is strongly influenced by both bottom-up and top-down processes. Moreover, the control of attention is biased by prior learning, such that attention is automatically captured by stimuli previously associated with either reward or threat. It is unknown whether value-oriented and threat-oriented mechanisms of selective information processing function independently of one another, or whether they interact with each other in the selection process. Here, we introduced the threat of electric shock into the value-driven attentional capture paradigm to examine whether the experience of threat influences the attention capturing quality of previously reward-associated stimuli. The results showed that value-driven attentional capture was blunted by the experience of threat. This contrasts with previous reports of threat potentiating attentional capture by physically salient stimuli, which we replicate here. Our findings demonstrate that threat selectively interferes with value-based but not salience-based attentional priority, consistent with a competitive relationship between value-based and threat-based information processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Neurobiology of value-driven attention

What we pay attention to is influenced by reward learning. Converging evidence points to the idea that associative reward learning changes how visual stimuli are processed in the brain, rendering learned reward cues difficult to ignore. Behavioral evidence distinguishes value-driven attention from other established control mechanisms, suggesting a distinct underlying neurobiological process. Recently, studies have begun to explore the neural substrates of this value-driven attention mechanism. Here, I review the progress that has been made in this area, and synthesize the findings to provide an integrative account of the neurobiology of value-driven attention. The proposed account can explain both attentional capture by previously rewarded targets and the modulatory effect of reward on priming, as well as the decoupling of reward history and prior task relevance in value-driven attention.

The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task

The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.

Interaction of Instrumental and Goal-Directed Learning Modulates Prediction Error Representations in the Ventral Striatum

Goal-directed and instrumental learning are both important controllers of human behavior. Learning about which stimulus event occurs in the environment and the reward associated with them allows humans to seek out the most valuable stimulus and move through the environment in a goal-directed manner. Stimulus-response associations are characteristic of instrumental learning, whereas response-outcome associations are the hallmark of goal-directed learning. Here we provide behavioral, computational, and neuroimaging results from a novel task in which stimulus-response and response-outcome associations are learned simultaneously but dominate behavior at different stages of the experiment. We found that prediction error representations in the ventral striatum depend on which type of learning dominates. Furthermore, the amygdala tracks the time-dependent weighting of stimulus-response versus response-outcome learning. Our findings suggest that the goal-directed and instrumental controllers dynamically engage the ventral striatum in representing prediction errors whenever one of them is dominating choice behavior.

SIGNIFICANCE STATEMENT

Converging evidence in human neuroimaging studies has shown that the reward prediction errors are correlated with activity in the ventral striatum. Our results demonstrate that this region is simultaneously correlated with a stimulus prediction error. Furthermore, the learning system that is currently dominating behavioral choice dynamically engages the ventral striatum for computing its prediction errors. This demonstrates that the prediction error representations are highly dynamic and influenced by various experimental context. This finding points to a general role of the ventral striatum in detecting expectancy violations and encoding error signals regardless of the specific nature of the reinforcer itself.

What is abnormal about addiction-related attentional biases?

BACKGROUND

The phenotype of addiction includes prominent attentional biases for drug cues, which play a role in motivating drug-seeking behavior and contribute to relapse. In a separate line of research, arbitrary stimuli have been shown to automatically capture attention when previously associated with reward in non-clinical samples.

METHODS AND RESULTS

Here, I argue that these two attentional biases reflect the same cognitive process. I outline five characteristics that exemplify attentional biases for drug cues: resistant to conflicting goals, robust to extinction, linked to dorsal striatal dopamine and to biases in approach behavior, and can distinguish between individuals with and without a history of drug dependence. I then go on to describe how attentional biases for arbitrary reward-associated stimuli share all of these features, and conclude by arguing that the attentional components of addiction reflect a normal cognitive process that promotes reward-seeking behavior.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications.

Neural activity in monkey amygdala during performance of a multisensory operant task

In this paper, we study the potential involvement of monkey amygdala in the evaluation of value encoding of visual and auditive stimuli associated with reward or no reward. We recorded the activity of 93 extracellular neurons from the monkey right amygdala, while performing a multisensory operant task. The activity of 78 task-related neurons was studied. Of these, 13 neurons (16%) responded to the value of visual stimuli, 22 neurons (28%) responded after the presentation of visual stimuli, 22 neurons (28%) showed an inhibition around the lever-pressing and were classified as action related neurons and 22 neurons (28%) responded after reward delivery. These findings suggest that neurons in the amygdala play a role in encoding value and processing visual information, participate in motor regulation and are sensitive to reward. The activity of these neurons did not change in the evaluation of auditive stimuli. These data support the hypothesis that amygdala neurons are specific to each sensory modality and that different groups of amygdala neurons process visual and auditive information.

Amygdala neural activity reflects spatial attention towards stimuli promising reward or threatening punishment

Humans and other animals routinely identify and attend to sensory stimuli so as to rapidly acquire rewards or avoid aversive experiences. Emotional arousal, a process mediated by the amygdala, can enhance attention to stimuli in a non-spatial manner. However, amygdala neural activity was recently shown to encode spatial information about reward-predictive stimuli, and to correlate with spatial attention allocation. If representing the motivational significance of sensory stimuli within a spatial framework reflects a general principle of amygdala function, then spatially selective neural responses should also be elicited by sensory stimuli threatening aversive events. Recordings from amygdala neurons were therefore obtained while monkeys directed spatial attention towards stimuli promising reward or threatening punishment. Neural responses encoded spatial information similarly for stimuli associated with both valences of reinforcement, and responses reflected spatial attention allocation. The amygdala therefore may act to enhance spatial attention to sensory stimuli associated with rewarding or aversive experiences.