Value-based attentional capture influences context-dependent decision-making

Reward-modulated attention deployment is driven by suppression, not attentional capture

One driving factor for attention deployment towards a stimulus is its associated value due to previous experience and learning history. Previous visual search studies found that when looking for a target, distractors associated with higher reward produce more interference (e.g., longer response times). The present study investigated the neural mechanism of such value-driven attention deployment. Specifically, we were interested in which of the three attention sub-processes are responsible for the interference that was repeatedly observed behaviorally: enhancement of relevant information, attentional capture by irrelevant information, or suppression of irrelevant information. We replicated earlier findings showing longer response times and lower accuracy when a target competed with a high-reward compared to a low-reward distractor. We also found a spatial gradient of interference: behavioral performance dropped with increasing proximity to the target. This gradient was steeper for high- than low-reward distractors. Event-related potentials of the EEG signal showed the reason for the reward-induced attentional bias: High-reward distractors required more suppression than low-reward distractors as evident in larger Pd components. This effect was only found for distractors near targets, showing the additional filtering needs required for competing stimuli in close proximity. As a result, fewer attentional resources can be distributed to the target when it competes with a high-reward distractor, as evident in a smaller target-N2pc amplitude. The distractor-N2pc, indicative of attentional capture, was neither affected by distance nor reward, showing that attentional capture alone cannot explain interference by stimuli of high value. In sum our results show that the higher need for suppression of high-value stimuli contributes to reward-modulated attention deployment and increased suppression can prevent attentional capture of high-value stimuli.

Conflicting Sensory Information Sharpens the Neural Representations of Early Selective Visuospatial Attention

Adaptive behaviors require the ability to resolve conflicting information caused by the processing of incompatible sensory inputs. Prominent theories of attention have posited that early selective attention helps mitigate cognitive interference caused by conflicting sensory information by facilitating the processing of task-relevant sensory inputs and filtering out behaviorally irrelevant information. Surprisingly, many recent studies that investigated the role of early selective attention on conflict mitigation have failed to provide positive evidence. Here, we examined changes in the selectivity of early visuospatial attention in male and female human subjects performing an attention-cueing Eriksen flanker task, where they discriminated the shape of a visual target surrounded by congruent or incongruent distractors. We used the inverted encoding model to reconstruct spatial representations of visual selective attention from the topographical patterns of amplitude modulations in alpha band oscillations in scalp EEG (∼8-12 Hz). We found that the fidelity of the alpha-based spatial reconstruction was significantly higher in the incongruent compared with the congruent condition. Importantly, these conflict-related modulations in the reconstruction fidelity occurred at a much earlier time window than those of the lateralized posterior event-related potentials associated with target selection and distractor suppression processes, as well as conflict-related modulations in the frontocentral negative-going wave and midline-frontal theta oscillations (∼3-7 Hz), thought to track executive control functions. Taken together, our data suggest that conflict resolution is supported by the cascade of neural processes underlying early selective visuospatial attention and frontal executive functions that unfold over time.

Influence of reinforcement learning on the inhibitory control of Internet gaming disorder

Reward processing dysfunction and inhibition control deficiency have been observed in Internet gaming disorder (IGD). However, it is still unclear whether the previous reinforcement learning depends on reward/punishment feedback influences on the cognitive inhibitory control of IGD. This study compared the differences between an IGD group and healthy people without game experiences in the probability selection task and the subsequent stop signal task by the method of behavioral experiments, in order to explore whether the reward learning ability is impaired in the IGD group. We also discuss the influence of previous reward learning on subsequent inhibition control. The results showed that (1) during the reward learning phase, the IGD group's accuracy was significantly lower than that of the control group; (2) compared with the control group, the IGD group's reaction times were longer in the transfer phase; (3) for no-go trials of the inhibitory control phase after reward learning, the accuracy of the reward-related stimulation in the IGD group was lower than that of punishment-related or neutral stimulation, but there was no significant difference among the three conditions in the control group. These findings indicated that the reinforcement learning ability of the IGD group was impaired, which further caused the abnormal response to reinforcement stimuli.

Value-modulated attentional capture is augmented by win-related sensory cues

Attentional prioritisation of stimuli in the environment plays an important role in overt choice. Previous research shows that prioritisation is influenced by the magnitude of paired rewards, in that stimuli signalling high-value rewards are more likely to capture attention than stimuli signalling low-value rewards; and this attentional bias has been proposed to play a role in addictive and compulsive behaviours. A separate line of research has shown that win-related sensory cues can bias overt choices. However, the role that these cues play in attentional selection is yet to be investigated. Participants in this study completed a visual search task in which they responded to a target shape in order to earn reward. The colour of a distractor signalled the magnitude of reward and type of feedback on each trial. Participants were slower to respond to the target when the distractor signalled high reward compared to when the distractor signalled low reward, suggesting that the high-reward distractors had increased attentional priority. Critically, the magnitude of this reward-related attentional bias was further increased for a high-reward distractor with post-trial feedback accompanied by win-related sensory cues. Participants also demonstrated an overt choice preference for the distractor that was associated with win-related sensory cues. These findings demonstrate that stimuli paired with win-related sensory cues are prioritised by the attention system over stimuli with equivalent physical salience and learned value. This attentional prioritisation may have downstream implications for overt choices, especially in gambling contexts where win-related sensory cues are common.

Multimodal investigations of emotional face processing and social trait judgment of faces

Faces are among the most important visual stimuli that humans perceive in everyday life. While extensive literature has examined emotional processing and social evaluations of faces, most studies have examined either topic using unimodal approaches. In this review, we promote the use of multimodal cognitive neuroscience approaches to study these processes, using two lines of research as examples: ambiguity in facial expressions of emotion and social trait judgment of faces. In the first set of studies, we identified an event-related potential that signals emotion ambiguity using electroencephalography and we found convergent neural responses to emotion ambiguity using functional neuroimaging and single-neuron recordings. In the second set of studies, we discuss how different neuroimaging and personality-dimensional approaches together provide new insights into social trait judgments of faces. In both sets of studies, we provide an in-depth comparison between neurotypicals and people with autism spectrum disorder. We offer a computational account for the behavioral and neural markers of the different facial processing between the two groups. Finally, we suggest new practices for studying the emotional processing and social evaluations of faces. All data discussed in the case studies of this review are publicly available.

Dissociable Neural Mechanisms Underlie the Effects of Attention on Visual Appearance and Response Bias

A prominent theoretical framework spanning philosophy, psychology, and neuroscience holds that selective attention penetrates early stages of perceptual processing to alter the subjective visual experience of behaviorally relevant stimuli. For example, searching for a red apple at the grocery store might make the relevant color appear brighter and more saturated compared with seeing the exact same red apple while searching for a yellow banana. In contrast, recent proposals argue that data supporting attention-related changes in appearance reflect decision- and motor-level response biases without concurrent changes in perceptual experience. Here, we tested these accounts by evaluating attentional modulations of EEG responses recorded from male and female human subjects while they compared the perceived contrast of attended and unattended visual stimuli rendered at different levels of physical contrast. We found that attention enhanced the amplitude of the P1 component, an early evoked potential measured over visual cortex. A linking model based on signal detection theory suggests that response gain modulations of the P1 component track attention-induced changes in perceived contrast as measured with behavior. In contrast, attentional cues induced changes in the baseline amplitude of posterior alpha band oscillations (∼9-12 Hz), an effect that best accounts for cue-induced response biases, particularly when no stimuli are presented or when competing stimuli are similar and decisional uncertainty is high. The observation of dissociable neural markers that are linked to changes in subjective appearance and response bias supports a more unified theoretical account and demonstrates an approach to isolate subjective aspects of selective information processing.SIGNIFICANCE STATEMENT Does attention alter visual appearance, or does it simply induce response bias? In the present study, we examined these competing accounts using EEG and linking models based on signal detection theory. We found that response gain modulations of the visually evoked P1 component best accounted for attention-induced changes in visual appearance. In contrast, cue-induced baseline shifts in alpha band activity better explained response biases. Together, these results suggest that attention concurrently impacts visual appearance and response bias, and that these processes can be experimentally isolated.

Value-based cognition and drug dependency

Value-based decision-making is thought to play an important role in drug dependency. Achieving elevated levels of euphoria or ameliorating dysphoria/pain may motivate goal-directed drug consumption in both drug-naïve and long-time users. In other words, drugs become viewed as the preferred means of attaining a desired internal state. The bias towards choosing drugs may affect one's cognition. Observed biases in learning, attention and memory systems within the brain gradually focus one's cognitive functions towards drugs and related cues to the exclusion of other stimuli. In this narrative review, the effects of drug use on learning, attention and memory are discussed with a particular focus on changes across brain-wide functional networks and the subsequent impact on behaviour. These cognitive changes are then incorporated into the cycle of addiction, an established model outlining the transition from casual drug use to chronic dependency. If drug use results in the elevated salience of drugs and their cues, the studies highlighted in this review strongly suggest that this salience biases cognitive systems towards the motivated pursuit of addictive drugs. This bias is observed throughout the cycle of addiction, possibly contributing to the persistent hold that addictive drugs have over the dependent. Taken together, the excessive valuation of drugs as the preferred means of achieving a desired internal state affects more than just decision-making, but also learning, attentional and mnemonic systems. This eventually narrows the focus of one's thoughts towards the pursuit and consumption of addictive drugs.

The past, present, and future of selection history

The last ten years of attention research have witnessed a revolution, replacing a theoretical dichotomy (top-down vs. bottom-up control) with a trichotomy (biased by current goals, physical salience, and selection history). This third new mechanism of attentional control, selection history, is multifaceted. Some aspects of selection history must be learned over time whereas others reflect much more transient influences. A variety of different learning experiences can shape the attention system, including reward, aversive outcomes, past experience searching for a target, target‒non-target relations, and more. In this review, we provide an overview of the historical forces that led to the proposal of selection history as a distinct mechanism of attentional control. We then propose a formal definition of selection history, with concrete criteria, and identify different components of experience-driven attention that fit within this definition. The bulk of the review is devoted to exploring how these different components relate to one another. We conclude by proposing an integrative account of selection history centered on underlying themes that emerge from our review.

Individual Alpha Frequency Determines the Impact of Bottom-Up Drive on Visual Processing

Endogenous alpha oscillations propagate from higher-order to early visual cortical regions, consistent with the observed modulation of these oscillations by top-down factors. However, bottom-up manipulations also influence alpha oscillations, and little is known about how these top-down and bottom-up processes interact to impact behavior. To address this, participants performed a detection task while viewing a stimulus flickering at multiple alpha band frequencies. Bottom-up drive at a participant's endogenous alpha frequency either impaired or enhanced perception, depending on the frequency, but not amplitude, of their endogenous alpha oscillation. Fast alpha drive impaired perceptual performance in participants with faster endogenous alpha oscillations, while participants with slower oscillations displayed enhanced performance. This interaction was reflected in slower endogenous oscillatory dynamics in participants with fast alpha oscillations and more rapid dynamics in participants with slow endogenous oscillations when receiving high-frequency bottom-up drive. This central tendency may suggest that driving visual circuits at alpha band frequencies that are away from the peak alpha frequency improves perception through dynamical interactions with the endogenous oscillation. As such, studies that causally manipulate neural oscillations via exogenous stimulation should carefully consider interacting effects of bottom-up drive and endogenous oscillations on behavior.

The influence of associative reward learning on motor inhibition

Stimuli that predict a rewarding outcome can cause difficulties to inhibit unfavourable behaviour. Research suggests that this is also the case for stimuli with a history of reward extending these effects on action control to situations, where reward is no longer accessible. We expand this line of research by investigating if previously reward-predictive stimuli promote behavioural activation and impair motor inhibition in a second unrelated task. In two experiments participants were trained to associate colours with a monetary reward or neutral feedback. Afterwards participants performed a cued go/no-go task, where cues appeared in the colours previously associated with feedback during training. In both experiments training resulted in faster responses in rewarded trials providing evidence of a value-driven response bias as long as reward was accessible. However, stimuli with a history of reward did not interfere with goal-directed action and inhibition in a subsequent task after removal of the reward incentives. While the first experiment was not conclusive regarding an impact of reward-associated cues on response inhibition, the second experiment, validated by Bayesian statistics, clearly questioned an effect of reward history on inhibitory control. This stands in contrast to earlier findings suggesting that the effect of reward history on subsequent action control is not as consistent as previously assumed. Our results show that participants are able to overcome influences from Pavlovian learning in a simple inhibition task. We discuss our findings with respect to features of the experimental design which may help or complicate overcoming behavioural biases induced by reward history.

Reward history impacts attentional orienting and inhibitory control on untrained tasks

It has been robustly shown that stimuli with reward history receive attentional priority. However, the majority of this research tests reward history effects on attentional bias using similar tasks for both the reward learning phase and the unrewarded testing phase, which limits our understanding of how the effects of reward history generalize beyond the trained tasks and mental sets. Across two new experiments, the current study addresses these issues by first associating reward with a stimulus in a visual search paradigm, and then testing value-driven effects of that stimulus in untrained and unrewarded tasks, including a cueing paradigm, a go/no-go task, and a delay discounting task. Results of Experiment 1 demonstrate that history of reward association in a visual search task generalizes to value-driven attentional bias in a different attention paradigm (i.e., cueing), indicating these effects are indeed attributable to imbued value that can transfer to other tasks beyond that in which the reward was trained. The results of Experiment 2 demonstrate that in addition to eliciting attentional orienting on untrained tasks, reward history can lead to better inhibitory control in the go/no-go task. We find no evidence for reward history effects in the delay discounting task. Together, these experiments demonstrate that when the reward association task is in the attention domain, reward history modulates attentional priority, and this effect generalizes to untrained and unrewarded tasks that utilize both spatial and nonspatial attention.

Incentive value and spatial certainty combine additively to determine visual priorities

How does the brain combine information predictive of the value of a visually guided task (incentive value) with information predictive of where task-relevant stimuli may occur (spatial certainty)? Human behavioural evidence indicates that these two predictions may be combined additively to bias visual selection (Additive Hypothesis), whereas neuroeconomic studies posit that they may be multiplicatively combined (Expected Value Hypothesis). We sought to adjudicate between these two alternatives. Participants viewed two coloured placeholders that specified the potential value of correctly identifying an imminent letter target if it appeared in that placeholder. Then, prior to the target’s presentation, an endogenous spatial cue was presented indicating the target’s more likely location. Spatial cues were parametrically manipulated with regard to the information gained (in bits). Across two experiments, performance was better for targets appearing in high versus low value placeholders and better when targets appeared in validly cued locations. Interestingly, as shown with a Bayesian model selection approach, these effects did not interact, clearly supporting the Additive Hypothesis. Even when conditions were adjusted to increase the optimality of a multiplicative operation, support for it remained. These findings refute recent theories that expected value computations are the singular mechanism driving the deployment of endogenous spatial attention. Instead, incentive value and spatial certainty seem to act independently to influence visual selection.

Previously Reward-Associated Stimuli Capture Spatial Attention in the Absence of Changes in the Corresponding Sensory Representations as Measured with MEG

Studies of selective attention typically consider the role of task goals or physical salience, but attention can also be captured by previously reward-associated stimuli, even if they are currently task irrelevant. One theory underlying this value-driven attentional capture (VDAC) is that reward-associated stimulus representations undergo plasticity in sensory cortex, thereby automatically capturing attention during early processing. To test this, we used magnetoencephalography to probe whether stimulus location and identity representations in sensory cortex are modulated by reward learning. We furthermore investigated the time course of these neural effects, and their relationship to behavioral VDAC. Male and female human participants first learned stimulus-reward associations. Next, we measured VDAC in a separate task by presenting these stimuli in the absence of reward contingency and probing their effects on the processing of separate target stimuli presented at different time lags. Using time-resolved multivariate pattern analysis, we found that learned value modulated the spatial selection of previously rewarded stimuli in posterior visual and parietal cortex from ∼260 ms after stimulus onset. This value modulation was related to the strength of participants' behavioral VDAC effect and persisted into subsequent target processing. Importantly, learned value did not influence cortical signatures of early processing (i.e., earlier than ∼200 ms); nor did it influence the decodability of stimulus identity. Our results suggest that VDAC is underpinned by learned value signals that modulate spatial selection throughout posterior visual and parietal cortex. We further suggest that VDAC can occur in the absence of changes in early visual processing in cortex.SIGNIFICANCE STATEMENT Attention is our ability to focus on relevant information at the expense of irrelevant information. It can be affected by previously learned but currently irrelevant stimulus-reward associations, a phenomenon termed "value-driven attentional capture" (VDAC). The neural mechanisms underlying VDAC remain unclear. It has been speculated that reward learning induces visual cortical plasticity, which modulates early visual processing to capture attention. Although we find that learned value modulates spatial signals in visual cortical areas, an effect that correlates with VDAC, we find no relevant signatures of changes in early visual processing in cortex.

Visual target detection in a distracting background relies on neural encoding of both visual targets and background

The ability to detect visual targets in complex background varies across individuals and are affected by factors such as stimulus saliency and top-down attention. Here, we investigated how the saliency of visual background (naturalistic cartoon video vs. blank screen) and top-down attention (single vs. dual tasks) separately affect individual ability to detect visual targets. Behaviorally, we found that target detection accuracy decreased and reaction time elongated when the background was salient or during dual tasking. The EEG response to visual background was recorded using a novel stimulus tagging technique. This response was strongest in occipital electrodes and was sensitive to background saliency but not dual tasking. In contrast, the event-related potential (ERP) evoked by the visual target was strongest in central electrodes, and was affected by both background saliency and dual tasking. With a cartoon background, the EEG responses to visual targets, presented in the central visual field, and the EEG responses to peripheral visual background could both predict individual target detection performance. When these two responses were combined, better prediction was achieved. These results suggest that neural processing of visual targets and background jointly contribute to individual visual target detection performance.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Attention Capture of Non-target Emotional Faces: An Evidence From Reward Learning

The aim of this study was to explore whether reward learning would affect the processing of targets when an emotional stimulus was task irrelevant. In the current study, using a visual search paradigm to establish an association between emotional faces and reward, an emotional face appeared as a task-irrelevant distractor during the test after reward learning, and participants were asked to judge the orientation of a line on the face. In experiment 1, no significant difference was found between the high reward-fear distractor condition and the no reward-neutral condition, but the response times of the high reward-fear condition were significantly longer than those of the low reward-happy condition. In experiment 2, there was no significant difference in participants' performance between high reward-happy and no reward-neutral responses. In addition, response times of the low reward-fear condition wear significantly longer than those of the high reward-happy and no reward-neutral conditions. The results show that reward learning affects attention bias of task-irrelevant emotional faces even when reward is absent. Moreover, the high reward selection history is more effective in weakening the emotional advantage of the processing advantage than the low reward.

Neural underpinnings of value-guided choice during auction tasks: An eye-fixation related potentials study

Values are attributed to goods during free viewing of objects which entails multi- and trans-saccadic cognitive processes. Using electroencephalographic eye-fixation related potentials, the present study investigated how neural signals related to value-guided choice evolved over time when viewing household and office products during an auction task. Participants completed a Becker-DeGroot-Marschak auction task whereby half of the stimuli were presented in either a free or forced bid protocol to obtain willingness-to-pay. Stimuli were assigned to three value categories of low, medium and high value based on subjective willingness-to-pay. Eye fixations were organised into five 800 ms time-bins spanning the objects total viewing time. Independent component analysis was applied to eye-fixation related potentials. One independent component (IC) was found to represent fixations for high value products with increased activation over the left parietal region of the scalp. An IC with a spatial maximum over a fronto-central region of the scalp coded the intermediate values. Finally, one IC displaying activity that extends over the right frontal scalp region responded to intermediate- and low-value items. Each of these components responded early on during viewing an object and remained active over the entire viewing period, both during free and forced bid trials. Results suggest that the subjective value of goods are encoded using sets of brain activation patterns which are tuned to respond uniquely to either low, medium, or high values. Data indicates that the right frontal region of the brain responds to low and the left frontal region to high values. Values of goods are determined at an early point in the decision making process and carried for the duration of the decision period via trans-saccadic processes.

When Conflict Cannot be Avoided: Relative Contributions of Early Selection and Frontal Executive Control in Mitigating Stroop Conflict

When viewing familiar stimuli (e.g., common words), processing is highly automatized such that it can interfere with the processing of incompatible sensory information. At least two mechanisms may help mitigate this interference. Early selection accounts posit that attentional processes filter out distracting sensory information to avoid conflict. Alternatively, late selection accounts hold that all sensory inputs receive full semantic analysis and that frontal executive mechanisms are recruited to resolve conflict. To test how these mechanisms operate to overcome conflict induced by highly automatized processing, we developed a novel version of the color-word Stroop task, where targets and distractors were simultaneously flickered at different frequencies. We measured the quality of early sensory processing by assessing the amplitude of steady-state visually evoked potentials (SSVEPs) elicited by targets and distractors. We also indexed frontal executive processes by assessing changes in frontal theta oscillations induced by color-word incongruency. We found that target- and distractor-related SSVEPs were not modulated by changes in the level of conflict whereas frontal theta activity increased on high compared to low conflict trials. These results suggest that frontal executive processes play a more dominant role in mitigating cognitive interference driven by the automatic tendency to process highly familiar stimuli.

Cost-benefit trade-offs in decision-making and learning

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes.

Value-based decision-making involves trading off the cost associated with an action–such as physical or mental effort–against its expected reward. Although facing conflicts between competing action alternatives is considered aversive and effortful, it remains unclear whether conflict also constitutes a cost in value-based decisions. We tested this hypothesis by combining a classic conflict (flanker) task with a reinforcement-learning task. Results showed that participants learned to maximise their earnings, but were nevertheless biased to follow irrelevant suggestions. Computational model-based analyses showed a greater choice bias with more uncertainty about the best action to make, supporting the hypothesis that the costs linked to conflict management were traded off against expected rewards. We additionally found that learning rates were reduced when following instructions, relative to when choosing freely what to do. Learning was further reduced by conflict between instructions and subjective action values. In short, we found that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and that different types of conflict may have different effects on learning about action outcomes.

Choice-theoretic foundations of the divisive normalization model

Recent advances in neuroscience suggest that a utility-like calculation is involved in how the brain makes choices, and that this calculation may use a computation known as divisive normalization. While this tells us how the brain makes choices, it is not immediately evident why the brain uses this computation or exactly what behavior is consistent with it. In this paper, we address both of these questions by proving a three-way equivalence theorem between the normalization model, an information-processing model, and an axiomatic characterization. The information-processing model views behavior as optimally balancing the expected value of the chosen object against the entropic cost of reducing stochasticity in choice. This provides an optimality rationale for why the brain may have evolved to use normalization-type models. The axiomatic characterization gives a set of testable behavioral statements equivalent to the normalization model. This answers what behavior arises from normalization. Our equivalence result unifies these three models into a single theory that answers the "how", "why", and "what" of choice behavior.

Value-driven attentional capture enhances distractor representations in early visual cortex

When a behaviorally relevant stimulus has been previously associated with reward, behavioral responses are faster and more accurate compared to equally relevant but less valuable stimuli. Conversely, task-irrelevant stimuli that were previously associated with a high reward can capture attention and distract processing away from relevant stimuli (e.g., seeing a chocolate bar in the pantry when you are looking for a nice, healthy apple). Although increasing the value of task-relevant stimuli systematically up-regulates neural responses in early visual cortex to facilitate information processing, it is not clear whether the value of task-irrelevant distractors influences behavior via competition in early visual cortex or via competition at later stages of decision-making and response selection. Here, we measured functional magnetic resonance imaging (fMRI) in human visual cortex while subjects performed a value-based learning task, and we applied a multivariate inverted encoding model (IEM) to assess the fidelity of distractor representations in early visual cortex. We found that the fidelity of neural representations related to task-irrelevant distractors increased when the distractors were previously associated with a high reward. This finding suggests that value-driven attentional capture begins with sensory modulations of distractor representations in early areas of visual cortex.

Electrophysiological indexes of option characteristic processing

Decision making is vital to human behavior and can be divided into multiple stages including option assessment, behavioral output, and feedback evaluation. Studying how people evaluate option characteristics in the option assessment stage would provide important knowledge on human decision making. Using the event-related potential (ERP) method, the present study investigated the neural mechanism of evaluating two types of option characteristics (i.e., reward magnitude and degree of uncertainty) in the temporal dimension. Thirty-five volunteers participated in a monetary gambling task, where they either accepted or rejected gambles. The ERP results showed a double dissociation pattern, with the early P1 component being sensitive to magnitude but insensitive to degree of uncertainty, while both the N2 and P3 components showed the opposite pattern. The results suggest that these two fundamental option features are assessed rapidly and separately in the human brain. Specifically, small magnitude elicited a larger P1 than did large magnitude, indicating that the perceptual and attentional processing of options is modulated by magnitude. Both the N2 and P3 amplitudes evoked by the risky context were larger than those evoked by the ambiguous one, reflecting that more cognitive conflicts and resources are involved in the former condition. Furthermore, the P1, but not the N2 or P3, amplitude was sensitive to decisions, suggesting that early attentional processes may contribute to human decision making. These findings may provide insight into the temporal mechanisms of option characteristic processing.

Value-driven attentional capture is modulated by the contents of working memory: An EEG study

Attention and working memory (WM) have previously been shown to interact closely when sensory information is being maintained. However, when non-sensory information is maintained in WM, the relationship between WM and sensory attention may be less strong. In the current study, we used electroencephalography to evaluate whether value-driven attentional capture (i.e., allocation of attention to a task-irrelevant feature previously associated with a reward) and its effects on either sensory or non-sensory WM performance might be greater than the effects of salient, non-reward-associated stimuli. In a training phase, 19 participants learned to associate a color with reward. Then, participants were presented with squares and encoded their locations into WM. Participants were instructed to convert the spatial locations either to another type of sensory representation or to an abstract, relational type of representation. During the WM delay period, task-irrelevant distractors, either previously-rewarded or non-rewarded, were presented, with a novel color distractor in the other hemifield. The results revealed lower alpha power and larger N2pc amplitude over posterior electrode sides contralateral to the previously rewarded color, compared to ipsilateral. These effects were mainly found during relational WM, compared to sensory WM, and only for the previously rewarded distractor color, compared to a previous non-rewarded target color or novel color. These effects were associated with modulations of WM performance. These results appear to reflect less capture of attention during maintenance of specific location information, and suggest that value-driven attentional capture can be mitigated as a function of the type of information maintained in WM.

Punishment-related memory-guided attention: Neural dynamics of perceptual modulation

Remembering the outcomes of past experiences allows us to generate future expectations and shape selection in the long-term. A growing number of studies has shown that learned positive reward values impact spatial memory-based attentional biases on perception. However, whether memory-driven attentional biases extend to punishment-related values has received comparatively less attention. Here, we manipulated whether recent spatial contextual memories became associated with successful avoidance of punishment (potential monetary loss). Behavioral and electrophysiological measures were collected from 27 participants during a subsequent memory-based attention task, in which we tested for the effect of punishment avoidance associations. Punishment avoidance significantly amplified effects of spatial contextual memories on visual search processes within natural scenes. Compared to non-associated scenes, contextual memories paired with punishment avoidance lead to faster responses to targets presented at remembered locations. Event-related potentials elicited by target stimuli revealed that acquired motivational value of specific spatial locations, by virtue of their association with past avoidance of punishment, dynamically affected neural signatures of early visual processing (indexed by larger P1 and earlier N1 potentials) and target selection (as indicated by reduced N2pc potentials). The present results extend our understanding of how memory, attention, and punishment-related mechanisms interact to optimize perceptual decision in real world environments.

Value-based attentional capture affects multi-alternative decision making

Humans and other animals often violate economic principles when choosing between multiple alternatives, but the underlying neurocognitive mechanisms remain elusive. A robust finding is that adding a third option can alter the relative preference for the original alternatives, but studies disagree on whether the third option's value decreases or increases accuracy. To shed light on this controversy, we used and extended the paradigm of one study reporting a positive effect. However, our four experiments with 147 human participants and a reanalysis of the original data revealed that the positive effect is neither replicable nor reproducible. In contrast, our behavioral and eye-tracking results are best explained by assuming that the third option's value captures attention and thereby impedes accuracy. We propose a computational model that accounts for the complex interplay of value, attention, and choice. Our theory explains how choice sets and environments influence the neurocognitive processes of multi-alternative decision making.

Having More Choices Changes How Human Observers Weight Stable Sensory Evidence

Decision-making becomes slower when more choices are available. Existing models attribute this slowing to poor sensory processing, to attenuated rates of sensory evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). However, studies have not isolated the effects of having more choices on sensory and decision-related processes from changes in task difficulty and divided attention. Here, we controlled task difficulty while independently manipulating the distribution of attention and the number of choices available to male and female human observers. We used EEG to measure steady-state visually evoked potentials (SSVEPs) and a frontal late positive deflection (LPD), EEG markers of sensory and postsensory decision-related processes, respectively. We found that dividing attention decreased SSVEP and LPD amplitudes, consistent with dampened sensory responses and slower rates of evidence accumulation, respectively. In contrast, having more choices did not alter SSVEP amplitude and led to a larger LPD. These results suggest that having more options largely spares early sensory processing and slows down decision-making via a selective increase in decision thresholds.SIGNIFICANCE STATEMENT When more choices are available, decision-making becomes slower. We tested whether this phenomenon is due to poor sensory processing, to reduced rates of evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). We measured choice modulations of sensory and decision-related neural responses using EEG. We also minimized potential confounds from changes in the distribution of attention and task difficulty, which often covary with having more choices. Dividing attention reduced the activity levels of both sensory and decision-related responses. However, having more choices did not change sensory processing and led to larger decision-related responses. These results suggest that having more choices spares sensory processing and selectively increases decision thresholds.

Multiple timescales of normalized value coding underlie adaptive choice behavior

Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.

Expectations Do Not Alter Early Sensory Processing during Perceptual Decision-Making

Two factors play important roles in shaping perception: the allocation of selective attention to behaviorally relevant sensory features, and prior expectations about regularities in the environment. Signal detection theory proposes distinct roles of attention and expectation on decision-making such that attention modulates early sensory processing, whereas expectation influences the selection and execution of motor responses. Challenging this classic framework, recent studies suggest that expectations about sensory regularities enhance the encoding and accumulation of sensory evidence during decision-making. However, it is possible, that these findings reflect well documented attentional modulations in visual cortex. Here, we tested this framework in a group of male and female human participants by examining how expectations about stimulus features (orientation and color) and expectations about motor responses impacted electroencephalography (EEG) markers of early sensory processing and the accumulation of sensory evidence during decision-making (the early visual negative potential and the centro-parietal positive potential, respectively). We first demonstrate that these markers are sensitive to changes in the amount of sensory evidence in the display. Then we show, counter to recent findings, that neither marker is modulated by either feature or motor expectations, despite a robust effect of expectations on behavior. Instead, violating expectations about likely sensory features and motor responses impacts posterior alpha and frontal theta oscillations, signals thought to index overall processing time and cognitive conflict. These findings are inconsistent with recent theoretical accounts and suggest instead that expectations primarily influence decisions by modulating post-perceptual stages of information processing.SIGNIFICANCE STATEMENT Expectations about likely features or motor responses play an important role in shaping behavior. Classic theoretical frameworks posit that expectations modulate decision-making by biasing late stages of decision-making including the selection and execution of motor responses. In contrast, recent accounts suggest that expectations also modulate decisions by improving the quality of early sensory processing. However, these effects could instead reflect the influence of selective attention. Here we examine the effect of expectations about sensory features and motor responses on a set of electroencephalography (EEG) markers that index early sensory processing and later post-perceptual processing. Counter to recent empirical results, expectations have little effect on early sensory processing but instead modulate EEG markers of time-on-task and cognitive conflict.

Fluctuations in instantaneous frequency predict alpha amplitude during visual perception

Rhythmic neural activity in the alpha band (8-13 Hz) is thought to have an important role in the selective processing of visual information. Typically, modulations in alpha amplitude and instantaneous frequency are thought to reflect independent mechanisms impacting dissociable aspects of visual information processing. However, in complex systems with interacting oscillators such as the brain, amplitude and frequency are mathematically dependent. Here, we record electroencephalography in human subjects and show that both alpha amplitude and instantaneous frequency predict behavioral performance in the same visual discrimination task. Consistent with a model of coupled oscillators, we show that fluctuations in instantaneous frequency predict alpha amplitude on a single trial basis, empirically demonstrating that these metrics are not independent. This interdependence suggests that changes in amplitude and instantaneous frequency reflect a common change in the excitatory and inhibitory neural activity that regulates alpha oscillations and visual information processing.

A Neural Signature Encoding Decisions under Perceptual Ambiguity

People often make perceptual decisions with ambiguous information, but it remains unclear whether the brain has a common neural substrate that encodes various forms of perceptual ambiguity. Here, we used three types of perceptually ambiguous stimuli as well as task instructions to examine the neural basis for both stimulus-driven and task-driven perceptual ambiguity. We identified a neural signature, the late positive potential (LPP), that encoded a general form of stimulus-driven perceptual ambiguity. In addition to stimulus-driven ambiguity, the LPP was also modulated by ambiguity in task instructions. To further specify the functional role of the LPP and elucidate the relationship between stimulus ambiguity, behavioral response, and the LPP, we employed regression models and found that the LPP was specifically associated with response latency and confidence rating, suggesting that the LPP encoded decisions under perceptual ambiguity. Finally, direct behavioral ratings of stimulus and task ambiguity confirmed our neurophysiological findings, which could not be attributed to differences in eye movements either. Together, our findings argue for a common neural signature that encodes decisions under perceptual ambiguity but is subject to the modulation of task ambiguity. Our results represent an essential first step toward a complete neural understanding of human perceptual decision making.

Changes of Attention during Value-Based Reversal Learning Are Tracked by N2pc and Feedback-Related Negativity

Previously learned reward values can have a pronounced impact, behaviorally and neurophysiologically, on the allocation of selective attention. All else constant, stimuli previously associated with a high value gain stronger attentional prioritization than stimuli previously associated with a low value. The N2pc, an ERP component indicative of attentional target selection, has been shown to reflect aspects of this prioritization, by changes of mean amplitudes closely corresponding to selective enhancement of high value target processing and suppression of high value distractor processing. What has remained unclear so far is whether the N2pc also reflects the flexible and repeated behavioral adjustments needed in a volatile task environment, in which the values of stimuli are reversed often and unannounced. Using a value-based reversal learning task, we found evidence that the N2pc amplitude flexibly and reversibly tracks value-based choices during the learning of reward associated stimulus colors. Specifically, successful learning of current value-contingencies was associated with reduced N2pc amplitudes, and this effect was more apparent for distractor processing, compared with target processing. In addition, following a value reversal the feedback related negativity(FRN), an ERP component that reflects feedback processing, was amplified and co-occurred with increased N2pc amplitudes in trials following low-value feedback. Importantly, participants that showed the greatest adjustment in N2pc amplitudes based on feedback were also the most efficient learners. These results allow further insight into how changes in attentional prioritization in an uncertain and volatile environment support flexible adjustments of behavior.

Funny money: the attentional role of monetary feedback detached from expected value

Stimuli associated with monetary reward can become powerful cues that effectively capture visual attention. We examined whether such value-driven attentional capture can be induced with monetary feedback in the absence of an expected cash payout. To this end, we implemented images of U.S. dollar bills as reward feedback. Participants knew in advance that they would not receive any money based on their performance. Our reward stimuli-$5 and $20 bill images-were thus dissociated from any practical utility. Strikingly, we observed a reliable attentional capture effect for the mere images of bills. Moreover, this finding generalized to Monopoly money. In two control experiments, we found no evidence in favor of nominal or symbolic monetary value. Hence, we claim that bill images are special monetary representations, such that there are strong associations between the defining visual features of bills and reward, probably due to a lifelong learning history. Together, we show that the motivation to earn cash plays a minor role when it comes to monetary rewards, while bill-defining visual features seem to be sufficient. These findings have the potential to influence human factor applications, such as gamification, and can be extended to novel value systems, such as the electronic cash Bitcoin being developed for use in mobile banking. Finally, our procedure represents a proof of concept on how images of money can be used to conserve expenditures in the experimental context.

Differences in Social Decision-Making between Proposers and Responders during the Ultimatum Game: An EEG Study

The Ultimatum Game (UG) is a typical paradigm to investigate social decision-making. Although the behavior of humans in this task is already well established, the underlying brain processes remain poorly understood. Previous investigations using event-related potentials (ERPs) revealed three major components related to cognitive processes in participants engaged in the responder condition, the early ERP component P2, the feedback-related negativity (FRN) and a late positive wave (late positive component, LPC). However, the comparison of the ERP waveforms between the responder and proposer conditions has never been studied. Therefore, to investigate condition-related electrophysiological changes, we applied the UG paradigm and compared parameters of the P2, LPC and FRN components in twenty healthy participants. For the responder condition, we found a significantly decreased amplitude and delayed latency for the P2 component, whereas the mean amplitudes of the LPC and FRN increased compared to the proposer condition. Additionally, the proposer condition elicited an early component consisting of a negative deflection around 190 ms, in the upward slope of the P2, probably as a result of early conflict-related processing. Using independent component analysis (ICA), we extracted one functional component time-locked to this deflection, and with source reconstruction (LAURA) we found the anterior cingulate cortex (ACC) as one of the underlying sources. Overall, our findings indicate that intensity and time-course of neuronal systems engaged in the decision-making processes diverge between both UG conditions, suggesting differential cognitive processes. Understanding the electrophysiological bases of decision-making and social interactions in controls could be useful to further detect which steps are impaired in psychiatric patients in their ability to attribute mental states (such as beliefs, intents, or desires) to oneself and others. This ability is called mentalizing (also known as theory of mind).

Two different mechanisms support selective attention at different phases of training

Selective attention supports the prioritized processing of relevant sensory information to facilitate goal-directed behavior. Studies in human subjects demonstrate that attentional gain of cortical responses can sufficiently account for attention-related improvements in behavior. On the other hand, studies using highly trained nonhuman primates suggest that reductions in neural noise can better explain attentional facilitation of behavior. Given the importance of selective information processing in nearly all domains of cognition, we sought to reconcile these competing accounts by testing the hypothesis that extensive behavioral training alters the neural mechanisms that support selective attention. We tested this hypothesis using electroencephalography (EEG) to measure stimulus-evoked visual responses from human subjects while they performed a selective spatial attention task over the course of ~1 month. Early in training, spatial attention led to an increase in the gain of stimulus-evoked visual responses. Gain was apparent within ~100 ms of stimulus onset, and a quantitative model based on signal detection theory (SDT) successfully linked the magnitude of this gain modulation to attention-related improvements in behavior. However, after extensive training, this early attentional gain was eliminated even though there were still substantial attention-related improvements in behavior. Accordingly, the SDT-based model required noise reduction to account for the link between the stimulus-evoked visual responses and attentional modulations of behavior. These findings suggest that training can lead to fundamental changes in the way attention alters the early cortical responses that support selective information processing. Moreover, these data facilitate the translation of results across different species and across experimental procedures that employ different behavioral training regimes.

Sensitivity to value-driven attention is predicted by how we learn from value

Reward learning is known to influence the automatic capture of attention. This study examined how the rate of learning, after high- or low-value reward outcomes, can influence future transfers into value-driven attentional capture. Participants performed an instrumental learning task that was directly followed by an attentional capture task. A hierarchical Bayesian reinforcement model was used to infer individual differences in learning from high or low reward. Results showed a strong relationship between high-reward learning rates (or the weight that is put on learning after a high reward) and the magnitude of attentional capture with high-reward colors. Individual differences in learning from high or low rewards were further related to performance differences when high- or low-value distractors were present. These findings provide novel insight into the development of value-driven attentional capture by showing how information updating after desired or undesired outcomes can influence future deployments of automatic attention.

On the feature specificity of value-driven attention

When an otherwise inconspicuous stimulus is learned to predict a reward, this stimulus will automatically capture visual attention. This learned attentional bias is not specific to the precise object previously associated with reward, but can be observed for different stimuli that share a defining feature with the reward cue. Under certain circumstances, value-driven attentional biases can even transfer to new contexts in which the reward cues were not previously experienced, and can also be evident for different exemplars of a stimulus category, suggesting some degree of tolerance in the scope of the underlying bias. Whether a match to a reward-predictive feature is necessary to support value-driven attention, or whether similar-looking features also receive some degree of elevated priority following associative reward learning, remains an open question. Here, I examine the impact of learned associations between reward and red- and green-colored stimuli on the processing of other colors. The findings show that even though other colors experienced during training were non-predictive with respect to reward, the speed with which targets possessing these colors were identified in a subsequent test phase was affected by their similarity to the high-value color. Thus, value-driven attentional biases for stimulus features are imprecise, as would be predicted by a sensory gain model of value-driven attention.

Going for It

What we pay attention to is influenced by reward learning. Previously reward-associated stimuli are difficult to ignore, automatically drawing our attention even when we know they are no longer beneficial to us. Recent research has suggested that these value-dependent attentional biases affect more than just perception, biasing an individual to act in such a way as to pursue the attended stimulus. Here, I review this evidence and argue for a direct link between attentional biases and shifts in action-related priorities. Given this link, abnormal or otherwise inappropriate reward-related attentional biases are hypothesized to contribute to undesired habits and poor economic choices.

What is abnormal about addiction-related attentional biases?

BACKGROUND

The phenotype of addiction includes prominent attentional biases for drug cues, which play a role in motivating drug-seeking behavior and contribute to relapse. In a separate line of research, arbitrary stimuli have been shown to automatically capture attention when previously associated with reward in non-clinical samples.

METHODS AND RESULTS

Here, I argue that these two attentional biases reflect the same cognitive process. I outline five characteristics that exemplify attentional biases for drug cues: resistant to conflicting goals, robust to extinction, linked to dorsal striatal dopamine and to biases in approach behavior, and can distinguish between individuals with and without a history of drug dependence. I then go on to describe how attentional biases for arbitrary reward-associated stimuli share all of these features, and conclude by arguing that the attentional components of addiction reflect a normal cognitive process that promotes reward-seeking behavior.

Gain- and Loss-Related Brain Activation Are Associated with Information Search Differences in Risky Gambles: An fMRI and Eye-Tracking Study

People differ in the way they approach and handle choices with unsure outcomes. In this study, we demonstrate that individual differences in the neural processing of gains and losses relates to attentional differences in the way individuals search for information in gambles. Fifty subjects participated in two independent experiments. Participants first completed an fMRI experiment involving financial gains and losses. Subsequently, they performed an eye-tracking experiment on binary choices between risky gambles, each displaying monetary outcomes and their respective probabilities. We find that individual differences in gain and loss processing relate to attention distribution. Individuals with a stronger reaction to gains in the ventromedial prefrontal cortex paid more attention to monetary amounts, while a stronger reaction in the ventral striatum to losses was correlated with an increased attention to probabilities. Reaction in the posterior cingulate cortex to losses was also found to correlate with an increased attention to probabilities. Our data show that individual differences in brain activity and differences in information search processes are closely linked.

Mechanisms of habitual approach: Failure to suppress irrelevant responses evoked by previously reward-associated stimuli

Reward learning has a powerful influence on the attention system, causing previously reward-associated stimuli to automatically capture attention. Difficulty ignoring stimuli associated with drug reward has been linked to addiction relapse, and the attention system of drug-dependent patients seems especially influenced by reward history. This and other evidence suggests that value-driven attention has consequences for behavior and decision-making, facilitating a bias to approach and consume the previously reward-associated stimulus even when doing so runs counter to current goals and priorities. Yet, a mechanism linking value-driven attention to behavioral responding and a general approach bias is lacking. Here we show that previously reward-associated stimuli escape inhibitory processing in a go/no-go task. Control experiments confirmed that this value-dependent failure of goal-directed inhibition could not be explained by search history or residual motivation, but depended specifically on the learned association between particular stimuli and reward outcome. When a previously high-value stimulus is encountered, the response codes generated by that stimulus are automatically afforded high priority, bypassing goal-directed cognitive processes involved in suppressing task-irrelevant behavior. (PsycINFO Database Record

The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications.