The role of reward prediction in the control of attention

Investigating the selective attention, attentional bias and smoking among Turkish University students: A cross-sectional study

The aim of this study was to investigate the relationship between selective attention, attentional bias and smoking in Turkish university students. The study was conducted with 64 individuals with an average age of 20.3 ± 0.83 years. While the d2 Test of Attention was used to measure various aspects of attention including processing speed, accuracy and selective attention; the Smoking Stroop Test evaluated the attentional bias toward smoking-related stimuli by comparing reaction times to smoking-related and neutral words. Also, the Fagerström Nicotine Dependence Test and Questionnaire of Smoking Urges assessed the level of nicotine dependence and cravings of participants. Findings showed that smoking status (B: 0.05, 95%CI:0.03,0.07) was statistically explanatory of attentional bias, but there was no significant difference in selective attention performance in terms of smoking status. These results are important for the development of intervention methods targeting cognitive processes associated with smoking.

Manipulating the reliability of target-color information modulates value-driven attentional capture

Previously rewarded stimuli slow response times (RTs) during visual search, despite being physically non-salient and no longer task-relevant or rewarding. Such value-driven attentional capture (VDAC) has been measured in a training-test paradigm. In the training phase, the search target is rendered in one of two colors (one predicting high reward and the other low reward). In this study, we modified this traditional training phase to include pre-cues that signaled reliable or unreliable information about the trial-to-trial color of the training phase search target. Reliable pre-cues indicated the upcoming target color with certainty, whereas unreliable pre-cues indicated the target was equally likely to be one of two distinct colors. Thus reliable and unreliable pre-cues provided certain and uncertain information, respectively, about the magnitude of the upcoming reward. We then tested for VDAC in a traditional test phase. We found that unreliably pre-cued distractors slowed RTs and drew more initial eye movements during search for the test-phase target, relative to reliably pre-cued distractors, thus providing novel evidence for an influence of information reliability on attentional capture. That said, our experimental manipulation also eliminated value-dependency (i.e., slowed RTs when a high-reward-predicting distractor was present relative to a low-reward-predicting distractor) for both kinds of distractors. Taken together, these results suggest that target-color uncertainty, rather than reward magnitude, played a critical role in modulating the allocation of value-driven attention in this study.

Value-driven attention and associative learning models: a computational simulation analysis

Value-driven attentional capture (VDAC) refers to a phenomenon by which stimulus features associated with greater reward value attract more attention than those associated with smaller reward value. To date, the majority of VDAC research has revealed that the relationship between reward history and attentional allocation follows associative learning rules. Accordingly, a mathematical implementation of associative learning models and multiple comparison between them can elucidate the underlying process and properties of VDAC. In this study, we implemented the Rescorla-Wagner, Mackintosh (Mac), Schumajuk-Pearce-Hall (SPH), and Esber-Haselgrove (EH) models to determine whether different models predict different outcomes when critical parameters in VDAC were adjusted. Simulation results were compared with experimental data from a series of VDAC studies by fitting two key model parameters, associative strength (V) and associability (α), using the Bayesian information criterion as a loss function. The results showed that SPH-V and EH- α outperformed other implementations of phenomena related to VDAC, such as expected value, training session, switching (or inertia), and uncertainty. Although V of models were sufficient to simulate VDAC when the expected value was the main manipulation of the experiment, α of models could predict additional aspects of VDAC, including uncertainty and resistance to extinction. In summary, associative learning models concur with the crucial aspects of behavioral data from VDAC experiments and elucidate underlying dynamics including novel predictions that need to be verified.

Value-directed memory selectivity relies on goal-directed knowledge of value structure prior to encoding in young and older adults

People are generally able to selectively attend and remember high-value over low-value information. Here, we investigated whether young and older adults would display typical value-based memory selectivity effects for to-be-learned item-value associations when goal-directed information about the meaning of associated values was presented before and after encoding. In two experiments, both young and older adults were presented with one (Experiment 1) or multiple (Experiment 2) lists of words that were arbitrarily paired with different numerical values (e.g., "door-8") or font colors (e.g., "door" presented in red), which indicated each word's value. In Experiment 1, participants were told that the numerical value indicated the relative importance of each item either before they studied the list (preencoding), after they studied it (postencoding), or not at all (no value control instructions). Older adults were significantly more selective in the preencoding condition relative to the other conditions, whereas younger adults were not selective in any condition on this single-list (numerical) value task of Experiment 1. In Experiment 2, young and older adults were tested on four additional lists of both pre- and postencoding trials each after studying and recalling four lists of words without any value instructions. Results from Experiment 2 revealed that both young and older adults selectively prioritized high-value words on the preencoding trials, but not on postencoding trials, on this color-based categorical (low-medium-high) value task. The present study highlights a critical role of goal-directed knowledge of value-based instructions prior to encoding to facilitate typically observed value-directed memory selectivity for important information. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Auditory and cross-modal attentional bias toward positive natural sounds: Behavioral and ERP evidence

Recently, researchers have expanded the investigation into attentional biases toward positive stimuli; however, few studies have examined attentional biases toward positive auditory information. In three experiments, the present study employed an emotional spatial cueing task using emotional sounds as cues and auditory stimuli (Experiment 1) or visual stimuli (Experiment 2 and Experiment 3) as targets to explore whether auditory or visual spatial attention could be modulated by positive auditory cues. Experiment 3 also examined the temporal dynamics of cross-modal auditory bias toward positive natural sounds using event-related potentials (ERPs). The behavioral results of the three experiments consistently demonstrated that response times to targets were faster after positive auditory cues than they were after neutral auditory cues in the valid condition, indicating that healthy participants showed a selective auditory attentional bias (Experiment 1) and cross-modal attentional bias (Experiment 2 and Experiment 3) toward positive natural sounds. The results of Experiment 3 showed that N1 amplitudes were more negative after positive sounds than they were after neutral sounds, which further provided electrophysiological evidence that positive auditory information enhances attention at early stages in healthy adults. The results of the experiments performed in the present study suggest that humans exhibit an attentional bias toward positive natural sounds.

Spatial task relevance modulates value-driven attentional capture

Attention tends to be attracted to visual features previously associated with reward. To date, nearly all existing studies examined value-associated stimuli at or near potential target locations, making such locations meaningful to inspect. The present experiments examined whether the attentional priority of a value-associated stimulus depends on its location-wise task relevance. In three experiments we used an RSVP task to compare the attentional demands of a value-associated peripheral distractor to that of a distractor associated with the top-down search goal. At a peripheral location that could never contain the target, a value-associated color did not capture attention. In contrast, at the same location, a distractor in a goal-matching color did capture attention. The results show that value-associated stimuli lose their attentional priority at task-irrelevant locations, in contrast to other types of stimuli that capture attention.

Dynamic modulation of neural feedback processing and attention during spatial probabilistic learning

Learned stimulus-reward associations can modulate behavior and the underlying neural processing of information. We investigated the cascade of these neurocognitive mechanisms involved in the learning of spatial stimulus-reward associations. Using electroencephalogram recordings while participants performed a probabilistic spatial reward learning task, we observed that the feedback-related negativity component was more negative in response to loss feedback compared to gain feedback but showed no modulation by learning. The late positive component became larger in response to losses as the learning set progressed but smaller in response to gains. In addition, feedback-locked alpha frequency oscillations measured over occipital sites were predictive of N2pc amplitudes—a marker of spatial attention orienting—observed on the next trial. This relationship was found to become stronger with learning set progression. Taken together, we elucidated neurocognitive dynamics underlying feedback processing during spatial reward learning, and the subsequent effects of these learned spatial stimulus-reward associations on spatial attention.

•

We can learn which spatial location relates to the highest probability of reward

•

Neural processing of feedback valence was not influenced by learning

•

LPC amplitude was dynamically modulated by learning, reflecting context updating

•

Feedback-locked alpha power was predictive of ensuing orientation of attention

The location independence of learned attentional flexibility

Individuals can adjust their shift readiness, known as attentional flexibility, according to the statistical structure of the environment. However, the extent to which these modulations in attentional flexibility are associated with a global readiness to shift attention to any location versus an anticipated shift to a single location remains unknown. Across two experiments, participants shifted attention among three rapid serial visual presentation (RSVP) streams of alphanumeric characters in response to embedded visual cues and made button presses in response to targets at the cued location. We manipulated the likelihood that participants would receive a cue that signaled a shift between two of the streams across blocks of trials. The likelihood of a cued shift of attention to the third location was held constant across all blocks. Participants demonstrated smaller target detection shift costs (Experiments 1 and 2) and shorter saccade latencies (Experiment 1) when the overall likelihood of shifting was high than when the overall shift likelihood was low. Critically, we observed evidence of both global shift readiness and location-specific shift readiness in both experiments such that participants shifted attention to the most-likely-to-be cued location the fastest, but still demonstrated a difference in the time to shift attention to the unlikely location according to the overall shift likelihood. Our findings provide evidence that moment-by-moment changes in attentional flexibility are not limited to an expectation to shift to a single location, but rather reflect, in part, a location-independent state of control.

The past, present, and future of selection history

The last ten years of attention research have witnessed a revolution, replacing a theoretical dichotomy (top-down vs. bottom-up control) with a trichotomy (biased by current goals, physical salience, and selection history). This third new mechanism of attentional control, selection history, is multifaceted. Some aspects of selection history must be learned over time whereas others reflect much more transient influences. A variety of different learning experiences can shape the attention system, including reward, aversive outcomes, past experience searching for a target, target‒non-target relations, and more. In this review, we provide an overview of the historical forces that led to the proposal of selection history as a distinct mechanism of attentional control. We then propose a formal definition of selection history, with concrete criteria, and identify different components of experience-driven attention that fit within this definition. The bulk of the review is devoted to exploring how these different components relate to one another. We conclude by proposing an integrative account of selection history centered on underlying themes that emerge from our review.

Selective memory disrupted in intra-modal dual-task encoding conditions

Given natural memory limitations, people can generally attend to and remember high-value over low-value information even when cognitive resources are depleted in older age and under divided attention during encoding, representing an important form of cognitive control. In the current study, we examined whether tasks requiring overlapping processing resources may impair the ability to selectively encode information in dual-task conditions. Participants in the divided-attention conditions of Experiment 1 completed auditory tone-distractor tasks that required them to discriminate between tones of different pitches (audio-nonspatial) or auditory channels (audio-spatial), while studying items in different locations in a grid (visual-spatial) differing in reward value. Results indicated that, while reducing overall memory accuracy, neither cross-modal auditory distractor task influenced participants' ability to selectively encode high-value items relative to a full attention condition, suggesting maintained cognitive control. Participants in Experiment 2 studied the same important visual-spatial information while completing demanding color (visual-nonspatial) or pattern (visual-spatial) discrimination tasks during study. While the cross-modal visual-nonspatial task did not influence memory selectivity, the intra-modal visual-spatial secondary task eliminated participants' sensitivity to item value. These results add novel evidence of conditions of impaired cognitive control, suggesting that the effectiveness of top-down, selective encoding processes is attenuated when concurrent tasks rely on overlapping processing resources.

Gotcha: Working memory prioritization from automatic attentional biases

Attention is an important resource for prioritizing information in working memory (WM), and it can be deployed both strategically and automatically. Most research investigating the relationship between WM and attention has focused on strategic efforts to deploy attentional resources toward remembering relevant information. However, such voluntary attentional control represents a mere subset of the attentional processes that select information to be encoded and maintained in WM (Theeuwes, Journal of Cognition, 1[1]: 29, 1-15, 2018). Here, we discuss three ways in which information becomes prioritized automatically in WM-physical salience, statistical learning, and reward learning. This review integrates findings from perception and working memory studies to propose a more sophisticated understanding of the relationship between attention and working memory.

Cannabinoid Modulation of Dopamine Release During Motivation, Periodic Reinforcement, Exploratory Behavior, Habit Formation, and Attention

Motivational and attentional processes energize action sequences to facilitate evolutionary competition and promote behavioral fitness. Decades of neuropharmacology, electrophysiology and electrochemistry research indicate that the mesocorticolimbic DA pathway modulates both motivation and attention. More recently, it was realized that mesocorticolimbic DA function is tightly regulated by the brain's endocannabinoid system and greatly influenced by exogenous cannabinoids-which have been harnessed by humanity for medicinal, ritualistic, and recreational uses for 12,000 years. Exogenous cannabinoids, like the primary psychoactive component of cannabis, delta-9-tetrahydrocannabinol, produce their effects by acting at binding sites for naturally occurring endocannabinoids. The brain's endocannabinoid system consists of two G-protein coupled receptors, endogenous lipid ligands for these receptor targets, and several synthetic and metabolic enzymes involved in their production and degradation. Emerging evidence indicates that the endocannabinoid 2-arachidonoylglycerol is necessary to observe concurrent increases in DA release and motivated behavior. And the historical pharmacology literature indicates a role for cannabinoid signaling in both motivational and attentional processes. While both types of behaviors have been scrutinized under manipulation by either DA or cannabinoid agents, there is considerably less insight into prospective interactions between these two important signaling systems. This review attempts to summate the relevance of cannabinoid modulation of DA release during operant tasks designed to investigate either motivational or attentional control of behavior. We first describe how cannabinoids influence DA release and goal-directed action under a variety of reinforcement contingencies. Then we consider the role that endocannabinoids might play in switching an animal's motivation from a goal-directed action to the search for an alternative outcome, in addition to the formation of long-term habits. Finally, dissociable features of attentional behavior using both the 5-choice serial reaction time task and the attentional set-shifting task are discussed along with their distinct influences by DA and cannabinoids. We end with discussing potential targets for further research regarding DA-cannabinoid interactions within key substrates involved in motivation and attention.

Using aversive conditioning with near-real-time feedback to shape eye movements during naturalistic viewing

Strategically shaping patterns of eye movements through training has manifold promising applications, with the potential to improve the speed and efficiency of visual search, improve the ability of humans to extract information from complex displays, and help correct disordered eye movement patterns. However, training how a person moves their eyes when viewing an image or scene is notoriously difficult, with typical approaches relying on explicit instruction and strategy, which have notable limitations. The present study introduces a novel approach to eye movement training using aversive conditioning with near-real-time feedback. Participants viewed indoor scenes (eight scenes presented over 48 trials) with the goal of remembering those scenes for a later memory test. During viewing, saccades meeting specific amplitude and direction criteria probabilistically triggered an aversive electric shock, which was felt within 50 ms after the eliciting eye movement, allowing for a close temporal coupling between an oculomotor behavior and the feedback intended to shape it. Results demonstrate a bias against performing an initial saccade in the direction paired with shock (Experiment 1) or generally of the amplitude paired with shock (Experiment 2), an effect that operates without apparent awareness of the relationship between shocks and saccades, persists into extinction, and generalizes to the viewing of novel images. The present study serves as a proof of concept concerning the implementation of near-real-time feedback in eye movement training.

Combined influence of valence and statistical learning on the control of attention: Evidence for independent sources of bias

Selection history exerts a powerful influence on the control of attention. Stimuli signalling reward and punishment capture attention even when physically non-salient and task-irrelevant. Repeated presentation of a salient distractor at a particular location generates learned suppression, resulting in reduced attentional processing at that location. A debate in the field concerns whether different components of selection history influence attention via a common underlying mechanism of learning-dependent control or via distinct, independent mechanisms. We probed this question with a particular focus on reward/punishment history and learned suppression. Participants were trained to suppress a particular location (high probability distractor location) and associate colours with reward or no outcome (no-reward). In a subsequent task, reward and no-reward distractors appeared in all locations equally often. In a separate experiment, we replaced reward with electric shocks. Reward and shock distractors captured attention more strongly than no-reward and no-shock distractors irrespective of their location. Distractors appearing in the high probability location showed reduced capture irrespective of their type. The results imply that reward and punishment learning and learned suppression have independent influences on the attentional system.

Uncertainty modulates value-driven attentional capture

The majority of previous studies on the value modulation of attention have shown that the magnitude of value-driven attentional bias correlates with the strength of reward association. However, relatively little is known about how uncertainty affects value-based attentional bias. We investigated whether attentional capture by previously rewarded stimuli is modulated by the uncertainty of the learned value without the influence of the strength of reward association. Participants were instructed to identify the line orientation in the target color circle. Importantly, each target color was associated with a different level of uncertainty by tuning the variation in reward delivery (Experiment 1) or reward magnitude (Experiment 2). Attentional interference for uncertainty-related distractors was greater than that for certainty distractors in Experiments 1 and 2. In addition, uncertainty-induced attentional bias disappeared earlier than attentional bias for certainty. The study demonstrated that uncertainty modulates value-based attentional capture in terms of strength and persistence, even when the effect of expected value remains constant.

How does the attention system learn from aversive outcomes?

Learning about aversive outcomes plays a role in the guidance of attention. Classical conditioning generates a bias to predictors of aversive outcomes, whereas instrumental learning potentiates a negatively reinforced avoidance behavior, which can be difficult to dissociate in the case of attention to aversively conditioned stimuli. The present study examined the relative contribution from these two learning processes to the control of attention. Participants were first provided an opportunity to avoid an electric shock by generating a saccade in the direction opposite one of two stimuli. In contradiction to the practiced avoidance behavior, such training resulted in a bias to orient toward the shock-associated stimulus, indicative of a more dominant role of classical conditioning in the control of attention. The findings are in parallel with the influence of positive reinforcement on attention, suggesting that the attention system may be guided by motivational relevance rather than a particular emotional valence. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Impact of relative and absolute values on selective attention

Valuable stimuli receive attentional priority. However, it is unknown whether the mechanism of the attentional priority is based on relative (e.g., higher) or absolute (e.g., 45 points) values. Therefore, we manipulated the relative and absolute values independently in a modified value-driven attentional capture paradigm. In the training phase, where associative learning occurs between color and reward value, two test target colors were each presented with another different target color (reference target colors) in separate context blocks. Therefore, each test target color had different reference points. In the test phase, the two test target colors were used as singleton distractor colors. In the training phase of Experiment 1, the absolute reward value of the test target colors was the same, but one had a higher value than its reference target color and the other had a lower value. In the test phase, the high relative value color distractor captured attention more, suggesting that the relative value of stimuli influenced selective attention. In Experiment 2 the relative value of the test target colors was the same, but the absolute value was higher for one. The high and low absolute value color distractors captured attention equally in the test phase, indicating little impact of the absolute value on selective attention. These findings suggest that the relative value, rather than absolute value, plays a critical role in the allocation of attention. Accordingly, the present study suggests that prospect theory (Kahneman & Tversky, Econometrica, 47 (2), 363-391, 1979) can be extended to earlier cognitive stages such as selective attention.

Specificity and persistence of statistical learning in distractor suppression

Statistical regularities in distractor location trigger suppression of high-probability distractor locations during visual search. The degree to which such suppression reflects generalizable, persistent changes in a spatial priority map has not been examined. We demonstrate that suppression of high-probability distractor locations persists after location probabilities are equalized and likely reflects a genuine reshaping of the priority map rather than more transient effects of selection history. Statistically learned suppression generalizes across contexts within a task during learning but does not generalize between task paradigms using unrelated stimuli in identical spatial locations. These findings suggest that stimulus features do play a role in learned spatial suppression, potentially gating the weights applied to a spatial priority map. However, the binding of location to context during learning is not automatic, in contrast to the previously reported interaction of location-based statistical learning and stimulus features. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Selection History-Driven Signal Suppression

The control of attention is influenced by current goals, physical salience, and selection history. Under certain conditions, physically salient stimuli can be strategically suppressed below baseline levels, facilitating visual search for a target. It is unclear whether such signal suppression is a broad mechanism of selective information processing that extends to other sources of attentional priority evoked by task-irrelevant stimuli, or whether it is particular to physically salient perceptual signals. Using eye movements, in the present study we highlight a case where a former-target-color distractor facilitates search for a target on a large percentage of trials. Our findings provide evidence that the principle of signal suppression extends to other sources of attentional priority beyond physical salience, and that selection history can be leveraged to strategically guide attention away from a stimulus.

Neural Mechanisms of Strategic Adaptation in Attentional Flexibility

Individuals are able to adjust their readiness to shift spatial attention, referred to as "attentional flexibility," according to the changing demands of the environment, but the neural mechanisms underlying learned adjustments in flexibility are unknown. In the current study, we used fMRI to identify the brain structures responsible for learning shift likelihood. Participants were cued to covertly hold or shift attention among continuous streams of alphanumeric characters and to indicate the parity of target stimuli. Unbeknown to the participants, the stream locations were predictive of the likelihood of having to shift (or hold) attention. Participants adapted their attentional flexibility according to contextual demands, such that the RT cost associated with shifting attention was smallest when shift cues were most likely. Learning model-derived shift prediction error scaled positively with activity within dorsal and ventral frontoparietal regions, documenting that these regions track and update shift likelihood. A complementary inverted encoding model analysis revealed that the pretrial difference in attentional selection strength between to-be-attended and to-be-ignored locations did not change with increasing shift likelihood. The behavioral improvement associated with learned flexibility may primarily arise from a speeding of the shift process rather than from preparatory broadening of attentional selection.

Learned prioritization yields attentional biases through selection history

While numerous studies have provided evidence for selection history as a robust influence on attentional allocation, it is unclear precisely which behavioral factors can result in this form of attentional bias. In the current study, we focus on “learned prioritization” as an underlying mechanism of selection history and its effects on selective attention. We conducted two experiments, each starting with a training phase to ensure that participants learned different stimulus priorities. This was accomplished via a visual search task in which a specific color was consistently more relevant when presented together with another given color. In Experiment 1, one color was always prioritized over another color and inferior to a third color, such that each color had an equal overall priority by the end of the training session. In Experiment 2, the three different colors had unequal priorities at the end of the training session. A subsequent testing phase in which participants had to search for a shape-defined target showed that only stimuli with unequal overall priorities (Experiment 2) affected attentional selection, with increased reaction times when a distractor was presented in a previously high-priority compared with a low-priority color. These results demonstrate that adopting an attentional set where certain stimuli are prioritized over others can result in a lingering attentional bias and further suggest that selection history does not equally operate on all previously selected stimuli. Finally, we propose that findings in value-driven attention studies where high-value and low-value signaling stimuli differentially capture attention may be a result of learned prioritization rather than reward.

Working Memory Load Enhances the Attentional Capture of Low Reward History

Attention priority of reward history, also called value-driven attentional capture (VDAC), is different from that of saliency or contingency. The magnitude of VDAC was found to be correlated with working memory capacity, but how cognitive control interacts with the attentional allocation of reward association is not clear. Here, we examined whether the distraction by learned color-reward association would change under different working memory load conditions. Participants were first trained with color-reward associations by searching a green/red circle with low/high reward. Then, during the test session, participants needed to search a unique shape while a green/red shape was either presented as a distractor or not shown at all. To manipulate the working memory load in the test, a digital memory task was integrated with the visual search task in half of the trials (memory load condition), but not in the other half (no-load condition). Consistent results were found in two experiments that the magnitude of attentional capture caused by low-value distractors was larger under memory load condition than under no-load condition, while there was no enough evidence supporting the influence of memory load on attentional capture by high-value distractors. These results suggested that working memory load, which occupied part of cognitive resources, reduced the priority of target information and might also modulate the strength of reward association holding in working memory. These findings extend the knowledge regarding the influence of working memory load on attentional capture of reward and suggest that reward-induced distraction is dynamic and could be modulated by cognitive control.

Selection history in context: Evidence for the role of reinforcement learning in biasing attention

Attention is biased towards learned predictors of reward. The influence of reward history on attentional capture has been shown to be context-specific: When particular stimulus features are associated with reward, these features only capture attention when viewed in the context in which they were rewarded. Selection history can also bias attention, such that prior target features gain priority independently of reward history. The contextual specificity of this influence of selection history on attention has not been examined. In the present study, we demonstrate that the consequences of repetitive selection on attention robustly generalize across context, such that prior target features capture attention even in contexts in which they were never seen previously. Our findings suggest that the learning underlying attention driven by outcome-independent selection history differs qualitatively from the learning underlying value-driven attention, consistent with a distinction between associative and reinforcement learning mechanisms.

Measuring attention to reward as an individual trait: the value-driven attention questionnaire (VDAQ)

Reward history is a powerful determinant of what we pay attention to. This influence of reward on attention varies substantially across individuals, being related to a variety of personality variables and clinical conditions. Currently, the ability to measure and quantify attention-to-reward is restricted to the use of psychophysical laboratory tasks, which limits research into the construct in a variety of ways. In the present study, we introduce a questionnaire designed to provide a brief and accessible means of assessing attention-to-reward. Scores on the questionnaire correlate with other measures known to be related to attention-to-reward and predict performance on multiple laboratory tasks measuring the construct. In demonstrating this relationship, we also provide evidence that attention-to-reward as measured in the lab, an automatic and implicit bias in information processing, is related to overt behaviors and motivations in everyday life as assessed via the questionnaire. Variation in scores on the questionnaire is additionally associated with a distinct biomarker in brain connectivity, and the questionnaire exhibits acceptable test-retest reliability. Overall, the Value-Driven Attention Questionnaire (VDAQ) provides a useful proxy-measure of attention-to-reward that is much more accessible than typical laboratory assessments.

Reward history but not search history explains value-driven attentional capture

In past years, an extensive amount of research has focused on how past experiences guide future attention. Humans automatically attend to stimuli previously associated with reward and stimuli that have been experienced during visual search, even when it is disadvantageous in present situations. Recently, the relationship between "reward history" and "search history" has been discussed critically. We review results from research on value-driven attentional capture (VDAC) with a focus on these two experience-based attentional selection processes and their distinction. To clarify inconsistencies, we examined VDAC within a design that allows a direct comparison with other mechanisms of attentional selection. Eighty-four healthy adults were trained to incidentally associate colors with reward (10 cents, 2 cents) or with no reward. In a subsequent visual search task, distraction by reward-associated and unrewarded stimuli was contrasted. In the training phase, reward signals facilitated performance. When these value-signaling stimuli appeared as distractors in the test phase, they continuously shaped attentional selection, despite their task irrelevance. Our findings clearly cannot be attributed to a history of target search. We conclude that once an association is established, value signals guide attention automatically in new situations, which can be beneficial or not, depending on the congruency with current goals.

Attention Combines Similarly in Covert and Overt Conditions

Attention is classically classified according to mode of engagement into voluntary and reflexive, and type of operation into covert and overt. The first distinguishes whether attention is elicited intentionally or by unexpected events; the second, whether attention is directed with or without eye movements. Recently, this taxonomy has been expanded to include automated orienting engaged by overlearned symbols and combined attention engaged by a combination of several modes of function. However, so far, combined effects were demonstrated in covert conditions only, and, thus, here we examined if attentional modes combined in overt responses as well. To do so, we elicited automated, voluntary, and combined orienting in covert, i.e., when participants responded manually and maintained central fixation, and overt cases, i.e., when they responded by looking. The data indicated typical effects for automated and voluntary conditions in both covert and overt data, with the magnitudes of the combined effect larger than the magnitude of each mode alone as well as their additive sum. No differences in the combined effects emerged across covert and overt conditions. As such, these results show that attentional systems combine similarly in covert and overt responses and highlight attention's dynamic flexibility in facilitating human behavior.

Pavlovian reward learning elicits attentional capture by reward-associated stimuli

Feature-reward association elicits value-driven attentional capture (VDAC) regardless of the task relevance of associated features. What are the necessary conditions for feature-reward associations in VDAC? Recent studies claim that VDAC is based on Pavlovian conditioning. In this study, we manipulated the temporal relationships among feature, response, and reward in reward learning to elucidate the necessary components of VDAC. We presented reward-associated features in a variety of locations in a flanker task to form a color-reward association (training phase) and then tested VDAC in a subsequent visual search task (test phase). In Experiment 1, we showed reward-associated features in a task display requiring response selection and observed VDAC, consistent with most previous studies. In Experiment 2, features presented at a fixation display before a task display also induced VDAC. Moreover, in Experiment 3, we reduced the time interval between features and rewards so that features appeared after a task display and we obtained marginally significant VDAC. However, no VDAC was observed when features and rewards were simultaneously presented in a feedback display in Experiments 4 and 5, suggesting that a direct association between feature and reward is not sufficient for VDAC. These results are in favor of the idea that response selection does not mediate feature-reward association in VDAC. Moreover, the evidence suggests that the time interval of feature and reward is flexible with some restriction in the learning of feature-reward association. The present study supports the hypothesis that theories of Pavlovian conditioning can account for feature-reward association in VDAC.

Do Reward-Related Distractors Impair Cognitive Performance? Perhaps Not

Over a hundred prior studies show that reward-related distractors capture attention. It is less clear, however, whether and when reward-related distractors affect performance on tasks that require cognitive control. In this experiment, we examined whether reward-related distractors impair performance during a demanding arithmetic task. Participants (N = 81) solved math problems, while they were exposed to task-irrelevant stimuli that were previously associated with monetary rewards (vs. not). Although we found some evidence for reward learning in the training phase, results from the test phase showed no evidence that reward-related distractors harm cognitive performance. This null effect was invariant across different versions of our task. We examined the results further with Bayesian analyses, which showed positive evidence for the null. Altogether, the present study showed that reward-related distractors did not harm performance on a mental arithmetic task. When considered together with previous studies, the present study suggests that the negative impact of reward-related distractors on cognitive control is not as straightforward as it may seem, and that more research is needed to clarify the circumstances under which reward-related distractors harm cognitive control.

Behavioural evidence for parallel outcome-sensitive and outcome-insensitive Pavlovian learning systems in humans

There is a dichotomy in instrumental conditioning between goal-directed actions and habits that are distinguishable on the basis of their relative sensitivity to changes in outcome value. It is less clear whether a similar distinction applies in Pavlovian conditioning, where responses have been found to be predominantly outcome sensitive. To test for both devaluation insensitive and devaluation sensitive Pavlovian conditioning in humans, we conducted four experiments combining Pavlovian conditioning and outcome devaluation procedures while measuring multiple conditioned responses. Our results suggest that Pavlovian conditioning involves two distinct types of learning: one that learns the current value of the outcome which is sensitive to devaluation, and one that learns about the spatial localisation of the outcome which is insensitive to devaluation. Our findings have implications for the mechanistic understanding of Pavlovian conditioning and provide a more nuanced understanding of Pavlovian mechanisms that might contribute to a number of psychiatric disorders.

Dissociable neural mechanisms underlie value-driven and selection-driven attentional capture

Stimuli associated with reward acquire the ability to automatically capture attention. It is also the case that, with sufficient training, former targets can acquire the ability to capture attention in the absence of extrinsic rewards. It remains unclear whether these two experience-dependent attentional biases share a common underlying mechanism. The present study examined the influence of selection history on attentional capture, and compared its neural correlates with those of value-driven attentional capture reported in Anderson et al. (2014a). Participants completed a four-day training in visual search for a specific colour target. In a subsequent test phase, they performed visual search for a shape-defined target in which colour was task-irrelevant. Response times were slower when a former target-colour distractor was present than when it was absent, replicating attentional capture by unrewarded former targets. Neuroimaging results revealed preferential activation by a former target-colour distractor in sensory areas. A more right lateralised pattern of activation was observed, compared to attentional capture by reward cues. No distractor-evoked activity was found in the caudate tail. These results imply that attentional capture by selection history is primarily driven by plasticity in sensory areas, and that reward history and selection history influence attention via dissociable underlying mechanisms.

Can Monetary Reward Modulate Social Attention?

Selective visual attention is a primary cognitive function, which allows the selection of the most relevant stimuli in the environment by prioritizing their processing. Several studies showed that this process can be influenced by both social signals, such as gaze direction (i.e., the Gaze Cueing Effect, GCE) and by the motivational valence of gratifying stimuli, such as monetary rewards. The aim of this study was to explore whether GCE could be modulated by a monetary reward. To this end, we created an experiment in which participants performed a gaze cuing task before and after an implicit learning task aiming to induce an association between gaze direction and monetary reward (experimental condition), or after a perceptual task (control condition). Statistical analyses were conducted following both a frequentist and a Bayesian approach. Results supported previous findings showing the presence of the GCE, i.e., faster responses in congruent trials when the target appeared in the gazed-at location. Interestingly, our results did not reveal significant differences among the conditions. Therefore, contrary to what was reported by previous attentional orienting studies with non-social stimuli, monetary reward does not seem to be able to modulate (or interfere with) the orienting of attention mediated by gaze direction as measured by the GCE. Taken together our results suggest that social signals such as gaze direction have a greater impact than monetary reward in orienting selective attention.

Previously rewarding environments enhance incidental memory formation

Declarative memory performance is superior for items that were encoded in temporal proximity to reward delivery or expectancy. How reward-predicting contexts affect subsequent declarative memory formation in those contexts are, however, unknown. Using an ecological experimental setup in the form of a naturalistic driving simulator task, we examined the effect that previously rewarded environments may have on incidental memory formation. After driving in two distinct environments, one of which associated with monetary reward, participants drove again in the environments, which were embedded with unique images on billboards. A recognition test 24 h later demonstrated that incidental memory was superior for items presented in the reward-associated environment. These findings suggest that environmental cues imbued with incentive salience promote memory processes even in the absence of reward.

Reduced Value-Driven Attentional Capture Among Children with ADHD Compared to Typically Developing Controls

The current study examined whether children with ADHD were more distracted by a stimulus previously associated with reward, but currently goal-irrelevant, than their typically-developing peers. In addition, we also probed the associated cognitive and motivational mechanisms by examining correlations with other behavioral tasks. Participants included 8-12 year-old children with ADHD (n = 30) and typically developing controls (n = 26). Children were instructed to visually search for color-defined targets and received monetary rewards for accurate responses. In a subsequent search task in which color was explicitly irrelevant, we manipulated whether a distractor item appeared in a previously reward-associated color. We examined whether children responded more slowly on trials with the previously-rewarded distractor present compared to trials without this distractor, a phenomenon referred to as value-driven attentional capture (VDAC), and whether children with and without ADHD differed in the extent to which they displayed VDAC. Correlations among working memory performance, immediate reward preference (delay discounting) and attentional capture were also examined. Children with ADHD were significantly less affected by the presence of the previously rewarded distractor than were control participants. Within the ADHD group, greater value-driven attentional capture was associated with poorer working memory. Although both ADHD and control participants were initially distracted by previously reward-associated stimuli, the magnitude of distraction was larger and persisted longer among control participants.

On the distinction between value-driven attention and selection history: Evidence from individuals with depressive symptoms

When predictive of extrinsic reward as targets, stimuli rapidly acquire the ability to automatically capture attention. Attentional biases for former targets of visual search also can develop without reward feedback but typically require much longer training. These learned biases towards former targets often are conceptualized within a single framework and might differ merely in degree. That is, both are the result of the reinforcement of selection history, with extrinsic reward for correct report of the target providing greater reinforcement than correct report alone. A direct test of this shared mechanisms hypothesis is lacking, however. Recent evidence demonstrates that depressed individuals present with blunted value-driven attentional biases. Based on the shared mechanisms hypothesis, we predicted that depressed individuals would similarly show blunted attentional biases for former targets following unrewarded training. To the contrary, however, we found that the effects of selection history on attention were robust and equivalent between individuals experiencing depressive symptoms and control participants, whereas attentional capture by previously reward-associated stimuli was blunted in depressed individuals. Our results suggest a qualitative distinction between the effects of reward history and the effects of selection history on attention, with depressive symptoms impairing the former while leaving the latter unaffected.

Multiple reward-cue contingencies favor expectancy over uncertainty in shaping the reward-cue attentional salience

Reward-predicting cues attract attention because of their motivational value. A debated question regards the conditions under which the cue's attentional salience is governed more by reward expectancy rather than by reward uncertainty. To help shedding light on this relevant issue, here, we manipulated expectancy and uncertainty using three levels of reward-cue contingency, so that, for example, a high level of reward expectancy (p = .8) was compared with the highest level of reward uncertainty (p = .5). In Experiment 1, the best reward-cue during conditioning was preferentially attended in a subsequent visual search task. This result was replicated in Experiment 2, in which the cues were matched in terms of response history. In Experiment 3, we implemented a hybrid procedure consisting of two phases: an omission contingency procedure during conditioning, followed by a visual search task as in the previous experiments. Crucially, during both phases, the reward-cues were never task relevant. Results confirmed that, when multiple reward-cue contingencies are explored by a human observer, expectancy is the major factor controlling both the attentional and the oculomotor salience of the reward-cue.

Changes of Attention during Value-Based Reversal Learning Are Tracked by N2pc and Feedback-Related Negativity

Previously learned reward values can have a pronounced impact, behaviorally and neurophysiologically, on the allocation of selective attention. All else constant, stimuli previously associated with a high value gain stronger attentional prioritization than stimuli previously associated with a low value. The N2pc, an ERP component indicative of attentional target selection, has been shown to reflect aspects of this prioritization, by changes of mean amplitudes closely corresponding to selective enhancement of high value target processing and suppression of high value distractor processing. What has remained unclear so far is whether the N2pc also reflects the flexible and repeated behavioral adjustments needed in a volatile task environment, in which the values of stimuli are reversed often and unannounced. Using a value-based reversal learning task, we found evidence that the N2pc amplitude flexibly and reversibly tracks value-based choices during the learning of reward associated stimulus colors. Specifically, successful learning of current value-contingencies was associated with reduced N2pc amplitudes, and this effect was more apparent for distractor processing, compared with target processing. In addition, following a value reversal the feedback related negativity(FRN), an ERP component that reflects feedback processing, was amplified and co-occurred with increased N2pc amplitudes in trials following low-value feedback. Importantly, participants that showed the greatest adjustment in N2pc amplitudes based on feedback were also the most efficient learners. These results allow further insight into how changes in attentional prioritization in an uncertain and volatile environment support flexible adjustments of behavior.

Funny money: the attentional role of monetary feedback detached from expected value

Stimuli associated with monetary reward can become powerful cues that effectively capture visual attention. We examined whether such value-driven attentional capture can be induced with monetary feedback in the absence of an expected cash payout. To this end, we implemented images of U.S. dollar bills as reward feedback. Participants knew in advance that they would not receive any money based on their performance. Our reward stimuli-$5 and $20 bill images-were thus dissociated from any practical utility. Strikingly, we observed a reliable attentional capture effect for the mere images of bills. Moreover, this finding generalized to Monopoly money. In two control experiments, we found no evidence in favor of nominal or symbolic monetary value. Hence, we claim that bill images are special monetary representations, such that there are strong associations between the defining visual features of bills and reward, probably due to a lifelong learning history. Together, we show that the motivation to earn cash plays a minor role when it comes to monetary rewards, while bill-defining visual features seem to be sufficient. These findings have the potential to influence human factor applications, such as gamification, and can be extended to novel value systems, such as the electronic cash Bitcoin being developed for use in mobile banking. Finally, our procedure represents a proof of concept on how images of money can be used to conserve expenditures in the experimental context.

Don't let it distract you: how information about the availability of reward affects attentional selection

Previous research has shown that attentional selection is affected by reward contingencies: previously selected and rewarded stimuli continue to capture attention even if the reward contingencies are no longer in place. In the current study, we investigated whether attentional selection also is affected by stimuli that merely signal the magnitude of reward available on a given trial but, crucially, have never had instrumental value. In a series of experiments, we show that a stimulus signaling high reward availability captures attention even when that stimulus is and was never physically salient or part of the task set, and selecting it is harmful for obtaining reward. Our results suggest that irrelevant reward-signaling stimuli capture attention, because participants have learned about the relationship between the stimulus and reward. Importantly, we only observed learning after initial attentional prioritization of the reward signaling stimulus. We conclude that nonsalient, task-irrelevant but reward-signaling stimuli can affect attentional selection above and beyond top-down or bottom-up attentional control, however, only after such stimuli were initially prioritized for selection.

Learned value and object perception: Accelerated perception or biased decisions?

Learned value is known to bias visual search toward valued stimuli. However, some uncertainty exists regarding the stage of visual processing that is modulated by learned value. Here, we directly tested the effect of learned value on preattentive processing using temporal order judgments. Across four experiments, we imbued some stimuli with high value and some with low value, using a nonmonetary reward task. In Experiment 1, we replicated the value-driven distraction effect, validating our nonmonetary reward task. Experiment 2 showed that high-value stimuli, but not low-value stimuli, exhibit a prior-entry effect. Experiment 3, which reversed the temporal order judgment task (i.e., reporting which stimulus came second), showed no prior-entry effect, indicating that although a response bias may be present for high-value stimuli, they are still reported as appearing earlier. However, Experiment 4, using a simultaneity judgment task, showed no shift in temporal perception. Overall, our results support the conclusion that learned value biases perceptual decisions about valued stimuli without speeding preattentive stimulus processing.

Sensitivity to value-driven attention is predicted by how we learn from value

Reward learning is known to influence the automatic capture of attention. This study examined how the rate of learning, after high- or low-value reward outcomes, can influence future transfers into value-driven attentional capture. Participants performed an instrumental learning task that was directly followed by an attentional capture task. A hierarchical Bayesian reinforcement model was used to infer individual differences in learning from high or low reward. Results showed a strong relationship between high-reward learning rates (or the weight that is put on learning after a high reward) and the magnitude of attentional capture with high-reward colors. Individual differences in learning from high or low rewards were further related to performance differences when high- or low-value distractors were present. These findings provide novel insight into the development of value-driven attentional capture by showing how information updating after desired or undesired outcomes can influence future deployments of automatic attention.

On the value-dependence of value-driven attentional capture

Findings from an increasingly large number of studies have been used to argue that attentional capture can be dependent on the learned value of a stimulus, or value-driven. However, under certain circumstances attention can be biased to select stimuli that previously served as targets, independent of reward history. Value-driven attentional capture, as studied using the training phase-test phase design introduced by Anderson and colleagues, is widely presumed to reflect the combined influence of learned value and selection history. However, the degree to which attentional capture is at all dependent on value learning in this paradigm has recently been questioned. Support for value-dependence can be provided through one of two means: (1) greater attentional capture by prior targets following rewarded training than following unrewarded training, and (2) greater attentional capture by prior targets previously associated with high compared to low value. Using a variant of the original value-driven attentional capture paradigm, Sha and Jiang (Attention, Perception, and Psychophysics, 78, 403-414, 2016) failed to find evidence of either, and raised criticisms regarding the adequacy of evidence provided by prior studies using this particular paradigm. To address this disparity, here we provided a stringent test of the value-dependence hypothesis using the traditional value-driven attentional capture paradigm. With a sufficiently large sample size, value-dependence was observed based on both criteria, with no evidence of attentional capture without rewards during training. Our findings support the validity of the traditional value-driven attentional capture paradigm in measuring what its name purports to measure.

Reward processing in the value-driven attention network: reward signals tracking cue identity and location

Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value.

No Apparent Influence of Reward upon Visual Statistical Learning

Humans are capable of detecting and exploiting a variety of environmental regularities, including stimulus-stimulus contingencies (e.g., visual statistical learning) and stimulus-reward contingencies. However, the relationship between these two types of learning is poorly understood. In two experiments, we sought evidence that the occurrence of rewarding events enhances or impairs visual statistical learning. Across all of our attempts to find such evidence, we employed a training stage during which we grouped shapes into triplets and presented triplets one shape at a time in an undifferentiated stream. Participants subsequently performed a surprise recognition task in which they were tested on their knowledge of the underlying structure of the triplets. Unbeknownst to participants, triplets were also assigned no-, low-, or high-reward status. In Experiments 1A and 1B, participants viewed shape streams while low and high rewards were “randomly” given, presented as low- and high-pitched tones played through headphones. Rewards were always given on the third shape of a triplet (Experiment 1A) or the first shape of a triplet (Experiment 1B), and high- and low-reward sounds were always consistently paired with the same triplets. Experiment 2 was similar to Experiment 1, except that participants were required to learn value associations of a subset of shapes before viewing the shape stream. Across all experiments, we observed significant visual statistical learning effects, but the strength of learning did not differ amongst no-, low-, or high-reward conditions for any of the experiments. Thus, our experiments failed to find any influence of rewards on statistical learning, implying that visual statistical learning may be unaffected by the occurrence of reward. The system that detects basic stimulus-stimulus regularities may operate independently of the system that detects reward contingencies.