Incidental biasing of attention from visual long-term memory

Statistically learned associations among objects bias attention

A growing body of research suggests that semantic relationships among objects can influence the control of attention. There is also some evidence that learned associations among objects can bias attention. However, it is unclear whether these findings are due to statistical learning or existing semantic relationships. In the present study, we examined whether statistically learned associations among objects can bias attention in the absence of existing semantic relationships. Participants searched for one of four targets among pairs of novel shapes and identified whether the target was present or absent from the display. In an initial training phase, each target was paired with an associated distractor in a fixed spatial configuration. In a subsequent test phase, each target could be paired with the previously associated distractor or a different distractor. In our first experiment, the previously associated distractor was always presented in the same pair as the target. Participants were faster to respond when this distractor was present on target-present trials. In our second experiment, the previously associated distractor was presented in a different pair than the target in the test phase. In this case, participants were slower to respond when this distractor was present on both target-present and target-absent trials. Together, these findings provide clear evidence that statistically learned associations among objects can bias attention, analogous to the effects of semantic relationships on attention.

Hippocampal mechanisms resolve competition in memory and perception

Behaving adaptively requires selection of relevant memories and sensations and suppression of competing ones. We hypothesized that these mechanisms are linked, such that hippocampal computations that resolve competition in memory also shape the precision of sensory representations to guide selective attention. We leveraged f MRI-based pattern similarity, receptive field modeling, and eye tracking to test this hypothesis in humans performing a memory-dependent visual search task. In the hippocampus, differentiation of competing memories predicted the precision of memory-guided eye movements. In visual cortex, preparatory coding of remembered target locations predicted search successes, whereas preparatory coding of competing locations predicted search failures due to interference. These effects were linked: stronger hippocampal memory differentiation was associated with lower competitor activation in visual cortex, yielding more precise preparatory representations. These results demonstrate a role for memory differentiation in shaping the precision of sensory representations, highlighting links between mechanisms that overcome competition in memory and perception.

The intersection of the retrieval state and internal attention

Large-scale brain states or distributed patterns of brain activity modulate downstream processing and behavior. Sustained attention and memory retrieval states impact subsequent memory, yet how these states relate to one another is unclear. I hypothesize that internal attention is a central process of the retrieval state. The alternative is that the retrieval state specifically reflects a controlled, episodic retrieval mode, engaged only when intentionally accessing events situated within a spatiotemporal context. To test my hypothesis, I developed a mnemonic state classifier independently trained to measure retrieval state evidence and applied this classifier to a spatial attention task. I find that retrieval state evidence increases during delay and response intervals when participants are maintaining spatial information. Critically, retrieval state evidence is positively related to the amount of maintained spatial location information and predicts target detection reaction times. Together, these findings support the hypothesis that internal attention is a central process of the retrieval state.

Repetition enhances the effects of activated long-term memory

Recent research indicates that visual long-term memory (vLTM) representations directly interface with perception and guide attention. This may be accomplished through a state known as activated LTM, however, little is known about the nature of activated LTM. Is it possible to enhance the attentional effects of these activated representations? And furthermore, is activated LTM discrete (i.e., a representation is either active or not active, but only active representations interact with perception) or continuous (i.e., there are different levels within the active state that all interact with perception)? To answer these questions, in the present study, we measured intrusion effects during a modified Sternberg task. Participants saw two lists of three complex visual objects, were cued that only one list was relevant for the current trial (the other list was, thus, irrelevant), and then their memory for the cued list was probed. Critically, half of the trials contained repeat objects (shown 10 times each), and half of the trials contained non-repeat objects (shown only once each). Results indicated that repetition enhanced activated LTM, as the intrusion effect (i.e., longer reaction times to irrelevant list objects than novel objects) was larger for repeat trials compared with non-repeat trials. These initial findings provide preliminary support that LTM activation is continuous, as the intrusion effect was not the same size for repeat and non-repeat trials. We conclude that researchers should repeat stimuli to increase the size of their effects and enhance how LTM representations interact with perception.

Long-term memory and working memory compete and cooperate to guide attention

Multiple types of memory guide attention: Both long-term memory (LTM) and working memory (WM) effectively guide visual search. Furthermore, both types of memories can capture attention automatically, even when detrimental to performance. It is less clear, however, how LTM and WM cooperate or compete to guide attention in the same task. In a series of behavioral experiments, we show that LTM and WM reliably cooperate to guide attention: Visual search is faster when both memories cue attention to the same spatial location (relative to when only one memory can guide attention). LTM and WM competed to guide attention in more limited circumstances: Competition only occurred when these memories were in different dimensions - particularly when participants searched for a shape and held an accessory color in mind. Finally, we found no evidence for asymmetry in either cooperation or competition: There was no evidence that WM helped (or hindered) LTM-guided search more than the other way around. This lack of asymmetry was found despite differences in LTM-guided and WM-guided search overall, and differences in how two LTMs and two WMs compete or cooperate with each other to guide attention. This work suggests that, even if only one memory is currently task-relevant, WM and LTM can cooperate to guide attention; they can also compete when distracting features are salient enough. This work elucidates interactions between WM and LTM during attentional guidance, adding to the literature on costs and benefits to attention from multiple active memories.

Using position rather than color at the traffic light - Covariation learning-based deviation from instructions in attention deficit/hyperactivity disorder

Based on instructions people can form task representations that shield relevant from seemingly irrelevant information. It has been documented that instructions can tie people to a particular way of performing a task despite that in principle a more efficient way could be learned and used. Since task shielding can lead to persistence of inefficient variants of task performance, it is relevant to test whether individuals with attention deficit/hyperactivity disorder (ADHD) - characterized by less task shielding - are more likely and quicker to escape a suboptimal instructed variant of performing a task. The paradigm used in this online study builds on the observation that in many environments different covarying features could be used to determine the appropriate response. For instance, as they approach a traffic light, drivers and pedestrians monitor the color (instructed stimulus feature) and/or the position of the signal (covarying stimulus feature, more efficient in case of reduced color sight). Similarly, we instructed participants to respond to the color of a stimulus without mentioning that color covaried with the position of the stimulus. In order to assess whether with practice participants would use the non-instructed feature position to an increasing extent, we compared reaction times and error rates for standard trials to trials in which color was either ambiguous or did not match the usual covariation. Results showed that the covariation learning task can be administered online to adult participants with and without ADHD. Performance differences suggested that with practice ADHD participants (n = 43 out of a total N = 245) might increase attention to non-instructed stimulus features. Yet, they used the non-instructed covarying stimulus feature to a similar extent as other participants. Together the results suggest that participants with ADHD do not lag behind in abandoning instructed task processing in favor of a learned alternative strategy.

Memory and Proactive Interference for spatially distributed items

Our ability to briefly retain information is often limited. Proactive Interference (PI) might contribute to these limitations (e.g., when items in recognition tests are difficult to reject after having appeared recently). In visual Working Memory (WM), spatial information might protect WM against PI, especially if encoding items together with their spatial locations makes item-location combinations less confusable than simple items without a spatial component. Here, I ask (1) if PI is observed for spatially distributed items, (2) if it arises among simple items or among item-location combinations, and (3) if spatial information affects PI at all. I show that, contrary to views that spatial information protects against PI, PI is reliably observed for spatially distributed items except when it is weak. PI mostly reflects items that appear recently or frequently as memory items, while occurrences as test items play a smaller role, presumably because their temporal context is easier to encode. Through mathematical modeling, I then show that interference occurs among simple items rather than item-location combinations. Finally, to understand the effects of spatial information, I separate the effects of (a) the presence and (b) the predictiveness of spatial information on memory and its susceptibility to PI. Memory is impaired when items are spatially distributed, but, depending on the analysis, unaffected by the predictiveness of spatial information. In contrast, the susceptibility to PI is unaffected by either manipulation. Visual memory is thus impaired by PI for spatially distributed items due to interference from recent memory items (rather than test items or item-location combinations).

The past, present, and future of selection history

The last ten years of attention research have witnessed a revolution, replacing a theoretical dichotomy (top-down vs. bottom-up control) with a trichotomy (biased by current goals, physical salience, and selection history). This third new mechanism of attentional control, selection history, is multifaceted. Some aspects of selection history must be learned over time whereas others reflect much more transient influences. A variety of different learning experiences can shape the attention system, including reward, aversive outcomes, past experience searching for a target, target‒non-target relations, and more. In this review, we provide an overview of the historical forces that led to the proposal of selection history as a distinct mechanism of attentional control. We then propose a formal definition of selection history, with concrete criteria, and identify different components of experience-driven attention that fit within this definition. The bulk of the review is devoted to exploring how these different components relate to one another. We conclude by proposing an integrative account of selection history centered on underlying themes that emerge from our review.

Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding

Our attention is critically important for what we remember. Prior measures of the relationship between attention and memory, however, have largely treated "attention" as a monolith. Here, across three experiments, we provide evidence for two dissociable aspects of attention that influence encoding into long-term memory. Using spatial cues together with a sensitive continuous report procedure, we find that long-term memory response error is affected by both trial-by-trial fluctuations of sustained attention and prioritization via covert spatial attention. Furthermore, using multivariate analyses of EEG, we track both sustained attention and spatial attention before stimulus onset. Intriguingly, even during moments of low sustained attention, there is no decline in the representation of the spatially attended location, showing that these two aspects of attention have robust but independent effects on long-term memory encoding. Finally, sustained and spatial attention predicted distinct variance in long-term memory performance across individuals. That is, the relationship between attention and long-term memory suggests a composite model, wherein distinct attentional subcomponents influence encoding into long-term memory. These results point toward a taxonomy of the distinct attentional processes that constrain our memories.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Premembering Experience: A Hierarchy of Time-Scales for Proactive Attention

Memories are about the past, but they serve the future. Memory research often emphasizes the former aspect: focusing on the functions that re-constitute (re-member) experience and elucidating the various types of memories and their interrelations, timescales, and neural bases. Here we highlight the prospective nature of memory in guiding selective attention, focusing on functions that use previous experience to anticipate the relevant events about to unfold-to "premember" experience. Memories of various types and timescales play a fundamental role in guiding perception and performance adaptively, proactively, and dynamically. Consonant with this perspective, memories are often recorded according to expected future demands. Using working memory as an example, we consider how mnemonic content is selected and represented for future use. This perspective moves away from the traditional representational account of memory toward a functional account in which forward-looking memory traces are informationally and computationally tuned for interacting with incoming sensory signals to guide adaptive behavior.

Incidental covariation learning leading to strategy change

As they approach a traffic light, drivers and pedestrians monitor the color (instructed stimulus feature) and/or the position of the signal (covarying stimulus feature) for response selection. Many studies have pointed out that instructions can effectively determine the stimulus features used for response selection in a task. This leaves open whether and how practice with a correlating alternative stimulus feature can lead to a strategy change from an instructed to a learned variant of performing the task. To address this question, we instructed participants to respond to the position of a stimulus within a reference frame, at the same time, during task performance, an unmentioned second stimulus feature, the color, covaried with stimulus position and allowed the use of an alternative response strategy. To assess the impact of the non-instructed stimulus feature of color on response selection throughout practice, the spatial position of the stimulus was ambiguous on some trials. Group average increases in color usage were based on a mixture of (1) participants who, despite extended practice on the covariation, exclusively relied on the instructed stimulus feature and (2) those who abruptly started to rely heavily on stimulus color to select responses in ambiguous trials. When the instructed and uninstructed feature predicted different actions, choices were still biased by the uninstructed color feature, albeit more weakly. A second experiment showed that the influence of color generalized across frequently and infrequently presented combinations of position and color. Strategy changes were accompanied by awareness in both experiments. The results suggest that incidental covariation learning can trigger spontaneous voluntary strategy change involving a re-configuration of the instructed task set.

Behavioral decoding of working memory items inside and outside the focus of attention

How we attend to our thoughts affects how we attend to our environment. Holding information in working memory can automatically bias visual attention toward matching information. By observing attentional biases on reaction times to visual search during a memory delay, it is possible to reconstruct the source of that bias using machine learning techniques and thereby behaviorally decode the content of working memory. Can this be done when more than one item is held in working memory? There is some evidence that multiple items can simultaneously bias attention, but the effects have been inconsistent. One explanation may be that items are stored in different states depending on the current task demands. Recent models propose functionally distinct states of representation for items inside versus outside the focus of attention. Here, we use behavioral decoding to evaluate whether multiple memory items-including temporarily irrelevant items outside the focus of attention-exert biases on visual attention. Only the single item in the focus of attention was decodable. The other item showed a brief attentional bias that dissipated until it returned to the focus of attention. These results support the idea of dynamic, flexible states of working memory across time and priority.

What Is Memory-Guided Attention? How Past Experiences Shape Selective Visuospatial Attention in the Present

What controls our attention? It is historically thought that there are two primary factors that determine selective attention: the perceptual salience of the stimuli and the goals based on the task at hand. However, this distinction doesn't neatly capture the varied ways our past experience can influence our ongoing mental processing. In this chapter, we aim to describe how past experience can be systematically characterized by different types of memory, and we outline experimental evidence suggesting how attention can then be guided by each of these different memory types. We highlight findings from human behavioral, neuroimaging, and neuropsychological work from the perspective of two related frameworks of human memory: the multiple memory systems (MMS) framework and the neural processing (NP) framework. The MMS framework underscores how memory can be separated based on consciousness (declarative and non-declarative memory), while the NP framework emphasizes different forms of memory as reflective of different brain processing modes (rapid encoding of flexible associations, slow encoding of rigid associations, and rapid encoding of single or unitized items). We describe how memory defined by these frameworks can guide our attention, even when they do not directly relate to perceptual salience or the goals concerning the current task. We close by briefly discussing theoretical implications as well as some interesting avenues for future research.

Five Factors that Guide Attention in Visual Search

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

The price of information: Increased inspection costs reduce the confirmation bias in visual search

In visual search, there is a confirmation bias such that attention is biased towards stimuli that match a target template, which has been attributed to covert costs of updating the templates that guide search [Rajsic, Wilson, & Pratt, 2015. Confirmation bias in visual search. Journal of Experimental Psychology: Human Perception and Performance. Advance online publication. doi:10.1037/xhp0000090]. In order to provide direct evidence for this speculation, the present study increased the cost of inspections in search by using gaze- and mouse-contingent searches, which restrict the manner in which information in search displays can be accrued, and incur additional motor costs (in the case of mouse-contingent searches). In a fourth experiment, we rhythmically mask elements in the search display to induce temporal inspection costs. Our results indicated that confirmation bias is indeed attenuated when inspection costs are increased. We conclude that confirmation bias results from the low-cost strategy of matching information to a single, concrete visual template, and that more sophisticated guidance strategies will be used when sufficiently beneficial. This demonstrates that search guidance itself comes at a cost, and that the form of guidance adopted in a given search depends on a comparison between guidance costs and the expected benefits of their implementation.