Attentional selection in visual perception, memory and action: a quest for cross-domain integration

Perception of direct gaze in a video-conference setting: the effects of position and size

A common problem in video conferences is gaze direction. In face-to-face communication, it is common that speaker and listener intermittently look at each other. In a video-conference setting, where multiple participants are on the screen, things are complicated and not necessarily optimal. If the listener feels looked at when the speaker looks into the camera, how tolerant is the listener for slight deviations? And does this depend on the position of the speaker’s tile on the screen, or the size of the tile? In a first experiment, participants from a student population judged whether they are looked at, while vertical gaze direction of the looker was varied. Furthermore, the position of the tile on the screen varied. The results showed that a slightly upward directed gaze was optimal for the direct gaze judgment, with a width of ± 4 degrees. Optimal gaze direction was somewhat higher for tiles at the bottom of the screen. A second experiment tested the effect of size on the perception of horizontal gaze directions. Size was found to increase the gaze cone. The paper concludes with some recommendations for a setup of video conference systems, optimized for perceived gaze contact.

Attentional biases to emotional scenes in schizophrenia: An eye-tracking study

Attentional biases to emotional information may play a key role in the onset and course of schizophrenia. The aim of this experiment was to examine the attentional processing of four emotional scenes in competition (happy, neutral, sad, threatening) in 53 patients with schizophrenia and 51 controls. The eye movements were recorded in a 20-seconds free-viewing task. The results were: (i) patients showed increased attention on threatening scenes, compared to controls, in terms of attentional engagement and maintenance; (ii) patients payed less attention to happy scenes than controls, in terms of attentional maintenance; (iii) whereas positive symptoms were associated with a late avoidance of sad scenes, negative symptoms were associated with heightened attention to threat. The findings suggest that a threat-related bias and a lack of sensitivity to positive information may represent an underlying psychological mechanism of schizophrenia. Importantly, schizophrenia symptoms modulated the attentional biases, which has aetiological and therapeutic implications.

Fixation durations in natural scene viewing are guided by peripheral scene content

Fixation durations provide insights into processing demands. We investigated factors controlling fixation durations during scene viewing in two experiments. In Experiment 1, we tested the degree to which fixation durations adapt to global scene processing difficulty by manipulating the contrast (from original contrast to isoluminant) and saturation (original vs. grayscale) of the entire scene. We observed longer fixation durations for lower levels of contrast, and longer fixation durations for grayscale than for color scenes. Thus fixation durations were globally slowed as visual information became more and more degraded, making scene processing increasingly more difficult. In Experiment 2, we investigated two possible sources for this slow-down. We used "checkerboard" stimuli in which unmodified patches alternated with patches from which luminance information had been removed (isoluminant patches). Fixation durations showed an inverted immediacy effect (longer, rather than shorter, fixation durations on unmodified patches) along with a parafoveal-on-foveal effect (shorter fixation durations, when an unmodified patch was fixated next). This effect was stronger when the currently fixated patch was isoluminant as opposed to unmodified. Our results suggest that peripheral scene information substantially affects fixation durations and are consistent with the notion of competition among the current and potential future fixation locations.

Reaching movements are attracted by stimuli that signal reward

When presented with a set of possible reach targets, the movement trajectory can reveal aspects of the underlying competition for action selection. Current goals and physical salience can affect the trajectory of reaching movements to be attracted towards a distractor. Some studies demonstrated that stimuli associated with reward can also cause an attraction when reaching towards the reward stimulus was previously rewarded and the reward stimulus was physically salient. Here we demonstrate that a non-salient stimulus that signals the availability of reward attracts reaching movements even when moving towards it was never necessary nor rewarded. Moreover, the attraction by reward is particularly evident with short-latency movements. We conclude that neither physical salience nor reinforcing the movement towards a stimulus is necessary for reward to gain priority in the selection for action.

Oculomotor capture by search-irrelevant features in visual working memory: on the crucial role of target-distractor similarity

When searching for varying targets in the environment, a target template has to be maintained in visual working memory (VWM). Recently, we showed that search-irrelevant features of a VWM template bias attention in an object-based manner, so that objects sharing such features with a VWM template capture the eyes involuntarily. Here, we investigated whether target-distractor similarity modulates capture strength. Participants saccaded to a target accompanied by a distractor. A single feature (e.g., shape) defined the target in each trial indicated by a cue, and the cue also varied in one irrelevant feature (e.g., color). The distractor matched the cue's irrelevant feature in half of the trials. Nine experiments showed that target-distractor similarity consistently influenced the degree of oculomotor capture. High target-distractor dissimilarity in the search-relevant feature reduced capture by the irrelevant feature (Experiments 1, 3, 6, 7). However, capture was reduced by high target-distractor similarity in the search-irrelevant feature (Experiments 1, 4, 5, 8). Strong oculomotor capture was observed if target-distractor similarity was reasonably low in the relevant and high in the irrelevant feature, irrespective of whether color or shape were relevant (Experiments 2 and 5). These findings argue for involuntary and object-based, top-down control by VWM templates, whereas its manifestation in oculomotor capture depends crucially on target-distractor similarity in relevant and irrelevant feature dimensions of the search object.

Do eye movements enhance visual memory retrieval?

When remembering an object at a given location, participants tend to return their gaze to that location even after the object has disappeared, known as Looking-at-Nothing (LAN). However, it is unclear whether LAN is associated with better memory performance. Previous studies reporting beneficial effects of LAN have often not systematically manipulated or assessed eye movements. We asked 20 participants to remember the location and identity of eight objects arranged in a circle, shown for 5 s. Participants were prompted to judge whether a location statement (e.g., "Star Right") was correct or incorrect, or referred to a previously unseen object. During memory retrieval, participants either fixated in the screen center or were free to move their eyes. Results reveal no difference in memory accuracy and response time between free-viewing and fixation while a LAN effect was found for saccades during free viewing, but not for microsaccades during fixation. Memory performance was better in those free-viewing trials in which participants made a saccade to the critical location, and scaled with saccade accuracy. These results indicate that saccade kinematics might be related to both memory performance and memory retrieval processes, but the strength of their link would differ between individuals and task demands.

The attractiveness of salient distractors to reaching movements is task dependent

Previous studies in visual attention and oculomotor research showed that a physically salient distractor does not always capture attention or the eyes. Under certain top-down task sets, a salient distractor can be actively suppressed, avoiding capture. Even though previous studies showed that reaching movements are also influenced by salient distractors, it is unclear if and how a mechanism of active suppression of distractors would affect reaching movements. Active suppression might also explain why some studies find reaching movements to curve towards a distractor, while others find reaching movements to curve away. In this study, we varied the top-down task set in two separate experiments by manipulating the certainty about the target location. Participants had to reach for a diamond present among three circles. In Experiments 1 and 3, participants had to search for the reach targets; hence, the target's location certainty was low. In Experiments 2 and 3, the target's location was cued before the reach; hence, the target's location certainty was high. We found that reaches curved towards the physically salient, color singleton, distractor in the search-to-reach task (Experiments 1 and 3), but not in the cued reach task (Experiments 2 and 3). Thus, the saliency of the distractor only attracted reaching movements when the certainty of the target's location was low. Our findings suggest that the attractiveness of physically salient distractors to reaching movements depends on the top-down task set. The results can be explained by the effect of active attentional suppression on the competition between movement plans.

High-precision voluntary movements are largely independent of preceding vertex potentials elicited by sudden sensory events

KEY POINTS

Salient and sudden sensory events generate a remarkably large response in the human brain, the vertex wave (VW). The VW is coupled with a modulation of a voluntarily-applied isometric force. In the present study, we tested whether the VW is also related to executing high-precision movements. The execution of a voluntary high-precision movement remains relatively independent of the brain activity reflected by the preceding VW. The apparent relationship between the positive VW and movement onset time is explained by goal-related but stimulus-independent neural activities. These results highlight the need to consider such goal-related but stimulus-independent neural activities when attempting to relate event-related potential amplitude with perceptual and behavioural performance.

ABSTRACT

Salient and fast-rising sensory events generate a large biphasic vertex wave (VW) in the human electroencephalogram (EEG). We recently reported that the VW is coupled with a modulation of concomitantly-applied isometric force. In the present study, in five experiments, we tested whether the VW is also related to high-precision visuomotor control. We obtained three results. First, the saliency-induced increase in VW amplitude was paralleled by a modulation in two of the five extracted movement parameters: a reduction in the onset time of the voluntary movement (P < 0.005) and an increase in movement accuracy (P < 0.005). Second, spontaneous trial-by-trial variability in vertex wave amplitude, for a given level of stimulus saliency, was positively correlated with movement onset time (P < 0.001 in four out of five experiments). Third, this latter trial-by-trial correlation was explained by a widespread EEG negativity independent of the occurrence of the positive VW, although overlapping in time with it. These results indicate that (i) the execution of a voluntary high-precision movement remains relatively independent of the neural processing reflected by the preceding VW, with (ii) the exception of movement onset time, for which saliency-based contextual effects are dissociated from trial-by-trial effects. These results also indicate that (iii) attentional effects can produce spurious correlations between event-related potentials (ERPs) and behavioural measures. Although sudden salient stimuli trigger characteristic EEG responses coupled with distinct reactive components within an ongoing isometric task, the results of the present study indicate that the execution of a subsequent voluntary movement appears largely protected from such saliency-based modulation, with the exception of movement onset time.

Task implementation and top-down control in continuous search

Evidence from continuous search suggests that targets are detected by default, whereas distractors are processed in considerable depth. These observations shed light on task implementation and top-down control. Task implementation builds on forming dynamic distractor models, based on continuous integration of distractor-related information. Top-down control builds on using these models for testing upcoming stimulus information.

Distractor Dwelling, Skipping, and Revisiting Determine Target Absent Performance in Difficult Visual Search

Some targets in visual search are more difficult to find than others. In particular, a target that is similar to the distractors is more difficult to find than a target that is dissimilar to the distractors. Efficiency differences between easy and difficult searches are manifest not only in target-present trials but also in target-absent trials. In fact, even physically identical displays are searched through with different efficiency depending on the searched-for target. Here, we monitored eye movements in search for a target similar to the distractors (difficult search) versus a target dissimilar to the distractors (easy search). We aimed to examine three hypotheses concerning the causes of differential search efficiencies in target-absent trials: (a) distractor dwelling (b) distractor skipping, and (c) distractor revisiting. Reaction times increased with target similarity which is consistent with existing theories and replicates earlier results. Eye movement data indicated guidance in target trials, even though search was very slow. Dwelling, skipping, and revisiting contributed to low search efficiency in difficult search, with dwelling being the strongest factor. It is argued that differences in dwell time account for a large amount of total search time differences.

Novelty biases attention and gaze in a surprise trial

While the classical distinction between task-driven and stimulus-driven biasing of attention appears to be a dichotomy at first sight, there seems to be a third category that depends on the contrast or discrepancy between active representations and the upcoming stimulus, and may be termed novelty, surprise, or prediction failure. For previous demonstrations of the discrepancy-attention link, stimulus-driven components (saliency) may have played a decisive role. The present study was conducted to evaluate the discrepancy-attention link in a display where novel and familiar stimuli are equated for saliency. Eye tracking was used to determine fixations on novel and familiar stimuli as a proxy for attention. Results show a prioritization of attention by the novel color, and a de-prioritization of the familiar color, which is clearly present at the second fixation, and spans over the next couple of fixations. Saliency, on the other hand, did not prioritize items in the display. The results thus reinforce the notion that novelty captures and binds attention.

When circles become triangular: how transsaccadic predictions shape the perception of shape

Human vision is characterized by a consistent pattern of saccadic eye movements. With each saccade, internal object representations change their retinal position and spatial resolution. This raises the question as to how peripheral perception is affected by imminent saccadic eye movements. Here, we suggest that saccades are accompanied by a prediction of their perceptual consequences (i.e., the foveation of the target object). Accordingly, peripheral perception should be biased toward previously associated foveal input. In this study, we first exposed participants to an altered visual stimulation where one object systematically changed its shape during saccades. Subsequently, participants had to judge the shape of briefly presented peripheral saccade targets. The results showed that targets were perceived as less curved for objects that previously changed from more circular in the periphery to more triangular in the fovea. Similarly, shapes were perceived as more curved for objects that previously changed from triangular to circular. Thus, peripheral perception seems to depend not solely on the current input but also on memorized experiences, enabling predictions about the perceptual consequences of saccadic eye movements.

Linking perception and action by structure or process? Toward an integrative perspective

Over the past decades cognitive neuroscience's renewed interest in action has intensified the search of principles explaining how the cognitive system links perception to action and vice versa. To date, at least two seemingly alternative approaches can be distinguished. Perception and action might be linked either by common representational structures, as assumed by the ideomotor approach, or by common attentional processes, as assumed by the attention approach. This article first reviews the evidence from different paradigms supporting each approach. It becomes clear that most studies selectively focus either on actions directed at goals outside the actors' perceptual range (supporting the ideomotor approach) or on actions directed at targets within the actors' perceptual range (supporting the attention approach). In a second step, I will try to reconcile both approaches by reviewing recent eye movement studies that abolish the classical combination of approach and goals under study. Demonstrating that both approaches cover target- as well as goal-directed actions, it is proposed that operations addressed in both conceptual frameworks interact with each other.

The interplay of attention and consciousness in visual search, attentional blink and working memory consolidation

Despite the acknowledged relationship between consciousness and attention, theories of the two have mostly been developed separately. Moreover, these theories have independently attempted to explain phenomena in which both are likely to interact, such as the attentional blink (AB) and working memory (WM) consolidation. Here, we make an effort to bridge the gap between, on the one hand, a theory of consciousness based on the notion of global workspace (GW) and, on the other, a synthesis of theories of visual attention. We offer a theory of attention and consciousness (TAC) that provides a unified neurocognitive account of several phenomena associated with visual search, AB and WM consolidation. TAC assumes multiple processing stages between early visual representation and conscious access, and extends the dynamics of the global neuronal workspace model to a visual attentional workspace (VAW). The VAW is controlled by executive routers, higher-order representations of executive operations in the GW, without the need for explicit saliency or priority maps. TAC leads to newly proposed mechanisms for illusory conjunctions, AB, inattentional blindness and WM capacity, and suggests neural correlates of phenomenal consciousness. Finally, the theory reconciles the all-or-none and graded perspectives on conscious representation.