Attentional landscapes in reaching and grasping

Peripheral and bimanual reaching in a stroke survivor with left visual neglect and extinction

Whether attentional deficits are accompanied by visuomotor impairments following posterior parietal lesions has been debated for quite some time. This single-case study investigated reaching in a stroke survivor (E.B.) with left visual neglect and visual extinction following right temporo-parietal-frontal strokes. Unlike most neglect patients, E.B. did not present left hemiparesis, homonymous hemianopia nor show evidence of motor neglect or extinction allowing us to examine, for the first time, if lateralised attentional deficits co-occur with deficits in peripheral and bimanual reaching. First, we found a classic optic ataxia field effect: E.B.'s accuracy was impaired when reaching to peripheral targets in her neglected left visual field (regardless of the hand used). Second, we found a larger bimanual cost for movement time in E.B. than controls when both hands reached to incongruent locations. E.B.'s visuomotor profile is similar to the one of patients with optic ataxia showing that attentional deficits are accompanied by visuomotor deficits in the affected field.

A spatial-attentional mechanism underlies action-related distortions of time judgment

Temporal binding has been understood as an illusion in timing judgment. When an action triggers an outcome (e.g. a sound) after a brief delay, the action is reported to occur later than if the outcome does not occur, and the outcome is reported to occur earlier than a similar outcome not caused by an action. We show here that an attention mechanism underlies the seeming illusion of timing judgment. In one method, participants watch a rotating clock hand and report event times by noting the clock hand position when the event occurs. We find that visual spatial attention is critically involved in shaping event time reports made in this way. This occurs because action and outcome events result in shifts of attention around the clock rim, thereby biasing the perceived location of the clock hand. Using a probe detection task to measure attention, we show a difference in the distribution of visual spatial attention between a single-event condition (sound only or action only) and a two-event agency condition (action plus sound). Participants accordingly report the timing of the same event (the sound or the action) differently in the two conditions: spatial attentional shifts masquerading as temporal binding. Furthermore, computational modeling based on the attention measure can reproduce the temporal binding effect. Studies that use time judgment as an implicit marker of voluntary agency should first discount the artefactual changes in event timing reports that actually reflect differences in spatial attention. The study also has important implications for related results in mental chronometry obtained with the clock-like method since Wundt, as attention may well be a critical confounding factor in the interpretation of these studies.

Comparing eye-hand coordination between controller-mediated virtual reality, and a real-world object interaction task

Virtual reality (VR) technology has advanced significantly in recent years, with many potential applications. However, it is unclear how well VR simulations mimic real-world experiences, particularly in terms of eye-hand coordination. This study compares eye-hand coordination from a previously validated real-world object interaction task to the same task re-created in controller-mediated VR. We recorded eye and body movements and segmented participants' gaze data using the movement data. In the real-world condition, participants wore a head-mounted eye tracker and motion capture markers and moved a pasta box into and out of a set of shelves. In the VR condition, participants wore a VR headset and moved a virtual box using handheld controllers. Unsurprisingly, VR participants took longer to complete the task. Before picking up or dropping off the box, participants in the real world visually fixated the box about half a second before their hand arrived at the area of action. This 500-ms minimum fixation time before the hand arrived was preserved in VR. Real-world participants disengaged their eyes from the box almost immediately after their hand initiated or terminated the interaction, but VR participants stayed fixated on the box for much longer after it was picked up or dropped off. We speculate that the limited haptic feedback during object interactions in VR forces users to maintain visual fixation on objects longer than in the real world, altering eye-hand coordination. These findings suggest that current VR technology does not replicate real-world experience in terms of eye-hand coordination.

Dissociation of perception and motor execution of lower limb in multi-directional movements

Estimating the action capability is vital for humans to move their bodies successfully. Researchers have proposed reachability as an overestimation of motor abilities by judging unreachable distances as reachable. The existing literature has mainly investigated the sagittal direction, but multi-directional reachability is unexplored. This study examined the relationship between perception and motor using the reaching of the lower limbs in multiple directions. We asked 16 adults to reach targets projected onto the floor at 21 locations (seven directions and three distances) to estimate the reaching time. We found that the reaching time slowed as the direction increased toward the contralateral side, but the subjective reaching time did not change with direction. Multiple regression analysis showed that the subjective reaching time could be calculated accurately, mainly using the duration from the toe leaving the ground to movement completion. These results suggest that changes in direction may not be perceived precisely by the motor system of the lower limbs and that the subjective reaching time was strongly affected by the time after the toe left the ground. Our findings provide novel insights into the relationship between motor and perception in multiple directions, which may provide a new strategy for the maximal performance of lower-limb movement.

Action space restructures visual working memory in prefrontal cortex

Visual working memory enables flexible behavior by decoupling sensory stimuli from behavioral actions. While previous studies have predominantly focused on the storage component of working memory, the role of future actions in shaping working memory remains unknown. To answer this question, we used two working memory tasks that allowed the dissociation of sensory and action components of working memory. We measured behavioral performance and neuronal activity in the macaque prefrontal cortex area, frontal eye fields. We show that the action space reshapes working memory, as evidenced by distinct patterns of memory tuning and attentional orienting between the two tasks. Notably, neuronal activity during the working memory period predicted future behavior and exhibited mixed selectivity in relation to the sensory space but linear selectivity relative to the action space. This linear selectivity was achieved through the rapid transformation from sensory to action space and was subsequently maintained as a stable cross-temporal population activity pattern. Combined, we provide direct physiological evidence of the action-oriented nature of frontal eye field neurons during memory tasks and demonstrate that the anticipation of behavioral outcomes plays a significant role in transforming and maintaining the contents of visual working memory.

Attention in flux

Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.

Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize

When we look at an image, its features are represented in our visual system in a highly distributed manner, calling for a mechanism that binds them into coherent object representations. There have been different proposals for the neuronal mechanisms that can mediate binding. One hypothesis is that binding is achieved by oscillations that synchronize neurons representing features of the same perceptual object. This view allows separate communication channels between different brain areas. Another hypothesis is that binding of features that are represented in different brain regions occurs when the neurons in these areas that respond to the same object simultaneously enhance their firing rate, which would correspond to directing object-based attention to these features. This review summarizes evidence in favor of and against these two hypotheses, examining the neuronal correlates of binding and assessing the time course of perceptual grouping. I conclude that enhanced neuronal firing rates bind features into coherent object representations, whereas oscillations and synchrony are unrelated to binding.

Functional connectivity fingerprints of the frontal eye field and inferior frontal junction suggest spatial versus nonspatial processing in the prefrontal cortex

Neuroimaging evidence suggests that the frontal eye field (FEF) and inferior frontal junction (IFJ) govern the encoding of spatial and nonspatial (such as feature- or object-based) representations, respectively, both during visual attention and working memory tasks. However, it is still unclear whether such contrasting functional segregation is also reflected in their underlying functional connectivity patterns. Here, we hypothesized that FEF has predominant functional coupling with spatiotopically organized regions in the dorsal ('where') visual stream whereas IFJ has predominant functional connectivity with the ventral ('what') visual stream. We applied seed-based functional connectivity analyses to temporally high-resolving resting-state magnetoencephalography (MEG) recordings. We parcellated the brain according to the multimodal Glasser atlas and tested, for various frequency bands, whether the spontaneous activity of each parcel in the ventral and dorsal visual pathway has predominant functional connectivity with FEF or IFJ. The results show that FEF has a robust power correlation with the dorsal visual pathway in beta and gamma bands. In contrast, anterior IFJ (IFJa) has a strong power coupling with the ventral visual stream in delta, beta and gamma oscillations. Moreover, while FEF is phase-coupled with the superior parietal lobe in the beta band, IFJa is phase-coupled with the middle and inferior temporal cortex in delta and gamma oscillations. We argue that these intrinsic connectivity fingerprints are congruent with each brain region's function. Therefore, we conclude that FEF and IFJ have dissociable connectivity patterns that fit their respective functional roles in spatial versus nonspatial top-down attention and working memory control.

Turning Attention Inside Out: How Working Memory Serves Behavior

Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.

Nine insights from internet engineering that help us understand brain network communication

Philosophers have long recognized the value of metaphor as a tool that opens new avenues of investigation. By seeing brains as having the goal of representation, the computer metaphor in its various guises has helped systems neuroscience approach a wide array of neuronal behaviors at small and large scales. Here I advocate a complementary metaphor, the internet. Adopting this metaphor shifts our focus from computing to communication, and from seeing neuronal signals as localized representational elements to seeing neuronal signals as traveling messages. In doing so, we can take advantage of a comparison with the internet's robust and efficient routing strategies to understand how the brain might meet the challenges of network communication. I lay out nine engineering strategies that help the internet solve routing challenges similar to those faced by brain networks. The internet metaphor helps us by reframing neuronal activity across the brain as, in part, a manifestation of routing, which may, in different parts of the system, resemble the internet more, less, or not at all. I describe suggestive evidence consistent with the brain's use of internet-like routing strategies and conclude that, even if empirical data do not directly implicate internet-like routing, the metaphor is valuable as a reference point for those investigating the difficult problem of network communication in the brain and in particular the problem of routing.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Staggered Onsets of Processing Relevant and Irrelevant Stimulus Features Produce Different Dynamics of Congruency Effects

The dynamics of congruency effects in conflict tasks can be analyzed by means of delta plots which depict the reaction-time differences between incongruent and congruent conditions across the quantiles of the reaction-time distributions. Delta plots exhibit a variety of different shapes. Here we test the hypothesis that staggered onsets of processing task-relevant and task-irrelevant features for response selection (together with a declining influence of the irrelevant feature) produce such variety. For this purpose, staggered onsets were implemented in two extensions of the Leaky, Competing Accumulator model. We show the cardinal capability of these models to produce different shapes of delta plots with different assumptions about temporal offsets between processing relevant and irrelevant stimulus features. Applying the models to experimental data, we first show that they can reproduce the delta plots observed with a conflict task with stimulus size as the irrelevant feature. For this task congruency effects are delayed and appear only at longer reaction times. Second, we fit the models to the results of two new Simon-task experiments with an experimentally controlled temporal offset in addition to the internal one. The experimentally induced variations of the shape of delta plots for this task could be reasonably well fitted by one of the two models that assumed an early start of response selection as soon as either the relevant or the irrelevant stimulus feature becomes available. We conclude that delta plots are crucially shaped by staggered onsets of processing relevant and irrelevant features for response selection.

Attention-Setting and Human Mental Function

This article provides an introduction to experimental research on top-down human attention in complex scenes, written for cognitive scientists in general. We emphasize the major effects of goals and intention on mental function, measured with behavioral experiments. We describe top-down attention as an open category of mental actions that initiates particular task sets, which are assembled from a wide range of mental processes. We call this attention-setting. Experiments on visual search, task switching, and temporal attention are described and extended to the important human time scale of seconds.

Visual attention around a hand location localized by proprioceptive information

Facilitation of visual processing has been reported in the space near the hand. To understand the underlying mechanism of hand proximity attention, we conducted experiments that isolated hand-related effects from top–down attention, proprioceptive information from visual information, the position effect from the influence of action, and the distance effect from the peripersonal effect. The flash-lag effect was used as an index of attentional modulation. Because the results showed that the flash-lag effect was smaller at locations near the hand, we concluded that there was a facilitation effect of the visual stimuli around the hand location identified through proprioceptive information. This was confirmed by conventional reaction time measures. We also measured steady-state visual evoked potential (SSVEP) in order to investigate the spatial properties of hand proximity attention and top–down attention. The results showed that SSVEP reflects the effect of top–down attention but not that of hand proximity attention. This suggests that the site of hand proximity attention is at a later stage of visual processing, assuming that SSVEP responds to neural activities at the early stages. The results of left-handers differed from those of right-handers, and this is discussed in relation to handedness variation.

Eye and hand movements disrupt attentional control

Voluntary attentional control is the ability to selectively focus on a subset of visual information in the presence of other competing stimuli–a marker of cognitive control enabling flexible, goal-driven behavior. To test its robustness, we contrasted attentional control with the most common source of attentional orienting in daily life: attention shifts prior to goal-directed eye and hand movements. In a multi-tasking paradigm, human participants attended at a location while planning eye or hand movements elsewhere. Voluntary attentional control suffered with every simultaneous action plan, even under reduced task difficulty and memory load–factors known to interfere with attentional control. Furthermore, the performance cost was limited to voluntary attention: We observed simultaneous attention benefits at two movement targets without attentional competition between them. This demonstrates that the visual system allows for the concurrent representation of multiple attentional foci. Since attentional control is extremely fragile and dominated by premotor attention shifts, we propose that action-driven selection plays the superordinate role for visual selection.

Connectivity modulations induced by reach&grasp movements: a multidimensional approach

Reach&grasp requires highly coordinated activation of different brain areas. We investigated whether reach&grasp kinematics is associated to EEG-based networks changes. We enrolled 10 healthy subjects. We analyzed the reach&grasp kinematics of 15 reach&grasp movements performed with each upper limb. Simultaneously, we obtained a 64-channel EEG, synchronized with the reach&grasp movement time points. We elaborated EEG signals with EEGLAB 12 in order to obtain event related synchronization/desynchronization (ERS/ERD) and lagged linear coherence between Brodmann areas. Finally, we evaluated network topology via sLORETA software, measuring network local and global efficiency (clustering and path length) and the overall balance (small-worldness). We observed a widespread ERD in α and β bands during reach&grasp, especially in the centro-parietal regions of the hemisphere contralateral to the movement. Regarding functional connectivity, we observed an α lagged linear coherence reduction among Brodmann areas contralateral to the arm involved in the reach&grasp movement. Interestingly, left arm movement determined widespread changes of α lagged linear coherence, specifically among right occipital regions, insular cortex and somatosensory cortex, while the right arm movement exerted a restricted contralateral sensory-motor cortex modulation. Finally, no change between rest and movement was found for clustering, path length and small-worldness. Through a synchronized acquisition, we explored the cortical correlates of the reach&grasp movement. Despite EEG perturbations, suggesting that the non-dominant reach&grasp network has a complex architecture probably linked to the necessity of a higher visual control, the pivotal topological measures of network local and global efficiency remained unaffected.

A Neural Dynamic Model of the Perceptual Grounding of Spatial and Movement Relations

How does the human brain link relational concepts to perceptual experience? For example, a speaker may say "the cup to the left of the computer" to direct the listener's attention to one of two cups on a desk. We provide a neural dynamic account for both perceptual grounding, in which relational concepts enable the attentional selection of objects in the visual array, and for the generation of descriptions of the visual array using relational concepts. In the model, activation in neural populations evolves dynamically under the influence of both inputs and strong interaction as formalized in dynamic field theory. Relational concepts are modeled as patterns of connectivity to perceptual representations. These generalize across the visual array through active coordinate transforms that center the representation of target objects in potential reference objects. How the model perceptually grounds or generates relational descriptions is probed in 104 simulations that systematically vary the spatial and movement relations employed, the number of feature dimensions used, and the number of matching and nonmatching objects. We explain how sequences of decisions emerge from the time- and state-continuous neural dynamics, how relational hypotheses are generated and either accepted or rejected, followed by the selection of new objects or the generation of new relational hypotheses. Its neural realism distinguishes the model from information processing accounts, its capacity to autonomously generate sequences of processing steps distinguishes it from deep neural network accounts. The model points toward a neural dynamic theory of higher cognition.

Evaluating the Influence of Visual Attentional Tracking on Pointing Movement Precision

Multiple object tracking (MOT) and goal-directed movements are both based on attentional processes. This study focused on the instant effect of attentive tracking with respect to pointing performance. We measured the precision of pointing to the targets of an MOT task immediately after the tracking period when all the objects were still in motion, and to the precision of pointing to moving objects in the task without attentional tracking. The results demonstrated that an increase in the attentional tracking load was the primary factor that decreased pointing precision, although visual load may also contribute to this influence. We also manipulated the relationship between the number of targets and the density of the MOT display (the total number of objects displayed), which affected the MOT accuracy but not the pointing precision.

The essential role of recurrent processing for figure-ground perception in mice

The segregation of figures from the background is an important step in visual perception. In primary visual cortex, figures evoke stronger activity than backgrounds during a delayed phase of the neuronal responses, but it is unknown how this figure-ground modulation (FGM) arises and whether it is necessary for perception. Here, we show, using optogenetic silencing in mice, that the delayed V1 response phase is necessary for figure-ground segregation. Neurons in higher visual areas also exhibit FGM and optogenetic silencing of higher areas reduced FGM in V1. In V1, figures elicited higher activity of vasoactive intestinal peptide-expressing (VIP) interneurons than the background, whereas figures suppressed somatostatin-positive interneurons, resulting in an increased activation of pyramidal cells. Optogenetic silencing of VIP neurons reduced FGM in V1, indicating that disinhibitory circuits contribute to FGM. Our results provide insight into how lower and higher areas of the visual cortex interact to shape visual perception.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Pre-saccadic attention spreads to stimuli forming a perceptual group with the saccade target

The pre-saccadic attention shift-a rapid increase in visual sensitivity at the target-is an inevitable precursor of saccadic eye movements. Saccade targets are often parts of the objects that are of interest to the active observer. Although the link between saccades and covert attention shifts is well established, it remains unclear if pre-saccadic attention selects the location of the eye movement target or rather the entire object that occupies this location. Indeed, several neurophysiological studies suggest that attentional modulations of neural activity in visual cortex spreads across parts of objects (e.g., elements grouped by Gestalt principles) that contain the target location of a saccade. To understand the nature of pre-saccadic attentional selection, we examined how visual sensitivity, measured in a challenging orientation discrimination task, changes during saccade preparation at locations that are perceptually grouped with the saccade target. In Experiment 1, using grouping by color in a delayed-saccade task, we found no consistent spread of attention to locations that formed a perceptual group with the saccade target. However, performance depended on the side of the stimulus arrangement relative to the saccade target location, an effect we discuss with respect to attentional momentum. In Experiment 2, employing stronger perceptual grouping cues (color and motion) and an immediate-saccade task, we obtained a reliable grouping effect: Attention spread to locations that were perceptually grouped with the saccade target while saccade preparation was underway. We also replicated the side effect observed in Experiment 1. These results provide evidence that the pre-saccadic attention spreads beyond the target location along the saccade direction, and selects scene elements that-based on Gestalt criteria-are likely to belong to the same object as the saccade target.

Action affects perception through modulation of attention

The present study explored the origin of perceptual changes repeatedly observed in the context of actions. In Experiment 1, participants tried to hit a circular target with a stylus movement under restricted feedback conditions. We measured the perception of target size during action planning and observed larger estimates for larger movement distances. In Experiment 2, we then tested the hypothesis that this action specific influence on perception is due to changes in the allocation of spatial attention. For this purpose, we replaced the hitting task by conditions of focused and distributed attention and measured the perception of the former target stimulus. The results revealed changes in the perceived stimulus size very similar to those observed in Experiment 1. These results indicate that action’s effects on perception root in changes of spatial attention.

Simple action planning can affect attentional allocation in subsequent visual search

Previous research has shown that mental representations of actions can influence performance on relevant tasks or dimensions even when there is no overt execution of the action. In this study, we examined whether cognitive processes prior to the physical execution of an action can elicit attentional bias towards irrelevant tasks or dimensions of that action. Participants performed two independent tasks-an action task and a search task-where they were instructed to plan an action and execute the action following the visual search task. We found that the same features of the object were prioritized in the subsequent search task when participants had planned an action response on the object in comparison to when they had not. This effect occurred even when the feature was irrelevant to the tasks or required action. Furthermore, the effect of action planning without physical response was found to be comparable to the effect of physical response. These results suggest that planning of a simple action can induce attentional bias to irrelevant features of objects even without physical action.

The brain dynamics of architectural affordances during transition

Action is a medium of collecting sensory information about the environment, which in turn is shaped by architectural affordances. Affordances characterize the fit between the physical structure of the body and capacities for movement and interaction with the environment, thus relying on sensorimotor processes associated with exploring the surroundings. Central to sensorimotor brain dynamics, the attentional mechanisms directing the gating function of sensory signals share neuronal resources with motor-related processes necessary to inferring the external causes of sensory signals. Such a predictive coding approach suggests that sensorimotor dynamics are sensitive to architectural affordances that support or suppress specific kinds of actions for an individual. However, how architectural affordances relate to the attentional mechanisms underlying the gating function for sensory signals remains unknown. Here we demonstrate that event-related desynchronization of alpha-band oscillations in parieto-occipital and medio-temporal regions covary with the architectural affordances. Source-level time-frequency analysis of data recorded in a motor-priming Mobile Brain/Body Imaging experiment revealed strong event-related desynchronization of the alpha band to originate from the posterior cingulate complex, the parahippocampal region as well as the occipital cortex. Our results firstly contribute to the understanding of how the brain resolves architectural affordances relevant to behaviour. Second, our results indicate that the alpha-band originating from the occipital cortex and parahippocampal region covaries with the architectural affordances before participants interact with the environment, whereas during the interaction, the posterior cingulate cortex and motor areas dynamically reflect the affordable behaviour. We conclude that the sensorimotor dynamics reflect behaviour-relevant features in the designed environment.

Attention updates the perceived position of moving objects

The information used by conscious perception may differ from that which drives certain actions. A dramatic illusion caused by an object's internal texture motion has been put forward as one example. The motion causes an illusory position shift that accumulates over seconds into a large effect, but targeting of the grating for a saccade (a rapid eye movement) is not affected by this illusion. While this has been described as a dissociation between perception and action, an alternative explanation is that rather than saccade targeting having privileged access to the correct position, a shift of attention that precedes saccades resets the accumulated illusory position shift to zero. In support of this possibility, we found that the accumulation of illusory position shift can be reset by transients near the moving object, creating an impression of the object returning to near its actual position. Repetitive luminance changes of the object also resulted in reset of the accumulation, but less so when attention to the object was reduced by a concurrent digit identification task. Finally, judgments of the object's positions around the time of saccade onset reflected the veridical rather than the illusory position. These results suggest that attentional shifts, including those preceding saccades, can update the perceived position of moving objects and mediate the previously reported dissociation between conscious perception and saccades.

Selective attention to real-world objects drives their emotional appraisal

Attentional manipulations have been shown to influence subsequent evaluations of objects and images. For example, images used as distractors in a visual search task are subsequently rated more negatively than are target images. One powerful manipulation of attention occurs when we plan and execute movements toward objects in our environment. Here, in two experiments, we show that selective attention to real-world objects subsequently improves emotional appraisal of those objects-an effect we term "target appreciation." Participants were presented with abstract images on three-dimensional objects, and were cued to either reach and grasp one of the two objects, or to respond to the cued object with a keyboard. Images presented on target objects were appraised more positively when compared with novel images. In contrast, images associated with obstacles or distractor objects were not appraised differently than novel images, despite the attentional suppression thought to be required to successfully avoid or ignore these objects. We speculate that this automatic appreciation of the objects of selective attention may be adaptive for organisms acting in complex environments.

Looking ahead: When do you find the next item in foraging visual search?

Many real-world visual tasks involve searching for multiple instances of a target (e.g., picking ripe berries). What strategies do observers use when collecting items in this type of search? Do they wait to finish collecting the current item before starting to look for the next target, or do they search ahead for future targets? We utilized behavioral and eye-tracking measures to distinguish between these two possibilities in foraging search. Experiment 1 used a color wheel technique in which observers searched for T shapes among L shapes while all items independently cycled through a set of colors. Trials were abruptly terminated, and observers reported both the color and location of the next target that they intended to click. Using observers’ color reports to infer target-finding times, we demonstrate that observers found the next item before the time of the click on the current target. We validated these results in Experiment 2 by recording fixation locations around the time of each click. Experiment 3 utilized a different procedure, in which all items were intermittently occluded during the trial. We then calculated a distribution of when targets were visible around the time of each click, allowing us to infer when they were most likely found. In a fourth and final experiment, observers indicated the locations of multiple future targets after the search was abruptly terminated. Together, our results provide converging evidence to demonstrate that observers can find the next target before collecting the current target and can typically forage one to two items ahead.

An exploratory study of gaze behaviour in young adults with developmental coordination disorder

Developmental Coordination Disorder (DCD) is characterized by substantial difficulties with motor coordination to the extent that it has a clear impact on the daily functioning of those who suffer from the disorder. Laboratory-based research indicated impaired oculomotor control in individuals with DCD. However, it is not clear how these oculomotor problems contribute to control and coordination in daily tasks. This study explored differences and similarities in gaze behaviour during reading and cup stacking between young adults with DCD and their matched typically developing counterparts (TD; aged 20-23 years). Gaze behaviour was recorded using eye-tracking, and hand movements were registered using a digital camera. Results of the reading tasks demonstrated similar behaviour between the groups, apart from a lower number of characters recorded per fixation in the DCD group. In cup stacking, the individuals with DCD were slower than their counterparts when three cups had to be displaced to a central target using the dominant hand. The gaze strategy of individuals with DCD involved systematic fixations on the cup or target prior to the hand movement to that cup or target, whereas these alternating saccades between cup and target were less obvious in the TD group. In the bimanual stacking task, where a pyramid of six cups had to be built on a central target using both hands, both groups mainly fixated the central target for the whole duration of the task, without distinct differences in gaze behaviour and duration of performance between individuals with and those without DCD. In conclusion, gaze behaviour of young adults with DCD shows differences from that of their typically developing counterparts that may be related to underlying oculomotor deficits in some but not all daily tasks.

Visual working memory and action: Functional links and bi-directional influences

Working memory bridges perception to action over extended delays, enabling flexible goal-directed behaviour. To date, studies of visual working memory – concerned with detailed visual representations such as shape and colour – have considered visual memory predominantly in the context of visual task demands, such as visual identification and search. Another key purpose of visual working memory is to directly inform and guide upcoming actions. Taking this as a starting point, I review emerging evidence for the pervasive bi-directional links between visual working memory and (planned) action, and discuss these links from the perspective of their common goal of enabling flexible and precise behaviour.

Gaze behaviour during walking in young adults with developmental coordination disorder

BACKGROUND

Individuals with Developmental Coordination Disorder (DCD) experience difficulty with motor coordination and this affects their daily functioning. Research indicated inferior visuospatial processing and oculomotor control in DCD. As visual information is essential for locomotor control, more insight in the gaze behaviour of this population during walking is required and crucial for gaze training interventions as a possible means to improve daily functioning of children and adults with DCD.

AIM

This study explored differences and similarities in gaze behaviour during walking between typically developing young adults and those with DCD.

METHODS AND PROCEDURES

Ten young adults with DCD (age: 22.13 ± 0.64) and ten typically developing individuals (age: 22.00 ± 1.05) completed a walking task in which they had to place their feet on irregularly placed targets wearing eye tracking glasses.

OUTCOMES AND RESULTS

Individuals with DCD walked slower and demonstrated a different gaze strategy compared to their neurotypical peers as they fixated almost each and every target sequentially. Typically developing individuals, on the other hand, directed gaze further along the path and often fixated areas around the targets.

CONCLUSIONS AND IMPLICATIONS

Despite adequate walking performance in daily situations in young adults with DCD, fundamental control deficits persist into adulthood. WHAT THIS PAPER ADDS?: This paper is the first to demonstrate differences in gaze behaviour between young adults with DCD and typically developing individuals in a task that resembles a task of daily living, as previous research focused on laboratory tasks. This is a valuable finding as DCD has a clear impact on the daily life. Furthermore, this study demonstrated that the fundamental control deficits of DCD persist into adulthood despite frequent performance and practice of these daily tasks. Lastly, these findings might contribute to the therapeutic potential of gaze training interventions to improve the daily functioning of children and adults with DCD.

Spatial attention enhances cortical tracking of quasi-rhythmic visual stimuli

Successfully interpreting and navigating our natural visual environment requires us to track its dynamics constantly. Additionally, we focus our attention on behaviorally relevant stimuli to enhance their neural processing. Little is known, however, about how sustained attention affects the ongoing tracking of stimuli with rich natural temporal dynamics. Here, we used MRI-informed source reconstructions of magnetoencephalography (MEG) data to map to what extent various cortical areas track concurrent continuous quasi-rhythmic visual stimulation. Further, we tested how top-down visuo-spatial attention influences this tracking process. Our bilaterally presented quasi-rhythmic stimuli covered a dynamic range of 4-20 ​Hz, subdivided into three distinct bands. As an experimental control, we also included strictly rhythmic stimulation (10 vs 12 ​Hz). Using a spectral measure of brain-stimulus coupling, we were able to track the neural processing of left vs. right stimuli independently, even while fluctuating within the same frequency range. The fidelity of neural tracking depended on the stimulation frequencies, decreasing for higher frequency bands. Both attended and non-attended stimuli were tracked beyond early visual cortices, in ventral and dorsal streams depending on the stimulus frequency. In general, tracking improved with the deployment of visuo-spatial attention to the stimulus location. Our results provide new insights into how human visual cortices process concurrent dynamic stimuli and provide a potential mechanism - namely increasing the temporal precision of tracking - for boosting the neural representation of attended input.

Is congruent movement training more effective than standard visual scanning therapy to ameliorate symptoms of visuospatial neglect? Study protocol of a randomised control trial

INTRODUCTION

Approximately 30% of all patients with stroke show visuospatial neglect (VSN). Currently, visual scanning therapy (VST) is applied in clinical settings to attenuate neglect symptoms. VST builds on the premise that eye movements to the affected hemifield lead to a concurrent shift of visual attention. Congruent movements with different effectors of the motor system, for example, eye and hand, can produce an even larger boost of attention compared with a single effector. This congruency principle may produce a powerful bias in the motor system, which may counteract the pathological biases in the attentional system of neglect patients. Therefore, an intervention with congruent eye and hand movements may result in greater attenuation of neglect compared with an intervention with single eye movements as applied in standard VST. The current randomised controlled trial will investigate the beneficial effects of this updated version of VST by comparing changes in performance on standard neuropsychological neglect tasks and severity of neglect in activities of daily living.

METHODS AND ANALYSIS

Thirty VSN patients in the subacute phase poststroke onset will be randomly assigned to one of two groups: congruent eye and hand movement training (experimental group) versus standard VST (control group). Each patient will receive 10 sessions of training, 30 min each, within 2 weeks. Performance on standard neuropsychological neglect tasks, a visual discrimination task, severity of neglect in ADL and eye movement characteristics before and after intervention will be compared for and between both groups.

ETHICS AND DISSEMINATION

This study has been approved by the ethical committee of the University Medical Centre Utrecht. All subjects will participate voluntarily and will give written informed consent. Results of this study will be published in peer-reviewed scientific journals and presented at international conferences.

TRIAL REGISTRATION NUMBER

NTR7005.

Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.

Dynamics of visual attention revealed in foraging tasks

Visual search tasks play a key role in theories of visual attention. But single-target search tasks may provide only a snapshot of attentional orienting. Foraging tasks with multiple targets of different types arguably provide a closer analogy to everyday attentional processing. Set-size effects have in the literature formed the basis for inferring how attention operates during visual search. We therefore measured the effects of absolute set-size (constant target-distractor ratio) and relative set-size (constant set-size but target-distractor ratio varies) on foraging patterns during "feature" foraging (targets differed from distractors on a single feature) and "conjunction" foraging (targets differed from distractors on a combination of two features). Patterns of runs of same target-type selection were similar regardless of whether absolute or relative set-size varied: long sequential runs during conjunction foraging but rapid switching between target types during feature foraging. But although foraging strategies differed between feature and conjunction foraging, surprisingly, intertarget times throughout foraging trials did not differ much between the conditions. Typical response time by set-size patterns for single-target visual search tasks were only observed for the last target during foraging. Furthermore, the foraging patterns within trials involved several distinct phases, that may serve as markers of particular attentional operations. Foraging tasks provide a remarkably intricate picture of attentional selection, far more detailed than traditional single-target visual search tasks, and well-known theories of visual attention have difficulty accounting for key aspects of the observed foraging patterns. Finally, we discuss how theoretical conceptions of attention could be modified to account for these effects.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Updating spatial working memory in a dynamic visual environment

The present review describes recent developments regarding the role of the eye movement system in representing spatial information and keeping track of locations of relevant objects. First, we discuss the active vision perspective and why eye movements are considered crucial for perception and attention. The second part focuses on the question of how the oculomotor system is used to represent spatial attentional priority, and the role of the oculomotor system in maintenance of this spatial information. Lastly, we discuss recent findings demonstrating rapid updating of information across saccadic eye movements. We argue that the eye movement system plays a key role in maintaining and rapidly updating spatial information. Furthermore, we suggest that rapid updating emerges primarily to make sure actions are minimally affected by intervening eye movements, allowing us to efficiently interact with the world around us.

Attentional Weighting in the Face Processing Network: A Magnetic Response Image-guided Magnetoencephalography Study Using Multiple Cyclic Entrainments

We recorded magnetoencephalography using a neural entrainment paradigm with compound face stimuli that allowed for entraining the processing of various parts of a face (eyes, mouth) as well as changes in facial identity. Our magnetic response image-guided magnetoencephalography analyses revealed that different subnodes of the human face processing network were entrained differentially according to their functional specialization. Whereas the occipital face area was most responsive to the rate at which face parts (e.g., the mouth) changed, and face patches in the STS were mostly entrained by rhythmic changes in the eye region, the fusiform face area was the only subregion that was strongly entrained by the rhythmic changes in facial identity. Furthermore, top-down attention to the mouth, eyes, or identity of the face selectively modulated the neural processing in the respective area (i.e., occipital face area, STS, or fusiform face area), resembling behavioral cue validity effects observed in the participants' RT and detection rate data. Our results show the attentional weighting of the visual processing of different aspects and dimensions of a single face object, at various stages of the involved visual processing hierarchy.

Adults with probable developmental coordination disorder selectively process early visual, but not tactile information during action preparation. An electrophysiological study

Developmental coordination disorder (DCD) is a neurodevelopmental condition affecting motor coordination in children and adults. Here, EEG signals elicited by visual and tactile stimuli were recorded while adult participants with and without probable DCD (pDCD) performed a motor task. The task cued reaching movements towards a location in visible peripersonal space as well as an area of unseen personal space. Event-related potentials elicited by visual and tactile stimuli revealed that visual processing was strongly affected by movement preparation in the pDCD group, even more than in controls. However, in contrast to the controls, tactile processing in unseen space was unaffected by movement preparation in the pDCD group. The selective use of sensory information from vision and proprioception is fundamental for the adaptive control of movements, and these findings suggest that this is impaired in DCD. Additionally, the pDCD group showed attenuated motor rhythms (beta: 13-30 Hz) over sensorimotor regions following cues to prepare movements towards unseen personal space. The results reveal that individuals with pDCD exhibit differences in the neural mechanisms of spatial selection and action preparation compared to controls, which may underpin the sustained difficulties they experience. These findings provide new insights into the neural mechanisms potentially disrupted in this highly prevalent disorder.

Tool use modulates early stages of visuo-tactile integration in far space: Evidence from event-related potentials

The neural representation of multisensory space near the body is modulated by the active use of long tools in non-human primates. Here, we investigated whether the electrophysiological correlates of visuo-tactile integration in near and far space were modulated by active tool use in healthy humans. Participants responded to a tactile target delivered to one hand while an irrelevant visual stimulus was presented ipsilaterally in near or far space. This crossmodal task was performed after the use of either short or long tools. Crucially, the P100 components elicited by visuo-tactile stimuli was enhanced on far as compared to near space trials after the use of long tools, while no such difference was present after short tool use. Thus, we found increased neural responses in brain areas encoding tactile stimuli to the body when visual stimuli were presented close to the tip of the tool after long tool use. This increased visuo-tactile integration on far space trials following the use of long tools might indicate a transient remapping of multisensory space. We speculate that performing voluntary actions with long tools strengthens the representation of sensory information arising within portions of space (i.e. the hand and the tip of the tool) that are most functionally relevant to one's behavioural goals.

Object-based attention in complex, naturalistic auditory streams

In vision, objects have been described as the 'units' on which non-spatial attention operates in many natural settings. Here, we test the idea of object-based attention in the auditory domain within ecologically valid auditory scenes, composed of two spatially and temporally overlapping sound streams (speech signal vs. environmental soundscapes in Experiment 1 and two speech signals in Experiment 2). Top-down attention was directed to one or the other auditory stream by a non-spatial cue. To test for high-level, object-based attention effects we introduce an auditory repetition detection task in which participants have to detect brief repetitions of auditory objects, ruling out any possible confounds with spatial or feature-based attention. The participants' responses were significantly faster and more accurate in the valid cue condition compared to the invalid cue condition, indicating a robust cue-validity effect of high-level, object-based auditory attention.

Visual Selection of the Future Reach Path in Obstacle Avoidance

In two EEG experiments, we studied the role of visual attention during the preparation of manual movements around an obstacle. Participants performed rapid hand movements to a goal position avoiding a central obstacle either on the left or right side, depending on the pitch of the acoustical go signal. We used a dot probe paradigm to analyze the deployment of spatial attention in the visual field during the motor preparation. Briefly after the go signal but still before the hand movement actually started, a visual transient was flashed either on the planned pathway of the hand (congruent trials) or on the opposite, movement-irrelevant side (incongruent trials). The P1/N1 components that were evoked by the onset of the dot probe were enhanced in congruent trials where the visual transient was presented on the planned path of the hand. The results indicate that, during movement preparation, attention is allocated selectively to the planned trajectory the hand is going to take around the obstacle.

Pushing attention to one side: Force field adaptation alters neural correlates of orienting and disengagement of spatial attention

Sensorimotor adaptation to wedge prisms can alter the balance of attention between left and right space in healthy adults, and improve symptoms of spatial neglect after stroke. Here we asked whether the orienting of spatial attention to visual stimuli is affected by a different form of sensorimotor adaptation that involves physical perturbations of arm movement, rather than distortion of visual feedback. Healthy participants performed a cued discrimination task before and after they made reaching movements to a central target. A velocity-dependent force field pushed the hand aside during each reach, and required participants to apply compensatory forces toward the opposite side. We used event-related potentials (ERPs) to determine whether electroencephalography (EEG) responses reflecting orienting (cue-locked N1) and disengagement (target-locked P1) of spatial attention are affected by adaptation to force fields. After adaptation, the cue-locked N1 was relatively larger for stimuli presented in the hemispace corresponding to the direction of compensatory hand force. P1 amplitudes evoked by invalidly cued targets presented on the opposite side were reduced. This suggests that force field adaptation boosted attentional orienting responses toward the side of hand forces, and impeded attentional disengagement from that side, mimicking previously reported effects of prism adaptation. Thus, remapping between motor commands and intended movement direction is sufficient to bias ERPs, reflecting changes in the orienting of spatial attention in the absence of visuo-spatial distortion or visuo-proprioceptive mismatch. Findings are relevant to theories of how sensorimotor adaptation can modulate attention, and may open new avenues for treatment of spatial neglect.

Separate and combined effects of action relevance and motivational value on visual working memory

Visual working memory contents can be selectively weighted according to differences in their task-relevance. In the present study, we examined the influence of two more indirect selection biases established by a concurrent task or learned reward associations: action relevance and motivational value. In three experiments, memory performance was assessed with the same color change detection task. Potential action relevance and motivational value were each determined by a specific feature of the memory items (location or shape, respectively) and manipulated orthogonally. Investigated separately (Experiments 1A and 1B), both selection biases modulated visual working memory. In combination (Experiment 2), action relevance and motivational value still each exerted an influence, but not in a fully independent and additive manner. While action relevance impacted performance irrespective of the reward associated with the items, an effect of motivational value was only observed for action-relevant items. These results support the notion that visual working memory is automatically biased as an inherent part of action planning. More generally, these findings highlight the versatile nature of visual working memory: Contents can be flexibly weighted to reflect differences in their importance, taking into account several sources of information.

High-precision voluntary movements are largely independent of preceding vertex potentials elicited by sudden sensory events

KEY POINTS

Salient and sudden sensory events generate a remarkably large response in the human brain, the vertex wave (VW). The VW is coupled with a modulation of a voluntarily-applied isometric force. In the present study, we tested whether the VW is also related to executing high-precision movements. The execution of a voluntary high-precision movement remains relatively independent of the brain activity reflected by the preceding VW. The apparent relationship between the positive VW and movement onset time is explained by goal-related but stimulus-independent neural activities. These results highlight the need to consider such goal-related but stimulus-independent neural activities when attempting to relate event-related potential amplitude with perceptual and behavioural performance.

ABSTRACT

Salient and fast-rising sensory events generate a large biphasic vertex wave (VW) in the human electroencephalogram (EEG). We recently reported that the VW is coupled with a modulation of concomitantly-applied isometric force. In the present study, in five experiments, we tested whether the VW is also related to high-precision visuomotor control. We obtained three results. First, the saliency-induced increase in VW amplitude was paralleled by a modulation in two of the five extracted movement parameters: a reduction in the onset time of the voluntary movement (P < 0.005) and an increase in movement accuracy (P < 0.005). Second, spontaneous trial-by-trial variability in vertex wave amplitude, for a given level of stimulus saliency, was positively correlated with movement onset time (P < 0.001 in four out of five experiments). Third, this latter trial-by-trial correlation was explained by a widespread EEG negativity independent of the occurrence of the positive VW, although overlapping in time with it. These results indicate that (i) the execution of a voluntary high-precision movement remains relatively independent of the neural processing reflected by the preceding VW, with (ii) the exception of movement onset time, for which saliency-based contextual effects are dissociated from trial-by-trial effects. These results also indicate that (iii) attentional effects can produce spurious correlations between event-related potentials (ERPs) and behavioural measures. Although sudden salient stimuli trigger characteristic EEG responses coupled with distinct reactive components within an ongoing isometric task, the results of the present study indicate that the execution of a subsequent voluntary movement appears largely protected from such saliency-based modulation, with the exception of movement onset time.

Collision Avoidance Behavior between Walkers: Global and Local Motion Cues

Daily activities require agents to interact with each other, such as during collision avoidance. The nature of visual information that is used for a collision free interaction requires further understanding. We aim to manipulate the nature of visual information in two forms, global and local information appearances. Sixteen healthy participants navigated towards a target in an immersive computer-assisted virtual environment (CAVE) using a joystick. A moving passive obstacle crossed the participant's trajectory perpendicularly at various pre-defined risks of collision distances. The obstacle was presented with one of five virtual appearances, associated to global motion cues (i.e., a cylinder or a sphere), or local motion cues (i.e., only the legs or the trunk). A full body virtual walker, showing both local and global motion cues, used as a reference condition. The final crossing distance was affected by the global motion appearances, however, appearance had no qualitative effect on motion adaptations. These findings contribute towards further understanding what information people use when interacting with others.

The profile of attention differs between locations orthogonal to and in line with reach direction

People make movements in a variety of directions when interacting with the world around them. It has been well documented that attention shifts to the goal of an upcoming movement, whether the movement is a saccade or a reach. However, recent evidence suggests that the direction of a movement may influence the spatial spread of attention (Stewart & Ma-Wyatt, 2015, Journal of Vision, 15(5), 10). We investigated whether the spatiotemporal profile of attention differs depending on where that location is situated relative to the direction of movement, and if this pattern is consistent across different movement effectors. We compared attentional facilitation at locations in line with or orthogonal to the movement, for reach-only, reach-plus-saccade, and saccade-only conditions. Results show that the spatiotemporal profile of attention differs across different movement combinations, and is also different at target locations orthogonal to and in line with the movement direction. Specifically, when a reach alone was made, there was a general decrease in attention at all locations during the movement and a general increase in attention at all locations with a saccade only. However, the concurrent reach and saccade condition showed a premovement attentional facilitation at locations orthogonal to movement direction, but not those in line with the movement direction. These results suggest attentional guidance may be more important at differing time points, depending on the type of movement.