Functional Use of Eye Movements for an Acting System

Rapid eye and hand responses in an interception task are differentially modulated by context-dependent predictability

When catching a falling ball or avoiding a collision with traffic, humans can quickly generate eye and limb responses to unpredictable changes in their environment. Mechanisms of limb and oculomotor control when responding to sudden changes in the environment have mostly been investigated independently. Here, we investigated eye-hand coordination in a rapid interception task where human participants used a virtual paddle to intercept a moving target. The target moved vertically down a computer screen and could suddenly jump to the left or right. In high-certainty blocks, the target always jumped; in low-certainty blocks, the target only jumped in a portion of the trials. Further, we manipulated response urgency by varying the time of target jumps, with early jumps requiring less urgent responses and late jumps requiring more urgent responses. Our results highlight differential effects of certainty and urgency on eye-hand coordination. Participants initiated both eye and hand responses earlier for high-certainty compared with low-certainty blocks. Hand reaction times decreased and response vigor increased with increasing urgency levels. However, eye reaction times were lowest for medium-urgency levels and eye vigor was unaffected by urgency. Across all trials, we found a weak positive correlation between eye and hand responses. Taken together, these results suggest that the limb and oculomotor systems use similar early sensorimotor processing; however, rapid responses are modulated differentially to attain system-specific sensorimotor goals.

Eyes and hand are both reliable at localizing somatosensory targets

Body representations (BR) for action are critical to perform accurate movements. Yet, behavioral measures suggest that BR are distorted even in healthy people. However, the upper limb has mostly been used as a probe so far, making difficult to decide whether BR are truly distorted or whether this depends on the effector used as a readout. Here, we aimed to assess in healthy humans the accuracy of the eye and hand effectors in localizing somatosensory targets, to determine whether they may probe BR similarly. Twenty-six participants completed two localization tasks in which they had to localize an unseen target (proprioceptive or tactile) with either their eyes or hand. Linear mixed model revealed in both tasks a larger horizontal (but not vertical) localization error for the ocular than for the manual localization performance. However, despite better hand mean accuracy, manual and ocular localization performance positively correlated to each other in both tasks. Moreover, target position also affected localization performance for both eye and hand responses: accuracy was higher for the more flexed position of the elbow in the proprioceptive task and for the thumb than for the index finger in the tactile task, thus confirming previous results of better performance for the thumb. These findings indicate that the hand seems to beat the eyes along the horizontal axis when localizing somatosensory targets, but the localization patterns revealed by the two effectors seemed to be related and characterized by the same target effect, opening the way to assess BR with the eyes when upper limb motor control is disturbed.

Towards Automatic Object Detection and Activity Recognition in Indoor Climbing

Rock climbing has propelled from niche sport to mainstream free-time activity and Olympic sport. Moreover, climbing can be studied as an example of a high-stakes perception-action task. However, understanding what constitutes an expert climber is not simple or straightforward. As a dynamic and high-risk activity, climbing requires a precise interplay between cognition, perception, and precise action execution. While prior research has predominantly focused on the movement aspect of climbing (i.e., skeletal posture and individual limb movements), recent studies have also examined the climber's visual attention and its links to their performance. To associate the climber's attention with their actions, however, has traditionally required frame-by-frame manual coding of the recorded eye-tracking videos. To overcome this challenge and automatically contextualize the analysis of eye movements in indoor climbing, we present deep learning-driven (YOLOv5) hold detection that facilitates automatic grasp recognition. To demonstrate the framework, we examined the expert climber's eye movements and egocentric perspective acquired from eye-tracking glasses (SMI and Tobii Glasses 2). Using the framework, we observed that the expert climber's grasping duration was positively correlated with total fixation duration (r = 0.807) and fixation count (r = 0.864); however, it was negatively correlated with the fixation rate (r = -0.402) and saccade rate (r = -0.344). The findings indicate the moments of cognitive processing and visual search that occurred during decision making and route prospecting. Our work contributes to research on eye-body performance and coordination in high-stakes contexts, and informs the sport science and expands the applications, e.g., in training optimization, injury prevention, and coaching.

Interacting with autistic virtual characters: intrapersonal synchrony of nonverbal behavior affects participants' perception

Temporal coordination of communicative behavior is not only located between but also within interaction partners (e.g., gaze and gestures). This intrapersonal synchrony (IaPS) is assumed to constitute interpersonal alignment. Studies show systematic variations in IaPS in individuals with autism, which may affect the degree of interpersonal temporal coordination. In the current study, we reversed the approach and mapped the measured nonverbal behavior of interactants with and without ASD from a previous study onto virtual characters to study the effects of the differential IaPS on observers (N = 68), both with and without ASD (crossed design). During a communication task with both characters, who indicated targets with gaze and delayed pointing gestures, we measured response times, gaze behavior, and post hoc impression formation. Results show that character behavior indicative of ASD resulted in overall enlarged decoding times in observers and this effect was even pronounced in observers with ASD. A classification of observer's gaze types indicated differentiated decoding strategies. Whereas non-autistic observers presented with a rather consistent eyes-focused strategy associated with efficient and fast responses, observers with ASD presented with highly variable decoding strategies. In contrast to communication efficiency, impression formation was not influenced by IaPS. The results underline the importance of timing differences in both production and perception processes during multimodal nonverbal communication in interactants with and without ASD. In essence, the current findings locate the manifestation of reduced reciprocity in autism not merely in the person, but in the interactional dynamics of dyads.

Sensorimotor prediction is used to direct gaze toward task-relevant locations in a goal-directed throwing task

Previous research has shown that action effects of self-generated movements are internally predicted before outcome feedback becomes available. To test whether these sensorimotor predictions are used to facilitate visual information uptake for feedback processing, we measured eye movements during the execution of a goal-directed throwing task. Participants could fully observe the effects of their throwing actions (ball trajectory and either hitting or missing a target) in most of the trials. In a portion of the trials, the ball trajectory was not visible, and participants only received static information about the outcome. We observed a large proportion of predictive saccades, shifting gaze toward the goal region before the ball arrived and outcome feedback became available. Fixation locations after predictive saccades systematically covaried with future ball positions in trials with continuous ball flight information, but notably also in trials with static outcome feedback and only efferent and proprioceptive information about the movement that could be used for predictions. Fixation durations at the chosen positions after feedback onset were modulated by action outcome (longer durations for misses than for hits) and outcome uncertainty (longer durations for narrow vs. clear outcomes). Combining both effects, durations were longest for narrow errors and shortest for clear hits, indicating that the chosen locations offer informational value for feedback processing. Thus, humans are able to use sensorimotor predictions to direct their gaze toward task-relevant feedback locations. Outcome-dependent saccade latency differences (miss vs. hit) indicate that also predictive valuation processes are involved in planning predictive saccades.NEW & NOTEWORTHY We elucidate the potential benefits of sensorimotor predictions, focusing on how the system actually uses this information to optimize feedback processing in goal-directed actions. Sensorimotor information is used to predict spatial parameters of movement outcomes, guiding predictive saccades toward future action effects. Saccade latencies and fixation durations are modulated by outcome quality, indicating that predictive valuation processes are considered and that the locations chosen are of high informational value for feedback processing.

Consequences of eye movements for spatial selectivity

What determines spatial tuning in the visual system? Standard views rely on the assumption that spatial information is directly inherited from the relative position of photoreceptors and shaped by neuronal connectivity.1,2 However, human eyes are always in motion during fixation,3,4,5,6 so retinal neurons receive temporal modulations that depend on the interaction of the spatial structure of the stimulus with eye movements. It has long been hypothesized that these modulations might contribute to spatial encoding,7,8,9,10,11,12 a proposal supported by several recent observations.13,14,15,16 A fundamental, yet untested, consequence of this encoding strategy is that spatial tuning is not hard-wired in the visual system but critically depends on how the fixational motion of the eye shapes the temporal structure of the signals impinging onto the retina. Here we used high-resolution techniques for eye-tracking17 and gaze-contingent display control18 to quantitatively test this distinctive prediction. We examined how contrast sensitivity, a hallmark of spatial vision, is influenced by fixational motion, both during normal active fixation and when the spatiotemporal stimulus on the retina is altered to mimic changes in fixational control. We showed that visual sensitivity closely follows the strength of the luminance modulations delivered within a narrow temporal bandwidth, so changes in fixational motion have opposite visual effects at low and high spatial frequencies. By identifying a key role for oculomotor activity in spatial selectivity, these findings have important implications for the perceptual consequences of abnormal eye movements, the sources of perceptual variability, and the function of oculomotor control.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

Eye-hand coordination all the way: from discrete to continuous hand movements

The differentiation between continuous and discrete actions is key for behavioral neuroscience. Although many studies have characterized eye-hand coordination during discrete (e.g., reaching) and continuous (e.g., pursuit tracking) actions, all these studies were conducted separately, using different setups and participants. In addition, how eye-hand coordination might operate at the frontier between discrete and continuous movements remains unexplored. Here we filled these gaps by means of a task that could elicit different movement dynamics. Twenty-eight participants were asked to simultaneously track with their eyes and a joystick a visual target that followed an unpredictable trajectory and whose position was updated at different rates (from 1.5 to 240 Hz). This procedure allowed us to examine actions ranging from discrete point-to-point movements (low refresh rate) to continuous pursuit (high refresh rate). For comparison, we also tested a manual tracking condition with the eyes fixed and a pure eye tracking condition (hand fixed). The results showed an abrupt transition between discrete and continuous hand movements around 3 Hz contrasting with a smooth trade-off between fixations and smooth pursuit. Nevertheless, hand and eye tracking accuracy remained strongly correlated, with each of these depending on whether the other effector was recruited. Moreover, gaze-cursor distance and lag were smaller when eye and hand performed the task conjointly than separately. Altogether, despite some dissimilarities in eye and hand dynamics when transitioning between discrete and continuous movements, our results emphasize that eye-hand coordination continues to smoothly operate and support the notion of synergies across eye movement types.NEW & NOTEWORTHY The differentiation between continuous and discrete actions is key for behavioral neuroscience. By using a visuomotor task in which we manipulate the target refresh rate to trigger different movement dynamics, we explored eye-hand coordination all the way from discrete to continuous actions. Despite abrupt changes in hand dynamics, eye-hand coordination continues to operate via a gradual trade-off between fixations and smooth pursuit, an observation confirming the notion of synergies across eye movement types.

Comparing eye-hand coordination between controller-mediated virtual reality, and a real-world object interaction task

Virtual reality (VR) technology has advanced significantly in recent years, with many potential applications. However, it is unclear how well VR simulations mimic real-world experiences, particularly in terms of eye-hand coordination. This study compares eye-hand coordination from a previously validated real-world object interaction task to the same task re-created in controller-mediated VR. We recorded eye and body movements and segmented participants' gaze data using the movement data. In the real-world condition, participants wore a head-mounted eye tracker and motion capture markers and moved a pasta box into and out of a set of shelves. In the VR condition, participants wore a VR headset and moved a virtual box using handheld controllers. Unsurprisingly, VR participants took longer to complete the task. Before picking up or dropping off the box, participants in the real world visually fixated the box about half a second before their hand arrived at the area of action. This 500-ms minimum fixation time before the hand arrived was preserved in VR. Real-world participants disengaged their eyes from the box almost immediately after their hand initiated or terminated the interaction, but VR participants stayed fixated on the box for much longer after it was picked up or dropped off. We speculate that the limited haptic feedback during object interactions in VR forces users to maintain visual fixation on objects longer than in the real world, altering eye-hand coordination. These findings suggest that current VR technology does not replicate real-world experience in terms of eye-hand coordination.

Perceptual-Cognitive Integration for Goal-Directed Action in Naturalistic Environments

Real-world actions require one to simultaneously perceive, think, and act on the surrounding world, requiring the integration of (bottom-up) sensory information and (top-down) cognitive and motor signals. Studying these processes involves the intellectual challenge of cutting across traditional neuroscience silos, and the technical challenge of recording data in uncontrolled natural environments. However, recent advances in techniques, such as neuroimaging, virtual reality, and motion tracking, allow one to address these issues in naturalistic environments for both healthy participants and clinical populations. In this review, we survey six topics in which naturalistic approaches have advanced both our fundamental understanding of brain function and how neurologic deficits influence goal-directed, coordinated action in naturalistic environments. The first part conveys fundamental neuroscience mechanisms related to visuospatial coding for action, adaptive eye-hand coordination, and visuomotor integration for manual interception. The second part discusses applications of such knowledge to neurologic deficits, specifically, steering in the presence of cortical blindness, impact of stroke on visual-proprioceptive integration, and impact of visual search and working memory deficits. This translational approach-extending knowledge from lab to rehab-provides new insights into the complex interplay between perceptual, motor, and cognitive control in naturalistic tasks that are relevant for both basic and clinical research.

The eyes anticipate where objects will move based on their shape

Imagine staring into a clear river, starving, desperately searching for a fish to spear and cook. You see a dark shape lurking beneath the surface. It doesn't resemble any sort of fish you've encountered before - but you're hungry. To catch it, you need to anticipate which way it will move when you lunge for it, to compensate for your own sensory and motor processing delays1,2,3. Yet you know nothing about the behaviour of this creature, and do not know in which direction it will try to escape. What cues do you then use to drive such anticipatory responses? Fortunately, many species4, including humans, have the remarkable ability to predict the directionality of objects based on their shape - even if they are unfamiliar and so we cannot rely on semantic knowledge about their movements5. While it is known that such directional inferences can guide attention5, we do not yet fully understand how such causal inferences are made, or the extent to which they enable anticipatory behaviours. Does the oculomotor system, which moves our eyes to optimise visual input, use directional inferences from shape to anticipate upcoming motion direction? Such anticipation is necessary to stabilise the moving object on the high-resolution fovea of the retina while tracking the shape, a primary goal of the oculomotor system6, and to guide any future interactions7,8. Here, we leveraged a well-known behaviour of the oculomotor system: anticipatory smooth eye movements (ASEM), where an increase in eye velocity is observed in the direction of a stimulus' expected motion, before the stimulus actually moves3, to show that the oculomotor system extracts directional information from shape, and uses this inference to predict and anticipate upcoming motion.

Differentiated, rather than shared, strategies for time-coordinated action in social and non-social domains in autistic individuals

Autism spectrum disorder (ASD) is a neurodevelopmental condition with a highly heterogeneous adult phenotype that includes social and non-social behavioral characteristics. The link between the characteristics assignable to the different domains remains unresolved. One possibility is that social and non-social behaviors in autism are modulated by a common underlying deficit. However, here we report evidence supporting an alternative concept that is individual-centered rather than deficit-centered. Individuals are assumed to have a distinctive style in the strategies they adopt to perform social and non-social tasks with these styles presumably being structured differently between autistic individuals and typically-developed (TD) individuals. We tested this hypothesis for the execution of time-coordinated (synchronized) actions. Participants performed (i) a social task that required synchronized gaze and pointing actions to interact with another person, and (ii) a non-social task that required finger-tapping actions synchronized to periodic stimuli at different time-scales and sensory modalities. In both tasks, synchronization behavior differed between ASD and TD groups. However, a principal component analysis of individual behaviors across tasks revealed associations between social and non-social features for the TD persons but such cross-domain associations were strikingly absent for autistic individuals. The highly differentiated strategies between domains in ASD are inconsistent with a general synchronization deficit and instead highlight the individualized developmental heterogeneity in the acquisition of domain-specific behaviors. We propose a cognitive model to help disentangle individual-centered from deficit-centered effects in other domains. Our findings reinforce the importance to identify individually differentiated phenotypes to personalize autism therapies.

The effect of impaired velocity signals on goal-directed eye and hand movements

Information about position and velocity is essential to predict where moving targets will be in the future, and to accurately move towards them. But how are the two signals combined over time to complete goal-directed movements? We show that when velocity information is impaired due to using second-order motion stimuli, saccades directed towards moving targets land at positions where targets were ~ 100 ms before saccade initiation, but hand movements are accurate. Importantly, the longer latencies of hand movements allow for additional time to process the sensory information available. When increasing the period of time one sees the moving target before making the saccade, saccades become accurate. In line with that, hand movements with short latencies show higher curvature, indicating corrections based on an update of incoming sensory information. These results suggest that movements are controlled by an independent and evolving combination of sensory information about the target's position and velocity.

Implicit Sensorimotor Adaptation Proceeds in Absence of Movement Execution

In implicit sensorimotor adaptation, a mismatch between the predicted and actual sensory feedback results in a sensory prediction error (SPE). Sensory predictions have long been thought to be linked to descending motor commands, implying a necessary contribution of movement execution to adaptation. However, recent work has shown that mere motor imagery (MI) also engages predictive mechanisms, opening up the possibility that MI might be sufficient to drive implicit adaptation. In a within-subject design in humans (n = 30), implicit adaptation was assessed in a center-out reaching task, following a single exposure to a visuomotor rotation. It was hypothesized that performing MI of a reaching movement while being provided with an animation of rotated visual feedback (MI condition) would lead to postrotation biases (PRBs) similar to those observed when the movement is executed (Execution condition). Results revealed that both the MI and Execution conditions led to significant directional biases following rotated trials. Yet the magnitude of these biases was significantly larger in the Execution condition. To further probe the contribution of MI to adaptation, a Control condition was conducted in which participants were presented with the same rotated visual animation as in the MI condition, but in which they were prevented from performing MI. Surprisingly, significant biases were also observed in the Control condition, suggesting that MI per se may not have accounted for adaptation. Overall, these results suggest that implicit adaptation can be partially supported by processes other than those that strictly pertain to generating motor commands, although movement execution does potentiate it.

Touching with the eyes: Oculomotor self-touch induces illusory body ownership

Self-touch plays a central role in the construction and plasticity of the bodily self. But which mechanisms support this role? Previous accounts emphasize the convergence of proprioceptive and tactile signals from the touching and the touched body parts. Here, we hypothesise that proprioceptive information is not necessary for self-touch modulation of body-ownership. Because eye movements do not rely on proprioceptive signals as limb movements do, we developed a novel oculomotor self-touch paradigm where voluntary eye movements generated corresponding tactile sensations. We then compared the effectiveness of eye versus hand self-touch movements in generating an illusion of owning a rubber hand. Voluntary oculomotor self-touch was as effective as hand-driven self-touch, suggesting that proprioception does not contribute to body ownership during self-touch. Self-touch may contribute to a unified sense of bodily self by binding voluntary actions toward our own body with their tactile consequences.

Training and proficiency level in endoscopic sinus surgery change residents' eye movements

Nose surgery is challenging and needs a lot of training for safe and efficient treatments. Eye tracking can provide an objective assessment to measure residents' learning curve. The aim of the current study was to assess residents' fixation duration and other dependent variables over the course of a dedicated training in functional endoscopic sinus surgery (FESS). Sixteen residents performed a FESS training over 18 sessions, split into three surgical steps. Eye movements in terms of percent fixation on the screen and average fixation duration were measured, in addition to residents' completion time, cognitive load, and surgical performance. Results indicated performance improvements in terms of completion time and surgical performance. Cognitive load and average fixation duration showed a significant change within the last step of training. Percent fixation on screen increased within the first step, and then stagnated. Results showed that eye movements and cognitive load differed between residents of different proficiency levels. In conclusion, eye tracking is a helpful objective measuring tool in FESS. It provides additional insights of the training level and changes with increasing performance. Expert-like gaze was obtained after half of the training sessions and increased proficiency in FESS was associated with increased fixation duration.

The influence of action on perception spans different effectors

Perception and action are fundamental processes that characterize our life and our possibility to modify the world around us. Several pieces of evidence have shown an intimate and reciprocal interaction between perception and action, leading us to believe that these processes rely on a common set of representations. The present review focuses on one particular aspect of this interaction: the influence of action on perception from a motor effector perspective during two phases, action planning and the phase following execution of the action. The movements performed by eyes, hands, and legs have a different impact on object and space perception; studies that use different approaches and paradigms have formed an interesting general picture that demonstrates the existence of an action effect on perception, before as well as after its execution. Although the mechanisms of this effect are still being debated, different studies have demonstrated that most of the time this effect pragmatically shapes and primes perception of relevant features of the object or environment which calls for action; at other times it improves our perception through motor experience and learning. Finally, a future perspective is provided, in which we suggest that these mechanisms can be exploited to increase trust in artificial intelligence systems that are able to interact with humans.

Intrapersonal synchrony analysis reveals a weaker temporal coherence between gaze and gestures in adults with autism spectrum disorder

The temporal encoding of nonverbal signals within individuals, referred to as intrapersonal synchrony (IaPS), is an implicit process and essential feature of human communication. Based on existing evidence, IaPS is thought to be a marker of nonverbal behavior characteristics in autism spectrum disorders (ASD), but there is a lack of empirical evidence. The aim of this study was to quantify IaPS in adults during an experimentally controlled real-life interaction task. A sample of adults with a confirmed ASD diagnosis and a matched sample of typically-developed adults were tested (N = 48). Participants were required to indicate the appearance of a target invisible to their interaction partner nonverbally through gaze and pointing gestures. Special eye-tracking software allowed automated extraction of temporal delays between nonverbal signals and their intrapersonal variability with millisecond temporal resolution as indices for IaPS. Likelihood ratio tests of multilevel models showed enlarged delays between nonverbal signals in ASD. Larger delays were associated with greater intrapersonal variability in delays. The results provide a quantitative constraint on nonverbal temporality in typically-developed adults and suggest weaker temporal coherence between nonverbal signals in adults with ASD. The results provide a potential diagnostic marker and inspire predictive coding theories about the role of IaPS in interpersonal synchronization processes.

Context matters during pick-and-place in VR: Impact on search and transport phases

When considering external assistive systems for people with motor impairments, gaze has been shown to be a powerful tool as it is anticipatory to motor actions and is promising for understanding intentions of an individual even before the action. Up until now, the vast majority of studies investigating the coordinated eye and hand movement in a grasping task focused on single objects manipulation without placing them in a meaningful scene. Very little is known about the impact of the scene context on how we manipulate objects in an interactive task. In the present study, it was investigated how the scene context affects human object manipulation in a pick-and-place task in a realistic scenario implemented in VR. During the experiment, participants were instructed to find the target object in a room, pick it up, and transport it to a predefined final location. Thereafter, the impact of the scene context on different stages of the task was examined using head and hand movement, as well as eye tracking. As the main result, the scene context had a significant effect on the search and transport phases, but not on the reach phase of the task. The present work provides insights into the development of potential supporting intention predicting systems, revealing the dynamics of the pick-and-place task behavior once it is realized in a realistic context-rich scenario.

Imagining interceptions: Eye movements as an online indicator of covert motor processes during motor imagery

The analysis of eye movements during motor imagery has been used to understand the influence of covert motor processes on visual-perceptual activity. There is evidence showing that gaze metrics seem to be affected by motor planning often dependent on the spatial and temporal characteristics of a task. However, previous research has focused on simulated actions toward static targets with limited empirical evidence of how eye movements change in more dynamic environments. The study examined the characteristics of eye movements during motor imagery for an interception task. Twenty-four participants were asked to track a moving target over a computer display and either mentally simulate an interception or rest. The results showed that smooth pursuit variables, such as duration and gain, were lower during motor imagery when compared to passive observation. These findings indicate that motor plans integrate visual-perceptual information based on task demands and that eye movements during imagery reflect such constraint.

Test of Gross Motor Development-3: Item Difficulty and Item Differential Functioning by Gender and Age with Rasch Analysis

The assessment of motor proficiency is essential across childhood to identify children’s strengths and difficulties and to provide adequate instruction and opportunities; assessment is a powerful tool to promote children’s development. This study aimed to investigate the hierarchal order of the Test of Gross Motor Development-Third Edition (TGMD-3) items regarding difficulty levels and the differential item functioning across gender and age group (3 to 5, 6 to 8, and 9 to 10 years old). Participants are 989 children (3 to 10.9 years; girls n = 491) who were assessed using TGMD-3. For locomotor skills, appropriate results reliability (alpha = 1.0), infit (M = 0.99; SD = 0.17), outfit (M = 1.18; SD = 0.64), and point-biserial correlations (rpb values from 0.14 to 0.58) were found; the trend was similar for ball skills: reliability (alpha = 1.0), infit (M = 0.99; SD = 0.13), outfit (M = 1.08; SD = 0.52); point-biserial correlations (rpb values from 0.06 to 0.59) were obtained. Two motor criteria: gallop, item-1, and one-hand forehand strike, item-4, were the most difficult items; in contrast, run, item-2, and two-hand catch, item-2, were the easiest items. Differential item functioning for age was observed in nine locomotor and ten ball skills items. These items were easier for older children compared to younger ones. The TGMD-3 has items with different difficulty levels capable of differential functioning across age groups.

Gaze behaviour: A window into distinct cognitive processes revealed by the Tower of London test

The analysis of gaze behaviour during complex tasks provides a promising non-invasive method to examine how specific eye movement patterns relate to various aspects of cognition and action. Notably, the association between aspects of gaze behaviour and subsequent goal-directed action during high-level visuospatial problem solving remains elusive. Therefore, the current study comprehensively examined gaze behaviour using traditional and entropy-based gaze analyses in healthy adults (N = 27) while they performed the Freiburg version of the Tower of London task. Results demonstrated that both gaze analyses provided crucial temporal and spatial information related to planning, solution elaboration and execution. Specifically, gaze biases toward task-relevant areas (i.e., the work space) and an increase in gaze complexity (i.e., gaze transition entropy) during optimal performance reflected changes in cognitive demands as task difficulty increased. A comparison between optimal and non-optimal performance revealed sub-optimal gaze patterns that occurred in the early stages of planning, which were taken to reflect poor information extraction from the task environment and impaired maintenance of information in visuospatial working memory. Gaze behaviour during movement execution indicated an increased need to extract and process information from the goal space. Consequently, movement execution time increased in order to reverse erroneous movements and re-sequence the problem solution. Taken together, the traditional and entropy-based gaze analyses applied in the present study provide a promising approach to identify eye movement patterns that support neurocognitive performance on tasks relying on visuospatial planning and problem solving.

Making a saccade enhances Stroop and Simon conflict control

Cognitive control is an important ability instantiated in many situations such as conflict control (e.g., Stroop/Simon task) and the control of eye movements (e.g., saccades). However, it is unclear whether eye movement control shares a common cognitive control system with the conflict control. In Experiment 1, we asked participants to make a prosaccade or antisaccade and then to identify the color of a lateralized color word (i.e., a Stroop-Simon stimulus). The stimulus onset asynchrony (SOA) between the saccadic cue and the Stroop-Simon stimulus was manipulated to be either short (200 ms) or long (600 ms). Results showed that the Stroop effect at the response level and the (negative) Simon effect were smaller when the SOA was short than long, demonstrating a decline of response control over time after making a saccade. Moreover, this temporal change of the Simon effect was more pronounced in the antisaccade session than in the prosaccade session. Furthermore, individuals who had better performance in the antisaccade task performed better in the response control of Stroop interference. When the saccade task was removed in Experiment 2, the temporal declines of the response control observed in Experiment 1 were absent. Experiment 3 replicated the key results of Experiment 1 by replacing the Stroop-Simon task with a typical Simon task and separately testing the typical Stroop and Simon tasks. Overall, our findings suggest that a common system is shared between the control of eye movements and the conflict control at the response level.

Eye-hand coordination during online reach corrections is task dependent

To produce accurate movements, the human motor system needs to deal with errors that can occur due to inherent noise, changes in the body, or disturbances in the environment. Here, we investigated the temporal coupling of rapid corrections of the eye and hand in response to a change in visual target location during the movement. In addition to a "classic" double-step task in which the target stepped to a new position, participants performed a set of modified double-step tasks in which the change in movement goal was indicated by the appearance of an additional target, or by a spatial or symbolic cue. We found that both the absolute correction latencies of the eye and hand and the relative eye-hand correction latencies were dependent on the visual characteristics of the target change, with increasingly longer latencies in tasks that required more visual and cognitive processing. Typically, the hand started correcting slightly earlier than the eye, especially when the target change was indicated by a symbolic cue, and in conditions where visual feedback of the hand position was provided during the reach. Our results indicate that the oculomotor and limb-motor system can be differentially influenced by processing requirements of the task and emphasize that temporal eye-hand coupling is flexible rather than rigid.NEW & NOTEWORTHY Eye movements support hand movements in many situations. Here, we used variations of a double-step task to investigate temporal coupling of corrective hand and eye movements in response to target displacements. Correction latency coupling depended on the visual and cognitive processing demands of the task. The hand started correcting before the eye, especially when the task required decoding a symbolic cue. These findings highlight the flexibility and task dependency of eye-hand coordination.

Automated tool detection with deep learning for monitoring kinematics and eye-hand coordination in microsurgery

In microsurgical procedures, surgeons use micro-instruments under high magnifications to handle delicate tissues. These procedures require highly skilled attentional and motor control for planning and implementing eye-hand coordination strategies. Eye-hand coordination in surgery has mostly been studied in open, laparoscopic, and robot-assisted surgeries, as there are no available tools to perform automatic tool detection in microsurgery. We introduce and investigate a method for simultaneous detection and processing of micro-instruments and gaze during microsurgery. We train and evaluate a convolutional neural network for detecting 17 microsurgical tools with a dataset of 7500 frames from 20 videos of simulated and real surgical procedures. Model evaluations result in mean average precision at the 0.5 threshold of 89.5-91.4% for validation and 69.7-73.2% for testing over partially unseen surgical settings, and the average inference time of 39.90 ± 1.2 frames/second. While prior research has mostly evaluated surgical tool detection on homogeneous datasets with limited number of tools, we demonstrate the feasibility of transfer learning, and conclude that detectors that generalize reliably to new settings require data from several different surgical procedures. In a case study, we apply the detector with a microscope eye tracker to investigate tool use and eye-hand coordination during an intracranial vessel dissection task. The results show that tool kinematics differentiate microsurgical actions. The gaze-to-microscissors distances are also smaller during dissection than other actions when the surgeon has more space to maneuver. The presented detection pipeline provides the clinical and research communities with a valuable resource for automatic content extraction and objective skill assessment in various microsurgical environments.

Predictive eye movements are adjusted in a Bayes-optimal fashion in response to unexpectedly changing environmental probabilities

This study examined the application of active inference to dynamic visuomotor control. Active inference proposes that actions are dynamically planned according to uncertainty about sensory information, prior expectations, and the environment, with motor adjustments serving to minimise future prediction errors. We investigated whether predictive gaze behaviours are indeed adjusted in this Bayes-optimal fashion during a virtual racquetball task. In this task, participants intercepted bouncing balls with varying levels of elasticity, under conditions of higher or lower environmental volatility. Participants' gaze patterns differed between stable and volatile conditions in a manner consistent with generative models of Bayes-optimal behaviour. Partially observable Markov models also revealed an increased rate of associative learning in response to unpredictable shifts in environmental probabilities, although there was no overall effect of volatility on this parameter. Findings extend active inference frameworks into complex and unconstrained visuomotor tasks and present important implications for a neurocomputational understanding of the visual guidance of action.

Development of eye-hand coordination in typically developing children and adolescents assessed using a reach-to-grasp sequencing task

Eye-hand coordination is required to accurately perform daily activities that involve reaching, grasping and manipulating objects. Studies using aiming, grasping or sequencing tasks have shown a stereotypical temporal coupling pattern where the eyes are directed to the object in advance of the hand movement, which may facilitate the planning and execution required for reaching. While the temporal coordination between the ocular and manual systems has been extensively investigated in adults, relatively little is known about the typical development of eye-hand coordination. Therefore, the current study addressed an important knowledge gap by characterizing the profile of eye-hand coupling in typically developing school-age children (n = 57) and in a cohort of adults (n = 30). Eye and hand movements were recorded concurrently during the performance of a bead threading task which consists of four distinct movements: reach to bead, grasp, reach to needle, and thread. Results showed a moderate to high correlation between eye and hand latencies in children and adults, supporting that both movements were planned in parallel. Eye and reach latencies, latency differences, and dwell time during grasping and threading, showed significant age-related differences, suggesting eye-hand coupling becomes more efficient in adolescence. Furthermore, visual acuity, stereoacuity and accommodative facility were also found to be associated with the efficiency of eye-hand coordination in children. Results from this study can serve as reference values when examining eye and hand movement during the performance of fine motor skills in children with neurodevelopmental disorders.

Topical Review: Perceptual-cognitive Skills, Methods, and Skill-based Comparisons in Interceptive Sports

SIGNIFICANCE

We give a comprehensive picture of perceptual-cognitive (PC) skills that could contribute to performance in interceptive sports. Both visual skills that are low level and unlikely influenced by experience and higher-level cognitive-attentional skills are considered, informing practitioners for identification and training and alerting researchers to gaps in the literature.Perceptual-cognitive skills and abilities are keys to success in interceptive sports. The interest in identifying which skills and abilities underpin success and hence should be selected and developed is likely going to grow as technologies for skill testing and training continue to advance. Many different methods and measures have been applied to the study of PC skills in the research laboratory and in the field, and research findings across studies have often been inconsistent. In this article, we provide definitional clarity regarding whether a skill is primarily visual attentional (ranging from fundamental/low-level skills to high-level skills) or cognitive. We review those skills that have been studied using sport-specific stimuli or tests, such as postural cue anticipation in baseball, as well as those that are mostly devoid of sport context, considered general skills, such as dynamic visual acuity. In addition to detailing the PC skills and associated methods, we provide an accompanying table of published research since 1995, highlighting studies (for various skills and sports) that have and have not differentiated across skill groups.

Computational Mechanisms Mediating Inhibitory Control of Coordinated Eye-Hand Movements

Significant progress has been made in understanding the computational and neural mechanisms that mediate eye and hand movements made in isolation. However, less is known about the mechanisms that control these movements when they are coordinated. Here, we outline our computational approaches using accumulation-to-threshold and race-to-threshold models to elucidate the mechanisms that initiate and inhibit these movements. We suggest that, depending on the behavioral context, the initiation and inhibition of coordinated eye-hand movements can operate in two modes-coupled and decoupled. The coupled mode operates when the task context requires a tight coupling between the effectors; a common command initiates both effectors, and a unitary inhibitory process is responsible for stopping them. Conversely, the decoupled mode operates when the task context demands weaker coupling between the effectors; separate commands initiate the eye and hand, and separate inhibitory processes are responsible for stopping them. We hypothesize that the higher-order control processes assess the behavioral context and choose the most appropriate mode. This computational mechanism can explain the heterogeneous results observed across many studies that have investigated the control of coordinated eye-hand movements and may also serve as a general framework to understand the control of complex multi-effector movements.

Motor learning by selection in visual working memory

Motor adaptation maintains movement accuracy over the lifetime. Saccadic eye movements have been used successfully to study the mechanisms and neural basis of adaptation. Using behaviorally irrelevant targets, it has been shown that saccade adaptation is driven by errors only in a brief temporal interval after movement completion. However, under natural conditions, eye movements are used to extract information from behaviorally relevant objects and to guide actions manipulating these objects. In this case, the action outcome often becomes apparent only long after movement completion, outside the supposed temporal window of error evaluation. Here, we show that saccade adaptation can be driven by error signals long after the movement when using behaviorally relevant targets. Adaptation occurred when a task-relevant target appeared two seconds after the saccade, or when a retro-cue indicated which of two targets, stored in visual working memory, was task-relevant. Our results emphasize the important role of visual working memory for optimal movement control.

The role of eye movements in manual interception: A mini-review

When we catch a moving object in mid-flight, our eyes and hands are directed toward the object. Yet, the functional role of eye movements in guiding interceptive hand movements is not yet well understood. This review synthesizes emergent views on the importance of eye movements during manual interception with an emphasis on laboratory studies published since 2015. We discuss the role of eye movements in forming visual predictions about a moving object, and for enhancing the accuracy of interceptive hand movements through feedforward (extraretinal) and feedback (retinal) signals. We conclude by proposing a framework that defines the role of human eye movements for manual interception accuracy as a function of visual certainty and object motion predictability.