Eye-hand interactions in tracing and drawing tasks

Effects of color cues on eye-hand coordination training with a mirror drawing task in virtual environment

Mirror drawing is a motor learning task that is used to evaluate and improve eye-hand coordination of users and can be implemented in immersive Virtual Reality (VR) Head-Mounted Displays (HMDs) for training purposes. In this paper, we investigated the effect of color cues on user motor performance in a mirror-drawing task between Virtual Environment (VE) and Real World (RW), with three different colors. We conducted a 5-day user study with twelve participants. The results showed that the participants made fewer errors in RW compared to VR, except for pre-training, which indicated that hardware and software limitations have detrimental effects on the motor learning of the participants across different realities. Furthermore, participants made fewer errors with the colors close to green, which is usually associated with serenity, contentment, and relaxation. According to our findings, VR headsets can be used to evaluate participants' eye-hand coordination in mirror drawing tasks to evaluate the motor-learning of participants. VE and RW training applications could benefit from our findings in order to enhance their effectiveness.

Visual-spatial and visuomotor functioning in adults with neurofibromatosis type 1

BACKGROUND

Neurofibromatosis type 1 (NF1) is a neurodevelopmental genetic disorder associated with visual-spatial and visuomotor deficits, which have not been studied well in adults with NF1.

METHODS

In 22 adults with NF1 and 31 controls, visuomotor functioning was assessed by measuring eye latency, hand latency and hand accuracy during visuomotor tasks. Visual-spatial functioning was assessed by measuring eye movement responses during the Visual Threshold Task.

RESULTS

The NF1 group had a significantly shorter eye latency than the control group and was less accurate in their hand movements during specific visuomotor tasks. The groups showed no differences in eye movement responses during the Visual Threshold Task and in hand latency during the visuomotor tasks.

CONCLUSIONS

In contrast to studies in children with NF1, we found no alterations in visual-spatial information processing in adults. Impairments in eye latency and hand accuracy during specific visuomotor tasks may indicate deficits in visuomotor functioning in adults with NF1.

Cerebral laterality of writing in right- and left- handers: A functional transcranial Doppler ultrasound study

The cerebral lateralization of written language has received very limited research attention in comparison to the wealth of studies on the cerebral lateralization of oral language. The purpose of the present study was to further our understanding of written language lateralization, by elucidating the relative contribution of language and motor functions. We compared written word generation with a task that has equivalent visuomotor demands but does not include language: the repeated drawing of symbols. We assessed cerebral laterality using functional transcranial Doppler ultrasound (fTCD), a non-invasive, perfusion-sensitive neuroimaging technique in 23 left- and 31 right-handed participants. Findings suggest that the linguistic aspect of written word generation recruited more left-hemispheric areas during writing, in right-handers compared to left-handers. This difference could be explained by greater variability in cerebral laterality patterns within left-handers or the possibility that the areas subserving language in left-handers are broader than in right-handers. Another explanation is that the attentional demands of the more novel symbol copying task (compared to writing) contributed more right-hemispheric activation in right-handers, but this could not be captured in left-handers due to ceiling effects. Future work could investigate such attentional demands using both simple and complex stimuli in the copying condition.

Influence of visual feedback persistence on visuo-motor skill improvement

Towards the larger goal of understanding factors relevant for improving visuo-motor control, we investigated the role of visual feedback for modulating the effectiveness of a simple hand-eye training protocol. The regimen comprised a series of curve tracing tasks undertaken over a period of one week by neurologically healthy individuals with their non-dominant hands. Our three subject groups differed in the training they experienced: those who received ‘Persistent’ visual-feedback by seeing their hand and trace evolve in real-time superimposed upon the reference patterns, those who received ‘Non-Persistent’ visual-feedback seeing their hand movement but not the emerging trace, and a ‘Control’ group that underwent no training. Improvements in performance were evaluated along two dimensions—accuracy and steadiness, to assess visuo-motor and motor skills, respectively. We found that persistent feedback leads to a significantly greater improvement in accuracy than non-persistent feedback. Steadiness, on the other hand, benefits from training irrespective of the persistence of feedback. Our results not only demonstrate the feasibility of rapid visuo-motor learning in adulthood, but more specifically, the influence of visual veridicality and a critical role for dynamically emergent visual information.

Using Head-Mounted Eye-Tracking to Study Handwriting Development

Even with the increasing use of technology in the classroom, handwriting remains a developmental foundation of education. When children fail to learn to write efficiently, they encounter cascading difficulties in using writing to communicate and learn content. Traditionally, the development of handwriting has been studied via neuropsychological testing or the moment-to-moment kinematics of pen movements. By measuring children’s handwriting with neuropsychological testing, investigators have determined that children’s visual-motor integration abilities predict children’s handwriting. Further, by measuring children’s pen movements while writing, investigators have determined that children’s handwriting becomes more fluent as they become skilled writers. Both of these literatures have advanced our understanding of handwriting substantially, but fall short of providing a full account of handwriting development. Here, we offer a perception–action account of handwriting development by describing how eye and hand movements become integrated during early writing. We describe how head-mounted eye-tracking technology can be used to measure children’s eye movements as they write in real-time. We illustrate this approach with findings from research on letter, form, and word copying in school-entry age children. We conclude by discussing how a perception–action approach can be extended for use with atypical populations.

Separate motor memories are formed when controlling different implicitly specified locations on a tool

Skillful manipulation requires forming and recalling memories of the dynamics of objects linking applied force to motion. It has been assumed that such memories are associated with entire objects. However, we often control different locations on an object, and these locations may be associated with different dynamics. We have previously demonstrated that multiple memories can be formed when participants are explicitly instructed to control different visual points marked on an object. A key question is whether this novel finding generalizes to more natural situations in which control points are implicitly defined by the task. To answer this question, we used objects with no explicit control points and tasks designed to encourage the use of distinct implicit control points. Participants moved a handle, attached to a robotic interface, to control the position of a rectangular object ("eraser") in the horizontal plane. Participants were required to move the eraser straight ahead to wipe away a column of dots ("dust"), located to either the left or right. We found that participants adapted to opposing dynamics when linked to the left and right dust locations, even though the movements required for these two contexts were the same. Control conditions showed this learning could not be accounted for by contextual cues or the fact that the task goal required moving in a straight line. These results suggest that people naturally control different locations on manipulated objects depending on the task context and that doing so affords the formation of separate motor memories. NEW & NOTEWORTHY Skilled manipulation requires forming motor memories of object dynamics, which have been assumed to be associated with entire objects. However, we recently demonstrated that people can form multiple memories when explicitly instructed to control different visual points on an object. In this article we show that this novel finding generalizes to more natural situations in which control points are implicitly defined by the task.

Precision in drawing and tracing tasks: Different measures for different aspects of fine motor control

Drawing and tracing tasks, by being relatively easy to execute and evaluate, have been incorporated in many paradigms used to study motor control. While these tasks are helpful when examining various aspects relative to the performance, the relationship in proficiency between these tasks was not evaluated to our knowledge. Seeing that drawing is thought to be an internally cued and tracing an externally cued task, differences in performances are to be expected. In this study, a quantitative evaluation of the precision of circle drawing and tracing, and spiral tracing was made on 150 healthy subjects. Our results show that, while precision is correlated when repeating drawing circles, tracing spirals, or tracing circles as well as between tracing spirals and tracing circles; there is no correlation when subjects performed drawing circles and tracing spirals or between drawing and tracing of circles. These results suggest that this lack of correlation is task dependent and not shape dependent. We suggest that the evaluation of fine motor control should include both a tracing and a drawing task, taking in consideration the precision in each task. We believe that this approach could help not only to evaluate fine motor control more accurately, but also to identify subjects who are more reliant on either internal or external cueing and to what extent.

Eye movements in interception with delayed visual feedback

The increased reliance on electronic devices such as smartphones in our everyday life exposes us to various delays between our actions and their consequences. Whereas it is known that people can adapt to such delays, the mechanisms underlying such adaptation remain unclear. To better understand these mechanisms, the current study explored the role of eye movements in interception with delayed visual feedback. In two experiments, eye movements were recorded as participants tried to intercept a moving target with their unseen finger while receiving delayed visual feedback about their own movement. In Experiment 1, the target randomly moved in one of two different directions at one of two different velocities. The delay between the participant’s finger movement and movement of the cursor that provided feedback about the finger movements was gradually increased. Despite the delay, participants followed the target with their gaze. They were quite successful at hitting the target with the cursor. Thus, they moved their finger to a position that was ahead of where they were looking. Removing the feedback showed that participants had adapted to the delay. In Experiment 2, the target always moved in the same direction and at the same velocity, while the cursor’s delay varied across trials. Participants still always directed their gaze at the target. They adjusted their movement to the delay on each trial, often succeeding to intercept the target with the cursor. Since their gaze was always directed at the target, and they could not know the delay until the cursor started moving, participants must have been using peripheral vision of the delayed cursor to guide it to the target. Thus, people deal with delays by directing their gaze at the target and using both experience from previous trials (Experiment 1) and peripheral visual information (Experiment 2) to guide their finger in a way that will make the cursor hit the target.

Evaluation of Handwriting Movement Kinematics: From an Ecological to a Magnetic Resonance Environment

Writing is a means of communication which requires complex motor, perceptual, and cognitive skills. If one of these abilities gets lost following traumatic events or due to neurological diseases, handwriting could deteriorate. Occupational therapy practitioners provide rehabilitation services for people with impaired handwriting. However, to determine the effectiveness of handwriting interventions no studies assessed whether the proposed treatments improved the kinematics of writing movement or had an effect at the level of the central nervous system. There is need to find new quantitative methodologies able to describe the behavioral and the neural outcomes of the rehabilitative interventions for handwriting. In the present study we proposed a combined approach that allowed evaluating the kinematic parameters of handwriting movements, acquired by means of a magnetic resonance-compatible tablet, and their neural correlates obtained simultaneously from a functional magnetic resonance imaging examination. Results showed that the system was reliable in term of reproducibility of the kinematic data during a test/re-test procedure. Further, despite the modifications with respect to an ecological writing movement condition, the kinematic parameters acquired inside the MR-environment were descriptive of individuals’ movement features. At last, the imaging protocol succeeded to show the activation of the cerebral regions associated with the production of writing movement in healthy people. From these findings, this methodology seems to be promising to evaluate the handwriting movement deficits and the potential alterations in the neural activity in those individuals who have handwriting difficulties. Finally, it would provide a mean to quantitatively assess the effect of a rehabilitative treatment.

Eye Movements in Drawing Simple Lines

As part of an investigation into real-world drawing, eye movements and eye – hand interactions have been recorded for twenty subjects of varying drawing experience drawing simple straight and curved lines and a square. Two modes of eye — hand behaviour were observed. In the first, named ‘close pursuit’, fixations closely followed the pencil with a sequence of small saccades. In the second, named ‘target locking’, a stable fixation was made on the end-point of the line throughout the entire drawing action. Depending on subject and type of line, close pursuit, target locking, or a combination of these modes was used regardless of previous drawing experience. The results are discussed in terms of the role of the eye in the control of the movement trajectory of the hand.

Lesion-symptom mapping of a complex figure copy task: A large-scale PCA study of the BCoS trial

Complex figure copying is a commonly used neuropsychological test. Here we explored the neural basis of the factors underlying complex figure copying (CFC), using data from the Birmingham Cognitive Screen (BCoS) in a large group of sub-acute, ischemic stroke patients (239). We computed two analyses: in the first we assessed the contribution of co-morbid deficits (i.e. in gesture processing, object use, visual neglect, pictures naming and sustained attention) to the lesions associated with CFC. In a second analysis a Principle Component Analysis (PCA) was used to isolate different underlying task components and to link to clinical neuroimaging scans. A voxel-based morphometry (VBM) analysis showed that poor CFC performance was associated with lesions to bi-lateral thalamus, lingual, right fusiform and right inferior parietal cortices (rIPC). The latter association with the posterior parietal cortex was diminished after controlling for neglect. Follow up analysis showed the neglect partially mediated the correlation of CFC and rIPC. The PCA revealed three main underlying components: (1) a component associated with high-level motor control common to different measures of apraxia and linked to the left postcentral gyrus, the right thalamus and middle frontal gyrus; (2) a visuo-motor transformation component unique to the CFC and associated with lesions to the posterior occipital and sensory cortices; (3) a component associated with multistep object use tasks which was correlated with lesions to the left inferior frontal orbital gyrus, the right fusiform and cerebellum. Using clinical symptoms, cognitive profiles and lesion mapping we showed that beyond visual perception, CFC performance is supported by three functional networks: one for high-level motor control, a visuo-motor transformation component, and multistep object use network.

The genesis of errors in drawing

The difficulty adults find in drawing objects or scenes from real life is puzzling, assuming that there are few gross individual differences in the phenomenology of visual scenes and in fine motor control in the neurologically healthy population. A review of research concerning the perceptual, motoric and memorial correlates of drawing ability was conducted in order to understand why most adults err when trying to produce faithful representations of objects and scenes. The findings reveal that accurate perception of the subject and of the drawing is at the heart of drawing proficiency, although not to the extent that drawing skill elicits fundamental changes in visual perception. Instead, the decisive role of representational decisions reveals the importance of appropriate segmentation of the visual scene and of the influence of pictorial schemas. This leads to the conclusion that domain-specific, flexible, top-down control of visual attention plays a critical role in development of skill in visual art and may also be a window into creative thinking.

Effect of reduced visual acuity on precision of two-dimensional tracing movements

PURPOSE

We intended to assess consequences of reduced visual acuity for performance in a natural simple motor task (tracing) using objective kinematic performance measures. Specifically, we intended to elucidate the kind of relationship between the task performance and best corrected binocular visual acuity and to determine the threshold of visual acuity when task performance starts to deteriorate.

METHODS

Ninety-five individuals with different best corrected visual acuity participated in the study (age 49±12 years, mean±SD, 27 men and 68 women). The participants manually traced maze-like visual patterns of different spatial complexity presented on the screen of a portable notebook computer using Clinical Kinematic Assessment Tool software. Tracing error was computed as performance measure in each trial with a spatial pattern matching technique - rigid point set registration method.

RESULTS

The segmented linear regression analysis showed that the relation between visual acuity and tracing errors was best described with a regression function having a break point between two data segments. Tracing performance was unaffected by values of visual acuity below 0.2 on logMAR scale, but when logMAR values increased above this critical limit (i.e. when visual acuity is further reduced), tracing errors linearly increased. The rate of the increase of the tracing error correlated with the complexity of visual stimulus shape.

CONCLUSION

Testing of fine motor functions with objective kinematic measures during visuomotor tasks may help differentiating between actual effects of reduced visual acuity on eye-hand coordination in individuals with similar levels of impairment of visual acuity.

DOES DYSLEXIA DEVELOP FROM LEFT-EYE DOMINANCE?

The purpose of this theoretical analysis and synthesis is to indicate how left-eye sighting dominance may lead to reading failure through dysfunctional right hemisphere letter encoding. Differing compensatory strategies are postulated to lead to outcomes that include the development of the phonologically impaired and phonologically proficient subtypes of dyslexia as well as specific spelling disability. Evidence is presented indicating that these disorders might be prevented by delaying the introduction of letter writing until the age of 8 years. Early childhood speech categorization in children genetically at-risk of developing dyslexia is also considered from this perspective. Convergent support for this premature writing hypothesis is provided by a comparison with the development of the left-hand inverted writing posture.

Visually guided tracking on a handheld device: can it be used to measure visuomotor skill in shift workers?

OBJECTIVE

We introduced a new visually controlled tracking task that can be assessed on a handheld device in shift workers to evaluate time-of-day dependent modulations in visuomotor performance.

BACKGROUND

Tracking tasks have been used to predict performance fluctuations depending on time of day mainly under laboratory conditions. One challenge to an extended use at the actual working site is the complex and fixed test setup consisting of a test unit, a monitor, and a manipulation object, such as a joystick.

METHOD

Participants followed an unpredictably moving target on the screen of a handheld device with an attachable stylus. A total of 11 shift workers (age range: 20-59, mean: 33.64, standard deviation: 10.56) were tested in the morning, the evening, and the night shift in 2-hr intervals with the tracking task and indicated their fatigue levels on visual analogue scales. We evaluated tracking precision by calculating the mean spatial deviation from the target for each session.

RESULTS

Tracking precision was significantly influenced by the interaction between shift and session, suggesting a clear time-of-day effect of visuomotor performance under real-life conditions. Tracking performance declined during early-morning hours whereas fatigue ratings increased.

CONCLUSION

These findings suggest that our setup is suitable to detect time-of-day dependent performance changes in visually guided tracking.

APPLICATION

Our task could be used to evaluate fluctuations in visuomotor coordination, a skill that is decisive in various production steps at the actual working place to assess productivity.

Manual control age and sex differences in 4 to 11 year old children

To what degree does being male or female influence the development of manual skills in pre-pubescent children? This question is important because of the emphasis placed on developing important new manual skills during this period of a child's education (e.g. writing, drawing, using computers). We investigated age and sex-differences in the ability of 422 children to control a handheld stylus. A task battery deployed using tablet PC technology presented interactive visual targets on a computer screen whilst simultaneously recording participant's objective kinematic responses, via their interactions with the on-screen stimuli using the handheld stylus. The battery required children use the stylus to: (i) make a series of aiming movements, (ii) trace a series of abstract shapes and (iii) track a moving object. The tasks were not familiar to the children, allowing measurement of a general ability that might be meaningfully labelled ‘manual control’, whilst minimising culturally determined differences in experience (as much as possible). A reliable interaction between sex and age was found on the aiming task, with girls' movement times being faster than boys in younger age groups (e.g. 4–5 years) but with this pattern reversing in older children (10–11 years). The improved performance in older boys on the aiming task is consistent with prior evidence of a male advantage for gross-motor aiming tasks, which begins to emerge during adolescence. A small but reliable sex difference was found in tracing skill, with girls showing a slightly higher level of performance than boys irrespective of age. There were no reliable sex differences between boys and girls on the tracking task. Overall, the findings suggest that prepubescent girls are more likely to have superior manual control abilities for performing novel tasks. However, these small population differences do not suggest that the sexes require different educational support whilst developing their manual skills.

Congruency of gaze metrics in action, imagery and action observation

The aim of this paper is to provide a review of eye movements during action execution, action observation, and movement imagery. Furthermore, the paper highlights aspects of congruency in gaze metrics between these states. The implications of the imagery, observation, and action gaze congruency are discussed in terms of motor learning and rehabilitation. Future research directions are outlined in order to further the understanding of shared gaze metrics between overt and covert states. Suggestions are made for how researchers and practitioners can structure action observation and movement imagery interventions to maximize (re)learning.

Kinematic property of target motion conditions gaze behavior and eye-hand synergy during manual tracking

This study investigated how frequency demand and motion feedback influenced composite ocular movements and eye-hand synergy during manual tracking. Fourteen volunteers conducted slow and fast force-tracking in which targets were displayed in either line-mode or wave-mode to guide manual tracking with target movement of direct position or velocity nature. The results showed that eye-hand synergy was a selective response of spatiotemporal coupling conditional on target rate and feedback mode. Slow and line-mode tracking exhibited stronger eye-hand coupling than fast and wave-mode tracking. Both eye movement and manual action led the target signal during fast-tracking, while the latency of ocular navigation during slow-tracking depended on the feedback mode. Slow-tracking resulted in more saccadic responses and larger pursuit gains than fast-tracking. Line-mode tracking led to larger pursuit gains but fewer and shorter gaze fixations than wave-mode tracking. During slow-tracking, incidences of saccade and gaze fixation fluctuated across a target cycle, peaking at velocity maximum and the maximal curvature of target displacement, respectively. For line-mode tracking, the incidence of smooth pursuit was phase-dependent, peaking at velocity maximum as well. Manual behavior of slow or line-mode tracking was better predicted by composite eye movements than that of fast or wave-mode tracking. In conclusion, manual tracking relied on versatile visual strategies to perceive target movements of different kinematic properties, which suggested a flexible coordinative control for the ocular and manual sensorimotor systems.

Drawing from memory: hand-eye coordination at multiple scales

Eyes move to gather visual information for the purpose of guiding behavior. This guidance takes the form of perceptual-motor interactions on short timescales for behaviors like locomotion and hand-eye coordination. More complex behaviors require perceptual-motor interactions on longer timescales mediated by memory, such as navigation, or designing and building artifacts. In the present study, the task of sketching images of natural scenes from memory was used to examine and compare perceptual-motor interactions on shorter and longer timescales. Eye and pen trajectories were found to be coordinated in time on shorter timescales during drawing, and also on longer timescales spanning study and drawing periods. The latter type of coordination was found by developing a purely spatial analysis that yielded measures of similarity between images, eye trajectories, and pen trajectories. These results challenge the notion that coordination only unfolds on short timescales. Rather, the task of drawing from memory evokes perceptual-motor encodings of visual images that preserve coarse-grained spatial information over relatively long timescales as well.

Changes in Timing and kinematics of goal directed eye-hand movements in early-stage Parkinson's disease

Objective

Many daily activities involve intrinsic or extrinsic goal-directed eye and hand movements. An extensive visuomotor coordination network including nigro-striatal pathways is required for efficient timing and positioning of eyes and hands. The aim of this study was to investigate how Parkinson’s disease (PD) affects eye-hand coordination in tasks with different cognitive complexity.

Methods

We used a touch screen, an eye-tracking device and a motion capturing system to quantify changes in eye-hand coordination in early-stage PD patients (H&Y < 2.5) and age-matched controls. Timing and kinematics of eye and hand were quantified in four eye-hand coordination tasks (pro-tapping, dual planning, anti-tapping and spatial memory task).

Results

In the pro-tapping task, saccade initiation towards extrinsic goals was not impaired. However, in the dual planning and anti-tapping task initiation of saccades towards intrinsic goals was faster in PD patients. Hand movements were differently affected: initiation of the hand movement was only delayed in the pro-tapping and dual planning task. Overall, hand movements in PD patients were slower executed compared to controls.

Interpretation

Whereas initiation of saccades in an extrinsic goal-directed task (pro-tapping task) is not affected, early stage PD patients have difficulty in suppressing reflexive saccades towards extrinsic goals in tasks where the endpoint is an intrinsic goal (e.g. dual planning and anti-tapping task). This is specific for eye movements, as hand movements have delayed responses in the pro-tapping and dual planning task. This suggests that reported impairment of the dorsolateral prefrontal cortex in early-stage PD patients affects only inhibition of eye movements. We conclude that timing and kinematics of eye and hand movements in visuomotor tasks are affected in PD patients. This result may have clinical significance by providing a behavioral marker for the early diagnosis of PD.

Eye-hand synergy and intermittent behaviors during target-directed tracking with visual and non-visual information

Visual feedback and non-visual information play different roles in tracking of an external target. This study explored the respective roles of the visual and non-visual information in eleven healthy volunteers who coupled the manual cursor to a rhythmically moving target of 0.5 Hz under three sensorimotor conditions: eye-alone tracking (EA), eye-hand tracking with visual feedback of manual outputs (EH tracking), and the same tracking without such feedback (EHM tracking). Tracking error, kinematic variables, and movement intermittency (saccade and speed pulse) were contrasted among tracking conditions. The results showed that EHM tracking exhibited larger pursuit gain, less tracking error, and less movement intermittency for the ocular plant than EA tracking. With the vision of manual cursor, EH tracking achieved superior tracking congruency of the ocular and manual effectors with smaller movement intermittency than EHM tracking, except that the rate precision of manual action was similar for both types of tracking. The present study demonstrated that visibility of manual consequences altered mutual relationships between movement intermittency and tracking error. The speed pulse metrics of manual output were linked to ocular tracking error, and saccade events were time-locked to the positional error of manual tracking during EH tracking. In conclusion, peripheral non-visual information is critical to smooth pursuit characteristics and rate control of rhythmic manual tracking. Visual information adds to eye-hand synchrony, underlying improved amplitude control and elaborate error interpretation during oculo-manual tracking.

Very slow search and reach: failure to maximize expected gain in an eye-hand coordination task

We examined an eye-hand coordination task where optimal visual search and hand movement strategies were inter-related. Observers were asked to find and touch a target among five distractors on a touch screen. Their reward for touching the target was reduced by an amount proportional to how long they took to locate and reach to it. Coordinating the eye and the hand appropriately would markedly reduce the search-reach time. Using statistical decision theory we derived the sequence of interrelated eye and hand movements that would maximize expected gain and we predicted how hand movements should change as the eye gathered further information about target location. We recorded human observers' eye movements and hand movements and compared them with the optimal strategy that would have maximized expected gain. We found that most observers failed to adopt the optimal search-reach strategy. We analyze and describe the strategies they did adopt.

WHAT THE DRAUGHTSMAN'S HAND TELLS THE DRAUGHTSMAN'S EYE: A SENSORIMOTOR ACCOUNT OF DRAWING

In this paper we address the challenging problem of sensorimotor integration, with reference to eye-hand coordination of an artificial agent engaged in a natural drawing task. Under the assumption that eye-hand coupling influences observed movements, a motor continuity hypothesis is exploited to account for how gaze shifts are constrained by hand movements. A Bayesian model of such coupling is presented in the form of a novel Dynamic Bayesian Network, namely an Input–Output Coupled Hidden Markov Model. Simulation results are compared to those obtained by eye-tracked human subjects involved in drawing experiments.

Role of the Rostral Superior Colliculus in Gaze Anchoring During Reach Movements

When reaching for an object, primates usually look at their target before touching it with the hand. This gaze movement prior to the arm movement allows target fixation, which is usually prolonged until the target is reached. In this manner, a stable image of the object is provided on the fovea during the reach, which is crucial for guiding the final part of the hand trajectory by visual feedback. Here we investigated a neural substrate possibly responsible for this behavior. In particular we tested the influence of reaching movements on neurons recorded at the rostral pole of the superior colliculus (rSC), an area classically related to fixation. Most rSC neurons showed a significant increase in their activity during reaching. Moreover, this increase was particularly high when the reaching movements were preceded by corresponding saccades to the targets to be reached, probably revealing a stronger coupling of the oculo-manual neural system during such a natural task. However, none of the parameters tested—including movement kinematics and target location—was found to be closely related to the observed increase in neural activity. Thus the increase in activity during reaching was found to be rather nonspecific except for its dependence on whether the reach was produced in isolation or in combination with a gaze movement. These results identify the rSC as a neural substrate sufficient for gaze anchoring during natural reaching movements, placing its activity at the core of the neural system dedicated to eye-hand coordination.

Influence of visual feedback and speed on micromanipulation accuracy

Accuracy in micromanipulation tasks is limited and it is important to identify various factors affecting it. This paper studies the effect of visual magnification, speed and handedness to micromanipulation accuracy using microscope and LCD screen for feedback. Magnification of visual feedback increases the accuracy, but large magnification does not provide further improvement beyond 16x. Further, we observed a trade off between speed and accuracy in tracing a circular path, i.e. faster speed reduces the speed control ability of the hand. Finally, dominant/non-dominant hand is found to affect accuracy in motion.

The contribution of the parietal lobes to speaking and writing

The left parietal lobe has been proposed as a major language area. However, parietal cortical function is more usually considered in terms of the control of actions, contributing both to attention and cross-modal integration of external and reafferent sensory cues. We used positron emission tomography to study normal subjects while they overtly generated narratives, both spoken and written. The purpose was to identify the parietal contribution to the modality-specific sensorimotor control of communication, separate from amodal linguistic and memory processes involved in generating a narrative. The majority of left and right parietal activity was associated with the execution of writing under visual and somatosensory control irrespective of whether the output was a narrative or repetitive reproduction of a single grapheme. In contrast, action-related parietal activity during speech production was confined to primary somatosensory cortex. The only parietal area with a pattern of activity compatible with an amodal central role in communication was the ventral part of the left angular gyrus (AG). The results of this study indicate that the cognitive processing of language within the parietal lobe is confined to the AG and that the major contribution of parietal cortex to communication is in the sensorimotor control of writing.

Visuomotor characterization of eye movements in a drawing task

Understanding visuomotor coordination requires the study of tasks that engage mechanisms for the integration of visual and motor information; in this paper we choose a paradigmatic yet little studied example of such a task, namely realistic drawing. On the one hand, our data indicate that the motor task has little influence on which regions of the image are overall most likely to be fixated: salient features are fixated most often. Viceversa, the effect of motor constraints is revealed in the temporal aspect of the scanpaths: (1) subjects direct their gaze to an object mostly when they are acting upon (drawing) it; and (2) in support of graphically continuous hand movements, scanpaths resemble edge-following patterns along image contours. For a better understanding of such properties, a computational model is proposed in the form of a novel kind of Dynamic Bayesian Network, and simulation results are compared with human eye-hand data.

Eye-hand strategies in copying complex lines

Eye movements and eye–hand interactions have been recorded for 10 beginner art students copying complex lines representing outlines of caricature heads seen in profile. Four copying conditions mimicking real-world drawing situations were tested: Direct copying where the original and copy were placed side by side, Direct Blind copying where the subject could not see the drawing hand and copy, Memory copying where the original was first memorized for drawing and subsequently hidden before drawing commenced, and Non-specific Memory copying where the original was encoded for facial recognition before being hidden and drawn from memory. We observed four very different eye–hand interaction strategies which provide evidence for the eye's dual role in the copying process: acquiring visual information in order to activate the visuomotor transformation and guiding the hand on the paper. The Direct copying strategies were best understood in terms of a Drawing Hypothesis stating that shape is the result of visuomotor mapping alone and, consequently, can be accurately drawn without vision of the drawing hand or paper. A double just-in-time mechanism is proposed whereby the eye refers alternatively to the original for shape and to the copy for spatial position just in time for the drawing action to proceed continuously.

Shape distortion produced by isolated mismatch between vision and proprioception

To investigate the nature of the visuomotor transformation, previous studies have used pointing tasks and examined how adaptation to a spatially localized mismatch between vision and proprioception generalizes across the workspace. Whereas some studies found extensive spatial generalization of single-point remapping, consistent with the hypothesis of a global realignment of visual and proprioceptive spaces, other studies reported limited transfer associated with variations in initial limb posture. Here, we investigated the effects of spatially localized remapping in the context of a visuomanual tracking task. Subjects tracked a visual target tracing a simple two-dimensional geometrical form without visual feedback except at a single point, where the visual display of the hand was shifted relative to its actual position. After adaptation, hand paths exhibited distortions relative to the visual templates that were inconsistent with the idea of a global realignment of visual and proprioceptive spaces. Results of a visuoproprioceptive matching task showed that these distortions were not limited to active movements but also affected perception of passive limb movements.

Differentiation between external and internal cuing: an fMRI study comparing tracing with drawing

Externally cued movement is thought to preferentially involve cerebellar and premotor circuits whereas internally generated movement recruits basal ganglia, pre-supplementary motor cortex (pre-SMA) and dorsolateral prefrontal cortex (DLPFC). Tracing and drawing are exemplar externally and internally guided actions and Parkinson's patients and cerebellar patients show deficits in tracking and drawing, respectively. In this study we aimed to examine this external/internal distinction in healthy subjects using functional imaging. Ten healthy subjects performed tracing and drawing of simple geometric shapes using pencil and paper while in a 3-T fMRI scanner. Results indicated that compared to tracing, drawing generated greater activation in the right cerebellar crus I, bilateral pre-SMA, right dorsal premotor cortex and right frontal eye field. Tracing did not recruit any additional activation compared to drawing except in striate and extrastriate visual areas. Therefore, drawing recruited areas more frequently associated with cognitively challenging tasks, attention and memory, but basal ganglia and cerebellar activity did not differentiate tracing from drawing in the hypothesised manner. As our paradigm was of a simple, repetitive and static design, these results suggest that the task familiarity and the temporal nature of visual feedback in tracking tasks, compared to tracing, may be important contributing factors towards the degree of cerebellar involvement. Future studies comparing dynamic with static external cues and visual feedback may clarify the role of the cerebellum and basal ganglia in the visual guidance of drawing actions.