Visual tracking of active and passive movements of the hand

Awareness of voluntary action, rather than body ownership, improves motor control

Awareness of the body is essential for accurate motor control. However, how this awareness influences motor control is poorly understood. The awareness of the body includes awareness of visible body parts as one’s own (sense of body ownership) and awareness of voluntary actions over that visible body part (sense of agency). Here, I show that sense of agency over a visible hand improves the initiation of movement, regardless of sense of body ownership. The present study combined the moving rubber hand illusion, which allows experimental manipulation of agency and body ownership, and the finger-tracking paradigm, which allows behavioral quantification of motor control by the ability to coordinate eye with hand movements. This eye–hand coordination requires awareness of the hand to track the hand with the eye. I found that eye–hand coordination is improved when participants experience a sense of agency over a tracked artificial hand, regardless of their sense of body ownership. This improvement was selective for the initiation, but not maintenance, of eye–hand coordination. These results reveal that the prospective experience of explicit sense of agency improves motor control, suggesting that artificial manipulation of prospective agency may be beneficial to rehabilitation and sports training techniques.

Active View and Passive View in Virtual Reality Have Different Impacts on Memory and Impression

Virtual reality (VR) through a head-mounted display (HMD) can provide new experiences. However, it remains unclear how the characteristics of HMDs affect users’ memory. To use HMDs more effectively and appropriately in several applied fields, including education, it is necessary to clarify what characteristics of HMDs affect users’ memory. A head-tracking function mounted on an HMD helps to detect the user’s head direction to enable a simulation experience akin to the real world. When we experience a simulation on an HMD, we actively perceive the visual world. In this study, we assessed how active/passive viewing affects users’ memory of VR content. We conducted a psychological experiment in which participants watched a movie on an HMD. In the active viewing condition, the presented view changed depending on the participant’s head direction. In the passive viewing condition, the presented view was a recorded movie that was shown to the participants in the active viewing condition. All participants took a memory test about the content presented in the movie on the day of viewing and 2 weeks later. The results showed that performance on the memory test in the active viewing condition was significantly lower than that in the passive viewing condition after 2 weeks. This result indicated that active viewing in VR inhibited users’ memory compared to passive viewing. The current study contributes to the development of new VR techniques, such as educational learning.

Robot End Effector Tracking Using Predictive Multisensory Integration

We propose a biologically inspired model that enables a humanoid robot to learn how to track its end effector by integrating visual and proprioceptive cues as it interacts with the environment. A key novel feature of this model is the incorporation of sensorimotor prediction, where the robot predicts the sensory consequences of its current body motion as measured by proprioceptive feedback. The robot develops the ability to perform smooth pursuit-like eye movements to track its hand, both in the presence and absence of visual input, and to track exteroceptive visual motions. Our framework makes a number of advances over past work. First, our model does not require a fiducial marker to indicate the robot hand explicitly. Second, it does not require the forward kinematics of the robot arm to be known. Third, it does not depend upon pre-defined visual feature descriptors. These are learned during interaction with the environment. We demonstrate that the use of prediction in multisensory integration enables the agent to incorporate the information from proprioceptive and visual cues better. The proposed model has properties that are qualitatively similar to the characteristics of human eye-hand coordination.

Three-dimensional binocular eye-hand coordination in normal vision and with simulated visual impairment

Sensorimotor coupling in healthy humans is demonstrated by the higher accuracy of visually tracking intrinsically-rather than extrinsically-generated hand movements in the fronto-parallel plane. It is unknown whether this coupling also facilitates vergence eye movements for tracking objects in depth, or can overcome symmetric or asymmetric binocular visual impairments. Human observers were therefore asked to track with their gaze a target moving horizontally or in depth. The movement of the target was either directly controlled by the observer's hand or followed hand movements executed by the observer in a previous trial. Visual impairments were simulated by blurring stimuli independently in each eye. Accuracy was higher for self-generated movements in all conditions, demonstrating that motor signals are employed by the oculomotor system to improve the accuracy of vergence as well as horizontal eye movements. Asymmetric monocular blur affected horizontal tracking less than symmetric binocular blur, but impaired tracking in depth as much as binocular blur. There was a critical blur level up to which pursuit and vergence eye movements maintained tracking accuracy independent of blur level. Hand-eye coordination may therefore help compensate for functional deficits associated with eye disease and may be employed to augment visual impairment rehabilitation.

Orienting to Targets by Looking and Pointing: Parallels and Interactions in Ocular and Manual Performance

Orienting to a target by looking and pointing is examined for parallels between the control of the two systems and interactions due to movement of the eyes and limb to the same target. Parallels appear early in orienting and may be due to common processing of spatial information for the ocular and manual systems. The eyes and limb both have shorter response latency to central visual and peripheral auditory targets. Each movement also has shorter latency and duration when the target presentation is short enough (200 msec) that no analysis of feedback of the target position is possible during the movement. Interactions appear at many stages of information processing for movement. Latency of ocular movement is much longer when the subject also points, and the eye and limb movement latencies are highly correlated for orienting to auditory targets. Final position of eyes and limb are significantly correlated only when target duration is short (200 msec). This illustrates that sensory information obtained before the movement begins is an important, but not the only, source of input about target position. Additional information that assists orienting may be passed from one system to another, since visual information gained by looking aided pointing to lights and proprioceptive information from the pointing hand seemed to assist the eyes in looking to sounds. Thus the production of this simple set of movements may be partly described by a cascade-type process of parallel analysis of spatial information for eye and hand control, but is also, later in the movement, assisted by cross-system interaction.

Role of motor execution in the ocular tracking of self-generated movements

When human observers track the movements of their own hand with their gaze, the eyes can start moving before the finger (i.e., anticipatory smooth pursuit). The signals driving anticipation could come from motor commands during finger motor execution or from motor intention and decision processes associated with self-initiated movements. For the present study, we built a mechanical device that could move a visual target either in the same direction as the participant's hand or in the opposite direction. Gaze pursuit of the target showed stronger anticipation if it moved in the same direction as the hand compared with the opposite direction, as evidenced by decreased pursuit latency, increased positional lead of the eye relative to target, increased pursuit gain, decreased saccade rate, and decreased delay at the movement reversal. Some degree of anticipation occurred for incongruent pursuit, indicating that there is a role for higher-level movement prediction in pursuit anticipation. The fact that anticipation was larger when target and finger moved in the same direction provides evidence for a direct coupling between finger and eye motor commands.

LRP predicts smooth pursuit eye movement onset during the ocular tracking of self-generated movements

Several studies have indicated that human observers are very efficient at tracking self-generated hand movements with their gaze, yet it is not clear whether this is simply a by-product of the predictability of self-generated actions or if it results from a deeper coupling of the somatomotor and oculomotor systems. In a first behavioral experiment we compared pursuit performance as observers either followed their own finger or tracked a dot whose motion was externally generated but mimicked their finger motion. We found that even when the dot motion was completely predictable in terms of both onset time and kinematics, pursuit was not identical to that produced as the observers tracked their finger, as evidenced by increased rate of catch-up saccades and by the fact that in the initial phase of the movement gaze was lagging behind the dot, whereas it was ahead of the finger. In a second experiment we recorded EEG in the attempt to find a direct link between the finger motor preparation, indexed by the lateralized readiness potential (LRP) and the latency of smooth pursuit. After taking into account finger movement onset variability, we observed larger LRP amplitudes associated with earlier smooth pursuit onset across trials. The same held across subjects, where average LRP onset correlated with average eye latency. The evidence from both experiments concurs to indicate that a strong coupling exists between the motor systems leading to eye and finger movements and that simple top-down predictive signals are unlikely to support optimal coordination.

Manual tracking enhances smooth pursuit eye movements

Previous studies have reported that concurrent manual tracking enhances smooth pursuit eye movements only when tracking a self-driven or a predictable moving target. Here, we used a control-theoretic approach to examine whether concurrent manual tracking enhances smooth pursuit of an unpredictable moving target. In the eye-hand tracking condition, participants used their eyes to track a Gaussian target that moved randomly along a horizontal axis. In the meantime, they used their dominant hand to move a mouse to control the horizontal movement of a Gaussian cursor to vertically align it with the target. In the eye-alone tracking condition, the target and cursor positions recorded in the eye-hand tracking condition were replayed, and participants only performed eye tracking of the target. Catch-up saccades were identified and removed from the recorded eye movements, allowing for a frequency-response analysis of the smooth pursuit response to unpredictable target motion. We found that the overall smooth pursuit gain was higher and the number of catch-up saccades made was less when eye tracking was accompanied by manual tracking than when not. We conclude that concurrent manual tracking enhances smooth pursuit. This enhancement is a fundamental property of eye-hand coordination that occurs regardless of the predictability of the target motion.

Proprioception plays a different role for sensorimotor adaptation to different distortions

If proprioceptive feedback is degraded by agonist-antagonist muscle vibration, then adaptation to rotated vision remains intact while adaptation to a velocity-dependent force field worsens. Here we evaluate whether this differential effect of vibration is related to the physical nature of the distortion - visual versus mechanical - or to their kinematic coupling to the subjects' hand - velocity versus position dependent. Subjects adapted to a velocity-dependent visual distortion, to a position-dependent force, or to a velocity-dependent force; one half of the subjects adapted with, and the other half without agonist-antagonist vibration at the wrist, elbow, and shoulder. We found, as before, that vibration slowed down adaptation to a velocity-dependent force. However, vibration did not modify adaptation to the other two distortions, nor did it influence the aftereffects of any distortion. From this we conclude that intact proprioception supports strategic compensatory processes when proprioceptive signals agree with visual ones, and provide relevant (dynamic) information not available to the visual system.

Introducing a feedback training system for guided home rehabilitation

As the number of people requiring orthopaedic intervention is growing, individualized physiotherapeutic rehabilitation and adequate postoperative care becomes increasingly relevant. The chances of improvement in the patients condition is directly related to the performance and consistency of the physiotherapeutic exercises.

In this paper a smart, cost-effective and easy to use Feedback Training System for home rehabilitation based on standard resistive elements is introduced. This ensures high accuracy of the exercises performed and offers guidance and control to the patient by offering direct feedback about the performance of the movements.

46 patients were recruited and performed standard physiotherapeutic training to evaluate the system. The results show a significant increase in the patient's ability to reproduce even simple physiotherapeutic exercises when being supported by the Feedback Training System. Thus physiotherapeutic training can be extended into the home environment whilst ensuring a high quality of training.

The Duncker illusion and eye-hand coordination

A moving background alters the perceived direction of target motion (the Duncker illusion). To test whether this illusion also affects pointing movements to remembered/extrapolated target locations, we constructed a display in which a target moved in a straight line and disappeared behind a band of moving random dots. Subjects were required to touch the spot where the target would emerge from the occlusion. The four directions of random-dot motion induced pointing errors that were predictable from the Duncker illusion. Because it has been previously established that saccadic direction is influenced by this illusion, gaze was subsequently recorded in a second series of experiments while subjects performed the pointing task and a similar task with eye-tracking only. In the pointing task, subjects typically saccaded to the lower border of the occlusion zone as soon as the target disappeared and then tried to maintain fixation at that spot. However, it was particularly obvious in the eye-tracking-only condition that horizontally moving random dots generally evoked an appreciable ocular following response, altering the gaze direction. Hand-pointing errors were related to the saccadic gaze error but were more highly correlated with final gaze errors (resulting from the initial saccade and the subsequent ocular following response). The results suggest a model of limb control in which gaze position can provide the target signal for limb movement.

Manual tracking in two dimensions

Manual tracking was studied by asking subjects to follow, with their finger, a target moving on a touch-sensitive video monitor. The target initially moved in a straight line at a constant speed and then, at a random point in time, made one abrupt change in direction. The results were approximated with a simple model according to which, after a reaction time, the hand moved in a straight line to intercept the target. Both the direction of hand motion and its peak speed could be predicted by assuming a constant time to intercept. This simple model was able to account for results obtained over a broad range of target speeds as well as the results of experiments in which both the speed and the direction of the target changed simultaneously. The results of an experiment in which the target acceleration was nonzero suggested that the error signals used during tracking are related to both speed and direction but poorly (if at all) to target acceleration. Finally, in an experiment in which target velocity remained constant along one axis but the perpendicular component underwent a step change, tracking along both axes was perturbed. This last finding demonstrates that tracking in two dimensions cannot be decomposed into its Cartesian components. However, an analytical model in a hand-centered frame of reference in which speed and direction are the controlled variables could account for much of the data.

The relationship of target and response complexity in coincident timing performance

This paper reports two experiments concerned with the interaction of response and target factors in a coincident timing situation. Coincident timing involves executing a response to intercept a moving target. The target response relationship was manipulated in the first experiment by providing some targets with no structure (linear path) and others containing 1 and 2 bounce points. In the second experiment, responses required 0, 1, or 2 reversals in direction. The results provide tentative support for the notion that structure in the target path could be linked to the key temporal response dynamics for improved performance consistency, and that subjects may alter their response structure to incorporate this linkage. This can be further extended to the advantage of simply moving while perceiving movement. This paper stresses the importance of examining the perceptual and motor requirements of the coincident timing skill in unison.

Eye movements of teenage children of schizophrenics: a possible inherited marker of susceptibility to the disease

Teenage children each having one schizophrenic parent showed deficits in oculomotor control which are also found in schizophrenic subjects compared with controls. They made significantly more saccadic eye movements interrupting smooth pursuit ocular tracking. They also made significantly more double-jump saccadic movements in a looking task. Since this latter difference was true only when the visual target was illuminated during the saccade (p less than 0.001), the cause was likely visual processing and not a fixation stability problem. The two measures correlated significantly for all subjects (r = 0.63), and half the experimentals had scores more than two standard deviations above the mean for controls. This suggests that a pattern of oculomotor dysfunction found in schizophrenics which comes from a single source may act as a marker for susceptibility of their offspring to the disease.

Parallel Ocular and Manual Tracking Responses to a Continuously Moving Visual Target

Continuous ocular and manual tracking of the same visual target moving horizontally in sinusoids at 0.75 Hz was measured by lag, RMS Error, and Gain. The best measures of accuracy of tracking, error and lag, were remarkably similar in the two systems and were affected similarly by presence of a background and changes in predictability of target movement. Details of within-system performance varied despite the over-all parallels. Gain was different in adjustment of proportion of saccadic to pursuit movement was affected by the presence of the hand, even though this did not affect tracking accuracy. The over-all parallel of response adjustment suggests that a suprasystem decision-maker sets general response goals and each motor system adjusts output details to match these goals.

Motor control of schizophrenics--I. Oculomotor control of schizophrenics: a deficit in sensory processing, not strictly in motor control

An in-depth analysis of eye movements of schizophrenics has demonstrated defects in several oculomotor subsystems, likely due to processing of sensory information and not to problems in the motor system itself. Schizophrenic subjects made more pursuit tracking errors, due to higher RMS errors, larger phase lag, and more saccades in pursuit. When making saccadic movements they had normal reaction times, saccade velocity and intersaccadic pause, but made more double-jump saccades and overshot the target. Only some had inability to hold steady fixation, but all had an attentional problem in restraining an inappropriate eye movement. The multiple defects suggest a processing problem that is not strictly motor and does not fit any standard definition of attention.

Eye-hand tracking using afterimages. Evidence that sense of effort is dependent on spatial constancy mechanisms

Oculomotor tracking of one's unseen hand is greatly enhanced when a positive afterimage of the arm is projected onto its changing apparent position. The "afterimage arm" also influences the perception of real arm motion. The nature of this influence indicates that sense of effort or will derives from spatial computations potentially involving sensory and motor information about the ongoing orientation of the entire body.

Adaptation to visual displacement: contribution of proprioceptive, visual, and attentional factors

A study is reported of the sensory and oculomotor factors that influence adaptation to visual displacement when only sight of the hand and of a target light is permitted. Ten subjects each participated in five counterbalanced sessions in which the relative contributions to adaptation of limb proprioception, oculomotor pursuit, retinal image displacement, and attention were determined. Recordings of eye position were taken throughout the sessions. A discordance between the visual direction of the hand and the proprioceptively specified position of the hand was essential but not sufficient for adaptation to occur. When such a discordance was present adaptation was enhanced by improving position sense of the arm by moving it passively, maximizing accuracy of visual tracking, and focusing attention on the hand. Attentional factors played a smaller part than has been reported for exposure situations involving error feedback about limb position. Eye-movement recordings showed that tracking the passively moved hand visually was vastly superior to tracking a target light undergoing comparable motion.

Adaptation to visual displacement with active and passive limb movements: effect of movement frequency and predictability of movement

Three experiments were performed to evaluate the influence of active and passive limb movements on adaptation to visual displacement. Over a wide frequency range (0·5-1·25 Hz) with constant amplitude, 30°, significant adaptation was achieved with active and passive movements. When arm movement frequency was constant at 1·0 Hz but amplitude of movement was varied, less adaptation was achieved for both active and passive movements than when amplitude was held constant. Even at a frequency above that of most naturally occurring limb movements, 1·67 Hz, and with variable amplitude motion, significant adaptation was achieved with active and passive limb movements. These findings emphasize the importance of visual-proprioceptive discordances for adaptation to visual displacement when only sight of the hand is permitted. Significant differences did not appear between the active and passive movement conditions in any of the experiments.