On-line feedback control of human visually guided slow ramp tracking: effects of spatial separation of visual cues

Differences in left and right lower limb control strategies in coping with visual tracking tasks during bipedal standing

Introduction

Differences in motor control between the lower limbs may influence the risk of sports injury and recovery from rehabilitation. In this study, differences in the visual feedback ability of the left and right lower limbs were assessed using visual target tracking tasks.

Methods

Thirty-four healthy young subjects (aged 20.4 ± 1.2 years) were asked to move their bodies back and forth while tracking a visual target displayed on a monitor in front of them for 30 s. The two target motions were sinusoidal (i.e., predictable patterns) and more complex (random) patterns. To assess the ability of the lower limbs to follow visual target tracking, antero-posterior CoP (right limb, CoPap-r; left limb, CoPap-l) and medio-lateral CoP (right limb, CoPml-r; left limb, CoPml-l) data were measured using a stabilometer. Tracking ability by visual feedback ability was calculated as the difference in displacement between the target signal and the trajectories of the right and left pressure centers as trapezoidal areas, and a smaller sum of area (SoA) over the entire measurement time was defined as a greater tracking ability.

Results

Regarding the SoA in the anterior-posterior CoP, the mean SoA in the sinusoidal and random tasks was significantly lower in the CoP-r data than in the CoP-l data, indicating that the right lower limb had a more remarkable ability to follow visual target tracking. Regarding the SoA in the medial-lateral direction (CoP), the mean SoA in the sinusoidal and random tasks did not significantly differ between the two legs.

Discussion

The right lower limb may have a tracking function activated by the target signal when responding to visual stimuli. Identifying the motor strategies of each lower limb in response to visual stimuli will not only help identify potential differences between each lower limb but also suggest the possibility of enhancing the role of each lower limb in balance control.

The microstructure of intra- and interpersonal coordination

Movements are naturally composed of submovements, i.e. recurrent speed pulses (2-3 Hz), possibly reflecting intermittent feedback-based motor adjustments. In visuomotor (unimanual) synchronization tasks, partners alternate submovements over time, indicating mutual coregulation. However, it is unclear whether submovement coordination is organized differently between and within individuals. Indeed, different types of information may be variably exploited for intrapersonal and interpersonal coordination. Participants performed a series of bimanual tasks alone or in pairs, with or without visual feedback (solo task only). We analysed the relative timing of submovements between their own hands or between their own hands and those of their partner. Distinct coordinative structures emerged at the submovement level depending on the relevance of visual feedback. Specifically, the relative timing of submovements (between partners/effectors) shifts from alternation to simultaneity and a mixture of both when coordination is achieved using vision (interpersonal), proprioception/efference-copy only (intrapersonal, without vision) or all information sources (intrapersonal, with vision), respectively. These results suggest that submovement coordination represents a behavioural proxy for the adaptive weighting of different sources of information within action-perception loops. In sum, the microstructure of movement reveals common principles governing the dynamics of sensorimotor control to achieve both intra- and interpersonal coordination.

Interpersonal synchronization of movement intermittency

Most animal species group together and coordinate their behavior in quite sophisticated manners for mating, hunting, or defense purposes. In humans, coordination at a macroscopic level (the pacing of movements) is evident both in daily life (e.g., walking) and skilled (e.g., music and dance) behaviors. By examining the fine structure of movement, we here show that interpersonal coordination is established also at a microscopic – submovement – level. Natural movements appear as marked by recurrent (2–3 Hz) speed breaks, i.e., submovements, that are traditionally considered the result of intermittency in (visuo)motor feedback-based control. In a series of interpersonal coordination tasks, we show that submovements produced by interacting partners are not independent but alternate tightly over time, reflecting online mutual adaptation. These findings unveil a potential core mechanism for behavioral coordination that is based on between-persons synchronization of the intrinsic dynamics of action-perception cycles.

•

Movements show intermittent speed pulses occurring at 2–3 Hz, called submovements

•

Submovements are actively coordinated in counter-phase by interacting partners

•

Submovements coordination depends on spatial alignment but not movement congruency

•

Behavioral coordination occurs both at macro- and microscopic movement scales

Effects of Future Information and Trajectory Complexity on Kinematic Signal and Muscle Activation during Visual-Motor Tracking

Visual-motor tracking movement is a common and essential behavior in daily life. However, the contribution of future information to visual-motor tracking performance is not well understood in current research. In this study, the visual-motor tracking performance with and without future-trajectories was compared. Meanwhile, three task demands were designed to investigate their impact. Eighteen healthy young participants were recruited and instructed to track a target on a screen by stretching/flexing their elbow joint. The kinematic signals (elbow joint angle) and surface electromyographic (EMG) signals of biceps and triceps were recorded. The normalized integrated jerk (NIJ) and fuzzy approximate entropy (fApEn) of the joint trajectories, as well as the multiscale fuzzy approximate entropy (MSfApEn) values of the EMG signals, were calculated. Accordingly, the NIJ values with the future-trajectory were significantly lower than those without future-trajectory (p-value < 0.01). The smoother movement with future-trajectories might be related to the increasing reliance of feedforward control. When the task demands increased, the fApEn values of joint trajectories increased significantly, as well as the MSfApEn of EMG signals (p-value < 0.05). These findings enrich our understanding about visual-motor control with future information.

Development of a quantitative evaluation system for visuo-motor control in three-dimensional virtual reality space

The process of learning a human's movement and motor control mechanisms by watching and mimicking human motions was based on visuo-motor control in three dimensional space. However, previous studies regarding the visuo-motor control in three dimensional space have focused on analyzing the tracking tasks along one-dimensional lines or two-dimensional planes using single or multi-joint movements. Therefore, in this study, we developed a new system to quantitatively evaluate visuo-motor control in three-dimensional space based on virtual reality (VR) environment. Our proposed system is designed to analyze circular tracking movements on frontal and sagittal planes in VR space with millimeter level accuracy. In particular, we compared the circular tracking movements under monocular and binocular vision conditions. The results showed that the accuracy of circular tracking movements drops approximately 4.5 times in monocular vision than that in binocular vision on both frontal and sagittal planes. We also found that significant difference can be observed between frontal and sagittal planes for only the accuracy of X-axis in both monocular and binocular visions.

Sensorimotor control of tracking movements at various speeds for stroke patients as well as age-matched and young healthy subjects

There are aging- and stroke-induced changes on sensorimotor control in daily activities, but their mechanisms have not been well investigated. This study explored speed-, aging-, and stroke-induced changes on sensorimotor control. Eleven stroke patients (affected sides and unaffected sides) and 20 control subjects (10 young and 10 age-matched individuals) were enrolled to perform elbow tracking tasks using sinusoidal trajectories, which included 6 target speeds (15.7, 31.4, 47.1, 62.8, 78.5, and 94.2 deg/s). The actual elbow angle was recorded and displayed on a screen as visual feedback, and three indicators, the root mean square error (RMSE), normalized integrated jerk (NIJ) and integral of the power spectrum density of normalized speed (IPNS), were used to investigate the strategy of sensorimotor control. Both NIJ and IPNS had significant differences among the four groups (P<0.01), and the values were ranked in the following order: young controls < age-matched controls Collapse

Key Words Collapse

MESH Headings

• Adult
• Case-Control Studies
• Elbow/physiopathology
• Female
• Humans
• Male
• Middle Aged
• Movement/physiology
• Psychomotor Performance/physiology
• Stroke/physiopathology
• Young Adult

Collapse

Grants Collapse

Affiliation(s)

Di Ao School of Engineering, Sun Yat-sen University, Guangzhou, Guang Dong, P. R. China Key Laboratory of Sensing Technology and Biomedical Instrument of GuangDong province, Guangzhou, Guang Dong, P. R. China
Rong Song School of Engineering, Sun Yat-sen University, Guangzhou, Guang Dong, P. R. China Key Laboratory of Sensing Technology and Biomedical Instrument of GuangDong province, Guangzhou, Guang Dong, P. R. China
Kai-yu Tong Division of Biomedical Engineering, Department of Electronic Engineering, the Chinese University of Hong Kong, Hong Kong, China

Collapse

A three-component model of the control error in manual tracking of continuous random signals

OBJECTIVE

The performance of human operators acting within closed-loop control systems is investigated in a classic tracking task. The dependence of the control error (tracking error) on the parameters display gain, k(display), and input signal frequency bandwidth, f(g), which alter task difficulty and presumably the control delay, is studied with the aim of functionally specifying it via a model.

BACKGROUND

The human operator as an element of a cascaded human-machine control system (e.g., car driving or piloting an airplane) codetermines the overall system performance. Control performance of humans in continuous tracking has been described in earlier studies.

METHOD

Using a handheld joystick, 10 participants tracked continuous random input signals. The parameters f(g) and k(display) were altered between experiments.

RESULTS

Increased task difficulty promoted lengthened control delay and, consequently, increased control error.Tracking performance degraded profoundly with target deflection components above 1 Hz, confirming earlier reports.

CONCLUSION

The control error is composed of a delay-induced component, a demand-based component, and a novel component: a human tracking limit. Accordingly, a new model that allows concepts of the observed control error to be split into these three components is suggested.

APPLICATION

To achieve optimal performance in control systems that include a human operator (e.g., vehicles, remote controlled rovers, crane control), (a) tasks should be kept as simple as possible to achieve shortest control delays, and (b) task components requiring higher-frequency (> 1 Hz) tracking actions should be avoided or automated by technical systems.

Submovement organization, pen pressure, and muscle activity are modulated to precision demands in 2D tracking

The authors investigated how tracking performance, submovement organization, pen pressure and muscle activity in forearm and shoulder muscles were affected by target size in a 2D tracking task performed with a pen on a digitizer tablet. Twenty-six subjects took part in an experiment, in which either a small dot or a large dot was tracked, while it moved quasirandomly across a computer screen at a constant velocity of 2 cm/s. The manipulation of precision level was successful, because mean distance to target and the standard deviation of this distance were significantly smaller with the small target than with the large target. With a small target, subjects trailed more behind the center of target and used submovements with larger amplitudes and of shorter duration, resulting in higher tracking accuracy. This change in submovement organization was accompanied by higher pen pressure, while at the same time muscle activity in the forearm extensors and flexors was increased, indicating higher endpoint stability. In conclusion, increased precision demands were accommodated by both a different organization of submovements and higher endpoint stability in a 2D tracking task performed with a pen on a digitizer tablet.

Overcoming the guidance effect in motor skill learning: feedback all the time can be beneficial

Extensive research has shown that augmented feedback presented too often can create a dependency on the feedback and hinder long-term memory formation of a motor skill. This dependency has been labeled the guidance effect, and one way to overcome the guidance effect is to reduce how often augmented feedback is presented during training. In two experiments, participants were presented with visual augmented feedback during every trial in a 5-min training interval. Participants were provided visual augmented feedback in the form of a Lissajous template of a 1:2 multi-frequency pattern and a cursor representing the coordination between the limbs. Some participants were trained with the cursor superimposed (behind group) on the Lissajous template, and others were trained with the cursor presented in a separate window (side group) from the Lissajous template. In experiment 1, motion of the end-effectors was constrained to the medial-lateral direction in the horizontal plane. In experiment 2, end-effector motion was possible in both the medial-lateral and anterior-posterior directions in the horizontal plane. The location of the cursor did not influence performance during the 5-min training interval in either experiment. After a 15-min break, a retention test performed without the visual feedback provided by the cursor revealed that the behind groups' performance was guided by the visual feedback in both experiments, whereas the side groups were able to perform without visual feedback. In experiment two, the side group's performance without feedback was influenced when anterior-posterior motion was not constrained; however, the extent of the guidance effect was significantly less compared to the behind trained group in both experiments. The results show that the emergence of guided motor performance depends on the format of the display that provides visually based augmented feedback, and not just on how often the feedback is provided. In conclusion, visually based augmented feedback leads to the simultaneous development of a spatial and motor representation of the task. The behind format led to a dependence on the spatial representation developed during training, while the side format facilitated the development of the motor representation as a means to overcome guidance.

The integration of temporally shifted visual feedback in a synchronization task: The role of perceptual stability in a visuo-proprioceptive conflict situation

The present study examined whether the beneficial role of coherently grouped visual motion structures for performing complex (interlimb) coordination patterns can be generalized to synchronization behavior in a visuo-proprioceptive conflict situation. To achieve this goal, 17 participants had to synchronize a self-moved circle, representing the arm movement, with a visual target signal corresponding to five temporally shifted visual feedback conditions (0%, 25%, 50%, 75%, and 100% of the target cycle duration) in three synchronization modes (in-phase, anti-phase, and intermediate). The results showed that the perception of a newly generated perceptual Gestalt between the visual feedback of the arm and the target signal facilitated the synchronization performance in the preferred in-phase synchronization mode in contrast to the less stable anti-phase and intermediate mode. Our findings suggest that the complexity of the synchronization mode defines to what extent the visual and/or proprioceptive information source affects the synchronization performance in the present unimanual synchronization task.

Learning an environment-actor coordination skill: visuomotor transformation and coherency of perceptual structure

The coordination dynamics of learning were examined in a visuomotor tracking task. Participants produced rhythmic elbow flexion-extension motions to learn a visually defined 90 degrees relative phase tracking pattern with an external sinusoidal signal. There were two visuomotor transformation groups, a correct feedback group and a mirrored feedback group with feedback representing the elbow's motion transformed by 180 degrees . In Experiment 1, the to-be-tracked signal and the participant's motion signal were superimposed within a single window display. In Experiment 2, the to-be-tracked signal and participant's signal were presented in separate windows. Before day 1 practice and 24 h after day 2 practice, participants attempted visually defined 0 degrees , 45 degrees , 90 degrees , 135 degrees , and 180 degrees relative phase tracking patterns either with or without visual feedback of the arm's motion. Before practice, only the 0 degrees and 180 degrees tracking patterns were stable. Practice led to a decrease in phase error toward the required 90 degrees relative phase pattern with a corresponding increase in stability in both the experiments. No effect of visual transformation on performance emerged during practice in the single window task, but did emerge in the two window task. The one window training facilitated transfer to the four unpracticed relative phase patterns, whereas the two window training display only facilitated transfer performance to a single unpracticed relative phase pattern. These findings suggest that the perceptual structure determined the degree of learning and transfer and interacted with the visuomotor transformation. The present findings are discussed with reference to how the visual display constrains the coherency of independent signals with regard to learning and transfer and the role of perceptual discrimination processes linked to transfer.

Afferent contributions to digit force coupling and force level variation during performance of non-lift pinch

Afferent contributions to the coordination of thumb and index finger forces during non-lift pinch were studied using an anesthetization case study design. Two subjects, one performing with and without digital anesthetization and one with intact sensation, produced dynamic pinch forces against a stable object, with and without visual feedback. Error corrections were less frequent post-anesthetization, and the cross correlation between digit forces was lower when sensation was removed. However, this decrease in cross correlation between digit forces seemed to reflect a loss in the magnitude of tightly coupled error corrections when sensation was removed, rather than more frequent deviations of force magnitude between the digit forces. Force-time output without visual feedback lacked these error corrections, and the correlation between digit forces remained high, irrespective of sensory status. Additionally, with vision occluded, the time rate of force change did not vary in a gradual manner as would be expected from a neural representation of a sinusoidal target, but was instead marked by sudden abrupt reversals of force rate of change, invariant of somatosensory status. The coupling of digit forces and rates of force change during non-lift pinch appear to be controlled primarily with feedforward mechanisms, where the lack of proprioceptive feedback does not seem to disrupt this coupling.

Manually controlled human balancing using visual, vestibular and proprioceptive senses involves a common, low frequency neural process

Ten subjects balanced their own body or a mechanically equivalent unstable inverted pendulum by hand, through a compliant spring linkage. Their balancing process was always characterized by repeated small reciprocating hand movements. These bias adjustments were an observable sign of intermittent alterations in neural output. On average, the adjustments occurred at intervals of approximately 400 ms. To generate appropriate stabilizing bias adjustments, sensory information about body or load movement is needed. Subjects used visual, vestibular or proprioceptive sensation alone and in combination to perform the tasks. We first ask, is the time between adjustments (bias duration) sensory specific? Vision is associated with slow responses. Other senses involved with balance are known to be faster. Our second question is; does bias duration depend on sensory abundance? An appropriate bias adjustment cannot occur until unplanned motion is unambiguously perceived (a sensory threshold). The addition of more sensory data should therefore expedite action, decreasing the mean bias adjustment duration. Statistical analysis showed that (1) the mean bias adjustment duration was remarkably independent of the sensory modality and (2) the addition of one or two sensory modalities made a small, but significant, decrease in the mean bias adjustment duration. Thus, a threshold effect can alter only a very minor part of the bias duration. The bias adjustment duration in manual balancing must reflect something more than visual sensation and perceptual thresholds; our suggestion is that it is a common central motor planning process. We predict that similar processes may be identified in the control of standing.

Eye-hand interactions in tracing and drawing tasks

We report a preliminary analysis of the interactions between eye and hand during tracing and drawing of four simple shapes. Eye and hand movements were recorded using the ASL 504 system and the Flock of Birds system, respectively. During tracing, pen tip and eye were tightly coupled, with participants making a series of small saccades just in front of the moving pen, interspersed with periods of smooth pursuit. During drawing, saccades were fewer and larger and pursuit was less frequent. Observed eye-hand interactions suggested a bidirectional relationship between the eye and hand. These findings are explained in terms of the differing degree that the two tasks employ visual detail, external or internal cues and eye-hand coordination.

The effect of changed visual feedback on intention tremor in multiple sclerosis

In patients with multiple sclerosis (MS), intention tremor amplitude is enhanced during the visually guided compared to the memory guided motor tasks. In the present study, the effect of visual feedback on intention tremor was investigated during visually guided wrist step-tracking tasks. Specifically, visual feedback of the hand was provided either instantly or averaged over different time windows. Thirteen MS patients with intention tremor and 14 healthy controls performed the wrist step-tracking task, while the visual representation of the actual hand position was displayed instantly or averaged over time windows of 150, 250 and 350 ms. It has been found in the patient group that, in association with a decreased initial error and decreased tremor amplitude on the screen, the amplitude of the actual performed tremor also decreased when visual feedback was changed. The tremor reduction was not different between conditions with manipulated feedback, although delays in presenting visual feedback of the hand position increased when the time window was larger. The reduction in overall tremor amplitude was unlikely related to other factors, such as eye fixation deficits or the speed of the primary hand movement. These results suggest that hand tremor severity is dependent on the visual feedback of position and movement errors.

Effects of correct and transformed visual feedback on rhythmic visuo-motor tracking: Tracking performance and visual search behavior

The effects of correct and transformed visual feedback on rhythmic unimanual visuo-motor tracking were examined, focusing on tracking performance (accuracy and stability) and visual search behavior. Twelve participants (reduced to 9 in the analyses) manually tracked an oscillating visual target signal in phase (by moving the hand in the same direction as the target signal) and in antiphase (by moving the hand in the opposite direction), while the frequency of the target signal was gradually increased to probe pattern stability. Besides a control condition without feedback, correct feedback (representing the actual hand movement) or mirrored feedback (representing the hand movement transformed by 180 degrees) were provided during tracking, resulting in either in-phase or antiphase visual motion of the target and feedback signal, depending on the tracking mode performed. The quality (accuracy and stability) of in-phase tracking was hardly affected by the two forms of feedback, whereas antiphase tracking clearly benefited from mirrored feedback but not from correct feedback. This finding extends previous results indicating that the performance of visuo-motor coordination tasks is aided by visual feedback manipulations resulting in coherently grouped (i.e., in-phase) visual motion structures. Further insights into visuo-motor tracking with and without feedback were garnered from the visual search patterns accompanying task performance. Smooth pursuit eye movements only occurred at lower oscillation frequencies and prevailed during in-phase tracking and when target and feedback signal moved in phase. At higher frequencies, point-of-gaze was fixated at a location that depended on the feedback provided and the resulting visual motion structures. During in-phase tracking the mirrored feedback was ignored, which explains why performance was not affected in this condition. Point-of-gaze fixations at one of the end-points were accompanied by reduced motor variability at this location, reflecting a form of visuo-motor anchoring that may support the pick up of discrete information as well as the control of hand movements to a desired location.

Effect of tactile feedback on movement speed and precision during work-related tasks using a computer mouse

Effects of tactile feedback on movement accuracy and speed were studied. Younger and older participants performed three tasks (1, select and drag word; 2, menu navigation; 3, select and drag cell) using commercial software and a mouse with or without tactile feedback. Task time and error number were recorded. Tasks were divided according to presence or absence of tactile feedback, and participants were divided into subgroups (high, average, low) based on Task 1 performance. Overall, older participants took longer (p < .0001) and made more errors (p < .001) than younger participants. There was an effect of feedback by task in younger participants for all six outcomes (p < 0.02). At the task level, with feedback, younger participants reduced performance time (13%) and errors (24%) on Task 1. Low- and average-performance younger participants benefited most from feedback for Task 1. Older low-performance participants also benefited from feedback for Task 1. For Task 3, older participants tended to take more time and make more errors with feedback. Tactile feedback may enhance performance when feedback is event related. Older people may not integrate sensation as well as younger individuals to enhance performance. Potential applications of this research include the development of tactile feedback interfaces to facilitate computer use.

The effects of enhanced visual feedback on human synchronization

The execution of actions not only reposes on the spatial and temporal organization of the movements as such but also on their appropriate imbedding into the environmental spatio-temporal constraints. Actually, performance outcome appears to be strongly influenced by the strength of the perception-action coupling. The present experiment wants to examine to what degree this coupling strength affects the spatial and spatio-temporal characteristics of a synchronization task. In particular, the effects of: (i) enhanced visual feedback; and (ii) a modification in the spatial organization of the task were investigated. To do so, a task was designed in which horizontal arm movements had to be synchronized with a target light moving horizontally or vertically at a sinusoidal speed. Subjects performed six experimental conditions representing three synchronization modes (horizontal in-phase, horizontal anti-phase and orthogonal) and two feedback conditions (no feedback and feedback). The results for movement amplitude and relative phase revealed the operation of task specific effects. Apparently, the availability of feedback at the perception-action coupling level provoked the use of different strategies to cope with the constraints of this synchronization task.