Human hand moves proactively to the external stimulus: an evolutional strategy for minimizing transient error

Preparing to move: Setting initial conditions to simplify interactions with complex objects

Humans dexterously interact with a variety of objects, including those with complex internal dynamics. Even in the simple action of carrying a cup of coffee, the hand not only applies a force to the cup, but also indirectly to the liquid, which elicits complex reaction forces back on the hand. Due to underactuation and nonlinearity, the object's dynamic response to an action sensitively depends on its initial state and can display unpredictable, even chaotic behavior. With the overarching hypothesis that subjects strive for predictable object-hand interactions, this study examined how subjects explored and prepared the dynamics of an object for subsequent execution of the target task. We specifically hypothesized that subjects find initial conditions that shorten the transients prior to reaching a stable and predictable steady state. Reaching a predictable steady state is desirable as it may reduce the need for online error corrections and facilitate feed forward control. Alternative hypotheses were that subjects seek to reduce effort, increase smoothness, and reduce risk of failure. Motivated by the task of 'carrying a cup of coffee', a simplified cup-and-ball model was implemented in a virtual environment. Human subjects interacted with this virtual object via a robotic manipulandum that provided force feedback. Subjects were encouraged to first explore and prepare the cup-and-ball before initiating a rhythmic movement at a specified frequency between two targets without losing the ball. Consistent with the hypotheses, subjects increased the predictability of interaction forces between hand and object and converged to a set of initial conditions followed by significantly decreased transients. The three alternative hypotheses were not supported. Surprisingly, the subjects' strategy was more effortful and less smooth, unlike the observed behavior in simple reaching movements. Inverse dynamics of the cup-and-ball system and forward simulations with an impedance controller successfully described subjects' behavior. The initial conditions chosen by the subjects in the experiment matched those that produced the most predictable interactions in simulation. These results present first support for the hypothesis that humans prepare the object to minimize transients and increase stability and, overall, the predictability of hand-object interactions.

Synchronization-based control for a collaborative robot

This article introduces a new control scheme for controlling a robotic manipulator in a collaborative task, allowing it to respond proactively to its partner's movements. Unlike conventional robotic systems, humans can operate in an unstructured, dynamic environment due to their ability to anticipate changes before they occur and react accordingly. Recreating this artificially by using a forward model would lead to the huge computational task of simulating a world full of complex nonlinear dynamics and autonomous human agents. In this study, a controller based on anticipating synchronization, where a 'leader' dynamical system is predicted by a coupled 'follower' with delayed self-feedback, is used to modify a robot's dynamical behaviour to follow that of a series of leaky integrators and harmonic oscillators. This allows the robot (follower) to be coupled with a collaborative partner (leader) to anticipate its movements, without a complete model of its behaviour. This is tested by tasking a simulated Baxter robot with performing a collaborative manual coordination task with an autonomous partner under a range of feedback delay conditions, confirming its ability to anticipate using oscillators instead of a detailed forward model.

Analysis of motor control strategy for frontal and sagittal planes of circular tracking movements using visual feedback noise from velocity change and depth information

We aim to investigate a control strategy for the circular tracking movement in a three-dimensional (3D) space based on the accuracy of the visual information. After setting the circular orbits for the frontal and sagittal planes in the 3D virtual space, the subjects track a target moving at a constant velocity. The analysis is applied to two parameters of the polar coordinates, namely, ΔR (the difference in the distance from the center of a circular orbit) and Δω (the difference in the angular velocity). The movement in the sagittal plane provides different depth information depending on the position of the target in orbit, unlike the task of the frontal plane. Therefore, the circular orbit is divided into four quadrants for a statistical analysis of ΔR. In the sagittal plane, the error was two to three times larger in quadrants 1 and 4 than in quadrants 2 and 3 close to the subject. Here, Δω is estimated using a frequency analysis; the lower the accuracy of the visual information, the greater the periodicity. When comparing two different planes, the periodicity in the sagittal plane was approximately 1.7 to 2 times larger than that of the frontal plane. In addition, the average angular velocity of the target and tracer was within 0.6% during a single cycle. We found that if the amount of visual information is reduced, an optimal feedback control strategy can be used to reduce the positional error within a specific area.

Analysis of Visuo Motor Control between Dominant Hand and Non-Dominant Hand for Effective Human-Robot Collaboration

The human-in-the-loop technology requires studies on sensory-motor characteristics of each hand for an effective human-robot collaboration. This study aims to investigate the differences in visuomotor control between the dominant (DH) and non-dominant hands in tracking a target in the three-dimensional space. We compared the circular tracking performances of the hands on the frontal plane of the virtual reality space in terms of radial position error (ΔR), phase error (Δθ), acceleration error (Δa), and dimensionless squared jerk (DSJ) at four different speeds for 30 subjects. ΔR and Δθ significantly differed at relatively high speeds (ΔR: 0.5 Hz; Δθ: 0.5, 0.75 Hz), with maximum values of ≤1% compared to the target trajectory radius. DSJ significantly differed only at low speeds (0.125, 0.25 Hz), whereas Δa significantly differed at all speeds. In summary, the feedback-control mechanism of the DH has a wider range of speed control capability and is efficient according to an energy saving model. The central nervous system (CNS) uses different models for the two hands, which react dissimilarly. Despite the precise control of the DH, both hands exhibited dependences on limb kinematic properties at high speeds (0.75 Hz). Thus, the CNS uses a different strategy according to the model for optimal results.

Analysis of Control Characteristics between Dominant and Non-Dominant Hands by Transient Responses of Circular Tracking Movements in 3D Virtual Reality Space

Human movement is a controlled result of the sensory-motor system, and the motor control mechanism has been studied through diverse movements. The present study examined control characteristics of dominant and non-dominant hands by analyzing the transient responses of circular tracking movements in 3D virtual reality space. A visual target rotated in a circular trajectory at four different speeds, and 29 participants tracked the target with their hands. The position of each subject’s hand was measured, and the following three parameters were investigated: normalized initial peak velocity (IPV2), initial peak time (IPT2), and time delay (TD2). The IPV2 of both hands decreased as target speed increased. The results of IPT2 revealed that the dominant hand reached its peak velocity 0.0423 s earlier than the non-dominant hand, regardless of target speed. The TD2 of the hands diminished by 0.0218 s on average as target speed increased, but the dominant hand statistically revealed a 0.0417-s shorter TD2 than the non-dominant hand. Velocity-control performances from the IPV2 and IPT2 suggested that an identical internal model controls movement in both hands, whereas the dominant hand is likely more experienced than the non-dominant hand in reacting to neural commands, resulting in better reactivity in the movement task.

EEG dynamical network analysis method reveals the neural signature of visual-motor coordination

Human visual-motor coordination is an essential function of movement control, which requires interactions of multiple brain regions. Understanding the cortical-motor coordination is important for improving physical therapy for motor disabilities. However, its underlying transient neural dynamics is still largely unknown. In this study, we applied an eigenvector-based dynamical network analysis method to investigate the functional connectivity calculated from electroencephalography (EEG) signals under visual-motor coordination task and to identify its meta-stable states dynamics. We first tested this signal processing on a simulated network to evaluate it in comparison with other dynamical methods, demonstrating that the eigenvector-based dynamical network analysis was able to correctly extract the dynamical features of the evolving networks. Subsequently, the eigenvector-based analysis was applied to EEG data collected under a visual-motor coordination experiment. In the EEG study with participants, the results of both topological analysis and the eigenvector-based dynamical analysis were able to distinguish different experimental conditions of visual tracking task. With the dynamical analysis, we showed that different visual-motor coordination states can be distinguished by investigating the meta-stable states dynamics of the functional connectivity.

A systematic evaluation of the evidence for perceptual control theory in tracking studies

Perceptual control theory (PCT) proposes that perceptual inputs are controlled to intentional 'reference' states by hierarchical negative feedback control, evidence for which comes from manual tracking experiments in humans. We reviewed these experiments to determine whether tracking is a process of perceptual control, and to assess the state-of-the-evidence for PCT. A systematic literature search was conducted of peer-review journal and book chapters in which tracking data were simulated with a PCT model (13 studies, 53 participants). We report a narrative review of these studies and a qualitative assessment of their methodological quality. We found evidence that individuals track to individual-specific endogenously-specified reference states and act against disturbances, and evidence that hierarchical PCT can simulate complex tracking. PCT's learning algorithm, reorganization, was not modelled. Limitations exist in the range of tracking conditions under which the PCT model has been tested. Future PCT research should apply the PCT methodology to identify control variables in real-world tasks and develop hierarchical PCT architectures for goal-oriented robotics to test the plausibility of PCT model-based action control.

Adaptive intermittent control: A computational model explaining motor intermittency observed in human behavior

It is a fundamental question how our brain performs a given motor task in a real-time fashion with the slow sensorimotor system. Computational theory proposed an influential idea of feed-forward control, but it has mainly treated the case that the movement is ballistic (such as reaching) because the motor commands should be calculated in advance of movement execution. As a possible mechanism for operating feed-forward control in continuous motor tasks (such as target tracking), we propose a control model called "adaptive intermittent control" or "segmented control," that brain adaptively divides the continuous time axis into discrete segments and executes feed-forward control in each segment. The idea of intermittent control has been proposed in the fields of control theory, biological modeling and nonlinear dynamical system. Compared with these previous models, the key of the proposed model is that the system speculatively determines the segmentation based on the future prediction and its uncertainty. The result of computer simulation showed that the proposed model realized faithful visuo-manual tracking with realistic sensorimotor delays and with less computational costs (i.e., with fewer number of segments). Furthermore, it replicated "motor intermittency", that is, intermittent discontinuities commonly observed in human movement trajectories. We discuss that the temporally segmented control is an inevitable strategy for brain which has to achieve a given task with small computational (or cognitive) cost, using a slow control system in an uncertain variable environment, and the motor intermittency is the side-effect of this strategy.

Periodic change in phase relationship between target and hand motion during visuo-manual tracking task: behavioral evidence for intermittent control

When one performs visuo-manual tracking tasks, velocity profile of hand movements shows discontinuous patterns even if the target moves smoothly. A crucial factor of this "intermittency" is considerable delay in the sensorimotor feedback loop, and several researchers have suggested that the cause is intermittent correction of motor commands. However, when and how the brain monitors task performance and updates motor commands in a continuous motor task is uncertain. We examined how tracking error was affected by the timing of target disappearance during a tracking task. Results showed that tracking error, defined as the average phase difference between target and hand, varied periodically in all conditions. Hand preceded target at one specific phase but followed it at another, implying that motor control was not performed in a temporally uniform manner. Tracking stability was evaluated by the variance in phase difference, and changed depending on the timing of target-removal. The variability was larger when target disappeared around turning points than that when it disappeared around the center of motion. This shows that visual information at turning points is more effectively exploited for motor control of sinusoidal target tracking, suggesting that our brain controls hand movements with intermittent reference to visual information.

Mechanism for synchronized motion between two humans in mutual tapping experiments: transition from alternative mode to synchronization mode

We performed mutual tapping experiments between two humans to investigate the conditions required for synchronized motion. A transition from an alternative mode to a synchronization mode was discovered under the same conditions as when a subject changed from a reactive mode to an anticipation mode in single tapping experiments. The experimental results suggest that the cycle time for each tapping motion is tuned by a proportional control that is based on synchronization errors and cycle time errors. As the tapping frequency increases, a mathematical model based on feedback control in the sensory-motor closed loop predicts a discrete transition of the mode as the gain factors of the proportional control decrease. The conditions for synchronization are shown as a consequence of the coupled dynamics based on the next feedback loop in the sensory-motor system.

Transition from an antiphase error-correction mode to a synchronization mode in mutual hand tracking

Proactive motion in hand tracking and in finger bending, in which the body motion occurs prior to the reference signal, has been previously reported when a periodic target signal was shown to the subjects at relatively high frequencies. These phenomena indicate that the human sensory-motor system tends to choose an anticipatory mode rather than a reactive mode, when the target motion is relatively fast. The present research was undertaken to study what kind of mode appears in the sensory-motor system when two persons were asked to track the hand position of the partner at various mean tracking frequency. The experimental results showed that a transition from a mutual error-correction mode to a synchronization mode occurred in the same region of the transition frequency with the one from a reactive error-correction mode to a proactive anticipatory mode reported previously in the target tracking experiments of the single subjects. Present research indicated that synchronization of body motion occurred only when both of the pair subjects operated in a proactive anticipatory mode. We also presented mathematical models to explain the behavior of the error-correction mode and the synchronization mode.

The Aspects, the Origin, and the Merit of Aware Computing

In this paper we tried to understand scientifically the awareness, a daily word. Some aspects of awareness, such as qualitative or quantitative, the targets of awareness, either the external world or the internal world, were discussed. Suggestion on the human awareness was described from the experimental results of visual hand tracking. The origin and the merit of awareness in the process of evolution of animals were discussed. Finally some characters of possible aware computers and aware robots were studied.

Semianalytical transient solution of a delayed differential equation and its application to the tracking motion in the sensory-motor system

We derived semianalytically the transient solution of a delayed differential equation that had been shown to be a simple but good model of the sensory-motor system. In the present Brief Report, we applied this transient solution for studying the global nature of the transient tracking motion when visual target information is changed suddenly. The results clarified that the dynamic error minimization principle in hand motion observed experimentally is robust over a wide range of the parameter space of the delay time, the time constant, and the feedforward parameter.

Binding of movement, sound and touch: multimodal coordination dynamics

Very little is known about the coordination of movement in combination with stimuli such as sound and touch. The present research investigates the hypothesis that both the type of action (e.g., a flexion or extension movement) and the sensory modality (e.g., auditory or tactile) determine the stability of multimodal coordination. We performed a parametric study in which the ability to synchronize movement, touch and sound was explored over a broad range of stimulus frequencies or rates. As expected, synchronization of finger movement with external auditory and tactile stimuli was successfully established and maintained across all frequencies. In the key experimental conditions, participants were instructed to synchronize peak flexion of the index finger with touch and peak extension with sound (and vice-versa). In this situation, tactile and auditory stimuli were delivered counter-phase to each other. Two key effects were observed. First, switching between multimodal coordination patterns occurred, with transitions selecting one multimodal pattern (flexion with sound and extension with touch) more often than its partner. This finding indicates that the stability of multimodal coordination is influenced by both the type of action and the stimulus modality. Second, at higher rates, transitions from coherent to incoherent phase relations between touch, movement and sound occurred, attesting to the breakdown of multimodal coordination. Because timing errors in multimodal coordination were systematically altered when compared to unimodal control conditions we are led to consider the role played by time delays in multimodal coordination dynamics.

Theoretical analysis of destabilization resonances in time-delayed stochastic second-order dynamical systems and some implications for human motor control

A linear stochastic delay differential equation of second order is studied that can be regarded as a Kramers model with time delay. An analytical expression for the stationary probability density is derived in terms of a Gaussian distribution. In particular, the variance as a function of the time delay is computed analytically for several parameter regimes. Strikingly, in the parameter regime close to the parameter regime in which the deterministic system exhibits Hopf bifurcations, we find that the variance as a function of the time delay exhibits a sequence of pronounced peaks. These peaks are interpreted as delay-induced destabilization resonances arising from oscillatory ghost instabilities. On the basis of the obtained theoretical findings, reinterpretations of previous human motor control studies and predictions for future human motor control studies are provided.

Delay Fokker-Planck equations, Novikov's theorem, and Boltzmann distributions as small delay approximations

We study time-delayed stochastic systems that can be described by means of so-called delay Fokker-Planck equations. Using Novikov's theorem, we first show that the theory of delay Fokker-Planck equations is on an equal footing with the theory of ordinary Fokker-Planck equations. Subsequently, we derive stationary distributions in the case of small time delays. In the case of additive noise systems, these distributions can be cast into the form of Boltzmann distributions involving effective potential functions.