Predictability and Robustness in the Manipulation of Dynamically Complex Objects

The information variables for controlling manual transfer of liquid-filled containers

It is a familiar but challenging task to manually transfer a liquid-filled container without spilling. The action requires stringent control because the dynamics of interacting with the non-rigid aqueous content is complex. In this work, we sought to discover what properties of a liquid-filled container were predictive of transfer without spilling performance. Two candidate variables were tested (Experiment 1): The distance between liquid surface and the container's rim (h) and the container's diameter (d). Participants attempted to transfer 15 containers (3 ds and 5 hs), one at a time and as fast as possible, without spilling. Kinematic analyses showed that the movement's peak velocity and the first peak acceleration were affected by h; the movement time and the frequency of acceleration change were affected by h and d in a hierarchical manner, where transfer without spilling was first affected by h and for full containers, the thick ones were moved more slowly and went through more acceleration change; for not so full containers, the container's diameter did not have any effect. Next, each of the 15 containers was compared with the other 14, and participants judged from a pair of displayed containers which one was more likely to be moved fast without spilling (Experiment 2). Perceived affordance was affected by h and d but not by whether containers were placed upright or tilted. In general, thinner and less full containers were judged as easier to be moved fast without spilling.

Next-Generation Personalized Medicine: Implementation of Variability Patterns for Overcoming Drug Resistance in Chronic Diseases

Chronic diseases are a significant healthcare problem. Partial or complete non-responsiveness to chronic therapies is a significant obstacle to maintaining the long-term effect of drugs in these patients. A high degree of intra- and inter-patient variability defines pharmacodynamics, drug metabolism, and medication response. This variability is associated with partial or complete loss of drug effectiveness. Regular drug dosing schedules do not comply with physiological variability and contribute to resistance to chronic therapies. In this review, we describe a three-phase platform for overcoming drug resistance: introducing irregularity for improving drug response; establishing a deep learning, closed-loop algorithm for generating a personalized pattern of irregularity for overcoming drug resistance; and upscaling the algorithm by implementing quantified personal variability patterns along with other individualized genetic and proteomic-based ways. The closed-loop, dynamic, subject-tailored variability-based machinery can improve the efficacy of existing therapies in patients with chronic diseases.

Human control of complex objects: Towards more dexterous robots

Manipulation of objects with underactuated dynamics remains a challenge for robots. In contrast, humans excel at 'tool use' and more insight into human control strategies may inform robotic control architectures. We examined human control of objects that exhibit complex - underactuated, nonlinear, and potentially chaotic dynamics, such as transporting a cup of coffee. Simple control strategies appropriate for unconstrained movements, such as maximizing smoothness, fail as interaction forces have to be compensated or preempted. However, predictive control based on internal models appears daunting when the objects have nonlinear and unpredictable dynamics. We hypothesized that humans learn strategies that make these interactions predictable. Using a virtual environment subjects interacted with a virtual cup and rolling ball using a robotic visual and haptic interface. Two different metrics quantified predictability: stability or contraction, and mutual information between controller and object. In point-to-point displacements subjects exploited the contracting regions of the object dynamics to safely navigate perturbations. Control contraction metrics showed that subjects used a controller that exponentially stabilized trajectories. During continuous cup-and-ball displacements subjects developed predictable solutions sacrificing smoothness and energy efficiency. These results may stimulate control strategies for dexterous robotic manipulators and human-robot interaction.

Separating neural influences from peripheral mechanics: the speed-curvature relation in mechanically constrained actions

While the study of unconstrained movements has revealed important features of neural control, generalizing those insights to more sophisticated object manipulation is challenging. Humans excel at physical interaction with objects, even when those objects introduce complex dynamics and kinematic constraints. This study examined humans turning a horizontal planar crank (radius 10.29 cm) at their preferred and three instructed speeds (with visual feedback), both in clockwise and counterclockwise directions. To explore the role of neuromechanical dynamics, the instructed speeds covered a wide range: fast (near the limits of performance), medium (near preferred speed), and very slow (rendering dynamic effects negligible). Because kinematically constrained movements involve significant physical interaction, disentangling neural control from the influences of biomechanics presents a challenge. To address it, we modeled the interactive dynamics to "subtract off" peripheral biomechanics from observed force and kinematic data, thereby estimating aspects of underlying neural action that may be expressed in terms of motion. We demonstrate the value of this method: remarkably, an approximately elliptical path emerged, and speed minima coincided with curvature maxima, similar to what is seen in unconstrained movements, even though the hand moved at nearly constant speed along a constant-curvature path. These findings suggest that the neural controller takes advantage of peripheral biomechanics to simplify physical interaction. As a result, patterns seen in unconstrained movements persist even when physical interaction prevents their expression in hand kinematics. The reemergence of a speed-curvature relation indicates that it is due, at least in part, to neural processes that emphasize smoothness and predictability.NEW & NOTEWORTHY Physically interacting with kinematic constraints is commonplace in everyday actions. We report a study of humans turning a crank, a circular constraint that imposes constant hand path curvature and hence should suppress variations of hand speed due to the power-law speed-curvature relation widely reported for unconstrained motions. Remarkably, we found that, when peripheral biomechanical factors are removed, a speed-curvature relation reemerges, indicating that it is, at least in part, of neural origin.

Scaling Analysis of Phase Fluctuations of Brain Networks in Dynamic Constrained Object Manipulation

In this study, we investigated the dynamic properties of oscillatory activities in the scalp electro-encephalographs (EEGs) of 20 participants involved in a novel dynamic manipulating task using a physical interface and a virtual feedback. The complexity of such a task a rises from the unexpected relationship between the magnitude of the motion and the feedback. The characterization of complex patterns arising from EEG is an important problem in identifying different mental intentions. We proposed a scaling analysis of phase fluctuation in the scalp EEG to discriminate the network states related to different EEG patterns, which correspond to manipulating the task with right or left movement intention. These intentions are generated while the participant is engaged in such a complex task. The phase characterization method was used to calculate the instantaneous phase from the operational EEG. Then, functional brain networks (FBNs) of 20 subjects based on the task-related EEG were constructed by phase synchronization. The degree features representing the structures and scaling components of brain networks are sensitive to the EEG patterns with left or right motor intention. The correlation between features and mental intentions was investigated by discriminant analysis. For 20 subjects, the average accuracy of state detection is [Formula: see text], and the average mean-squared error (MSE) is [Formula: see text]. The brain state depicted by the results is related to high awareness, the phase characterization is of the effectiveness in EEG processing and FBN construction and the difference of control intentions can be explored by the phase characterization method. This finding may be relevant to understanding some neuronal mechanisms underlying the attention and some applications of closed-loop control for the safety operation of tools.

Portable Motion-Analysis Device for Upper-Limb Research, Assessment, and Rehabilitation in Non-Laboratory Settings

This study presents the design and feasibility testing of an interactive portable motion-analysis device for the assessment of upper-limb motor functions in clinical and home settings. The device engages subjects to perform tasks that imitate activities of daily living, e.g. drinking from a cup and moving other complex objects. Sitting at a magnetic table subjects hold a 3D printed cup with an adjustable magnet and move this cup on the table to targets that can be drawn on the table surface. A ball rolling inside the cup can enhance the task challenge by introducing additional dynamics. A single video camera with a portable computer tracks real-time kinematics of the cup and the rolling ball using a custom-developed, color-based computer-vision algorithm. Preliminary verification with marker-based 3D-motion capture demonstrated that the device produces accurate kinematic measurements. Based on the real-time 2D cup coordinates, audio-visual feedback about performance can be delivered to increase motivation. The feasibility of using this device in clinical diagnostics is demonstrated on 2 neurotypical children and also 3 children with upper-extremity impairments in the hospital, where conventional motion-analysis systems are difficult to use. The device meets key needs for clinical practice: 1) a portable solution for quantitative motor assessment for upper-limb movement disorders at non-laboratory clinical settings, 2) a low-cost rehabilitation device that can increase the volume of in-home physical therapy, and 3) the device affords testing and training a variety of motor tasks inspired by daily challenges to enhance self-confidence to participate in day-to-day activities.

Predictability, force, and (anti)resonance in complex object control

Manipulation of complex objects as in tool use is ubiquitous and has given humans an evolutionary advantage. This study examined the strategies humans choose when manipulating an object with underactuated internal dynamics, such as a cup of coffee. The dynamics of the object renders the temporal evolution complex, possibly even chaotic, and difficult to predict. A cart-and-pendulum model, loosely mimicking coffee sloshing in a cup, was implemented in a virtual environment with a haptic interface. Participants rhythmically manipulated the virtual cup containing a rolling ball; they could choose the oscillation frequency, whereas the amplitude was prescribed. Three hypotheses were tested: 1) humans decrease interaction forces between hand and object; 2) humans increase the predictability of the object dynamics; and 3) humans exploit the resonances of the coupled object-hand system. Analysis revealed that humans chose either a high-frequency strategy with antiphase cup-and-ball movements or a low-frequency strategy with in-phase cup-and-ball movements. Counter to hypothesis 1, they did not decrease interaction force; instead, they increased the predictability of the interaction dynamics, quantified by mutual information, supporting hypothesis 2. To address hypothesis 3, frequency analysis of the coupled hand-object system revealed two resonance frequencies separated by an antiresonance frequency. The low-frequency strategy exploited one resonance, whereas the high-frequency strategy afforded more choice, consistent with the frequency response of the coupled system; both strategies avoided the antiresonance. Hence, humans did not prioritize small interaction forces but rather strategies that rendered interactions predictable. These findings highlight that physical interactions with complex objects pose control challenges not present in unconstrained movements. NEW & NOTEWORTHY Daily actions involve manipulation of complex nonrigid objects, which present a challenge since humans have no direct control of the whole object. We used a virtual-reality experiment and simulations of a cart-and-pendulum system coupled to hand movements with impedance to analyze the manipulation of this underactuated object. We showed that participants developed strategies that increased the predictability of the object behavior by exploiting the resonance structure of the object but did not minimize the hand-object interaction force.