Feedback Type May Change the EMG Pattern and Kinematics During Robot Supported Upper Limb Reaching Task

Haptic interfaces and virtual reality (VR) technology have been increasingly introduced in rehabilitation, facilitating the provision of various feedback and task conditions. However, correspondence between the feedback/task conditions and movement strategy during reaching tasks remains a question. To investigate movement strategy, we assessed velocity parameters and peak latency of electromyography. Ten neuromuscularly intact volunteers participated in the measurement using haptic interface and VR. Concurrent visual feedback and various terminal feedback (e.g., visual, haptic, visual and haptic) were given. Additionally, the object size for the reaching task was changed. The results demonstrated terminal haptic feedback had a significant impact on kinematic parameters; showed [Formula: see text] s ([Formula: see text]) shorter movement time and [Formula: see text] m/s ([Formula: see text]) higher mean velocity compared to no terminal feedback. Also, smaller peak latency was observed in different muscle regions based on the object size.

Collision avoidance analysis of human-robot physical interaction based on null-space impedance control of a dynamic reference arm plane

When the terminal upper limb rehabilitation robot is used for motion-assisted training, collisions between the manipulator links and the human upper limb may occur due to the null-space self-motion of the redundant manipulator. A null-space impedance control method based on a dynamic reference arm plane is proposed to realize collision avoidance during human-robot physical interaction motion for the collision problem between the manipulator links and the human upper limb. Firstly, a dynamic model and a Cartesian impedance controller of the manipulator are established. Then, the null-space impedance controller of the redundant manipulator is established based on the dynamic reference plane, which manages the null-space self-motion of the redundant manipulator to prevent collision between the manipulator links and the human upper limb. Finally, it is experimentally verified that the method proposed in this paper can effectively manage the null-space self-motion of the redundant manipulator, and thus achieve collision avoidance during the human-robot physical interaction motion. This research has significant potential in improving the safety and feasibility of motion-assisted training with rehabilitation robots.

Virtual Stiffness: A Novel Biomechanical Approach to Estimate Limb Stiffness of a Multi-Muscle and Multi-Joint System

In recent years, different groups have developed algorithms to control the stiffness of a robotic device through the electromyographic activity collected from a human operator. However, the approaches proposed so far require an initial calibration, have a complex subject-specific muscle model, or consider the activity of only a few pairs of antagonist muscles. This study described and tested an approach based on a biomechanical model to estimate the limb stiffness of a multi-joint, multi-muscle system from muscle activations. The "virtual stiffness" method approximates the generated stiffness as the stiffness due to the component of the muscle-activation vector that does not generate any endpoint force. Such a component is calculated by projecting the vector of muscle activations, estimated from the electromyographic signals, onto the null space of the linear mapping of muscle activations onto the endpoint force. The proposed method was tested by using an upper-limb model made of two joints and six Hill-type muscles and data collected during an isometric force-generation task performed with the upper limb. The null-space projection of the muscle-activation vector approximated the major axis of the stiffness ellipse or ellipsoid. The model provides a good approximation of the voluntary stiffening performed by participants that could be directly implemented in wearable myoelectric controlled devices that estimate, in real-time, the endpoint forces, or endpoint movement, from the mapping between muscle activation and force, without any additional calibrations.

Research on Hybrid Force Control of Redundant Manipulator with Reverse Task Priority

This paper presents the reverse priority impedance control of manipulators with reference to redundant robots of a given task. The reverse priority kinematic control of redundant manipulators is first expressed in detail. The motion in the joint space is derived following the opposite order compared with the classical task priority-based solution. Then the Cartesian impedance control is combined with the reverse priority impedance control to solve the reverse hierarchical impedance controlled, so that the Cartesian impedance behavior can be divided into the primary priority impedance control and the secondary priority impedance control. Furthermore, the secondary impedance control task will not disturb the primary impedance control task. The motion in the joint space is affected following the opposite order and working in the corresponding projection operators. The primary impedance control tasks are implemented at the end, so as to avoid the possible deformations caused by the singularities occurring in the secondary impedance control tasks. Hence, the proposed reverse priority impedance control of manipulator can achieve the desired impedance control tasks with proper hierarchy. In this paper, the simulation experiments of the manipulator will verify the proposed reverse priority control algorithm.

Dynamic Balancing of Humanoid Robot with Proprioceptive Actuation: Systematic Design of Algorithm, Software, and Hardware

For humanoid robots, maintaining a dynamic balance against uncertain disturbance is crucial, and this function can be achieved by coordinating the whole body to perform multiple tasks simultaneously. Researchers generally accept hierarchical whole-body control (WBC) to address this function. Although experts can build feasible hierarchies using prior knowledge, real-time WBC is still challenging because it often requires a quadratic program with multiple inequality constraints. In addition, the torque tracking performance of the WBC algorithm will be affected by uncertain factors such as joint friction for a large transmission ratio proprioceptive-actuated robot. Therefore, the balance control of physical robots requires a systematic solution. In this study, a robot control system with high computing power and real-time communication ability, UBTMaster, is implemented to achieve a reduced WBC in real time. Based on these, a whole-body control scheme based on task priority for the dynamic balance of humanoid robots is implemented. After realizing the joint friction model identification, finally, a variety of balancing scenarios are tested on the Walker3 humanoid robot driven by the proprioceptive actuators to verify the effectiveness of the proposed scheme. The Walker3 robot exhibits excellent balance when multiple external disturbances occur simultaneously. For example, the two feet of the robot are subjected to tilt and displacement perturbations, respectively, while the torso is subjected to external shocks simultaneously. The experimental results show that the dynamic balance of the robot under multiple external disturbances can be achieved by using strictly hierarchical real-time WBC with a systematic design.

Exploring planet geology through force-feedback telemanipulation from orbit

Current space exploration roadmaps envision exploring the surface geology of celestial bodies with robots for both scientific research and in situ resource utilization. In such unstructured, poorly lit, complex, and remote environments, automation is not always possible, and some tasks, such as geological sampling, require direct teleoperation aided by force-feedback (FF). The operator would be on an orbiting spacecraft, and poor bandwidth, high latency, and packet loss from orbit to ground mean that safe, stable, and transparent interaction is a substantial technical challenge. For this scenario, a control method was developed that ensures stability at high delay without reduction in speed or loss of positioning accuracy. At the same time, a new level of safety is achieved not only through FF itself but also through an intrinsic property of the approach preventing hard impacts. On the basis of this method, a tele-exploration scenario was simulated in the Analog-1 experiment with an astronaut on the International Space Station (ISS) using a 6-degree-of-freedom (DoF) FF capable haptic input device to control a mobile robot with manipulator on Earth to collect rock samples. The 6-DoF FF telemanipulation from space was performed at a round-trip communication delay constantly between 770 and 850 milliseconds and an average packet loss of 1.27%. This experiment showcases the feasibility of a complete space exploration scenario via haptic telemanipulation under spaceflight conditions. The results underline the benefits of this control method for safe and accurate interactions and of haptic feedback in general.

Exodex Adam—A Reconfigurable Dexterous Haptic User Interface for the Whole Hand

Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments.

Data-driven artificial and spiking neural networks for inverse kinematics in neurorobotics

Inverse kinematics is fundamental for computational motion planning. It is used to derive an appropriate state in a robot's configuration space, given a target position in task space. In this work, we investigate the performance of fully connected and residual artificial neural networks as well as recurrent, learning-based, and deep spiking neural networks for conventional and geometrically constrained inverse kinematics. We show that while highly parameterized data-driven neural networks with tens to hundreds of thousands of parameters exhibit sub-ms inference time and sub-mm accuracy, learning-based spiking architectures can provide reasonably good results with merely a few thousand neurons. Moreover, we show that spiking neural networks can perform well in geometrically constrained task space, even when configured to an energy-conserved spiking rate, demonstrating their robustness. Neural networks were evaluated on NVIDIA's Xavier and Intel's neuromorphic Loihi chip.

Enabling impedance-based physical human–multi–robot collaboration: Experiments with four torque-controlled manipulators

Robotics research into multi-robot systems so far has concentrated on implementing intelligent swarm behavior and contact-less human interaction. Studies of haptic or physical human-robot interaction, by contrast, have primarily focused on the assistance offered by a single robot. Consequently, our understanding of the physical interaction and the implicit communication through contact forces between a human and a team of multiple collaborative robots is limited. We here introduce the term Physical Human Multi-Robot Collaboration (PHMRC) to describe this more complex situation, which we consider highly relevant in future service robotics. The scenario discussed in this article covers multiple manipulators in close proximity and coupled through physical contacts. We represent this set of robots as fingers of an up-scaled agile robot hand. This perspective enables us to employ model-based grasping theory to deal with multi-contact situations. Our torque-control approach integrates dexterous multi-manipulator grasping skills, optimization of contact forces, compensation of object dynamics, and advanced impedance regulation into a coherent compliant control scheme. For this to achieve, we contribute fundamental theoretical improvements. Finally, experiments with up to four collaborative KUKA LWR IV+ manipulators performed both in simulation and real world validate the model-based control approach. As a side effect, we notice that our multi-manipulator control framework applies identically to multi-legged systems, and we execute it also on the quadruped ANYmal subject to non-coplanar contacts and human interaction.

Energy budgets for coordinate invariant robot control in physical human–robot interaction

In this work we consider the current certification process of applications with physical human–robot interaction (pHRI). Two major hazards are collisions and clamping scenarios. The implementation of safety measures in pHRI applications typically depends strongly on coordinates, e.g., to monitor the robot velocity or to predict external forces. We show that the current certification process does not, in general, guarantee a safe robot behavior. In particular, in unstructured environments it is not possible to predict all risks in advance. We therefore propose to control the energy of the robot, which is a coordinate invariant entity. For an impedance controlled robot, the total energy consists of potential energy and kinetic energy. The energy flow from task description to physical interaction follows a strict causality. We assign a safe energy budget for the robot. With this energy budget, the presented controller auto-tunes its parameters to limit the exchanged kinetic energy during a collision and the potential energy during clamping scenarios. In contact, the robot behaves compliantly and therefore eliminates clamping danger. After contact, the robot automatically continues to follow the desired trajectory. With this approach the number of safety-related parameters to be determined can be reduced to one energy value, which has the potential to significantly speed up the commissioning of pHRI applications. The proposed technique is validated by experiments.

Whole-Body Teleoperation and Shared Control of Redundant Robots with Applications to Aerial Manipulation

This paper introduces a passivity-based control framework for multi-task time-delayed bilateral teleoperation and shared control of kinematically-redundant robots. The proposed method can be seen as extension of state-of-the art hierarchical whole-body control as it allows for some of the tasks to be commanded by a remotely-located human operator through a haptic device while the others are autonomously performed. The operator is able to switch among tasks at any time without compromising the stability of the system. To enforce the passivity of the communication channel as well as to dissipate the energy generated by the null-space projectors used to enforce the hierarchy among the tasks, the Time-Domain Passivity Approach (TDPA) is applied. The efficacy of the approach is demonstrated through its application to the DLR Suspended Aerial Manipulator (SAM) in a real telemanipulation scenario with variable time delay, jitter, and package loss.

Task Feasibility Maximization Using Model-Free Policy Search and Model-Based Whole-Body Control

Producing feasible motions for highly redundant robots, such as humanoids, is a complicated and high-dimensional problem. Model-based whole-body control of such robots can generate complex dynamic behaviors through the simultaneous execution of multiple tasks. Unfortunately, tasks are generally planned without close consideration for the underlying controller being used, or the other tasks being executed, and are often infeasible when executed on the robot. Consequently, there is no guarantee that the motion will be accomplished. In this work, we develop a proof-of-concept optimization loop which automatically improves task feasibility using model-free policy search in conjunction with model-based whole-body control. This combination allows problems to be solved, which would be otherwise intractable using simply one or the other. Through experiments on both the simulated and real iCub humanoid robot, we show that by optimizing task feasibility, initially infeasible complex dynamic motions can be realized—specifically, a sit-to-stand transition. These experiments can be viewed in the accompanying Video S1.

A Multi-Priority Controller for Industrial Macro-Micro Manipulation

In this study, a control algorithm is proposed and evaluated for a special type of kinematically redundant manipulator. This manipulator is comprised of two mechanisms, macro and micro mechanisms, with distinct acceleration and work space characteristics. A control algorithm is devised to minimize the task completion duration and the overall actuator effort with respect to the conventional manipulator. A general framework multi-priority controller for macro-micro manipulators is introduced by utilizing virtual dynamics, which is introduced in null-space projection to achieve secondary tasks. The proposed controller is evaluated on a simulation model based on a previously constructed macro-micro manipulator for planar laser cutting. Task completion duration and the total actuator effort are investigated and the results are compared.

Decoupled nonlinear adaptive control of position and stiffness for pneumatic soft robots

This article addresses the problem of simultaneous and robust closed-loop control of joint stiffness and position, for a class of antagonistically actuated pneumatic soft robots with rigid links and compliant joints. By introducing a first-order dynamic equation for the stiffness variable and using the additional control degree of freedom, embedded in the null space of the pneumatic actuator matrix, an innovative control approach is introduced comprising an adaptive compensator and a dynamic decoupler. The proposed solution builds upon existing adaptive control theory and provides a technique for closing the loop on joint stiffness in pneumatic variable stiffness actuators. Under a very mild assumption involving the inertia and actuator matrices, the solution is able to cope with uncertainties of the model and, when the desired stiffness is constant or slowly varying, also of the pneumatic actuator. Position and stiffness decoupling is achieved by the introduction of a first-order differential equation for an internal state variable of the controller, which takes into account the time derivative of pressure in the stiffness dynamics. A formal proof of the stability of the position and stiffness tracking errors is provided. An appealing property of the approach is that it does not require higher derivatives of position or any derivatives of stiffness. The solution is validated with respect to several use-cases, first in simulation and then via a real pneumatic soft robot with McKibben muscles. A comparison with respect to existing techniques reveals a more robust position and stiffness tracking skill.

Exploiting null space potentials to control arm robots compliantly performing nonlinear tactile tasks

In this article, two new compliant control architectures are introduced that utilize null space solutions to decouple force and position control. They are capable to interact with uncertain surfaces and environments with varying materials and require fewer parameters to be tuned than the common architectures – hybrid or impedance control. The general concept behind these approaches allows to consider manipulators with six degrees of freedom as redundant by creating a virtual redundancy with a reduced work space. It will be demonstrated that the introduced approaches are superior regarding orthogonal separation of the Cartesian degrees of freedom and avoid inner singularities. To demonstrate their performance, the controllers are tested on a standard industrial robot (Stäubli, RX90B, six degrees of freedom) that actuates two different biomechanically inspired models of the human knee joint.

A singularity handling algorithm based on operational space control for six-degree-of-freedom anthropomorphic manipulators

In this article, we analyze the singularities of six-degree-of-freedom anthropomorphic manipulators and design a singularity handling algorithm that can smoothly go through singular regions. We show that the boundary singularity and the internal singularity points of six-degree-of-freedom anthropomorphic manipulators can be identified through a singularity analysis, although they do not possess the nice kinematic decoupling property as six-degree-of-freedom industrial manipulators. Based on this discovery, our algorithm adopts a switching strategy to handle these two cases. For boundary singularities, the algorithm modifies the control input to fold the manipulator back from the singular straight posture. For internal singularities, the algorithm controls the manipulator with null space motion. We show that this strategy allows a manipulator to move within singular regions and back to non-singular regions, so the usable workspace is increased compared with conventional approaches. The proposed algorithm is validated in simulations and real-time control experiments.

Scaling Our World View: How Monoamines Can Put Context Into Brain Circuitry

Monoamines are presumed to be diffuse metabotropic neuromodulators of the topographically and temporally precise ionotropic circuitry which dominates CNS functions. Their malfunction is strongly implicated in motor and cognitive disorders, but their function in behavioral and cognitive processing is scarcely understood. In this paper, the principles of such a monoaminergic function are conceptualized for locomotor control. We find that the serotonergic system in the ventral spinal cord scales ionotropic signals and shows topographic order that agrees with differential gain modulation of ionotropic subcircuits. Whereas the subcircuits can collectively signal predictive models of the world based on life-long learning, their differential scaling continuously adjusts these models to changing mechanical contexts based on sensory input on a fast time scale of a few 100 ms. The control theory of biomimetic robots demonstrates that this precision scaling is an effective and resource-efficient solution to adapt the activation of individual muscle groups during locomotion to changing conditions such as ground compliance and carried load. Although it is not unconceivable that spinal ionotropic circuitry could achieve scaling by itself, neurophysiological findings emphasize that this is a unique functionality of metabotropic effects since recent recordings in sensorimotor circuitry conflict with mechanisms proposed for ionotropic scaling in other CNS areas. We substantiate that precision scaling of ionotropic subcircuits is a main functional principle for many monoaminergic projections throughout the CNS, implying that the monoaminergic circuitry forms a network within the network composed of the ionotropic circuitry. Thereby, we provide an early-level interpretation of the mechanisms of psychopharmacological drugs that interfere with the monoaminergic systems.

An Extended Passive Motion Paradigm for Human-Like Posture and Movement Planning in Redundant Manipulators

A major challenge in robotics and computational neuroscience is relative to the posture/movement problem in presence of kinematic redundancy. We recently addressed this issue using a principled approach which, in conjunction with nonlinear inverse optimization, allowed capturing postural strategies such as Donders' law. In this work, after presenting this general model specifying it as an extension of the Passive Motion Paradigm, we show how, once fitted to capture experimental postural strategies, the model is actually able to also predict movements. More specifically, the passive motion paradigm embeds two main intrinsic components: joint damping and joint stiffness. In previous work we showed that joint stiffness is responsible for static postures and, in this sense, its parameters are regressed to fit to experimental postural strategies. Here, we show how joint damping, in particular its anisotropy, directly affects task-space movements. Rather than using damping parameters to fit a posteriori task-space motions, we make the a priori hypothesis that damping is proportional to stiffness. This remarkably allows a postural-fitted model to also capture dynamic performance such as curvature and hysteresis of task-space trajectories during wrist pointing tasks, confirming and extending previous findings in literature.

A New Objective Function for Obstacle Avoidance by Redundant Service Robot Arms

The performance of task-space tracking control of kinematically redundant robots regulating self-motion to ensure obstacle avoidance is studied and discussed. As the sub-task objective, the links of the kinematically redundant assistive robot should avoid any collisions with the patient that is being assisted. The shortcomings of the obstacle avoidance algorithms are discussed and a new obstacle avoidance algorithm is proposed. The performance of the proposed algorithm is validated with tests that were carried out using the virtual model of a seven degrees-of-freedom robot arm. The test results indicate that the developed controller for the robot manipulator is successful in both accomplishing the main-task and the sub-task objectives.