Modelling natural and artificial hands with synergies

Rewiring the evolution of the human hand: How the embodiment of a virtual bionic tool improves behavior

Humans are the most versatile tool users among animals. Accordingly, our manual skills evolved alongside the shape of the hand. In the future, further evolution may take place: humans may merge with their tools, and technology may integrate into our biology in a way that blurs the line between the two. So, the question is whether humans can embody a bionic tool (i.e., experience it as part of their body) and thus if this would affect behavior. We investigated in virtual reality how the substitution of the hand with a virtual grafting of an end-effector, either non-naturalistic (a bionic tool) or naturalistic (a hand), impacts embodiment and behavior. Across four experiments, we show that the virtual grafting of a bionic tool elicits a sense of embodiment similar to or even stronger than its natural counterpart. In conclusion, the natural usage of bionic tools can rewire the evolution of human behavior.

A soft supernumerary hand for rehabilitation in sub-acute stroke: a pilot study

In patients with subacute stroke, task specific training (TST) has been shown to accelerate functional recovery of the upper limb. However, many patients do not have sufficient active extension of the fingers to perform this treatment. In these patients, here we propose a new rehabilitation technique in which TST is performed through a soft robotic hand (SoftHand-X). In short, the extension of the robotic fingers is controlled by the patient through his residual, albeit minimal, active extension of the fingers or wrist, while the patient was required to relax the muscles to achieve full flexion of the robotic fingers. TST with SoftHand-X was attempted in 27 subacute stroke patients unable to perform TST due to insufficient active extension of the fingers. Four patients (14.8%) were able to perform the proposed treatment (10 daily sessions of 60 min each). They reported an excellent level of participation. After the treatment, both clinical score of spasticity and its electromyographic correlate (stretch reflex) decreased. In subacute stroke patients, TST using SoftHand-X is a well-accepted treatment, resulting in a decrease of spasticity. At present, it can be applied only in a small proportion of the patients who cannot perform conventional TST, though extensions are possible.

Co-optimization of robotic design and skill inspired by human hand evolution

During evolution of the human hand, evolutionary morphology has been closely related to behavior in complicated environments. Numerous researchers have revealed that learned skills have affected hand evolution. Inspired by this phenomenon, a co-optimization approach for underactuated hands is proposed that takes grasping skills and structural parameters into consideration. In our proposal, hand design, especially the underactuated mechanism, can be parameterized and shared with all the local agents. These mechanical parameters can be updated globally by the independent agents. In addition, we also train human-like 'feeling' of grasping: grasping stability is estimated in advance before the object drops, which can speed up grasping training. In this paper, our method is instantiated to address the optimization problem for the torsion spring mechanical parameters of an underactuated robotic hand with multi-actuators, and then the optimized results are transferred to the actual physical robotic hand to test the improvement of grasping. This collaborative evolution process leverages the dexterity of the multi-actuators and the adaptivity of the underactuated mechanism.

Adaptable Poly-Articulated Bionic Hands EnhanceBoth Performance and User's Perception in Bilateral Amputation: A Case Study

This article evaluates and compares the performance and perception of prosthetic devices based on different design principles, a traditional rigid gripper and an adaptable poly-articulated hand, in a pre- and post-training protocol with an individual with bilateral amputation. As a representative of the first class, we use commercial hands (Ottobock's MyoHand VariPlus Speed), which is a widely adopted model by prosthesis users worldwide. We compare these with two SoftHand Pro hands, which are experimental prototypes exhibiting 19 articulations actuated by one single motor, and are inspired by human hand motor control models. Results show that the individual with bilateral amputation, who was a non-expert myoelectric user, achieved better performance with adaptive poly-articulated hands. Furthermore, the acceptation, satisfaction and perceived functionality of the user were considerably higher for the SoftHand Pro. An observational analysis of the patient's behaviour by experienced therapists suggests that adaptable poly-articulated hands reduced compensatory movements and cognitive load. Using soft technologies may be especially advantageous for individuals with bilateral amputation, who present a very limited residual mobility and can largely benefit from the active use of their artificial arms in everyday life.

Sequential contact-based adaptive grasping for robotic hands

This paper proposes a novel type of grasping strategy that draws inspiration from the role of touch and the importance of wrist motions in human grasping. The proposed algorithm, which we call Sequential Contact-based Adaptive Grasping, can be used to reactively modify a given grasp plan according to contacts arising between the hand and the object. This technique, based on a systematic constraint categorization and an iterative task inversion procedure, is shown to lead to synchronized motions of the fingers and the wrist, as it can be observed in humans, and to increase grasp success rate by substantially mitigating the relevant problems of object slippage during hand closure and of uncertainties caused by the environment and by the perception system. After describing the grasping problem in its quasi-static aspects, the algorithm is derived and discussed with some simple simulations. The proposed method is general as it can be applied to different kinds of robotic hands. It refines a priori defined grasp plans and significantly reduces their accuracy requirements by relying only on a forward kinematic model and elementary contact information. The efficacy of our approach is confirmed by experimental results of tests performed on a collaborative robot manipulator equipped with a state-of-the-art underactuated soft hand.

Design, Implementation, and Evaluation of a Variable Stiffness Transradial Hand Prosthesis

We present the design, implementation, and experimental evaluation of a low-cost, customizable, easy-to-use transradial hand prosthesis capable of adapting its compliance. Variable stiffness actuation (VSA) of the prosthesis is based on antagonistically arranged tendons coupled to nonlinear springs driven through a Bowden cable based power transmission. Bowden cable based antagonistic VSA can, not only regulate the stiffness and the position of the prosthetic hand but also enables a light-weight and low-cost design, by the opportunistic placement of motors, batteries, and controllers on any convenient location on the human body, while nonlinear springs are conveniently integrated inside the forearm. The transradial hand prosthesis also features tendon driven underactuated compliant fingers that allow natural adaption of the hand shape to wrap around a wide variety of object geometries, while the modulation of the stiffness of their drive tendons enables the prosthesis to perform various tasks with high dexterity. The compliant fingers of the prosthesis add inherent robustness and flexibility, even under impacts. The control of the variable stiffness transradial hand prosthesis is achieved by an sEMG based natural human-machine interface.

Reducing Anthropomorphic Hand Degrees of Actuation with Grasp-Function-Dependent and Joint-Element-Sparse Hand Synergies

In this paper, a set of grasp-function-dependent and joint-element-sparse hand synergies was proposed. First, hand synergies were extracted from five basic categories of movements by principal component analysis (PCA). Then, varimax rotation was applied on these synergies, so each sparse synergy only represented a limited number of joints. Next, according to the contribution to these sparse synergies, finger joints were clustered into different joint modules. Finally, integrating the joint modules in different categories of hand movements, the minimum number of actuators and joint synergic modules for anthropomorphic hands were determined. The results showed that using 5 groups of joint modules and 7–9 actuators we can achieve the best performance of grasp function and motion flexibility. Furthermore, through the reasonable design of adaptive and hyperextension functional joint modules, anthropomorphic hands can better meet the requirements of different tasks like power grasping and precision pinching. Comparing with traditional finger-based actuation strategy, the joint coupling scheme achieved better anthropomorphic performance and larger workspace. These above findings will benefit the development of mechanical structure design and control method of anthropomorphic hands.

Object Manipulation with an Anthropomorphic Robotic Hand via Deep Reinforcement Learning with a Synergy Space of Natural Hand Poses

Anthropomorphic robotic hands are designed to attain dexterous movements and flexibility much like human hands. Achieving human-like object manipulation remains a challenge especially due to the control complexity of the anthropomorphic robotic hand with a high degree of freedom. In this work, we propose a deep reinforcement learning (DRL) to train a policy using a synergy space for generating natural grasping and relocation of variously shaped objects using an anthropomorphic robotic hand. A synergy space is created using a continuous normalizing flow network with point clouds of haptic areas, representing natural hand poses obtained from human grasping demonstrations. The DRL policy accesses the synergistic representation and derives natural hand poses through a deep regressor for object grasping and relocation tasks. Our proposed synergy-based DRL achieves an average success rate of 88.38% for the object manipulation tasks, while the standard DRL without synergy space only achieves 50.66%. Qualitative results show the proposed synergy-based DRL policy produces human-like finger placements over the surface of each object including apple, banana, flashlight, camera, lightbulb, and hammer.

Impairment and Compensation in Dexterous Upper-Limb Function After Stroke. From the Direct Consequences of Pyramidal Tract Lesions to Behavioral Involvement of Both Upper-Limbs in Daily Activities

Impairments in dexterous upper limb function are a significant cause of disability following stroke. While the physiological basis of movement deficits consequent to a lesion in the pyramidal tract is well demonstrated, specific mechanisms contributing to optimal recovery are less apparent. Various upper limb interventions (motor learning methods, neurostimulation techniques, robotics, virtual reality, and serious games) are associated with improvements in motor performance, but many patients continue to experience significant limitations with object handling in everyday activities. Exactly how we go about consolidating adaptive motor behaviors through the rehabilitation process thus remains a considerable challenge. An important part of this problem is the ability to successfully distinguish the extent to which a given gesture is determined by the neuromotor impairment and that which is determined by a compensatory mechanism. This question is particularly complicated in tasks involving manual dexterity where prehensile movements are contingent upon the task (individual digit movement, grasping, and manipulation…) and its objective (placing, two step actions…), as well as personal factors (motivation, acquired skills, and life habits…) and contextual cues related to the environment (presence of tools or assistive devices…). Presently, there remains a lack of integrative studies which differentiate processes related to structural changes associated with the neurological lesion and those related to behavioral change in response to situational constraints. In this text, we shall question the link between impairments, motor strategies and individual performance in object handling tasks. This scoping review will be based on clinical studies, and discussed in relation to more general findings about hand and upper limb function (manipulation of objects, tool use in daily life activity). We shall discuss how further quantitative studies on human manipulation in ecological contexts may provide greater insight into compensatory motor behavior in patients with a neurological impairment of dexterous upper-limb function.

Research Trends and Future Perspectives in Marine Biomimicking Robotics

Mechatronic and soft robotics are taking inspiration from the animal kingdom to create new high-performance robots. Here, we focused on marine biomimetic research and used innovative bibliographic statistics tools, to highlight established and emerging knowledge domains. A total of 6980 scientific publications retrieved from the Scopus database (1950-2020), evidencing a sharp research increase in 2003-2004. Clustering analysis of countries collaborations showed two major Asian-North America and European clusters. Three significant areas appeared: (i) energy provision, whose advancement mainly relies on microbial fuel cells, (ii) biomaterials for not yet fully operational soft-robotic solutions; and finally (iii), design and control, chiefly oriented to locomotor designs. In this scenario, marine biomimicking robotics still lacks solutions for the long-lasting energy provision, which presently hinders operation autonomy. In the research environment, identifying natural processes by which living organisms obtain energy is thus urgent to sustain energy-demanding tasks while, at the same time, the natural designs must increasingly inform to optimize energy consumption.

A soft, synergy-based robotic glove for grasping assistance

This paper presents a soft, tendon-driven, robotic glove designed to augment grasp capability and provide rehabilitation assistance for postspinal cord injury patients. The basis of the design is an underactuation approach utilizing postural synergies of the hand to support a large variety of grasps with a single actuator. The glove is lightweight, easy to don, and generates sufficient hand closing force to assist with activities of daily living. Device efficiency was examined through a characterization of the power transmission elements, and output force production was observed to be linear in both cylindrical and pinch grasp configurations. We further show that, as a result of the synergy-inspired actuation strategy, the glove only slightly alters the distribution of forces across the fingers, compared to a natural, unassisted grasping pattern. Finally, a preliminary case study was conducted using a participant suffering from an incomplete spinal cord injury (C7). It was found that through the use of the glove, the participant was able to achieve a 50% performance improvement (from four to six blocks) in a standard Box and Block test.

The Vicarios Virtual Reality Interface for Remote Robotic Teleoperation

Intuitive interaction is the cornerstone of accurate and effective performance in remote robotic teleoperation. It requires high-fidelity in control actions as well as perception (vision, haptic, and other sensory feedback) of the remote environment. This paper presentsVicarios, a Virtual Reality (VR) based interface with the aim of facilitating intuitive real-time remote teleoperation, while utilizing the inherent benefits of VR, including immersive visualization, freedom of user viewpoint selection, and fluidity of interaction through natural action interfaces.Vicariosaims to enhance the situational awareness, using the concept ofviewpoint-independent mappingbetween the operator and the remote scene, thereby giving the operator better control in the perception-action loop. The article describes the overall system ofVicarios, with its software, hardware, and communication framework. A comparative user study quantifies the impact of the interface and its features, including immersion and instantaneous user viewpoint changes, termed “teleporting”, on users’ performance. The results show that users’ performance with the VR-based interface was either similar to or better than the baseline condition of traditional stereo video feedback, approving the realistic nature of theVicariosinterface. Furthermore, including the teleporting feature in VR significantly improved participants’ performance and their appreciation for it, which was evident in the post-questionnaire results.Vicarioscapitalizes on the intuitiveness and flexibility of VR to improve accuracy in remote teleoperation.

Finger interdependence and unintentional force drifts: Lessons from manipulations of visual feedback

We explored the phenomenon of unintentional finger force drift by using visual feedback on the force produced either by explicitly instructed (master) finger pairs or by non-instructed (enslaved) finger pairs. In particular, we drew contrasting predictions from two hypotheses: that force drifts represented consequences of drifts in effector referent coordinates at the level of individual fingers vs. at the level of finger modes (hypothetical variables accounting for the finger force interdependence). Subjects performed accurate force production with two fingers of a hand, index-ring or middle-little. They received visual feedback on the force produced either by the master fingers or by the other two, enslaved, fingers. The feedback scale was adjusted to ensure that the subjects did not know the difference between these two, randomly presented, conditions. Under feedback on the master finger force, enslaved force showed a consistent drift upward. Under feedback on the enslaved finger force, master force showed a consistent drift downward. The subjects were unaware of the force drifts, which could reach over 35% of the initial force magnitude. The data support the hypothesis on drifts in the referent coordinate at the level of individual digits, not finger modes, as the origin of unintentional force drifts. The consistent increase in the relative amount of force produced by the enslaved fingers suggests that the commonly used methods to quantify enslaving should include relatively brief force production tasks.

Sensing form - finger gaiting as key to tactile object exploration - a data glove analysis of a prototypical daily task

Background

Motor hand skill and associated dexterity is important for meeting the challenges of daily activity and an important resource post-stroke. In this context, the present study investigated the finger movements of right-handed subjects during tactile manipulation of a cuboid, a prototypical task underlying tactile exploration. During one motor act, the thumb and fingers of one hand surround the cuboid in a continuous and regular manner. While the object is moved by the guiding thumb, the opposed supporting fingers are replaced once they reach their joint limits by free fingers, a mechanism termed finger gaiting.

Methods

For both hands of 22 subjects, we acquired the time series of consecutive manipulations of a cuboid at a frequency of 1 Hz, using a digital data glove consisting of 29 sensors. Using principle component analysis, we decomposed the short action into motor patterns related to successive manipulations of the cuboid. The components purport to represent changing grasp configurations involving the stabilizing fingers and guiding thumb. The temporal features of the components permits testing whether the distinct configurations occur at the frequency of 1 Hz, i.e. within the time window of 1 s, and, thus, taxonomic classification of the manipulation as finger gaiting.

Results

The fraction of variance described by the principal components indicated that three components described the salient features of the single motor acts for each hand. Striking in the finger patterns was the prominent and varying roles of the MCP and PIP joints of the fingers, and the CMC joint of the thumb. An important aspect of the three components was their representation of distinct finger configurations within the same motor act. Principal component and graph theory analysis confirmed modular, functionally synchronous action of the involved joints. The computation of finger trajectories in one subject illustrated the workspace of the task, which differed for the right and left hands.

Conclusion

In this task one complex motor act of 1 s duration could be described by three elementary and hierarchically ordered grasp configurations occurring at the prescribed frequency of 1 Hz. Therefore, these configurations represent finger gaiting, described until now only in artificial systems, as the principal mechanism underlying this prototypical task.

Trial registration

clinicaltrials.gov, NCT02865642, registered 12 August 2016.

Exploring augmented grasping capabilities in a multi-synergistic soft bionic hand

BACKGROUND

State-of-the-art bionic hands incorporate hi-tech devices which try to overcome limitations of conventional single grip systems. Unfortunately, their complexity often limits mechanical robustness and intuitive prosthesis control. Recently, the translation of neuroscientific theories (i.e. postural synergies) in software and hardware architecture of artificial devices is opening new approaches for the design and control of upper-limb prostheses.

METHODS

Following these emerging principles, previous research on the SoftHand Pro, which embeds one physical synergy, showed promising results in terms of intuitiveness, robustness, and grasping performance. To explore these principles also in hands with augmented capabilities, this paper describes the SoftHand 2 Pro, a second generation of the device with 19 degrees-of-freedom and a second synergistic layer. After a description of the proposed device, the work explores a continuous switching control method based on a myoelectric pattern recognition classifier.

RESULTS

The combined system was validated using standardized assessments with able-bodied and, for the first time, amputee subjects. Results show an average improvement of more than 30% of fine grasp capabilities and about 10% of hand function compared with the first generation SoftHand Pro.

CONCLUSIONS

Encouraging results suggest how this approach could be a viable way towards the design of more natural, reliable, and intuitive dexterous hands.

Reach to Grasp Planning for a Synergy-Controlled Robotic Hand based on Pesudo-Distance Formulation

In the past several years, grasp analysis of multi-fingered robotic hands has been actively studied through the use of posture synergies. In these grasping planning algorithms, a formulated optimization is usually performed in the hand’s low-dimensional representation together with the hand’s position and orientation. The optimization terminates at a stable grasp, often after repeated trials with different initial guesses. Furthermore, there is no guarantee that the generated grasp leads to a smooth reach-to-grasp trajectory since the grasping planning process mostly concerns hand poses with the fingers proximal to the object. A unified theoretical framework of a gradient-based iterative algorithm is hence proposed in this paper to plan a reach-to-grasp task, predicting the grasp quality and adjusting the hand’s posture synergies, position and orientation during the approaching phase to achieve a stable grasp. The grasp quality measurement is adopted from a highly efficient pseudo-distance formulation. Stable power grasp and precision pinch can be consistently and intentionally planned with different contact conditions specified in the formulation, which means that an intention for planning a power grasp would not generate a pinch result. Several numerical simulation case studies are presented to demonstrate the effectiveness of the proposed algorithm.

Trends and challenges in robot manipulation

Dexterous manipulation is one of the primary goals in robotics. Robots with this capability could sort and package objects, chop vegetables, and fold clothes. As robots come to work side by side with humans, they must also become human-aware. Over the past decade, research has made strides toward these goals. Progress has come from advances in visual and haptic perception and in mechanics in the form of soft actuators that offer a natural compliance. Most notably, immense progress in machine learning has been leveraged to encapsulate models of uncertainty and to support improvements in adaptive and robust control. Open questions remain in terms of how to enable robots to deal with the most unpredictable agent of all, the human.

Human-Inspired Representation of Object-Specific Grasps for Anthropomorphic Hands

The human hand is a complex, highly-articulated system, which has been the source of inspiration in designing humanoid robotic and prosthetic hands. Understanding the functionality of the human hand is crucial for the design, efficient control and transfer of human versatility and dexterity to such anthropomorphic robotic hands. Although research in this area has made significant advances, the synthesis of grasp configurations, based on observed human grasping data, is still an unsolved and challenging task. In this work we derive a novel, constrained autoencoder model, that encodes human grasping data in a compact representation. This representation encodes both the grasp type in a three-dimensional latent space and the object size as an explicit parameter constraint allowing the direct synthesis of object-specific grasps. We train the model on 2250 grasps generated by 15 subjects using 35 diverse objects from the KIT and YCB object sets. In the evaluation we show that the synthesized grasp configurations are human-like and have a high probability of success under pose uncertainty.

Kinematic Synthesis and Design of the Robust Closed Loop Articulated Minimally actuated (CLAM) Hand

The paper presents a comprehensive design process for the development of the minimally actuated Closed Loop Articulated Mechanical (CLAM) hand. Each of the fingers is designed as a planar one degree of freedom eight-bar linkage with an anthropomorphic backbone chain. The fingers movement is based on experimentally obtained physiological precision grasping task, with incorporated second-order task specifications, related to maintaining fingertip–body contact with a minimum number of fingers. Instead of actuating individual joints in each finger, the mechanism generates the desired anthropomorphic grasping trajectory using a single actuator in each finger. The paper offers not only details on multi-loop articulated hands design based on anthropometric data and physiological task with second-order effects for maintaining the object–fingertip contact, but also shows how this class of hands that have been considered mostly for adaptive grasping can be successfully utilized for precision grasping. The minimal number of fingers and actuators can simplify the control, resulting in a robust, lightweight, and cost-effective solution for the precision grasping of a variety of objects with different shapes and geometries.

Hand–object configuration estimation using particle filters for dexterous in-hand manipulation

We consider the problem of in-hand dexterous manipulation with a focus on unknown or uncertain hand–object parameters, such as hand configuration, object pose within hand, and contact positions. In particular, in this work we formulate a generic framework for hand–object configuration estimation using underactuated hands as an example. Owing to the passive reconfigurability and the lack of encoders in the hand’s joints, it is challenging to estimate, plan, and actively control underactuated manipulation. By modeling the grasp constraints, we present a particle filter-based framework to estimate the hand configuration. Specifically, given an arbitrary grasp, we start by sampling a set of hand configuration hypotheses and then randomly manipulate the object within the hand. While observing the object’s movements as evidence using an external camera, which is not necessarily calibrated with the hand frame, our estimator calculates the likelihood of each hypothesis to iteratively estimate the hand configuration. Once converged, the estimator is used to track the hand configuration in real time for future manipulations. Thereafter, we develop an algorithm to precisely plan and control the underactuated manipulation to move the grasped object to desired poses. In contrast to most other dexterous manipulation approaches, our framework does not require any tactile sensing or joint encoders, and can directly operate on any novel objects, without requiring a model of the object a priori. We implemented our framework on both the Yale Model O hand and the Yale T42 hand. The results show that the estimation is accurate for different objects, and that the framework can be easily adapted across different underactuated hand models. In the end, we evaluated our planning and control algorithm with handwriting tasks, and demonstrated the effectiveness of the proposed framework.

Skin Stretch Haptic Feedback to Convey Closure Information in Anthropomorphic, Under-Actuated Upper Limb Soft Prostheses

Restoring hand function in individuals with upper limb loss is a challenging task, made difficult by the complexity of human hands from both a functional and sensory point of view. Users of commercial prostheses, even sophisticated devices, must visually attend to the hand to know its state, since in most cases they are not provided with any direct sensory information. Among the different types of haptic feedback that can be delivered, particularly information on hand opening is likely to reduce the requirement of constant visual attention. In recent years, there has been a trend of using underactuated, compliant multi-fingered hands as upper limb prostheses, in part due to their simplicity and ease of use attributed to low degree-of-freedom (d.o.f.) actuation. The trend toward underactuation encourages the design of one d.o.f. haptic devices to provide intuitive sensory feedback from the prosthesis. However, mapping the closure of a multi-d.o.f. prosthetic hand to a simple and intuitive haptic cue is not a trivial task. In this paper, we explore the use of a one d.o.f. skin stretch haptic device, the rice haptic rocker, to provide intuitive proprioceptive feedback indicating overall hand closure of an underactuated prosthesis. The benefits and challenges of the system are assessed in multi-tasking and reduced vision scenarios for an object-size discrimination task, in an effort to simulate challenges in daily life, and are compared against the haptic resolution of the device using the just noticeable difference. Finally, an evaluation done with a prosthesis user, in the form of a truncated version of the Activities Measure for Upper Limb Amputees (AM-ULA), shows possible benefits of the addition of haptic feedback in tasks with reduced visual attention.

Motor modules account for active perception of force

Despite longstanding evidence suggesting a relation between action and perception, the mechanisms underlying their integration are still unclear. It has been proposed that to simplify the sensorimotor integration processes underlying active perception, the central nervous system (CNS) selects patterns of movements aimed at maximizing sampling of task-related sensory input. While previous studies investigated the action-perception loop focusing on the role of higher-level features of motor behavior (e.g., kinematic invariants, effort), the present study explored and quantified the contribution of lower-level organization of motor control. We tested the hypothesis that the coordinated recruitment of group of muscles (i.e., motor modules) engaged to counteract an external force contributes to participants’ perception of the same force. We found that: 1) a model describing the modulation of a subset of motor modules involved in the motor task accounted for about 70% of participants’ perceptual variance; 2) an alternative model, incompatible with the motor modules hypothesis, accounted for significantly lower variance of participants’ detection performance. Our results provide empirical evidence of the potential role played by muscle activation patterns in active perception of force. They also suggest that a modular organization of motor control may mediate not only coordination of multiple muscles, but also perceptual inference.

Kinematic synergies of hand grasps: a comprehensive study on a large publicly available dataset

Background

Hand grasp patterns require complex coordination. The reduction of the kinematic dimensionality is a key process to study the patterns underlying hand usage and grasping. It allows to define metrics for motor assessment and rehabilitation, to develop assistive devices and prosthesis control methods. Several studies were presented in this field but most of them targeted a limited number of subjects, they focused on postures rather than entire grasping movements and they did not perform separate analysis for the tasks and subjects, which can limit the impact on rehabilitation and assistive applications. This paper provides a comprehensive mapping of synergies from hand grasps targeting activities of daily living. It clarifies several current limits of the field and fosters the development of applications in rehabilitation and assistive robotics.

Methods

In this work, hand kinematic data of 77 subjects, performing up to 20 hand grasps, were acquired with a data glove (a 22-sensor CyberGlove II data glove) and analyzed. Principal Component Analysis (PCA) and hierarchical cluster analysis were used to extract and group kinematic synergies that summarize the coordination patterns available for hand grasps.

Results

Twelve synergies were found to account for > 80% of the overall variation. The first three synergies accounted for more than 50% of the total amount of variance and consisted of: the flexion and adduction of the Metacarpophalangeal joint (MCP) of fingers 3 to 5 (synergy #1), palmar arching and flexion of the wrist (synergy #2) and opposition of the thumb (synergy #3). Further synergies refine movements and have higher variability among subjects.

Conclusion

Kinematic synergies are extracted from a large number of subjects (77) and grasps related to activities of daily living (20). The number of motor modules required to perform the motor tasks is higher than what previously described. Twelve synergies are responsible for most of the variation in hand grasping. The first three are used as primary synergies, while the remaining ones target finer movements (e.g. independence of thumb and index finger). The results generalize the description of hand kinematics, better clarifying several limits of the field and fostering the development of applications in rehabilitation and assistive robotics.

Scaling Our World View: How Monoamines Can Put Context Into Brain Circuitry

Monoamines are presumed to be diffuse metabotropic neuromodulators of the topographically and temporally precise ionotropic circuitry which dominates CNS functions. Their malfunction is strongly implicated in motor and cognitive disorders, but their function in behavioral and cognitive processing is scarcely understood. In this paper, the principles of such a monoaminergic function are conceptualized for locomotor control. We find that the serotonergic system in the ventral spinal cord scales ionotropic signals and shows topographic order that agrees with differential gain modulation of ionotropic subcircuits. Whereas the subcircuits can collectively signal predictive models of the world based on life-long learning, their differential scaling continuously adjusts these models to changing mechanical contexts based on sensory input on a fast time scale of a few 100 ms. The control theory of biomimetic robots demonstrates that this precision scaling is an effective and resource-efficient solution to adapt the activation of individual muscle groups during locomotion to changing conditions such as ground compliance and carried load. Although it is not unconceivable that spinal ionotropic circuitry could achieve scaling by itself, neurophysiological findings emphasize that this is a unique functionality of metabotropic effects since recent recordings in sensorimotor circuitry conflict with mechanisms proposed for ionotropic scaling in other CNS areas. We substantiate that precision scaling of ionotropic subcircuits is a main functional principle for many monoaminergic projections throughout the CNS, implying that the monoaminergic circuitry forms a network within the network composed of the ionotropic circuitry. Thereby, we provide an early-level interpretation of the mechanisms of psychopharmacological drugs that interfere with the monoaminergic systems.

Optimizing the structure and movement of a robotic bat with biological kinematic synergies

In this article, we present methods to optimize the design and flight characteristics of a biologically inspired bat-like robot. In previous, work we have designed the topological structure for the wing kinematics of this robot; here we present methods to optimize the geometry of this structure, and to compute actuator trajectories such that its wingbeat pattern closely matches biological counterparts. Our approach is motivated by recent studies on biological bat flight that have shown that the salient aspects of wing motion can be accurately represented in a low-dimensional space. Although bats have over 40 degrees of freedom (DoFs), our robot possesses several biologically meaningful morphing specializations. We use principal component analysis (PCA) to characterize the two most dominant modes of biological bat flight kinematics, and we optimize our robot’s parametric kinematics to mimic these. The method yields a robot that is reduced from five degrees of actuation (DoAs) to just three, and that actively folds its wings within a wingbeat period. As a result of mimicking synergies, the robot produces an average net lift improvesment of 89% over the same robot when its wings cannot fold.

The SoftHand Pro: Functional evaluation of a novel, flexible, and robust myoelectric prosthesis

Roughly one quarter of active upper limb prosthetic technology is rejected by the user, and user surveys have identified key areas requiring improvement: function, comfort, cost, durability, and appearance. Here we present the first systematic, clinical assessment of a novel prosthetic hand, the SoftHand Pro (SHP), in participants with transradial amputation and age-matched, limb-intact participants. The SHP is a robust and functional prosthetic hand that minimizes cost and weight using an underactuated design with a single motor. Participants with limb loss were evaluated on functional clinical measures before and after a 6-8 hour training period with the SHP as well as with their own prosthesis; limb-intact participants were tested only before and after SHP training. Participants with limb loss also evaluated their own prosthesis and the SHP (following training) using subjective questionnaires. Both objective and subjective results were positive and illuminated the strengths and weaknesses of the SHP. In particular, results pre-training show the SHP is easy to use, and significant improvement in the Activities Measure for Upper Limb Amputees in both groups following a 6-8 hour training highlights the ease of learning the unique features of the SHP (median improvement: 4.71 and 3.26 and p = 0.009 and 0.036 for limb loss and limb-intact groups, respectively). Further, we found no difference in performance compared to participant's own commercial devices in several clinical measures and found performance surpassing these devices on two functional tasks, buttoning a shirt and using a cell phone, suggesting a functional prosthetic design. Finally, improvements are needed in the SHP design and/or training in light of poor results in small object manipulation. Taken together, these results show the promise of the SHP, a flexible and adaptive prosthetic hand, and pave a path forward to ensuring higher functionality in future.

The Combinatorial Creature: Cortical Phenotypes within and across Lifetimes

The neocortex is one of the most distinctive structures of the mammalian brain, yet also one of the most varied in terms of both size and organization. Multiple processes have contributed to this variability, including evolutionary mechanisms (i.e., alterations in gene sequence) that alter the size, organization, and connections of neocortex, and activity dependent mechanisms that can also modify these same features. Thus, changes to the neocortex can occur over different time-scales, including within a single generation. This combination of genetic and activity dependent mechanisms that create a given cortical phenotype allows the mammalian neocortex to rapidly and flexibly adjust to different body and environmental contexts, and in humans permits culture to impact brain construction.

Replicating Human Hand Synergies Onto Robotic Hands: A Review on Software and Hardware Strategies

This review reports the principal solutions proposed in the literature to reduce the complexity of the control and of the design of robotic hands taking inspiration from the organization of the human brain. Several studies in neuroscience concerning the sensorimotor organization of the human hand proved that, despite the complexity of the hand, a few parameters can describe most of the variance in the patterns of configurations and movements. In other words, humans exploit a reduced set of parameters, known in the literature as synergies, to control their hands. In robotics, this dimensionality reduction can be achieved by coupling some of the degrees of freedom (DoFs) of the robotic hand, that results in a reduction of the needed inputs. Such coupling can be obtained at the software level, exploiting mapping algorithm to reproduce human hand organization, and at the hardware level, through either rigid or compliant physical couplings between the joints of the robotic hand. This paper reviews the main solutions proposed for both the approaches.

HapPro: A Wearable Haptic Device for Proprioceptive Feedback

OBJECTIVE

Myoelectric hand prostheses have reached a considerable technological level and gained an increasing attention in assistive robotics. However, their abandonment rate remains high, with unintuitive control and lack of sensory feedback being major causes. Among the different types of sensory information, proprioception, e.g., information on hand aperture, is crucial to successfully perform everyday actions. Despite the many attempts in literature to restore and convey this type of feedback, much remains to be done to close the action-perception loop in prosthetic devices.

METHODS

With this as motivation, in this paper we introduce HapPro, a wearable, noninvasive haptic device that can convey proprioceptive information for a prosthetic hand. The device was used with an under-actuated, simple to control anthropomorphic robotic hand, providing information about hand aperture by mapping it to the position of a wheel that can run on the user's forearm. Tests with 43 able bodied subjects and one amputee subject were conducted in order to quantify the effectiveness of HapPro as a feedback device.

RESULTS

HapPro provided a good level of accuracy for item discrimination. Participants also reported the device to be intuitive and effective in conveying proprioceptive cues. Similar results were obtained in the proof-of-concept experiment with an amputee subject.

CONCLUSIONS

Results show that HapPro is able to convey information on the opening of a prosthetic hand in a noninvasive way.

SIGNIFICANCE

Using this device for proprioceptive feedback could improve usability of myoelectric prostheses, potentially reducing abandonment and increasing quality of life for their users.

Autonomy in Rehabilitation Robotics: An Intersection

Within the field of human rehabilitation, robotic machines are used both to rehabilitate the body and to perform functional tasks. Robotics autonomy able to perceive the external world and reason about high-level control decisions, however, seldom is present in these machines. For functional tasks in particular, autonomy could help to decrease the operational burden on the human and perhaps even to increase access-and this potential only grows as human motor impairments become more severe. There are however serious, and often subtle, considerations to introducing clinically-feasible robotics autonomy to rehabilitation robots and machines. Today the fields of robotics autonomy and rehabilitation robotics are largely separate. The topic of this article is at the intersection of these fields: the introduction of clinically-feasible autonomy solutions to rehabilitation robots, and opportunities for autonomy within the rehabilitation domain.

A Systematic Review on Muscle Synergies: From Building Blocks of Motor Behavior to a Neurorehabilitation Tool

The central nervous system (CNS) is believed to utilize specific predefined modules, called muscle synergies (MS), to accomplish a motor task. Yet questions persist about how the CNS combines these primitives in different ways to suit the task conditions. The MS hypothesis has been a subject of debate as to whether they originate from neural origins or nonneural constraints. In this review article, we present three aspects related to the MS hypothesis: (1) the experimental and computational evidence in support of the existence of MS, (2) algorithmic approaches for extracting them from surface electromyography (EMG) signals, and (3) the possible role of MS as a neurorehabilitation tool. We note that recent advances in computational neuroscience have utilized the MS hypothesis in motor control and learning. Prospective advances in clinical, medical, and engineering sciences and in fields such as robotics and rehabilitation stand to benefit from a more thorough understanding of MS.

Grasp specific and user friendly interface design for myoelectric hand prostheses

This paper presents the design and characterisation of a hand prosthesis and its user interface, focusing on performing the most commonly used grasps in activities of daily living (ADLs). Since the operation of a multi-articulated powered hand prosthesis is difficult to learn and master, there is a significant rate of abandonment by amputees in preference for simpler devices. In choosing so, amputees chose to live with fewer features in their prosthesis that would more reliably perform the basic operations. In this paper, we look simultaneously at a hand prosthesis design method that aims for a small number of grasps, a low complexity user interface and an alternative method to the current use of EMG as a preshape selection method through the use of a simple button; to enable amputees to get to and execute the intended hand movements intuitively, quickly and reliably. An experiment is reported at the end of the paper comparing the speed and accuracy with which able-bodied naive subjects are able to select the intended preshapes through the use of a simplified EMG method and a simple button. It is shown that the button was significantly superior in the speed of successful task completion and marginally superior in accuracy (success of first attempt).

Preliminary results toward a naturally controlled multi-synergistic prosthetic hand

Robotic hands embedding human motor control principles in their mechanical design are getting increasing interest thanks to their simplicity and robustness, combined with good performance. Another key aspect of these hands is that humans can use them very effectively thanks to the similarity of their behavior with real hands. Nevertheless, controlling more than one degree of actuation remains a challenging task. In this paper, we take advantage of these characteristics in a multi-synergistic prosthesis. We propose an integrated setup composed of Pisa/IIT SoftHand 2 and a control strategy which simultaneously and proportionally maps the human hand movements to the robotic hand. The control technique is based on a combination of non-negative matrix factorization and linear regression algorithms. It also features a real-time continuous posture compensation of the electromyographic signals based on an IMU. The algorithm is tested on five healthy subjects through an experiment in a virtual environment. In a separate experiment, the efficacy of the posture compensation strategy is evaluated on five healthy subjects and, finally, the whole setup is successfully tested in performing realistic daily life activities.

Grasp Performance of a Soft Synergy-Based Prosthetic Hand: A Pilot Study

Current prosthetic hands are frequently rejected in part due to limited functionality and versatility. We assessed the feasibility of a novel prosthetic hand, the SoftHand Pro (SHP), whose design combines soft robotics and hand postural synergies. Able-bodied subjects ( ) tracked cursor motion by opening and closing the SHP and performed a grasp-lift-hold-release (GLHR) task with a sensorized cylindrical object of variable weight. The SHP control was driven by electromyographic (EMG) signals from two antagonistic muscles. Although the time to perform the GLHR task was longer for the SHP than native hand for the first few trials (10.2 ± 1.4 s and 2.13 ± 0.09 s, respectively), performance was much faster on subsequent trials (~5 s). The SHP steady-state grip force was significantly modulated as a function of object weight ( ). For the native hand, however, peak and steady-state grip forces were modulated to a greater extent (+68% and +91%, respectively). These changes were mediated by the modulation of EMG amplitude and co-contraction. These data suggest that the SHP has a promise for prosthetic applications and point-to-design modifications that could improve the SHP.

On neuromechanical approaches for the study of biological and robotic grasp and manipulation

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank and open-minded assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas at the interface of neuromechanics, neuroscience, rehabilitation and robotics.

Postural Hand Synergies during Environmental Constraint Exploitation

Humans are able to intuitively exploit the shape of an object and environmental constraints to achieve stable grasps and perform dexterous manipulations. In doing that, a vast range of kinematic strategies can be observed. However, in this work we formulate the hypothesis that such ability can be described in terms of a synergistic behavior in the generation of hand postures, i.e., using a reduced set of commonly used kinematic patterns. This is in analogy with previous studies showing the presence of such behavior in different tasks, such as grasping. We investigated this hypothesis in experiments performed by six subjects, who were asked to grasp objects from a flat surface. We quantitatively characterized hand posture behavior from a kinematic perspective, i.e., the hand joint angles, in both pre-shaping and during the interaction with the environment. To determine the role of tactile feedback, we repeated the same experiments but with subjects wearing a rigid shell on the fingertips to reduce cutaneous afferent inputs. Results show the persistence of at least two postural synergies in all the considered experimental conditions and phases. Tactile impairment does not alter significantly the first two synergies, and contact with the environment generates a change only for higher order Principal Components. A good match also arises between the first synergy found in our analysis and the first synergy of grasping as quantified by previous work. The present study is motivated by the interest of learning from the human example, extracting lessons that can be applied in robot design and control. Thus, we conclude with a discussion on implications for robotics of our findings.

A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.

Hand Grasping Synergies As Biometrics

Recently, the need for more secure identity verification systems has driven researchers to explore other sources of biometrics. This includes iris patterns, palm print, hand geometry, facial recognition, and movement patterns (hand motion, gait, and eye movements). Identity verification systems may benefit from the complexity of human movement that integrates multiple levels of control (neural, muscular, and kinematic). Using principal component analysis, we extracted spatiotemporal hand synergies (movement synergies) from an object grasping dataset to explore their use as a potential biometric. These movement synergies are in the form of joint angular velocity profiles of 10 joints. We explored the effect of joint type, digit, number of objects, and grasp type. In its best configuration, movement synergies achieved an equal error rate of 8.19%. While movement synergies can be integrated into an identity verification system with motion capture ability, we also explored a camera-ready version of hand synergies-postural synergies. In this proof of concept system, postural synergies performed well, but only when specific postures were chosen. Based on these results, hand synergies show promise as a potential biometric that can be combined with other hand-based biometrics for improved security.

Synergy-Based Bilateral Port: A Universal Control Module for Tele-Manipulation Frameworks Using Asymmetric Master-Slave Systems

Endowing tele-manipulation frameworks with the capability to accommodate a variety of robotic hands is key to achieving high performances through permitting to flexibly interchange the end-effector according to the task considered. This requires the development of control policies that not only cope with asymmetric master–slave systems but also whose high-level components are designed in a unified space in abstraction from the devices specifics. To address this dual challenge, a novel synergy port is developed that resolves the kinematic, sensing, and actuation asymmetries of the considered system through generating motion and force feedback references in the hardware-independent hand postural synergy space. It builds upon the concept of the Cartesian-based synergy matrix, which is introduced as a tool mapping the fingertips Cartesian space to the directions oriented along the grasp principal components. To assess the effectiveness of the proposed approach, the synergy port has been integrated into the control system of a highly asymmetric tele-manipulation framework, in which the 3-finger hand exoskeleton HEXOTRAC is used as a master device to control the SoftHand, a robotic hand whose transmission system relies on a single motor to drive all joints along a soft synergistic path. The platform is further enriched with the vision-based motion capture system Optitrack to monitor the 6D trajectory of the user’s wrist, which is used to control the robotic arm on which the SoftHand is mounted. Experiments have been conducted with the humanoid robot COMAN and the KUKA LWR robotic manipulator. Results indicate that this bilateral interface is highly intuitive and allows users with no prior experience to reach, grasp, and transport a variety of objects exhibiting very different shapes and impedances. In addition, the hardware and control solutions proved capable of accommodating users with different hand kinematics. Finally, the proposed control framework offers a universal, flexible, and intuitive interface allowing for the performance of effective tele-manipulations.

A Prosthetic Hand Body Area Controller Based on Efficient Pattern Recognition Control Strategies

Poliarticulated prosthetic hands represent a powerful tool to restore functionality and improve quality of life for upper limb amputees. Such devices offer, on the same wearable node, sensing and actuation capabilities, which are not equally supported by natural interaction and control strategies. The control in state-of-the-art solutions is still performed mainly through complex encoding of gestures in bursts of contractions of the residual forearm muscles, resulting in a non-intuitive Human-Machine Interface (HMI). Recent research efforts explore the use of myoelectric gesture recognition for innovative interaction solutions, however there persists a considerable gap between research evaluation and implementation into successful complete systems. In this paper, we present the design of a wearable prosthetic hand controller, based on intuitive gesture recognition and a custom control strategy. The wearable node directly actuates a poliarticulated hand and wirelessly interacts with a personal gateway (i.e., a smartphone) for the training and personalization of the recognition algorithm. Through the whole system development, we address the challenge of integrating an efficient embedded gesture classifier with a control strategy tailored for an intuitive interaction between the user and the prosthesis. We demonstrate that this combined approach outperforms systems based on mere pattern recognition, since they target the accuracy of a classification algorithm rather than the control of a gesture. The system was fully implemented, tested on healthy and amputee subjects and compared against benchmark repositories. The proposed approach achieves an error rate of 1.6% in the end-to-end real time control of commonly used hand gestures, while complying with the power and performance budget of a low-cost microcontroller.

Multidigit force control during unconstrained grasping in response to object perturbations

Because of the complex anatomy of the human hand, in the absence of external constraints, a large number of postures and force combinations can be used to attain a stable grasp. Motor synergies provide a viable strategy to solve this problem of motor redundancy. In this study, we exploited the technical advantages of an innovative sensorized object to study unconstrained hand grasping within the theoretical framework of motor synergies. Participants were required to grasp, lift, and hold the sensorized object. During the holding phase, we repetitively applied external disturbance forces and torques and recorded the spatiotemporal distribution of grip forces produced by each digit. We found that the time to reach the maximum grip force during each perturbation was roughly equal across fingers, consistent with a synchronous, synergistic stiffening across digits. We further evaluated this hypothesis by comparing the force distribution of human grasping vs. robotic grasping, where the control strategy was set by the experimenter. We controlled the global hand stiffness of the robotic hand and found that this control algorithm produced a force pattern qualitatively similar to human grasping performance. Our results suggest that the nervous system uses a default whole hand synergistic control to maintain a stable grasp regardless of the number of digits involved in the task, their position on the objects, and the type and frequency of external perturbations.NEW & NOTEWORTHY We studied hand grasping using a sensorized object allowing unconstrained finger placement. During object perturbation, the time to reach the peak force was roughly equal across fingers, consistently with a synergistic stiffening across fingers. Force distribution of a robotic grasping hand, where the control algorithm is based on global hand stiffness, was qualitatively similar to human grasping. This suggests that the central nervous system uses a default whole hand synergistic control to maintain a stable grasp.

A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.