How Positioning Wearable Haptic Interfaces on Limbs Influences Virtual Embodiment

With increasing use of computer applications and robotic devices in our everyday life, and with the advent of metaverse, there is an urgent need of developing new types of interfaces that facilitate a more intuitive interaction in physical and virtual space. In this work, we investigate the influence of the location of haptic feedback devices on embodiment of virtual hands and user load during an interactive pick-and-place task. To do this, we conducted a user study with a 3x2 repeated measure experiment design: feedback position is varied between the distal phalanx of the index finger and the thumb, the proximal phalanx of the index finger and the thumb, and the wrist. These conditions of feedback are tested with the stimuli applied synchronously to the participant in one case, and with an additional delay of 350 ms in the second case. The results show that the location of the haptic feedback device does not affect embodiment, whereas the delay, i.e., whether the feedback is applied synchronously or asynchronously, affects embodiment. This suggests that for pick-and-place tasks, haptic feedback devices can be placed on the user's wrist without compromising performance making the hands to remain free, allowing unobstructed hand visibility for precise motion tracking, thereby improving accuracy.

Detachable Robotic Grippers for Human-Robot Collaboration

Collaborative robots promise to add flexibility to production cells thanks to the fact that they can work not only close to humans but also with humans. The possibility of a direct physical interaction between humans and robots allows to perform operations that were inconceivable with industrial robots. Collaborative soft grippers have been recently introduced to extend this possibility beyond the robot end-effector, making humans able to directly act on robotic hands. In this work, we propose to exploit collaborative grippers in a novel paradigm in which these devices can be easily attached and detached from the robot arm and used also independently from it. This is possible only with self-powered hands, that are still quite uncommon in the market. In the presented paradigm not only hands can be attached/detached to/from the robot end-effector as if they were simple tools, but they can also remain active and fully functional after detachment. This ensures all the advantages brought in by tool changers, that allow for quick and possibly automatic tool exchange at the robot end-effector, but also gives the possibility of using the hand capabilities and degrees of freedom without the need of an arm or of external power supplies. In this paper, the concept of detachable robotic grippers is introduced and demonstrated through two illustrative tasks conducted with a new tool changer designed for collaborative grippers. The novel tool changer embeds electromagnets that are used to add safety during attach/detach operations. The activation of the electromagnets is controlled through a wearable interface capable of providing tactile feedback. The usability of the system is confirmed by the evaluations of 12 users.

Integration of a Passive Exoskeleton and a Robotic Supernumerary Finger for Grasping Compensation in Chronic Stroke Patients: The SoftPro Wearable System

Upper-limb impairments are all-pervasive in Activities of Daily Living (ADLs). As a consequence, people affected by a loss of arm function must endure severe limitations. To compensate for the lack of a functional arm and hand, we developed a wearable system that combines different assistive technologies including sensing, haptics, orthotics and robotics. The result is a device that helps lifting the forearm by means of a passive exoskeleton and improves the grasping ability of the impaired hand by employing a wearable robotic supernumerary finger. A pilot study involving 3 patients, which was conducted to test the capability of the device to assist in performing ADLs, confirmed its usefulness and serves as a first step in the investigation of novel paradigms for robotic assistance.

Hand closure model for planning top grasps with soft robotic hands

Automating the act of grasping is one of the most compelling challenges in robotics. In recent times, a major trend has gained the attention of the robotic grasping community: soft manipulation. Along with the design of intrinsically soft robotic hands, it is important to devise grasp planning strategies that can take into account the hand characteristics, but are general enough to be applied to different robotic systems. In this article, we investigate how to perform top grasps with soft hands according to a model-based approach, using both power and precision grasps. The so-called closure signature (CS) is used to model closure motions of soft hands by associating to them a preferred grasping direction. This direction can be aligned to a suitable direction over the object to achieve successful top grasps. The CS-alignment is here combined with a recently developed AI-driven grasp planner for rigid grippers that is adjusted and used to retrieve an estimate of the optimal grasp to be performed on the object. The resulting grasp planner is tested with multiple experimental trials with two different robotic hands. A wide set of objects with different shapes was grasped successfully.

Exploiting Robot Hand Compliance and Environmental Constraints for Edge Grasps

This paper presents a method to grasp objects that cannot be picked directly from a table, using a soft, underactuated hand. These grasps are achieved by dragging the object to the edge of a table, and grasping it from the protruding part, performing so-called slide-to-edge grasps. This type of approach, which uses the environment to facilitate the grasp, is named Environmental Constraint Exploitation (ECE), and has been shown to improve the robustness of grasps while reducing the planning effort. The paper proposes two strategies, namely Continuous Slide and Grasp and Pivot and Re-Grasp, that are designed to deal with different objects. In the first strategy, the hand is positioned over the object and assumed to stick to it during the sliding until the edge, where the fingers wrap around the object and pick it up. In the second strategy, instead, the sliding motion is performed using pivoting, and thus the object is allowed to rotate with respect to the hand that drags it toward the edge. Then, as soon as the object reaches the desired position, the hand detaches from the object and moves to grasp the object from the side. In both strategies, the hand positioning for grasping the object is implemented using a recently proposed functional model for soft hands, the closure signature, whereas the sliding motion on the table is executed by using a hybrid force-velocity controller. We conducted 320 grasping trials with 16 different objects using a soft hand attached to a collaborative robot arm. Experiments showed that the Continuous Slide and Grasp is more suitable for small objects (e.g., a credit card), whereas the Pivot and Re-Grasp performs better with larger objects (e.g., a big book). The gathered data were used to train a classifier that selects the most suitable strategy to use, according to the object size and weight. Implementing ECE strategies with soft hands is a first step toward their use in real-world scenarios, where the environment should be seen more as a help than as a hindrance.

Human-Robot Team Interaction Through Wearable Haptics for Cooperative Manipulation

The interaction of robot teams and single human in teleoperation scenarios is beneficial in cooperative tasks, for example, the manipulation of heavy and large objects in remote or dangerous environments. The main control challenge of the interaction is its asymmetry, arising because robot teams have a relatively high number of controllable degrees of freedom compared to the human operator. Therefore, we propose a control scheme that establishes the interaction on spaces of reduced dimensionality taking into account the low number of human command and feedback signals imposed by haptic devices. We evaluate the suitability of wearable haptic fingertip devices for multi-contact teleoperation in a user study. The results show that the proposed control approach is appropriate for human-robot team interaction and that the wearable haptic fingertip devices provide suitable assistance in cooperative manipulation tasks.

Can Wearable Haptic Devices Foster the Embodiment of Virtual Limbs?

Increasing presence is one of the primary goals of virtual reality research. A crucial aspect is that users are capable of distinguishing their self from the external virtual world. The hypothesis we investigate is that wearable haptics play an important role in the body experience and could thereby contribute to the immersion of the user in the virtual environment. A within-subject study (n=32) comparing the embodiment of a virtual hand with different implementations of haptic feedback (force feedback, vibrotactile feedback, and no haptic feedback) is presented. Participants wore a glove with haptic feedback devices at thumb and index finger. They were asked to put virtual cubes on a moving virtual target. Touching a virtual object caused vibrotactile-feedback, force-feedback or no feedback depending on the condition. These conditions were provided both synchronously and asynchronously. Embodiment was assessed quantitatively with the proprioceptive drift and subjectively via a questionnaire. Results show that haptic feedback significantly improves the subjective embodiment of a virtual hand and that force feedback leads to stronger responses to certain subscales of subjective embodiment. These outcomes are useful guidelines for wearable haptic designer and represent a basis for further research concerning human body experience, in reality, and in virtual environments.

Replicating Human Hand Synergies Onto Robotic Hands: A Review on Software and Hardware Strategies

This review reports the principal solutions proposed in the literature to reduce the complexity of the control and of the design of robotic hands taking inspiration from the organization of the human brain. Several studies in neuroscience concerning the sensorimotor organization of the human hand proved that, despite the complexity of the hand, a few parameters can describe most of the variance in the patterns of configurations and movements. In other words, humans exploit a reduced set of parameters, known in the literature as synergies, to control their hands. In robotics, this dimensionality reduction can be achieved by coupling some of the degrees of freedom (DoFs) of the robotic hand, that results in a reduction of the needed inputs. Such coupling can be obtained at the software level, exploiting mapping algorithm to reproduce human hand organization, and at the hardware level, through either rigid or compliant physical couplings between the joints of the robotic hand. This paper reviews the main solutions proposed for both the approaches.

Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications

Although Augmented Reality (AR) has been around for almost five decades, only recently we have witnessed AR systems and applications entering in our everyday life. Representative examples of this technological revolution are the smartphone games "Pokémon GO" and "Ingress" or the Google Translate real-time sign interpretation app. Even if AR applications are already quite compelling and widespread, users are still not able to physically interact with the computer-generated reality. In this respect, wearable haptics can provide the compelling illusion of touching the superimposed virtual objects without constraining the motion or the workspace of the user. In this paper, we present the experimental evaluation of two wearable haptic interfaces for the fingers in three AR scenarios, enrolling 38 participants. In the first experiment, subjects were requested to write on a virtual board using a real chalk. The haptic devices provided the interaction forces between the chalk and the board. In the second experiment, subjects were asked to pick and place virtual and real objects. The haptic devices provided the interaction forces due to the weight of the virtual objects. In the third experiment, subjects were asked to balance a virtual sphere on a real cardboard. The haptic devices provided the interaction forces due to the weight of the virtual sphere rolling on the cardboard. Providing haptic feedback through the considered wearable device significantly improved the performance of all the considered tasks. Moreover, subjects significantly preferred conditions providing wearable haptic feedback.

Toward wearable supernumerary robotic fingers to compensate missing grasping abilities in hemiparetic upper limb

This paper presents the design, analysis, fabrication, experimental characterization, and evaluation of two prototypes of robotic extra fingers that can be used as grasp compensatory devices for a hemiparetic upper limb. The devices are the results of experimental sessions with chronic stroke patients and consultations with clinical experts. Both devices share a common principle of work, which consists in opposing the device to the paretic hand or wrist so to restrain the motion of an object. They can be used by chronic stroke patients to compensate for grasping in several activities of daily living (ADLs) with a particular focus on bimanual tasks. The robotic extra fingers are designed to be extremely portable and wearable. They can be wrapped as bracelets when not being used, to further reduce the encumbrance. Both devices are intrinsically compliant and driven by a single actuator through a tendon system. The motion of the robotic devices can be controlled using an electromyography-based interface embedded in a cap. The interface allows the user to control the device motion by contracting the frontalis muscle. The performance characteristics of the devices have been measured experimentally and the shape adaptability has been confirmed by grasping various objects with different shapes. We tested the devices through qualitative experiments based on ADLs involving five chronic stroke patients. The prototypes successfully enabled the patients to complete various bimanual tasks. Results show that the proposed robotic devices improve the autonomy of patients in ADLs and allow them to complete tasks that were previously impossible to perform.

A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.

A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.

An EMG Interface for the Control of Motion and Compliance of a Supernumerary Robotic Finger

In this paper, we propose a novel electromyographic (EMG) control interface to control motion and joints compliance of a supernumerary robotic finger. The supernumerary robotic fingers are a recently introduced class of wearable robotics that provides users additional robotic limbs in order to compensate or augment the existing abilities of natural limbs without substituting them. Since supernumerary robotic fingers are supposed to closely interact and perform actions in synergy with the human limbs, the control principles of extra finger should have similar behavior as human’s ones including the ability of regulating the compliance. So that, it is important to propose a control interface and to consider the actuators and sensing capabilities of the robotic extra finger compatible to implement stiffness regulation control techniques. We propose EMG interface and a control approach to regulate the compliance of the device through servo actuators. In particular, we use a commercial EMG armband for gesture recognition to be associated with the motion control of the robotic device and surface one channel EMG electrodes interface to regulate the compliance of the robotic device. We also present an updated version of a robotic extra finger where the adduction/abduction motion is realized through ball bearing and spur gears mechanism. We have validated the proposed interface with two sets of experiments related to compensation and augmentation. In the first set of experiments, different bimanual tasks have been performed with the help of the robotic device and simulating a paretic hand since this novel wearable system can be used to compensate the missing grasping abilities in chronic stroke patients. In the second set, the robotic extra finger is used to enlarge the workspace and manipulation capability of healthy hands. In both sets, the same EMG control interface has been used. The obtained results demonstrate that the proposed control interface is intuitive and can successfully be used, not only to control the motion of a supernumerary robotic finger but also to regulate its compliance. The proposed approach can be exploited also for the control of different wearable devices that has to actively cooperate with the human limbs.

Compensating Hand Function in Chronic Stroke Patients Through the Robotic Sixth Finger

A novel solution to compensate hand grasping abilities is proposed for chronic stroke patients. The goal is to provide the patients with a wearable robotic extra-finger that can be worn on the paretic forearm by means of an elastic band. The proposed prototype, the Robotic Sixth Finger, is a modular articulated device that can adapt its structure to the grasped object shape. The extra-finger and the paretic hand act like the two parts of a gripper cooperatively holding an object. We evaluated the feasibility of the approach with four chronic stroke patients performing a qualitative test, the Frenchay Arm Test. In this proof of concept study, the use of the Robotic Sixth Finger has increased the total score of the patients by two points in a five points scale. The subjects were able to perform the two grasping tasks included in the test that were not possible without the robotic extra-finger. Adding a robotic opposing finger is a very promising approach that can significantly improve the functional compensation of the chronic stroke patient during everyday life activities.

Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands

The term 'synergy' - from the Greek synergia - means 'working together'. The concept of multiple elements working together towards a common goal has been extensively used in neuroscience to develop theoretical frameworks, experimental approaches, and analytical techniques to understand neural control of movement, and for applications for neuro-rehabilitation. In the past decade, roboticists have successfully applied the framework of synergies to create novel design and control concepts for artificial hands, i.e., robotic hands and prostheses. At the same time, robotic research on the sensorimotor integration underlying the control and sensing of artificial hands has inspired new research approaches in neuroscience, and has provided useful instruments for novel experiments. The ambitious goal of integrating expertise and research approaches in robotics and neuroscience to study the properties and applications of the concept of synergies is generating a number of multidisciplinary cooperative projects, among which the recently finished 4-year European project "The Hand Embodied" (THE). This paper reviews the main insights provided by this framework. Specifically, we provide an overview of neuroscientific bases of hand synergies and introduce how robotics has leveraged the insights from neuroscience for innovative design in hardware and controllers for biomedical engineering applications, including myoelectric hand prostheses, devices for haptics research, and wearable sensing of human hand kinematics. The review also emphasizes how this multidisciplinary collaboration has generated new ways to conceptualize a synergy-based approach for robotics, and provides guidelines and principles for analyzing human behavior and synthesizing artificial robotic systems based on a theory of synergies.