Haptic Feedback, Performance and Arousal: A Comparison Study in an Immersive VR Motor Skill Training Task

This paper investigates the relationship between fine motor skill training in VR, haptic feedback, and physiological arousal. To do so, we present the design and development of a motor skill task (buzzwire), along with a custom vibrotactile feedback attachment for the Geomagic Touch haptic device. A controlled experiment following a between-subjects design was conducted with 73 participants, studying the role of three feedback conditions - visual/kinesthetic, visual/vibrotactile and visual only - on the learning and performance of the considered task and the arousal levels of the participants. Results indicate that performance improved in all three feedback conditions after the considered training session. However, participants reported no change in self-efficacy and in terms of presence and task load (NASA-TLX). All three feedback conditions also showed similar arousal levels. Further analysis revealed that positive changes in performance were linked to higher arousal levels. These results suggest the potential of haptic feedback to affect arousal levels and encourage further research into using this relationship to improve motor skill training in VR.

A Wearable Haptic Device for the Hand with Interchangeable End-Effectors

This paper presents a 4-degrees-of-freedom (4-DoF) hand wearable haptic device for Virtual Reality (VR). It is designed to support different end-effectors, that can be easily exchanged so as to provide a wide range of haptic sensations. The device is composed of a static upper body, secured to the back of the hand, and the (changeable) end-effector, placed in contact with the palm. The two parts of the device are connected by two articulated arms, actuated by four servo motors housed on the upper body and along the arms. The paper summarizes the design and kinematics of the wearable haptic device and presents a position control scheme able to actuate a broad range of end-effectors. As a proof of concept, we present and evaluate three representative end-effectors during interactions in VR, rendering the sensation of interacting (E1) with rigid slanted surfaces and sharp edges having different orientations, (E2) with curved surfaces having different curvatures, and (E3) with soft surfaces having different stiffness characteristics. A few additional end-effector designs are discussed. A human-subjects evaluation in immersive VR shows the broad applicability of the device, able to render rich interactions with a diverse set of virtual objects.

PREDICTOR: A Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation

Safety and ergonomics of Physical Human-Robot Collaboration (PHRC) are crucial to make human-robot collaborative systems trustworthy and make a significant impact in real-world applications. One big obstacle to the development of relevant research is the lack of a general platform for evaluating the safety and ergonomics of proposed PHRC systems. This paper aims to create a Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation (PREDICTOR). PREDICTOR consists of a dual-arm robot system and a VR headset as its hardware and contains physical simulation, haptic rendering and visual rendering modules as its software. The dual-arm robot system is used as an integrated admittance-type haptic device, which senses the force/torque applied by a human operator as an input to drive the simulation of a PHRC system and constrains the handles' motion to match their virtual counterparts in the simulation. The motion of the PHRC system in the simulation is fed back to the operator through the VR headset. PREDICTOR combines haptics and VR to emulate PHRC tasks in a safe environment since the interactive forces are monitored to avoid any risky events. PREDICTOR also brings flexibility as different PHRC tasks can be easily set up by changing the PHRC system model and the robot controller in the simulation. The effectiveness and performance of PREDICTOR were evaluated by experiments.

Investigating the effectiveness of immersive VR skill training and its link to physiological arousal

This paper details the motivations, design, and analysis of a study using a fine motor skill training task in both VR and physical conditions. The objective of this between-subjects study was to (a) investigate the effectiveness of immersive virtual reality for training participants in the 'buzz-wire' fine motor skill task compared to physical training and (b) investigate the link between participants' arousal with their improvements in task performance. Physiological arousal levels in the form of  electro-dermal activity (EDA) and ECG (Electrocardiogram) data were collected from 87 participants, randomly distributed across the two conditions. Results indicated that VR training is as good as, or even slightly better than, training in physical training in improving task performance. Moreover, the participants in the VR condition reported an increase in self-efficacy and immersion, while marginally significant differences were observed in the presence and the temporal demand (retrieved from NASA-TLX measurements). Participants in the VR condition showed on average less arousal than those in the physical condition. Though correlation analyses between performance metrics and arousal levels did not depict any statistically significant results, a closer examination of EDA values revealed that participants with lower arousal levels during training, across conditions, demonstrated better improvements in performance than those with higher arousal. These findings demonstrate the effectiveness of VR in training and the potential of using arousal and training performance data for designing adaptive VR training systems. This paper also discusses implications for researchers who consider using biosensors and VR for motor skill experiments.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-022-00699-3.

Design of Personalized Wearable Haptic Interfaces to Account for Fingertip Size and shape

The size and shape of fingertips vary significantly across humans, making it challenging to design wearable fingertip interfaces suitable for everyone. Although deemed important, this issue has often been neglected due to the difficulty of customizing devices for each different user. This article presents an innovative approach for automatically adapting the hardware design of a wearable haptic interface for a given user. We consider a three-DoF fingertip cutaneous device, composed of a static body and a mobile platform linked by three articulated legs. The mobile platform is capable of making and breaking contact with the finger pulp and re-angle to replicate contacts with arbitrarily-oriented surfaces. We analyze the performance of this device as a function of its main geometrical dimensions. Then, starting from the user's fingertip characteristics, we define a numerical procedure that best adapts the dimension of the device to: (i) maximize the range of renderable haptic stimuli; (ii) avoid unwanted contacts between the device and the skin; (iii) avoid singular configurations; and (iv) minimize the device encumbrance and weight. Together with the mechanical analysis and evaluation of the adapted design, we present a MATLAB script that calculates the device dimensions customized for a target fingertip as well as an online CAD utility for generating a ready-to-print STL file of the personalized design.

Effectiveness of Virtual Versus Physical Training: The Case of Assembly Tasks, Trainer's Verbal Assistance, and Task Complexity

Virtual immersive training (VIT) systems based on gamification of tasks are increasingly employed to train assembly workers. In this article, we present a study that compares the effectiveness of virtual and physical training for teaching a bimanual assembly task and in a novel approach, we introduce task complexity (T$_{\mathrm{CXB}}$ CXB ) as an indicator of assembly errors during final assembly. In a between-subjects experiment, 100 participants were trained to assemble a 3-D cube in one of the four conditions (physical, virtual and with trainer's verbal assistance or not). The results demonstrate that the best-performing conditions, both in terms of successful assemblies and time performance, are the ones that the physical objects are included in the training, whereas no significant difference is found when the trainer's verbal assistance is present or absent during training. Additionally, we address the validity of a practical T$_{\mathrm{CXB}}$ CXB list as a tool for supporting the design of VIT systems.

Human-Robot Team Interaction Through Wearable Haptics for Cooperative Manipulation

The interaction of robot teams and single human in teleoperation scenarios is beneficial in cooperative tasks, for example, the manipulation of heavy and large objects in remote or dangerous environments. The main control challenge of the interaction is its asymmetry, arising because robot teams have a relatively high number of controllable degrees of freedom compared to the human operator. Therefore, we propose a control scheme that establishes the interaction on spaces of reduced dimensionality taking into account the low number of human command and feedback signals imposed by haptic devices. We evaluate the suitability of wearable haptic fingertip devices for multi-contact teleoperation in a user study. The results show that the proposed control approach is appropriate for human-robot team interaction and that the wearable haptic fingertip devices provide suitable assistance in cooperative manipulation tasks.

Linear Integration of Tactile and Non-tactile Inputs Mediates Estimation of Fingertip Relative Position

While skin, joints and muscles receptors alone provide lower level information about individual variables (e.g., exerted limb force and limb displacement), the distance between limb endpoints (i.e., relative position) has to be extracted from high level integration of somatosensory and motor signals. In particular, estimation of fingertip relative position likely involves more complex sensorimotor transformations than those underlying hand or arm position sense: the brain has to estimate where each fingertip is relative to the hand and where fingertips are relative to each other. It has been demonstrated that during grasping, feedback of digit position drives rapid adjustments of fingers force control. However, it has been shown that estimation of fingertips' relative position can be biased by digit forces. These findings raise the question of how the brain combines concurrent tactile (i.e., cutaneous mechanoreceptors afferents induced by skin pressure and stretch) and non-tactile (i.e., both descending motor command and joint/muscle receptors signals associated to muscle contraction) digit force-related inputs for fingertip distance estimation. Here we addressed this question by quantifying the contribution of tactile and non-tactile force-related inputs for the estimation of fingertip relative position. We asked subjects to match fingertip vertical distance relying only on either tactile or non-tactile inputs from the thumb and index fingertip, and compared their performance with the condition where both types of inputs were combined. We found that (a) the bias in the estimation of fingertip distance persisted when tactile inputs and non-tactile force-related signals were presented in isolation; (b) tactile signals contributed the most to the estimation of fingertip distance; (c) linear summation of the matching errors relying only on either tactile or non-tactile inputs was comparable to the matching error when both inputs were simultaneously available. These findings reveal a greater role of tactile signals for sensing fingertip distance and suggest a linear integration mechanism with non-tactile inputs for the estimation of fingertip relative position.

A Three Revolute-Revolute-Spherical Wearable Fingertip Cutaneous Device for Stiffness Rendering

We present a novel three Revolute-Revolute-Spherical (3RRS) wearable fingertip device for the rendering of stiffness information. It is composed of a static upper body and a mobile end-effector. The upper body is located on the nail side of the finger, supporting three small servo motors, and the mobile end-effector is in contact with the finger pulp. The two parts are connected by three articulated legs, actuated by the motors. The end-effector can move toward the user's fingertip and rotate it to simulate contacts with arbitrarily-oriented surfaces. Moreover, a vibrotactile motor placed below the end-effector conveys vibrations to the fingertip. The proposed device weights 25 g for 35 x 50 x 48 mm dimensions. To test the effectiveness of our wearable haptic device and its level of wearability, we carried out two experiments, enrolling 30 human subjects in total. The first experiment tested the capability of our device in differentiating stiffness information, while the second one focused on evaluating its applicability in an immersive virtual reality scenario. Results showed the effectiveness of the proposed wearable solution, with a JND for stiffness of 208.5   17.2 N/m. Moreover, all subjects preferred the virtual interaction experience when provided with wearable cutaneous feedback, even if results also showed that subjects found our device still a bit difficult to use.

Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications

Although Augmented Reality (AR) has been around for almost five decades, only recently we have witnessed AR systems and applications entering in our everyday life. Representative examples of this technological revolution are the smartphone games "Pokémon GO" and "Ingress" or the Google Translate real-time sign interpretation app. Even if AR applications are already quite compelling and widespread, users are still not able to physically interact with the computer-generated reality. In this respect, wearable haptics can provide the compelling illusion of touching the superimposed virtual objects without constraining the motion or the workspace of the user. In this paper, we present the experimental evaluation of two wearable haptic interfaces for the fingers in three AR scenarios, enrolling 38 participants. In the first experiment, subjects were requested to write on a virtual board using a real chalk. The haptic devices provided the interaction forces between the chalk and the board. In the second experiment, subjects were asked to pick and place virtual and real objects. The haptic devices provided the interaction forces due to the weight of the virtual objects. In the third experiment, subjects were asked to balance a virtual sphere on a real cardboard. The haptic devices provided the interaction forces due to the weight of the virtual sphere rolling on the cardboard. Providing haptic feedback through the considered wearable device significantly improved the performance of all the considered tasks. Moreover, subjects significantly preferred conditions providing wearable haptic feedback.

Optimization-Based Wearable Tactile Rendering

Novel wearable tactile interfaces offer the possibility to simulate tactile interactions with virtual environments directly on our skin. But, unlike kinesthetic interfaces, for which haptic rendering is a well explored problem, they pose new questions about the formulation of the rendering problem. In this work, we propose a formulation of tactile rendering as an optimization problem, which is general for a large family of tactile interfaces. Based on an accurate simulation of contact between a finger model and the virtual environment, we pose tactile rendering as the optimization of the device configuration, such that the contact surface between the device and the actual finger matches as close as possible the contact surface in the virtual environment. We describe the optimization formulation in general terms, and we also demonstrate its implementation on a thimble-like wearable device. We validate the tactile rendering formulation by analyzing its force error, and we show that it outperforms other approaches.

Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems

Cutaneous haptic feedback can be used to enhance the performance of robotic teleoperation systems while guaranteeing their safety. Delivering ungrounded cutaneous cues to the human operator conveys in fact information about the forces exerted at the slave side and does not affect the stability of the control loop. In this work we analyze the feasibility, effectiveness, and implications of providing solely cutaneous feedback in robotic teleoperation. We carried out two peg-in-hole experiments, both in a virtual environment and in a real (teleoperated) environment. Two novel 3-degree-of-freedom fingertip cutaneous displays deliver a suitable amount of cutaneous feedback at the thumb and index fingers. Results assessed the feasibility and effectiveness of the proposed approach. Cutaneous feedback was outperformed by full haptic feedback provided by grounded haptic interfaces, but it outperformed conditions providing no force feedback at all. Moreover, cutaneous feedback always kept the system stable, even in the presence of destabilizing factors such as communication delays and hard contacts.

Towards wearability in fingertip haptics: a 3-DoF wearable device for cutaneous force feedback

Wearability will significantly increase the use of haptics in everyday life, as has already happened for audio and video technologies. The literature on wearable haptics is mainly focused on vibrotactile stimulation, and only recently, wearable devices conveying richer stimuli, like force vectors, have been proposed. This paper introduces design guidelines for wearable haptics and presents a novel 3-DoF wearable haptic interface able to apply force vectors directly to the fingertip. It consists of two platforms: a static one, placed on the back of the finger, and a mobile one, responsible for applying forces at the finger pad. The structure of the device resembles that of parallel robots, where the fingertip is placed in between the static and the moving platforms. This work presents the design of the wearable display, along with the quasi-static modeling of the relationship between the applied forces and the platform's orientation and displacement. The device can exert up to 1.5 N, with a maximum platform inclination of 30 degree. To validate the device and verify its effectiveness, a curvature discrimination experiment was carried out: employing the wearable device together with a popular haptic interface improved the performance with respect of employing the haptic interface alone.