A survey on bimanual haptic interaction

Exploring Bimanual Haptic Feedback for Spatial Search in Virtual Reality

Spatial search tasks are common and crucial in many Virtual Reality (VR) applications. Traditional methods to enhance the performance of spatial search often employ sensory cues such as visual, auditory, or haptic feedback. However, the design and use of bimanual haptic feedback with two VR controllers for spatial search in VR remains largely unexplored. In this work, we explored bimanual haptic feedback with various combinations of haptic properties, where four types of bimanual haptic feedback were designed, for spatial search tasks in VR. Two experiments were designed to evaluate the effectiveness of bimanual haptic feedback on spatial direction guidance and search in VR. The results from the first experiment reveal that our proposed bimanual haptic schemes significantly enhanced the recognition of spatial directions in terms of accuracy and speed compared to spatial audio feedback. The second experiment's findings suggest that the performance of bimanual haptic feedback was comparable to or even better than the visual arrow, especially in reducing the angle of head movement and enhancing searching targets behind the participants, which was supported by subjective feedback as well. Based on these findings, we have derived a set of design recommendations for spatial search using bimanual haptic feedback in VR.

Blind people can actively manipulate virtual objects with a novel tactile device

Frequently in rehabilitation, visually impaired persons are passive agents of exercises with fixed environmental constraints. In fact, a printed tactile map, i.e. a particular picture with a specific spatial arrangement, can usually not be edited. Interaction with map content, instead, facilitates the learning of spatial skills because it exploits mental imagery, manipulation and strategic planning simultaneously. However, it has rarely been applied to maps, mainly because of technological limitations. This study aims to understand if visually impaired people can autonomously build objects that are completely virtual. Specifically, we investigated if a group of twelve blind persons, with a wide age range, could exploit mental imagery to interact with virtual content and actively manipulate it by means of a haptic device. The device is mouse-shaped and designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. Spatial information can be mentally constructed by integrating local tactile cues, given by the device, with global proprioceptive cues, given by hand and arm motion. The experiment consisted of a bi-manual task, in which one hand explored some basic virtual objects and the other hand acted on a keyboard to change the position of one object in real-time. The goal was to merge basic objects into more complex objects, like a puzzle. The experiment spanned different resolutions of the tactile information. We measured task accuracy, efficiency, usability and execution time. The average accuracy in solving the puzzle was 90.5%. Importantly, accuracy was linearly predicted by efficiency, measured as the number of moves needed to solve the task. Subjective parameters linked to usability and spatial resolutions did not predict accuracy; gender modulated the execution time, with men being faster than women. Overall, we show that building purely virtual tactile objects is possible in absence of vision and that the process is measurable and achievable in partial autonomy. Introducing virtual tactile graphics in rehabilitation protocols could facilitate the stimulation of mental imagery, a basic element for the ability to orient in space. The behavioural variable introduced in the current study can be calculated after each trial and therefore could be used to automatically measure and tailor protocols to specific user needs. In perspective, our experimental setup can inspire remote rehabilitation scenarios for visually impaired people.

The Impact of Stiffness in Bimanual Versus Dyadic Interactions Requiring Force Exchange

During daily activities, humans routinely manipulate objects bimanually or with the help of a partner. This work explored how bimanual and dyadic coordination modes are impacted by the object's stiffness, which conditions inter-limb haptic communication. For this, we recruited 20 healthy participants who performed a virtual task inspired by object handling, where we looked at the initiation of force exchange and its continued maintenance while tracking. Our findings suggest that while individuals and dyads displayed different motor behaviours, which may stem from the dyad members' need to estimate their partner's actions, they exhibited similar tracking accuracy. For both coordination modes, increased stiffness resulted in better tracking accuracy and more correlated motions, but required a larger effort through increased average torque. These results suggest that stiffness may be a key consideration in applications such as rehabilitation, where bimanual or external physical assistance is often provided.

Differences in coordination between dominant and non-dominant hands in tele-operation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083449 DOI: 10.1109/embc40787.2023.10340192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

The intuitive tele-operation of industrial robot arms is necessary for the teaching of autonomous movement. We developed a novel interface, namely the iFeel Desktop Haptic Device, for operating robots intuitively. However, when a user uses two interfaces with two hands, there are differences in coordination between the dominant and non-dominant hands. In this paper, we investigated the differences in coordination between dominant and non-dominant hands using two devices for the development of cooperative control. In an experiment, when the participants manipulated the two interfaces to operate a virtual tracking system, we measured the tracking error in each degree of freedom (i.e., X, Y, Z, pitch, yaw, and roll directions). The results show that there were significant differences between the dominant and non-dominant hands for movement in the X, Y, Z, and pitch directions. We conclude that the operations that involve more body parts have a greater difference between dominant and non-dominant hands.

Cardiac X-ray image-based haptic guidance for robot-assisted coronary intervention: a feasibility study

PURPOSE

Effective and efficient haptic guidance is desirable for tele-operated robotic surgery because it has a potential to enhance surgeon's skills, especially in coronary interventions where surgeon loses both an eye-hand coordination and a direct sight to the organ. This paper proposes a novel haptic guidance procedure-both kinesthetic and cutaneous, which solely depends upon X-ray images, for tele-robotic system that assists an efficient navigation of the guidewire towards the target location during a coronary intervention.

METHODS

Proposed methodology requires cardiologists to draw virtual fixtures (VFs) on angiograms as a preoperative procedure. During an operation, these VFs direct the guidewire to the desired coronary vessel. For this, the position and orientation of guidewire tip are calculated with respect to VFs' anatomy, using image processing on the real-time 2D fluoroscopic images. The haptic feedbacks are then rendered on to the master device depending on the interaction with attractive and repulsive, guidance and forbidden region VFs.

RESULTS

A feasibility study in the laboratory environment is performed by using a webcam as an image acquisition device and a phantom-based coronary vessel model. The subsequent statistical analysis shows that, on an average, a decrease of approx. 37% in task completion time is observed with haptic feedback. Moreover, haptic guidance is found effective for most difficult branch, whereas there is a minimal significance of such haptics for the easiest branch.

CONCLUSIONS

The proposed haptic guidance procedure may assist cardiologists for an efficient and effective guidewire navigation during a surgical procedure. The cutaneous haptics (vibration feedback) is found more helpful in coronary interventions compared with kinesthetic haptics (force feedback).

A Lightweight Accessible Wearable Robotic Interface for Bimanual Haptic Manipulations

Wearable devices with bimanual force feedback enable natural and cooperative manipulations within an unrestricted space. Weight and cost have a great influence on the potential applications of a haptic device. This paper presents a wearable robotic interface with bimanual force feedback that has considerably reduced weight and cost. To make the reaction force less perceivable than the interaction force, a waist-worn scheme is adopted. The interface mainly consists of a belt, a fastening tape, two serial robotic arms, and two electronics units and batteries. The robotic arms located on both sides of the belt are capable of 3-DoF position tracking and force feedback for each hand. The whole interface is lightweight (only 2.4 kg) and accessible. Furthermore, it is also easy to wear and the operator can wear it only by putting the belt on the waist and fastening the tape, reducing his/her dependency on additional assistance. The interface is optimized to obtain desirable force output and a dexterous workspace without singularity. To evaluate its performance in bimanual cooperative manipulations, an experiment in the virtual environment was conducted. The experimental results showed the subjects had more efficient and stable cooperative manipulations with bimanual force feedback than without force feedback.

Salient Properties in Bimanual Haptic Volume Perception: Influence of Object Shape, Finger Pair, and Schizotypal Personality Traits

Bimanual haptic volume perception refers to somatosensory access to volume information through both hands, and the characteristics that influence this perception remain unclear. This article investigated the influence of target object shapes and finger pairs on bimanual haptic perception; in addition, associations of bimanual haptic impairment and schizotypal features in nonpsychotic individuals were investigated. Twenty blindfolded participants bimanually discriminated volume variations in regular solid objects under different shape (tetrahedron, cube, or sphere) and finger pair (high- or low-sensitivity pairs) conditions using a newly developed bimanual haptic volume presentation device. Discrimination thresholds were then associated with schizotypal traits using the Chinese version of the Schizotypal Personality Questionnaire. Target object shape and finger pairs significantly influenced bimanual haptic volume perception. Volume discrimination thresholds were significantly higher with the tetrahedron stimuli than the cubic or spherical stimuli in high-sensitivity pair conditions, but no significant differences among shapes were found in low-sensitivity pair conditions. Moreover, volume discrimination thresholds with high-sensitivity pairs were correlated with the paranoid score of the schizotypal personality questionnaire. The findings provide initial evidence toward understanding the nature of bimanual haptic volume perception, including the properties of objects, individuals, and object-individual interfaces.

Haptic shape discrimination and interhemispheric communication

In three experiments participants haptically discriminated object shape using unimanual (single hand explored two objects) and bimanual exploration (both hands were used, but each hand, left or right, explored a separate object). Such haptic exploration (one versus two hands) requires somatosensory processing in either only one or both cerebral hemispheres; previous studies related to the perception of shape/curvature found superior performance for unimanual exploration, indicating that shape comparison is more effective when only one hemisphere is utilized. The current results, obtained for naturally shaped solid objects (bell peppers, Capsicum annuum) and simple cylindrical surfaces demonstrate otherwise: bimanual haptic exploration can be as effective as unimanual exploration, showing that there is no necessary reduction in ability when haptic shape comparison requires interhemispheric communication. We found that while successive bimanual exploration produced high shape discriminability, the participants’ bimanual performance deteriorated for simultaneous shape comparisons. This outcome suggests that either interhemispheric interference or the need to attend to multiple objects simultaneously reduces shape discrimination ability. The current results also reveal a significant effect of age: older adults’ shape discrimination abilities are moderately reduced relative to younger adults, regardless of how objects are manipulated (left hand only, right hand only, or bimanual exploration).

The Role of Visual and Haptic Feedback During Dynamically Coupled Bimanual Manipulation

The mechanisms that underlie the control of bimanual actions in which the two hands act separately to manipulate different objects (uncoupled independent control) has been well studied. In contrast, much less is known about how the central nervous system controls bimanual actions that require the two hands act cooperatively to manipulate a single object (dynamically coupled control). Furthermore, there is scant research into the manual lateralization and role assignment in the processing of visual and haptic feedback during dynamically coupled bimanual tasks. In this experiment, we examined the role of the dominant and non-dominant hands during a dynamically coupled bimanual task in which visual and haptic feedback regarding object penetration were manipulated. Twelve subjects performed a bimanual grasp and reach task towards different target locations in the workspace by using two identical wrist robotic devices. Results showed haptic feedback is necessary for task completion, and that hand specialization plays a fundamental role in spatial and temporal coordination between the two limbs.

Non-Colocated Kinesthetic Display Limits Compliance Discrimination in the Absence of Terminal Force Cues

An important goal of haptic display is to make available the action/reaction relationships that define interactions between the body and the physical world. While in physical world interactions reaction cues invariably impinge on the same part of the body involved in action (reaction and action are colocated), a haptic interface is quite capable of rendering feedback to a separate body part than that used for producing exploratory actions (non-colocated action and reaction). This most commonly occurs with the use of vibrotactile display, in which a cutaneous cue has been substituted for a kinesthetic cue (a kind of sensory substitution). In this paper, we investigate whether non-colocated force and displacement cues degrade the perception of compliance. Using a custom non-colocated kinesthetic display in which one hand controls displacement and the other senses force, we ask participants to discriminate between two virtual springs with matched terminal force and adjustable non-linearity. An additional condition includes one hand controlling displacement while the other senses force encoded in a vibrotactile cue. Results show that when the terminal force cue is unavailable, and even when sensory substitution is not involved, non-colocated kinesthetic displays degrade compliance discrimination relative to colocated kinesthetic displays. Compliance discrimination is also degraded with vibrotactile display of force. These findings suggest that non-colocated kinesthetic displays and, likewise, cutaneous sensory substitution displays should be avoided when discrimination of compliance is necessary for task success.