Pseudo-Haptic Feedback in Teleoperation

Engineering Liquid-Vapor Phase Transition for Refreshable Haptic Interfaces

Haptics as a communication medium has been increasingly emphasized across various disciplines. Recent efforts have focused on developing various haptic stimulation technologies; however, most of them suffer from critical drawbacks stemming from their bulk, complexity, large power input, or high cost. Here, we describe a strategy to design portable and affordable refreshable haptic interfaces composed of an array of individually addressable and controllable liquid pouch motor-based haptic units embedded in either rigid or flexible substrates for different application contexts. The pouch motor filled with low boiling fluid, under a controlled manner, expands or contracts by Joule heating or cooling, enabling the haptic pin in contact to be protruded or retracted. Programming the actuation sequence of an array of haptic units enables the haptic interface to apply different stimuli to the skin to convey corresponding information. We finally demonstrate the applications to portable rigid braille displays and flexible epidermal VR devices. This study opens the avenue to the design of ubiquitous refreshable haptic interfaces that is portable, affordable, scalable, and uninjurious.

Phase Optimization for Multipoint Haptic Feedback Based on Ultrasound Array

Ultrasound-based haptic feedback is a potential technology for human–computer interaction (HCI) with the advantages of a low cost, low power consumption and a controlled force. In this paper, phase optimization for multipoint haptic feedback based on an ultrasound array was investigated, and the corresponding experimental verification is provided. A mathematical model of acoustic pressure was established for the ultrasound array, and then a phase-optimization model for an ultrasound transducer was constructed. We propose a pseudo-inverse (PINV) algorithm to accurately determine the phase contribution of each transducer in the ultrasound array. By controlling the phase difference of the ultrasound array, the multipoint focusing forces were formed, leading to various shapes such as geometries and letters, which can be visualized. Because the unconstrained PINV solution results in unequal amplitudes for each transducer, a weighted amplitude iterative optimization was deployed to further optimize the phase solution, by which the uniform amplitude distributions of each transducer were obtained. For the purpose of experimental verification, a platform of ultrasound haptic feedback consisting of a Field Programmable Gate Array (FPGA), an electrical circuit and an ultrasound transducer array was prototyped. The haptic performances of a single point, multiple points and dynamic trajectory were verified by controlling the ultrasound force exerted on the liquid surface. The experimental results demonstrate that the proposed phase-optimization model and theoretical results are effective and feasible, and the acoustic pressure distribution is consistent with the simulation results.

Survey of Pseudo-Haptics: Haptic Feedback Design and Application Proposals

In the last two decades, the design of pseudo-haptics as a haptic presentation method that does not require a mechanical feedback device has been proposed in various research papers. Moreover, applications using pseudo-haptics have been proposed and evaluated in various contexts. However, the findings from these studies have not yet been comprehensively organized in a survey paper in the recent times. In this article, findings from a series of individual prior studies were summarized from the design through to the application proposals. First, we summarize visual stimuli designs based on the target haptic object properties to induce pseudo-haptics. Second, we summarize two special issues when designing pseudo-haptics; (1) workaround design for the visualized mismatch of visual stimuli and user input; and (2) the combination design of pseudo-haptics and physical stimuli. Third, application proposals that use pseudo-haptics for training, assistance, and entertainment are presented. This survey paper would help not only researchers in academia but also application developers who intend to use pseudo-haptics as a haptic presentation method.

Vision-Based Suture Tensile Force Estimation in Robotic Surgery

Compared to laparoscopy, robotics-assisted minimally invasive surgery has the problem of an absence of force feedback, which is important to prevent a breakage of the suture. To overcome this problem, surgeons infer the suture force from their proprioception and 2D image by comparing them to the training experience. Based on this idea, a deep-learning-based method using a single image and robot position to estimate the tensile force of the sutures without a force sensor is proposed. A neural network structure with a modified Inception Resnet-V2 and Long Short Term Memory (LSTM) networks is used to estimate the suture pulling force. The feasibility of proposed network is verified using the generated DB, recording the interaction under the condition of two different artificial skins and two different situations (in vivo and in vitro) at 13 viewing angles of the images by changing the tool positions collected from the master-slave robotic system. From the evaluation conducted to show the feasibility of the interaction force estimation, the proposed learning models successfully estimated the tensile force at 10 unseen viewing angles during training.

Impact of haptic feedback on applied intracorporeal forces using a novel surgical robotic system-a randomized cross-over study with novices in an experimental setup

Background

Most currently used surgical robots have no force feedback; the next generation displays forces visually. A novel single-port robotic surgical system called FLEXMIN has been developed. Through an outer diameter of 38 mm, two instruments are teleoperated from a surgeon’s control console including true haptic force feedback. One additional channel incorporates a telescope, another is free for special instrument functions.

Methods

This randomized cross-over study analyzed the effect of haptic feedback on the application of intracorporeal forces. In a standardized experiment setup, the subjects had to draw circles with the surgical robot as gently as possible. The applied forces, the required time spans, and predefined error rates were measured.

Results

Without haptic feedback, the maximum forces (median/IQR) were 6.43 N/2.96 N. With haptic feedback, the maximum forces were lower (3.57 N/1.94 N, p < 0.001). Also, the arithmetic means of the force progression (p < 0.001) and their standard deviations (p < 0.001) were lower. Not significant were the shorter durations and lower error rates. No sequence effect of force or duration was detected. No characteristic learning or fatigue curve was observed.

Conclusions

In the experiment setup, the true haptic force feedback can reduce the applied intracorporeal robotic force to one-half when considering the aspects maximum, means, and standard deviation. Other test tasks are needed to validate the influence of force feedback on surgical efficiency and safety.

Haptics in Teleoperated Medical Interventions: Force Measurement, Haptic Interfaces and Their Influence on User's Performance

OBJECTIVES

Haptics in teleoperated medical interventions enables measurement and transfer of force information to the operator during robot-environment interaction. This paper provides an overview of the current research in this domain and guidelines for future investigations.

METHODS

We review current technologies in force measurement and haptic devices as well as their experimental evaluation and influence on user's performance.

RESULTS

Force sensing is moving away from the conventional proximal measurement methods to distal sensing and contact-less methods. Wearable devices that deliver haptic feedback on different body parts are increasingly playing an important role. Performance and accuracy improvement are the widely reported benefits of haptic feedback, while there is a debate on its effect on task completion time and exerted force.

CONCLUSION

With the surge of new ideas, there is a need for better and more systematic validation of the new sensing and feedback technology, through better user studies and novel methods like validated benchmarks and new taxonomies.

SIGNIFICANCE

This review investigates haptics from sensing to interfaces within the context of user's performance and the validation procedures to highlight salient advances. It provides guidelines to future developments and highlights the shortcomings in the field.

Magnetic Levitation Haptic Augmentation for Virtual Tissue Stiffness Perception

Haptic-based tissue stiffness perception is essential for palpation training system, which can provide the surgeon haptic cues for improving the diagnostic abilities. However, current haptic devices, such as Geomagic Touch, fail to provide immersive and natural haptic interaction in virtual surgery due to the inherent mechanical friction, inertia, limited workspace and flawed haptic feedback. To tackle this issue, we design a novel magnetic levitation haptic device based on electromagnetic principles to augment the tissue stiffness perception in virtual environment. Users can naturally interact with the virtual tissue by tracking the motion of magnetic stylus using stereoscopic vision so that they can accurately sense the stiffness by the magnetic stylus, which moves in the magnetic field generated by our device. We propose the idea that the effective magnetic field (EMF) is closely related to the coil attitude for the first time. To fully harness the magnetic field and flexibly generate the specific magnetic field for obtaining required haptic perception, we adopt probability clouds to describe the requirement of interactive applications and put forward an algorithm to calculate the best coil attitude. Moreover, we design a control interface circuit and present a self-adaptive fuzzy proportion integration differentiation (PID) algorithm to precisely control the coil current. We evaluate our haptic device via a series of quantitative experiments which show the high consistency of the experimental and simulated magnetic flux density, the high accuracy (0.28 mm) of real-time 3D positioning and tracking of the magnetic stylus, the low power consumption of the adjustable coil configuration, and the tissue stiffness perception accuracy improvement by 2.38 percent with the self-adaptive fuzzy PID algorithm. We conduct a user study with 22 participants, and the results suggest most of the users can clearly and immersively perceive different tissue stiffness and easily detect the tissue abnormality. Experimental results demonstrate that our magnetic levitation haptic device can provide accurate tissue stiffness perception augmentation with natural and immersive haptic interaction.

Influence of surgical gloves on haptic perception thresholds

BACKGROUND

Impairment of haptic perception by surgical gloves could reduce requirements on haptic systems for surgery. While grip forces and manipulation capabilities were not impaired in previous studies, no data is available for perception thresholds.

METHODS

Absolute and differential thresholds (20 dB above threshold) of 24 subjects were measured for frequencies of 25 and 250 Hz with a Ψ-method. Effects of wearing a surgical glove, moisture on the contact surface and subject's experience with gloves were incorporated in a full-factorial experimental design.

RESULTS

Absolute thresholds of 12.8 dB and -29.6 dB (means for 25 and 250 Hz, respectively) and differential thresholds of -12.6 dB and -9.5 dB agree with previous studies. A relevant effect of the frequency on absolute thresholds was found. Comparisons of glove- and no-glove-conditions did not reveal a significant mean difference.

CONCLUSIONS

Wearing a single surgical glove does not affect absolute and differential haptic perception thresholds.