Tactile display of softness on fingertip

A Lightweight and Affordable Wearable Haptic Controller for Robot-Assisted Microsurgery

In robot-assisted microsurgery (RAMS), surgeons often face the challenge of operating with minimal feedback, particularly lacking in haptic feedback. However, most traditional desktop haptic devices have restricted operational areas and limited dexterity. This report describes a novel, lightweight, and low-budget wearable haptic controller for teleoperated microsurgical robotic systems. We designed a wearable haptic interface entirely made using off-the-shelf material-PolyJet Photopolymer, fabricated using liquid and solid hybrid 3D co-printing technology. This interface was designed to resemble human soft tissues and can be wrapped around the fingertips, offering direct contact feedback to the operator. We also demonstrated that the device can be easily integrated with our motion tracking system for remote microsurgery. Two motion tracking methods, marker-based and marker-less, were compared in trajectory-tracking experiments at different depths to find the most effective motion tracking method for our RAMS system. The results indicate that within the 4 to 8 cm tracking range, the marker-based method achieved exceptional detection rates. Furthermore, the performance of three fusion algorithms was compared to establish the unscented Kalman filter as the most accurate and reliable. The effectiveness of the wearable haptic controller was evaluated through user studies focusing on the usefulness of haptic feedback. The results revealed that haptic feedback significantly enhances depth perception for operators during teleoperated RAMS.

SORI: A softness-rendering interface to unravel the nature of softness perception

Tactile perception of softness serves a critical role in the survival, well-being, and social interaction among various species, including humans. This perception informs activities from food selection in animals to medical palpation for disease detection in humans. Despite its fundamental importance, a comprehensive understanding of how softness is neurologically and cognitively processed remains elusive. Previous research has demonstrated that the somatosensory system leverages both cutaneous and kinesthetic cues for the sensation of softness. Factors such as contact area, depth, and force play a particularly critical role in sensations experienced at the fingertips. Yet, existing haptic technologies designed to explore this phenomenon are limited, as they often couple force and contact area, failing to provide a real-world experience of softness perception. Our research introduces the softness-rendering interface (SORI), a haptic softness display designed to bridge this knowledge gap. Unlike its predecessors, SORI has the unique ability to decouple contact area and force, thereby allowing for a quantitative representation of softness sensations at the fingertips. Furthermore, SORI incorporates individual physical fingertip properties and model-based softness cue estimation and mapping to provide a highly personalized experience. Utilizing this method, SORI quantitatively replicates the sensation of softness on stationary, dynamic, homogeneous, and heterogeneous surfaces. We demonstrate that SORI accurately renders the surfaces of both virtual and daily objects, thereby presenting opportunities across a range of fields, from teleoperation to medical technology. Finally, our proposed method and SORI will expedite psychological and neuroscience research to unlock the nature of softness perception.

Haptic Sensing and Feedback Techniques toward Virtual Reality

Haptic interactions between human and machines are essential for information acquisition and object manipulation. In virtual reality (VR) system, the haptic sensing device can gather information to construct virtual elements, while the haptic feedback part can transfer feedbacks to human with virtual tactile sensation. Therefore, exploring high-performance haptic sensing and feedback interface imparts closed-loop haptic interaction to VR system. This review summarizes state-of-the-art VR-related haptic sensing and feedback techniques based on the hardware parts. For the haptic sensor, we focus on mechanism scope (piezoresistive, capacitive, piezoelectric, and triboelectric) and introduce force sensor, gesture translation, and touch identification in the functional view. In terms of the haptic feedbacks, methodologies including mechanical, electrical, and elastic actuators are surveyed. In addition, the interactive application of virtual control, immersive entertainment, and medical rehabilitation is also summarized. The challenges of virtual haptic interactions are given including the accuracy, durability, and technical conflicts of the sensing devices, bottlenecks of various feedbacks, as well as the closed-loop interaction system. Besides, the prospects are outlined in artificial intelligence of things, wise information technology of medicine, and multimedia VR areas.

Soft Sensors and Actuators for Wearable Human-Machine Interfaces

Haptic human-machine interfaces (HHMIs) combine tactile sensation and haptic feedback to allow humans to interact closely with machines and robots, providing immersive experiences and convenient lifestyles. Significant progress has been made in developing wearable sensors that accurately detect physical and electrophysiological stimuli with improved softness, functionality, reliability, and selectivity. In addition, soft actuating systems have been developed to provide high-quality haptic feedback by precisely controlling force, displacement, frequency, and spatial resolution. In this Review, we discuss the latest technological advances of soft sensors and actuators for the demonstration of wearable HHMIs. We particularly focus on highlighting material and structural approaches that enable desired sensing and feedback properties necessary for effective wearable HHMIs. Furthermore, promising practical applications of current HHMI technology in various areas such as the metaverse, robotics, and user-interactive devices are discussed in detail. Finally, this Review further concludes by discussing the outlook for next-generation HHMI technology.

PopTouch: A Submillimeter Thick Dynamically Reconfigured Haptic Interface with Pressable Buttons

The interactions with touchscreens rely heavily on vision: The virtual buttons and virtual sliders on a touchscreen provide no mechanical sense of the object they seek to represent. This work presents PopTouch: a 500 µm thick flexible haptic display that creates pressable physical buttons on demand. PopTouch can be mounted directly on touchscreens or any other smooth surface, flat, or curved. The buttons of PopTouch are independently controlled hydraulically amplified electrostatic zipping taxels (tactile pixels) that generate 1.5 mm of out of plane displacement. When pressed by the user, the buttons provide intuitive mechanical feedback thanks to a snap-through characteristic in their force-displacement profile. The snap-through threshold can be as high as 4 N, and is tuned by design and actuation parameters. This work presents two versions of PopTouch: a transparent PopTouch for integration on Touchscreens with built-in touch sensing, such as smartphones and a sensorized PopTouch, with embedded thin-film piezoelectric sensors on each taxel, for integration on substrates without built-in touch sensing, such as a steering wheel. PopTouch adds static and vibrating button-like haptics to any device thanks to its thin profile, flexibility, low power consumption (6 mW per button), rapid refresh rate (2 Hz), and freely configured array format.

Universal pictures: A lithophane codex helps teenagers with blindness visualize nanoscopic systems

People with blindness have limited access to the high-resolution graphical data and imagery of science. Here, a lithophane codex is reported. Its pages display tactile and optical readouts for universal visualization of data by persons with or without eyesight. Prototype codices illustrated microscopy of butterfly chitin-from N-acetylglucosamine monomer to fibril, scale, and whole insect-and were given to high schoolers from the Texas School for the Blind and Visually Impaired. Lithophane graphics of Fischer-Spier esterification reactions and electron micrographs of biological cells were also 3D-printed, along with x-ray structures of proteins (as millimeter-scale 3D models). Students with blindness could visualize (describe, recall, distinguish) these systems-for the first time-at the same resolution as sighted peers (average accuracy = 88%). Tactile visualization occurred alongside laboratory training, synthesis, and mentoring by chemists with blindness, resulting in increased student interest and sense of belonging in science.

SmartSpring: A Low-Cost Wearable Haptic VR Display with Controllable Passive Feedback

With the development of virtual reality, the practical requirements of the wearable haptic interface have been greatly emphasized. While passive haptic devices are commonly used in virtual reality, they lack generality and are difficult to precisely generate continuous force feedback to users. In this work, we present SmartSpring, a new solution for passive haptics, which is inexpensive, lightweight and capable of providing controllable force feedback in virtual reality. We propose a hybrid spring-linkage structure as the proxy and flexibly control the mechanism for adjustable system stiffness. By analyzing the structure and force model, we enable a smart transform of the structure for producing continuous force signals. We quantitatively examine the real-world performance of SmartSpring to verify our model. By asymmetrically moving or actively pressing the end-effector, we show that our design can further support rendering torque and stiffness. Finally, we demonstrate the SmartSpring in a series of scenarios with user studies and a just noticeable difference analysis. Experimental results show the potential of the developed haptic display in virtual reality.

HaptGlove-Untethered Pneumatic Glove for Multimode Haptic Feedback in Reality-Virtuality Continuum

Novel haptics technologies are urgently needed to bridge the gap between entirely physical world and fully digital environment to render a more realistic and immersive human-computer interaction. Current virtual reality (VR) haptic gloves either deliver limited haptic feedback or are bulky and heavy. The authors develop a haptic glove or HaptGlove, an untethered and lightweight pneumatic glove, that allows users to "physically" interact in a VR environment and enables both kinesthetic and cutaneous sensations naturally and realistically. Integrated with five pairs of haptic feedback modules and fiber sensors, HaptGlove provides variable stiffness force feedback and fingertip force and vibration feedback, allowing users to touch, press, grasp, squeeze, and pull various virtual objects and feel the dynamic haptic changes. Significant improvements in VR realism and immersion are observed in a user study with participants achieving 78.9% accuracy in sorting six virtual balls of different stiffnesses. Importantly, HaptGlove facilitates VR training, education, entertainment, and socialization in a reality-virtuality continuum.

Wrapping Haptic Displays Around Robot Arms to Communicate Learning

Humans can leverage physical interaction to teach robot arms. As the human kinesthetically guides the robot through demonstrations, the robot learns the desired task. While prior works focus on how the robot learns, it is equally important for the human teacher to understand what their robot is learning. Visual displays can communicate this information; however, we hypothesize that visual feedback alone misses out on the physical connection between the human and robot. In this paper we introduce a novel class of soft haptic displays that wrap around the robot arm, adding signals without affecting that interaction. We first design a pneumatic actuation array that remains flexible in mounting. We then develop single and multi-dimensional versions of this wrapped haptic display, and explore human perception of the rendered signals during psychophysic tests and robot learning. We ultimately find that people accurately distinguish single-dimensional feedback with a Weber fraction of 11.4%, and identify multi-dimensional feedback with 94.5% accuracy. When physically teaching robot arms, humans leverage the single- and multi-dimensional feedback to provide better demonstrations than with visual feedback: our wrapped haptic display decreases teaching time while increasing demonstration quality. This improvement depends on the location and distribution of the wrapped haptic display.

Perception Accuracy of a Multi-Channel Tactile Feedback System for Assistive Technology

Assistive technology uses multi-modal feedback devices, focusing on the visual, auditory, and haptic modalities. Tactile devices provide additional information via touch sense. Perception accuracy of vibrations depends on the spectral and temporal attributes of the signal, as well as on the body parts they are attached to. The widespread use of AR/VR devices, wearables, and gaming interfaces requires information about the usability of feedback devices. This paper presents results of an experiment using an 8-channel tactile feedback system with vibrators placed on the wrists, arms, ankles, and forehead. Different vibration patterns were designed and presented using sinusoidal frequency bursts on 2, 4, and 8 channels. In total, 27 subjects reported their sensation formally and informally on questionnaires. Results indicate that 2 and 4 channels could be used simultaneously with high accuracy, and the transducers' optimal placement (best sensitivity) is on the wrists, followed by the ankles. Arm and head positions were inferior and generally inadequate for signal presentation. For optimal performance, signal length should exceed 500 ms. Furthermore, the amplitude level and temporal pattern of the presented signals have to be used for carrying information rather than the frequency of the vibration.

SuP-Ring: A pneumatic tactile display with substitutional representation of contact force components using normal indentation

BACKGROUND

Shear force is important for tumour detection and can contribute to minimally invasive surgery (MIS). A popular method uses lateral skin stretch to produce shear force but has some limitations.

METHODS

We have developed a ring-type pneumatic tactile display that employs normal indentation substituted for lateral skin stretch to represent normal and shear feedback, called SuP-Ring. Psychophysical experiments were conducted to evaluate how users perceive the provided feedback and the effectiveness of SuP-Ring in tumour localisation.

RESULTS

The experimental results show that the participants could perceive the provided normal and shear feedback well. Shear feedback enables users to enhance their performance in localising the tumour and normal feedback could contribute to ensuring the safety requirements in MIS.

CONCLUSIONS

The proposed tactile display could be useful for intraoperative tumour localisation and has the potential to be used in a wide variety of applications.

Integrating Tactile Feedback Technologies Into Home-Based Telerehabilitation: Opportunities and Challenges in Light of COVID-19 Pandemic

The COVID-19 pandemic has highlighted the need for advancing the development and implementation of novel means for home-based telerehabilitation in order to enable remote assessment and training for individuals with disabling conditions in need of therapy. While somatosensory input is essential for motor function, to date, most telerehabilitation therapies and technologies focus on assessing and training motor impairments, while the somatosensorial aspect is largely neglected. The integration of tactile devices into home-based rehabilitation practice has the potential to enhance the recovery of sensorimotor impairments and to promote functional gains through practice in an enriched environment with augmented tactile feedback and haptic interactions. In the current review, we outline the clinical approaches for stimulating somatosensation in home-based telerehabilitation and review the existing technologies for conveying mechanical tactile feedback (i.e., vibration, stretch, pressure, and mid-air stimulations). We focus on tactile feedback technologies that can be integrated into home-based practice due to their relatively low cost, compact size, and lightweight. The advantages and opportunities, as well as the long-term challenges and gaps with regards to implementing these technologies into home-based telerehabilitation, are discussed.