A Frequency-Domain Analysis of Haptic Gratings

Enhancing touch sensibility by sensory retraining in a sensory discrimination task via haptic rendering

Stroke survivors are commonly affected by somatosensory impairment, hampering their ability to interpret somatosensory information. Somatosensory information has been shown to critically support movement execution in healthy individuals and stroke survivors. Despite the detrimental effect of somatosensory impairments on performing activities of daily living, somatosensory training—in stark contrast to motor training—does not represent standard care in neurorehabilitation. Reasons for the neglected somatosensory treatment are the lack of high-quality research demonstrating the benefits of somatosensory interventions on stroke recovery, the unavailability of reliable quantitative assessments of sensorimotor deficits, and the labor-intensive nature of somatosensory training that relies on therapists guiding the hands of patients with motor impairments. To address this clinical need, we developed a virtual reality-based robotic texture discrimination task to assess and train touch sensibility. Our system incorporates the possibility to robotically guide the participants' hands during texture exploration (i.e., passive touch) and no-guided free texture exploration (i.e., active touch). We ran a 3-day experiment with thirty-six healthy participants who were asked to discriminate the odd texture among three visually identical textures –haptically rendered with the robotic device– following the method of constant stimuli. All participants trained with the passive and active conditions in randomized order on different days. We investigated the reliability of our system using the Intraclass Correlation Coefficient (ICC). We also evaluated the enhancement of participants' touch sensibility via somatosensory retraining and compared whether this enhancement differed between training with active vs. passive conditions. Our results showed that participants significantly improved their task performance after training. Moreover, we found that training effects were not significantly different between active and passive conditions, yet, passive exploration seemed to increase participants' perceived competence. The reliability of our system ranged from poor (in active condition) to moderate and good (in passive condition), probably due to the dependence of the ICC on the between-subject variability, which in a healthy population is usually small. Together, our virtual reality-based robotic haptic system may be a key asset for evaluating and retraining sensory loss with minimal supervision, especially for brain-injured patients who require guidance to move their hands.

Characterizing Detection Thresholds for Six Orthogonal Modes of Vibrotactile Display Via Stylus With Precision Grasp

In this paper, we characterize the detection thresholds in six orthogonal modes of vibrotactile haptic display via stylus, including three orthogonal force directions and three orthogonal torque directions at the haptic interaction point. A psychophysical study is performed to determine detection thresholds over the frequency range 20-250 Hz, for six distinct styluses. Analysis of variance is used to test the hypothesis that force signals, as well as torque signals, applied in different directions have different detection thresholds. We find that people are less sensitive to force signals parallel to the stylus than to those orthogonal to the stylus at low frequencies, and far more sensitive to torque signals about the stylus than to those orthogonal to the stylus. Optimization techniques are used to determine four independent two-parameter models to describe the frequency-dependent thresholds for each of the orthogonal force and torque modes for a stylus that is approximately radially symmetric; six independent models are required if the stylus is not well approximated as radially symmetric. Finally, we provide a means to estimate the model parameters given stylus parameters, for a range of styluses, and to estimate the coupling between orthogonal modes.

Vibrotactile Display of Patterned Surface Textures With Kinesthetic Haptic Devices Using Balanced Impulses

Kinesthetic haptic devices are designed primarily to display quasistatic and low-bandwidth forces and moments. Existing methods for vibrotactile display sometimes introduce haptic and/or audio artifacts. In this article, we propose a method to display vibrotactile stimulus signals of moderate to high frequency (20-500 Hz) using kinesthetic haptic devices with a standard 1 kHz haptic update rate. Our method combines symmetric square-wave signals whose periods are even multiples of the haptic update period with asymmetric square-wave signals whose periods are odd multiples of the haptic update period, while ensuring that the positive and negative impulses are balanced in both cases, and utilizing the just noticeable difference in frequency discrimination to avoid the need to display other frequencies. For frequencies at which the above method is insufficient, corresponding to a small band near 400 Hz for a 1 kHz update rate, we utilize a signal-mixing method. Our complete method is then extended to render haptic gratings by measuring scanning velocity, converting the local spatial frequency to its equivalent instantaneous temporal frequency, and displaying a single full-period vibration event. In a series of human-subject studies, we showed that our proposed method is preferred over existing methods for vibrotactile display of signals with relatively high-frequency content.

Building a Navigable Fine Texture Design Space

Friction modulation technology enables the creation of textural effects on flat haptic displays. However, an intuitive and manageably small design space for construction of such haptic textures remains an unfulfilled goal for user interface designers. In this paper, we explore perceptually relevant features of fine texture for use in texture construction and modification. Beginning with simple sinusoidal patterns of friction force that vary in frequency and amplitude, we define irregularity, essentially a variable amount of introduced noise, as a third building block of a texture pattern. We demonstrate using multidimensional scaling that all three parameters are scalable features perceptually distinct from each other. Additionally, participants' verbal descriptions of this 3-dimensional design space provide insight into their intuitive interpretation of the physical parameter changes.

Estimation of Torso Vibrotactile Thresholds Using Eccentric Rotating Mass Motors

The characterization of vibrotactile perception is crucial to accurately configure haptic devices and create appropriate stimuli for improving user performance in human-machine interaction systems. This article presents a study aiming to determine the absolute and differential vibrotactile thresholds in different areas of the torso to develop reliable haptic patterns to be displayed using a haptic vest. In the 'absolute threshold' experiment, we measure the minimum detectable vibration using a forced-choice task. Furthermore, in the 'differential threshold' experiment, we measure the minimum frequency change needed for users to discriminate two successive vibrotactile stimuli using a vibration matching task. The first experiment does not show differences between absolute thresholds, opening up the possibility of setting a unique minimal vibration for creating haptic patterns. Similarly, the second experiment does not show differences between differential thresholds. Moreover, as these thresholds follow Weber's law, it is viable to estimate any upper or lower differential threshold for any reference stimulus using a K-value. These results are a first step for creating vibrotactile patterns over the torso with the employed eccentric rotating mass motors. Moreover, the whole study provides a method to obtain these psychophysical values since the usage of different motors can change these results.

Phase Difference Between Normal and Shear Forces During Tactile Exploration Represents Textural Features

Contact forces and skin deformation induced during tactile exploration have been investigated in the frequency domain to understand finger-material interaction. Their power spectra are one of the representative feature quantities that have been associated with the surface properties of materials. However, thus far, the phase information of these quantities has not been studied. Furthermore, most previous studies focused on uni-dimensional signals such as forces in either the normal or tangential directions. We investigated the phase differences between normal and shear forces induced during tactile exploration. The results showed that the phase differences between these two axial forces differ among materials and that they exhibit features different from their power spectra. These results indicate that the phase difference between two axial forces should be taken into account to understand the finger-material interactions during tactile exploration.

Towards Multisensory Perception: Modeling and Rendering Sounds of Tool-Surface Interactions

Touch-produced sounds in tool-surface interactions convey rich information about textured surface properties and provide direct feedback about how users interact with the surface. This article presents a statistical learning-based approach for modeling and rendering touch-produced sounds in real time. We apply a data-driven modeling method, which recreates highly realistic sounds using audio signals recorded from unconstrained tool-surface interactions. The recorded sound is segmented, and each segment is labeled with the average velocity during that time. We model each segment with wavelet tree models using a moving window approach. Each window is analyzed by fast wavelet transform and is then broken down into a tree structure. During rendering, we use the user's current velocity to select tree models and synthesize new sounds by breadth-first search and inverse wavelet transform. We conducted a user study to evaluate the realism of our virtual sounds and their effect on human's perception of the texture dimensions in the presence of simultaneous real haptic cues. The results showed that in the presence of haptic cues, the virtual sound can more completely capture the texture's roughness and hardness than haptic cues alone. However, the perception on slipperiness depended mainly on touch.

Detection and Discrimination Thresholds for Haptic Gratings on Electrostatic Tactile Displays

Designing algorithmsfor rendering haptic texture on electrostatic tactile displays requires a quantitative understanding of human perception. In this paper, we report detection and discrimination thresholds for haptic gratings rendered on such displays based on the waveform and amplitude of the applied voltage. The haptic gratings consist of functions that describe the variation in voltage amplitude as a function of the position of finger on the display. Four types of virtual haptic gratings are considered in two experiments. In Experiment I, we estimate the absolute detection thresholds of haptic gratings for four different voltage amplitude functions, consisting of spatial waveforms with sinusoidal, square, triangle, or sawtooth shape. In Experiment II, we report discrimination thresholds for haptic gratings at five reference values of the voltage amplitude (80, 120, 160, 200, and 240 Vpp) for each of the voltage amplitude functions used in Experiment I. The results indicate that the detection thresholds for the four virtual haptic gratings are between 30 and 36 Vpp, and that the JND increases with the increase of voltage amplitudes. In addition, the JNDs of the four virtual gratings differ significantly, with the lowest and highest values being given by the triangle and sawtooth waveform, respectively.

Effect of Waveform on Tactile Perception by Electrovibration Displayed on Touch Screens

In this study, we investigated the effect of input voltage waveform on our haptic perception of electrovibration on touch screens. Through psychophysical experiments performed with eight subjects, we first measured the detection thresholds of electrovibration stimuli generated by sinusoidal and square voltages at various fundamental frequencies. We observed that the subjects were more sensitive to stimuli generated by square wave voltage than sinusoidal one for frequencies lower than 60 Hz. Using Matlab simulations, we showed that the sensation difference of waveforms in low fundamental frequencies occurred due to the frequency-dependent electrical properties of human skin and human tactile sensitivity. To validate our simulations, we conducted a second experiment with another group of eight subjects. We first actuated the touch screen at the threshold voltages estimated in the first experiment and then measured the contact force and acceleration acting on the index fingers of the subjects moving on the screen with a constant speed. We analyzed the collected data in the frequency domain using the human vibrotactile sensitivity curve. The results suggested that Pacinian channel was the primary psychophysical channel in the detection of the electrovibration stimuli caused by all the square-wave inputs tested in this study. We also observed that the measured force and acceleration data were affected by finger speed in a complex manner suggesting that it may also affect our haptic perception accordingly.

A Physics-Based Vibrotactile Feedback Library for Collision Events

We present PhysVib: a software solution on the mobile platform extending an open-source physics engine in a multi-rate rendering architecture for automatic vibrotactile feedback upon collision events. PhysVib runs concurrently with a physics engine at a low update rate and generates vibrotactile feedback commands at a high update rate based on the simulation results of the physics engine using an exponentially-decaying sinusoidal model. We demonstrate through a user study that this vibration model is more appropriate to our purpose in terms of perceptual quality than more complex models based on sound synthesis. We also evaluated the perceptual performance of PhysVib by comparing eight vibrotactile rendering methods. Experimental results suggested that PhysVib enables more realistic vibrotactile feedback than the other methods as to perceived similarity to the visual events. PhysVib is an effective solution for providing physically plausible vibrotactile responses while reducing application development time to great extent.

Modeling and rendering realistic textures from unconstrained tool-surface interactions

Texture gives real objects an important perceptual dimension that is largely missing from virtual haptic interactions due to limitations of standard modeling and rendering approaches. This paper presents a set of methods for creating a haptic texture model from tool-surface interaction data recorded by a human in a natural and unconstrained manner. The recorded high-frequency tool acceleration signal, which varies as a function of normal force and scanning speed, is segmented and modeled as a piecewise autoregressive (AR) model. Each AR model is labeled with the source segment's median force and speed values and stored in a Delaunay triangulation to create a model set for a given texture. We use these texture model sets to render synthetic vibration signals in real time as a user interacts with our TexturePad system, which includes a Wacom tablet and a stylus augmented with a Haptuator. We ran a human-subject study with two sets of ten participants to evaluate the realism of our virtual textures and the strengths and weaknesses of this approach. The results indicated that our virtual textures accurately capture and recreate the roughness of real textures, but other modeling and rendering approaches are required to completely match surface hardness and slipperiness.

The physical basis of perceived roughness in virtual sinusoidal textures

Using a high-fidelity haptic interface based on magnetic levitation, subjects explored virtual sinusoidal textures with a frictionless probe and reported the subjective magnitude of perceived roughness. A psychophysical function was obtained spanning 33 levels of spatial periods from 0.025 to 6.00 mm. Kinematic and dynamic variables were recorded at 1,000 Hz and used to derive a set of variables to correlate with the psychophysical outcome. These included position, velocity, kinetic energy, instantaneous force (based on acceleration), mean force, and variability of the z-axis force signal from the power spectral density. The analysis implicates power of the force signal as the physical correlate of perceived roughness of sinusoidal textures. The relationship between power and roughness held across the range of spatial periods examined.

Application of psychophysical techniques to haptic research

Various psychophysical methods have been used to study human haptic perception, although the selection of a particular method is often based on convention, rather than an analysis of which technique is optimal for the question being addressed. In this review, classical psychophysical techniques used to measure sensory thresholds are described as well as more modern methods such as adaptive procedures and those associated with signal detection theory. Details are provided as to how these techniques should be implemented to measure absolute and difference thresholds and factors that influence subjects' responses are noted. In addition to the methods used to measure sensory thresholds, the techniques available for measuring the perception of suprathreshold stimuli are presented. These scaling methods are reviewed in the context of the various stimulus and response biases that influence how subjects respond to stimuli. The importance of understanding the factors that influence perceptual processing is highlighted throughout the review with reference to experimental studies of haptic perception.

Discrimination of real and virtual surfaces with sinusoidal and triangular gratings using the fingertip and stylus

Two-interval two-alternative forced-choice discrimination experiments were conducted separately for sinusoidal and triangular textured surface gratings from which amplitude (i.e., height) discrimination thresholds were estimated. Participants (group sizes: n = 4 to 7) explored one of these texture types either by fingertip on real gratings (Finger real), by stylus on real gratings (Stylus real), or by stylus on virtual gratings (Stylus virtual). The real gratings were fabricated from stainless steel by an electrical discharge machining process while the virtual gratings were rendered via a programmable force-feedback device. All gratings had a 2.5-mm spatial period. On each trial, participants compared test gratings with 55, 60, 65, or 70 μm amplitudes against a 50-μm reference. The results indicate that discrimination thresholds did not differ significantly between sinusoidal and triangular gratings. With sinusoidal and triangular data combined, the average (mean + standard error) for the Stylus-real threshold (2.5 ± 0.2 μm) was significantly smaller (p <; 0.01) than that for the Stylus-virtual condition (4.9 ± 0.2 μm). Differences between the Finger-real threshold (3.8 ± 0.2 μm) and those from the other two conditions were not statistically significant. Further studies are needed to better understand the differences in perceptual cues resulting from interactions with real and virtual gratings.

Lossy data compression of vibrotactile material-like textures

Tactile content will be delivered over the Internet in the near future. Vibrotactile material-like textures that resemble the surfaces of wood, leather, etc., are representative of such content. We performed lossy compression of texture data for reducing the data size. We confirmed the effectiveness of two compression strategies: quantization and truncation of data beneath a shifted perceptual threshold curve. In the quantization strategy, the amplitude spectra of vibrotactile textures could be quantized in 14 steps. This reduced the data size to approximately one quarter without any noticeable quality deterioration. The method for truncating frequency components with amplitudes smaller than a shifted perceptual threshold curve was also effective, and it was preferable to the automatic deletion of subthreshold amplitudes. We reduced the data size of vibrotactile material textures to 10-20 percent of their original size by combining the lossy data compression strategy with Huffman coding, which is a lossless data compression method. Lossy compression algorithms will enhance the online delivery of vibrotactile material-like textures by decreasing their data size without significant loss of quality.

Virtual Active Touch: Perception of Virtual Gratings Wavelength through Pointing-Stick Interface

Tactile feedback enhances the usability and enjoyment of human-computer interfaces. Many feedback techniques have been devised to present tactile stimuli corresponding to a user's hand movements taking account of the concept of active touch. However, hand movements may not necessarily be required for achieving natural tactile feedback. Here, we propose a virtual-active-touch method that achieves haptic perception without actual/direct hand movements. In this method, a cursor manipulated by a force-input device is regarded as a virtual finger of the operator on the screen. Tactile feedback is provided to the operator in accordance with cursor movements. To validate the translation of virtual roughness gratings, we compare the virtual-active-touch interface with an interface that involves actual hand movements. By using the appropriate force-to-velocity gain for the pointing-stick interface, we show that the virtual-active-touch method presents the surface wavelengths of the gratings, which is a fundamental property for texture roughness, and that the gain significantly influences the textures experienced by the operators. Furthermore, we find that the perceived wavelengths of objects scaled and viewed on a small screen are skewed. We conclude that although some unique problems remain to be solved, we may be able to perceive the surface wavelengths solely with the intentions of active touch through virtual-active-touch interfaces.

Tactile and Haptic Illusions

This paper surveys the research literature on robust tactile and haptic illusions. The illusions are organized into two categories. The first category relates to objects and their properties, and is further differentiated in terms of haptic processing of material versus geometric object properties. The second category relates to haptic space, and is further differentiated in terms of the observer's own body versus external space. The illusions are initially described and where possible addressed in terms of their functional properties and/or underlying neural processes. The significance of these illusions for the design of tactile and haptic displays is also discussed. We conclude by briefly considering a number of important general themes that have emerged in the materials surveyed.

What you can't feel won't hurt you: Evaluating haptic hardware using a haptic contrast sensitivity function

In this paper, we extend the concept of the contrast sensitivity function - used to evaluate video projectors - to the evaluation of haptic devices. We propose using human observers to determine if vibrations rendered using a given haptic device are accompanied by artifacts detectable to humans. This determination produces a performance measure that carries particular relevance to applications involving texture rendering. For cases in which a device produces detectable artifacts, we have developed a protocol that localizes deficiencies in device design and/or hardware implementation. In this paper, we present results from human vibration detection experiments carried out using three commercial haptic devices and one high performance voice coil motor. We found that all three commercial devices produced perceptible artifacts when rendering vibrations near human detection thresholds. Our protocol allowed us to pinpoint the deficiencies, however, and we were able to show that minor modifications to the haptic hardware were sufficient to make these devices well suited for rendering vibrations, and by extension, the vibratory components of textures. We generalize our findings to provide quantitative design guidelines that ensure the ability of haptic devices to proficiently render the vibratory components of textures.