Interpersonal Transmission of Vibrotactile Feedback Via Smart Bracelets: Mechanics and Perception

The importance of interpersonal touch for social well-being is widely recognized, and haptic technology offers a promising avenue for augmenting these interactions. We presented smart bracelets that use vibrotactile feedback to augment social interactions, such as handshakes, by transmitting vibrations between two people. This work conducts mechanical and perceptual experiments to investigate key factors affecting the delivery of interpersonal vibrotactile feedback via bracelets. Our results show that low-frequency vibrations elicited through tangential actuation are efficiently transmitted from the wrist to the hand, with amplitude varying based on distance, frequency, and actuation direction. We also found that vibrations transmitted to different locations on the hand can be felt by a second person, with perceptual intensity correlated with oscillation magnitude at the touched location. Additionally, we demonstrate how wrist-interfaced devices can elicit spatial vibration patterns throughout the hand surface, which can be manipulated by the frequency and direction of actuation at the wrist. Our experiments provide important insights into the human factors associated with interpersonal vibrotactile feedback and have significant implications for the design of technologies that promote social well-being.

Rendering Dynamic Source Motion in Surface Haptics via Wave Focusing

Emerging surface haptic technologies can display localized haptic feedback anywhere on a touch surface by focusing mechanical waves generated via sparse arrays of actuators. However, rendering complex haptic scenes with such displays is challenging due to the infinite number of physical degrees of freedom intrinsic to such continuum mechanical systems. Here, we present computational focusing methods for rendering dynamic tactile sources. They can be applied to a variety of surface haptic devices and media, including those that exploit flexural waves in thin plates and solid waves in elastic media. We describe an efficient rendering technique based on time-reversal of waves emitted from a moving source and motion path discretization. We combine these with intensity regularization methods that reduce focusing artifacts, improve power output, and increase dynamic range. We demonstrate the utility of this approach in experiments with a surface display that uses elastic wave focusing to render dynamic sources, achieving millimeter-scale resolution in experiments. Results of a behavioral experiment show that participants could readily feel and interpret rendered source motion, attaining 99% accuracy across a wide range of motion speeds.

Shear shock waves mediate haptic holography via focused ultrasound

Emerging holographic haptic interfaces focus ultrasound in air to enable their users to touch, feel, and manipulate three-dimensional virtual objects. However, current holographic haptic systems furnish tactile sensations that are diffuse and faint, with apparent spatial resolutions that are far coarser than would be theoretically predicted from acoustic focusing. Here, we show how the effective spatial resolution and dynamic range of holographic haptic displays are determined by ultrasound-driven elastic wave transport in soft tissues. Using time-resolved optical imaging and numerical simulations, we show that ultrasound-based holographic displays excite shear shock wave patterns in the skin. The spatial dimensions of these wave patterns can exceed nominal focal dimensions by more than an order of magnitude. Analyses of data from behavioral and vibrometry experiments indicate that shock formation diminishes perceptual acuity. For holographic haptic displays to attain their potential, techniques for circumventing shock wave artifacts, or for exploiting these phenomena, are needed.

Dual-Mode Organic Electrochemical Transistors Based on Self-Doped Conjugated Polyelectrolytes for Reconfigurable Electronics

Reconfigurable organic logic devices are promising candidates for next generations of efficient computing systems and adaptive electronics. Ideally, such devices would be of simple structure and design, be power efficient, and compatible with high-throughput microfabrication techniques. This work reports an organic reconfigurable logic gate based on novel dual-mode organic electrochemical transistors (OECTs), which employ a self-doped conjugated polyelectrolyte as the active material, which then allows the transistors to operate in both depletion mode and enhancement mode. Furthermore, mode switching is accomplished by simply altering the polarity of the applied gate and drain voltages, which can be done on the fly. In contrast, achieving similar mode-switching functionality with other organic transistors typically requires complex molecular design or multi-device engineering. It in shown that dual-mode functionality is enabled by the concurrent existence of anion doping and cation dedoping of the films. A device physics model that accurately describes the behavior of these transistors is developed. Finally, the utility of these dual-mode transistors for implementing reconfigurable logic by fabricating a logic gate that may be switched between logic gates AND to NOR, and OR to NAND on the fly is demonstrated.

Efficient Fabrication of Organic Electrochemical Transistors via Wet Chemical Processing

A wet processing method to fabricate high-performance organic electrochemical transistors (OECTs) is reported. Wet chemical processing enables a simple and reliable patterning step, substituting several complex and expensive cleanroom procedures in the fabrication of OECTs. We fabricate depletion-mode OECTs based on poly(3,4-ethylenedioxythiophene) polystyrene sulfonate (PEDOT:PSS) and enhancement-mode OECTs based on a conjugated polyelectrolyte PCPDTBT-SO₃K on rigid and flexible substrates using this wet processing method. We show that the wet chemical processing step can also serve as a chemical treatment to enhance the electrical properties of the active material in OECTs. To highlight the potential of the fabrication process in applications, a transistor-based chemical sensor is demonstrated, capable of detecting methylene blue, a popular redox reporter in biodetection and immunoassays, with good detectivity. Given the tremendous potential of OECTs in emerging technologies such as biosensing and neuromorphic computing, this simple fabrication process established herein will render the OECT platform more accessible for research and applications.

Elastohydrodynamic friction of robotic and human fingers on soft micropatterned substrates

Frictional sliding between patterned surfaces is of fundamental and practical importance in the haptic engineering of soft materials. In emerging applications such as remote surgery and soft robotics, thin fluid films between solid surfaces lead to a multiphysics coupling between solid deformation and fluid dissipation. Here, we report a scaling law that governs the peak friction values of elastohydrodynamic lubrication on patterned surfaces. These peaks, absent in smooth tribopairs, arise due to a separation of length scales in the lubricant flow. The framework is generated by varying the geometry, elasticity and fluid properties of soft tribopairs and measuring the lubricated friction with a triborheometer. The model correctly predicts the elastohydrodynamic lubrication friction of a bioinspired robotic fingertip and human fingers. Its broad applicability can inform the future design of robotic hands or grippers in realistic conditions, and open up new ways of encoding friction into haptic signals.

Tactile Echoes: Multisensory Augmented Reality for the Hand

Touch interactions are central to many human activities, but there are few technologies for computationally augmenting free-hand interactions with real environments. Here, we describe Tactile Echoes, a finger-wearable system for augmenting touch interactions with physical objects. This system captures and processes touch-elicited vibrations in real-time in order to enliven tactile experiences. In this article, we process these signals via a parametric signal processing network in order to generate responsive tactile and auditory feedback. Just as acoustic echoes are produced through the delayed replication and modification of sounds, so are Tactile Echoes produced through transformations of vibrotactile inputs in the skin. The echoes also reflect the contact interactions and touched objects involved. A transient tap produces discrete echoes, while a continuous slide yields sustained feedback. We also demonstrate computational and spatial tracking methods that allow these effects to be selectively assigned to different objects or actions. A large variety of distinct multisensory effects can be designed via ten processing parameters. We investigated how Tactile Echoes are perceived in several perceptual experiments using multidimensional scaling methods. This allowed us to deduce low-dimensional, semantically grounded perceptual descriptions. We present several virtual and augmented reality applications of Tactile Echoes. In a user study, we found that these effects made interactions more responsive and engaging. Our findings show how to endow a large variety of touch interactions with expressive multisensory effects.

Beyond Soft Hands: Efficient Grasping With Non-Anthropomorphic Soft Grippers

Grasping and manipulation are challenging tasks that are nonetheless critical for many robotic systems and applications. A century ago, robots were conceived as humanoid automata. While conceptual at the time, this viewpoint remains influential today. Many robotic grippers have been inspired by the dexterity and functionality of the prehensile human hand. However, multi-fingered grippers that emulate the hand often integrate many kinematic degrees-of-freedom, and thus complex mechanisms, which must be controlled in order to grasp and manipulate objects. Soft fingers can facilitate grasping through intrinsic compliance, enabling them to conform to diverse objects. However, as with conventional fingered grippers, grasping via soft fingers involves challenges in perception, computation, and control, because fingers must be placed so as to achieve force closure, which depends on the shape and pose of the object. Emerging soft robotics research on non-anthropomorphic grippers has yielded new techniques that can circumvent fundamental challenges associated with grasping via fingered grippers. Common to many non-anthropomorphic soft grippers are mechanisms for morphological deformation or adhesion that simplify the grasping of diverse objects in different poses, without detailed knowledge of the object geometry. These advantages may allow robots to be used in challenging applications, such as logistics or rapid manufacturing, with lower cost and complexity. In this perspective, we examine challenges associated with grasping via anthropomorphic grippers. We describe emerging soft, non-anthropomorphic grasping methods, and how they may reduce grasping complexities. We conclude by proposing several research directions that could expand the capabilities of robotic systems utilizing non-anthropomorphic grippers.

Rendering Spatiotemporal Haptic Effects Via the Physics of Waves in the Skin

A major challenge in haptic engineering has been to design practical methods to efficiently stimulate distributed areas of skin. Here, we show how to use a single actuator to generate vibrotactile stimuli which cause sensations of temporally varying spatial extent. Through optical vibrometry methods, we show that vibrational stimuli applied at the fingertip elicit waves in the finger that propagate proximally toward the hand and show how the frequency-dependent damping behavior of skin causes propagation distances to decrease rapidly with increasing frequency of stimulation. Utilizing these results, we design haptic stimuli applied through a single actuator that produces wavefields that expand or contract in size. In a perception experiment, participants accurately (median $>$95%) identified these stimuli as expanding or contracting without prior exposure or training. As a potential application, we used these effects as haptic cues for interactions in virtual reality. We show through a second experiment that the spatiotemporal haptic stimuli were rated as significantly more engaging than conventional vibrotactile stimuli. These findings demonstrate how the physics of waves in skin can be utilized to excite spatiotemporal tactile effects over large surface areas with a single actuator, and inform methods to utilize the effects in practical applications.

Proprioceptive Localization of the Fingers: Coarse, Biased, and Context-Sensitive

The proprioceptive sense provides somatosensory information about positions of parts of the body, information that is essential for guiding behavior and monitoring the body. Few studies have investigated the perceptual localization of individual fingers, despite their importance for tactile exploration and fine manipulation. We present two experiments assessing the performance of proprioceptive localization of multiple fingers, either alone or in combination with visual cues. In the first experiment, we used a virtual reality paradigm to assess localization of multiple fingers. Surprisingly, the errors averaged 3.7 cm per digit, which represents a significant fraction of the range of motion of any finger. Both random and systematic errors were large. The latter included participant-specific biases and participant-independent distortions that evoked similar observations from prior studies of perceptual representations of hand shape. In a second experiment, we introduced visual cues about positions of nearby fingers, and observed that this contextual information could greatly decrease localization errors. The results suggest that only coarse proprioceptive information is available through somatosensation, and that finer information may not be necessary for fine motor behavior. These findings may help elucidate human hand function, and inform new applications to the design of human-computer interfaces or interactions in virtual reality.

Compression of dynamic tactile information in the human hand

A key problem in the study of the senses is to describe how sense organs extract perceptual information from the physics of the environment. We previously observed that dynamic touch elicits mechanical waves that propagate throughout the hand. Here, we show that these waves produce an efficient encoding of tactile information. The computation of an optimal encoding of thousands of naturally occurring tactile stimuli yielded a compact lexicon of primitive wave patterns that sparsely represented the entire dataset, enabling touch interactions to be classified with an accuracy exceeding 95%. The primitive tactile patterns reflected the interplay of hand anatomy with wave physics. Notably, similar patterns emerged when we applied efficient encoding criteria to spiking data from populations of simulated tactile afferents. This finding suggests that the biomechanics of the hand enables efficient perceptual processing by effecting a preneuronal compression of tactile information.

Fluidic Fabric Muscle Sheets for Wearable and Soft Robotics

Conformable robotic systems are attractive for applications in which they may actuate structures with large surface areas, provide forces through wearable garments, or enable autonomous robotic systems. We present a new family of soft actuators that we refer to as Fluidic Fabric Muscle Sheets (FFMS). They are composite fabric structures that integrate fluidic transmissions based on arrays of elastic tubes. These sheet-like actuators can strain, squeeze, bend, and conform to hard or soft objects of arbitrary shapes or sizes, including the human body. We show how to design and fabricate FFMS actuators via facile apparel engineering methods, including computerized sewing techniques that determine the stress and strain distributions that can be generated. We present a simple mathematical model that proves effective for predicting their performance. FFMS can operate at frequencies of 5 Hz or more, achieve engineering strains exceeding 100%, and exert forces >115 times their weight. They can be safely used in intimate contact with the human body even when delivering stresses exceeding 10⁶ Pascals. We demonstrate their versatility for actuating a variety of bodies or structures, and in configurations that perform multiaxis actuation, including bending and shape change. As we also show, FFMS can be used to exert forces on body tissues for wearable and biomedical applications. We demonstrate several potential use cases, including a miniature steerable robot, a glove for grasp assistance, garments for applying compression to the extremities, and devices for actuating small body regions or tissues via localized skin stretch.

Dynamics and Perception in the Thermal Grill Illusion

A basic challenge in perception research is to understand how sensory inputs from physical environments and the body are integrated in order to facilitate perceptual inferences. Thermal perception, which arises through heat transfer between extrinsic sources and body tissues, is an integral part of natural haptic experiences, and thermal feedback technologies have potential applications in wearable computing, virtual reality, and other areas. While physics dictates that thermal percepts can be slow, often unfolding over timescales measured in seconds, much faster perceptual responses can occur in the thermal grill illusion. The latter refers to a burning-like sensation that can be evoked when innocuous warm and cool stimuli are applied to the skin in juxtaposed fashion. Here, we show that perceptual response times to the thermal grill illusion decrease systematically with perceived intensity. Using results from behavioral experiments in combination with a physics-based description of tissue heating, we develop a simple model explaining the perception of the illusion through the evolution of internal tissue temperatures. The results suggest that improved understanding of the physical mechanisms of tissue heating may aid our understanding of thermal perception, as exemplified by the thermal grill illusion, and might point toward more efficient methods for thermal feedback.

Detection and Discrimination Thresholds for Haptic Gratings on Electrostatic Tactile Displays

Designing algorithmsfor rendering haptic texture on electrostatic tactile displays requires a quantitative understanding of human perception. In this paper, we report detection and discrimination thresholds for haptic gratings rendered on such displays based on the waveform and amplitude of the applied voltage. The haptic gratings consist of functions that describe the variation in voltage amplitude as a function of the position of finger on the display. Four types of virtual haptic gratings are considered in two experiments. In Experiment I, we estimate the absolute detection thresholds of haptic gratings for four different voltage amplitude functions, consisting of spatial waveforms with sinusoidal, square, triangle, or sawtooth shape. In Experiment II, we report discrimination thresholds for haptic gratings at five reference values of the voltage amplitude (80, 120, 160, 200, and 240 Vpp) for each of the voltage amplitude functions used in Experiment I. The results indicate that the detection thresholds for the four virtual haptic gratings are between 30 and 36 Vpp, and that the JND increases with the increase of voltage amplitudes. In addition, the JNDs of the four virtual gratings differ significantly, with the lowest and highest values being given by the triangle and sawtooth waveform, respectively.

Contact geometry and mechanics predict friction forces during tactile surface exploration

When we touch an object, complex frictional forces are produced, aiding us in perceiving surface features that help to identify the object at hand, and also facilitating grasping and manipulation. However, even during controlled tactile exploration, sliding friction forces fluctuate greatly, and it is unclear how they relate to the surface topography or mechanics of contact with the finger. We investigated the sliding contact between the finger and different relief surfaces, using high-speed video and force measurements. Informed by these experiments, we developed a friction force model that accounts for surface shape and contact mechanical effects, and is able to predict sliding friction forces for different surfaces and exploration speeds. We also observed that local regions of disconnection between the finger and surface develop near high relief features, due to the stiffness of the finger tissues. Every tested surface had regions that were never contacted by the finger; we refer to these as “tactile blind spots”. The results elucidate friction force production during tactile exploration, may aid efforts to connect sensory and motor function of the hand to properties of touched objects, and provide crucial knowledge to inform the rendering of realistic experiences of touch contact in virtual reality.

Stretchable, Twisted Conductive Microtubules for Wearable Computing, Robotics, Electronics, and Healthcare

Stretchable and flexible multifunctional electronic components, including sensors and actuators, have received increasing attention in robotics, electronics, wearable, and healthcare applications. Despite advances, it has remained challenging to design analogs of many electronic components to be highly stretchable, to be efficient to fabricate, and to provide control over electronic performance. Here, we describe highly elastic sensors and interconnects formed from thin, twisted conductive microtubules. These devices consist of twisted assemblies of thin, highly stretchable (>400%) elastomer tubules filled with liquid conductor (eutectic gallium indium, EGaIn), and fabricated using a simple roller coating process. As we demonstrate, these devices can operate as multimodal sensors for strain, rotation, contact force, or contact location. We also show that, through twisting, it is possible to control their mechanical performance and electronic sensitivity. In extensive experiments, we have evaluated the capabilities of these devices, and have prototyped an array of applications in several domains of stretchable and wearable electronics. These devices provide a novel, low cost solution for high performance stretchable electronics with broad applications in industry, healthcare, and consumer electronics, to emerging product categories of high potential economic and societal significance.

On Frictional Forces between the Finger and a Textured Surface during Active Touch

We investigated forces felt by a bare finger in sliding contact with a textured surface, and how they depend on properties of the surface and contact interaction. Prior research has shed light on haptic texture perception. Nevertheless, how texture-produced forces depend on the properties of a touched object or the way that it is touched is less clear. To address this, we designed an apparatus to accurately measure contact forces between a sliding finger and a textured surface. We fabricated textured surfaces, and measured spatial variations in forces produced as subjects explored the surfaces with a bare finger. We analyzed variations in these force signals, and their dependence on object geometry and contact parameters. We observed a number of phenomena, including transient stick-slip behavior, nonlinearities, phase variations, and large force fluctuations, in the form of aperiodic signal components that proved difficult to model for fine surfaces. Moreover, metrics such as total harmonic distortion and normalized variance decreased as the spatial scale of the stimuli increased. The results of this study suggest that surface geometry and contact parameters are insufficient to account for force production during such interactions. Moreover, the results shed light on perceptual challenges solved by the haptic system during active touch sensing of surface texture.

Fast Physically Accurate Rendering of Multimodal Signatures of Distributed Fracture in Heterogeneous Materials

This paper proposes a fast, physically accurate method for synthesizing multimodal, acoustic and haptic, signatures of distributed fracture in quasi-brittle heterogeneous materials, such as wood, granular media, or other fiber composites. Fracture processes in these materials are challenging to simulate with existing methods, due to the prevalence of large numbers of disordered, quasi-random spatial degrees of freedom, representing the complex physical state of a sample over the geometric volume of interest. Here, I develop an algorithm for simulating such processes, building on a class of statistical lattice models of fracture that have been widely investigated in the physics literature. This algorithm is enabled through a recently published mathematical construction based on the inverse transform method of random number sampling. It yields a purely time domain stochastic jump process representing stress fluctuations in the medium. The latter can be readily extended by a mean field approximation that captures the averaged constitutive (stress-strain) behavior of the material. Numerical simulations and interactive examples demonstrate the ability of these algorithms to generate physically plausible acoustic and haptic signatures of fracture in complex, natural materials interactively at audio sampling rates.

Linear and nonlinear subspace analysis of hand movements during grasping

This study investigated nonlinear patterns of coordination, or synergies, underlying whole-hand grasping kinematics. Prior research has shed considerable light on roles played by such coordinated degrees-of-freedom (DOF), illuminating how motor control is facilitated by structural and functional specializations in the brain, peripheral nervous system, and musculoskeletal system. However, existing analyses suppose that the patterns of coordination can be captured by means of linear analyses, as linear combinations of nominally independent DOF. In contrast, hand kinematics is itself highly nonlinear in nature. To address this discrepancy, we sought to to determine whether nonlinear synergies might serve to more accurately and efficiently explain human grasping kinematics than is possible with linear analyses. We analyzed motion capture data acquired from the hands of individuals as they grasped an array of common objects, using four of the most widely used linear and nonlinear dimensionality reduction algorithms. We compared the results using a recently developed algorithm-agnostic quality measure, which enabled us to assess the quality of the dimensional reductions that resulted by assessing the extent to which local neighborhood information in the data was preserved. Although qualitative inspection of this data suggested that nonlinear correlations between kinematic variables were present, we found that linear modeling, in the form of Principle Components Analysis, could perform better than any of the nonlinear techniques we applied.

Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions

Locomotion generates multisensory information about walked-upon objects. How perceptual systems use such information to get to know the environment remains unexplored. The ability to identify solid (e.g., marble) and aggregate (e.g., gravel) walked-upon materials was investigated in auditory, haptic or audio-haptic conditions, and in a kinesthetic condition where tactile information was perturbed with a vibromechanical noise. Overall, identification performance was better than chance in all experimental conditions and for both solids and the better identified aggregates. Despite large mechanical differences between the response of solids and aggregates to locomotion, for both material categories discrimination was at its worst in the auditory and kinesthetic conditions and at its best in the haptic and audio-haptic conditions. An analysis of the dominance of sensory information in the audio-haptic context supported a focus on the most accurate modality, haptics, but only for the identification of solid materials. When identifying aggregates, response biases appeared to produce a focus on the least accurate modality--kinesthesia. When walking on loose materials such as gravel, individuals do not perceive surfaces by focusing on the most accurate modality, but by focusing on the modality that would most promptly signal postural instabilities.

Vibration influences haptic perception of surface compliance during walking

Background

The haptic perception of ground compliance is used for stable regulation of dynamic posture and the control of locomotion in diverse natural environments. Although rarely investigated in relation to walking, vibrotactile sensory channels are known to be active in the discrimination of material properties of objects and surfaces through touch. This study investigated how the perception of ground surface compliance is altered by plantar vibration feedback.

Methodology/Principal Findings

Subjects walked in shoes over a rigid floor plate that provided plantar vibration feedback, and responded indicating how compliant it felt, either in subjective magnitude or via pairwise comparisons. In one experiment, the compliance of the floor plate was also varied. Results showed that perceived compliance of the plate increased monotonically with vibration feedback intensity, and depended to a lesser extent on the temporal or frequency distribution of the feedback. When both plate stiffness (inverse compliance) and vibration amplitude were manipulated, the effect persisted, with both factors contributing to compliance perception. A significant influence of vibration was observed even for amplitudes close to psychophysical detection thresholds.

Conclusions/Significance

These findings reveal that vibrotactile sensory channels are highly salient to the perception of surface compliance, and suggest that correlations between vibrotactile sensory information and motor activity may be of broader significance for the control of human locomotion than has been previously acknowledged.

Touch Is Everywhere: Floor Surfaces as Ambient Haptic Interfaces

Floor surfaces are notable for the diverse roles that they play in our negotiation of everyday environments. Haptic communication via floor surfaces could enhance or enable many computer-supported activities that involve movement on foot. In this paper, we discuss potential applications of such interfaces in everyday environments and present a haptically augmented floor component through which several interaction methods are being evaluated. We describe two approaches to the design of structured vibrotactile signals for this device. The first is centered on a musical phrase metaphor, as employed in prior work on tactile display. The second is based upon the synthesis of rhythmic patterns of virtual physical impact transients. We report on an experiment in which participants were able to identify communication units that were constructed from these signals and displayed via a floor interface at well above chance levels. The results support the feasibility of tactile information display via such interfaces and provide further indications as to how to effectively design vibrotactile signals for them.