Haptic Interaction Design for Everyday Interfaces

When to use vibrotactile displays? A meta-analysis for the role of vibrotactile displays in human-computer interaction

OBJECTIVE

This study aims to investigate the benefits of unimodal tactile displays relative to other modal displays and the performance gains of adding redundant tactile displays by integrating empirical studies.

BACKGROUND

Tactile displays have attracted increasing attention in recent years due to their unique advantages. Synthesizing experimental data is necessary to analyze the performance benefits of tactile displays for participants and better help practitioners in utilizing them.

METHOD

Five meta-analyses were conducted. Two meta-analyses compared the participants' performance between tactile and other modal displays (visual vs. tactile and auditory vs. tactile). Three meta-analyses examined the performance gains of adding redundant tactile displays based on other modal displays (visual vs. visual + tactile, auditory vs. auditory + tactile, and visual + auditory vs. visual + auditory + tactile). The related moderator variables, the types of presented information and concurrent tasks, were analyzed.

RESULTS

Little evidence shows the performance difference between tactile and auditory displays. Tactile displays are more beneficial than visual displays for presenting alert information or in the situation with a visual concurrent task. The performance gains of adding redundant tactile displays to other modal displays also depend on the specific type of presented information and the concurrent task.

CONCLUSION

When using tactile displays to convey information, interface designers should consider the specific type of presented information and the concurrent tasks.

APPLICATIONS

The present study's findings can provide some implications for designers to utilize tactile displays when they construct and implement information displays.

Where's My Button? Evaluating the User Experience of Surface Haptics in Featureless Automotive User Interfaces

Advancements in user interface technologies and demands of design engineering led to increasing implementation of large and mostly flat interactive surfaces in automotive. Recent discussions in the context of in-vehicle usage of touchscreens advocate for the use of haptic feedback to restore the explore- and feel-qualities typically experienced in traditional physical button interfaces that contribute to intuitive, eyes-free, and tactually rich interactions. Haptic technologies that include a friction modulation approach seem especially promising to convey a high-quality feeling. This research reports an experience-oriented evaluation of an electrostatic friction haptic display in an in-vehicle direct touch interaction context. The evaluation was based on an automotive multitask setting (primary driving-task and secondary target-selection-task) with a 2 × 2 feedback modality design (factors haptic/audio with levels absent/present). The objective variables (response time, errors, and performance on the primary task) did not differ between feedback modalities. Any additional feedback to a visual baseline enhanced the user experience, with the multimodal feedback being preferred by most participants. Surface haptics was perceived as a novel yet unexpected type of haptic feedback. We discuss the implications for the haptic design of programmable friction displays and provide an initial set of guidelines for this innovative technology.

Function Follows Form: Using the Aesthetic Association Principle to Enhance Haptic Interface Design

Novel tangible user interface technologies facilitate current trends toward seamless user interfaces. They enable the design of yet unseen interfaces and thus the creation of a new kind of haptic language. In order to use the benefits of a touch-and-feel design for a positive user experience, carefully designed haptic feedback plays an important role by providing aesthetically pleasing and sustainable product features. Haptic feedback may exceed mere acquiring of buttons and input-confirmation but enable orientation and even identification of functionality governed by the haptic impression. We employed the aesthetic association principle as a deeply grounded psychological mechanism that assists effective linkage between haptic form factors and associated functional attributes. In order to illustrate this powerful principle, we analyzed the specific associations between certain main haptic surface qualities and associated functional aspects. In a series of three subsequent studies (Pre-Study 1: perception, Pre-Study 2: similarity, and Main Study: association), we explored paradigmatic associations of that kind to develop guidelines which forms are distinct to be used in interfaces. We show how forms are implicitly categorized into functional qualities (on/off, more-less, selection), using a multidimensional scaling procedure and explore explicit form-functionality associations, using a think-aloud method in the context of an automotive interface. For a series of forms, we revealed clear associative relations to specific functions. We will discuss the general value and opportunities of an association-based approach to user experience in order to create intuitive user interfaces. We will also develop ideas for specific areas of applications.

Haptic Devices Based on Real-Time Dynamic Models of Multibody Systems

Multibody modeling of mechanical systems can be applied to various applications. Human-in-the-loop interfaces represent a growing research field, for which increasingly more devices include a dynamic multibody model to emulate the system physics in real-time. In this scope, reliable and highly dynamic sensors, to both validate those models and to measure in real-time the physical system behavior, have become crucial. In this paper, a multibody modeling approach in relative coordinates is proposed, based on symbolic equations of the physical system. The model is running in a ROS environment, which interacts with sensors and actuators. Two real-time applications with haptic feedback are presented: a piano key and a car simulator. In the present work, several sensors are used to characterize and validate the multibody model, but also to measure the system kinematics and dynamics within the human-in-the-loop process, and to ultimately validate the haptic device behavior. Experimental results for both developed devices confirm the interest of an embedded multibody model to enhance the haptic feedback performances. Besides, model parameters variations during the experiments illustrate the infinite possibilities that such model-based configurable haptic devices can offer.

Multibody-Based Piano Action: Validation of a Haptic Key

A piano key prototype actuated by a custom-made linear actuator is proposed to enhance the touch of digital pianos by reproducing the force feedback of an acoustic piano action. This paper presents the design and the validation of the haptic device. The approach exploits a multibody model to compute the action dynamics and the corresponding force on the key in real time. More specifically, a grand piano model that includes the five action bodies, its geometry and the specific force laws, is computed in the haptic device. A presizing step along with Finite Element Method (FEM) analysis produced an especially made actuator satisfying the design requirements, in particular the highly dynamic nature of the force to be transmitted. Force peaks, up to 50 (N) in less than 20 (ms), are reachable with low power consumption. Compared to previous solutions: (i) the key physical characteristics are preserved; (ii) the feedback is based on a real-time multibody model that is easily configurable and interchangeable; (iii) an experimental validation of the actuator within the prototype is developed and demonstrates its feasibility. The results confirm that the voice coil can produce suitable haptic feedback. In particular, rendering a grand piano action within the device shows promising haptic force profiles.

Eye-Tracking-Based Classification of Information Search Behavior Using Machine Learning: Evidence from Experiments in Physical Shops and Virtual Reality Shopping Environments

How can we tailor assistance systems, such as recommender systems or decision support systems, to consumers’ individual shopping motives? How can companies unobtrusively identify shopping motives without explicit user input? We demonstrate that eye movement data allow building reliable prediction models for identifying goal-directed and exploratory shopping motives. Our approach is validated in a real supermarket and in an immersive virtual reality supermarket. Several managerial implications of using gaze-based classification of information search behavior are discussed: First, the advent of virtual shopping environments makes using our approach straightforward as eye movement data are readily available in next-generation virtual reality devices. Virtual environments can be adapted to individual needs once shopping motives are identified and can be used to generate more emotionally engaging customer experiences. Second, identifying exploratory behavior offers opportunities for marketers to adapt marketing communication and interaction processes. Personalizing the shopping experience and profiling customers’ needs based on eye movement data promises to further increase conversion rates and customer satisfaction. Third, eye movement-based recommender systems do not need to interrupt consumers and thus do not take away attention from the purchase process. Finally, our paper outlines the technological basis of our approach and discusses the practical relevance of individual predictors.

Perception of a Haptic Stimulus Presented Under the Foot Under Workload

It is clear that the haptic channel can be exploited as a communication medium for several tasks of everyday life. Here we investigated whether such communication can be altered in a cognitive load situation. We studied the perception of a vibrotactile stimulus presented under the foot when the attention is loaded by another task (cognitive load). The results demonstrated a significant influence of workload on the perception of the vibrotactile stimulus. Overall, we observed that the average score in the single-task (at rest) condition was greater than the overall mean score in the dual-task conditions (counting forwards, counting backwards, and walking). The walking task was the task that most influenced the perception of the vibrotactile stimulus presented under the foot.

Improvements in the learnability of smartphone haptic interfaces for visually impaired users

We report the results of a study on the learnability of haptic icons used as alerts or notifications in smartphones. The aim was to explore the feasibility of using haptic icons to create assistive technologies for people with visual impairments. We compared the performance and satisfaction of users with different visual capacities (visually impaired vs. sighted) and using different learning processes (with or without a reinforcement learning stage). The reinforcement learning stage improves the recognition rate in both types of users, although the improvement obtained by the visually impaired users is even better as their recognition rates become very similar to those obtained by the sighted users. Finally, it was observed that the better recognized haptic icons are those assigned to the most employed applications by the user.

A Theoretical Framework of Haptic Processing in Automotive User Interfaces and Its Implications on Design and Engineering

Driving a car is a highly visual task. Despite the trend towards increased driver assistance and autonomous vehicles, drivers still need to interact with the car for both driving and non-driving relevant tasks, at times simultaneously. The often-resulting high cognitive load is a safety issue, which can be addressed by providing the driver with alternative feedback modalities, such as haptics. Recent trends in the automotive industry are moving towards the seamless integration of control elements through touch-sensitive surfaces. Psychological knowledge on optimally utilizing haptic technologies remains limited. The literature on automotive haptic feedback consists mainly of singular findings without putting them into a broader user context with respect to haptic design of interfaces. Moreover, haptic feedback has primarily been limited to the confirmation of control actions rather than the searching or finding of control elements, the latter of which becomes particularly important considering the current trends. This paper presents an integrated framework on haptic processing in automotive user interfaces and provides guidelines for haptic design of user interfaces in car interiors.

Getting Your Hands Dirty Outside the Lab: A Practical Primer for Conducting Wearable Vibrotactile Haptics Research

As haptics have become an ingrained part of our wearable experience, particularly through phones, smartwatches, and fitness trackers, significant research effort has been conducted to find new ways of using wearable haptics to convey information, especially while we are on-the-go. In this paper, instead of focusing on aspects of haptic information design, such as tacton encoding methods, actuators, and technical fabrication of devices, we address the more general recurring issues and "gotchas" that arise when moving from core haptic perceptual studies and in-lab wearable experiments to real world testing of wearable vibrotactile haptic systems. We summarize key issues for practitioners to take into account when designing and carrying out in-the-wild wearable haptic user studies, as well as for user studies in a lab environment that seek to simulate real-world conditions. We include not only examples from published work and commercial sources, but also hard-won illustrative examples derived from issues and failures from our own haptic studies. By providing a broad-based, accessible overview of recurring issues, we expect that both novice and experienced haptic researchers will find suggestions that will improve their own mobile wearable haptic studies.

Active and Passive Haptic Perception of Shape: Passive Haptics Can Support Navigation

Real-time haptic interactions occur under two exploration modes: active and passive. In this paper, we present a series of experiments that evaluate the main perceptual characteristics of both exploration modes. In particular, we focus on haptic shape recognition as it represents a fundamental task in many applications using haptic environments. The results of four experiments conducted with a group of 10 voluntary subjects show that the differences in motor activity between active and passive haptics ease the perception of surfaces for the first case and the perception of pathways for the latter. In addition, the guidance nature of passive haptics makes the pathway direction easy to recognize. This work shows that this last observation could find application in more challenging tasks such as navigation in space.

Applications of Future Technologies to Detect Skill Decay and Improve Procedural Performance

Medical simulation training has progressed in its use of incorporating various technologies to provide quality training interfaces from novices to experts. The purpose of this paper is to explore modeling, simulation and visualization training technology interfaces to improve precision learning, rigorous, objective assessment, and performance improvement feedback for clinical procedural skill training and sustainment. Technologies to include augmented reality (AR), haptic technology and computer vision will be defined and clarified. It is believed that by exploring the combination of using AR, haptics and computer vision technologies it is possible to develop a fully immersive learning system that can automate mentoring while detecting and measuring gross and fine motor skills. Such a system can be used to predict or delay the onset of skills decay (SD) by capturing rigorous, objective measures, and human performance metrics that can provide feedback to individual performers for skills improvement in real time.

Computer-aided trauma simulation system with haptic feedback is easy and fast for oral-maxillofacial surgeons to learn and use

PURPOSE

Computer-assisted surgical (CAS) planning tools have become widely available in craniomaxillofacial surgery, but are time consuming and often require professional technical assistance to simulate a case. An initial oral and maxillofacial (OM) surgical user experience was evaluated with a newly developed CAS system featuring a bimanual sense of touch (haptic).

MATERIALS AND METHODS

Three volunteer OM surgeons received a 5-minute verbal introduction to the use of a newly developed haptic-enabled planning system. The surgeons were instructed to simulate mandibular fracture reductions of 3 clinical cases, within a 15-minute time limit and without a time limit, and complete a questionnaire to assess their subjective experience with the system. Standard landmarks and linear and angular measurements between the simulated results and the actual surgical outcome were compared.

RESULTS

After the 5-minute instruction, all 3 surgeons were able to use the system independently. The analysis of standardized anatomic measurements showed that the simulation results within a 15-minute time limit were not significantly different from those without a time limit. Mean differences between measurements of surgical and simulated fracture reductions were within current resolution limitations in collision detection, segmentation of computed tomographic scans, and haptic devices. All 3 surgeons reported that the system was easy to learn and use and that they would be comfortable integrating it into their daily clinical practice for trauma cases.

CONCLUSION

A CAS system with a haptic interface that capitalizes on touch and force feedback experience similar to operative procedures is fast and easy for OM surgeons to learn and use.

Design and analysis of a recoil-type vibrotactile transducer

This article describes the design of a high-bandwidth, iron-less, recoil-based electromagnetic vibrotactile actuator. Its working principle, the theoretical analysis, the method used to determine its transfer function, its scaling properties and its design constraints are discussed along with its fabrication and possible improvements.

Putting Haptics into the Ambience

In an attentionally overloaded world, relief will come only from interfaces between humans and computation that are able to provide information in the background of our sensory and cognitive processes. Haptic displays may have a special role to play in this emerging movement toward ambient interfaces, because the touch sense is well suited to present many types of information in a way that treads lightly on our mental resources. This paper offers an introduction to the notion of ambient information display, and explores why and how the haptic channel could contribute. It begins with a discussion of the attentional problems posed by contemporary interface technology, and a broad overview of ambient interfaces themselves: their purpose, specification, features, and some general examples. Sense is made of the haptic ambient design space through a morphology of the functionality and social configurations exhibited by existing and envisioned examples. Finally, reflections on design principles and challenges for ambient haptic interfaces are aimed at inspiring, shaping, and informing future development in this area.