Dynamic vibrotactile signals for forward collision avoidance warning systems

Investigating looming tactile takeover requests with various levels of urgency in automated vehicles

Designing an effective takeover request (TOR) in conditionally automated vehicles is crucial to ensure driving safety when the system reaches its limit. In our study, we aimed to investigate the effects of looming tactile TORs (whose urgency is dynamically mapped to the situation's criticality as the vehicle approaches the upcoming obstacle) on takeover performance and subjective experience compared with conventional non-looming TORs (several tactile pulses with consistent inter-pulse intervals). In addition, the impact of the TOR urgency level (with urgency levels matched or unmatched to the situation's criticality) was considered. A total of 30 participants were recruited for this study. They were first asked to map the urgency of tactile signals to the criticality of takeover situations with various times to collision according to the recorded video clips. The looming TORs were constructed based on these mapping results. Then, a simulated driving experiment, employing a within-subject design, was conducted to explore the effects of the tactile TOR type (looming vs. non-looming) and urgency level (less urgency vs. matched urgency vs. greater urgency) on takeover performance and drivers' subjective experience. The results showed that the looming TOR can lead to a shorter takeover time and less maximum lateral acceleration compared with the non-looming TOR. Drivers also rated the looming TOR as more useful. Therefore, the looming TOR has great application potential for enhancing driving safety in automated vehicles. In addition, we found that as the TOR's level of urgency increased, the takeover time decreased. However, the TOR with an urgency level matched to the situation's criticality received higher usefulness and satisfaction ratings, suggesting that there was an important trade-off between the advantage of high-urgency TORs in speeding up driver responses and its cost of a poor experience. The findings of our study shed some light on the design and implementation of the takeover warning system for related practitioners.

Between-Tactor Display Using Dynamic Tactile Stimuli for Directional Cueing in Vibrating Environments

Torso-worn vibrotactile devices have been used in many studies on directional cueing and navigation in environments where visual feedback is limited. These devices aim to indicate directions with high resolution while using the smallest possible number of vibration motors (tactors). Resolution can be increased using between-tactor displays, but their performance in vibrating environments (e.g., a helicopter) are unknown. This study proposes a between-tactor display using dynamic stimuli and verifies its effectiveness when the user sits in a vibrating chair. We developed a waist belt device that displays 12 directions using 6 tactors. Static stimuli display virtual (between-tactor) locations by constantly vibrating two adjacent tactors equally, whereas dynamic stimuli move the virtual vibration position back and forth between tactors. We performed two studies in which participants felt tactile stimuli and used a joystick to move a cursor on a screen to a target in the perceived direction. Direction recognition accuracy and task completion time were measured under combined conditions of two belt orientations (tactor alignments), with and without chair vibration, and with and without audio white noise to mask tactor sound. In all conditions, dynamic stimuli increased recognition accuracy while maintaining task completion time compared to static stimuli.

Planning lane changes using advance visual and haptic information

Taking a motor planning perspective, this study investigates whether haptic force cues displayed on the steering wheel are more effective than visual cues in signaling the direction of an upcoming lane change. Licensed drivers drove in a fixed-base driving simulator equipped with an active steering system for realistic force feedback. They were instructed to make lane changes upon registering a directional cue. Cues were delivered according to the movement precuing technique employing a pair of precues and imperative cues which could be either visual, haptic, or crossmodal (a visual precue with a haptic imperative cue, and vice versa). The main dependent variable was response time. Additional analyses were conducted on steering wheel angle profiles and the rate of initial steering errors. Conditions with a haptic imperative cue produced considerably faster responses than conditions with a visual imperative cue, irrespective of the precue modality. Valid and invalid precues produced the typical gains and costs, with one exception. There appeared to be little cost in response time or initial steering errors associated with invalid cueing when both cues were haptic. The results are consistent with the hypothesis that imperative haptic cues facilitate action selection while visual stimuli require additional time-consuming cognitive processing.

Effect of a looming visual cue on situation awareness and perceived urgency in response to a takeover request

This study aimed to investigate the effect of a looming visual cue on situation awareness and perceived urgency in response to a takeover request (TOR), and to explore the underlying mechanisms of this effect through three experiments. In Experiment 1, the optimal size and speed of a red disk were determined, which were effective in capturing looming motion and conveying the urgency of the situation. The results indicated that both looming speed and size ratio had significant effects on situation awareness and perceived urgency. In Experiment 2, the effects of looming stimuli were compared with dimming stimuli, and the results showed that the looming visual cue was more effective in promoting perceived urgency and situation awareness. The results also indicated that the looming visual cue attracted more visual attention than the dimming visual cue, in line with previous studies. Experiment 3 utilized a driving simulator to test the effectiveness of the looming visual cue in promoting fast and appropriate responses to TORs in complex driving scenarios. The results showed that the looming visual cue was more effective in promoting perceived urgency and enhancing situation awareness, especially in highly complex driving situations. Overall, the findings suggest that the looming visual cue is a powerful tool for promoting fast and appropriate responses to TORs and enhancing situation awareness, particularly in complex driving scenarios. These results have important implications for designing effective TOR systems and improving driver safety on the road.

Constructing the behavioral sequence of the takeover process-TOR, behavior characteristics and phases division: A real vehicle experiment

Autonomous driving will still use human-machine co-driving to handle complex situations for a long term, which requires the driver to control the vehicle and avoid hazards by executing appropriate behavioral sequences after takeover prompts. Previous studies focused on the division of static behavioral indicators and major phases in the initial phase of takeover, while lacking the construction of behavioral sequences based on the dynamic changes of behavioral characteristics during the takeover process. This study divides the takeover process in a detailed manner and investigates the impact of audio types on the behavioral sequence at each phase. 20 professional drivers performed the NDRT in autonomous driving mode on real roads, and after receiving audio prompts, they took over the vehicle and performed hazard avoidance maneuvers. The results show that the behavioral characteristics could construct the behavioral sequence of different phases, with the dynamic characteristics of the takeover operation change. In addition, different types of audio prompts will affect the timing of the takeover operation and its driving performance. Choosing different audio prompts or combinations can help improve the effect of taking over the vehicle. This study helps to provide guidance on the design of human-machine interaction for behavior optimization at different phases, so that guiding the driver to take over the vehicle safely and effectively.

Speech-based takeover requests in conditionally automated driving: Effects of different voices on the driver takeover performance

This study explored the possibility of applying personalized takeover requests (TORs) in an automated driving system (ADS), which required drivers to regain control when the system reached its limits. A driving simulator experiment was conducted to investigate how speech-based TOR voices impacted driver performance in takeover scenarios with two lead time conditions in conditionally automated driving (level 3). Eighteen participants drove in three sessions, with each session having a different TOR voice (a synthesized male voice, a synthesized female voice, and a significant other voice). Two scenarios with a lead time of 5 s and two scenarios with a lead time of 12 s were provided per session. The driver takeover time and quality data were collected. A follow-up interview was conducted to gain a clearer understanding of the drivers' psychological feelings about each TOR voice during takeovers. Changes in takeover time and takeover quality caused by TOR voices were similar in both lead time conditions, except for the lateral acceleration. The synthesized male voice led to a larger maximum lateral acceleration than the other two voices in the 5 s condition. Interestingly, most drivers preferred choosing the synthesized female voice for future takeovers and showed negative attitudes toward the significant other voice. Our results implied that choosing TOR voices should consider the drivers' daily voice-usage habits as well as specific context of use, and personalized TOR voices should be incorporated into the ADS prudently.

Cognitive task analysis and workload classification

Automation can be utilized to relieve humans of difficult and repetitive tasks in many domains, presenting the opportunity for safer and more efficient systems. This increase in automation has led to new supervisory roles for human operators where humans monitor feedback from autonomous systems and provide input when necessary. Optimizing these roles requires tools for evaluation of task complexity and resulting operator cognitive workload. Cognitive task analysis is a process for modeling the cognitive actions required of a human during a task. This work presents an enhanced version of this process: Cognitive Task Analysis and Workload Classification (CTAWC). The goal of developing CTAWC was to provide a standardized process to decompose cognitive tasks in enough depth to allow for precise identification of sources of cognitive workload. CTAWC has the following advantages over conventional CTA methodology:•Integrates standard terminology from existing taxonomies for task classification to describe expected operator cognitive workload during task performance.•Provides a framework to evaluate adequate cognitive depth when decomposing cognitive tasks.•Provides a standard model upon which to build an empirical study to evaluate task complexity.

Post-trip safety interventions: State-of-the-art, challenges, and practical implications

INTRODUCTION

Currently, risky driving behaviour is a major contributor to road crashes and as a result, wide array of tools have been developed in order to record and improve driving behaviour. Within that group of tools, interventions have been indicated to significantly enhance driving behaviour and road safety. This study critically reviews monitoring technologies that provide post-trip interventions, such as retrospective visual feedback, gamification, rewards or penalties, in order to inform an appropriate driver mentoring strategy delivered after each trip.

METHOD

The work presented here is part of the European Commission H2020 i-DREAMS project. The reviewed platform characteristics were obtained through commercially available solutions as well as a comprehensive literature search in popular scientific databases, such as Scopus and Google Scholar. Focus was given on state-of-the-art-technologies for post-trip interventions utilized in four different transport modes (i.e. car, truck, bus and rail) associated with risk prevention and mitigation.

RESULTS

The synthesized results revealed that smartphone applications and web-based platforms are the most accepted, frequently and easiest to use tools in cars, buses and trucks across all papers considered, while limited evidence of post-trip interventions in -rail was found. The majority of smartphone applications detected mobile phone use and harsh events and provided individual performance scores, while in-vehicle systems provided delayed visual reports through a web-based platform.

CONCLUSIONS

Gamification and appropriate rewards appeared to be effective solutions, as it was found that they keep drivers motivated in improving their driving skills, but it was clear that these cannot be performed in isolation and a combination with other strategies (i.e. driver coaching and support) might be beneficial. Nevertheless, as there is no holistic and cross-modal post-trip intervention solution developed in real-world environments, challenges associated with post-trip feedback provision and suggestions on practical implementation are also provided.

Peripersonal space in the front, rear, left and right directions for audio-tactile multisensory integration

Peripersonal space (PPS) is important for humans to perform body–environment interactions. However, many previous studies only focused on the specific direction of the PPS, such as the front space, despite suggesting that there were PPSs in all directions. We aimed to measure and compare the peri-trunk PPS in four directions (front, rear, left, and right). To measure the PPS, we used a tactile and an audio stimulus because auditory information is available at any time in all directions. We used the approaching and receding task-irrelevant sounds in the experiment. Observers were asked to respond as quickly as possible when a tactile stimulus was applied to a vibrator on their chest. We found that peri-trunk PPS representations exist with an approaching sound, irrespective of the direction.

Feeling the Beat (and Seeing It, Too): Vibrotactile, Visual, and Bimodal Rate Discrimination

Beats are among the basic units of perceptual experience. Produced by regular, intermittent stimulation, beats are most commonly associated with audition, but the experience of a beat can result from stimulation in other modalities as well. We studied the robustness of visual, vibrotactile, and bimodal signals as sources of beat perception. Subjects attempted to discriminate between pulse trains delivered at 3 Hz or at 6 Hz. To investigate signal robustness, we intentionally degraded signals on two-thirds of the trials using temporal-domain noise. On these trials, inter-pulse intervals (IPIs) were stochastic, perturbed independently from the nominal IPI by random samples from zero-mean Gaussian distributions with different variances. These perturbations produced directional changes in the IPIs, which either increased or decreased the likelihood of confusing the two pulse rates. In addition to affording an assay of signal robustness, this paradigm made it possible to gauge how subjects' judgments were influenced by successive IPIs. Logistic regression revealed a strong primacy effect: subjects' decisions were disproportionately influenced by a trial's initial IPIs. Response times and parameter estimates from drift-diffusion modeling showed that information accumulates more rapidly with bimodal stimulation than with either unimodal stimulus alone. Analysis of error rates within each condition suggested consistently optimal decision making, even with increased IPI variability. Finally, beat information delivered by vibrotactile signals proved just as robust as information conveyed by visual signals, confirming vibrotactile stimulation's potential as a communication channel.

Implied tactile motion: Localizing dynamic stimulations on the skin

We report two experiments designed to investigate how the implied motion of tactile stimuli influences perceived location. Predicting the location of sensory input is especially important as far as the perception of, and interaction with, the external world is concerned. Using two different experimental approaches, an overall pattern of localization shifts analogous to what has been described previously in the visual and auditory modalities is reported. That is, participants perceive the last location of a dynamic stimulus further along its trajectory than is objectively the case. In Experiment 1, participants judged whether the last vibration in a sequence of three was located closer to the wrist or to the elbow. In Experiment 2, they indicated the last location on a ruler attached to their forearm. We further pinpoint the effects of implied motion on tactile localization by investigating the independent influences of motion direction and perceptual uncertainty. Taken together, these findings underline the importance of dynamic information in localizing tactile stimuli on the skin.

Effects of auditory and tactile warning on response to visual hazards under a noisy environment

A warning signal presented via a visual or an auditory cue might interfere with auditory or visual information inside and outside a vehicle. On the other hand, such interference would be certainly reduced if a tactile cue is used. Therefore, it is expected that tactile cues would be promising as warning signals, especially in a noisy environment. In order to determine the most suitable modality of cue (warning) to a visual hazard in noisy environments, auditory and tactile cues were examined in this study. The condition of stimulus onset asynchrony (SOA) was set to 0ms, 500ms, and 1000ms. Two types of noises were used: white noise and noise outside a vehicle recorded in a real-world driving environment. The noise level LA_eq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber of each type of noise was adjusted to approximately 60 dB (A), 70 dB (A), and 80 dB (A). As a result, it was verified that tactile warning was more effective than auditory warning. When the noise outside a vehicle from a real-driving environment was used as the noise inside the experimental chamber, the reaction time to the auditory warning was not affected by the noise level.

Comparing spatially static and dynamic vibrotactile take-over requests in the driver seat

Vibrotactile stimuli can be effective as warning signals, but their effectiveness as directional take-over requests in automated driving is yet unknown. This study aimed to investigate the correct response rate, reaction times, and eye and head orientation for static versus dynamic directional take-over requests presented via vibrating motors in the driver seat. In a driving simulator, eighteen participants performed three sessions: 1) a session involving no driving (Baseline), 2) driving a highly automated car without additional task (HAD), and 3) driving a highly automated car while performing a mentally demanding task (N-Back). Per session, participants received four directional static (in the left or right part of the seat) and four dynamic (moving from one side towards the opposite left or right of the seat) take-over requests via two 6×4 motor matrices embedded in the seat back and bottom. In the Baseline condition, participants reported whether the cue was left or right, and in the HAD and N-Back conditions participants had to change lanes to the left or to the right according to the directional cue. The correct response rate was operationalized as the accuracy of the self-reported direction (Baseline session) and the accuracy of the lane change direction (HAD & N-Back sessions). The results showed that the correct response rate ranged between 94% for static patterns in the Baseline session and 74% for dynamic patterns in the N-Back session, although these effects were not statistically significant. Steering wheel touch and steering input reaction times were approximately 200ms faster for static patterns than for dynamic ones. Eye tracking results revealed a correspondence between head/eye-gaze direction and lane change direction, and showed that head and eye-gaze movements where initiated faster for static vibrations than for dynamic ones. In conclusion, vibrotactile stimuli presented via the driver seat are effective as warnings, but their effectiveness as directional take-over requests may be limited. The present study may encourage further investigation into how to get drivers safely back into the loop.

Warning Drivers about Impending Collisions Using Vibrotactile Flow

Vibrotactile collision warning signals that create a sensation of motion across a driver's body result in faster brake reaction times (BRTs) to potential collision events. To date, however, such warnings have only simulated linear motion. We extended this research by exploring the effectiveness of collision warnings that incorporate vibrotactile patterns or "vibrotactile flow". In Experiment 1, expanding and contracting vibrotactile flow warnings were compared with a static warning (all tactors activated simultaneously) and a no warning condition in a car following scenario. Both vibrotactile flow warnings produced significantly faster BRTs than the static and no warning conditions. However, there was no directional effect. That is, there was no significant difference between contracting and expanding signals. Warnings that utilize vibrotactile flow therefore appear to provide an effective means of informing drivers about potential collision events. However, unlike comparable warnings utilizing linear motion, their effectiveness does not seem to depend on the precise relationship between the warning and collision events. Experiment 2 demonstrated that a tactile warning incorporating linear motion produced significantly faster BRTs than an expanding vibrotactile flow warning. Taken together, these results suggest that vibrotactile warnings that simulate linear motion may be more effective than vibrotactile flow warnings.

Tactile warning signals for in-vehicle systems

The last few years have seen growing interest in the design of tactile warning signals to direct driver attention to potentially dangerous road situations (e.g. an impending crash) so that they can initiate an avoidance maneuver in a timely manner. In this review, we highlight the potential uses of such warning signals for future collision warning systems and compare them with more traditional visual and auditory warnings. Basic tactile warning signals are capable of promoting driver alertness, which has been demonstrated to be beneficial for forward collision avoidance (when compared to a no warning baseline condition). However, beyond their basic alerting function, directional tactile warning signals are now increasingly being utilized to shift the attention of the driver toward locations of interest, and thus to further facilitate their speeded responses to potential collision events. Currently, many researchers are focusing their efforts on the development of meaningful (iconic) tactile warning signals. For instance, dynamic tactile warnings (varying in their intensity and/or location) can potentially be used to convey meaningful information to drivers. Finally, we highlight the future research that will be needed in order to explore how to present multiple directional warnings using dynamic tactile cues, thus forming an integrated collision avoidance system for future in-vehicle use.

A comparison of different informative vibrotactile forward collision warnings: does the warning need to be linked to the collision event?

Recent research demonstrates that auditory and vibrotactile forward collision warnings presenting a motion signal (e.g., looming or apparent motion across the body surface) can facilitate speeded braking reaction times (BRTs). The purpose of the present study was to expand on this work by directly comparing warning signals in which the motion conveyed was constant across all collision events with signals in which the speed of motion was dependent on the closing velocity (CV). Two experiments were conducted using a simulated car-following task and BRTs were measured. In Experiment 1, increasing intensity (looming) vibrotactile signals were presented from a single tactor attached to the driver's waist. When the increase in intensity was CV-linked, BRTs were significantly faster as compared to a no-warning condition, however, they were not significantly different from constant intensity and CV-independent looming warnings. In Experiment 2, a vertical array of three tactors was used to create motion either towards (upwards) or away (downwards) from the driver's head. When the warning signal presented upwards motion that was CV-linked, BRTs were significantly faster than all other warning types. Downwards warnings led to a significantly higher number of brake activations in false alarm situations as compared to upwards moving warnings. The effectiveness of dynamic tactile collision warnings would therefore appear to depend on both the link between the warning and collision event and on the directionality of the warning signal.