Multisensory warning signals for event perception and safe driving

Planning lane changes using advance visual and haptic information

Taking a motor planning perspective, this study investigates whether haptic force cues displayed on the steering wheel are more effective than visual cues in signaling the direction of an upcoming lane change. Licensed drivers drove in a fixed-base driving simulator equipped with an active steering system for realistic force feedback. They were instructed to make lane changes upon registering a directional cue. Cues were delivered according to the movement precuing technique employing a pair of precues and imperative cues which could be either visual, haptic, or crossmodal (a visual precue with a haptic imperative cue, and vice versa). The main dependent variable was response time. Additional analyses were conducted on steering wheel angle profiles and the rate of initial steering errors. Conditions with a haptic imperative cue produced considerably faster responses than conditions with a visual imperative cue, irrespective of the precue modality. Valid and invalid precues produced the typical gains and costs, with one exception. There appeared to be little cost in response time or initial steering errors associated with invalid cueing when both cues were haptic. The results are consistent with the hypothesis that imperative haptic cues facilitate action selection while visual stimuli require additional time-consuming cognitive processing.

Visual and steering behaviours during lane departures: a longitudinal study of interactions between lane departure warning system, driving task and driving experience

Lane Departure Warning Systems (LDWS) generate a warning in case of imminent lane departure. LDWS have proven to be effective and associated human-machine cooperation modelled. In this study, LDWS acceptance and its impact on visual and steering behaviour have been investigated over 6 weeks for novice and experienced drivers. Unprovoked lane departures were analysed along three driving tasks gradually more demanding. These observations were compared to a baseline condition without automation. The number of lane departures and their duration were dramatically reduced by LDWS, and a narrower visual spread of search during lane departure events was recorded. The findings confirmed LDWS effectiveness and suggested that these benefits are supported by visuo-attentional guidance. No specific influence of driving experience on LDWS was found, suggesting that similar cognitive processes are engaged with or without driving experience. Drivers' acceptance of LDWS lowered after automation use, but LDWS effectiveness remained stable during prolonged use.Practitioner summary: Lane Departure Warning Systems (LDWS) have been designed to prevent lane departure crashes. Here, LDWS assessment over a 6-week period showed a major drop in the number of lane departure events increasing over time. LDWS effectiveness is supported by the guidance of drivers' visual attention during lane departure events.

Multisensory GPS impact on spatial representation in an immersive virtual reality driving game

Individuals are increasingly relying on GPS devices to orient and find their way in their environment and research has pointed to a negative impact of navigational systems on spatial memory. We used immersive virtual reality to examine whether an audio–visual navigational aid can counteract the negative impact of visual only or auditory only GPS systems. We also examined the effect of spatial representation preferences and abilities when using different GPS systems. Thirty-four participants completed an IVR driving game including 4 GPS conditions (No GPS; audio GPS; visual GPS; audio–visual GPS). After driving one of the routes in one of the 4 GPS conditions, participants were asked to drive to a target landmark they had previously encountered. The audio–visual GPS condition returned more accurate performance than the visual and no GPS condition. General orientation ability predicted the distance to the target landmark for the visual and the audio–visual GPS conditions, while landmark preference predicted performance in the audio GPS condition. Finally, the variability in end distance to the target landmark was significantly reduced in the audio–visual GPS condition when compared to the visual and audio GPS conditions. These findings support theories of spatial cognition and inform the optimisation of GPS designs.

Effectiveness of Lateral Auditory Collision Warnings: Should Warnings Be Toward Danger or Toward Safety?

OBJECTIVE

The present study investigated the design of spatially oriented auditory collision-warning signals to facilitate drivers' responses to potential collisions.

BACKGROUND

Prior studies on collision warnings have mostly focused on manual driving. It is necessary to examine the design of collision warnings for safe takeover actions in semi-autonomous driving.

METHOD

In a video-based semi-autonomous driving scenario, participants responded to pedestrians walking across the road, with a warning tone presented in either the avoidance direction or the collision direction. The time interval between the warning tone and the potential collision was also manipulated. In Experiment 1, pedestrians always started walking from one side of the road to the other side. In Experiment 2, pedestrians appeared in the middle of the road and walked toward either side of the road.

RESULTS

In Experiment 1, drivers reacted to the pedestrian faster with collision-direction warnings than with avoidance-direction warnings. In Experiment 2, the difference between the two warning directions became nonsignificant. In both experiments, shorter time intervals to potential collisions resulted in faster reactions but did not influence the effect of warning direction.

CONCLUSION

The collision-direction warnings were advantageous over the avoidance-direction warnings only when they occurred at the same lateral location as the pedestrian, indicating that this advantage was due to the capture of attention by the auditory warning signals.

APPLICATION

The present results indicate that drivers would benefit most when warnings occur at the side of potential collision objects rather than the direction of a desirable action during semi-autonomous driving.

Effect of mapping characteristic on audiovisual warning: Evidence from a simulated driving study

Advanced driver assistance systems (ADAS) can enhance road safety by sending warning signals to drivers. Multimodal signals are gaining attention in ADAS warning design because they offer redundant information that facilitates human-system communication. However, no consensus has been reached on which multimodal design offers optimal benefit to road safety. Icons iconically map the real world and are associated with fast recognition and response time. Therefore, this study aims to investigate whether visual and auditory icons will benefit the effectiveness of audiovisual multimodal warnings. Thirty-two participants (16 females) experienced four types of unimodal warnings (high and low mapping visual warnings and high and low mapping auditory warnings) and four types of audiovisual warnings (high mapping visual + high mapping auditory warning, low mapping visual + low mapping auditory warning, high mapping visual + low mapping auditory warning, and low mapping visual + high mapping auditory warning) in simulated driving conditions. Visual warnings are presented in a head-up display. Results showed that multimodal warnings outperformed unimodal warnings (i.e., modality effect). We found mapping effect in audiovisual warnings, but only high mapping auditory constituents benefited warning effectiveness. Eye movement results revealed that the high mapping constituents might distract drivers from the road. This study adds evidence that multimodal warnings can offer extra benefits to drivers and high mapping auditory signals should be included in multimodal warning design to achieve better driving performance.

May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality

When interacting with technology, attention is mainly driven by audiovisual and increasingly haptic stimulation. Olfactory stimuli are widely neglected, although the sense of smell influences many of our daily life choices, affects our behavior, and can catch and direct our attention. In this study, we investigated the effect of smell and sound on visuospatial attention in a virtual environment. We implemented the Bells Test, an established neuropsychological test to assess attentional and visuospatial disorders, in virtual reality (VR). We conducted an experiment with 24 participants comparing the performance of users under three experimental conditions (smell, sound, and smell and sound). The results show that multisensory stimuli play a key role in driving the attention of the participants and highlight asymmetries in directing spatial attention. We discuss the relevance of the results within and beyond human-computer interaction (HCI), particularly with regard to the opportunity of using VR for rehabilitation and assessment procedures for patients with spatial attention deficits.

Auditory versus visual spatial stimulus-response mappings in tracking and discrete dual task performance: implications for human-machine interface design

A discrete four-choice response task with auditory signal presentation and a joystick-controlled visual tracking task was used to investigate how spatial compatibility influences the dual-task performance of different display-control settings. It was found that the more incompatible the stimulus-response mapping, the longer the delay for both tasks, presumably because of the longer stimulus encoding time required for the incompatible conditions. A comparison of the findings of this study with those of past experiments on visual visual setting shows that the dual-task performance in a cross-modality (auditory visual) setting was significantly better than that in an intra-modality (visual visual) setting because of visual scanning required in the intra-modal dual tasks. However, when the locations of visual visual tasks were close enough such that ambient and focal vision was concurrently used for information processing, the dual-task performance of intra-modality (visual visual) configuration was slightly better than that of the cross-modality (auditory visual) configuration. Practitioner Summary: The effect of spatial compatibility with auditory signal presentation in multiple display-control configurations was examined in a dual-task paradigm. The results provided important and useful ergonomics design implications and consequent recommendations for intra- and cross- modal interface design. The results should facilitate human-machine system design and improve overall system performance.

Impact of ambient sound on risk perception in humans: neuroeconomic investigations

Research in the field of multisensory perception shows that what we hear can influence what we see in a wide range of perceptual tasks. It is however unknown whether this extends to the visual perception of risk, despite the importance of the question in many applied domains where properly assessing risk is crucial, starting with financial trading. To fill this knowledge gap, we ran interviews with professional traders and conducted three laboratory studies using judgments of financial asset risk as a testbed. We provide evidence that the presence of ambient sound impacts risk perception, possibly due to the combination of facilitatory and synesthetic effects of general relevance to the perception of risk in many species as well as humans. We discuss the implications of our findings for various applied domains (e.g., financial, medical, and military decision-making), and raise new questions for future research.

Auditory Messages for Intersection Movement Assist (IMA) Systems: Effects of Speech- and Nonspeech-Based Cues

OBJECTIVE

The objective of this study was to assess the effects of different warning messages for an Intersection Movement Assist (IMA) based on drivers' ability to avoid a potential safety hazard.

BACKGROUND

An IMA system can detect hazards and warn drivers when it is unsafe to enter an intersection. The effects of different warning information conveyed by these systems are still unknown.

METHOD

A driving simulator study with 80 participants was conducted with a red light running (RLR) scenario using a 5 (warnings) x 2 (training) between-subject design. IMA warnings included the messages "Danger," "Brake now," "Vehicle on your left," a beep, and no IMA warning. Training was provided to half of the participants. Analysis of variance and logistic regression models were used to examine differences in drivers' avoidance behavior.

RESULTS

The analyses showed that all tested warning messages can significantly enhance drivers' avoidance performance. Significant differences were observed in crash occurrence, avoidance behavior (i.e., reaction time and speed change), and eye movements (i.e., fixation pattern and time to first fixation). The effects of training also differed given the warning message provided.

CONCLUSION

The "Brake now" message performed best in reducing crash involvement and prompted better avoidance performance. The "Danger" and "Vehicle on your left" messages improved drivers' hazard detection ability. The training showed a potential to enhance the effectiveness of nonspeech warning messages.

APPLICATION

The findings of this study can help designers and engineers better design IMA warning messages for RLR scenarios.

Superior side sound localisation performance in a full-chassis driving simulator

Alerts presented through the auditory modality improve drivers' crash avoidance performance in driving simulations, but drivers' ability to accurately localise the source of the auditory alerts is understudied. Because the results of driving simulation studies may hinge on assumptions that sound locations are accurately perceived by drivers, this study used a sound localisation task in a full-chassis driving simulator. Twenty-nine participants engaged in a sound localisation task while seated in the driving simulator. Performance was assessed by sound localisation accuracy, relative directional error, and participant confidence across seven sound sources surrounding the simulator. Performance was best when sounds were presented in left and right cardinal regions, and poorest when presented from the front and rear. Participants were less confident in their localisation judgments when sounds were presented from the rear. Practitioner summary: Drivers' ability to accurately localise auditory alerts is understudied. Participants performed an auditory localisation task with external sounds while seated in a full-chassis driving simulator. Participants were better detecting sounds from the sides instead of the front and rear. This has implications for external auditory alarms during driving. Abbreviations: ANOVA: analysis of variance; dB: decibel; f: frequency; Hz: hertz; LHD: left hand drive; ms: milliseconds; RTI: realtime technologies; s: seconds; SPL: sound pressure level.

Age, cognitive load, and multimodal effects on driver response to directional warning

Inattention can be considered a primary cause of vehicular accidents or crashes, and in-car warning signals are applied to alert the driver to take action even in automated vehicles. Because of age related decline of the older driver's abilities, in-car warning signals may need adjustment to the older driver. We therefore investigated the effects of uni-, bi- and trimodal directional warnings (i.e., light, sound, vibration) on young and older drivers' responses in a driving simulator. A young group of 15 drivers (20-25 years of age) and an older group of 16 drivers (65-79 years of age) participated. In the simulations, warning signal was presented at the left, the center, or the right in front of the participant. With a warning at the left, the center, and the right the correct response was to steer to the right, brake, and steer to the left, respectively. The main results showed the older drivers' responses were slower for each type of warning compared with the young drivers' responses. Overall, the responses were slower with an added cognitively loading task. The only multimodal type of warning inducing overall faster response than its constituent warning types was the vibration-sound, and only for the older drivers. Additionally, with the groups' responses collapsed, such a true multimodal effect on response time also showed for the center vibration-sound warning (i.e., braking response). The only multimodal warning showing clear reduction in response errors compared with its constituent warning types was the vibration-sound for the older drivers during extra cognitive load. The main conclusion is that older drivers can benefit from bimodal warning, as compared with unimodal, in terms of faster and more accurate response. The potential superiority of trimodal warning is nevertheless argued.

Earcons Versus Auditory Icons in Communicating Computing Events: Learning and User Preference

This article investigates the effectiveness and user preferences of auditory icons and earcons in communicating various computing events. A controlled data collection exercise revealed that participants more quickly learned the relationships between computing events and auditory icons than the relationships between the same computing events and earcons. Results from a second data collection exercise showed that participants not only preferred to hear earcons rather than auditory icons, but indicated that auditory icons would be more irritating after repeated hearings. Taken together, these results present an interesting conundrum for systems designers: The more effective mode of communication is less preferred by users.

Practice makes better - Learning effects of driving with a multi-stage collision warning

Advanced driver assistance systems like (forward) collision warnings can increase traffic safety. As safety-critical situations (especially in urban traffic) can be diverse, integrated adaptive systems (such as multi-stage warnings) need to be developed and examined in a variety of use cases over time instead of the more common approach of testing only one-time effectiveness in the most relevant use case. Thus, this driving simulator experiment investigated a multi-stage collision warning in partially repetitive trials (T) of various safety-critical situations (scenarios confronting drivers with hazards in form of pedestrians, obstacles or preceding vehicles). Its output adapted according to the drivers' behavior in two warning stages (W1 - warning for moderate deceleration in less critical situations; W2 - urgent warning for strong, fast braking in more critical situations). To analyze how much drivers benefit from the assistance when allowed practice with it, the driving behavior and subjective ratings of 24 participants were measured over four trials. They comprised a baseline without assistance (T1) and three further trials with assistance - a learning phase repeating the scenarios from T1 twice (T2 + T3) and a concluding transfer drive with new scenarios (T4). As expected, the situation criticality in the urgent warning (W2) scenarios was rated higher than in the warning (W1) scenarios. While the brake reaction time differed more between the W1 scenarios, the applied brake force differed more between the W2 scenarios. However, the scenario factor often interacted with the trial factor. Since in later warning stages reaction time reductions become finite, the reaction strength gains importance. Overall the drivers benefited from the assistance. Both warning stages led to faster brake reactions (of similar strength) in all three assisted trials compared to the baseline, which additionally improved successively over time (T1-T3, T1 vs. T4, T2 vs. T4). Moreover, the drivers applied the gained knowledge from the learning phase to various new situations (transfer: faster brake reactions in T4 compared to T1 or T2). The well accepted and positively rated (helpful and understandable) two-stage collision warning can thus be recommended as it facilitates accident mitigation by earlier decelerations. Practice with advanced driver assistance systems (even in driving simulators) should be endorsed to maximize their benefits for traffic safety and accident prevention.

Validation of Essential Acoustic Parameters for Highly Urgent In-Vehicle Collision Warnings

Objective The aim of this study was to validate the importance of key acoustic criteria for use as in-vehicle forward collision warning (FCW) systems. Background Despite recent advances in vehicle safety, automobile crashes remain one of the leading causes of death. As automation allows for more control of noncritical functions by the vehicle, the potential for disengagement and distraction from the driving task also increases. It is, therefore, as important as ever that in-vehicle safety-critical interfaces are intuitive and unambiguous, promoting effective collision avoidance responses upon first exposure even under divided-attention conditions. Method The current study used a driving simulator to assess the effectiveness of two warnings, one that met all essential acoustic parameters, one that met only some essential parameters, and a no-warning control in the context of a lead vehicle-following task in conjunction with a cognitive distractor task and collision event. Results Participants receiving an FCW comprising five essential acoustic components had improved collision avoidance responses relative to a no-warning condition and an FCW missing essential elements on their first exposure. Responses to a consistently good warning (GMU Prime) improved with subsequent exposures, whereas continued exposure to the less optimal FCW (GMU Sub-Prime) resulted in poorer performance even relative to receiving no warning at all. Conclusions This study provides support for previous warning design studies and for the validity of five key acoustic parameters essential for the design of effective in-vehicle FCWs. Application Results from this study have implications for the design of auditory FCWs and in-vehicle display design.

Influence of lane departure warnings onset and reliability on car drivers' behaviors

Lane departures represent an important cause of road crashes. The objective of the present study was to assess the effects of an auditory Lane Departure Warning System (LDWS) for partial and full lane departures (onset manipulation) combined with missed warnings (reliability manipulation: 100% reliable, 83% reliable and 66% reliable) on drivers' performances and acceptance. Several studies indicate that LDWS improves drivers' performances during lane departure episodes. However, little is known about the effects of the warning onset and reliability of LDWS. Results of studies which looked at forward collision warning systems show that early warnings tend to improve drivers' performances and receive a better trust judgement from the drivers when compared to later warnings. These studies also suggest that reliable assistances are more effective and trusted than unreliable ones. In the present study, lane departures were brought about by means of a distraction task whilst drivers simulated driving in a fixed-base simulator with or without an auditory LDWS. Results revealed steering behaviors improvements with LDWS. More effective recovery maneuvers were found with partial lane departure warnings than with full lane departure warnings and assistance unreliability did not impair significantly drivers' behaviors. Regarding missed lane departure episodes, drivers were found to react later and spend more time out of the driving lane when compared to properly warned lane departures, as if driving without assistance. Subjectively, LDWS did not reduce mental workload and partial lane departure warnings were judged more trustworthy than full lane departure ones. Data suggests the use of partial lane departure warnings when designing LDWS and that even unreliable LDWS may draw benefits compared to no assistance.

What a car does to your perception: Distance evaluations differ from within and outside of a car

Almost a century ago it was first suggested that cars can be interpreted as tools, but consequences of this assumption were never tested. Research on hand-held tools that are used to manipulate objects in the environment suggests that perception of near space is extended by using tools. Literature on environment perception finds perception of far space to be modulated by the observer's potential to act in the environment. Here we argue that a car increases the action potential and modulates perception of far space in a way similar to how hand-held tools modulate perception of near space. Five distances (4 to 20 meters) were estimated by pedestrians and drivers before and after driving/walking. Drivers underestimated all distances to a larger percentage than did pedestrians. Underestimation was even stronger after driving. We conclude that cars modulate the perception of far distances because they modulate the driver's perception, like a tool typically does, and change the perceived action potential.

Neuroscience-Inspired Design: From Academic Neuromarketing to Commercially Relevant Research

Companies and organizations the world over wish to understand, predict, and ultimately change the behavior of those whom they interact with, advise, or else provide services for: be it the accident-prone driver out on the roads, the shopper bombarded by a myriad of alternative products on the supermarket shelf, or the growing proportion of the population who are clinically obese. The hope is that by understanding more about the mind, using recent advances in neuroscience, more effective interventions can be designed. But just what insights can a neuroscience-inspired approach offer over-and-above more traditional, not to mention contemporary, behavioral methods? This article focuses on three key areas: neuroergonomics, neuromarketing, and neurogastronomy. The utility of the neuroscience-inspired approach is illustrated with a number of concrete real-world examples. Practical challenges with commercial neuromarketing research, including the cost, timing, ethics/legality and access to scanners (in certain countries), and the limited ecological validity of the situations in which people are typically tested are also discussed. This commentary highlights a number of the key challenges associated with translating academic neuroscience research into commercial neuromarketing applications.

Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual search

In joint tasks, adjusting to the actions of others is critical for success. For joint visual search tasks, research has shown that when search partners visually receive information about each other's gaze, they use this information to adjust to each other's actions, resulting in faster search performance. The present study used a visual, a tactile and an auditory display, respectively, to provide search partners with information about each other's gaze. Results showed that search partners performed faster when the gaze information was received via a tactile or auditory display in comparison to receiving it via a visual display or receiving no gaze information. Findings demonstrate the effectiveness of tactile and auditory displays for receiving task-relevant information in joint tasks and are applicable to circumstances in which little or no visual information is available or the visual modality is already taxed with a demanding task such as air-traffic control. Practitioner Summary: The present study demonstrates that tactile and auditory displays are effective for receiving information about actions of others in joint tasks. Findings are either applicable to circumstances in which little or no visual information is available or when the visual modality is already taxed with a demanding task.

Warning Drivers about Impending Collisions Using Vibrotactile Flow

Vibrotactile collision warning signals that create a sensation of motion across a driver's body result in faster brake reaction times (BRTs) to potential collision events. To date, however, such warnings have only simulated linear motion. We extended this research by exploring the effectiveness of collision warnings that incorporate vibrotactile patterns or "vibrotactile flow". In Experiment 1, expanding and contracting vibrotactile flow warnings were compared with a static warning (all tactors activated simultaneously) and a no warning condition in a car following scenario. Both vibrotactile flow warnings produced significantly faster BRTs than the static and no warning conditions. However, there was no directional effect. That is, there was no significant difference between contracting and expanding signals. Warnings that utilize vibrotactile flow therefore appear to provide an effective means of informing drivers about potential collision events. However, unlike comparable warnings utilizing linear motion, their effectiveness does not seem to depend on the precise relationship between the warning and collision events. Experiment 2 demonstrated that a tactile warning incorporating linear motion produced significantly faster BRTs than an expanding vibrotactile flow warning. Taken together, these results suggest that vibrotactile warnings that simulate linear motion may be more effective than vibrotactile flow warnings.

Depth: the Forgotten Dimension in Multisensory Research

The last quarter of a century has seen a dramatic rise of interest in the spatial constraints on multisensory integration. However, until recently, the majority of this research has investigated integration in the space directly in front of the observer. The space around us, however, extends in three spatial dimensions in the front and to the rear beyond such a limited area. The question to be addressed in this review concerns whether multisensory integration operates according to the same rules throughout the whole of three-dimensional space. The results reviewed here not only show that the space around us seems to be divided into distinct functional regions, but they also suggest that multisensory interactions are modulated by the region of space in which stimuli happen to be presented. We highlight a number of key limitations with previous research in this area, including: (1) The focus on only a very narrow region of two-dimensional space in front of the observer; (2) the use of static stimuli in most research; (3) the study of observers who themselves have been mostly static; and (4) the study of isolated observers. All of these factors may change the way in which the senses interact at any given distance, as can the emotional state/personality of the observer. In summarizing these salient issues, we hope to encourage researchers to consider these factors in their own research in order to gain a better understanding of the spatial constraints on multisensory integration as they affect us in our everyday life.

Car Gestures - Advisory warning using additional steering wheel angles

Advisory warning systems (AWS) notify the driver about upcoming hazards. This is in contrast to the majority of currently deployed advanced driver assistance systems (ADAS) that manage emergency situations. The target of this study is to investigate the effectiveness, acceptance, and controllability of a specific kind of AWS that uses the haptic information channel for warning the driver. This could be beneficial, as alternatives for using the visual modality can help to reduce the risk of visual overload. The driving simulator study (N=24) compared an AWS based on additional steering wheel angle control (Car Gestures) with a visual warning presented in a simulated head-up display (HUD). Both types of warning were activated 3.5s before the hazard object was reached. An additional condition of unassisted driving completed the experimental design. The subjects encountered potential hazards in a variety of urban situations (e.g. a pedestrian standing on the curbs). For the investigated situations, subjective ratings show that a majority of drivers prefer visual warnings over haptic information via gestures. An analysis of driving behavior indicates that both warning approaches guide the vehicle away from the potential hazard. Whereas gestures lead to a faster lateral driving reaction (compared to HUD warnings), the visual warnings result in a greater safety benefit (measured by the minimum distance to the hazard object). A controllability study with gestures in the wrong direction (i.e. leading toward the hazard object) shows that drivers are able to cope with wrong haptic warnings and safety is not reduced compared to unassisted driving as well as compared to (correct) haptic gestures and visual warnings.

Audiovisual crossmodal cuing effects in front and rear space

The participants in the present study had to make speeded elevation discrimination responses to visual targets presented to the left or right of central fixation following the presentation of a task-irrelevant auditory cue on either the same or opposite side. In Experiment 1, the cues were presented from in front of the participants (from the same azimuthal positions as the visual targets). A standard crossmodal exogenous spatial cuing effect was observed, with participants responding significantly faster in the elevation discrimination task to visual targets when both the auditory cues and the visual targets were presented on the same side. Experiment 2 replicated the exogenous spatial cuing effect for frontal visual targets following both front and rear auditory cues. The results of Experiment 3 demonstrated that the participants had little difficulty in correctly discriminating the location from which the sounds were presented. Thus, taken together, the results of the three experiments reported here demonstrate that the exact co-location of auditory cues and visual targets is not necessary to attract spatial attention. Implications of these results for the design of real-world warning signals are discussed.

Tactile warning signals for in-vehicle systems

The last few years have seen growing interest in the design of tactile warning signals to direct driver attention to potentially dangerous road situations (e.g. an impending crash) so that they can initiate an avoidance maneuver in a timely manner. In this review, we highlight the potential uses of such warning signals for future collision warning systems and compare them with more traditional visual and auditory warnings. Basic tactile warning signals are capable of promoting driver alertness, which has been demonstrated to be beneficial for forward collision avoidance (when compared to a no warning baseline condition). However, beyond their basic alerting function, directional tactile warning signals are now increasingly being utilized to shift the attention of the driver toward locations of interest, and thus to further facilitate their speeded responses to potential collision events. Currently, many researchers are focusing their efforts on the development of meaningful (iconic) tactile warning signals. For instance, dynamic tactile warnings (varying in their intensity and/or location) can potentially be used to convey meaningful information to drivers. Finally, we highlight the future research that will be needed in order to explore how to present multiple directional warnings using dynamic tactile cues, thus forming an integrated collision avoidance system for future in-vehicle use.

Dynamic vibrotactile warning signals for frontal collision avoidance: towards the torso versus towards the head

Three experiments were conducted to assess the effectiveness of dynamic vibrotactile warning signals with different spatial patterns and to compare dynamic towards-torso and towards-head vibrotactile warnings in a simulated driving task. The results revealed that embedding additional stimuli between the participant's hands and waist in the towards-torso cues (Experiment 1) and increasing the spatial distance between adjacent stimuli in the towards-head cues (Experiment 2) did not result in any further benefits in braking response times (BRTs). The triple towards-head cues resulting from the sequential operation of three pairs of stimuli on the torso gave rise to a significant advantage over the static cues; however, it did not outperform the dynamic towards-torso cues with just two pairs of stimuli. Taken together, these results demonstrated the promise of dynamic vibrotactile warnings (especially, the towards-torso warnings) in terms of the future design of more effective rear-end collision warnings.

PRACTITIONER SUMMARY

Three experiments assessed the effectiveness of dynamic towards-torso and towards-head vibrotactile warning signals in a simulated driving task. The results demonstrated the promise of dynamic vibrotactile warnings (especially, the towards-torso vibrotactile warnings) in terms of the future design of more effective frontal collision warnings.

Tactual perception: a review of experimental variables and procedures

This paper reviews the literature on tactual perception. Throughout this review, we will highlight some of the most relevant aspects in the touch literature: type of stimuli; type of participants; type of tactile exploration; and finally, the interaction between touch and other senses. Regarding type of stimuli, we will analyse studies with abstract stimuli such as vibrations, with two- and three-dimensional stimuli, and also concrete stimuli, considering the relation between familiar and unfamiliar stimuli and the haptic perception of faces. Under the "type of participants" topic, we separated studies with blind participants, studies with children and adults, and also performed an overview of sex differences in performance. The type of tactile exploration is explored considering conditions of active and passive touch, the relevance of movement in touch and the relation between haptic exploration and time. Finally, interactions between touch and vision, touch and smell and touch and taste are explored in the last topic. The review ends with an overall conclusion on the state of the art for the tactual perception literature. With this work, we intend to present an organised overview of the main variables in touch experiments, compiling aspects reported in the tactual literature, and attempting to provide both a summary of previous findings, and a guide to the design of future works on tactual perception and memory, through a presentation of implications from previous studies.

“This is your heart speaking. Call 911.”

Early warning for heart attacks could save many lives. We conducted three studies to design and evaluate multimodal alarms and patient training for an implanted heart attack detector. An implanted device provided vibrotactile alarms subcutaneously, and a pager-like device provided auditory and visual alarms. Temporal alarm patterns connoted an urgent emergency alarm (“Call 911”) and a less urgent alarm (“See your doctor”). In the third, clinical, study, most patients (94%) correctly responded to the alarms at 1, 3, and 6 months after device implantation. Subcutaneous vibrotactile alarms show great potential for use in critical medical applications.

Driver behaviour at rail level crossings: responses to flashing lights, traffic signals and stop signs in simulated rural driving

Australian road and railway authorities have made a concerted effort to reduce the number of rail level crossings, particularly the higher risk passive crossings that are protected by devices such as 'give way' or 'stop' signs. To improve this situation, passive level crossings are often upgraded with active controls such as flashing red lights. Traffic signals may provide good safety outcomes at level crossings but remain untested. The primary purpose of this research was to compare driver behaviour at two railway level crossings with active controls, flashing red lights and traffic signals, to behaviour at the current standard passive level crossing control, a stop sign. Participants drove the MUARC advanced driving simulator for 30 min. During the simulated drive, participants were exposed to three level crossing scenarios. Each scenario consisted of one of three level crossing control types, and was associated with an oncoming train. Mean vehicle speed on approach to the level crossings decreased more rapidly in response to flashing lights than to traffic signals. While speed on approach was lowest for the stop-sign condition, the number of non-compliant drivers (i.e., those who did not stop) at the crossing was highest for this condition. While results indicate that traffic signals at rail level crossings do not appear to offer any safety benefits over and above flashing red lights, further avenues of research are proposed to reach more definitive conclusions. Compliance was lowest for the passive crossing control which provides further support for the ongoing passive crossing upgrades in Australia.

Tonic and phasic EEG and behavioral changes induced by arousing feedback

This study investigates brain dynamics and behavioral changes in response to arousing auditory signals presented to individuals experiencing momentary cognitive lapses during a sustained-attention task. Electroencephalographic (EEG) and behavioral data were simultaneously collected during virtual-reality (VR) based driving experiments, in which subjects were instructed to maintain their cruising position and compensate for randomly induced lane deviations using the steering wheel. 30-channel EEG data were analyzed by independent component analysis and the short-time Fourier transform. Across subjects and sessions, intermittent performance during drowsiness was accompanied by characteristic spectral augmentation or suppression in the alpha- and theta-band spectra of a bilateral occipital component, corresponding to brief periods of normal (wakeful) and hypnagogic (sleeping) awareness and behavior. Arousing auditory feedback was delivered to the subjects in half of the non-responded lane-deviation events, which immediately agitated subject's responses to the events. The improved behavioral performance was accompanied by concurrent spectral suppression in the theta- and alpha-bands of the bilateral occipital component. The effects of auditory feedback on spectral changes lasted 30s or longer. The results of this study demonstrate the amount of cognitive state information that can be extracted from noninvasively recorded EEG data and the feasibility of online assessment and rectification of brain networks exhibiting characteristic dynamic patterns in response to momentary cognitive challenges.

Auditory displays as occasion setters

OBJECTIVE

The aim of this study was to evaluate whether representational sounds that capture the richness of experience of a collision enhance performance in braking to avoid a collision relative to other forms of warnings in a driving simulator.

BACKGROUND

There is increasing interest in auditory warnings that are informative about their referents. But as well as providing information about some intended object, warnings may be designed to set the occasion for a rich body of information about the outcomes of behavior in a particular context. These richly informative warnings may offer performance advantages, as they may be rapidly processed by users.

METHOD

An auditory occasion setter for a collision (a recording of screeching brakes indicating imminent collision) was compared with two other auditory warnings (an abstract and an "environmental" sound), a speech message, a visual display, and no warning in a fixed-base driving simulator as interfaces to a collision avoidance system. The main measure was braking response times at each of two headways (1.5 s and 3 s) to a lead vehicle.

RESULTS

The occasion setter demonstrated statistically significantly faster braking responses at each headway in 8 out of 10 comparisons (with braking responses equally fast to the abstract warning at 1.5 s and the environmental warning at 3 s).

CONCLUSION

Auditory displays that set the occasion for an outcome in a particular setting and for particular behaviors may offer small but critical performance enhancements in time-critical applications.

APPLICATION

The occasion setter could be applied in settings where speed of response by users is of the essence.

The effects of multisensory targets on saccadic trajectory deviations: eliminating age differences

The present study had two aims. First, to determine if bimodal audio-visual targets allow for greater inhibition of visual distractors, which in turn may lead to greater saccadic trajectory deviations away from those distractors. Second, to determine if bimodal targets can reduce age differences in the ability to generate deviations away, as older adults tend to benefit more from multisensory integration than younger adults. The results show that bimodal targets produced larger deviations away than unimodal targets, but only when the distractor preceded the target, and this effect was comparable across age groups. Furthermore, in contrast to previous research, older adults in this study showed similar deviations away from distractors to those of younger adults. These findings suggest that age differences in the production of trajectory deviations away are not inevitable and that multisensory integration may be an important means for increasing top-down inhibition of irrelevant distraction.

Capturing spatial attention with multisensory cues: a review

The last 30 years have seen numerous studies demonstrating unimodal and crossmodal spatial cuing effects. However, surprisingly few studies have attempted to investigate whether multisensory cues might be any more effective in capturing a person's spatial attention than unimodal cues. Indeed, until very recently, the consensus view was that multisensory cues were, in fact, no more effective. However, the results of several recent studies have overturned this conclusion, by showing that multisensory cues retain their attention-capturing ability under conditions of perceptual load (i.e., when participants are simultaneously engaged in a concurrent attention-demanding task) while their constituent signals (when presented unimodally) do not. Here we review the empirical literature on multisensory spatial cuing effects and highlight the implications that this research has for the design of more effective warning signals in applied settings.

Tactile and Multisensory Spatial Warning Signals for Drivers

The last few years have seen many exciting developments in the area of tactile and multisensory interface design. One of the most rapidly-moving practical application areas for these findings is in the development of warning signals and information displays for drivers. For instance, tactile displays can be used to awaken sleepy drivers, to capture the attention of distracted drivers, and even to present more complex information to drivers who may be visually-overloaded. This review highlights the most important potential costs and benefits associated with the use of tactile and multisensory information displays in a vehicular setting. Multisensory displays that are based on the latest cognitive neuroscience research findings can capture driver attention significantly more effective than their unimodal (i.e., tactile) counterparts. Multisensory displays can also be used to transmit information more efficiently, as well as to reduce driver workload. Finally, we highlight the key research questions currently awaiting further research, including questions such as: Are tactile warning signals really intuitive? Are there certain regions of the body (or in the space surrounding the body) where tactile/multisensory warning signals are particularly effective? To what extent is the spatial coincidence and temporal synchrony of the individual sensory signals critical to determining the effectiveness of multisensory displays? And, finally, how does the issue of compliance vs. reliance (or the 'cry wolf' phenomenon associated with the presentation of signals that are perceived as false alarms) influence the effectiveness of tactile and/or multisensory warning signals?